diff --git a/.github/ISSUE_TEMPLATE/bug.yml b/.github/ISSUE_TEMPLATE/bug.yml index c11e3e85cdf..d81faabc400 100644 --- a/.github/ISSUE_TEMPLATE/bug.yml +++ b/.github/ISSUE_TEMPLATE/bug.yml @@ -19,6 +19,8 @@ body: options: - Media3 main branch - Media3 pre-release (alpha, beta or RC not in this list) + - Media3 1.11.1 + - Media3 1.11.0 - Media3 1.10.1 - Media3 1.10.0 - Media3 1.9.2 diff --git a/RELEASENOTES.md b/RELEASENOTES.md index fdb642707a5..cd8abbc1ad9 100644 --- a/RELEASENOTES.md +++ b/RELEASENOTES.md @@ -3,32 +3,161 @@ ### Unreleased changes * Common Library: + * Add a central `Flags` registry to manage experimental feature flags and + canary mode rollout. Migrated many existing `@ExperimentalApi` methods + to use the new flags API. + * Add `MediaMetadata.playlistId`, + `MediaMetadata.Builder.setPlaylistId(String)`, + `MediaMetadata.categories`, and + `MediaMetadata.Builder.setCategories(List)`. * ExoPlayer: - * Add `Format.selectionPriority` representing the relative preference of - the `Format` in track selection - ([#3236](https://github.com/androidx/media/pull/3236)). + * Fix an issue where DASH multi-period playback gets stuck when the next + period's first chunk fails to load + ([#3396](https://github.com/androidx/media/issues/3396)). + * Report `wasCanceled = true` in `AnalyticsListener.onLoadError` and + `MediaSourceEventListener.onLoadError` when load retries are exhausted. + * Support registering multiple `VideoFrameMetadataListener` instances via + `ExoPlayer.addVideoFrameMetadataListener` and + `ExoPlayer.removeVideoFrameMetadataListener`, deprecating + `ExoPlayer.setVideoFrameMetadataListener` and + `ExoPlayer.clearVideoFrameMetadataListener`. + * Add support for ads in multi-period content (e.g., DASH) by splitting + and offsetting the `AdPlaybackState` for each period. + * Fix race condition in `PreCacheHelper` where canceling an ongoing + preparation request could still start background downloads. + * Limit `setLoadOnlySelectedTracks(true)` on `ProgressiveMediaSource` and + `DefaultMediaSourceFactory` to only filter out unselected video and + image tracks. Audio and text tracks are always loaded to allow seamless + track switching without re-buffering. + * Enable `setLoadOnlySelectedTracks(true)` by default on + `ProgressiveMediaSource` and `DefaultMediaSourceFactory`. This saves + memory and allows `DefaultLoadControl` buffer allocation calculations to + work more accurately when not all tracks are selected. Note that this + may cause additional buffering if unselected video or image tracks are + newly enabled mid-playback. + * Fix an issue where Player.getCurrentPosition() could return stale values + (updating only a few times per second) when dynamic scheduling is + enabled ([#3286](https://github.com/androidx/media/issues/3286)). + * Fix an issue where an unexpected decrease or reset in the `AudioTrack` + playback head position could be incorrectly treated as an integer + overflow, causing playback to stall in buffering + ([#3407](https://github.com/androidx/media/issues/3407)). + * Remove experimental + `DefaultMediaCodecAdapterFactory.setAsyncCryptoSynchronizationEnabled(boolean)`. + * Avoid unnecessary renderer re-enables and glitches when returning from + interstitials/ads into content clipped at the end + ([#3371](https://github.com/androidx/media/issues/3371)). + * Fix an issue where a renderer error on an upcoming playlist item could + prematurely interrupt current playback when + `Flags.FLAG_PER_STREAM_MEDIA_PROGRESSION` is enabled. + * Fix a `NullPointerException` that could occur if `ExoPlayer` + transitioned renderers to an unprepared media period when + `Flags.FLAG_PER_STREAM_MEDIA_PROGRESSION` is enabled. + * Fix an issue where video frames could be unexpectedly skipped or dropped + during dynamic scheduling when + `Flags.FLAG_VIDEO_RENDERER_DURATION_TO_PROGRESS` is enabled. + * Fix an `ArrayIndexOutOfBoundsException` in analytics listeners when + stopping playback during an ad on a live stream + ([#3423](https://github.com/androidx/media/issues/3423)). + * Fix an issue where `PreloadMediaSource` could unexpectedly post preload + event when `preload()` is never called + ([#3408](https://github.com/androidx/media/issues/3408)). + * Fix an issue where `Format.subsampleOffsetUs` was ignored for metadata + tracks, causing incorrect playback timestamps in decoded SCTE-35 splice + commands ([#3413](https://github.com/androidx/media/issues/3413)). + * Fix a memory leak in `PreloadMediaSource` and `DefaultPreloadManager` + when `preload()` is called multiple times on the same source + ([#3445](https://github.com/androidx/media/issues/3445)). + * Fix a `NullPointerException` when releasing `AdsMediaSource` while a + content timeline refresh is pending + ([#3442](https://github.com/androidx/media/issues/3442)). * CompositionPlayer: + * Support configuring the frame rate of video frame aggregation via + `Composition.Builder.setVideoFrameAggregationParameters` for playback + workflows. + * Fix frame drops in multi-sequence compositions caused by timestamp + quantization or rounding mismatches, by selecting the frame with the + closest timestamp during frame aggregation. This is enabled from API 29 + only, as it requires holding one additional frame in flight per video + sequence. +* Decoder extensions (FFmpeg, VP9, AV1, etc.): + * Opus extension: Fix native crash when decoding invalid Opus data. * Transformer: + * Support configuring the frame rate of video frame aggregation via + `Composition.Builder.setVideoFrameAggregationParameters` for export + workflows. + * Fix frame drops in multi-sequence compositions caused by timestamp + quantization or rounding mismatches, by selecting the frame with the + closest timestamp during frame aggregation. This is enabled from API 29 + only, as it requires holding one additional frame in flight per video + sequence. + * Respect `Transformer#setAudioMimeType(AUDIO_RAW)` when `AudioProcessors` + are used. Prior to this fix, requesting `MimeTypes.AUDIO_RAW` and + applying an `AudioProcessor` resulted in compressed output. * Track Selection: + * Fix an issue where an SDR track was incorrectly preferred over a + supported Dolby Vision track on some devices + ([#3135](https://github.com/androidx/media/issues/3135)). * Extractors: - * AVI: Fix an issue where playing files without keyframe flags on audio - tracks caused audio loss and `OutOfMemoryError`. - * MP3: Use gapless-aware durations from Xing/Info headers - ([#3183](https://github.com/androidx/media/issues/3183)). - * MP3: Adjust LAME/Xing encoder delay and padding metadata to match - decoded PCM trimming - ([#3200](https://github.com/androidx/media/pull/3200)). + * Mp4Extractor: Fix `dOps` box parsing for Opus in MP4 where `PreSkip`, + `InputSampleRate`, and `OutputGain` were not converted from big-endian + to little-endian, causing the first ~299 ms of audio to be dropped when + no edit list is present + ([#3431](https://github.com/androidx/media/issues/3431)). + * Remove `Mp4Extractor.FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES`, + `Mp4Extractor.FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES_H265`, + `FragmentedMp4Extractor.FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES`, and + `FragmentedMp4Extractor.FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES_H265` + as within-GOP sample dependency parsing for H.264 and H.265 is enabled + by default. + * Mp4Extractor: Fix location metadata parsing from `xyz` boxes containing + altitude coordinates to populate `Mp4LocationData`. + * Add `ExtractorUtil.getFramesPerEncodedSample` to calculate the number of + audio frames per sample for encoded audio formats + ([#3367](https://github.com/androidx/media/issues/3367)). + * Add `FLAG_READ_XMP_METADATA` to `Mp4Extractor` to extract XMP metadata + from top-level UUID boxes. + * Fix corrupted AC-3, DTS and LPCM audio when playing DVD-style MPEG-PS + content, by stripping the `private_stream_1` sub-stream header from each + PES packet in `PsExtractor` before the payload is passed to the audio + reader ([#3327](https://github.com/androidx/media/issues/3327)). + * Fix failure in `FragmentedMp4Extractor` when a version 1 `tfdt` box + encodes a negative `baseMediaDecodeTime` + ([#3412](https://github.com/androidx/media/issues/3412)). + * MPEG-TS: Support parsing H.264 and H.265 frame packing arrangement SEI + messages to populate `Format.stereoMode` + ([#3419](https://github.com/androidx/media/issues/3419)). + * Parse Dolby Vision Profile 20 configuration data and reject unsupported + Dolby Vision profiles instead of playing back as standard HEVC with + incorrect colors ([#3427](https://github.com/androidx/media/pull/3427)). + * MP3: Revert LAME/Xing encoder delay & padding metadata adjustment + because it's incompatible with many `MediaCodec` MP3 decoders + ([#3425](https://github.com/androidx/media/issues/3425)). + * Matroska: Fix `NullPointerException` when parsing nested `ChapterAtom` + elements ([#3438](https://github.com/androidx/media/issues/3438)). * Inspector: + * Support retrieving XMP metadata from MP4 files using + `MetadataRetriever`. * Inspector Frame: * Audio: - * Remove fallback path of EAC3-JOC to EAC3 for playback on Google Pixel - devices as their standard EAC3 decoders cannot decode EAC3-JOC streams - ([#3257](https://github.com/androidx/media/pull/3257)). + * Add workaround for devices that invoke + `android.media.AudioDeviceCallback` on the wrong thread + ([#3386](https://github.com/androidx/media/issues/3386)). + * Fix potential stuck playbacks after track transitions with compressed + offload. * Video: - * Fix bug in `VideoFrameReleaseHelper` where display changes could trigger - redundant `Choreographer` callbacks and excessive CPU wakeups on the - main thread. + * Fix reporting of late video frames with identical release timestamps so + that they are reported as dropped instead of skipped. * Text: + * Add `MediaItem.SubtitleConfiguration.timeOffsetUs` to shift the + timestamps of sideloaded subtitles relative to the media. The offset can + be changed during playback with `Player.replaceMediaItem(int, + MediaItem)` without interrupting playback + ([#1976](https://github.com/androidx/media/issues/1976)). + * SubRip: Add support for injecting a `CharsetDetector` into + `DefaultSubtitleParserFactory` to detect the character encoding of + standalone SubRip subtitles without a byte order mark + ([#2247](https://github.com/androidx/media/issues/2247)). * Metadata: * Image: * DataSource: @@ -36,66 +165,240 @@ * Effect: * Effect Lottie: * Muxers: + * Fix `dOps` box writing for Opus in `Mp4Muxer` and `FragmentedMp4Muxer` + where `PreSkip`, `InputSampleRate`, and `OutputGain` were written in + little-endian instead of big-endian + ([#3431](https://github.com/androidx/media/issues/3431)). + * Support writing GPS altitude in `Mp4LocationData` via `Mp4Muxer`. + * Add `Muxer.Factory.create(ParcelFileDescriptor)` to allow muxing + directly to a `ParcelFileDescriptor`. + * Write `tfdt` (Track Fragment Base Media Decode Time) box by default in + `traf` boxes of fragmented MP4 files to provide media decode timestamps. + * Support writing E-AC-3 and E-AC-3 JOC (Dolby Atmos) `dec3` container + boxes in `Mp4Muxer`. + * Add WebM format support to `MediaMuxerCompat`. + * Write `mfra` (Movie Fragment Random Access) box at the end of fragmented + MP4 files to allow seeking via `FragmentedMp4Extractor` using + `FLAG_READ_MFRA_FOR_SEEK_MAP`. + * Fix crash in `Mp4Writer` and `FragmentedMp4Writer` when writing an EOS + sample before any other samples are written. + * Fix container compliance when muxing Dolby content into MP4 or + Fragmented MP4 files by advertising the dby1 compatible brand in the + container header. + ([#3369](https://github.com/androidx/media/pull/3369)). + * Support 32-bit float PCM audio tracks in `Mp4Muxer` and + `FragmentedMp4Muxer` + ([#3389](https://github.com/androidx/media/issues/3389)). + * Reject unsupported Dolby Vision profiles in `Mp4Muxer` and + `FragmentedMp4Muxer` when adding tracks. + * Support 24-bit and 32-bit integer PCM audio tracks in `Mp4Muxer` and + `FragmentedMp4Muxer`. * IMA extension: + * Forward player pause and resume events to the IMA SDK in + `ImaServerSideAdInsertionMediaSource`. * Session: - * Fix double-downscaling of artwork in `MediaSession` when the image size - is close to the platform limit, resolving blurriness in notifications - ([#3134](https://github.com/androidx/media/issues/3134)). - * Fix `ForegroundServiceStartNotAllowedException` crash in - `MediaNotificationManager` when an asynchronous artwork bitmap load - callback attempts to start the foreground service while the app is in - the background ([#3270](https://github.com/androidx/media/issues/3270)). - * Fix an issue where `PositionInfo` objects received by a - `MediaController` were missing their original `Timeline` UIDs. The - session now reliably translates and propagates these UIDs during state - updates. + * Add `MediaConstants.EXTRAS_KEY_PLAYLIST_ID`, + `MediaConstants.EXTRAS_KEY_PLAYLIST_TITLE`, + `MediaConstants.EXTRAS_KEY_PLAYLIST_ARTWORK_URI`, and + `MediaConstants.EXTRAS_KEY_CATEGORIES` to populate playlist and item + metadata from legacy metadata and descriptions, and + `MediaConstants.EXTRAS_KEY_SUGGESTED_TYPE` and + `MediaConstants.EXTRAS_KEY_TTL_SECONDS` for suggestion + `LibraryParams.extras`. + * Fix `SecurityException` crash when creating or releasing a + `MediaSession` on devices that enforce strict `PendingIntent` limits. + * Fix issue where `MediaSession.getControllerForCurrentRequest()` returned + `null` during play requests on Android 12 (API 31 and 32) when + `MediaSessionService` was not yet in the foreground. + * Fix `IllegalArgumentException` crash when a `MediaController` connects + to a legacy session that reports a negative maximum volume. + * Fix an issue where calling `MediaController.setMediaItem` or + `setMediaItems` followed by `play()` on a paused legacy + `MediaSessionCompat` could dispatch an unsupported `onPrepareFrom*` + callback instead of `onPlayFrom*`, or dispatch `onPlay()` without + re-initializing the requested media item. * UI: - * Fix shutter suppression bug in `PlayerView` and Compose - `PresentationState` that caused crashes on timelines that do not support - period UID lookup - ([#3264](https://github.com/androidx/media/issues/3264)). + * Add `TrackSelectionState` and `TrackSelectionParametersState` classes + and their corresponding Composable state remember functions to the + `media3-ui-compose` module. + * Introduce `PresentationState.videoAspectRatio` and make + `Modifier.resizeWithContentScale` take `aspectRatio: Float?` instead of + `sourceSizeDp: Size?`. + * Add `CuesState` class and `rememberCuesState` Composable to the + `media3-ui-compose` module for listening to player's current cues. + * Add scrubbing mode support to `ProgressSlider` if it is based on + `ExoPlayer` or `CompositionPlayer`. + * Add `showArtwork` support to `PresentationState` and use it to time + rendering of the provided optional `artwork` Composable in + `ContentFrame` and `Player`. Add `Artwork` Composable to be used as the + default value for the `artwork` slot. + * Add new fields to `PlayPauseButtonState` (`showBuffering`, `showRetry`, + `showReplay`) and use them in `PlayPauseButton` to render more icons and + a buffering indicator. + * Add `PlaybackState` class to allow custom playback state observation, + and introduce `BufferingIndicator` as a button-independent component. + * Fix an issue where `PlayerView` and `PlayerControlView` showed ad + markers at period boundaries for live postroll placeholders in + multi-period live streams. * Downloads: * OkHttp extension: * Cronet extension: * RTMP extension: * HLS extension: - * Parse `SCORE` attribute of `EXT-X-STREAM-INF` and - `EXT-X-I-FRAME-STREAM-INF`, and populate it to - `Format.selectionPriority` - ([#3236](https://github.com/androidx/media/pull/3236)). + * Fix regression where `Format.id` and `Format.metadata` were `null` for + exposed HLS tracks + ([#3402](https://github.com/androidx/media/issues/3402)). + * Clip asset-list ads to `X-PLAYOUT-LIMIT` and skip those that fall + entirely past the limit, and default the content resume offset to 0 when + an interstitial omits `X-RESUME-OFFSET` and its duration is unknown even + if `X-PLAYOUT-LIMIT` is set. + * Fix an issue where resolving an `X-ASSET-LIST` interstitial overwrote + the content resume offset contributions of other interstitials in the + same ad group. * DASH extension: - * Add support for parsing multiple `Location` elements from the manifest - (MPD) and performing location fallback in `DashMediaSource` when - manifest loads fail. + * Parse CICP color information (`ColourPrimaries`, + `TransferCharacteristics`, `MatrixCoefficients`, and + `VideoFullRangeFlag`) from `EssentialProperty` and + `SupplementalProperty` descriptors into `Format.colorInfo` + ([#3426](https://github.com/androidx/media/issues/3426)). + * Support whitespace-separated lists of `@id` values in trick mode + (`http://dashif.org/guidelines/trickmode`) descriptor `@value` + attributes ([#3315](https://github.com/androidx/media/issues/3315)). + * Support parsing `` in `DashManifestParser`, + exposing it in `AdaptationSet.producerReferenceTimes`, and using it to + populate `Timeline.Window.presentationStartTimeMs` and + `Timeline.Window.windowStartTimeMs` for VOD streams. + * Add support for DASH Content Steering + ([#1689](https://github.com/androidx/media/issues/1689)). + * Fix a `NullPointerException` that could occur when parsing a DASH + manifest with a missing attribute in `` element + ([#3436](https://github.com/androidx/media/issues/3436)). * Smooth Streaming extension: * RTSP extension: + * Fix an `IllegalStateException` crash that occurred when processing + delayed network responses after the RTSP client was closed. + * Fix issue where rapid scrubbing could incorrectly trigger the TCP + protocol fallback mechanic. + * Fix an `IllegalStateException` crash that can occur when processing + delayed network responses after the RTSP client resets state during + track reselection, session teardown, or uri redirection. + * Fix issue where seeking again prior to playback restart could cause an + `IllegalStateException` crash. + * Fix issue where seeking backward while a seek is already pending could + cause audio dropout and video playback freeze. + * Fix an issue where H.264 and H.265 keyframe access units ending with an + aggregation or non-IDR packet were not flagged as keyframes, causing + playback to remain stuck in buffering + ([#3434](https://github.com/androidx/media/issues/3434)). * Decoder extensions (FFmpeg, VP9, AV1, etc.): - * Add monochrome video support to bundled dav1d JNI. * MIDI extension: * Leanback extension: * Cast extension: - * Add `MediaRouteButtonState` and `rememberMediaRouteButtonState` to - support observing the visibility state of the media route picker, and - add `state` parameter to `MediaRouteButton` - ([3172](https://github.com/androidx/media/issues/3172)). - * Fix bug where `RemoteCastPlayer` could get permanently stuck in a - track-masking state after a track selection, causing it to ignore - subsequent track updates from the Cast receiver - ([#3237](https://github.com/androidx/media/issues/3237)). - * Fix bug where `DefaultMediaItemConverter` throws a - `NullPointerException` when trying to join a cast session started from a - non-Media3 sender. + * Introduce `SessionAvailabilityListener.SessionUnavailableReason` to + represent the reason for session unavailability, and provide an + overloaded method for + `SessionAvailabilityListener.onCastSessionUnavailable` to surface it. + * Fix an issue where playback would unexpectedly resume locally after + stopping a Cast session. + * Log an error when `RemoteMediaClient` operations (such as play, pause, + speed change, repeat mode, volume, mute, or track selection) fail in + `RemoteCastPlayer` + ([#3373](https://github.com/androidx/media/issues/3373)). + * Fix `ArrayIndexOutOfBoundsException` in `RemoteCastPlayer` when the Cast + receiver queue changes or shrinks while a seek operation is pending + ([#3372](https://github.com/androidx/media/issues/3372)). + * Fix an issue with the default Cast receiver where the timeline only + populated media items immediately surrounding the active item rather + than the entire playlist. + ([#3129](https://github.com/androidx/media/issues/3129)). + * Update the timeline and playback position immediately when modifying the + playlist (`setMediaItems`, `addMediaItems`, `moveMediaItems`, + `removeMediaItems`) or seeking in `RemoteCastPlayer`, and queue + successive playlist operations so they execute reliably in order + ([#2639](https://github.com/androidx/media/issues/2639), + [#3272](https://github.com/androidx/media/issues/3272)). + * Fix an issue where calling `addMediaItems` on an empty queue in + `RemoteCastPlayer` failed to load the media items on the Cast receiver. + ([#2402](https://github.com/androidx/media/issues/2402)). * Test Utilities: - * Add `setPlaybackLooper` to `TestExoPlayerBuilder`. * Remove deprecated symbols: - * Remove `androidx.media3.common.C.generateAudioSessionIdV21`. Use - `androidx.media3.common.util.Util.generateAudioSessionId` instead. + * Remove `DashUtil.buildDataSpec(Representation, RangedUri, int)` and + `DashUtil.buildDataSpec(Representation, String, RangedUri, int)`. Use + `DashUtil.buildDataSpec(Representation, BaseUrl, RangedUri, int, + Map)` instead. +* Build: + * Increase Gradle `minSdk` to 24. ## 1.11 -### 1.11.0-alpha01 (2026-06-23) +### 1.11.1 (2026-09-10) + +This release includes the following changes since the +[1.11.0 release](#1110-2026-08-05): + +* ExoPlayer: + * Move the flag to enable/disable HAGC metadata for progressive media from + `MediaSource.Factory` to `ExtractorsFactory`. The previous + `setExperimentalEnableHagcPlayback` method is removed and replaced by + `ExtractorsFactory.setParseHagcMetadata`. This also resolves an issue + where disabling didn't work for Matroska/WebM containers. + * Fix a playback stall caused when prewarming a non-transitioning + secondary renderer. + * Fix pre-warming issue where the `Surface` should be returned to the + primary renderer when a seek resets and disables both renderers. + * Fix `ArrayIndexOutOfBoundsException` when a live timeline refresh moves + the default position past a server-side inserted ad that is currently + being played ([#3348](https://github.com/androidx/media/issues/3348)). + * Fix a scrubbing mode issue where stale video frames could be briefly + displayed when seeking with + `ScrubbingModeParameters.allowSkippingMediaCodecFlush` enabled. +* Extractors: + * Matroska: Fix issue where Tracks placed after clusters wouldn't result + in a seekable timeline + ([#3377](https://github.com/androidx/media/issues/3377)). +* Audio: + * Fix buffer size calculation in `SilenceSkippingAudioProcessor` so that + the minimum silence duration is not incorrectly scaled down by the frame + size ([#3271](https://github.com/androidx/media/pull/3271)). + * Fix 8-bit PCM handling in `PcmAudioUtil` to treat samples as unsigned as + defined by Android + ([#3271](https://github.com/androidx/media/pull/3271)). + * Fix offload issue in which playback could stall during pre-roll or + gapless transitions due to limited hardware buffer sizes. + * Fix bug in `DefaultAudioSink` where release count doesn't decrease when + playback thread is no longer alive + ([#3338](https://github.com/androidx/media/issues/3338)). +* Session: + * Fix `NullPointerException` when an in-process `MediaController` is + released from a `Player.Listener` callback + ([#3375](https://github.com/androidx/media/issues/3375)). + * Fix deadlock on the main thread when a legacy `MediaBrowser` connects to + a service returning an asynchronous result from `onGetLibraryRoot()` or + `onConnectAsync()` + ([#3393](https://github.com/androidx/media/issues/3393)). +* HLS extension: + * Fix calculation of content resume offset when the target segment for + snapping is not yet in the playlist + ([#3322](https://github.com/androidx/media/issues/3322)). + * Fix an issue where a fully consumed `HlsMediaChunk` retries loading on + receiving `EOFException` from the extractor + ([#3350](https://github.com/androidx/media/issues/3350)). +* DASH extension: + * Fix incorrect sample timestamp calculation for image tracks with a + `presentationTimeOffset` + ([#3334](https://github.com/androidx/media/issues/3334)). +* Decoder extensions (FFmpeg, VP9, AV1, etc.): + * Opus: Fix memory corruption when multiple `OpusDecoder` instances are + initialized concurrently. + * MPEG-H: Fix memory leak, truncation of non-ASCII characters, and + potential native crash under low-memory conditions when sending commands + to the `MpeghUiManager` + ([#3365](https://github.com/androidx/media/issues/3365)). -This release includes the following changes since [1.10.1](#1101-2026-05-12): +### 1.11.0 (2026-08-05) + +This release includes the following changes since the +[1.10.1 release](#1101-2026-05-12): * Common library: * Add `Format.channelMask` to explicitly represent the audio channel mask, @@ -109,8 +412,6 @@ This release includes the following changes since [1.10.1](#1101-2026-05-12): count to decrease when an ad group is fully processed (`hasUnplayedAds()` is `false`), accommodating dynamic ad group resizing during reset workflows. - * Add support for ads in multi-period content (e.g., DASH) by splitting - and offsetting the `AdPlaybackState` for each period. * Add `getFlags()` and `FLAG_STRICT_DURATION` to `SampleStream` to allow streams to report flags, and update renderers to check these flags dynamically. @@ -180,6 +481,11 @@ This release includes the following changes since [1.10.1](#1101-2026-05-12): metadata out-of-band to the decoder on API 37+ devices. This behavior can be disabled via `DefaultMediaSourceFactory.setExperimentalEnableHagcPlayback(false)`. + * Add `Format.selectionPriority` representing the relative preference of + the `Format` in track selection + ([#3236](https://github.com/androidx/media/pull/3236)). + * Fix potential scrubbing mode issue where scrubbing could stall when + seeking to 'end of stream'. * CompositionPlayer: * Fix an issue where `TrackSelector` instances were not being released. * Transformer: @@ -189,6 +495,9 @@ This release includes the following changes since [1.10.1](#1101-2026-05-12): `AdaptiveTrackSelection.Factory.setTrackFormatComparator` to allow custom format ordering and ABR selection priority beyond bitrate-only ordering. + * Fix a timing-dependent concurrency issue where rapid sequential updates + to track selection parameters could result in some updates being lost or + overwritten. * Extractors: * MP4, MP3, and FLAC: Add `FLAG_DISABLE_ARTWORK_METADATA` to allow discarding attached pictures and cover art metadata during container @@ -233,6 +542,13 @@ This release includes the following changes since [1.10.1](#1101-2026-05-12): ([#3268](https://github.com/androidx/media/pull/3268)). * MP4: Allow variable length `sgpd` boxes in fragmented MP4 files ([#3243](https://github.com/androidx/media/pull/3243)). + * AVI: Fix an issue where playing files without keyframe flags on audio + tracks caused audio loss and `OutOfMemoryError`. + * MP3: Use gapless-aware durations from Xing/Info headers + ([#3183](https://github.com/androidx/media/issues/3183)). + * MP3: Adjust LAME/Xing encoder delay and padding metadata to match + decoded PCM trimming + ([#3200](https://github.com/androidx/media/pull/3200)). * Audio: * Add a 100ms grace period in ExoPlayer's audio renderers when transitioning from ready to not-ready state, to debounce transient @@ -263,6 +579,9 @@ This release includes the following changes since [1.10.1](#1101-2026-05-12): time-based audio processing ([#418](https://github.com/androidx/media/issues/418)). * Add support for big endian 32-bit and 64-bit floating point PCM. + * Remove fallback path of EAC3-JOC to EAC3 for playback on Google Pixel + devices as their standard EAC3 decoders cannot decode EAC3-JOC streams + ([#3257](https://github.com/androidx/media/pull/3257)). * Video: * Fix video joining logic to count dropped buffers during joining as skipped instead of dropped. @@ -273,6 +592,9 @@ This release includes the following changes since [1.10.1](#1101-2026-05-12): * Allow configuration of the threshold used to schedule frames early. The default is 50ms but one can customize the value through `MediaCodecVideoRenderer.Builder.setMaxEarlyUsThreshold()`. + * Fix bug in `VideoFrameReleaseHelper` where display changes could trigger + redundant `Choreographer` callbacks and excessive CPU wakeups on the + main thread. * Text: * TTML: Fallback to `displayAlign` from `style` for regions ([#2559](https://github.com/androidx/media/issues/2559)). @@ -334,6 +656,23 @@ This release includes the following changes since [1.10.1](#1101-2026-05-12): `Callback.onConnect` is a candidate to be deprecated. * Fix issue where Bluetooth AVRCP browsing didn't recognise `MediaLibraryService` on API 36 and 37. + * Fix double-downscaling of artwork in `MediaSession` when the image size + is close to the platform limit, resolving blurriness in notifications + ([#3134](https://github.com/androidx/media/issues/3134)). + * Fix `ForegroundServiceStartNotAllowedException` crash in + `MediaNotificationManager` when an asynchronous artwork bitmap load + callback attempts to start the foreground service while the app is in + the background ([#3270](https://github.com/androidx/media/issues/3270)). + * Fix an issue where `PositionInfo` objects received by a + `MediaController` were missing their original `Timeline` UIDs. The + session now reliably translates and propagates these UIDs during state + updates. + * Fix crash by catching `ForegroundServiceStartNotAllowedException` in + `MediaSessionService.stopSelfSafely()` + ([#3310](https://github.com/androidx/media/issues/3310)). + * Fix unexpected `onMediaItemTransition()` callback on `MediaController` + when seeking inside the same media item + ([#3248](https://github.com/androidx/media/issues/3248)). * UI: * Fix first frame rendering at the wrong size when `ContentFrame` is recomposed mid-playback @@ -364,18 +703,49 @@ This release includes the following changes since [1.10.1](#1101-2026-05-12): (`ui-compose`) to support preloading of `MediaItems` in sliding window UI environments. Showcase this functionality in `demo-compose` with a new `ShortFormPlayerScreen`. + * Fix shutter suppression bug in `PlayerView` and Compose + `PresentationState` that caused crashes on timelines that do not support + period UID lookup + ([#3264](https://github.com/androidx/media/issues/3264)). +* Downloads: + * Resolve HLS variables in media playlist when fetched for download + ([#3258](https://github.com/androidx/media/issues/3258)). * Ktor extension: * Add a new `media3-datasource-ktor` extension module providing a `KtorDataSource` backed by the [Ktor](https://ktor.io/) HTTP stack. + * Fix `KtorDataSource` loading the entire HTTP response body into memory + when opening a connection + ([#3305](https://github.com/androidx/media/issues/3305)). * HLS extension: * Add support for HLS Content Steering and Pathway Cloning ([#1689](https://github.com/androidx/media/issues/1689)). * Add `setWithAssetListReset` to `HlsInterstitialsAdsLoader` to allow apps to reset fully processed asset list interstitials back to an unavailable state for on-demand re-resolution. + * Parse `SCORE` attribute of `EXT-X-STREAM-INF` and + `EXT-X-I-FRAME-STREAM-INF`, and populate it to + `Format.selectionPriority` + ([#3236](https://github.com/androidx/media/pull/3236)). + * Fix bug where scheduled HLS playlist refreshes continue even if the + playlist is no longer primary or active + ([#3254](https://github.com/androidx/media/issues/3254)). + * Fix calculation of content resume offset when resolving interstitial + asset lists ([#3322](https://github.com/androidx/media/issues/3322)). + * Fix unexpected audio track timestamp discontinuity when resuming low + latency live playback after a playlist request failure + ([#3311](https://github.com/androidx/media/issues/3311)). + * Fix bug in `HlsMediaPeriod.getStreamKeys()` where the ID3 track in audio + renditions is considered as subtitles + ([#3333](https://github.com/androidx/media/issues/3333)). * DASH extension: * Separate CEA closed captions into different `TrackGroup`s based on language ([#3113](https://github.com/androidx/media/issues/3113)). + * Add support for parsing multiple `Location` elements from the manifest + (MPD) and performing location fallback in `DashMediaSource` when + manifest loads fail. + * Fix crash (`IndexOutOfBoundsException`) when opening manifests with an + empty `` element + ([#3326](https://github.com/androidx/media/issues/3326)). * RTSP extension: * Fix transient playback stalls or failures during RTSP playback preparation caused by UDP port binding errors. @@ -385,6 +755,23 @@ This release includes the following changes since [1.10.1](#1101-2026-05-12): * Add new `CastParams.getShowSystemOutputSwitcherOnCastIconClick()` option, which triggers the use of the SystemUI Output Switcher instead of the in-app Cast device picker when available. + * Add `MediaRouteButtonState` and `rememberMediaRouteButtonState` to + support observing the visibility state of the media route picker, and + add `state` parameter to `MediaRouteButton` + ([#3172](https://github.com/androidx/media/issues/3172)). + * Fix bug where `RemoteCastPlayer` could get permanently stuck in a + track-masking state after a track selection, causing it to ignore + subsequent track updates from the Cast receiver + ([#3237](https://github.com/androidx/media/issues/3237)). + * Fix bug where `DefaultMediaItemConverter` throws a + `NullPointerException` when trying to join a cast session started from a + non-Media3 sender. + * Prevent `CastPlayer` from transferring `MediaItems` that are not + playable when changing the active player + ([#3199](https://github.com/androidx/media/issues/3199)). + * Fix bug where unconfigured live streams (`liveConfiguration == UNSET`) + were queued as `STREAM_TYPE_BUFFERED` + ([#3318](https://github.com/androidx/media/issues/3318)). * Test utilities: * Expand `DataSourceContractTest` to include support for `POST` requests with headers & bodies, add some `POST` resources to @@ -394,6 +781,7 @@ This release includes the following changes since [1.10.1](#1101-2026-05-12): `DataSource.open()`. Resources that may result in `C.LENGTH_UNSET` should indicate this by using `TestResource.Builder.setMayResolveToUnknownLength(boolean)`. + * Add `setPlaybackLooper` to `TestExoPlayerBuilder`. * Remove deprecated symbols: * Remove `androidx.media3.exoplayer.MetadataRetriever`. Use `androidx.media3.inspector.MetadataRetriever` instead. @@ -408,12 +796,26 @@ This release includes the following changes since [1.10.1](#1101-2026-05-12): `androidx.media3.extractor.DiscardingTrackOutput` instead. * Remove `androidx.media3.extractor.DummyExtractorOutput`. Use `androidx.media3.extractor.NoOpExtractorOutput` instead. + * Remove `androidx.media3.common.C.generateAudioSessionIdV21`. Use + `androidx.media3.common.util.Util.generateAudioSessionId` instead. * Build * Updated the `README.md` instructions for depending on Media3 locally to use Gradle Composite Builds (`includeBuild`). The `core_settings.gradle` script has been removed, and the `androidxMediaModulePrefix` and `androidxMediaEnableMidiModule` properties are no longer used. +### 1.11.0-rc01 (2026-07-22) + +Use the 1.11.0 [stable version](#1110-2026-08-05). + +### 1.11.0-beta01 (2026-07-08) + +Use the 1.11.0 [stable version](#1110-2026-08-05). + +### 1.11.0-alpha01 (2026-06-23) + +Use the 1.11.0 [stable version](#1110-2026-08-05). + ## 1.10 ### 1.10.1 (2026-05-12) diff --git a/api.txt b/api.txt index 97e4216e2d2..1616ab4035c 100644 --- a/api.txt +++ b/api.txt @@ -1646,7 +1646,7 @@ package androidx.media3.exoplayer.audio { public static final class AudioTrackAudioOutputProvider.Builder { ctor public AudioTrackAudioOutputProvider.Builder(@Nullable _: android.content.Context); method public build(): androidx.media3.exoplayer.audio.AudioTrackAudioOutputProvider; - method @RequiresApi(24) public setAudioTrackBuilderModifier(_: java.util.function.BiConsumer): androidx.media3.exoplayer.audio.AudioTrackAudioOutputProvider.Builder; + method public setAudioTrackBuilderModifier(_: java.util.function.BiConsumer): androidx.media3.exoplayer.audio.AudioTrackAudioOutputProvider.Builder; } public class ForwardingAudioOutput implements androidx.media3.exoplayer.audio.AudioOutput { @@ -1791,6 +1791,7 @@ package androidx.media3.muxer { method public stop(): void; method public writeSampleData(_: int, _: java.nio.ByteBuffer, _: android.media.MediaCodec.BufferInfo): void; field public static final OUTPUT_FORMAT_MP4: int = 0; // 0x0 + field public static final OUTPUT_FORMAT_WEBM: int = 1; // 0x1 } } diff --git a/build-logic-settings/src/main/kotlin/androidx/media3/buildlogic/Media3Modules.kt b/build-logic-settings/src/main/kotlin/androidx/media3/buildlogic/Media3Modules.kt index 236aa800b32..6a5bba51578 100644 --- a/build-logic-settings/src/main/kotlin/androidx/media3/buildlogic/Media3Modules.kt +++ b/build-logic-settings/src/main/kotlin/androidx/media3/buildlogic/Media3Modules.kt @@ -115,6 +115,13 @@ object Media3Modules { ), "lib-effect-ndk" to Media3Module("libraries/effect_ndk", "media3-effect-ndk", "Media3 Effect NDK module"), + "lib-effect-play-services" to + Media3Module( + "libraries/effect_play_services", + "media3-effect-play-services", + "Media3 Effect Play Services module", + allowKt = false, + ), "lib-exoplayer" to Media3Module( "libraries/exoplayer", @@ -233,6 +240,8 @@ object Media3Modules { "test-exoplayer-playback" to Media3Module("libraries/test_exoplayer_playback", includeInCompositeBuild = false), "test-proguard" to Media3Module("libraries/test_proguard", includeInCompositeBuild = false), + "test-proguard-minimal" to + Media3Module("libraries/test_proguard_minimal", includeInCompositeBuild = false), "test-session-common" to Media3Module("libraries/test_session_common", includeInCompositeBuild = false), "test-session-current" to diff --git a/build-logic/build.gradle.kts b/build-logic/build.gradle.kts index 1ea60326a58..7c9d5f67a9f 100644 --- a/build-logic/build.gradle.kts +++ b/build-logic/build.gradle.kts @@ -21,7 +21,7 @@ repositories { dependencies { implementation("androidx.media3.buildlogic:build-logic-settings") - implementation(libs.android.gradle.plugin) - implementation(libs.android.gradle.api) - implementation(libs.kotlin.gradle.plugin) + compileOnly(libs.android.gradle.plugin) + compileOnly(libs.android.gradle.api) + compileOnly(libs.kotlin.gradle.plugin) } diff --git a/build-logic/src/main/kotlin/androidx/media3/buildlogic/StripGlslCommentsTask.kt b/build-logic/src/main/kotlin/androidx/media3/buildlogic/StripGlslCommentsTask.kt new file mode 100644 index 00000000000..b540363af45 --- /dev/null +++ b/build-logic/src/main/kotlin/androidx/media3/buildlogic/StripGlslCommentsTask.kt @@ -0,0 +1,72 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.buildlogic + +import org.gradle.api.DefaultTask +import org.gradle.api.file.ConfigurableFileCollection +import org.gradle.api.file.DirectoryProperty +import org.gradle.api.tasks.CacheableTask +import org.gradle.api.tasks.InputFiles +import org.gradle.api.tasks.OutputDirectory +import org.gradle.api.tasks.PathSensitive +import org.gradle.api.tasks.PathSensitivity +import org.gradle.api.tasks.TaskAction + +/** + * Copies GLSL shaders to the `raw` subdirectory of a resource directory, with comments removed to + * reduce APK size. + * + * Line breaks are kept, so line numbers in shader compilation errors match the source files. + */ +@CacheableTask +abstract class StripGlslCommentsTask : DefaultTask() { + + /** The GLSL files to copy. */ + @get:InputFiles + @get:PathSensitive(PathSensitivity.NAME_ONLY) + abstract val shaders: ConfigurableFileCollection + + /** The resource directory to write the shaders to. */ + @get:OutputDirectory abstract val outputDirectory: DirectoryProperty + + @TaskAction + fun stripComments() { + val resourceDirectory = outputDirectory.get().asFile + resourceDirectory.deleteRecursively() + val rawDirectory = resourceDirectory.resolve("raw").apply { mkdirs() } + for (shader in shaders) { + rawDirectory.resolve(shader.name).writeText(stripGlslComments(shader.readText())) + } + } +} + +// GLSL has no string literals, so a regular expression is enough to find the comments. +private val GLSL_COMMENT = Regex("""//[^\n]*|/\*[\s\S]*?\*/""") + +/** + * Returns [glsl] with its comments removed. + * + * Each comment is replaced with the line breaks it contains, or with a space if it has none. Then + * trailing whitespace is removed from each line. + */ +internal fun stripGlslComments(glsl: String): String = + glsl + .replace(GLSL_COMMENT) { comment -> + "\n".repeat(comment.value.count { it == '\n' }).ifEmpty { " " } + } + .lines() + .joinToString("\n") { it.trimEnd() } diff --git a/build-logic/src/main/kotlin/androidx/media3/buildlogic/commonConfig.kt b/build-logic/src/main/kotlin/androidx/media3/buildlogic/commonConfig.kt index 9b013c79038..dd157b9a081 100644 --- a/build-logic/src/main/kotlin/androidx/media3/buildlogic/commonConfig.kt +++ b/build-logic/src/main/kotlin/androidx/media3/buildlogic/commonConfig.kt @@ -64,6 +64,9 @@ fun Project.configureCommonConfig(android: CommonExtension, libs: VersionCatalog } } + // TODO: b/502167525 - Upgrade source/targetCompatibility to JavaVersion.VERSION_11. + // AndroidX bytecode has moved to Java 11 (b/457821470). When androidx.compose.bom is updated, + // Kotlin consumers calling inline Compose APIs will require JVM target 11. compileOptions.apply { sourceCompatibility = JavaVersion.VERSION_1_8 targetCompatibility = JavaVersion.VERSION_1_8 diff --git a/build.gradle.kts b/build.gradle.kts index 0c8ecea043a..7c3d7cba18a 100644 --- a/build.gradle.kts +++ b/build.gradle.kts @@ -34,6 +34,9 @@ buildscript { } plugins { + alias(libs.plugins.android.application) apply false + alias(libs.plugins.android.library) apply false + alias(libs.plugins.kotlin.android) apply false id("media3.android-application") apply false id("media3.android-library") apply false alias(libs.plugins.kotlin.compose.compiler) apply false diff --git a/demos/cast/src/main/res/mipmap-hdpi/ic_launcher.png b/demos/cast/src/main/res/mipmap-hdpi/ic_launcher.png index 52e8dc93d98..981a8bb5c70 100644 Binary files a/demos/cast/src/main/res/mipmap-hdpi/ic_launcher.png and b/demos/cast/src/main/res/mipmap-hdpi/ic_launcher.png differ diff --git a/demos/cast/src/main/res/mipmap-mdpi/ic_launcher.png b/demos/cast/src/main/res/mipmap-mdpi/ic_launcher.png index b55576eff33..d10e3fc5fa9 100644 Binary files a/demos/cast/src/main/res/mipmap-mdpi/ic_launcher.png and b/demos/cast/src/main/res/mipmap-mdpi/ic_launcher.png differ diff --git a/demos/cast/src/main/res/mipmap-xhdpi/ic_launcher.png b/demos/cast/src/main/res/mipmap-xhdpi/ic_launcher.png index ca84d6a60e7..3cf84e34255 100644 Binary files a/demos/cast/src/main/res/mipmap-xhdpi/ic_launcher.png and b/demos/cast/src/main/res/mipmap-xhdpi/ic_launcher.png differ diff --git a/demos/cast/src/main/res/mipmap-xxhdpi/ic_launcher.png b/demos/cast/src/main/res/mipmap-xxhdpi/ic_launcher.png index 27ab9b1054e..9f35c9aeb76 100644 Binary files a/demos/cast/src/main/res/mipmap-xxhdpi/ic_launcher.png and b/demos/cast/src/main/res/mipmap-xxhdpi/ic_launcher.png differ diff --git a/demos/cast/src/main/res/mipmap-xxxhdpi/ic_launcher.png b/demos/cast/src/main/res/mipmap-xxxhdpi/ic_launcher.png index d1eb9b78cf1..48fd939c651 100644 Binary files a/demos/cast/src/main/res/mipmap-xxxhdpi/ic_launcher.png and b/demos/cast/src/main/res/mipmap-xxxhdpi/ic_launcher.png differ diff --git a/demos/compose/build.gradle.kts b/demos/compose/build.gradle.kts index 394ac6c7a60..568a719a80f 100644 --- a/demos/compose/build.gradle.kts +++ b/demos/compose/build.gradle.kts @@ -20,6 +20,13 @@ plugins { android { namespace = "androidx.media3.demo.compose" + // TODO: b/502167525 - Remove this temporary JVM 11 override once commonConfig.kt + // is migrated to Java 11 and the alpha28 dependency is reverted back to the BOM. + compileOptions { + sourceCompatibility = JavaVersion.VERSION_11 + targetCompatibility = JavaVersion.VERSION_11 + } + buildTypes { getByName("release") { isShrinkResources = true @@ -38,13 +45,20 @@ dependencies { implementation(libs.androidx.activity.compose) implementation(libs.androidx.appcompat) implementation(libs.androidx.lifecycle.runtime.compose) + implementation(libs.androidx.lifecycle.viewmodel) implementation(libs.androidx.navigation.compose) implementation(libs.material) implementation(libs.kotlinx.coroutines.guava) + implementation(project(":lib-cast")) implementation(project(":lib-exoplayer")) + implementation(project(":lib-effect")) implementation(project(":lib-inspector")) + implementation(project(":lib-inspector-frame")) implementation(project(":lib-ui-compose-material3")) + // TODO(b/555710150): Revert this once our Gradle BOM is updated to a stable version that + // includes the M3 API changes (Slider). + implementation("androidx.compose.material3:material3:1.5.0-alpha28") // For detecting and debugging leaks only. LeakCanary is not needed for demo app to work. debugImplementation(libs.leakcanary.android) diff --git a/demos/compose/src/main/AndroidManifest.xml b/demos/compose/src/main/AndroidManifest.xml index d6d6f8e33a9..595b159d373 100644 --- a/demos/compose/src/main/AndroidManifest.xml +++ b/demos/compose/src/main/AndroidManifest.xml @@ -20,6 +20,7 @@ + + + diff --git a/demos/compose/src/main/assets/artwork/ic_launcher_round.webp b/demos/compose/src/main/assets/artwork/ic_launcher_round.webp new file mode 100644 index 00000000000..9126ae37cbc Binary files /dev/null and b/demos/compose/src/main/assets/artwork/ic_launcher_round.webp differ diff --git a/demos/compose/src/main/assets/media.exolist.json b/demos/compose/src/main/assets/media.exolist.json index 17d6a5b30a3..a6139db3936 100644 --- a/demos/compose/src/main/assets/media.exolist.json +++ b/demos/compose/src/main/assets/media.exolist.json @@ -41,7 +41,7 @@ ] }, { - "name": "Audios (w/ artwork)", + "name": "Artwork", "playlist": [ { "name": "The Way Of Waking Up", @@ -58,18 +58,52 @@ "artwork_uri": "asset:///artwork/album_kai_engel.png" }, { - "name": "Jazz in Paris", + "name": "Audio and Asset URI Artwork", "album": "Jazz & Blues", "artist": "Media Right Productions", "uri": "https://storage.googleapis.com/exoplayer-test-media-1/mp3/Jazz_In_Paris.mp3", "artwork_uri": "asset:///artwork/album_sea.png" }, { - "name": "On the Bach", + "name": "Audio and HTTPS Artwork", "album": "Cinematic", "artist": "Jingle Punks", "uri": "https://storage.googleapis.com/exoplayer-test-media-1/mp3/On_the_Bach.mp3", - "artwork_uri": "asset:///artwork/album_drinks.png" + "artwork_uri": "https://storage.googleapis.com/exoplayer-test-media-1/mp3/album_art_3.jpg" + }, + { + "name": "Audio and Embedded Artwork", + "album": "Google Play", + "uri": "https://storage.googleapis.com/exoplayer-test-media-0/play.mp3" + }, + { + "name": "Audio and Content URI Artwork", + "artist": "Jingle Punks", + "uri": "https://storage.googleapis.com/exoplayer-test-media-1/mp3/On_the_Bach.mp3", + "artwork_uri": "content://androidx.media3.demo.compose/artwork/album_sea.png" + }, + { + "name": "Video and Asset URI Artwork", + "uri": "https://storage.googleapis.com/exoplayer-test-media-1/mp4/android-screens-10s.mp4", + "artwork_uri": "asset:///artwork/album_road.png" + }, + { + "name": "Audio and Resource URI Artwork", + "artist": "Jingle Punks", + "uri": "https://storage.googleapis.com/exoplayer-test-media-1/mp3/On_the_Bach.mp3", + "artwork_uri": "android.resource://androidx.media3.demo.compose/mipmap/ic_launcher" + }, + { + "name": "Audio and WebP Artwork", + "artist": "Jingle Punks", + "uri": "https://storage.googleapis.com/exoplayer-test-media-1/mp3/On_the_Bach.mp3", + "artwork_uri": "asset:///artwork/ic_launcher_round.webp" + }, + { + "name": "Audio and Invalid Artwork", + "artist": "Jingle Punks", + "uri": "https://storage.googleapis.com/exoplayer-test-media-1/mp3/On_the_Bach.mp3", + "artwork_uri": "asset:///artwork/does_not_exist.png" } ] } @@ -110,10 +144,6 @@ "name": "Screens audio (FMP4)", "uri": "https://storage.googleapis.com/exoplayer-test-media-1/gen-3/screens/dash-vod-single-segment/audio-141.mp4" }, - { - "name": "Google Play (MP3)", - "uri": "https://storage.googleapis.com/exoplayer-test-media-0/play.mp3" - }, { "name": "Google Play (Ogg, Vorbis)", "uri": "https://storage.googleapis.com/exoplayer-test-media-1/ogg/play.ogg" @@ -162,6 +192,10 @@ "name": "AV1 Rooftop (Everyframe T35)", "uri": "https://storage.googleapis.com/exoplayer-test-media-1/hagc/rooftop_av1_t35track_everyframe_v0.mp4" }, + { + "name": "HEVC Rooftop (Everyframe T35)", + "uri": "https://storage.googleapis.com/exoplayer-test-media-1/hagc/rooftop_hevc_t35track_everyframe_v0.mp4" + }, { "name": "HEVC Lights (SDR Base, Sparse T35)", "uri": "https://storage.googleapis.com/exoplayer-test-media-1/hagc/lights_hevc_sdrbase_t35track_v0.mp4" diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/BitmapContentProvider.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/BitmapContentProvider.kt new file mode 100644 index 00000000000..608ead5445b --- /dev/null +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/BitmapContentProvider.kt @@ -0,0 +1,83 @@ +/* + * Copyright 2023 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.demo.compose + +import android.content.ContentProvider +import android.content.ContentValues +import android.content.Context +import android.database.Cursor +import android.net.Uri +import android.os.ParcelFileDescriptor +import java.io.File + +/** + * Provides artwork for content URIs. + * + *

A bitmap file in the asset folder with path 'artwork/album1.png' can be referenced as artwork + * URI with 'content://androidx.media3.demo.compose/artwork/album1.png'. + * 'androidx.media3.demo.compose' is the authority declared for the content provider in + * 'AndroidManifest.xml'. + * + *

For demo use only. + */ +class BitmapContentProvider : ContentProvider() { + + override fun onCreate() = true + + override fun openFile(uri: Uri, mode: String): ParcelFileDescriptor? { + val ctx = context ?: return super.openFile(uri, mode) + val assetPath = uri.path?.drop(1) ?: return super.openFile(uri, mode) + + val file = copyAssetFileToCacheDirectory(ctx, assetPath) + return ParcelFileDescriptor.open(file, ParcelFileDescriptor.MODE_READ_ONLY) + } + + private fun copyAssetFileToCacheDirectory(context: Context, assetPath: String): File { + val publicFile = File(context.cacheDir, assetPath.replace("/", "_")) + synchronized(this) { + if (!publicFile.exists()) { + context.assets.open(assetPath).use { input -> + publicFile.outputStream().use { output -> input.copyTo(output) } + } + } + } + return publicFile + } + + // No-op implementations of abstract ContentProvider methods. + + override fun insert(uri: Uri, values: ContentValues?): Uri? = null + + override fun query( + uri: Uri, + projection: Array?, + selection: String?, + selectionArgs: Array?, + sortOrder: String?, + ): Cursor? = null + + override fun update( + uri: Uri, + values: ContentValues?, + selection: String?, + selectionArgs: Array?, + ) = 0 + + override fun delete(uri: Uri, selection: String?, selectionArgs: Array?) = 0 + + override fun getType(uri: Uri): String? = null +} diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/DemoComposeApplication.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/DemoComposeApplication.kt new file mode 100644 index 00000000000..673aefad19a --- /dev/null +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/DemoComposeApplication.kt @@ -0,0 +1,32 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.demo.compose + +import android.app.Application +import androidx.annotation.OptIn +import androidx.media3.cast.Cast +import androidx.media3.cast.CastParams +import androidx.media3.common.util.UnstableApi + +/** The main application class for the Compose demo app. */ +class DemoComposeApplication : Application() { + @OptIn(UnstableApi::class) // Cast is an unstable API. + override fun onCreate() { + super.onCreate() + Cast.getSingletonInstance(this) + .initialize(CastParams.Builder().setShowSystemOutputSwitcherOnCastButtonClick(true).build()) + } +} diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/MainActivity.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/MainActivity.kt index 9c25d4318f3..7ea80b5c485 100644 --- a/demos/compose/src/main/java/androidx/media3/demo/compose/MainActivity.kt +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/MainActivity.kt @@ -32,35 +32,52 @@ import androidx.compose.material3.dynamicDarkColorScheme import androidx.compose.material3.dynamicLightColorScheme import androidx.compose.material3.lightColorScheme import androidx.compose.runtime.Composable +import androidx.compose.runtime.LaunchedEffect import androidx.compose.runtime.collectAsState import androidx.compose.runtime.getValue import androidx.compose.ui.Modifier import androidx.compose.ui.platform.LocalContext +import androidx.media3.demo.compose.layout.EditingPlayerScreen import androidx.media3.demo.compose.layout.LongFormPlayerScreen import androidx.media3.demo.compose.layout.PlayerFormatScreen import androidx.media3.demo.compose.layout.SampleChooserScreen import androidx.media3.demo.compose.layout.ShortFormPlayerScreen -import androidx.media3.demo.compose.viewmodel.ComposeDemoViewModel +import androidx.media3.demo.compose.viewmodel.NavigationViewModel +import androidx.media3.demo.compose.viewmodel.PlayerLifecycleViewModel import androidx.navigation.compose.NavHost import androidx.navigation.compose.composable import androidx.navigation.compose.rememberNavController class MainActivity : ComponentActivity() { - private val sharedViewModel: ComposeDemoViewModel by viewModels() + private val navigationViewModel: NavigationViewModel by viewModels() + private val playerViewModel: PlayerLifecycleViewModel by viewModels() override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) enableEdgeToEdge() - setContent { ComposeDemoApp(modifier = Modifier.fillMaxSize(), viewModel = sharedViewModel) } + setContent { + ComposeDemo( + navigationViewModel = navigationViewModel, + playerViewModel = playerViewModel, + modifier = Modifier.fillMaxSize(), + ) + } } } @Composable -private fun ComposeDemoApp(modifier: Modifier = Modifier, viewModel: ComposeDemoViewModel) { +private fun ComposeDemo( + navigationViewModel: NavigationViewModel, + playerViewModel: PlayerLifecycleViewModel, + modifier: Modifier = Modifier, +) { val context = LocalContext.current val isDarkTheme = isSystemInDarkTheme() val supportsDynamicColor = Build.VERSION.SDK_INT >= Build.VERSION_CODES.S + val activity = context as? ComponentActivity + LaunchedEffect(isDarkTheme) { activity?.enableEdgeToEdge() } + val colorScheme = when { supportsDynamicColor && isDarkTheme -> dynamicDarkColorScheme(context) @@ -80,30 +97,41 @@ private fun ComposeDemoApp(modifier: Modifier = Modifier, viewModel: ComposeDemo composable(ROUTE_SAMPLE_CHOOSER) { SampleChooserScreen( onPlaylistClick = { selectedPlaylistName, selectedMedia -> - viewModel.selectPlaylistName(selectedPlaylistName) - viewModel.selectMediaItems(selectedMedia) + navigationViewModel.selectPlaylistName(selectedPlaylistName) + navigationViewModel.selectMediaItems(selectedMedia) navController.navigate(ROUTE_PLAYER_FORMAT_CHOOSER) }, modifier = modifier.statusBarsPadding(), ) } composable(ROUTE_PLAYER_FORMAT_CHOOSER) { + val mediaItems by navigationViewModel.mediaItems.collectAsState() PlayerFormatScreen( onLongFormClick = { navController.navigate(ROUTE_LONG_FORM_PLAYER) }, onShortFormClick = { navController.navigate(ROUTE_SHORT_FORM_PLAYER) }, + onEditingClick = + if (mediaItems.size == 1) { + { navController.navigate(ROUTE_EDITING_PLAYER) } + } else null, modifier = modifier.statusBarsPadding(), ) } composable(ROUTE_LONG_FORM_PLAYER) { - val mediaItems by viewModel.mediaItems.collectAsState() - val playlistName by viewModel.playlistName.collectAsState() - LongFormPlayerScreen(playlistName, mediaItems) + val mediaItems by navigationViewModel.mediaItems.collectAsState() + val playlistName by navigationViewModel.playlistName.collectAsState() + LongFormPlayerScreen(playlistName, mediaItems, playerViewModel) } composable(ROUTE_SHORT_FORM_PLAYER) { - val mediaItems by viewModel.mediaItems.collectAsState() - val playlistName by viewModel.playlistName.collectAsState() + val mediaItems by navigationViewModel.mediaItems.collectAsState() + val playlistName by navigationViewModel.playlistName.collectAsState() ShortFormPlayerScreen(playlistName, mediaItems) } + composable(ROUTE_EDITING_PLAYER) { + val mediaItems by navigationViewModel.mediaItems.collectAsState() + if (mediaItems.size == 1) { + EditingPlayerScreen(mediaItems.single(), playerViewModel) + } + } } } } @@ -112,3 +140,4 @@ private const val ROUTE_SAMPLE_CHOOSER = "sample_chooser" private const val ROUTE_PLAYER_FORMAT_CHOOSER = "player_format_chooser" private const val ROUTE_LONG_FORM_PLAYER = "long_form_player" private const val ROUTE_SHORT_FORM_PLAYER = "short_form_player" +private const val ROUTE_EDITING_PLAYER = "editing_player" diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/buttons/CcButton.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/buttons/CcButton.kt new file mode 100644 index 00000000000..f99dfebb06e --- /dev/null +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/buttons/CcButton.kt @@ -0,0 +1,84 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.demo.compose.buttons + +import androidx.compose.foundation.layout.size +import androidx.compose.material3.Icon +import androidx.compose.material3.IconButton +import androidx.compose.material3.IconButtonDefaults +import androidx.compose.runtime.Composable +import androidx.compose.runtime.LaunchedEffect +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.setValue +import androidx.compose.ui.Modifier +import androidx.compose.ui.res.painterResource +import androidx.compose.ui.unit.dp +import androidx.media3.common.C +import androidx.media3.common.Player +import androidx.media3.ui.compose.material3.R +import androidx.media3.ui.compose.state.observeState + +@Composable +internal fun CcButton(player: Player?, modifier: Modifier = Modifier) { + var isCcEnabled by + remember(player) { + mutableStateOf(player?.currentTracks?.isTypeSelected(C.TRACK_TYPE_TEXT) ?: false) + } + var hasTextTrack by + remember(player) { + mutableStateOf(player?.currentTracks?.containsType(C.TRACK_TYPE_TEXT) ?: false) + } + + val observer = + remember(player) { + player?.observeState(Player.EVENT_TRACKS_CHANGED) { + isCcEnabled = player.currentTracks.isTypeSelected(C.TRACK_TYPE_TEXT) + hasTextTrack = player.currentTracks.containsType(C.TRACK_TYPE_TEXT) + } + } + + LaunchedEffect(observer) { observer?.observe() } + + IconButton( + onClick = { + player?.let { + val currentlyTextSelected = it.currentTracks.isTypeSelected(C.TRACK_TYPE_TEXT) + it.trackSelectionParameters = + it.trackSelectionParameters + .buildUpon() + .setTrackTypeDisabled(C.TRACK_TYPE_TEXT, currentlyTextSelected) + .setSelectTextByDefault(!currentlyTextSelected) + .build() + } + }, + modifier = modifier, + enabled = hasTextTrack, + colors = IconButtonDefaults.iconButtonColors(), + ) { + Icon( + modifier = Modifier.size(24.dp), + painter = + painterResource( + if (isCcEnabled) R.drawable.media3_icon_subtitles + else R.drawable.media3_icon_subtitles_off + ), + contentDescription = if (isCcEnabled) "Disable closed captions" else "Enable closed captions", + ) + } +} diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/buttons/LabeledProgressSlider.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/buttons/LabeledProgressSlider.kt index 7db63fb50d6..68dc1ada55c 100644 --- a/demos/compose/src/main/java/androidx/media3/demo/compose/buttons/LabeledProgressSlider.kt +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/buttons/LabeledProgressSlider.kt @@ -16,6 +16,7 @@ package androidx.media3.demo.compose.buttons +import androidx.compose.foundation.Canvas import androidx.compose.foundation.background import androidx.compose.foundation.interaction.DragInteraction import androidx.compose.foundation.interaction.FocusInteraction @@ -25,7 +26,6 @@ import androidx.compose.foundation.interaction.PressInteraction import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.padding import androidx.compose.foundation.shape.RoundedCornerShape -import androidx.compose.material3.ExperimentalMaterial3Api import androidx.compose.material3.MaterialTheme import androidx.compose.material3.Slider import androidx.compose.material3.SliderColors @@ -33,6 +33,7 @@ import androidx.compose.material3.SliderDefaults import androidx.compose.material3.SliderState import androidx.compose.material3.Text import androidx.compose.runtime.Composable +import androidx.compose.runtime.DisposableEffect import androidx.compose.runtime.LaunchedEffect import androidx.compose.runtime.getValue import androidx.compose.runtime.mutableFloatStateOf @@ -44,6 +45,9 @@ import androidx.compose.runtime.rememberCoroutineScope import androidx.compose.runtime.setValue import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier +import androidx.compose.ui.geometry.CornerRadius +import androidx.compose.ui.geometry.RoundRect +import androidx.compose.ui.graphics.Path import androidx.compose.ui.layout.layout import androidx.compose.ui.layout.onSizeChanged import androidx.compose.ui.unit.Dp @@ -51,7 +55,9 @@ import androidx.compose.ui.unit.DpSize import androidx.compose.ui.unit.dp import androidx.media3.common.Player import androidx.media3.common.util.Util +import androidx.media3.exoplayer.ExoPlayer import androidx.media3.ui.compose.indicators.ProgressIndicator +import androidx.media3.ui.compose.material3.util.isScrubbingModeEnabled import androidx.media3.ui.compose.state.ProgressStateWithTickCount import kotlinx.coroutines.CoroutineScope @@ -74,12 +80,17 @@ import kotlinx.coroutines.CoroutineScope * [Interactions][androidx.compose.foundation.interaction.Interaction] for this slider. You can * create and pass in your own `remember`ed instance to observe `Interactions` and customize the * appearance / behavior of this slider in different states. + * @param thumbTrackGapSize The gap between the thumb and the track. Set to `0.dp` to remove the gap + * completely. * @param thumb A custom thumb to be displayed on the slider, it is placed on top of the track. The - * lambda receives a [SliderState] which is used to obtain the current active track. If `null`, a - * standard Material3 `SliderDefaults.Thumb` will be shown. + * lambda receives the [ProgressStateWithTickCount], the current [SliderState], and a boolean + * indicating whether changing progress is enabled. If `null`, a standard Material3 + * `SliderDefaults.Thumb` will be shown. + * @param track A custom track to be displayed on the slider, it is placed underneath the thumb. The + * lambda receives the [ProgressStateWithTickCount], the current [SliderState], and a boolean + * indicating whether changing progress is enabled. If `null`, a standard Material3 + * `SliderDefaults.Track` will be shown. */ -// TODO: b/304811984 - publish this Slider when the overload with thumb/track is stabilized -@OptIn(ExperimentalMaterial3Api::class) @Composable fun LabeledProgressSlider( player: Player?, @@ -89,25 +100,60 @@ fun LabeledProgressSlider( scope: CoroutineScope = rememberCoroutineScope(), colors: SliderColors = SliderDefaults.colors(), interactionSource: MutableInteractionSource = remember { MutableInteractionSource() }, - thumb: (@Composable (ProgressStateWithTickCount, SliderState) -> Unit)? = - { progressState, sliderState -> - SeekTimeThumb(progressState, sliderState, interactionSource) + thumbTrackGapSize: Dp = ThumbTrackGapSize, + thumb: (@Composable (ProgressStateWithTickCount, SliderState, Boolean) -> Unit)? = + { progressState, sliderState, enabled -> + SeekTimeThumb(progressState, sliderState, interactionSource, enabled) + }, + track: @Composable ((ProgressStateWithTickCount, SliderState, Boolean) -> Unit)? = + { progressState, sliderState, enabled -> + BufferingTrack( + progressState, + sliderState, + enabled, + thumbTrackGapSize = thumbTrackGapSize, + colors = colors, + interactionSource = interactionSource, + ) }, ) { var sliderWidthPx by remember { mutableIntStateOf(0) } + val exoPlayer = remember(player) { player as? ExoPlayer } ProgressIndicator(player, totalTickCount = sliderWidthPx, scope) { var isDragging by remember { mutableStateOf(false) } var seekPosition by remember { mutableFloatStateOf(0f) } + // Fallback for navigating away before the trailing invoke in onValueChangeFinished is called + DisposableEffect(exoPlayer) { + onDispose { + exoPlayer?.setScrubbingModeEnabled(false) + isDragging = false + } + } + // Cache the result to avoid repeated scrubbing mode checks + var scrubbingEnabledForThisPlayer by remember(player) { mutableStateOf(false) } + + val sliderValue = if (isDragging) seekPosition else currentPositionProgress + val sliderState = remember { SliderState(value = sliderValue) } + sliderState.value = sliderValue + Slider( - value = if (isDragging) seekPosition else currentPositionProgress, + state = sliderState, onValueChange = { + if (!isDragging) { + exoPlayer?.setScrubbingModeEnabled(true) + scrubbingEnabledForThisPlayer = exoPlayer.isScrubbingModeEnabled() + } isDragging = true seekPosition = it + // Dispatch seeks only if they are less expensive (scrubbing mode enabled) + if (scrubbingEnabledForThisPlayer) updateCurrentPositionProgress(seekPosition) onValueChange?.invoke(it) + sliderState.value = it }, onValueChangeFinished = { updateCurrentPositionProgress(seekPosition) + exoPlayer?.setScrubbingModeEnabled(false) isDragging = false onValueChangeFinished?.invoke() }, @@ -119,8 +165,20 @@ fun LabeledProgressSlider( enabled = changingProgressEnabled, ) } else { - - thumb(this, sliderState) + thumb(this, sliderState, changingProgressEnabled) + } + }, + track = { sliderState -> + if (track == null) { + SliderDefaults.Track( + colors = colors, + enabled = changingProgressEnabled, + sliderState = sliderState, + thumbTrackGapSize = thumbTrackGapSize, + trackInsideCornerSize = if (thumbTrackGapSize == 0.dp) 0.dp else TrackInsideCornerSize, + ) + } else { + track(this, sliderState, changingProgressEnabled) } }, // Beware the order: This measurement will happen first and it is unaware of any final size @@ -137,14 +195,14 @@ fun LabeledProgressSlider( } } -@OptIn(ExperimentalMaterial3Api::class) // For SliderState @Composable private fun SeekTimeThumb( progressState: ProgressStateWithTickCount, sliderState: SliderState, interactionSource: MutableInteractionSource, + enabled: Boolean = true, ) { - LabeledThumb(interactionSource = interactionSource) { + LabeledThumb(interactionSource = interactionSource, enabled = enabled) { Text( text = Util.getStringForTime(progressState.progressToPosition(sliderState.value)), modifier = @@ -163,6 +221,7 @@ private fun SeekTimeThumb( internal fun LabeledThumb( interactionSource: MutableInteractionSource, modifier: Modifier = Modifier, + enabled: Boolean = true, textOffset: Dp = 8.dp, label: @Composable (() -> Unit)? = null, ) { @@ -202,10 +261,124 @@ internal fun LabeledThumb( label() } } - SliderDefaults.Thumb(interactionSource = interactionSource, thumbSize = ThumbSize) + SliderDefaults.Thumb( + interactionSource = interactionSource, + enabled = enabled, + thumbSize = ThumbSize, + ) + } +} + +@Composable +private fun BufferingTrack( + progressState: ProgressStateWithTickCount, + sliderState: SliderState, + enabled: Boolean, + modifier: Modifier = Modifier, + thumbTrackGapSize: Dp = ThumbTrackGapSize, + colors: SliderColors = SliderDefaults.colors(), + interactionSource: MutableInteractionSource? = null, +) { + val interactions = remember { mutableStateListOf() } + if (interactionSource != null) { + LaunchedEffect(interactionSource) { + interactionSource.interactions.collect { interaction -> + when (interaction) { + is PressInteraction.Press -> interactions.add(interaction) + is PressInteraction.Release -> interactions.remove(interaction.press) + is PressInteraction.Cancel -> interactions.remove(interaction.press) + is DragInteraction.Start -> interactions.add(interaction) + is DragInteraction.Stop -> interactions.remove(interaction.start) + is DragInteraction.Cancel -> interactions.remove(interaction.start) + is FocusInteraction.Focus -> interactions.add(interaction) + is FocusInteraction.Unfocus -> interactions.remove(interaction.focus) + } + } + } + } + + Box(modifier = modifier, contentAlignment = Alignment.Center) { + SliderDefaults.Track( + sliderState = sliderState, + enabled = enabled, + colors = colors, + thumbTrackGapSize = thumbTrackGapSize, + trackInsideCornerSize = if (thumbTrackGapSize == 0.dp) 0.dp else TrackInsideCornerSize, + ) + + val isDragging = interactions.isNotEmpty() + val currentBufferedPos = progressState.bufferedPositionProgress + var draggingBufferedPos by remember { mutableFloatStateOf(-1f) } + + if (isDragging && draggingBufferedPos < 0f) { + draggingBufferedPos = currentBufferedPos + } else if (!isDragging) { + draggingBufferedPos = -1f + } + + val bufferedPos = + if (isDragging && draggingBufferedPos >= 0f) draggingBufferedPos else currentBufferedPos + + Canvas(modifier = Modifier.matchParentSize()) { + val thumbPos = sliderState.coercedValueAsFraction + + val startX = size.width * thumbPos + val endX = size.width * bufferedPos + + val trackHeight = 16.dp.toPx() + + val thumbWidth = if (isDragging) PressedThumbWidth else ThumbWidth + // The gap from the center of the thumb to the start of the visible track + val visualGap = + if (thumbTrackGapSize > 0.dp) { + (thumbWidth.toPx() / 2f) + thumbTrackGapSize.toPx() + } else { + 0f + } + val visualStartX = (startX + visualGap).coerceIn(0f, size.width) + val visualEndX = endX.coerceIn(0f, size.width) + + if (visualEndX > visualStartX) { + val bufferingColor = + if (enabled) { + colors.activeTrackColor.copy(alpha = colors.activeTrackColor.alpha * 0.5f) + } else { + colors.disabledActiveTrackColor.copy( + alpha = colors.disabledActiveTrackColor.alpha * 0.5f + ) + } + + val trackInsideCornerRadius = + if (thumbTrackGapSize > 0.dp) TrackInsideCornerSize.toPx() else 0f + val fullCornerRadius = trackHeight / 2f + + val path = + Path().apply { + addRoundRect( + RoundRect( + left = visualStartX, + top = (size.height - trackHeight) / 2f, + right = visualEndX, + bottom = (size.height + trackHeight) / 2f, + topLeftCornerRadius = CornerRadius(trackInsideCornerRadius), + bottomLeftCornerRadius = CornerRadius(trackInsideCornerRadius), + topRightCornerRadius = CornerRadius(fullCornerRadius), + bottomRightCornerRadius = CornerRadius(fullCornerRadius), + ) + ) + } + + drawPath(path = path, color = bufferingColor) + } + } } } +private val ThumbTrackGapSize = + 6.dp // = androidx.compose.material3.tokens.SliderTokens.ActiveHandleLeadingSpace +private val TrackInsideCornerSize = 2.dp private val ThumbWidth = 4.dp // = androidx.compose.material3.tokens.SliderTokens.HandleWidth +private val PressedThumbWidth = + 2.dp // = androidx.compose.material3.tokens.SliderTokens.PressedHandleWidth private val ThumbHeight = 44.0.dp // = androidx.compose.material3.tokens.SliderTokens.HandleHeight private val ThumbSize = DpSize(ThumbWidth, ThumbHeight) diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/buttons/PlaybackSpeedButton.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/buttons/PlaybackSpeedButton.kt index b7a560e407c..8d7eab66f7f 100644 --- a/demos/compose/src/main/java/androidx/media3/demo/compose/buttons/PlaybackSpeedButton.kt +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/buttons/PlaybackSpeedButton.kt @@ -32,12 +32,14 @@ import androidx.compose.material3.ModalBottomSheet import androidx.compose.material3.OutlinedButton import androidx.compose.material3.OutlinedIconButton import androidx.compose.material3.SheetState +import androidx.compose.material3.SheetValue import androidx.compose.material3.Slider import androidx.compose.material3.SliderDefaults +import androidx.compose.material3.SliderState import androidx.compose.material3.Text import androidx.compose.material3.TextButton import androidx.compose.material3.contentColorFor -import androidx.compose.material3.rememberModalBottomSheetState +import androidx.compose.material3.rememberBottomSheetState import androidx.compose.runtime.Composable import androidx.compose.runtime.getValue import androidx.compose.runtime.mutableStateOf @@ -82,7 +84,6 @@ import kotlin.math.round * content is a [PlaybackSpeedBottomSheet]. */ @androidx.annotation.OptIn(UnstableApi::class) -@OptIn(ExperimentalMaterial3Api::class) // for rememberModalBottomSheetState @Composable fun PlaybackSpeedBottomSheetButton( player: Player?, @@ -154,7 +155,7 @@ fun PlaybackSpeedBottomSheet( onDismissRequest: () -> Unit, modifier: Modifier = Modifier, contentModifier: Modifier = Modifier, - sheetState: SheetState = rememberModalBottomSheetState(), + sheetState: SheetState = rememberBottomSheetState(initialValue = SheetValue.Hidden), presetSpeeds: List = listOf(0.25f, 1.0f, 1.25f, 1.5f, 2.0f), speedStep: Float = 0.05f, speedRange: ClosedFloatingPointRange = 0.25f..2.0f, @@ -269,11 +270,18 @@ private fun PlaybackSpeedSlider( speedStep: Float, modifier: Modifier = Modifier, ) { + val steps = ((speedRange.endInclusive - speedRange.start) / speedStep).toInt() - 1 + val sliderState = + remember(speedRange, speedStep) { + SliderState(value = state.playbackSpeed, steps = steps, trackRange = speedRange) + } + sliderState.value = state.playbackSpeed Slider( - value = state.playbackSpeed, - onValueChange = { newSpeed -> state.updatePlaybackSpeed(newSpeed) }, - valueRange = speedRange, - steps = ((speedRange.endInclusive - speedRange.start) / speedStep).toInt() - 1, + state = sliderState, + onValueChange = { newSpeed -> + state.updatePlaybackSpeed(newSpeed) + sliderState.value = newSpeed + }, colors = SliderDefaults.colors( activeTickColor = Color.Transparent, diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/buttons/SettingsButton.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/buttons/SettingsButton.kt index 0781e6963ac..7fd47909b04 100644 --- a/demos/compose/src/main/java/androidx/media3/demo/compose/buttons/SettingsButton.kt +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/buttons/SettingsButton.kt @@ -81,7 +81,6 @@ internal fun SettingsBottomSheet( horizontalAlignment = Alignment.CenterHorizontally, ) { ListItem( - headlineContent = { Text("Playback Speed") }, trailingContent = { // Custom PlaybackSpeed button, brings up a bottom sheet, not default toggle PlaybackSpeedBottomSheetButton( @@ -91,9 +90,10 @@ internal fun SettingsBottomSheet( ) }, colors = ListItemDefaults.colors(containerColor = Color.Transparent), - ) + ) { + Text("Playback Speed") + } ListItem( - headlineContent = { Text("Shuffle Mode") }, trailingContent = { // Custom shuffle button, switch-toggle, not icon button ShuffleButton(player) { @@ -101,9 +101,10 @@ internal fun SettingsBottomSheet( } }, colors = ListItemDefaults.colors(containerColor = Color.Transparent), - ) + ) { + Text("Shuffle Mode") + } ListItem( - headlineContent = { Text("Repeat Mode") }, trailingContent = { // Default repeat button RepeatButton( @@ -113,12 +114,15 @@ internal fun SettingsBottomSheet( ) }, colors = ListItemDefaults.colors(containerColor = Color.Transparent), - ) + ) { + Text("Repeat Mode") + } ListItem( - headlineContent = { Text("Content Scale") }, trailingContent = { TextButton(onClick = onContentScaleChange) { Text(contentScale) } }, colors = ListItemDefaults.colors(containerColor = Color.Transparent), - ) + ) { + Text("Content Scale") + } } } } diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/data/parser.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/data/parser.kt index 62d88297fc6..7b5aee746b3 100644 --- a/demos/compose/src/main/java/androidx/media3/demo/compose/data/parser.kt +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/data/parser.kt @@ -20,7 +20,6 @@ import android.content.Context import android.net.Uri import android.util.JsonReader import androidx.core.net.toUri -import androidx.core.util.Preconditions.checkState import androidx.media3.common.C import androidx.media3.common.MediaItem import androidx.media3.common.MediaMetadata @@ -117,7 +116,7 @@ private fun readEntry(reader: JsonReader, insidePlaylist: Boolean): PlaylistHold "subtitle_mime_type" -> subtitleMimeType = reader.nextString() "subtitle_language" -> subtitleLanguage = reader.nextString() "playlist" -> { - checkState(!insidePlaylist, "Invalid nesting of playlists") + check(!insidePlaylist) { "Invalid nesting of playlists" } children = mutableListOf() reader.beginArray() while (reader.hasNext()) { diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/editing/ClippingSlider.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/editing/ClippingSlider.kt index b50102a7df2..5cb6e3b6863 100644 --- a/demos/compose/src/main/java/androidx/media3/demo/compose/editing/ClippingSlider.kt +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/editing/ClippingSlider.kt @@ -18,8 +18,9 @@ package androidx.media3.demo.compose.editing import android.graphics.Bitmap import androidx.compose.foundation.Canvas import androidx.compose.foundation.Image +import androidx.compose.foundation.interaction.DragInteraction import androidx.compose.foundation.interaction.MutableInteractionSource -import androidx.compose.foundation.interaction.collectIsDraggedAsState +import androidx.compose.foundation.interaction.PressInteraction import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.Spacer @@ -27,18 +28,21 @@ import androidx.compose.foundation.layout.aspectRatio import androidx.compose.foundation.layout.fillMaxHeight import androidx.compose.foundation.layout.fillMaxSize import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.layout.wrapContentHeight import androidx.compose.foundation.shape.RoundedCornerShape -import androidx.compose.material3.ExperimentalMaterial3Api import androidx.compose.material3.RangeSlider +import androidx.compose.material3.RangeSliderState import androidx.compose.material3.Slider import androidx.compose.material3.SliderDefaults +import androidx.compose.material3.SliderState import androidx.compose.runtime.Composable -import androidx.compose.runtime.Immutable import androidx.compose.runtime.LaunchedEffect import androidx.compose.runtime.getValue import androidx.compose.runtime.mutableIntStateOf +import androidx.compose.runtime.mutableLongStateOf import androidx.compose.runtime.mutableStateOf import androidx.compose.runtime.remember +import androidx.compose.runtime.rememberCoroutineScope import androidx.compose.runtime.rememberUpdatedState import androidx.compose.runtime.setValue import androidx.compose.runtime.snapshotFlow @@ -61,7 +65,10 @@ import androidx.compose.ui.graphics.painter.Painter import androidx.compose.ui.layout.ContentScale import androidx.compose.ui.layout.layout import androidx.compose.ui.layout.onSizeChanged +import androidx.compose.ui.platform.LocalDensity import androidx.compose.ui.res.painterResource +import androidx.compose.ui.unit.DpSize +import androidx.compose.ui.unit.dp import androidx.compose.ui.unit.offset import androidx.media3.common.C import androidx.media3.common.Player @@ -72,7 +79,9 @@ import androidx.media3.ui.compose.state.observeState import androidx.media3.ui.compose.state.rememberProgressStateWithTickCount import com.google.common.collect.ImmutableList import kotlin.math.roundToInt +import kotlinx.coroutines.CoroutineScope import kotlinx.coroutines.coroutineScope +import kotlinx.coroutines.flow.first import kotlinx.coroutines.launch /** The ratio of the total width of a clipping thumb to the width of the image row. */ @@ -92,27 +101,44 @@ private const val CLIPPING_THUMB_PLAIN_WIDTH_RATIO = * / 2 before the image row end. */ private const val CLIPPING_TRACK_WIDTH_RATIO = 1f - CLIPPING_THUMB_WIDTH_RATIO -/** The ratio of the maximum position slider length to the width of the image row. */ -private const val POSITION_SLIDER_MAX_LENGTH_RATIO = 1f - 2 * CLIPPING_THUMB_PLAIN_WIDTH_RATIO +/** The ratio of the maximum progress slider length to the width of the image row. */ +private const val PROGRESS_SLIDER_MAX_LENGTH_RATIO = 1f - 2 * CLIPPING_THUMB_PLAIN_WIDTH_RATIO /** The ratio of the clipping frame's horizontal bar thickness to the total height of the slider. */ private const val CLIPPING_FRAME_THICKNESS_RATIO = 0.05f /** The minimum clipping progress delta required to prevent the clipping thumbs from overlapping. */ private const val MIN_CLIPPING_DELTA_FOR_NO_OVERLAP = 2f * (CLIPPING_THUMB_WIDTH_RATIO - CLIPPING_THUMB_PLAIN_WIDTH_RATIO) / - POSITION_SLIDER_MAX_LENGTH_RATIO + PROGRESS_SLIDER_MAX_LENGTH_RATIO /** * A small epsilon value used to check if a progress value is close to the boundaries (0.0 or 1.0). */ private const val BOUNDARY_EPSILON = 1e-3f +/** + * A small positive weight used for layout spacers to avoid zero-weight exceptions in [Row] layout. + */ +private const val MIN_LAYOUT_WEIGHT = 1e-4f + +/** The width of the playback position thumb. */ +private val POSITION_THUMB_WIDTH = 4.dp + +/** The ratio of the playback position thumb height to the slider track height. */ +private const val POSITION_THUMB_HEIGHT_RATIO = 1.1f + +// TODO: b/505719491 +// - Implement accessibility requirements +// - Move to material3 module and mark API unstable +// - Add tests +// - Optimize ProgressSlider recomposition scope (e.g. sliderColors reading playbackProgress on +// every frame) /** * A Material3 clipping slider that allows users to select a clipping range and track playback * position. * * This component displays a row of bitmaps representing the media content, with a range slider - * overlaid to define the start and end clipping points. A secondary position slider allows for + * overlaid to define the start and end clipping points. A secondary progress slider allows for * seeking within the selected range. * * This component does not update the player's clipping configuration. The caller is intended to @@ -120,21 +146,27 @@ private const val BOUNDARY_EPSILON = 1e-3f * experience. * * @param player The [Player] whose content to clip. - * @param clippingRangeMs The selected clipping range in milliseconds. To set the end of the - * clipping range to the full duration of the media, the caller should pass [C.TIME_END_OF_SOURCE] - * as the end time in the [LongRange]. + * @param bitmaps A list of [Bitmap] instances to display as a background preview for the slider. + * All bitmaps in the list should have the same size. Because the slider's aspect ratio depends on + * the number of bitmaps, the list size should remain fixed, as changing it dynamically will cause + * the slider layout to resize each time the list grows or shrinks (for example, when loading + * frames asynchronously, pass a fixed-size list upfront using placeholder bitmaps and update + * elements as frames load). If this list is empty, the component will render an empty [Box] + * instead. + * @param modifier The [Modifier] to be applied to the slider. + * @param initialClippingRangeMs The initial clipping range in milliseconds. * @param onClippingRangeChange A callback that is invoked continuously as one of the clipping * thumbs is being dragged. The [LongRange] represents the clipping start and end positions in * milliseconds (or [C.TIME_END_OF_SOURCE] as the end position when the clip is untrimmed at the - * end of the media) and should be used to update [clippingRangeMs]. - * @param bitmaps A list of [Bitmap] instances to display as a background preview for the slider. - * They should all have the same size. If this list is empty, the component will render an empty - * [Box] instead. - * @param modifier The [Modifier] to be applied to the slider. + * end of the media). * @param onClippingRangeChangeFinished A callback that is invoked when the user finishes dragging a - * clipping thumb. This callback shouldn't be used to update the range slider values (use - * [onClippingRangeChange] for that), but rather to know when the user has completed selecting a - * new value by ending a drag. + * clipping thumb. + * @param onProgressChange A callback that is invoked continuously when the user drags the playback + * position thumb. The [Float] value represents the playback progress as a fraction between 0.0 + * and 1.0. + * @param onProgressChangeFinished A callback that is invoked when the user finishes dragging the + * playback position thumb. + * @param scope The [CoroutineScope] to use for listening to player progress updates. * @param minClippedDurationMs The minimum allowed duration of the clipped range in milliseconds. * The slider will prevent the user from selecting a range shorter than this value. * @param colors The [ClippingSliderColors] used to style the slider. @@ -144,129 +176,154 @@ private const val BOUNDARY_EPSILON = 1e-3f * being painted. The second boolean indicates whether the thumb has reached its absolute boundary * within the media (start of media for the start thumb, end of the media for the end thumb). */ -// TODO: b/505719491 -// - Implement accessibility requirements -// - Implement color defaults -// - Decide and test what the slider should look like for RTL locales -// - Remove @OptIn(ExperimentalMaterial3Api::class) annotations once the RangeSlider is stable -// - Move to material3 module and mark API unstable -// - Add tests -@OptIn(ExperimentalMaterial3Api::class) @Composable fun ClippingSlider( player: Player?, - clippingRangeMs: LongRange, - onClippingRangeChange: (LongRange) -> Unit, bitmaps: ImmutableList, modifier: Modifier = Modifier, + initialClippingRangeMs: LongRange = 0L..C.TIME_END_OF_SOURCE, + onClippingRangeChange: ((LongRange) -> Unit)? = null, onClippingRangeChangeFinished: (() -> Unit)? = null, + onProgressChange: ((Float) -> Unit)? = null, + onProgressChangeFinished: (() -> Unit)? = null, + scope: CoroutineScope = rememberCoroutineScope(), minClippedDurationMs: Long = 1000L, - colors: ClippingSliderColors, + colors: ClippingSliderColors = ClippingSliderDefaults.colors(), shape: RoundedCornerShape = RoundedCornerShape(percent = 30), clippingThumbPainter: @Composable (isStart: Boolean, isAtLimit: Boolean) -> Painter = - defaultClippingThumbPainterIcon, + defaultClippingThumbPainter, ) { - if (bitmaps.isEmpty()) { - Box(modifier) - return - } - val sliderAspectRatio = - remember(bitmaps) { - val firstBitmap = bitmaps[0] - require(firstBitmap.width > 0 && firstBitmap.height > 0) { - "Bitmap should have positive width and height" - } - (bitmaps.size * firstBitmap.width).toFloat() / firstBitmap.height.toFloat() - } var positionTickCount by remember { mutableIntStateOf(0) } val state = - rememberClippingSliderState(player, positionTickCount, clippingRangeMs, minClippedDurationMs) - val currentOnClippingRangeChange by rememberUpdatedState(onClippingRangeChange) + rememberClippingSliderState( + player = player, + positionTickCount = positionTickCount, + initialClippingRangeMs = initialClippingRangeMs, + minClippedDurationMs = minClippedDurationMs, + onClippingRangeChange = onClippingRangeChange, + onClippingRangeChangeFinished = onClippingRangeChangeFinished, + onProgressChange = onProgressChange, + onProgressChangeFinished = onProgressChangeFinished, + scope = scope, + ) - LaunchedEffect(clippingRangeMs, minClippedDurationMs, state, state.durationMs) { + LaunchedEffect(state.isUserInteracting, minClippedDurationMs, state.durationMs) { if (state.isUserInteracting) return@LaunchedEffect - val wasAdjusted = state.syncExternalRange(clippingRangeMs, minClippedDurationMs) + val wasAdjusted = state.enforceConstraints(minClippedDurationMs) if (wasAdjusted) { - currentOnClippingRangeChange(state.clippingRangeMs) + state.onClippingRangeChange?.invoke(state.clippingRangeMs) } } - Box( + ClippingSlider( + state = state, + bitmaps = bitmaps, modifier = - modifier.aspectRatio(sliderAspectRatio).onSizeChanged { size -> - positionTickCount = (POSITION_SLIDER_MAX_LENGTH_RATIO * size.width).roundToInt() - } - ) { - ImageRow(bitmaps, Modifier.fillMaxWidth().clip(shape)) - ClippedImagesFilter( - clippingRangeProvider = { state.clippingRange }, - colors.clippedFilterColor, - Modifier.fillMaxSize().clip(shape), - ) - ClippingRangeSlider( - state = state, - onClippingRangeChange = onClippingRangeChange, - onClippingRangeChangeFinished = onClippingRangeChangeFinished, - minClippedDurationMs = minClippedDurationMs, - colors = colors, - shape = shape, - thumbPainter = clippingThumbPainter, - modifier = Modifier.fillMaxSize(), - ) - PositionSlider(state, colors.positionThumbColor, Modifier.fillMaxSize()) - } + modifier.onSizeChanged { size -> + positionTickCount = (PROGRESS_SLIDER_MAX_LENGTH_RATIO * size.width).roundToInt() + }, + onClippingRangeChange = onClippingRangeChange, + onClippingRangeChangeFinished = onClippingRangeChangeFinished, + onProgressChange = onProgressChange, + onProgressChangeFinished = onProgressChangeFinished, + colors = colors, + shape = shape, + clippingThumbPainter = clippingThumbPainter, + ) } /** - * Represents the colors used in a [ClippingSlider] to customize its appearance. + * A Material3 clipping slider that allows users to select a clipping range and track playback + * position using a [ClippingSliderState]. * - * @property clippingFrameColor The color of the selection frame and of the clipping handles. - * @property clippingIconColor The color used to tint the icons within the clipping thumbs. - * @property positionThumbColor The color of the thumb representing the current playback position. - * @property clippedFilterColor The color of the filter applied to the clipped areas of the slider. - * The image bitmaps underneath remain visible if the alpha is less than 1. + * @param state The [ClippingSliderState] controlling the slider. + * @param bitmaps A list of [Bitmap] instances to display as a background preview for the slider. + * All bitmaps in the list should have the same size. Because the slider's aspect ratio depends on + * the number of bitmaps, the list size should remain fixed, as changing it dynamically will cause + * the slider layout to resize each time the list grows or shrinks (for example, when loading + * frames asynchronously, pass a fixed-size list upfront using placeholder bitmaps and update + * elements as frames load). If this list is empty, the component will render an empty [Box] + * instead. + * @param modifier The [Modifier] to be applied to the slider. + * @param onClippingRangeChange A callback that is invoked continuously when the user drags a + * clipping thumb. + * @param onClippingRangeChangeFinished A callback that is invoked when the user finishes dragging a + * clipping thumb. + * @param onProgressChange A callback that is invoked continuously when the user drags the playback + * position thumb. + * @param onProgressChangeFinished A callback that is invoked when the user finishes dragging the + * playback position thumb. + * @param colors The [ClippingSliderColors] used to style the slider. + * @param shape The [RoundedCornerShape] used to define the slider's shape. + * @param clippingThumbPainter A composable lambda that provides icons for the clipping thumbs. The + * first boolean passed to the lambda indicates which of the two clipping handles is currently + * being painted. The second boolean indicates whether the thumb has reached its absolute boundary + * within the media (start of media for the start thumb, end of the media for the end thumb). */ -@Immutable -class ClippingSliderColors( - val clippingFrameColor: Color, - val clippingIconColor: Color, - val positionThumbColor: Color, - val clippedFilterColor: Color, +@Composable +private fun ClippingSlider( + state: ClippingSliderState, + bitmaps: ImmutableList, + modifier: Modifier = Modifier, + onClippingRangeChange: ((LongRange) -> Unit)? = null, + onClippingRangeChangeFinished: (() -> Unit)? = null, + onProgressChange: ((Float) -> Unit)? = null, + onProgressChangeFinished: (() -> Unit)? = null, + colors: ClippingSliderColors = ClippingSliderDefaults.colors(), + shape: RoundedCornerShape = RoundedCornerShape(percent = 30), + clippingThumbPainter: @Composable (isStart: Boolean, isAtLimit: Boolean) -> Painter = + defaultClippingThumbPainter, ) { - - /** Returns a copy of this ClippingSliderColors, optionally overriding some of the values. */ - fun copy( - clippingFrameColor: Color = this.clippingFrameColor, - clippingIconColor: Color = this.clippingIconColor, - positionThumbColor: Color = this.positionThumbColor, - clippedFilterColor: Color = this.clippedFilterColor, - ): ClippingSliderColors = - ClippingSliderColors( - clippingFrameColor, - clippingIconColor, - positionThumbColor, - clippedFilterColor, - ) - - override fun equals(other: Any?): Boolean { - if (this === other) return true - if (other !is ClippingSliderColors) return false - - if (clippingFrameColor != other.clippingFrameColor) return false - if (clippingIconColor != other.clippingIconColor) return false - if (positionThumbColor != other.positionThumbColor) return false - if (clippedFilterColor != other.clippedFilterColor) return false - - return true + if (onClippingRangeChange != null) { + state.onClippingRangeChange = onClippingRangeChange + } + if (onClippingRangeChangeFinished != null) { + state.onClippingRangeChangeFinished = onClippingRangeChangeFinished + } + if (onProgressChange != null) { + state.onProgressChange = onProgressChange + } + if (onProgressChangeFinished != null) { + state.onProgressChangeFinished = onProgressChangeFinished } + if (bitmaps.isEmpty()) { + Box(modifier) + return + } + val sliderAspectRatio = + remember(bitmaps) { + val firstBitmap = bitmaps[0] + require(firstBitmap.width > 0 && firstBitmap.height > 0) { + "Bitmap should have positive width and height" + } + (bitmaps.size * firstBitmap.width).toFloat() / firstBitmap.height.toFloat() + } - override fun hashCode(): Int { - var result = clippingFrameColor.hashCode() - result = 31 * result + clippingIconColor.hashCode() - result = 31 * result + positionThumbColor.hashCode() - result = 31 * result + clippedFilterColor.hashCode() - return result + Box(modifier = modifier.aspectRatio(sliderAspectRatio)) { + ClippingRangeSlider( + rangeSliderState = state.rangeSliderState, + bitmaps = bitmaps, + modifier = Modifier.fillMaxSize(), + enabled = state.changingProgressEnabled && state.durationMs > 0, + startThumbInteractionSource = state.startThumbInteractionSource, + endThumbInteractionSource = state.endThumbInteractionSource, + clippingRangeProvider = { state.clippingRange }, + onValueChangeFinished = { state.onRangeSliderValueChangeFinished() }, + colors = colors, + shape = shape, + clippingThumbPainter = clippingThumbPainter, + ) + ProgressSlider( + state = state.progressSliderState, + modifier = Modifier.fillMaxSize(), + enabled = state.changingProgressEnabled && state.durationMs > 0, + trackRange = state.activeValueRange, + onValueChange = { state.onProgressSliderValueChange(it) }, + onValueChangeFinished = { state.onProgressSliderValueChangeFinished() }, + interactionSource = state.progressThumbInteractionSource, + positionThumbColor = colors.positionThumbColor, + ) } } @@ -286,132 +343,135 @@ private fun ImageRow(bitmaps: ImmutableList, modifier: Modifier = Modifi } /** - * A composable that draws a visual filter (overlay) over the clipped (inactive) areas of the - * slider. + * A composable that draws a visual filter (overlay) over the inactive track areas of the slider. * * This component dims the regions of the image row that fall outside the active clipping range (to * the left of the start thumb and to the right of the end thumb). * * @param clippingRangeProvider A provider that returns the current active clipping range as a * fraction (0.0 to 1.0) of the total duration. - * @param clippedFilterColor The color (usually translucent) used to overlay the clipped areas. + * @param inactiveTrackColor The color (usually translucent) used to overlay the inactive track + * areas. * @param modifier The [Modifier] to be applied to this composable. */ @Composable -private fun ClippedImagesFilter( +private fun InactiveTrackFilter( clippingRangeProvider: () -> ClosedFloatingPointRange, - clippedFilterColor: Color, + inactiveTrackColor: Color, modifier: Modifier = Modifier, ) { Canvas(modifier) { val clippingRange = clippingRangeProvider() val width = size.width val height = size.height - val positionSliderStart = logicalToVisualPositionSliderStart(clippingRange.start) * width - if (positionSliderStart > 0f) { - drawRect(clippedFilterColor, size = Size(positionSliderStart, height)) + val progressSliderStart = logicalToVisualProgressSliderStart(clippingRange.start) * width + if (progressSliderStart > 0f) { + drawRect(inactiveTrackColor, size = Size(progressSliderStart, height)) } - val positionSliderEnd = logicalToVisualPositionSliderEnd(clippingRange.endInclusive) * width - if (positionSliderEnd < width) { + val progressSliderEnd = logicalToVisualProgressSliderEnd(clippingRange.endInclusive) * width + if (progressSliderEnd < width) { drawRect( - clippedFilterColor, - topLeft = Offset(x = positionSliderEnd, y = 0f), - size = Size(width - positionSliderEnd, height), + inactiveTrackColor, + topLeft = Offset(x = progressSliderEnd, y = 0f), + size = Size(width - progressSliderEnd, height), ) } } } -@OptIn(ExperimentalMaterial3Api::class) +/** + * A composable that renders the range slider used to select the clipping range over preview + * bitmaps. + * + * This component displays the background [ImageRow], the [InactiveTrackFilter] overlay, and a + * [RangeSlider] configured with custom thumbs ([ClippingThumb]) and track ([ClippingTrack]). + * + * @param rangeSliderState The [RangeSliderState] controlling the slider thumbs. + * @param bitmaps A list of [Bitmap] instances to display as a background preview for the slider. + * All bitmaps in the list should have the same size. Because the slider's aspect ratio depends on + * the number of bitmaps, the list size should remain fixed, as changing it dynamically will cause + * the slider layout to resize each time the list grows or shrinks (for example, when loading + * frames asynchronously, pass a fixed-size list upfront using placeholder bitmaps and update + * elements as frames load). If this list is empty, the [ClippingSlider] component will render an + * empty [Box] instead. + * @param modifier The [Modifier] to be applied to this composable. + * @param enabled Whether interaction with the range slider is enabled. + * @param startThumbInteractionSource The [MutableInteractionSource] for the start clipping thumb. + * @param endThumbInteractionSource The [MutableInteractionSource] for the end clipping thumb. + * @param clippingRangeProvider A provider returning the current clipping range as fractions (0.0 to + * 1.0) of the total duration. + * @param colors The [ClippingSliderColors] used to style the slider. + * @param shape The [RoundedCornerShape] used to define the slider's shape. + * @param clippingThumbPainter A composable lambda that provides icons for the clipping thumbs. The + * first boolean passed to the lambda indicates which of the two clipping handles is currently + * being painted. The second boolean indicates whether the thumb has reached its absolute boundary + * within the media (start of media for the start thumb, end of the media for the end thumb). + */ @Composable private fun ClippingRangeSlider( - state: ClippingSliderState, - onClippingRangeChange: (LongRange) -> Unit, - onClippingRangeChangeFinished: (() -> Unit)?, - minClippedDurationMs: Long, - colors: ClippingSliderColors, - shape: RoundedCornerShape, - thumbPainter: @Composable (isStart: Boolean, isAtLimit: Boolean) -> Painter, + rangeSliderState: RangeSliderState, + bitmaps: ImmutableList, modifier: Modifier = Modifier, + enabled: Boolean = true, + startThumbInteractionSource: MutableInteractionSource = remember { MutableInteractionSource() }, + endThumbInteractionSource: MutableInteractionSource = remember { MutableInteractionSource() }, + clippingRangeProvider: () -> ClosedFloatingPointRange = { + clippingRangeFromSliderRange(rangeSliderState.startValue..rangeSliderState.endValue) + }, + onValueChangeFinished: (() -> Unit)? = null, + colors: ClippingSliderColors = ClippingSliderDefaults.colors(), + shape: RoundedCornerShape = RoundedCornerShape(percent = 30), + clippingThumbPainter: @Composable (isStart: Boolean, isAtLimit: Boolean) -> Painter = + defaultClippingThumbPainter, ) { - val startThumbInteractionSource = remember { MutableInteractionSource() } - val endThumbInteractionSource = remember { MutableInteractionSource() } - val isDraggingStartThumb by startThumbInteractionSource.collectIsDraggedAsState() - val isDraggingEndThumb by endThumbInteractionSource.collectIsDraggedAsState() - val isDragging = isDraggingStartThumb || isDraggingEndThumb - val clippingSliderRange = sliderRangeFromClippingRange(state.clippingRange) - val minProgressDelta = state.calculateMinProgressDelta(minClippedDurationMs) - // While onValueChange and onValueChangeFinished handle ordinary range updates faster upfront, - // this observer acts as a safety net for gesture cancellations (e.g. system interrupts like phone - // calls). - LaunchedEffect(isDragging) { - if (isDragging) { - state.pause() - } - state.isUserInteracting = isDragging + Box(modifier = modifier) { + ImageRow(bitmaps, Modifier.fillMaxWidth().clip(shape)) + InactiveTrackFilter( + clippingRangeProvider = clippingRangeProvider, + colors.inactiveTrackColor, + Modifier.fillMaxSize().clip(shape), + ) + RangeSlider( + state = rangeSliderState, + // TODO: b/505719491 - Once onValueChange and onValueChangeFinished callbacks are added to + // this stateful overload, move them here from RangeSliderState, and implement minRangeDelta + // clamping logic + modifier = Modifier.fillMaxSize(), + enabled = enabled, + onValueChangeFinished = onValueChangeFinished, + startThumbInteractionSource = startThumbInteractionSource, + endThumbInteractionSource = endThumbInteractionSource, + startThumb = { + val isStart = true + val isAtLimit = rangeSliderState.startValue <= BOUNDARY_EPSILON + ClippingThumb( + isStart, + colors, + shape, + clippingThumbPainter(isStart, isAtLimit), + Modifier.fillMaxWidth(CLIPPING_THUMB_WIDTH_RATIO).fillMaxHeight(), + ) + }, + endThumb = { + val isStart = false + val isAtLimit = rangeSliderState.endValue >= 1f - BOUNDARY_EPSILON + ClippingThumb( + isStart, + colors, + shape, + clippingThumbPainter(isStart, isAtLimit), + Modifier.fillMaxWidth(CLIPPING_THUMB_WIDTH_RATIO).fillMaxHeight(), + ) + }, + track = { + ClippingTrack( + clippingSliderRangeProvider = { rangeSliderState.startValue..rangeSliderState.endValue }, + colors.clippingFrameColor, + Modifier.fillMaxSize(), + ) + }, + ) } - RangeSlider( - value = clippingSliderRange, - onValueChange = { newClippingSliderRange -> - if (!isDragging) return@RangeSlider // Filter out tapping events - state.isUserInteracting = true - state.pause() - val proposedRange = clippingRangeFromSliderRange(newClippingSliderRange) - var constrainedStart = proposedRange.start - var constrainedEnd = proposedRange.endInclusive - if (constrainedEnd - constrainedStart < minProgressDelta) { - if (isDraggingStartThumb) { - constrainedStart = (constrainedEnd - minProgressDelta).coerceAtLeast(0f) - } else { - constrainedEnd = (constrainedStart + minProgressDelta).coerceAtMost(1f) - } - } - state.clippingRange = constrainedStart..constrainedEnd - onClippingRangeChange(state.clippingRangeMs) - }, - modifier = modifier, - enabled = state.durationMs > 0, - onValueChangeFinished = { - if (!isDragging) return@RangeSlider - state.committedClippingRange = state.clippingRange - val snapPosition = - if (isDraggingStartThumb) state.clippingRange.start else state.clippingRange.endInclusive - state.seekTo(snapPosition) - state.isUserInteracting = false - onClippingRangeChangeFinished?.invoke() - }, - startInteractionSource = startThumbInteractionSource, - endInteractionSource = endThumbInteractionSource, - startThumb = { - val isStart = true - val isAtLimit = clippingSliderRange.start <= BOUNDARY_EPSILON - ClippingThumb( - isStart, - colors, - shape, - thumbPainter(isStart, isAtLimit), - Modifier.fillMaxWidth(CLIPPING_THUMB_WIDTH_RATIO).fillMaxHeight(), - ) - }, - endThumb = { - val isStart = false - val isAtLimit = clippingSliderRange.endInclusive >= 1f - BOUNDARY_EPSILON - ClippingThumb( - isStart, - colors, - shape, - thumbPainter(isStart, isAtLimit), - Modifier.fillMaxWidth(CLIPPING_THUMB_WIDTH_RATIO).fillMaxHeight(), - ) - }, - track = { - ClippingTrack( - clippingSliderRangeProvider = { clippingSliderRange }, - colors.clippingFrameColor, - Modifier.fillMaxSize(), - ) - }, - ) } /** @@ -484,7 +544,7 @@ private fun ClippingThumb( contentDescription = null, modifier = Modifier.fillMaxHeight().fillMaxWidth(CLIPPING_THUMB_PLAIN_RATIO), contentScale = ContentScale.FillBounds, - colorFilter = ColorFilter.tint(colors.clippingIconColor), + colorFilter = ColorFilter.tint(colors.clippingThumbIconColor), ) } } @@ -533,83 +593,93 @@ private fun ClippingTrack( ) } +/** + * A composable that renders the playback position scrubber slider within the clipping bounds. + * + * @param state The [SliderState] controlling the playback position thumb. + * @param modifier The [Modifier] to be applied to this composable. + * @param enabled Whether interaction with the progress slider is enabled. + * @param trackRange The allowed range of values for the progress slider, corresponding to the + * committed clipping range. + * @param interactionSource The [MutableInteractionSource] for the progress slider. + * @param positionThumbColor The color used to render the playback position thumb. + */ @Composable -@OptIn(ExperimentalMaterial3Api::class) -private fun PositionSlider( - state: ClippingSliderState, - positionThumbColor: Color, +private fun ProgressSlider( + state: SliderState, modifier: Modifier = Modifier, + enabled: Boolean = true, + trackRange: ClosedFloatingPointRange = state.trackRange, + onValueChange: ((Float) -> Unit)? = null, + onValueChangeFinished: (() -> Unit)? = null, + interactionSource: MutableInteractionSource = remember { MutableInteractionSource() }, + positionThumbColor: Color = ClippingSliderDefaults.colors().positionThumbColor, ) { - var dragPosition by remember { mutableStateOf(null) } - val interactionSource = remember { MutableInteractionSource() } - val isDragging by interactionSource.collectIsDraggedAsState() - // While onValueChange and onValueChangeFinished handle ordinary scrubber updates faster upfront, - // this observer acts as a safety net for gesture cancellations (e.g. system interrupts like phone - // calls). - LaunchedEffect(isDragging) { - if (isDragging) { - state.pause() - } - state.isUserInteracting = isDragging - } - // Use state.committedClippingRange to compute the position slider position so it only updates - // when dragging the clipping thumbs finishes. - val visualPositionSliderStart = - logicalToVisualPositionSliderStart(state.committedClippingRange.start) - val visualPositionSliderEnd = - logicalToVisualPositionSliderEnd(state.committedClippingRange.endInclusive) + // Use trackRange (which corresponds to the pre-drag clipping bounds) to compute the progress + // slider layout so it remains visually stable during drag gestures. + val visualProgressSliderStart = logicalToVisualProgressSliderStart(trackRange.start) + val visualProgressSliderEnd = logicalToVisualProgressSliderEnd(trackRange.endInclusive) + val density = LocalDensity.current + var sliderHeight by remember { mutableStateOf(0.dp) } Row(modifier) { - Spacer(modifier = Modifier.weight(visualPositionSliderStart)) - // TODO: b/505719491 - Make the thumb height match the ClippingSlider height - Slider( - value = - dragPosition - // Coerce within the active clipping range so the position thumb sticks to the clipping - // thumb when the clipping thumb crosses the position thumb. Then coerce within - // committedClippingRange to ensure the value stays within Slider.valueRange while player - // position updates lag behind a seek. - ?: state.playbackProgress - ?.coerceIn(state.clippingRange) - ?.coerceIn(state.committedClippingRange) - ?: state.committedClippingRange.start, - onValueChange = { - state.isUserInteracting = true - state.pause() - dragPosition = it - }, - enabled = state.changingProgressEnabled, + Spacer(modifier = Modifier.weight(visualProgressSliderStart.coerceAtLeast(MIN_LAYOUT_WEIGHT))) + Box( modifier = - Modifier.weight(visualPositionSliderEnd - visualPositionSliderStart) + Modifier.weight( + (visualProgressSliderEnd - visualProgressSliderStart).coerceAtLeast(MIN_LAYOUT_WEIGHT) + ) .fillMaxHeight() - // Expand the slider by 1px on each side to avoid rounding gaps. Row measurement rounds - // weighted dimensions to the nearest integer pixel, which can leave a sub-pixel gap - // between the clipping thumbs and the slider where the background image is visible. This - // 1px overlap ensures the seam is always covered. - .layout { measurable, constraints -> - val expandedConstraints = constraints.offset(horizontal = 2) - val placeable = measurable.measure(expandedConstraints) - layout(placeable.width - 2, placeable.height) { placeable.place(x = -1, y = 0) } - }, - valueRange = state.committedClippingRange, - onValueChangeFinished = { - dragPosition?.let { state.seekTo(it) } - dragPosition = null - state.isUserInteracting = false - }, - interactionSource = interactionSource, - colors = + .onSizeChanged { size -> sliderHeight = with(density) { size.height.toDp() } } + ) { + val sliderColors = SliderDefaults.colors( - thumbColor = - if (state.playbackProgress != null) positionThumbColor else Color.Transparent, + thumbColor = positionThumbColor, activeTrackColor = Color.Transparent, inactiveTrackColor = Color.Transparent, - disabledThumbColor = - if (state.playbackProgress != null) positionThumbColor else Color.Transparent, + disabledThumbColor = Color.Transparent, disabledActiveTrackColor = Color.Transparent, disabledInactiveTrackColor = Color.Transparent, - ), + ) + + Slider( + state = state, + enabled = enabled, + onValueChange = onValueChange ?: { state.value = it }, + onValueChangeFinished = onValueChangeFinished, + modifier = + Modifier.fillMaxWidth() + // Use unbounded height so the slider and its position thumb can exceed the track height + // (sliderHeight) without being clamped by the parent Box's maxHeight constraint. + .wrapContentHeight(unbounded = true) + // Expand the slider by 1px on each side to avoid rounding gaps. Row measurement rounds + // weighted dimensions to the nearest integer pixel, which can leave a sub-pixel gap + // between the clipping thumbs and the slider where the background image is visible. + // This 1px overlap ensures the seam is always covered. + .layout { measurable, constraints -> + val expandedConstraints = constraints.offset(horizontal = 2) + val placeable = measurable.measure(expandedConstraints) + layout(placeable.width - 2, placeable.height) { placeable.place(x = -1, y = 0) } + }, + interactionSource = interactionSource, + colors = sliderColors, + thumb = { + if (enabled) { + SliderDefaults.Thumb( + interactionSource = interactionSource, + colors = sliderColors, + thumbSize = + DpSize( + width = POSITION_THUMB_WIDTH, + height = sliderHeight * POSITION_THUMB_HEIGHT_RATIO, + ), + ) + } + }, + ) + } + Spacer( + modifier = Modifier.weight((1f - visualProgressSliderEnd).coerceAtLeast(MIN_LAYOUT_WEIGHT)) ) - Spacer(modifier = Modifier.weight(1f - visualPositionSliderEnd)) } } @@ -617,11 +687,24 @@ private fun PositionSlider( private fun rememberClippingSliderState( player: Player?, positionTickCount: Int, - initialClippingRangeMs: LongRange, + initialClippingRangeMs: LongRange = 0L..C.TIME_END_OF_SOURCE, minClippedDurationMs: Long, + onClippingRangeChange: ((LongRange) -> Unit)? = null, + onClippingRangeChangeFinished: (() -> Unit)? = null, + onProgressChange: ((Float) -> Unit)? = null, + onProgressChangeFinished: (() -> Unit)? = null, + scope: CoroutineScope = rememberCoroutineScope(), ): ClippingSliderState { val positionProgressState = - rememberProgressStateWithTickCount(player, totalTickCount = positionTickCount) + rememberProgressStateWithTickCount( + player = player, + totalTickCount = positionTickCount, + scope = scope, + ) + val currentOnClippingRangeChange by rememberUpdatedState(onClippingRangeChange) + val currentOnClippingRangeChangeFinished by rememberUpdatedState(onClippingRangeChangeFinished) + val currentOnProgressChange by rememberUpdatedState(onProgressChange) + val currentOnProgressChangeFinished by rememberUpdatedState(onProgressChangeFinished) val clippingSliderState = remember(player, positionProgressState) { ClippingSliderState( @@ -631,31 +714,36 @@ private fun rememberClippingSliderState( minClippedDurationMs, ) } + clippingSliderState.onClippingRangeChange = currentOnClippingRangeChange + clippingSliderState.onClippingRangeChangeFinished = currentOnClippingRangeChangeFinished + clippingSliderState.onProgressChange = currentOnProgressChange + clippingSliderState.onProgressChangeFinished = currentOnProgressChangeFinished + clippingSliderState.minClippedDurationMs = minClippedDurationMs LaunchedEffect(clippingSliderState) { clippingSliderState.observe() } return clippingSliderState } /** - * Returns the visual ratio for the position slider start compared to the width of the image row. + * Returns the visual ratio for the progress slider start compared to the width of the image row. * - * The position slider lies between the plain portions of the clipping handles. + * The progress slider lies between the plain portions of the clipping handles. * * @param clippingStart The clipping start position of the media, expressed as a ratio of the * duration (value between 0 and 1). */ -private fun logicalToVisualPositionSliderStart(clippingStart: Float): Float = - (clippingStart * POSITION_SLIDER_MAX_LENGTH_RATIO) + CLIPPING_THUMB_PLAIN_WIDTH_RATIO +private fun logicalToVisualProgressSliderStart(clippingStart: Float): Float = + (clippingStart * PROGRESS_SLIDER_MAX_LENGTH_RATIO) + CLIPPING_THUMB_PLAIN_WIDTH_RATIO /** - * Returns the visual ratio for the position slider end compared to the width of the image row. + * Returns the visual ratio for the progress slider end compared to the width of the image row. * - * The position slider lies between the plain portions of the clipping handles. + * The progress slider lies between the plain portions of the clipping handles. * * @param clippingEnd The clipping end position of the media, expressed as a ratio of the duration * (value between 0 and 1). */ -private fun logicalToVisualPositionSliderEnd(clippingEnd: Float): Float = - (clippingEnd * POSITION_SLIDER_MAX_LENGTH_RATIO) + CLIPPING_THUMB_PLAIN_WIDTH_RATIO +private fun logicalToVisualProgressSliderEnd(clippingEnd: Float): Float = + (clippingEnd * PROGRESS_SLIDER_MAX_LENGTH_RATIO) + CLIPPING_THUMB_PLAIN_WIDTH_RATIO /** * Converts the clipping slider start and end positions to the actual clipping end points. @@ -676,13 +764,13 @@ private fun clippingRangeFromSliderRange( // at the start of the slider) and the actual clipping start (corresponding to the inside of the // left thumb), divided by the total width. val clippingStartWidthRatio = clippingSliderRange.start * CLIPPING_TRACK_WIDTH_RATIO - val clippingStart = clippingStartWidthRatio / POSITION_SLIDER_MAX_LENGTH_RATIO + val clippingStart = clippingStartWidthRatio / PROGRESS_SLIDER_MAX_LENGTH_RATIO // The distance between the min clipping start and the actual clipping end (corresponding to the // inside of the right thumb), divided by the total width. val clippingEndWidthRatio = clippingSliderRange.endInclusive * CLIPPING_TRACK_WIDTH_RATIO - 2 * CLIPPING_THUMB_PLAIN_WIDTH_RATIO + CLIPPING_THUMB_WIDTH_RATIO - val clippingEnd = clippingEndWidthRatio / POSITION_SLIDER_MAX_LENGTH_RATIO + val clippingEnd = clippingEndWidthRatio / PROGRESS_SLIDER_MAX_LENGTH_RATIO return clippingStart..clippingEnd } @@ -695,7 +783,7 @@ private fun clippingRangeFromSliderRange( * This is the inverse of [clippingRangeFromSliderRange]. * * @param clippingRange The clipping range that should be applied to the media, expressed as a - * fraction of the total duration. Both values should be between 0 and 1. + * fraction of the total duration. Both values are between 0 and 1. * @return The clipping slider range. Both values are between 0 and 1. */ private fun sliderRangeFromClippingRange( @@ -704,19 +792,19 @@ private fun sliderRangeFromClippingRange( // The distance between the min slider start (corresponding to the center of the // left thumb when at the start of the slider) and the actual slider start // (corresponding to the center of the left thumb), divided by the total width. - val sliderStartWidthRatio = clippingRange.start * POSITION_SLIDER_MAX_LENGTH_RATIO + val sliderStartWidthRatio = clippingRange.start * PROGRESS_SLIDER_MAX_LENGTH_RATIO val sliderStart = sliderStartWidthRatio / CLIPPING_TRACK_WIDTH_RATIO // The distance between the min slider start (corresponding to the center of the // left thumb when at the start of the slider) and the actual slider end // (corresponding to the center of the right thumb), divided by the total width. val sliderEndWidthRatio = - clippingRange.endInclusive * POSITION_SLIDER_MAX_LENGTH_RATIO + + clippingRange.endInclusive * PROGRESS_SLIDER_MAX_LENGTH_RATIO + 2 * CLIPPING_THUMB_PLAIN_WIDTH_RATIO - CLIPPING_THUMB_WIDTH_RATIO val sliderEnd = sliderEndWidthRatio / CLIPPING_TRACK_WIDTH_RATIO return sliderStart..sliderEnd } -private val defaultClippingThumbPainterIcon: +private val defaultClippingThumbPainter: @Composable (isStart: Boolean, isAtLimit: Boolean) -> Painter = @Composable { isStart, isAtLimit -> @@ -725,19 +813,95 @@ private val defaultClippingThumbPainterIcon: else painterResource(R.drawable.media3_icon_clip_thumb_right_arrow) } +/** + * Calculates the minimum progress delta required to prevent the clipping thumbs from overlapping. + */ +private fun calculateMinRangeDelta(minClippedDurationMs: Long, durationMs: Long): Float = + if (durationMs <= 0) { + MIN_CLIPPING_DELTA_FOR_NO_OVERLAP + } else { + maxOf(minClippedDurationMs.toFloat() / durationMs, MIN_CLIPPING_DELTA_FOR_NO_OVERLAP) + .coerceAtMost(1f) + } + +/** + * State object that manages a [ClippingSlider] connected to a [Player]. + * + * @param player The [Player] to observe and control, or `null`. + * @param positionProgressState The [ProgressStateWithTickCount] tracking media progress. + * @param initialClippingRangeMs The initial clipping range in milliseconds. + * @param initialMinClippedDurationMs The initial minimum allowed duration between clipping start + * and end in milliseconds. + */ private class ClippingSliderState( private val player: Player?, private val positionProgressState: ProgressStateWithTickCount, - initialClippingRangeMs: LongRange, - minClippedDurationMs: Long, + private val initialClippingRangeMs: LongRange, + initialMinClippedDurationMs: Long, ) { + var minClippedDurationMs by mutableLongStateOf(initialMinClippedDurationMs) + var onClippingRangeChange: ((LongRange) -> Unit)? = null + var onClippingRangeChangeFinished: (() -> Unit)? = null + var onProgressChange: ((Float) -> Unit)? = null + var onProgressChangeFinished: (() -> Unit)? = null + val startThumbInteractionSource = MutableInteractionSource() + val endThumbInteractionSource = MutableInteractionSource() + val progressThumbInteractionSource = MutableInteractionSource() + private var lastChangedBoundaryIsStart = true + val rangeSliderState = RangeSliderState(startValue = 0f, endValue = 1f) + + fun onRangeSliderValueChangeFinished() { + val snapPosition = + if (lastChangedBoundaryIsStart) clippingRange.start else clippingRange.endInclusive + seekTo(snapPosition) + isClipping = false + isUserInteracting = false + preDragClippingRange = clippingRange + updateProgressSliderRange(clippingRange) + onClippingRangeChangeFinished?.invoke() + } + + var progressSliderState: SliderState by mutableStateOf(createProgressSliderState(0f..1f)) + private set + + private fun createProgressSliderState( + range: ClosedFloatingPointRange, + initialValue: Float = range.start, + ): SliderState = + SliderState( + // Coerce within the active clipping range so the position thumb sticks to the clipping + // thumb when the clipping thumb crosses the position thumb. + value = initialValue.coerceIn(range), + trackRange = range, + ) + + fun onProgressSliderValueChange(newValue: Float) { + val coerced = + if (clippingRange.start < clippingRange.endInclusive) { + newValue.coerceIn(clippingRange) + } else { + newValue + } + progressSliderState.value = coerced + onProgressChange?.invoke(coerced) + } + + fun onProgressSliderValueChangeFinished() { + seekTo(progressSliderState.value) + isUserInteracting = false + onProgressChangeFinished?.invoke() + } + + private fun updateProgressSliderRange(range: ClosedFloatingPointRange) { + if (range.start >= range.endInclusive) return + if (progressSliderState.trackRange == range) return + val currentValue = progressSliderState.value.coerceIn(range) + progressSliderState = createProgressSliderState(range, currentValue) + } + /** The current clipping range expressed as a fraction of the total duration (0 to 1). */ - var clippingRange by mutableStateOf(0f..1f) - /** - * The clipping range that has been confirmed by the user. This is typically updated when a drag - * operation on the clipping thumbs finishes. - */ - var committedClippingRange by mutableStateOf(0f..1f) + val clippingRange: ClosedFloatingPointRange + get() = clippingRangeFromSliderRange(rangeSliderState.startValue..rangeSliderState.endValue) /** * The current clipping range in milliseconds, preserving [C.TIME_END_OF_SOURCE] when the end @@ -761,9 +925,30 @@ private class ClippingSliderState( val playbackProgress: Float? get() = if (durationMs > 0) positionProgressState.currentPositionProgress else null - /** Whether the user is actively interacting with the clipping slider. */ + /** + * Whether the user is actively interacting with the slider (either adjusting clipping or + * scrubbing progress). + */ var isUserInteracting by mutableStateOf(false) + /** Whether the user is actively dragging one of the clipping thumbs. */ + private var isClipping by mutableStateOf(false) + + /** + * The clipping range as a fraction of the total duration (0 to 1) captured before the current + * interaction started. + */ + var preDragClippingRange by mutableStateOf(0f..1f) + private set + + /** + * The value range used by the progress slider to layout its track. Anchors to + * [preDragClippingRange] while [isClipping] is true so that the progress scrubber layout remains + * visually stable while clipping thumbs are being dragged. + */ + val activeValueRange: ClosedFloatingPointRange + get() = if (isClipping) preDragClippingRange else clippingRange + /** Whether changing the playback progress is enabled. */ val changingProgressEnabled: Boolean get() = positionProgressState.changingProgressEnabled @@ -788,7 +973,7 @@ private class ClippingSliderState( } init { - val unused = syncExternalRange(initialClippingRangeMs, minClippedDurationMs) + setClippingRange(initialClippingRangeMs) } fun pause() { @@ -804,42 +989,65 @@ private class ClippingSliderState( /** Converts a fraction (0 to 1) of the total duration into a position in milliseconds. */ fun progressToPosition(progress: Float) = positionProgressState.progressToPosition(progress) + fun onDragStarted(isStart: Boolean) { + lastChangedBoundaryIsStart = isStart + preDragClippingRange = clippingRange + isClipping = true + isUserInteracting = true + pause() + } + + /** + * Sets the clipping range, enforcing constraints against media duration and + * [minClippedDurationMs]. + * + * @param clippingRangeMs The desired clipping range in milliseconds. + */ + fun setClippingRange(clippingRangeMs: LongRange) { + val (newRange, _) = calculateClippingRangeProgress(clippingRangeMs, minClippedDurationMs) + applyClippingRange(newRange, updateSlider = true) + } + /** - * Syncs the state with an externally provided clipping range, enforcing constraints and seeking - * the player if necessary. + * Validates the internal clipping range against current constraints and media duration. If the + * bounds violate constraints (e.g., they are too close together or exceed media bounds), they are + * clamped to legal values. * - * @param clippingRangeMs The new desired clipping range in milliseconds. - * @param minClippedDurationMs The minimum permitted duration between clipping start and end in - * milliseconds. - * @return Whether the requested range was adjusted to enforce constraints or match source bounds. + * @param minClippedDurationMs The minimum permitted duration between clipping start and end. + * @return Whether the internal range was adjusted to enforce constraints. */ - fun syncExternalRange(clippingRangeMs: LongRange, minClippedDurationMs: Long): Boolean { + fun enforceConstraints(minClippedDurationMs: Long): Boolean { + this.minClippedDurationMs = minClippedDurationMs val (newRange, wasAdjusted) = - calculateClippingRangeProgress(clippingRangeMs, minClippedDurationMs) - clippingRange = newRange - committedClippingRange = clippingRange + calculateClippingRangeProgress(this.clippingRangeMs, minClippedDurationMs) + applyClippingRange(newRange, updateSlider = wasAdjusted) + return wasAdjusted + } + + private fun applyClippingRange(newRange: ClosedFloatingPointRange, updateSlider: Boolean) { + if (updateSlider) { + val sliderRange = sliderRangeFromClippingRange(newRange) + if (rangeSliderState.startValue != sliderRange.start) { + rangeSliderState.startValue = sliderRange.start + } + if (rangeSliderState.endValue != sliderRange.endInclusive) { + rangeSliderState.endValue = sliderRange.endInclusive + } + preDragClippingRange = newRange + updateProgressSliderRange(newRange) + } + + // Ensure the player's current position hasn't been left outside the new bounds val currentProgress = playbackProgress if (currentProgress != null) { - if (currentProgress < clippingRange.start) { - seekTo(clippingRange.start) - } else if (currentProgress > clippingRange.endInclusive) { - seekTo(clippingRange.endInclusive) + if (currentProgress < newRange.start) { + seekTo(newRange.start) + } else if (currentProgress > newRange.endInclusive) { + seekTo(newRange.endInclusive) } } - return wasAdjusted } - /** - * Calculates the minimum progress delta required to prevent the clipping thumbs from overlapping. - */ - fun calculateMinProgressDelta(minClippedDurationMs: Long): Float = - if (durationMs <= 0) { - MIN_CLIPPING_DELTA_FOR_NO_OVERLAP - } else { - maxOf(minClippedDurationMs.toFloat() / durationMs, MIN_CLIPPING_DELTA_FOR_NO_OVERLAP) - .coerceAtMost(1f) - } - private fun play() { if (playWhenReady) return playWhenReady = true @@ -866,10 +1074,10 @@ private class ClippingSliderState( if (clippingRangeMs.last == C.TIME_END_OF_SOURCE) durationMs else clippingRangeMs.last val rawEnd = originalEndMs.toFloat() / durationMs var end = rawEnd.coerceIn(0f, 1f) - val minProgressDelta = calculateMinProgressDelta(minClippedDurationMs) - if (end - start < minProgressDelta) { - end = (start + minProgressDelta).coerceAtMost(1f) - start = (end - minProgressDelta).coerceAtLeast(0f) + val minRangeDelta = calculateMinRangeDelta(minClippedDurationMs, durationMs) + if (end - start < minRangeDelta) { + end = (start + minRangeDelta).coerceAtMost(1f) + start = (end - minRangeDelta).coerceAtLeast(0f) } val wasAdjusted = rawStart != start || rawEnd != end return (start..end) to wasAdjusted @@ -877,7 +1085,105 @@ private class ClippingSliderState( suspend fun observe() { coroutineScope { + // Observe Player events (isPlaying, playWhenReady) to keep playback states synchronized. launch { playerStateObserver?.observe() } + + // Apply initial clipping range once media duration is known. + launch { + snapshotFlow { durationMs }.first { it > 0 } + setClippingRange(initialClippingRangeMs) + } + + // Update progress slider state range when activeValueRange changes. + launch { + snapshotFlow { activeValueRange }.collect { range -> updateProgressSliderRange(range) } + } + + // Update progress slider state as playback progresses. + launch { + snapshotFlow { playbackProgress } + .collect { progress -> + if ( + !isUserInteracting && + progress != null && + clippingRange.start < clippingRange.endInclusive + ) { + progressSliderState.value = progress.coerceIn(clippingRange) + } + } + } + + // Listen for touch/drag gestures on the start thumb to pause playback and record interaction. + launch { + startThumbInteractionSource.interactions.collect { interaction -> + if (interaction is DragInteraction.Start || interaction is PressInteraction.Press) { + onDragStarted(isStart = true) + } + } + } + + // Listen for touch/drag gestures on the end thumb to pause playback and record interaction. + launch { + endThumbInteractionSource.interactions.collect { interaction -> + if (interaction is DragInteraction.Start || interaction is PressInteraction.Press) { + onDragStarted(isStart = false) + } + } + } + + // Listen for touch/drag gestures on the progress thumb to pause playback and record + // interaction. + launch { + progressThumbInteractionSource.interactions.collect { interaction -> + if (interaction is DragInteraction.Start || interaction is PressInteraction.Press) { + isUserInteracting = true + pause() + } + } + } + + // Observe slider movements to enforce minRangeDelta constraints. + // TODO: b/505719491 - Potentially migrate to RangeSlider's onValueChange callback + // minRangeDelta clamping logic can be there instead of observing snapshotFlow here. + launch { + var previousRange = rangeSliderState.startValue..rangeSliderState.endValue + snapshotFlow { rangeSliderState.startValue..rangeSliderState.endValue } + .collect { currentRange -> + if (currentRange.start != previousRange.start) { + lastChangedBoundaryIsStart = true + } else if (currentRange.endInclusive != previousRange.endInclusive) { + lastChangedBoundaryIsStart = false + } + previousRange = currentRange + + if (isClipping) { + val minDelta = calculateMinRangeDelta(minClippedDurationMs, durationMs) + val proposedRange = clippingRangeFromSliderRange(currentRange) + var start = proposedRange.start + var end = proposedRange.endInclusive + if (end - start < minDelta) { + if (lastChangedBoundaryIsStart) { + start = (end - minDelta).coerceAtLeast(0f) + } else { + end = (start + minDelta).coerceAtMost(1f) + } + val clampedSliderRange = sliderRangeFromClippingRange(start..end) + if (rangeSliderState.startValue != clampedSliderRange.start) { + rangeSliderState.startValue = clampedSliderRange.start + } + if (rangeSliderState.endValue != clampedSliderRange.endInclusive) { + rangeSliderState.endValue = clampedSliderRange.endInclusive + } + } + if (clippingRange.start < clippingRange.endInclusive) { + progressSliderState.value = progressSliderState.value.coerceIn(clippingRange) + } + onClippingRangeChange?.invoke(clippingRangeMs) + } + } + } + + // Loop playback back to clipping start when the player reaches or exceeds the clipping end. launch { snapshotFlow { val progress = playbackProgress diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/editing/ClippingSliderDefaults.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/editing/ClippingSliderDefaults.kt new file mode 100644 index 00000000000..c96db9ed4c5 --- /dev/null +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/editing/ClippingSliderDefaults.kt @@ -0,0 +1,102 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.demo.compose.editing + +import androidx.compose.material3.MaterialTheme +import androidx.compose.runtime.Composable +import androidx.compose.runtime.Immutable +import androidx.compose.ui.graphics.Color + +/** + * Represents the colors used in a [ClippingSlider] to customize its appearance. + * + * @property clippingFrameColor The color of the selection frame and of the clipping handles. + * @property clippingThumbIconColor The color used to tint the icons within the clipping thumbs. + * @property positionThumbColor The color of the thumb representing the current playback position. + * @property inactiveTrackColor The color of the filter applied to the inactive areas outside the + * selected clipping range on the slider track. The underlying content remains visible if the + * color's alpha is less than 1f. + */ +@Immutable +class ClippingSliderColors( + val clippingFrameColor: Color, + val clippingThumbIconColor: Color, + val positionThumbColor: Color, + val inactiveTrackColor: Color, +) { + + /** Returns a copy of this ClippingSliderColors, optionally overriding some of the values. */ + fun copy( + clippingFrameColor: Color = this.clippingFrameColor, + clippingThumbIconColor: Color = this.clippingThumbIconColor, + positionThumbColor: Color = this.positionThumbColor, + inactiveTrackColor: Color = this.inactiveTrackColor, + ): ClippingSliderColors = + ClippingSliderColors( + clippingFrameColor, + clippingThumbIconColor, + positionThumbColor, + inactiveTrackColor, + ) + + override fun equals(other: Any?): Boolean { + if (this === other) return true + if (other !is ClippingSliderColors) return false + + if (clippingFrameColor != other.clippingFrameColor) return false + if (clippingThumbIconColor != other.clippingThumbIconColor) return false + if (positionThumbColor != other.positionThumbColor) return false + if (inactiveTrackColor != other.inactiveTrackColor) return false + + return true + } + + override fun hashCode(): Int { + var result = clippingFrameColor.hashCode() + result = 31 * result + clippingThumbIconColor.hashCode() + result = 31 * result + positionThumbColor.hashCode() + result = 31 * result + inactiveTrackColor.hashCode() + return result + } +} + +/** Contains the default values used by [ClippingSlider]. */ +object ClippingSliderDefaults { + + /** + * Creates a [ClippingSliderColors] with the default colors used in a [ClippingSlider]. + * + * @param clippingFrameColor The color of the selection frame and of the clipping handles. + * @param clippingThumbIconColor The color used to tint the icons within the clipping thumbs. + * @param positionThumbColor The color of the thumb representing the current playback position. + * @param inactiveTrackColor The color of the filter applied to the inactive areas outside the + * selected clipping range on the slider track. + */ + @Composable + fun colors( + clippingFrameColor: Color = MaterialTheme.colorScheme.primary, + clippingThumbIconColor: Color = MaterialTheme.colorScheme.onPrimary, + positionThumbColor: Color = MaterialTheme.colorScheme.inversePrimary, + inactiveTrackColor: Color = MaterialTheme.colorScheme.scrim.copy(alpha = 0.6f), + ): ClippingSliderColors = + ClippingSliderColors( + clippingFrameColor = clippingFrameColor, + clippingThumbIconColor = clippingThumbIconColor, + positionThumbColor = positionThumbColor, + inactiveTrackColor = inactiveTrackColor, + ) +} + diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/editing/VideoCropper.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/editing/VideoCropper.kt new file mode 100644 index 00000000000..5ca41adee4c --- /dev/null +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/editing/VideoCropper.kt @@ -0,0 +1,995 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.demo.compose.editing + +import androidx.compose.animation.core.animateFloatAsState +import androidx.compose.foundation.Canvas +import androidx.compose.foundation.gestures.awaitEachGesture +import androidx.compose.foundation.gestures.awaitFirstDown +import androidx.compose.foundation.gestures.calculateCentroid +import androidx.compose.foundation.gestures.calculatePan +import androidx.compose.foundation.gestures.calculateZoom +import androidx.compose.foundation.layout.Box +import androidx.compose.foundation.layout.BoxScope +import androidx.compose.foundation.layout.PaddingValues +import androidx.compose.foundation.layout.fillMaxSize +import androidx.compose.foundation.shape.RoundedCornerShape +import androidx.compose.runtime.Composable +import androidx.compose.runtime.Immutable +import androidx.compose.runtime.Stable +import androidx.compose.runtime.State +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableFloatStateOf +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.rememberUpdatedState +import androidx.compose.runtime.setValue +import androidx.compose.ui.Modifier +import androidx.compose.ui.draw.clipToBounds +import androidx.compose.ui.geometry.CornerRadius +import androidx.compose.ui.geometry.Offset +import androidx.compose.ui.geometry.Rect +import androidx.compose.ui.geometry.RoundRect +import androidx.compose.ui.geometry.Size +import androidx.compose.ui.geometry.lerp +import androidx.compose.ui.graphics.ClipOp +import androidx.compose.ui.graphics.Path +import androidx.compose.ui.graphics.StrokeCap +import androidx.compose.ui.graphics.StrokeJoin +import androidx.compose.ui.graphics.TransformOrigin +import androidx.compose.ui.graphics.drawscope.Stroke +import androidx.compose.ui.graphics.drawscope.clipPath +import androidx.compose.ui.graphics.graphicsLayer +import androidx.compose.ui.graphics.lerp +import androidx.compose.ui.input.pointer.PointerInputScope +import androidx.compose.ui.input.pointer.pointerInput +import androidx.compose.ui.input.pointer.positionChanged +import androidx.compose.ui.layout.layout +import androidx.compose.ui.layout.onSizeChanged +import androidx.compose.ui.platform.LocalDensity +import androidx.compose.ui.platform.LocalLayoutDirection +import androidx.compose.ui.unit.Constraints +import androidx.compose.ui.unit.Density +import androidx.compose.ui.unit.Dp +import androidx.compose.ui.unit.LayoutDirection +import androidx.compose.ui.unit.dp +import androidx.compose.ui.unit.toSize +import androidx.compose.ui.util.lerp +import androidx.media3.common.Player +import androidx.media3.ui.compose.ContentFrame +import kotlin.math.roundToInt + +/** The radius of the touch target for the corners of the crop overlay. */ +private val CORNER_TOUCH_TARGET_RADIUS = 24.dp + +/** + * A reusable video cropping widget. + * + * This component displays the video and a crop overlay. It allows the user to resize the crop area + * by dragging its corners (preserving the aspect ratio) and pan or zoom the video inside the crop + * area. + * + * @param state The [VideoCropperState] object holding the state. Created via + * [rememberVideoCropperState]. + * @param modifier The modifier to be applied to the cropping widget. + * @param onCropRectChangeFinished A callback that is invoked when a user interaction (dragging a + * corner, or panning or zooming the video) finishes. + * @param cropFrameControls An optional overlay to be rendered on top of the crop frame. + * @param colors The [VideoCropperColors] used to style the cropping widget. Defaults to + * [VideoCropperDefaults.colors]. + * @param cropFramePadding The padding values applied to the crop area boundaries, relative to the + * layout bounds of the [VideoCropper] composable. Defaults to + * [VideoCropperDefaults.CropFramePadding]. + * @param bracketThickness The stroke thickness of the corner brackets. Defaults to + * [VideoCropperDefaults.BracketThickness]. + * @param bracketLength The arm length of the corner brackets. Defaults to + * [VideoCropperDefaults.BracketLength]. + * @param cropFrameBorderThickness The stroke thickness of the frame border around the crop area. + * Defaults to [VideoCropperDefaults.CropFrameBorderThickness]. + * @param cropFrameShape The [RoundedCornerShape] defining the corner rounding radius of the crop + * frame and brackets. Defaults to [VideoCropperDefaults.CropFrameShape]. + * @param minCropSize The minimum width and height of the crop frame. Must be strictly positive. + * Defaults to [VideoCropperDefaults.MinCropSize]. + */ +// TODO: b/505719256 +// - Implement accessibility requirements +// - Implement color defaults +// - Move to material3 module and mark API unstable +// - Add tests +// - Add support for flexible aspect ratio +// - Add support for HDR +@Composable +fun VideoCropper( + state: VideoCropperState, + modifier: Modifier = Modifier, + onCropRectChangeFinished: (() -> Unit)? = null, + cropFrameControls: (@Composable BoxScope.() -> Unit)? = null, + colors: VideoCropperColors = VideoCropperDefaults.colors(), + cropFramePadding: PaddingValues = VideoCropperDefaults.CropFramePadding, + bracketThickness: Dp = VideoCropperDefaults.BracketThickness, + bracketLength: Dp = VideoCropperDefaults.BracketLength, + cropFrameBorderThickness: Dp = VideoCropperDefaults.CropFrameBorderThickness, + cropFrameShape: RoundedCornerShape = VideoCropperDefaults.CropFrameShape, + minCropSize: Dp = VideoCropperDefaults.MinCropSize, +) { + require(minCropSize > 0.dp) { "minCropSize must be strictly positive, but was $minCropSize" } + + val gestureState = remember { GestureState() } + var cropperSize by remember { mutableStateOf(Size.Zero) } + + val density = LocalDensity.current + val layoutDirection = LocalLayoutDirection.current + val isReady = !state.videoSize.isEmpty() && !cropperSize.isEmpty() + val interactingProgressState = + animateFloatAsState(targetValue = if (state.isInteracting) 1f else 0f) + val videoFitRect = + if (isReady) calculateVideoFitRect(cropperSize = cropperSize, videoSize = state.videoSize) + else Rect.Zero + val cropConstraintRect = + if (isReady) { + calculateCropConstraintRect( + cropperSize = cropperSize, + paddingValues = cropFramePadding, + density = density, + layoutDirection = layoutDirection, + ) + } else { + Rect.Zero + } + val transformation = + remember( + isReady, + state, + gestureState, + interactingProgressState, + videoFitRect, + cropConstraintRect, + ) { + if (isReady) { + CropFrameTransformation( + cropperState = state, + gestureState = gestureState, + interactionProgressState = interactingProgressState, + videoFitRect = videoFitRect, + cropConstraintRect = cropConstraintRect, + ) + } else { + null + } + } + + val style = + CropFrameStyle( + colors = colors, + bracketThickness = bracketThickness, + bracketLength = bracketLength, + cropFrameBorderThickness = cropFrameBorderThickness, + cropFrameShape = cropFrameShape, + minCropSize = minCropSize, + ) + + Box(modifier = modifier.clipToBounds().onSizeChanged { cropperSize = it.toSize() }) { + // Render the player container unconditionally so the player can attach to the surface and + // expose the video size. + VideoPlayerContainer(player = state.player, transformation = transformation) + if (transformation != null) { + CropFrameContainer( + cropperState = state, + gestureState = gestureState, + transformation = transformation, + videoFitRect = videoFitRect, + style = style, + onCropRectChangeFinished = onCropRectChangeFinished, + cropFrameControls = cropFrameControls, + ) + } + } +} + +/** + * A container that renders the [ContentFrame] video surface and applies the animated scale and + * translation offsets. + */ +@Composable +private fun VideoPlayerContainer( + player: Player?, + transformation: CropFrameTransformation?, + modifier: Modifier = Modifier, +) { + ContentFrame( + player = player, + modifier = + modifier.fillMaxSize().graphicsLayer { + val scale = transformation?.scale ?: 1f + scaleX = scale + scaleY = scale + val translation = transformation?.translation ?: Offset.Zero + translationX = translation.x + translationY = translation.y + transformOrigin = TransformOrigin(0f, 0f) + }, + ) +} + +/** + * A container that wraps the crop frame drawing, controls overlay, and gesture detection. + * + * This composable intercepts touch gestures on the screen. It distinguishes between dragging the + * corners of the crop frame and panning or zooming the video itself. It updates the crop rectangle + * coordinates and triggers the corresponding callbacks. + * + * @param cropperState The active [VideoCropperState]. + * @param gestureState The [GestureState] tracking active gesture interaction. + * @param transformation The calculated [CropFrameTransformation] containing the animated scales and + * translations. + * @param videoFitRect The baseline unscaled video rectangle (in pixels). + * @param style The [CropFrameStyle] configuration. + * @param modifier The modifier to be applied to the crop frame container. + * @param onCropRectChangeFinished A callback invoked when the user finishes + * dragging/panning/zooming. + * @param cropFrameControls An optional overlay content composable. + */ +@Composable +private fun CropFrameContainer( + cropperState: VideoCropperState, + gestureState: GestureState, + transformation: CropFrameTransformation, + videoFitRect: Rect, + style: CropFrameStyle, + modifier: Modifier = Modifier, + onCropRectChangeFinished: (() -> Unit)? = null, + cropFrameControls: (@Composable BoxScope.() -> Unit)? = null, +) { + val density = LocalDensity.current + val currentTransformation by rememberUpdatedState(transformation) + val currentVideoFitRect by rememberUpdatedState(videoFitRect) + val currentMinCropSizePx by rememberUpdatedState(with(density) { style.minCropSize.toPx() }) + val currentOnCropRectChangeFinished by rememberUpdatedState(onCropRectChangeFinished) + val currentTouchTargetSizePx by + rememberUpdatedState(with(density) { CORNER_TOUCH_TARGET_RADIUS.toPx() }) + Box( + modifier = + modifier.fillMaxSize().pointerInput(Unit) { + detectCropTransformGestures( + onGestureStart = { touchPoint -> + gestureState.startGesture( + transformation = currentTransformation, + touchPoint = touchPoint, + touchTargetSize = currentTouchTargetSizePx, + ) + cropperState.isInteracting = true + }, + onGesture = { centroid, pan, zoom, pointerCount -> + cropperState.cropRect = + gestureState.processGesture( + centroid = centroid, + pan = pan, + zoom = zoom, + pointerCount = pointerCount, + transformation = currentTransformation, + videoFitRect = currentVideoFitRect, + minCropSize = currentMinCropSizePx, + ) + }, + onGestureEnd = { + gestureState.finishGesture() + cropperState.isInteracting = false + currentOnCropRectChangeFinished?.invoke() + }, + ) + } + ) { + CropperOverlay(transformation = transformation, style = style) + cropFrameControls?.let { controls -> + CropFrameControlsOverlay(transformation = transformation, content = controls) + } + } +} + +/** + * A custom canvas that draws the dim overlay, the solid black crop frame border, and the corner + * brackets. + */ +@Composable +private fun CropperOverlay( + transformation: CropFrameTransformation, + style: CropFrameStyle, + modifier: Modifier = Modifier, +) { + val cropFramePath = remember { Path() } + val borderPath = remember { Path() } + val bracketsPath = remember { Path() } + val density = LocalDensity.current + val bracketStroke = + remember(density, style.bracketThickness) { + val thicknessPx = with(density) { style.bracketThickness.toPx() } + Stroke(width = thicknessPx, cap = StrokeCap.Round, join = StrokeJoin.Round) + } + + Canvas(modifier = modifier.fillMaxSize()) { + // Define the crop frame path as a rounded rectangle. + val cropFrame = transformation.cropFrame + val topLeftRadius = style.cropFrameShape.topStart.toPx(cropFrame.size, this) + val topRightRadius = style.cropFrameShape.topEnd.toPx(cropFrame.size, this) + val bottomRightRadius = style.cropFrameShape.bottomEnd.toPx(cropFrame.size, this) + val bottomLeftRadius = style.cropFrameShape.bottomStart.toPx(cropFrame.size, this) + cropFramePath.reset() + cropFramePath.addRoundRect( + RoundRect( + rect = cropFrame, + topLeft = CornerRadius(topLeftRadius), + topRight = CornerRadius(topRightRadius), + bottomRight = CornerRadius(bottomRightRadius), + bottomLeft = CornerRadius(bottomLeftRadius), + ) + ) + + // Draw the dimming overlay and the outer border. + val overlayColor = + lerp( + style.colors.idleOverlayColor, + style.colors.interactingOverlayColor, + transformation.progress, + ) + val borderThicknessPx = style.cropFrameBorderThickness.toPx() + val borderRect = cropFrame.inflate(borderThicknessPx) + borderPath.reset() + borderPath.addRoundRect( + RoundRect( + rect = borderRect, + topLeft = CornerRadius(if (topLeftRadius > 0f) topLeftRadius + borderThicknessPx else 0f), + topRight = + CornerRadius(if (topRightRadius > 0f) topRightRadius + borderThicknessPx else 0f), + bottomRight = + CornerRadius(if (bottomRightRadius > 0f) bottomRightRadius + borderThicknessPx else 0f), + bottomLeft = + CornerRadius(if (bottomLeftRadius > 0f) bottomLeftRadius + borderThicknessPx else 0f), + ) + ) + clipPath(path = cropFramePath, clipOp = ClipOp.Difference) { + drawRect(color = overlayColor) + drawPath(path = borderPath, color = style.colors.borderColor) + } + + // Draw the corner brackets. + val bracketThicknessPx = style.bracketThickness.toPx() + val bracketOffset = borderThicknessPx + bracketThicknessPx / 2f + val bracketLengthPx = style.bracketLength.toPx() + bracketsPath.reset() + bracketsPath.addCornerBracket( + corner = cropFrame.topLeft + Offset(-bracketOffset, -bracketOffset), + dirX = 1f, + dirY = 1f, + length = bracketLengthPx, + radius = topLeftRadius + bracketOffset, + ) + bracketsPath.addCornerBracket( + corner = cropFrame.topRight + Offset(bracketOffset, -bracketOffset), + dirX = -1f, + dirY = 1f, + length = bracketLengthPx, + radius = topRightRadius + bracketOffset, + ) + bracketsPath.addCornerBracket( + corner = cropFrame.bottomLeft + Offset(-bracketOffset, bracketOffset), + dirX = 1f, + dirY = -1f, + length = bracketLengthPx, + radius = bottomLeftRadius + bracketOffset, + ) + bracketsPath.addCornerBracket( + corner = cropFrame.bottomRight + Offset(bracketOffset, bracketOffset), + dirX = -1f, + dirY = -1f, + length = bracketLengthPx, + radius = bottomRightRadius + bracketOffset, + ) + drawPath(color = style.colors.bracketColor, path = bracketsPath, style = bracketStroke) + } +} + +/** + * A custom layout wrapper that measures and places the crop controls overlay exactly on top of the + * active crop frame bounds. + */ +@Composable +private fun CropFrameControlsOverlay( + transformation: CropFrameTransformation, + modifier: Modifier = Modifier, + content: @Composable BoxScope.() -> Unit, +) { + Box( + modifier = + modifier.layout { measurable, _ -> + val cropFrame = transformation.cropFrame + val width = cropFrame.width.roundToInt() + val height = cropFrame.height.roundToInt() + val placeable = measurable.measure(Constraints.fixed(width, height)) + layout(width, height) { + placeable.placeWithLayer(x = cropFrame.left.roundToInt(), y = cropFrame.top.roundToInt()) + } + } + ) { + content() + } +} + +/** + * Calculates the baseline unscaled screen rectangle where the video is displayed inside + * [cropperSize], fitting [videoSize] while preserving its aspect ratio (letterboxed or + * pillarboxed). + * + * This represents the 1x baseline boundaries of the video surface before any zoom or translation + * transformations are applied. + */ +private fun calculateVideoFitRect(cropperSize: Size, videoSize: Size): Rect = + calculateMaxRectInContainer( + aspectRatio = videoSize.width / videoSize.height, + container = Rect(Offset.Zero, cropperSize), + ) + +/** + * Calculates the largest rectangle with the specified aspect ratio that can fit inside a container + * rectangle, centering it. + */ +private fun calculateMaxRectInContainer(aspectRatio: Float, container: Rect): Rect { + val containerAspectRatio = container.width / container.height + val width: Float + val height: Float + if (aspectRatio > containerAspectRatio) { + width = container.width + height = container.width / aspectRatio + } else { + height = container.height + width = container.height * aspectRatio + } + val left = container.left + (container.width - width) / 2 + val top = container.top + (container.height - height) / 2 + return Rect(left = left, top = top, right = left + width, bottom = top + height) +} + +/** + * Calculates the bounding rectangle that constrains the crop frame, by applying [paddingValues] to + * the [cropperSize]. + * + * The crop frame cannot be resized or positioned outside of this returned rectangle. + * + * @param cropperSize The total size of the cropper component, in pixels. + * @param paddingValues The padding to apply to the boundaries. + * @param density The density to use for converting DP padding values to pixels. + * @param layoutDirection The layout direction to resolve left/right padding. + * @return The constraint rectangle in pixel coordinates. + */ +private fun calculateCropConstraintRect( + cropperSize: Size, + paddingValues: PaddingValues, + density: Density, + layoutDirection: LayoutDirection, +): Rect { + val left = + with(density) { paddingValues.calculateLeftPadding(layoutDirection).toPx() } + .coerceIn(0f, cropperSize.width - 1f) + val top = + with(density) { paddingValues.calculateTopPadding().toPx() } + .coerceIn(0f, cropperSize.height - 1f) + val right = + (cropperSize.width - + with(density) { paddingValues.calculateRightPadding(layoutDirection).toPx() }) + .coerceIn(left + 1f, cropperSize.width) + val bottom = + (cropperSize.height - with(density) { paddingValues.calculateBottomPadding().toPx() }).coerceIn( + top + 1f, + cropperSize.height, + ) + return Rect(left = left, top = top, right = right, bottom = bottom) +} + +private fun Rect.scale(scale: Float): Rect { + return Rect( + left = left * scale, + top = top * scale, + right = right * scale, + bottom = bottom * scale, + ) +} + +/** + * Appends a rounded corner bracket contour to this [Path] starting from the specified [corner] + * vertex extending along [dirX] and [dirY] directions. + * + * All coordinates and dimensions are in pixels. + * + * @param corner The corner vertex coordinate on screen. + * @param dirX Horizontal direction multiplier (+1f for right, -1f for left). + * @param dirY Vertical direction multiplier (+1f for down, -1f for up). + * @param length The total length of each bracket arm. + * @param radius The corner rounding radius where the bracket arms meet. + */ +private fun Path.addCornerBracket( + corner: Offset, + dirX: Float, + dirY: Float, + length: Float, + radius: Float, +) { + moveTo(corner.x + dirX * length, corner.y) + lineTo(corner.x + dirX * radius, corner.y) + val centerX = corner.x + dirX * radius + val centerY = corner.y + dirY * radius + arcTo( + rect = + Rect( + left = centerX - radius, + top = centerY - radius, + right = centerX + radius, + bottom = centerY + radius, + ), + startAngleDegrees = -90f * dirY, + sweepAngleDegrees = -90f * dirX * dirY, + forceMoveTo = false, + ) + lineTo(corner.x, corner.y + dirY * length) +} + +/** + * Detects single-finger (corner dragging or panning) and multi-touch (zooming and panning) gestures + * on [PointerInputScope]. + */ +private suspend fun PointerInputScope.detectCropTransformGestures( + onGestureStart: (touchPoint: Offset) -> Unit, + onGesture: (centroid: Offset, pan: Offset, zoom: Float, pointerCount: Int) -> Unit, + onGestureEnd: () -> Unit, +) { + awaitEachGesture { + val initialTouch = awaitFirstDown(requireUnconsumed = false) + onGestureStart(initialTouch.position) + var isInteracting = true + do { + val event = awaitPointerEvent() + val canceled = event.changes.any { it.isConsumed } + if (!canceled) { + val centroid = event.calculateCentroid(useCurrent = true) + val panChange = event.calculatePan() + val zoomChange = event.calculateZoom() + val pointerCount = event.changes.count { it.pressed } + if (zoomChange != 1f || panChange != Offset.Zero) { + onGesture(centroid, panChange, zoomChange, pointerCount) + } + for (change in event.changes) { + if (change.positionChanged()) { + change.consume() + } + } + } else { + isInteracting = false + } + } while (isInteracting && event.changes.any { it.pressed }) + onGestureEnd() + } +} + +/** + * State holder class managing gesture tracking and transformation calculations during corner drag, + * panning, and zooming gestures. + * + * This class serves as the source of truth for the active gesture session, tracking mutable state + * that must persist across recompositions (such as the active corner and the accumulated drag + * translation). It is decoupled from layout-specific details (like container padding) and UI + * transition animations, which are managed by [CropFrameTransformation]. + */ +@Stable +private class GestureState { + + /** The corner currently being dragged, or `null` if not dragging a corner. */ + var activeCorner by mutableStateOf(null) + private set + + /** + * The video scale used during an active interaction. + * + * - During corner dragging and panning, it remains constant to freeze the video zoom level. + * - During zooming, it updates dynamically to apply the user's zoom changes. + */ + var interactingVideoScale by mutableFloatStateOf(1f) + private set + + /** + * The video translation used during an active interaction (in pixels). + * + * - During corner dragging, it remains constant to freeze the video in place. + * - During panning or zooming, it updates dynamically to apply the user's pan changes. + */ + var interactingVideoTranslation by mutableStateOf(Offset.Zero) + private set + + /** + * The bounds of the crop frame captured at the start of the interaction (in pixels). + * + * This rect remains strictly fixed (frozen) during the gesture. During a pan or zoom gesture, the + * visual crop frame stays stationary on screen while the video scales and moves underneath it; + * this frozen rect is used as a reference to calculate the new crop rectangle relative to the + * video. + */ + var frozenCropFrame: Rect = Rect.Zero + private set + + /** + * Initializes the interaction state at the start of a gesture. + * + * Detects if the touch point is near any corner of the crop frame to initiate a corner resize. If + * not, it initiates a video pan/zoom interaction. + * + * All coordinates and sizes are expressed in pixels. + * + * @param transformation The active [CropFrameTransformation]. + * @param touchPoint The touch point coordinate. + * @param touchTargetSize The radius around corners within which a touch is registered as a corner + * drag. + */ + fun startGesture( + transformation: CropFrameTransformation, + touchPoint: Offset, + touchTargetSize: Float, + ) { + activeCorner = + detectCorner( + cropFrame = transformation.cropFrame, + touchPoint = touchPoint, + touchTargetSize = touchTargetSize, + ) + interactingVideoScale = transformation.nonInteractingScale + interactingVideoTranslation = transformation.nonInteractingTranslation + frozenCropFrame = transformation.cropFrame + } + + /** + * Processes the crop rectangle or video transformation based on gesture changes. + * + * If a corner is active, it resizes the crop frame. If panning or zooming the video, it + * translates and scales the video under the stationary crop frame. + * + * All coordinates, offsets, and sizes are expressed in pixels. + * + * @param centroid The focal point around which zooming occurs on screen. + * @param pan The pan displacement vector for this gesture step. + * @param zoom The incremental zoom factor for this gesture step. + * @param pointerCount The number of pointers currently pressed. + * @param transformation The active [CropFrameTransformation]. + * @param videoFitRect The bounds of the video when fit inside the [VideoCropper] container. + * @param minCropSize The minimum allowed size of the crop frame on screen. + * @return The updated normalized crop rectangle. + */ + fun processGesture( + centroid: Offset, + pan: Offset, + zoom: Float, + pointerCount: Int, + transformation: CropFrameTransformation, + videoFitRect: Rect, + minCropSize: Float, + ): Rect { + if (activeCorner != null && pointerCount > 1) { + // Transition from corner resizing to video transformation if a second pointer touches down. + frozenCropFrame = transformation.interactingCropFrame + activeCorner = null + } + + val corner = activeCorner + if (corner != null) { + // Resize the crop frame. + val newCropFrame = + resizeCropFrame( + cropFrame = transformation.interactingCropFrame, + corner = corner, + dragAmount = pan, + aspectRatio = transformation.cropAspectRatio, + bounds = transformation.interactingVideoRect, + minCropSize = minCropSize, + ) + + // Convert the on-screen crop frame pixel bounds into normalized [0..1] video coordinates + // relative to the video's active on-screen bounds. + val newCropRect = + Rect( + left = + (newCropFrame.left - transformation.interactingVideoRect.left) / + transformation.interactingVideoRect.width, + top = + (newCropFrame.top - transformation.interactingVideoRect.top) / + transformation.interactingVideoRect.height, + right = + (newCropFrame.right - transformation.interactingVideoRect.left) / + transformation.interactingVideoRect.width, + bottom = + (newCropFrame.bottom - transformation.interactingVideoRect.top) / + transformation.interactingVideoRect.height, + ) + return newCropRect + } + + // Otherwise, zoom and pan the video under the stationary crop frame. + + val minScaleX = frozenCropFrame.width / videoFitRect.width + val minScaleY = frozenCropFrame.height / videoFitRect.height + val minVideoScale = maxOf(minScaleX, minScaleY) + val maxScaleX = frozenCropFrame.width / minCropSize + val maxScaleY = frozenCropFrame.height / minCropSize + val maxVideoScale = maxOf(minVideoScale, minOf(maxScaleX, maxScaleY)) + val newVideoScale = (interactingVideoScale * zoom).coerceIn(minVideoScale, maxVideoScale) + + // Calculate new screen position of the video's top-left corner after scaling around centroid + // and panning. + val previousVideoTopLeft = + videoFitRect.topLeft * interactingVideoScale + interactingVideoTranslation + val scaleRatio = newVideoScale / interactingVideoScale + val tentativeTopLeft = centroid - (centroid - previousVideoTopLeft) * scaleRatio + pan + + // Clamp the video position so it fully covers the stationary on-screen crop frame. Use minOf + // to guard against float rounding discrepancies where minimum > maximum. + val videoWidth = videoFitRect.width * newVideoScale + val minVideoLeft = frozenCropFrame.right - videoWidth + val maxVideoLeft = frozenCropFrame.left + val clampedVideoLeft = + tentativeTopLeft.x.coerceIn(minOf(minVideoLeft, maxVideoLeft), maxVideoLeft) + + val videoHeight = videoFitRect.height * newVideoScale + val minVideoTop = frozenCropFrame.bottom - videoHeight + val maxVideoTop = frozenCropFrame.top + val clampedVideoTop = tentativeTopLeft.y.coerceIn(minOf(minVideoTop, maxVideoTop), maxVideoTop) + + interactingVideoScale = newVideoScale + interactingVideoTranslation = + Offset(clampedVideoLeft, clampedVideoTop) - videoFitRect.topLeft * newVideoScale + + // Convert the stationary on-screen crop frame position into normalized [0..1] video + // coordinates, clamped to avoid floating-point rounding violations. + val normalizedLeft = ((frozenCropFrame.left - clampedVideoLeft) / videoWidth).coerceIn(0f, 1f) + val normalizedTop = ((frozenCropFrame.top - clampedVideoTop) / videoHeight).coerceIn(0f, 1f) + val normalizedRight = + ((frozenCropFrame.right - clampedVideoLeft) / videoWidth).coerceIn(normalizedLeft, 1f) + val normalizedBottom = + ((frozenCropFrame.bottom - clampedVideoTop) / videoHeight).coerceIn(normalizedTop, 1f) + return Rect( + left = normalizedLeft, + top = normalizedTop, + right = normalizedRight, + bottom = normalizedBottom, + ) + } + + /** Resets the interaction state when a gesture finishes or is canceled. */ + fun finishGesture() { + activeCorner = null + } + + /** + * Determines which corner of the crop frame, if any, is close to the given touch point. + * + * @param cropFrame The current bounds of the on-screen crop frame. + * @param touchPoint The touch point coordinate. + * @param touchTargetSize The radius around corners within which a touch is registered as a corner + * drag. + * @return The detected [Corner], or `null` if the touch point is not near any corner. + */ + private fun detectCorner(cropFrame: Rect, touchPoint: Offset, touchTargetSize: Float): Corner? { + val touchTargetSizeSquared = touchTargetSize * touchTargetSize + return when { + (touchPoint - cropFrame.topLeft).getDistanceSquared() < touchTargetSizeSquared -> + Corner.TOP_LEFT + (touchPoint - cropFrame.topRight).getDistanceSquared() < touchTargetSizeSquared -> + Corner.TOP_RIGHT + (touchPoint - cropFrame.bottomLeft).getDistanceSquared() < touchTargetSizeSquared -> + Corner.BOTTOM_LEFT + (touchPoint - cropFrame.bottomRight).getDistanceSquared() < touchTargetSizeSquared -> + Corner.BOTTOM_RIGHT + else -> null + } + } + + /** + * Resizes the [cropFrame] crop frame rectangle by dragging a specific [corner] by [dragAmount], + * while strictly preserving [aspectRatio] and remaining within [bounds] and above [minCropSize]. + * + * To maintain the exact aspect ratio during unconstrained diagonal dragging, the touch coordinate + * is projected perpendicularly onto a linear constraint line passing through the stationary + * anchor corner (diagonally opposite to [corner]) with slope determined by [aspectRatio]. + * + * All coordinates, offsets, and sizes are expressed in pixels. + * + * @param cropFrame The current crop frame rectangle before applying [dragAmount]. + * @param corner The active [Corner] being dragged by the user. + * @param dragAmount The touch displacement vector for this drag event. + * @param aspectRatio The required width-to-height aspect ratio of the crop frame. + * @param bounds The maximum allowable screen bounding box constraining the crop frame. + * @param minCropSize The minimum allowable width and height for the crop frame. + * @return The resized and clamped crop frame rectangle. + */ + private fun resizeCropFrame( + cropFrame: Rect, + corner: Corner, + dragAmount: Offset, + aspectRatio: Float, + bounds: Rect, + minCropSize: Float, + ): Rect { + val isLeft = corner == Corner.TOP_LEFT || corner == Corner.BOTTOM_LEFT + val isTop = corner == Corner.TOP_LEFT || corner == Corner.TOP_RIGHT + + // The anchor is the stationary corner diagonally opposite to the dragged corner. + val anchorX = if (isLeft) cropFrame.right else cropFrame.left + val anchorY = if (isTop) cropFrame.bottom else cropFrame.top + + // Signs indicate whether dragging along the positive axes (+x / +y) expands (+1f) or + // contracts (-1f) the frame's width and height relative to the stationary anchor. + val signX = if (isLeft) -1f else 1f + val signY = if (isTop) -1f else 1f + + // Tentative dimensions from applying the drag delta. + val tentativeWidth = cropFrame.width + dragAmount.x * signX + val tentativeHeight = cropFrame.height + dragAmount.y * signY + + // Project (tentativeWidth, tentativeHeight) onto the aspect ratio line. + val projectedHeight = + (tentativeWidth * aspectRatio + tentativeHeight) / (aspectRatio * aspectRatio + 1f) + + // Determine the allowable height range constrained by bounds and minCropSize. + val maxBoundaryWidth = if (signX > 0) bounds.right - anchorX else anchorX - bounds.left + val maxBoundaryHeight = if (signY > 0) bounds.bottom - anchorY else anchorY - bounds.top + val maxHeight = minOf(maxBoundaryWidth / aspectRatio, maxBoundaryHeight) + val minHeight = maxOf(minCropSize, minCropSize / aspectRatio).coerceAtMost(maxHeight) + + // Clamp height and derive width. + val newHeight = projectedHeight.coerceIn(minHeight, maxHeight) + val newWidth = newHeight * aspectRatio + + val left = if (signX > 0) anchorX else anchorX - newWidth + val top = if (signY > 0) anchorY else anchorY - newHeight + return Rect(left = left, top = top, right = left + newWidth, bottom = top + newHeight) + } +} + +/** + * Derives and provides the scale, translation, and crop frame bounds for both interacting (active + * dragging/panning/zooming) and non-interacting (static preview) states. + * + * Custom property getters dynamically read state from [VideoCropperState] and [GestureState] during + * layout, draw, or gesture handling phases to prevent unnecessary recompositions during active + * interactions. + * + * During the non-interacting state, the crop frame is centered and fits within the widget's + * padding, while the video is scaled and translated so that the cropped region fills the crop + * frame. During the interacting state, either the video zoom and position are frozen (when resizing + * the crop frame), or the crop frame is frozen (when panning or zooming the video). + * + * This class is a presentation-focused, read-only adapter remembered in composition. It combines + * the active interaction state from [GestureState] and [VideoCropperState] with container layout + * constraints to derive the final visual targets. + * + * All coordinates and offsets in this class are in pixels. + * + * @param cropperState The active [VideoCropperState]. + * @param gestureState The active [GestureState]. + * @param interactionProgressState The animated interaction progress state (0f = idle, 1f = + * interacting). + * @param videoFitRect The baseline unscaled video rectangle (in pixels). + * @param cropConstraintRect The bounding rectangle constraining the crop frame (in pixels). + * @property cropAspectRatio The active aspect ratio of the crop frame. + * @property nonInteractingScale The target video scale when the user is not interacting. + * @property nonInteractingTranslation The target video translation when the user is not + * interacting. + * @property interactingVideoRect The video bounds during active interaction. + * @property interactingCropFrame The crop frame bounds during active interaction. + * @property progress The current interaction progress (0f = idle, 1f = interacting). + * @property cropFrame The current animated crop frame bounds. + * @property scale The current animated video scale. + * @property translation The current animated video translation offset. + */ +@Stable +private class CropFrameTransformation( + private val cropperState: VideoCropperState, + private val gestureState: GestureState, + private val interactionProgressState: State, + private val videoFitRect: Rect, + private val cropConstraintRect: Rect, +) { + + val cropAspectRatio: Float + get() = + cropperState.targetAspectRatio + ?: ((cropperState.cropRect.width * cropperState.videoSize.width) / + (cropperState.cropRect.height * cropperState.videoSize.height)) + + val nonInteractingScale: Float + get() = nonInteractingCropFrame.width / (videoFitRect.width * cropperState.cropRect.width) + + val nonInteractingTranslation: Offset + get() = + nonInteractingCropFrame.topLeft - + videoFitRect.topLeft * nonInteractingScale - + Offset( + x = cropperState.cropRect.left * videoFitRect.width * nonInteractingScale, + y = cropperState.cropRect.top * videoFitRect.height * nonInteractingScale, + ) + + val interactingVideoRect: Rect + get() = + videoFitRect + .scale(gestureState.interactingVideoScale) + .translate(gestureState.interactingVideoTranslation) + + val interactingCropFrame: Rect + get() = + if (gestureState.activeCorner == null) { + gestureState.frozenCropFrame + } else { + val videoRect = interactingVideoRect + Rect( + left = videoRect.left + cropperState.cropRect.left * videoRect.width, + top = videoRect.top + cropperState.cropRect.top * videoRect.height, + right = videoRect.left + cropperState.cropRect.right * videoRect.width, + bottom = videoRect.top + cropperState.cropRect.bottom * videoRect.height, + ) + } + + val progress: Float + get() = interactionProgressState.value + + val cropFrame: Rect + get() = + lerp( + start = nonInteractingCropFrame, + stop = interactingCropFrame, + fraction = interactionProgressState.value, + ) + + val scale: Float + get() = + lerp( + start = nonInteractingScale, + stop = gestureState.interactingVideoScale, + fraction = interactionProgressState.value, + ) + + val translation: Offset + get() = + lerp( + start = nonInteractingTranslation, + stop = gestureState.interactingVideoTranslation, + fraction = interactionProgressState.value, + ) + + private val nonInteractingCropFrame: Rect + get() = + calculateMaxRectInContainer(aspectRatio = cropAspectRatio, container = cropConstraintRect) +} + +/** Encapsulates visual styling and measurement configurations for rendering the crop frame. */ +@Immutable +private data class CropFrameStyle( + val colors: VideoCropperColors, + val bracketThickness: Dp, + val bracketLength: Dp, + val cropFrameBorderThickness: Dp, + val cropFrameShape: RoundedCornerShape, + val minCropSize: Dp, +) + +private enum class Corner { + TOP_LEFT, + TOP_RIGHT, + BOTTOM_LEFT, + BOTTOM_RIGHT, +} diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/editing/VideoCropperDefaults.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/editing/VideoCropperDefaults.kt new file mode 100644 index 00000000000..c42262b35fc --- /dev/null +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/editing/VideoCropperDefaults.kt @@ -0,0 +1,100 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.demo.compose.editing + +import androidx.compose.foundation.layout.PaddingValues +import androidx.compose.foundation.shape.RoundedCornerShape +import androidx.compose.runtime.Composable +import androidx.compose.runtime.Immutable +import androidx.compose.ui.graphics.Color +import androidx.compose.ui.unit.Dp +import androidx.compose.ui.unit.dp + +/** + * Represents the colors used in a [VideoCropper] to customize its appearance. + * + * @property idleOverlayColor The color of the mask applied to the area outside the crop frame when + * the user is not interacting. + * @property interactingOverlayColor The color of the mask applied to the area outside the crop + * frame when the user is interacting. + * @property bracketColor The color of the corner brackets. + * @property borderColor The color of the crop frame border. + */ +@Immutable +class VideoCropperColors( + val idleOverlayColor: Color, + val interactingOverlayColor: Color, + val bracketColor: Color, + val borderColor: Color, +) { + + /** Returns a copy of this VideoCropperColors, optionally overriding some of the values. */ + fun copy( + idleOverlayColor: Color = this.idleOverlayColor, + interactingOverlayColor: Color = this.interactingOverlayColor, + bracketColor: Color = this.bracketColor, + borderColor: Color = this.borderColor, + ): VideoCropperColors = + VideoCropperColors( + idleOverlayColor = idleOverlayColor, + interactingOverlayColor = interactingOverlayColor, + bracketColor = bracketColor, + borderColor = borderColor, + ) + + override fun equals(other: Any?): Boolean { + if (this === other) return true + if (other !is VideoCropperColors) return false + return idleOverlayColor == other.idleOverlayColor && + interactingOverlayColor == other.interactingOverlayColor && + bracketColor == other.bracketColor && + borderColor == other.borderColor + } + + override fun hashCode(): Int { + var result = idleOverlayColor.hashCode() + result = 31 * result + interactingOverlayColor.hashCode() + result = 31 * result + bracketColor.hashCode() + result = 31 * result + borderColor.hashCode() + return result + } +} + +/** Contains default values used by [VideoCropper]. */ +object VideoCropperDefaults { + + /** Creates a [VideoCropperColors] with the default colors used in a [VideoCropper]. */ + @Composable + fun colors( + idleOverlayColor: Color = Color.Black, + interactingOverlayColor: Color = Color.Black.copy(alpha = 0.5f), + bracketColor: Color = Color.White, + borderColor: Color = Color.Black, + ): VideoCropperColors = + VideoCropperColors( + idleOverlayColor = idleOverlayColor, + interactingOverlayColor = interactingOverlayColor, + bracketColor = bracketColor, + borderColor = borderColor, + ) + + val CropFramePadding: PaddingValues = PaddingValues(36.dp) + val BracketThickness: Dp = 4.dp + val BracketLength: Dp = 24.dp + val CropFrameBorderThickness: Dp = 4.dp + val CropFrameShape: RoundedCornerShape = RoundedCornerShape(8.dp) + val MinCropSize: Dp = 40.dp +} diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/editing/VideoCropperState.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/editing/VideoCropperState.kt new file mode 100644 index 00000000000..588bbc06514 --- /dev/null +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/editing/VideoCropperState.kt @@ -0,0 +1,203 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.demo.compose.editing + +import androidx.annotation.FloatRange +import androidx.compose.runtime.Composable +import androidx.compose.runtime.LaunchedEffect +import androidx.compose.runtime.SideEffect +import androidx.compose.runtime.Stable +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.setValue +import androidx.compose.ui.geometry.Rect +import androidx.compose.ui.geometry.Size +import androidx.media3.common.Player +import androidx.media3.common.VideoSize +import androidx.media3.ui.compose.state.PlayerStateObserver +import androidx.media3.ui.compose.state.observeState +import kotlin.math.abs + +/** + * A small epsilon value used to compare aspect ratios to account for floating point imprecision. + */ +private const val BOUNDARY_EPSILON = 1e-3f + +/** + * Remembers the value of [VideoCropperState] created based on the input parameters and launch a + * coroutine to listen to [Player's][Player] changes. If the [Player] instance changes between + * compositions, produces and remembers a new value. + * + * @param player The [Player] whose video content to crop. + * @param initialCropRect The initial crop rectangle normalized in [0..1] relative to the video + * dimensions. Must have positive width and height within [0..1]. If [targetAspectRatio] is set + * and does not match the aspect ratio of [initialCropRect], the crop rectangle will automatically + * be re-centered and resized to the largest possible rectangle matching [targetAspectRatio] once + * video dimensions are available. + * @param targetAspectRatio The desired target aspect ratio for the crop frame (e.g. 1f for square, + * 1.77f for 16:9). Must be strictly positive and finite, or null to preserve the aspect ratio of + * [initialCropRect]. + */ +@Composable +fun rememberVideoCropperState( + player: Player?, + initialCropRect: Rect = Rect(0f, 0f, 1f, 1f), + @FloatRange(from = 0.0, fromInclusive = false) targetAspectRatio: Float? = null, +): VideoCropperState { + val cropperState = + remember(player) { + VideoCropperState( + player = player, + initialCropRect = initialCropRect, + targetAspectRatio = targetAspectRatio, + ) + } + SideEffect { cropperState.targetAspectRatio = targetAspectRatio } + LaunchedEffect(player) { cropperState.observe() } + return cropperState +} + +/** + * A state holder object that manages the video cropper state. + * + * Use [rememberVideoCropperState] to create an instance that survives recompositions. + * + * @property cropRect The current crop rectangle normalized in [0..1] relative to the video + * dimensions. Coordinates must be within [0..1] with positive width and height. If + * [targetAspectRatio] is set and does not match the aspect ratio of [cropRect], the crop + * rectangle will automatically be re-centered and resized to the largest possible rectangle + * matching [targetAspectRatio] once video dimensions are available. + * @property targetAspectRatio The desired target aspect ratio for the crop frame. Must be strictly + * positive and finite, or null to preserve the aspect ratio of [cropRect]. + * @property isInteracting Whether the user is currently interacting with the cropper (dragging a + * corner, panning or zooming the video). + */ +@Stable +class VideoCropperState( + internal val player: Player?, + initialCropRect: Rect = Rect(0f, 0f, 1f, 1f), + @FloatRange(from = 0.0, fromInclusive = false) targetAspectRatio: Float? = null, +) { + + var cropRect: Rect + get() = _cropRect.value + set(newCropRect) { + validateCropRect(newCropRect) + _cropRect.value = newCropRect + } + + @setparam:FloatRange(from = 0.0, fromInclusive = false) + var targetAspectRatio: Float? + get() = _targetAspectRatio.value + set(newTargetAspectRatio) { + validateTargetAspectRatio(newTargetAspectRatio) + _targetAspectRatio.value = newTargetAspectRatio + updateCropRect() + } + + var isInteracting by mutableStateOf(false) + internal set + + internal var videoSize: Size by mutableStateOf(player?.videoSize?.toDisplaySize() ?: Size.Zero) + private set + + private val _cropRect = mutableStateOf(initialCropRect) + private val _targetAspectRatio = mutableStateOf(targetAspectRatio) + + private val playerStateObserver: PlayerStateObserver? = + player?.observeState(Player.EVENT_VIDEO_SIZE_CHANGED) { + val size = player.videoSize.toDisplaySize() + if (!size.isEmpty()) { + videoSize = size + updateCropRect() + } + } + + init { + validateCropRect(initialCropRect) + validateTargetAspectRatio(targetAspectRatio) + updateCropRect() + } + + suspend fun observe() { + playerStateObserver?.observe() + } + + private fun validateCropRect(cropRect: Rect) { + require( + cropRect.left in 0f..1f && + cropRect.top in 0f..1f && + cropRect.right in 0f..1f && + cropRect.bottom in 0f..1f && + cropRect.left < cropRect.right && + cropRect.top < cropRect.bottom + ) { + "cropRect coordinates must be within [0, 1] with positive width and height, but was $cropRect" + } + } + + private fun validateTargetAspectRatio(targetAspectRatio: Float?) { + require(targetAspectRatio == null || (targetAspectRatio > 0f && targetAspectRatio.isFinite())) { + "targetAspectRatio must be positive and finite, but was $targetAspectRatio" + } + } + + private fun updateCropRect() { + if (videoSize.isEmpty()) return + val currentTargetRatio = targetAspectRatio ?: return + val videoAspectRatio = videoSize.width / videoSize.height + val currentRatio = (cropRect.width / cropRect.height) * videoAspectRatio + if (abs(currentRatio - currentTargetRatio) > BOUNDARY_EPSILON) { + cropRect = calculateCropRectForAspectRatio(videoAspectRatio, currentTargetRatio) + } + } + + /** + * Calculates the largest centered crop rectangle in normalized [0..1] coordinates for a given + * target aspect ratio. + */ + private fun calculateCropRectForAspectRatio( + videoAspectRatio: Float, + targetAspectRatio: Float, + ): Rect { + if (targetAspectRatio <= 0f || videoAspectRatio <= 0f) { + return Rect(0f, 0f, 1f, 1f) + } + return if (targetAspectRatio <= videoAspectRatio) { + val normWidth = (targetAspectRatio / videoAspectRatio).coerceAtMost(1f) + val left = (1f - normWidth) / 2f + Rect(left = left, top = 0f, right = left + normWidth, bottom = 1f) + } else { + val normHeight = (videoAspectRatio / targetAspectRatio).coerceAtMost(1f) + val top = (1f - normHeight) / 2f + Rect(left = 0f, top = top, right = 1f, bottom = top + normHeight) + } + } + + /** + * Converts this [VideoSize] to a Compose [Size] representing the display dimensions, scaling the + * width based on [VideoSize.pixelWidthHeightRatio]. + */ + private fun VideoSize.toDisplaySize(): Size { + if (width <= 0 || height <= 0) return Size.Zero + val safeRatio = + if (pixelWidthHeightRatio <= 0f || pixelWidthHeightRatio.isNaN()) 1f + else pixelWidthHeightRatio + val displayWidth = width * safeRatio + return Size(displayWidth, height.toFloat()) + } +} diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/layout/EditingPlayerScreen.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/layout/EditingPlayerScreen.kt new file mode 100644 index 00000000000..9b7336e9cfa --- /dev/null +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/layout/EditingPlayerScreen.kt @@ -0,0 +1,193 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.demo.compose.layout + +import android.content.Context +import android.graphics.Bitmap +import androidx.activity.compose.BackHandler +import androidx.compose.foundation.background +import androidx.compose.foundation.layout.Arrangement +import androidx.compose.foundation.layout.Box +import androidx.compose.foundation.layout.Row +import androidx.compose.foundation.layout.fillMaxSize +import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.layout.navigationBarsPadding +import androidx.compose.foundation.layout.statusBarsPadding +import androidx.compose.material3.ExperimentalMaterial3Api +import androidx.compose.material3.Icon +import androidx.compose.material3.IconButton +import androidx.compose.material3.LocalContentColor +import androidx.compose.material3.MaterialTheme +import androidx.compose.runtime.Composable +import androidx.compose.runtime.CompositionLocalProvider +import androidx.compose.runtime.State +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.produceState +import androidx.compose.runtime.saveable.rememberSaveable +import androidx.compose.runtime.setValue +import androidx.compose.ui.Alignment +import androidx.compose.ui.Modifier +import androidx.compose.ui.platform.LocalContext +import androidx.compose.ui.res.painterResource +import androidx.media3.common.MediaItem +import androidx.media3.common.Player +import androidx.media3.common.util.ExperimentalApi +import androidx.media3.common.util.Log +import androidx.media3.demo.compose.R +import androidx.media3.demo.compose.editing.ClippingSlider +import androidx.media3.demo.compose.viewmodel.PlayerLifecycleViewModel +import androidx.media3.demo.compose.viewmodel.rememberPlayerWithLifecycle +import androidx.media3.effect.Presentation +import androidx.media3.effect.Presentation.LAYOUT_SCALE_TO_FIT_WITH_CROP +import androidx.media3.exoplayer.mediacodec.MediaCodecSelector +import androidx.media3.inspector.frame.FrameExtractor +import androidx.media3.ui.compose.material3.Player +import androidx.media3.ui.compose.material3.PlayerDefaults +import androidx.media3.ui.compose.material3.buttons.MuteButton +import androidx.media3.ui.compose.state.rememberProgressStateWithTickCount +import com.google.common.collect.ImmutableList +import java.util.Collections +import kotlinx.coroutines.CancellationException +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.guava.await +import kotlinx.coroutines.withContext + +private const val BITMAP_COUNT = 8 + +@OptIn(ExperimentalMaterial3Api::class) +@androidx.annotation.OptIn(ExperimentalApi::class) +@Composable +internal fun EditingPlayerScreen( + mediaItem: MediaItem, + playerViewModel: PlayerLifecycleViewModel, + modifier: Modifier = Modifier, +) { + val context = LocalContext.current + val player by rememberPlayerWithLifecycle(playerViewModel, mediaItem) + val durationMs = rememberProgressStateWithTickCount(player).durationMs + // A list state that will hold the extracted preview frames of the video for the clipping track. + val bitmaps by rememberExtractedFrames(context, mediaItem, durationMs) + var showCropping by rememberSaveable { mutableStateOf(false) } + + BackHandler(enabled = showCropping) { showCropping = false } + + CompositionLocalProvider(LocalContentColor provides MaterialTheme.colorScheme.primary) { + if (showCropping) { + VideoCropperScreen(player = player, modifier = modifier.fillMaxSize()) + } else { + EditingPlayerScreen( + player = player, + bitmaps = bitmaps, + mediaItem = mediaItem, + onCropClick = { showCropping = true }, + modifier = modifier.fillMaxSize(), + ) + } + } +} + +@OptIn(ExperimentalMaterial3Api::class) +@androidx.annotation.OptIn(ExperimentalApi::class) +@Composable +internal fun EditingPlayerScreen( + player: Player?, + bitmaps: ImmutableList, + mediaItem: MediaItem, + onCropClick: () -> Unit, + modifier: Modifier = Modifier, +) { + Box(modifier.background(MaterialTheme.colorScheme.background).statusBarsPadding()) { + Player( + player = player, + showControls = true, + bottomControls = { player, showControls -> + PlayerDefaults.BottomControls( + player, + showControls, + modifier = Modifier.fillMaxWidth().navigationBarsPadding(), + above = { + Row( + modifier = Modifier.fillMaxWidth(), + horizontalArrangement = Arrangement.SpaceBetween, + verticalAlignment = Alignment.CenterVertically, + ) { + IconButton(onClick = onCropClick) { + Icon( + painter = painterResource(R.drawable.media3_icon_crop), + contentDescription = "Crop", + ) + } + MuteButton(player) + } + }, + progressSlider = { player -> + ClippingSlider( + player = player, + bitmaps = bitmaps, + onClippingRangeChangeFinished = { + Log.d("EditingPlayerScreen", "onClippingRangeChangeFinished") + }, + ) + }, + ) + }, + ) + } +} + +@Composable +private fun rememberExtractedFrames( + context: Context, + mediaItem: MediaItem, + durationMs: Long, + bitmapCount: Int = BITMAP_COUNT, +): State> { + return produceState(initialValue = ImmutableList.of(), mediaItem, durationMs, bitmapCount) { + if (durationMs <= 0L) return@produceState + + try { + withContext(Dispatchers.IO) { + FrameExtractor.Builder(context, mediaItem) + .setEffects( + listOf(Presentation.createForWidthAndHeight(50, 50, LAYOUT_SCALE_TO_FIT_WITH_CROP)) + ) + .setMediaCodecSelector(MediaCodecSelector.DEFAULT) + .build() + .use { frameExtractor -> + val positions = + List(bitmapCount) { i -> i * durationMs / (bitmapCount - 1).coerceAtLeast(1) } + val futureFrames = positions.map { positionMs -> frameExtractor.getFrame(positionMs) } + // Fast-path: Await the first frame and immediately publish it to the UI. Because the + // slider's aspect ratio depends on the number of bitmaps, the list size should remain + // fixed, as changing it dynamically would cause the slider layout to resize each time + // the list grows or shrinks. + val firstBitmap = futureFrames.first().await().bitmap + value = ImmutableList.copyOf(Collections.nCopies(bitmapCount, firstBitmap)) + + // Slow-path: Await all remaining frames and publish the final list + val allBitmaps = futureFrames.map { it.await().bitmap } + value = ImmutableList.copyOf(allBitmaps) + } + } + } catch (e: CancellationException) { + throw e // Don't swallow cancellation + } catch (e: Exception) { + Log.e("EditingPlayerScreen", "Error extracting frames", e) + } + } +} diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/layout/LongFormPlayerScreen.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/layout/LongFormPlayerScreen.kt index 2ec4a387da0..9e500c2646a 100644 --- a/demos/compose/src/main/java/androidx/media3/demo/compose/layout/LongFormPlayerScreen.kt +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/layout/LongFormPlayerScreen.kt @@ -16,11 +16,10 @@ package androidx.media3.demo.compose.layout -import android.content.Context -import android.os.Build import androidx.compose.foundation.background import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.fillMaxSize import androidx.compose.foundation.layout.fillMaxWidth import androidx.compose.foundation.layout.navigationBarsPadding @@ -28,6 +27,8 @@ import androidx.compose.foundation.layout.padding import androidx.compose.foundation.layout.statusBarsPadding import androidx.compose.foundation.shape.RoundedCornerShape import androidx.compose.material3.Button +import androidx.compose.material3.ButtonDefaults +import androidx.compose.material3.ElevatedButton import androidx.compose.material3.ExperimentalMaterial3Api import androidx.compose.material3.LocalContentColor import androidx.compose.material3.MaterialTheme @@ -35,6 +36,7 @@ import androidx.compose.material3.Text import androidx.compose.runtime.Composable import androidx.compose.runtime.CompositionLocalProvider import androidx.compose.runtime.LaunchedEffect +import androidx.compose.runtime.collectAsState import androidx.compose.runtime.getValue import androidx.compose.runtime.mutableIntStateOf import androidx.compose.runtime.mutableStateOf @@ -44,29 +46,41 @@ import androidx.compose.runtime.saveable.rememberSaveable import androidx.compose.runtime.setValue import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier +import androidx.compose.ui.graphics.Color +import androidx.compose.ui.graphics.ColorFilter +import androidx.compose.ui.graphics.drawscope.DrawScope +import androidx.compose.ui.graphics.painter.Painter import androidx.compose.ui.input.pointer.PointerIcon import androidx.compose.ui.input.pointer.pointerHoverIcon import androidx.compose.ui.layout.onGloballyPositioned import androidx.compose.ui.layout.onSizeChanged import androidx.compose.ui.platform.LocalContext import androidx.compose.ui.platform.LocalDensity +import androidx.compose.ui.res.painterResource import androidx.compose.ui.unit.IntSize import androidx.compose.ui.unit.dp -import androidx.lifecycle.compose.LifecycleResumeEffect -import androidx.lifecycle.compose.LifecycleStartEffect +import androidx.media3.cast.MediaRouteButton +import androidx.media3.cast.rememberMediaRouteButtonState import androidx.media3.common.MediaItem -import androidx.media3.common.MediaMetadata import androidx.media3.common.Player import androidx.media3.common.util.ExperimentalApi +import androidx.media3.datasource.DataSourceBitmapLoader +import androidx.media3.demo.compose.R +import androidx.media3.demo.compose.buttons.CcButton import androidx.media3.demo.compose.buttons.LabeledProgressSlider import androidx.media3.demo.compose.buttons.SettingsBottomSheet import androidx.media3.demo.compose.buttons.SettingsButton -import androidx.media3.demo.compose.text.CurrentItemInfo +import androidx.media3.demo.compose.text.CastingOverlay import androidx.media3.demo.compose.text.FastForwardOverlay import androidx.media3.demo.compose.text.PlaylistInfoBottomSheet import androidx.media3.demo.compose.text.SeekOverlay import androidx.media3.demo.compose.text.SeekOverlayState +import androidx.media3.demo.compose.text.rememberCastState +import androidx.media3.demo.compose.viewmodel.PlayerLifecycleViewModel +import androidx.media3.demo.compose.viewmodel.rememberPlayerWithLifecycle import androidx.media3.exoplayer.ExoPlayer +import androidx.media3.ui.compose.Artwork +import androidx.media3.ui.compose.material3.MiniController import androidx.media3.ui.compose.material3.Player import androidx.media3.ui.compose.material3.PlayerDefaults import androidx.media3.ui.compose.material3.buttons.MuteButton @@ -74,85 +88,77 @@ import androidx.media3.ui.compose.state.rememberPlayPauseButtonState import androidx.media3.ui.compose.state.rememberPlaybackSpeedState import androidx.media3.ui.compose.state.rememberSeekBackButtonState import androidx.media3.ui.compose.state.rememberSeekForwardButtonState -import androidx.media3.ui.compose.text.CurrentMediaItemBox import kotlin.time.Duration.Companion.milliseconds import kotlinx.coroutines.Job import kotlinx.coroutines.delay import kotlinx.coroutines.launch @Composable -fun LongFormPlayerScreen( +internal fun LongFormPlayerScreen( playlistName: String, mediaItems: List, + playerViewModel: PlayerLifecycleViewModel, modifier: Modifier = Modifier, ) { - val context = LocalContext.current - var player by remember { mutableStateOf(null) } - - // See the following resources - // https://developer.android.com/topic/libraries/architecture/lifecycle#onStop-and-savedState - // https://developer.android.com/develop/ui/views/layout/support-multi-window-mode#multi-window_mode_configuration - // https://developer.android.com/develop/ui/compose/layouts/adaptive/support-multi-window-mode#android_9 - - if (Build.VERSION.SDK_INT > 23) { - // Initialize/release in onStart()/onStop() only because in a multi-window environment multiple - // apps can be visible at the same time. The apps that are out-of-focus are paused, but video - // playback should continue. - LifecycleStartEffect(mediaItems, playlistName) { - player = initializePlayer(context, playlistName, mediaItems) - onStopOrDispose { - player?.apply { release() } - player = null - } - } - } else { - // Call to onStop() is not guaranteed, hence we release the Player in onPause() instead - LifecycleResumeEffect(mediaItems, playlistName) { - player = initializePlayer(context, playlistName, mediaItems) - onPauseOrDispose { - player?.apply { release() } - player = null - } - } - } - + val player by + rememberPlayerWithLifecycle(playerViewModel, mediaItems, playlistName, useCast = true) + val localPlayer by playerViewModel.localPlayer.collectAsState() CompositionLocalProvider(LocalContentColor provides MaterialTheme.colorScheme.primary) { - LongFormPlayerScreen(player, modifier = modifier.fillMaxSize()) + LongFormPlayerScreen(player, localPlayer, modifier = modifier.fillMaxSize()) } } @OptIn(ExperimentalMaterial3Api::class) @androidx.annotation.OptIn(ExperimentalApi::class) @Composable -internal fun LongFormPlayerScreen(player: Player?, modifier: Modifier = Modifier) { +internal fun LongFormPlayerScreen( + player: Player?, + localPlayer: ExoPlayer?, + modifier: Modifier = Modifier, +) { val density = LocalDensity.current val scope = rememberCoroutineScope() - var currentContentScaleIndex by remember { mutableIntStateOf(0) } + var currentContentScaleIndex by rememberSaveable { mutableIntStateOf(0) } var showPlaylist by rememberSaveable { mutableStateOf(false) } - var showCurrentMediaItemInfo by rememberSaveable { mutableStateOf(false) } + var showMiniController by rememberSaveable { mutableStateOf(false) } var showSettings by rememberSaveable { mutableStateOf(false) } var bottomControlsHeight by remember { mutableStateOf(0.dp) } + val castState = rememberCastState(player) + val isRemotePlayback = castState.isRemotePlayback + val mediaRouteButtonState = rememberMediaRouteButtonState() - var showControls by remember { mutableStateOf(true) } + var showControls by rememberSaveable { mutableStateOf(true) } var anyPointerDown by remember { mutableStateOf(false) } val playPauseButtonState = rememberPlayPauseButtonState(player) - val hideJob = remember { mutableStateOf(null) } + val hideJobHolder = remember { JobHolder() } fun scheduleHideControls() { - hideJob.value?.cancel() + hideJobHolder.job?.cancel() if (!anyPointerDown) { - hideJob.value = scope.launch { + hideJobHolder.job = scope.launch { delay(CONTROLS_VISIBILITY_TIMEOUT) showControls = false } } } - LaunchedEffect(showControls, anyPointerDown, showSettings) { - if (showControls && !anyPointerDown && !showSettings) { + LaunchedEffect( + showControls, + anyPointerDown, + showSettings, + mediaRouteButtonState.isPickerVisible, + isRemotePlayback, + ) { + if ( + showControls && + !anyPointerDown && + !showSettings && + !mediaRouteButtonState.isPickerVisible && + !isRemotePlayback + ) { scheduleHideControls() } else { - hideJob.value?.cancel() + hideJobHolder.job?.cancel() } } @@ -161,6 +167,20 @@ internal fun LongFormPlayerScreen(player: Player?, modifier: Modifier = Modifier var showFastForward by remember { mutableStateOf(false) } val playbackSpeedState = rememberPlaybackSpeedState(player) + val context = LocalContext.current + val bitmapLoader = remember(context) { DataSourceBitmapLoader.Builder(context).build() } + + val errorPainter = + rememberTintedPainter( + painterResource(R.drawable.media3_icon_broken_image), + MaterialTheme.colorScheme.primary, + ) + val fallbackPainter = + rememberTintedPainter( + painterResource(R.drawable.media3_icon_default_album_image), + MaterialTheme.colorScheme.primary, + ) + Box( modifier .background(MaterialTheme.colorScheme.background) @@ -169,7 +189,17 @@ internal fun LongFormPlayerScreen(player: Player?, modifier: Modifier = Modifier ) { Player( player = player, - showControls = showControls, + artwork = { p -> + Artwork( + player = p, + contentDescription = null, + modifier = Modifier.fillMaxSize(), + bitmapLoader = bitmapLoader, + error = errorPainter, + fallback = fallbackPainter, + ) + }, + showControls = if (isRemotePlayback) true else showControls, modifier = Modifier.onGloballyPositioned { coordinates -> size = coordinates.size } .playerGestures( @@ -178,7 +208,7 @@ internal fun LongFormPlayerScreen(player: Player?, modifier: Modifier = Modifier showControls = true scheduleHideControls() }, - onToggleControls = { showControls = !showControls }, + onToggleControls = { if (!isRemotePlayback) showControls = !showControls }, playbackSpeedState = playbackSpeedState, seekBackButtonState = rememberSeekBackButtonState(player), seekForwardButtonState = rememberSeekForwardButtonState(player), @@ -188,7 +218,12 @@ internal fun LongFormPlayerScreen(player: Player?, modifier: Modifier = Modifier showControls = false seekOverlayState.show(it) }, - fastForwardActionArea = { offset -> offset.x >= size.width / 2 }, + // Only allow fast-forwarding if we are NOT showing the play button (i.e., we are + // playing) + // and the press is on the right half of the screen. + fastForwardActionArea = { offset -> + !playPauseButtonState.showPlay && offset.x >= size.width / 2 + }, onFastForward = { showControls = false showFastForward = it @@ -202,12 +237,26 @@ internal fun LongFormPlayerScreen(player: Player?, modifier: Modifier = Modifier }, ), contentScale = CONTENT_SCALES[currentContentScaleIndex].second, + shutter = { + Box(Modifier.fillMaxSize().background(Color.Black)) + CastingOverlay(castState, Modifier.fillMaxSize()) + }, topControls = { player, showControls -> - PlayerDefaults.TopControls(player, showControls, Modifier.fillMaxWidth()) { - SettingsButton( - Modifier.padding(horizontal = 15.dp).align(Alignment.CenterEnd), - onSettingsClick = { showSettings = true }, - ) + PlayerDefaults.TopControls( + player, + showControls, + Modifier.fillMaxWidth().padding(horizontal = 15.dp), + ) { + Row(Modifier.align(Alignment.CenterEnd)) { + CcButton(player = player) + MediaRouteButton(state = mediaRouteButtonState) + SettingsButton(onSettingsClick = { showSettings = true }) + } + } + }, + centerControls = { player, showControls -> + if (!isRemotePlayback) { + PlayerDefaults.CenterControls(player, showControls, Modifier.fillMaxWidth()) } }, bottomControls = { player, showControls -> @@ -218,19 +267,25 @@ internal fun LongFormPlayerScreen(player: Player?, modifier: Modifier = Modifier Modifier.fillMaxWidth().navigationBarsPadding().onSizeChanged { bottomControlsHeight = with(density) { it.height.toDp() } }, - above = { - Box(Modifier.fillMaxWidth()) { MuteButton(player, Modifier.align(Alignment.CenterEnd)) } + above = { player -> + if (isRemotePlayback) { + PlayerDefaults.CenterControls(player, showControls, Modifier.fillMaxWidth()) + } else { + Box(Modifier.fillMaxWidth()) { + MuteButton(player, Modifier.align(Alignment.CenterEnd)) + } + } + }, + progressSlider = { + val sliderPlayer = if (isRemotePlayback) player else localPlayer + LabeledProgressSlider(sliderPlayer) }, - progressSlider = { LabeledProgressSlider(it) }, ) }, ) Column(Modifier.align(Alignment.TopStart)) { PlaylistButton(onClick = { showPlaylist = true }) - PlayingNowButton( - showCurrentMediaItemInfo, - onClick = { showCurrentMediaItemInfo = !showCurrentMediaItemInfo }, - ) + PlayingNowButton(showMiniController, onClick = { showMiniController = !showMiniController }) } SeekOverlay( state = seekOverlayState, @@ -251,23 +306,23 @@ internal fun LongFormPlayerScreen(player: Player?, modifier: Modifier = Modifier ), ) } - if (showCurrentMediaItemInfo) { - CurrentMediaItemBox(player) { - Box( - Modifier.align(Alignment.BottomCenter) - .padding(bottom = bottomControlsHeight + 10.dp) - .fillMaxWidth() - .background(MaterialTheme.colorScheme.primaryContainer.copy(alpha = 0.5f)) - ) { - CurrentItemInfo(meta = mediaMetadata, Modifier.padding(8.dp)) - } - } + if (showMiniController) { + MiniController( + player = player, + modifier = + Modifier.fillMaxWidth() + .align(Alignment.BottomCenter) + .padding(bottom = bottomControlsHeight + 10.dp), + bitmapLoader = bitmapLoader, + defaultArtwork = fallbackPainter, + ) } if (showPlaylist) { PlaylistInfoBottomSheet( player = player, onDismissRequest = { showPlaylist = false }, modifier = Modifier.fillMaxWidth(), + bitmapLoader = bitmapLoader, ) } if (showSettings) { @@ -291,18 +346,30 @@ private fun PlaylistButton(modifier: Modifier = Modifier, onClick: () -> Unit) { @Composable private fun PlayingNowButton(visible: Boolean, modifier: Modifier = Modifier, onClick: () -> Unit) { - Button(onClick, modifier) { Text("Playing\nNow" + if (visible) " <<" else " >>") } + ElevatedButton( + onClick = onClick, + modifier = modifier, + elevation = + ButtonDefaults.elevatedButtonElevation(defaultElevation = if (visible) 0.dp else 8.dp), + ) { + Text("Playing Now") + } } -private fun initializePlayer( - context: Context, - playlistName: String, - mediaItems: List, -): Player = - ExoPlayer.Builder(context).build().apply { - setMediaItems(mediaItems) - setPlaylistMetadata(MediaMetadata.Builder().setTitle(playlistName).build()) - prepare() +@Composable +private fun rememberTintedPainter(painter: Painter, tint: Color): Painter { + return remember(painter, tint) { + object : Painter() { + override val intrinsicSize + get() = painter.intrinsicSize + + override fun DrawScope.onDraw() { + with(painter) { draw(size, colorFilter = ColorFilter.tint(tint)) } + } + } } +} private val CONTROLS_VISIBILITY_TIMEOUT = 3000.milliseconds + +private class JobHolder(var job: Job? = null) diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/layout/PlayerFormatScreen.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/layout/PlayerFormatScreen.kt index ceb5ea0387c..da0c9c2fdd5 100644 --- a/demos/compose/src/main/java/androidx/media3/demo/compose/layout/PlayerFormatScreen.kt +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/layout/PlayerFormatScreen.kt @@ -31,6 +31,7 @@ import androidx.compose.ui.unit.dp fun PlayerFormatScreen( onLongFormClick: () -> Unit, onShortFormClick: () -> Unit, + onEditingClick: (() -> Unit)? = null, modifier: Modifier = Modifier, ) { Column( @@ -41,5 +42,9 @@ fun PlayerFormatScreen( Button(onClick = onLongFormClick) { Text(text = "Long-Form Player") } Spacer(modifier = Modifier.height(16.dp)) Button(onClick = onShortFormClick) { Text(text = "Short-Form Player") } + if (onEditingClick != null) { + Spacer(modifier = Modifier.height(16.dp)) + Button(onClick = onEditingClick) { Text(text = "Editing Player") } + } } } diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/layout/SampleChooserScreen.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/layout/SampleChooserScreen.kt index 0ca49b5991c..2a0d6015f53 100644 --- a/demos/compose/src/main/java/androidx/media3/demo/compose/layout/SampleChooserScreen.kt +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/layout/SampleChooserScreen.kt @@ -95,7 +95,6 @@ fun SampleChooserScreen( } items(group.playlists) { playlist -> ListItem( - headlineContent = { Text(playlist.name) }, modifier = Modifier.clickable(enabled = !isLoading) { coroutineScope.launch { @@ -111,7 +110,9 @@ fun SampleChooserScreen( containerColor = MaterialTheme.colorScheme.surfaceVariant, headlineColor = MaterialTheme.colorScheme.onSurfaceVariant, ), - ) + ) { + Text(playlist.name) + } } } } diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/layout/ShortFormPlayerScreen.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/layout/ShortFormPlayerScreen.kt index ea0d7e86878..34fdf076116 100644 --- a/demos/compose/src/main/java/androidx/media3/demo/compose/layout/ShortFormPlayerScreen.kt +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/layout/ShortFormPlayerScreen.kt @@ -35,6 +35,8 @@ import androidx.media3.exoplayer.ExoPlayer import androidx.media3.ui.compose.lifecycle.SlidingWindowEffect import androidx.media3.ui.compose.lifecycle.rememberPooledPlayer import androidx.media3.ui.compose.material3.Player +import androidx.media3.ui.compose.material3.indicator.BufferingIndicator +import androidx.media3.ui.compose.state.SHOW_BUFFERING_WHEN_PLAYING import androidx.media3.ui.compose.state.rememberPlayPauseButtonState @OptIn(ExperimentalApi::class) @@ -104,6 +106,9 @@ internal fun ShortFormPlayerScreen( showControls = false, contentScale = ContentScale.Crop, modifier = Modifier.fillMaxSize().noRippleClickable(playPauseButtonState::onClick), + centerControls = { p, _ -> + BufferingIndicator(player = p, displayMode = SHOW_BUFFERING_WHEN_PLAYING) + }, ) } } diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/layout/VideoCropperScreen.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/layout/VideoCropperScreen.kt new file mode 100644 index 00000000000..c33f7aa30ff --- /dev/null +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/layout/VideoCropperScreen.kt @@ -0,0 +1,106 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.demo.compose.layout + +import androidx.compose.foundation.background +import androidx.compose.foundation.horizontalScroll +import androidx.compose.foundation.layout.Arrangement +import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.Row +import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.layout.navigationBarsPadding +import androidx.compose.foundation.layout.padding +import androidx.compose.foundation.layout.statusBarsPadding +import androidx.compose.foundation.rememberScrollState +import androidx.compose.material3.Button +import androidx.compose.material3.MaterialTheme +import androidx.compose.material3.OutlinedButton +import androidx.compose.material3.Text +import androidx.compose.runtime.Composable +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.saveable.rememberSaveable +import androidx.compose.runtime.setValue +import androidx.compose.ui.Alignment +import androidx.compose.ui.Modifier +import androidx.compose.ui.geometry.Rect +import androidx.compose.ui.unit.dp +import androidx.media3.common.Player +import androidx.media3.demo.compose.editing.VideoCropper +import androidx.media3.demo.compose.editing.VideoCropperDefaults +import androidx.media3.demo.compose.editing.rememberVideoCropperState + +private enum class AspectRatioPreset(val label: String, val ratio: Float?) { + ORIGINAL("Original", null), + SQUARE("1:1", 1f), + SIXTEEN_NINE("16:9", 16f / 9f), + NINE_SIXTEEN("9:16", 9f / 16f), + FOUR_THREE("4:3", 4f / 3f), + THREE_FOUR("3:4", 3f / 4f), + THREE_TWO("3:2", 3f / 2f), + TWO_THREE("2:3", 2f / 3f), + FIVE_FOUR("5:4", 5f / 4f), + FOUR_FIVE("4:5", 4f / 5f), +} + +@Composable +internal fun VideoCropperScreen(player: Player?, modifier: Modifier = Modifier) { + var selectedPreset by rememberSaveable { mutableStateOf(AspectRatioPreset.ORIGINAL) } + val cropperState = + rememberVideoCropperState(player = player, targetAspectRatio = selectedPreset.ratio) + + Column( + modifier = + modifier + .background(MaterialTheme.colorScheme.background) + .statusBarsPadding() + .navigationBarsPadding() + ) { + VideoCropper( + state = cropperState, + modifier = Modifier.weight(1f).fillMaxWidth(), + colors = + VideoCropperDefaults.colors( + idleOverlayColor = MaterialTheme.colorScheme.background, + interactingOverlayColor = MaterialTheme.colorScheme.background.copy(alpha = 0.6f), + bracketColor = MaterialTheme.colorScheme.primary, + borderColor = MaterialTheme.colorScheme.background, + ), + ) + + Row( + modifier = Modifier.fillMaxWidth().horizontalScroll(rememberScrollState()).padding(16.dp), + horizontalArrangement = Arrangement.spacedBy(8.dp), + verticalAlignment = Alignment.CenterVertically, + ) { + for (preset in AspectRatioPreset.entries) { + val isSelected = selectedPreset == preset + val onClick = { + selectedPreset = preset + if (preset == AspectRatioPreset.ORIGINAL) { + cropperState.cropRect = Rect(0f, 0f, 1f, 1f) + } + } + if (isSelected) { + Button(onClick = onClick) { Text(text = preset.label) } + } else { + OutlinedButton(onClick = onClick) { Text(text = preset.label) } + } + } + } + } +} diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/layout/modifiers.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/layout/modifiers.kt index 1dbcae975ab..504e93df556 100644 --- a/demos/compose/src/main/java/androidx/media3/demo/compose/layout/modifiers.kt +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/layout/modifiers.kt @@ -86,7 +86,8 @@ internal fun Modifier.reportPointerDown(onPointerDownChange: (Boolean) -> Unit): * * @param onPointerDownChange Lambda invoked with `true` when any pointer goes down, and `false` * when the last pointer goes up. - * @param onPointerMove Lambda invoked when the pointer moves within the Composable. + * @param onPointerMove Lambda invoked when a pointer hovers/moves within the Composable without + * being pressed. * @param onToggleControls Lambda invoked to show/hide player controls, typically used on a single * tap. * @param seekBackButtonState State object for handling seek back actions and providing seek @@ -158,7 +159,7 @@ internal fun Modifier.playerGestures( } val isAnyPressed = event.changes.any { it.pressed } onPointerDownChange?.invoke(isAnyPressed) - if (event.type == PointerEventType.Move) { + if (event.type == PointerEventType.Move && !isAnyPressed) { onPointerMove?.invoke() } } diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/shortform/ShortFormState.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/shortform/ShortFormState.kt index 68a861c9cbb..fcd00b4e393 100644 --- a/demos/compose/src/main/java/androidx/media3/demo/compose/shortform/ShortFormState.kt +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/shortform/ShortFormState.kt @@ -16,6 +16,7 @@ package androidx.media3.demo.compose.shortform import android.content.Context +import androidx.annotation.GuardedBy import androidx.annotation.MainThread import androidx.compose.runtime.Composable import androidx.compose.runtime.DisposableEffect @@ -58,16 +59,6 @@ internal class ShortFormState( private var playerCounter = 0 - private val cacheDelegate = lazy { - val downloadDirectory = context.getExternalFilesDir(null) ?: context.filesDir - SimpleCache( - /* cacheDir = */ File(downloadDirectory, "precache"), - /* evictor = */ NoOpCacheEvictor(), - /* databaseProvider = */ StandaloneDatabaseProvider(context), - ) - } - private val cache: Cache by cacheDelegate - private val targetPreloadStatusControl = object : TargetPreloadStatusControl { var currentPlayingIndex = C.INDEX_UNSET @@ -99,7 +90,7 @@ internal class ShortFormState( val preloadManagerBuilder = DefaultPreloadManager.Builder(context, targetPreloadStatusControl) .setLoadControl(loadControl) - .setCache(cache) + .setCache(getDownloadCache(context)) playerPool = PlayerPool(playerPoolCapacity) { @@ -150,6 +141,30 @@ internal class ShortFormState( if (::playerPool.isInitialized) playerPool.release() isReady = false } + + private companion object { + private const val PRECACHE_CONTENT_DIRECTORY = "precache" + private val lock = Any() + + @GuardedBy("lock") private var downloadCache: Cache? = null + + private fun getDownloadCache(context: Context): Cache = + synchronized(lock) { + downloadCache + ?: run { + val appContext = context.applicationContext + val downloadDirectory = + appContext.getExternalFilesDir(/* type= */ null) ?: appContext.filesDir + val downloadContentDirectory = File(downloadDirectory, PRECACHE_CONTENT_DIRECTORY) + SimpleCache( + downloadContentDirectory, + NoOpCacheEvictor(), + StandaloneDatabaseProvider(appContext), + ) + .also { downloadCache = it } + } + } + } } @Composable diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/text/Cast.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/text/Cast.kt new file mode 100644 index 00000000000..5feea7890d2 --- /dev/null +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/text/Cast.kt @@ -0,0 +1,115 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.demo.compose.text + +import androidx.annotation.OptIn +import androidx.compose.foundation.background +import androidx.compose.foundation.layout.Arrangement +import androidx.compose.foundation.layout.Box +import androidx.compose.foundation.layout.Column +import androidx.compose.foundation.layout.Spacer +import androidx.compose.foundation.layout.height +import androidx.compose.foundation.layout.size +import androidx.compose.material3.Icon +import androidx.compose.material3.MaterialTheme +import androidx.compose.material3.Text +import androidx.compose.runtime.Composable +import androidx.compose.runtime.LaunchedEffect +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.setValue +import androidx.compose.ui.Alignment +import androidx.compose.ui.Modifier +import androidx.compose.ui.res.painterResource +import androidx.compose.ui.unit.dp +import androidx.media3.cast.R as CastR +import androidx.media3.common.DeviceInfo +import androidx.media3.common.Player +import androidx.media3.common.util.UnstableApi +import androidx.media3.ui.compose.state.PlayerStateObserver +import androidx.media3.ui.compose.state.observeState + +/** + * Remembers the value of [CastState] created based on the passed [Player] and launches a coroutine + * to listen to [Player's][Player] changes. If the [Player] instance changes between compositions, + * produce and remember a new value. + */ +@OptIn(UnstableApi::class) +@Composable +internal fun rememberCastState(player: Player?): CastState { + val castState = remember(player) { CastState(player) } + LaunchedEffect(player) { castState.observe() } + return castState +} + +/** + * State that converts the necessary information from the [Player] to correctly deal with a UI + * component representing casting state. + */ +@OptIn(UnstableApi::class) +internal class CastState(private val player: Player?) { + var isRemotePlayback by + mutableStateOf(player?.deviceInfo?.playbackType == DeviceInfo.PLAYBACK_TYPE_REMOTE) + private set + + var deviceName by mutableStateOf(player?.deviceInfo?.routingControllerName ?: "") + private set + + private val playerStateObserver: PlayerStateObserver? = + player?.observeState(Player.EVENT_DEVICE_INFO_CHANGED) { + isRemotePlayback = player.deviceInfo.playbackType == DeviceInfo.PLAYBACK_TYPE_REMOTE + deviceName = player.deviceInfo.routingControllerName ?: "" + } + + /** + * Subscribes to updates from [Player.Events] and listens to [Player.EVENT_DEVICE_INFO_CHANGED] in + * order to determine whether playback is happening on a remote device. + */ + suspend fun observe() = playerStateObserver?.observe() +} + +@OptIn(UnstableApi::class) +@Composable +internal fun CastingOverlay(state: CastState, modifier: Modifier = Modifier) { + if (state.isRemotePlayback) { + Box( + modifier = modifier.background(MaterialTheme.colorScheme.surface.copy(alpha = 0.9f)), + contentAlignment = Alignment.Center, + ) { + Column( + horizontalAlignment = Alignment.CenterHorizontally, + verticalArrangement = Arrangement.Center, + ) { + Icon( + painter = painterResource(CastR.drawable.media_route_button_connected), + contentDescription = null, + modifier = Modifier.size(64.dp), + tint = MaterialTheme.colorScheme.primary, + ) + Spacer(modifier = Modifier.height(16.dp)) + Text( + text = + if (state.deviceName.isNotEmpty()) "Casting to ${state.deviceName}" + else "Casting to remote device", + style = MaterialTheme.typography.titleMedium, + color = MaterialTheme.colorScheme.onSurface, + ) + } + } + } +} diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/text/CurrentItemInfo.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/text/CurrentItemInfo.kt index c5d0d28c3ab..a71635805a8 100644 --- a/demos/compose/src/main/java/androidx/media3/demo/compose/text/CurrentItemInfo.kt +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/text/CurrentItemInfo.kt @@ -16,35 +16,24 @@ package androidx.media3.demo.compose.text -import android.graphics.Bitmap -import androidx.compose.foundation.Image -import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.Row -import androidx.compose.foundation.layout.padding -import androidx.compose.foundation.layout.size import androidx.compose.material3.Text import androidx.compose.runtime.Composable -import androidx.compose.runtime.LaunchedEffect -import androidx.compose.runtime.getValue -import androidx.compose.runtime.mutableStateOf -import androidx.compose.runtime.remember -import androidx.compose.runtime.setValue import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier -import androidx.compose.ui.graphics.asImageBitmap -import androidx.compose.ui.platform.LocalContext -import androidx.compose.ui.unit.dp import androidx.media3.common.C import androidx.media3.common.MediaMetadata import androidx.media3.common.util.Util -import androidx.media3.datasource.DataSourceBitmapLoader -import kotlinx.coroutines.guava.await @Composable -internal fun CurrentItemInfo(meta: MediaMetadata, modifier: Modifier = Modifier) { +internal fun CurrentItemInfo( + meta: MediaMetadata, + modifier: Modifier = Modifier, + artwork: @Composable () -> Unit = {}, +) { Row(modifier, verticalAlignment = Alignment.CenterVertically) { - Artwork(meta, Modifier.padding(end = 16.dp).size(64.dp)) + artwork() Column { Text("Title: ${meta.title ?: "Unknown Title"}") Text("Artist: ${meta.artist ?: "Unknown Artist"}") @@ -52,16 +41,3 @@ internal fun CurrentItemInfo(meta: MediaMetadata, modifier: Modifier = Modifier) } } } - -@Composable -private fun Artwork(meta: MediaMetadata, modifier: Modifier = Modifier) { - val context = LocalContext.current - val bitmapLoader = remember(context) { DataSourceBitmapLoader.Builder(context).build() } - var bitmap by remember(meta) { mutableStateOf(null) } - LaunchedEffect(meta) { - bitmap = runCatching { bitmapLoader.loadBitmapFromMetadata(meta)?.await() }.getOrNull() - } - - bitmap?.let { Image(bitmap = it.asImageBitmap(), contentDescription = "Artwork", modifier) } - ?: Box(modifier) -} diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/text/PlaylistInfo.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/text/PlaylistInfo.kt index 0f9bb2ba096..1b01f7cbeb0 100644 --- a/demos/compose/src/main/java/androidx/media3/demo/compose/text/PlaylistInfo.kt +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/text/PlaylistInfo.kt @@ -21,6 +21,7 @@ import androidx.compose.foundation.clickable import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.fillMaxWidth import androidx.compose.foundation.layout.padding +import androidx.compose.foundation.layout.size import androidx.compose.foundation.lazy.LazyColumn import androidx.compose.foundation.shape.RoundedCornerShape import androidx.compose.material3.Card @@ -35,7 +36,9 @@ import androidx.compose.ui.Modifier import androidx.compose.ui.unit.dp import androidx.media3.common.C import androidx.media3.common.Player +import androidx.media3.common.util.BitmapLoader import androidx.media3.common.util.Util +import androidx.media3.ui.compose.Artwork import androidx.media3.ui.compose.state.PlaylistState import androidx.media3.ui.compose.state.rememberPlaylistState @@ -45,6 +48,7 @@ internal fun PlaylistInfoBottomSheet( player: Player?, onDismissRequest: () -> Unit, modifier: Modifier = Modifier, + bitmapLoader: BitmapLoader? = null, ) { val playlistState = rememberPlaylistState(player) ModalBottomSheet( @@ -55,16 +59,21 @@ internal fun PlaylistInfoBottomSheet( Column(horizontalAlignment = Alignment.CenterHorizontally) { PlaylistTitle(playlistState) PlaylistStats(playlistState) - PlaylistAsCards(playlistState, onDismissRequest) + PlaylistAsCards(playlistState, onDismissRequest, bitmapLoader) } } } @Composable -private fun PlaylistAsCards(playlistState: PlaylistState, onDismissRequest: () -> Unit) { +private fun PlaylistAsCards( + playlistState: PlaylistState, + onDismissRequest: () -> Unit, + bitmapLoader: BitmapLoader? = null, +) { LazyColumn(horizontalAlignment = Alignment.CenterHorizontally) { items(playlistState.mediaItemCount) { index -> val chosen = index == playlistState.currentMediaItemIndex + val meta = playlistState.getMediaItemAt(index).mediaMetadata Card( shape = RoundedCornerShape(4.dp), border = BorderStroke(6.dp, MaterialTheme.colorScheme.inversePrimary).takeIf { chosen }, @@ -81,7 +90,17 @@ private fun PlaylistAsCards(playlistState: PlaylistState, onDismissRequest: () - onDismissRequest() }, ) { - CurrentItemInfo(playlistState.getMediaItemAt(index).mediaMetadata, Modifier.padding(16.dp)) + CurrentItemInfo( + meta = meta, + modifier = Modifier.padding(16.dp), + artwork = { + Artwork( + metadata = meta, + modifier = Modifier.padding(end = 16.dp).size(64.dp), + bitmapLoader = bitmapLoader, + ) + }, + ) } } } diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/viewmodel/ComposeDemoViewModel.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/viewmodel/ComposeDemoViewModel.kt deleted file mode 100644 index 67079ec596e..00000000000 --- a/demos/compose/src/main/java/androidx/media3/demo/compose/viewmodel/ComposeDemoViewModel.kt +++ /dev/null @@ -1,38 +0,0 @@ -/* - * Copyright 2025 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ - -package androidx.media3.demo.compose.viewmodel - -import androidx.lifecycle.ViewModel -import androidx.media3.common.MediaItem -import kotlinx.coroutines.flow.MutableStateFlow -import kotlinx.coroutines.flow.asStateFlow - -internal class ComposeDemoViewModel : ViewModel() { - private val _mediaItems = MutableStateFlow>(emptyList()) - val mediaItems = _mediaItems.asStateFlow() - - private val _playlistName = MutableStateFlow("") - val playlistName = _playlistName.asStateFlow() - - fun selectMediaItems(mediaItems: List) { - _mediaItems.value = mediaItems - } - - fun selectPlaylistName(playlistName: String) { - _playlistName.value = playlistName - } -} diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/viewmodel/NavigationViewModel.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/viewmodel/NavigationViewModel.kt new file mode 100644 index 00000000000..ef73f9e1dbe --- /dev/null +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/viewmodel/NavigationViewModel.kt @@ -0,0 +1,75 @@ +/* + * Copyright 2025 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.demo.compose.viewmodel + +import android.os.Bundle +import androidx.lifecycle.SavedStateHandle +import androidx.lifecycle.ViewModel +import androidx.media3.common.MediaItem +import androidx.media3.common.MediaLibraryInfo +import kotlinx.coroutines.flow.MutableStateFlow +import kotlinx.coroutines.flow.asStateFlow + +/** + * [ViewModel] responsible for holding and sharing user-selected sample data across Compose + * Navigation destinations. + * + * This ViewModel is capable of: + * - Passing the selected [mediaItems] and [playlistName] from the sample selection screen to player + * destinations. + * - Saving and restoring playlist metadata and [MediaItem] lists via [SavedStateHandle], allowing + * playback screens to recover after OS process death (`adb shell am kill`). + */ +internal class NavigationViewModel(private val savedStateHandle: SavedStateHandle) : ViewModel() { + private val _mediaItems = + MutableStateFlow( + savedStateHandle.get>(KEY_MEDIA_ITEMS)?.map { + MediaItem.fromBundle(it, MediaLibraryInfo.INTERFACE_VERSION) + } ?: emptyList() + ) + /** The currently selected list of [MediaItem]s to play. */ + val mediaItems = _mediaItems.asStateFlow() + + private val _playlistName = + MutableStateFlow(savedStateHandle.get(KEY_PLAYLIST_NAME) ?: "") + /** The display name of the currently selected playlist. */ + val playlistName = _playlistName.asStateFlow() + + /** + * Updates the selected [mediaItems] and serializes them into [SavedStateHandle] using + * `toBundleIncludeLocalConfiguration()` to preserve local URIs and subtitle configurations across + * process death. + */ + fun selectMediaItems(mediaItems: List) { + savedStateHandle[KEY_MEDIA_ITEMS] = + ArrayList( + mediaItems.map { it.toBundleIncludeLocalConfiguration(MediaLibraryInfo.INTERFACE_VERSION) } + ) + _mediaItems.value = mediaItems + } + + /** Updates the selected [playlistName] and stores it into [SavedStateHandle]. */ + fun selectPlaylistName(playlistName: String) { + savedStateHandle[KEY_PLAYLIST_NAME] = playlistName + _playlistName.value = playlistName + } + + companion object { + private const val KEY_MEDIA_ITEMS = "media_items" + private const val KEY_PLAYLIST_NAME = "playlist_name" + } +} diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/viewmodel/PlayerLifecycleViewModel.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/viewmodel/PlayerLifecycleViewModel.kt new file mode 100644 index 00000000000..b4550722afc --- /dev/null +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/viewmodel/PlayerLifecycleViewModel.kt @@ -0,0 +1,203 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.demo.compose.viewmodel + +import android.app.Application +import android.os.Bundle +import androidx.lifecycle.AndroidViewModel +import androidx.lifecycle.SavedStateHandle +import androidx.media3.cast.CastPlayer +import androidx.media3.common.C +import androidx.media3.common.MediaItem +import androidx.media3.common.MediaMetadata +import androidx.media3.common.Player +import androidx.media3.common.TrackSelectionParameters +import androidx.media3.exoplayer.ExoPlayer +import kotlin.math.max +import kotlinx.coroutines.flow.MutableStateFlow +import kotlinx.coroutines.flow.StateFlow +import kotlinx.coroutines.flow.asStateFlow + +/** + * [AndroidViewModel] that manages an application-wide singleton [Player] instance and automatically + * saves and restores its playback state across configuration changes and OS process death. + * + * This ViewModel is capable of: + * - Initializing the [ExoPlayer] when entering the player screen ([initializePlayer]) and releasing + * hardware codecs and threads when leaving or backgrounding ([releasePlayer]). + * - State Persistence via [SavedStateHandle]: Storing playback position ([startPosition]), playlist + * index ([startItemIndex]), play-when-ready state ([startAutoPlay]), volume ([startVolume]), and + * track selection parameters ([startTrackSelectionParameters]) into the system saved state bundle + * without requiring Activity or Composable boilerplate. + * - Handling loading of the data with the [loadPlaylist] method that handles idempotent playlist + * comparison, optional metadata assignment, state restoration + */ +internal class PlayerLifecycleViewModel( + application: Application, + private val savedStateHandle: SavedStateHandle, +) : AndroidViewModel(application) { + + private val _localPlayer = MutableStateFlow(null) + val localPlayer: StateFlow = _localPlayer.asStateFlow() + + private val _player = MutableStateFlow(null) + val player: StateFlow = _player.asStateFlow() + + private var startTrackSelectionParameters: TrackSelectionParameters + get() = + savedStateHandle.get(KEY_TRACK_SELECTION_PARAMETERS)?.let { + TrackSelectionParameters.fromBundle(it) + } ?: TrackSelectionParameters.DEFAULT + set(value) { + savedStateHandle[KEY_TRACK_SELECTION_PARAMETERS] = value.toBundle() + } + + private var startAutoPlay: Boolean + get() = savedStateHandle.get(KEY_AUTO_PLAY) ?: true + set(value) { + savedStateHandle[KEY_AUTO_PLAY] = value + } + + private var startItemIndex: Int + get() = savedStateHandle.get(KEY_ITEM_INDEX) ?: C.INDEX_UNSET + set(value) { + savedStateHandle[KEY_ITEM_INDEX] = value + } + + private var startPosition: Long + get() = savedStateHandle.get(KEY_POSITION) ?: C.TIME_UNSET + set(value) { + savedStateHandle[KEY_POSITION] = value + } + + private var startVolume: Float + get() = savedStateHandle.get(KEY_VOLUME) ?: 1f + set(value) { + savedStateHandle[KEY_VOLUME] = value + } + + /** + * Initializes the singleton [Player] instance if it has not already been created. + * + * @param useCast Whether to wrap the local [ExoPlayer] in a [CastPlayer]. + * + * Applies the saved [startTrackSelectionParameters], [startAutoPlay], and [startVolume] from + * [SavedStateHandle]. + */ + fun initializePlayer(useCast: Boolean = false) { + if (_player.value == null) { + val exoPlayer = + ExoPlayer.Builder(getApplication()).build().apply { + trackSelectionParameters = startTrackSelectionParameters + playWhenReady = startAutoPlay + volume = startVolume + } + if (useCast) { + _localPlayer.value = exoPlayer + _player.value = CastPlayer.Builder(getApplication()).setLocalPlayer(exoPlayer).build() + } else { + _localPlayer.value = null + _player.value = exoPlayer + } + } + } + + /** + * Loads a single [mediaItem] and an optional [playlistName] into the player, then prepares it. + */ + fun loadPlaylist(mediaItem: MediaItem, playlistName: String? = null) { + loadPlaylist(listOf(mediaItem), playlistName) + } + + /** + * Loads a list of [mediaItems] and an optional [playlistName] into the player, then prepares it. + * + * When loading into a newly created player instance (`mediaItemCount == 0`), this method + * atomically restores any saved [startItemIndex] and [startPosition] via + * `setMediaItems(mediaItems, startItemIndex, startPosition)` to avoid default position resets. + */ + fun loadPlaylist(mediaItems: List, playlistName: String? = null) { + _player.value?.apply { + if (isPlaylistDifferent(mediaItems)) { + val wasEmpty = mediaItemCount == 0 + if (wasEmpty && startItemIndex != C.INDEX_UNSET) { + setMediaItems(mediaItems, startItemIndex, startPosition) + } else { + setMediaItems(mediaItems) + } + if (!playlistName.isNullOrEmpty()) { + setPlaylistMetadata(MediaMetadata.Builder().setTitle(playlistName).build()) + } + prepare() + } else if (!playlistName.isNullOrEmpty() && playlistMetadata.title == null) { + setPlaylistMetadata(MediaMetadata.Builder().setTitle(playlistName).build()) + } + } + } + + private fun Player.isPlaylistDifferent(mediaItems: List): Boolean = + mediaItemCount != mediaItems.size || + mediaItems.indices.any { getMediaItemAt(it) != mediaItems[it] } + + /** + * Saves the current playback state into [SavedStateHandle] and releases the hardware codecs and + * player resources. + * + * Called when the composable screen is paused or stopped (e.g. app backgrounded or screen + * rotated). + */ + fun releasePlayer() { + _player.value?.let { player -> + startAutoPlay = player.playWhenReady + startItemIndex = player.currentMediaItemIndex + startPosition = max(0L, player.contentPosition) + startVolume = player.volume + startTrackSelectionParameters = player.trackSelectionParameters + } + // Release standalone ExoPlayer OR CastPlayer (which auto-releases the underlying localPlayer) + _player.value?.release() + _localPlayer.value = null + _player.value = null + } + + override fun onCleared() { + releasePlayer() + clearSavedState() + } + + /** + * Resets all saved playback state properties in [SavedStateHandle] back to their defaults. + * + * Called when navigating away from the player screen so that subsequent playlists start from the + * beginning (`0:00`) instead of resuming a previous session's timestamp. + */ + fun clearSavedState() { + startAutoPlay = true + startItemIndex = C.INDEX_UNSET + startPosition = C.TIME_UNSET + startVolume = 1f + startTrackSelectionParameters = TrackSelectionParameters.DEFAULT + } + + companion object { + private const val KEY_TRACK_SELECTION_PARAMETERS = "track_selection_parameters" + private const val KEY_ITEM_INDEX = "item_index" + private const val KEY_POSITION = "position" + private const val KEY_AUTO_PLAY = "auto_play" + private const val KEY_VOLUME = "volume" + } +} diff --git a/demos/compose/src/main/java/androidx/media3/demo/compose/viewmodel/rememberPlayerWithLifecycle.kt b/demos/compose/src/main/java/androidx/media3/demo/compose/viewmodel/rememberPlayerWithLifecycle.kt new file mode 100644 index 00000000000..475db1699ca --- /dev/null +++ b/demos/compose/src/main/java/androidx/media3/demo/compose/viewmodel/rememberPlayerWithLifecycle.kt @@ -0,0 +1,115 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.demo.compose.viewmodel + +import android.app.Activity +import android.content.Context +import android.content.ContextWrapper +import androidx.compose.runtime.Composable +import androidx.compose.runtime.DisposableEffect +import androidx.compose.runtime.LaunchedEffect +import androidx.compose.runtime.State +import androidx.compose.runtime.collectAsState +import androidx.compose.runtime.getValue +import androidx.compose.runtime.rememberUpdatedState +import androidx.compose.ui.platform.LocalContext +import androidx.lifecycle.Lifecycle +import androidx.lifecycle.LifecycleEventObserver +import androidx.lifecycle.compose.LocalLifecycleOwner +import androidx.media3.common.MediaItem +import androidx.media3.common.Player + +/** + * Connects the [PlayerLifecycleViewModel] to the Compose UI lifecycle and automatically handles + * initialization, playlist loading, state restoration, and release. + * + * See the following resources: + * - https://developer.android.com/topic/libraries/architecture/lifecycle#onStop-and-savedState + * - https://developer.android.com/develop/ui/views/layout/support-multi-window-mode#multi-window_mode_configuration + * - https://developer.android.com/develop/ui/compose/layouts/adaptive/support-multi-window-mode#android_9 + * + * @return A [State] containing the current [Player] instance. + */ +@Composable +internal fun rememberPlayerWithLifecycle( + playerViewModel: PlayerLifecycleViewModel, + mediaItems: List, + playlistName: String? = null, + useCast: Boolean = false, +): State { + val context = LocalContext.current + val lifecycleOwner = LocalLifecycleOwner.current + val player = playerViewModel.player.collectAsState() + val currentMediaItems by rememberUpdatedState(mediaItems) + val currentPlaylistName by rememberUpdatedState(playlistName) + val currentUseCast by rememberUpdatedState(useCast) + + DisposableEffect(lifecycleOwner) { + val observer = LifecycleEventObserver { _, event -> + val activity = context.findActivity() + when (event) { + Lifecycle.Event.ON_START -> { + playerViewModel.initializePlayer(currentUseCast) + playerViewModel.loadPlaylist(currentMediaItems, currentPlaylistName) + } + Lifecycle.Event.ON_STOP -> { + // Ignore simple configuration changes like rotation (see android:configChanges) + if (activity?.isChangingConfigurations != true) { + playerViewModel.releasePlayer() + } + } + else -> {} + } + } + lifecycleOwner.lifecycle.addObserver(observer) + onDispose { lifecycleOwner.lifecycle.removeObserver(observer) } + } + + DisposableEffect(Unit) { + // When the user navigates back (to SampleChooserScreen), release the player. + onDispose { + val activity = context.findActivity() + if (activity?.isChangingConfigurations != true) { + playerViewModel.releasePlayer() + playerViewModel.clearSavedState() + } + } + } + + LaunchedEffect(mediaItems, playlistName, useCast) { + playerViewModel.initializePlayer(useCast) + playerViewModel.loadPlaylist(mediaItems, playlistName) + } + + return player +} + +@Composable +internal fun rememberPlayerWithLifecycle( + playerViewModel: PlayerLifecycleViewModel, + mediaItem: MediaItem, + playlistName: String? = null, + useCast: Boolean = false, +): State = + rememberPlayerWithLifecycle(playerViewModel, listOf(mediaItem), playlistName, useCast) + +internal fun Context.findActivity(): Activity? = + when (this) { + is Activity -> this + is ContextWrapper -> baseContext.findActivity() + else -> null + } diff --git a/demos/compose/src/main/res/drawable/media3_icon_broken_image.xml b/demos/compose/src/main/res/drawable/media3_icon_broken_image.xml new file mode 100644 index 00000000000..04e03725ab9 --- /dev/null +++ b/demos/compose/src/main/res/drawable/media3_icon_broken_image.xml @@ -0,0 +1,25 @@ + + + + + diff --git a/libraries/session/src/main/res/layout/media3_notification_media_action.xml b/demos/compose/src/main/res/drawable/media3_icon_crop.xml similarity index 57% rename from libraries/session/src/main/res/layout/media3_notification_media_action.xml rename to demos/compose/src/main/res/drawable/media3_icon_crop.xml index c06f54d708f..d77f0cbeea8 100644 --- a/libraries/session/src/main/res/layout/media3_notification_media_action.xml +++ b/demos/compose/src/main/res/drawable/media3_icon_crop.xml @@ -1,5 +1,4 @@ - - - - + + + diff --git a/demos/compose/src/main/res/drawable/media3_icon_default_album_image.xml b/demos/compose/src/main/res/drawable/media3_icon_default_album_image.xml new file mode 100644 index 00000000000..a83b5b04566 --- /dev/null +++ b/demos/compose/src/main/res/drawable/media3_icon_default_album_image.xml @@ -0,0 +1,25 @@ + + + + + diff --git a/demos/compose/src/main/res/mipmap-hdpi/ic_launcher.png b/demos/compose/src/main/res/mipmap-hdpi/ic_launcher.png index adaa93220eb..e5a228fb765 100644 Binary files a/demos/compose/src/main/res/mipmap-hdpi/ic_launcher.png and b/demos/compose/src/main/res/mipmap-hdpi/ic_launcher.png differ diff --git a/demos/compose/src/main/res/mipmap-mdpi/ic_launcher.png b/demos/compose/src/main/res/mipmap-mdpi/ic_launcher.png index 9b6f7d5e806..a058cced802 100644 Binary files a/demos/compose/src/main/res/mipmap-mdpi/ic_launcher.png and b/demos/compose/src/main/res/mipmap-mdpi/ic_launcher.png differ diff --git a/demos/compose/src/main/res/mipmap-xhdpi/ic_launcher.png b/demos/compose/src/main/res/mipmap-xhdpi/ic_launcher.png index 2101026c9fe..b2e07ad4ff8 100644 Binary files a/demos/compose/src/main/res/mipmap-xhdpi/ic_launcher.png and b/demos/compose/src/main/res/mipmap-xhdpi/ic_launcher.png differ diff --git a/demos/compose/src/main/res/mipmap-xxhdpi/ic_launcher.png b/demos/compose/src/main/res/mipmap-xxhdpi/ic_launcher.png index 223ec8bd113..c40bc4e6d37 100644 Binary files a/demos/compose/src/main/res/mipmap-xxhdpi/ic_launcher.png and b/demos/compose/src/main/res/mipmap-xxhdpi/ic_launcher.png differ diff --git a/demos/compose/src/main/res/mipmap-xxxhdpi/ic_launcher.png b/demos/compose/src/main/res/mipmap-xxxhdpi/ic_launcher.png index 698ed68c429..b5340fa2881 100644 Binary files a/demos/compose/src/main/res/mipmap-xxxhdpi/ic_launcher.png and b/demos/compose/src/main/res/mipmap-xxxhdpi/ic_launcher.png differ diff --git a/demos/composition/build.gradle.kts b/demos/composition/build.gradle.kts index adb7ca62fe2..2c547c17e82 100644 --- a/demos/composition/build.gradle.kts +++ b/demos/composition/build.gradle.kts @@ -38,17 +38,15 @@ android { } dependencies { - implementation(libs.androidx.appcompat) implementation(libs.material) implementation(project(":lib-effect")) - implementation(project(":lib-effect-ndk")) + implementation(project(":lib-effect-play-services")) implementation(project(":lib-exoplayer")) implementation(project(":lib-exoplayer-dash")) implementation(project(":lib-muxer")) implementation(project(":lib-transformer")) implementation(project(":lib-inspector")) - implementation(project(":lib-ui")) - implementation(project(":lib-ui-compose")) + implementation(project(":lib-ui-compose-material3")) implementation(libs.androidx.annotation) implementation(libs.androidx.lifecycle.viewmodel) implementation(libs.androidx.activity.compose) diff --git a/demos/composition/src/main/java/androidx/media3/demo/composition/CompositionPreviewActivity.kt b/demos/composition/src/main/java/androidx/media3/demo/composition/CompositionPreviewActivity.kt index b0a9e3bb81d..0d08bb5ff34 100644 --- a/demos/composition/src/main/java/androidx/media3/demo/composition/CompositionPreviewActivity.kt +++ b/demos/composition/src/main/java/androidx/media3/demo/composition/CompositionPreviewActivity.kt @@ -20,10 +20,10 @@ import android.content.pm.ActivityInfo import android.content.pm.PackageManager import android.os.Build.VERSION.SDK_INT import android.os.Bundle +import androidx.activity.ComponentActivity import androidx.activity.compose.setContent import androidx.activity.enableEdgeToEdge import androidx.activity.viewModels -import androidx.appcompat.app.AppCompatActivity import androidx.compose.foundation.layout.fillMaxSize import androidx.compose.foundation.layout.fillMaxWidth import androidx.compose.foundation.layout.padding @@ -53,11 +53,10 @@ import androidx.media3.demo.composition.ui.theme.CompositionDemoTheme import androidx.media3.demo.composition.ui.theme.spacing /** - * An Activity ([AppCompatActivity]) that previews compositions, using - * [androidx.media3.transformer.CompositionPlayer]. + * An Activity that previews compositions, using [androidx.media3.transformer.CompositionPlayer]. */ @OptIn(ExperimentalMaterial3Api::class) -class CompositionPreviewActivity : AppCompatActivity() { +class CompositionPreviewActivity : ComponentActivity() { override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) enableEdgeToEdge() diff --git a/demos/composition/src/main/java/androidx/media3/demo/composition/CompositionPreviewViewModel.kt b/demos/composition/src/main/java/androidx/media3/demo/composition/CompositionPreviewViewModel.kt index a784c9bf937..6532dea3d81 100644 --- a/demos/composition/src/main/java/androidx/media3/demo/composition/CompositionPreviewViewModel.kt +++ b/demos/composition/src/main/java/androidx/media3/demo/composition/CompositionPreviewViewModel.kt @@ -20,9 +20,10 @@ import android.net.Uri import android.os.Build.VERSION.SDK_INT import android.os.SystemClock import android.provider.OpenableColumns -import android.view.SurfaceView +import android.util.Rational import androidx.annotation.GuardedBy import androidx.annotation.OptIn +import androidx.annotation.RequiresApi import androidx.compose.runtime.getValue import androidx.compose.runtime.mutableStateOf import androidx.compose.runtime.setValue @@ -45,7 +46,7 @@ import androidx.media3.common.util.Log import androidx.media3.common.util.Size import androidx.media3.common.util.Util import androidx.media3.common.util.Util.usToMs -import androidx.media3.common.video.HardwareBufferFrame +import androidx.media3.common.video.FrameProcessor import androidx.media3.demo.composition.MatrixTransformationFactory.createDizzyCropEffect import androidx.media3.demo.composition.data.CompositionPreviewState import androidx.media3.demo.composition.data.ExportState @@ -56,16 +57,16 @@ import androidx.media3.demo.composition.data.MediaState import androidx.media3.demo.composition.data.OutputSettingsState import androidx.media3.demo.composition.data.Preset import androidx.media3.effect.DebugTraceUtil +import androidx.media3.effect.DefaultGlFrameProcessor +import androidx.media3.effect.HardwareBufferJni import androidx.media3.effect.LanczosResample import androidx.media3.effect.MultipleInputVideoGraph import androidx.media3.effect.Presentation import androidx.media3.effect.RgbFilter -import androidx.media3.effect.SimpleGlFrameProcessor import androidx.media3.effect.StaticOverlaySettings -import androidx.media3.effect.ndk.HardwareBufferJni +import androidx.media3.effect.playservices.EnhancementSessionDecorator import androidx.media3.inspector.MetadataRetriever import androidx.media3.transformer.Composition -import androidx.media3.transformer.CompositionFrameMetadata import androidx.media3.transformer.CompositionPlayer import androidx.media3.transformer.EditedMediaItem import androidx.media3.transformer.EditedMediaItemSequence @@ -76,6 +77,7 @@ import androidx.media3.transformer.InAppFragmentedMp4Muxer import androidx.media3.transformer.InAppMp4Muxer import androidx.media3.transformer.JsonUtil import androidx.media3.transformer.Transformer +import androidx.media3.transformer.VideoFrameAggregationParameters import com.google.common.base.Stopwatch import com.google.common.base.Ticker import java.io.File @@ -109,10 +111,11 @@ class CompositionPreviewViewModel(application: Application) : AndroidViewModel(a val EXPORT_STARTED_MESSAGE = application.resources.getString(R.string.export_started) val FAILED_LOAD_MEDIA_MESSAGE = application.resources.getString(R.string.failed_load_media) val FAILED_GET_DURATION_MESSAGE = application.resources.getString(R.string.failed_get_duration) - val API_33_REQUIRED_MESSAGE = - application.resources.getString(R.string.api_33_required_packet_consumer) - internal var frameConsumerEnabled: Boolean = false - internal var surfaceView: SurfaceView? = null + val API_28_REQUIRED_MESSAGE = + application.resources.getString(R.string.api_28_required_frame_processor) + private val API_33_REQUIRED_GMS_VIDEO_ENHANCEMENT_MESSAGE = + application.resources.getString(R.string.api_33_required_gms_video_enhancement) + internal var frameProcessorEnabled: Boolean = false private var transformer: Transformer? = null private var outputFile: File? = null private var preparedComposition: Composition? = null @@ -149,6 +152,7 @@ class CompositionPreviewViewModel(application: Application) : AndroidViewModel(a hdrMode = Composition.HDR_MODE_KEEP_HDR, audioMimeType = SAME_AS_INPUT_OPTION, videoMimeType = SAME_AS_INPUT_OPTION, + frameAggregationFps = UNSET_OPTION, muxerOption = MUXER_OPTIONS[0], ), exportState = ExportState(), @@ -197,8 +201,7 @@ class CompositionPreviewViewModel(application: Application) : AndroidViewModel(a } override fun onCleared() { - super.onCleared() - releaseAndRecreatePlayer() + releasePlayer() cancelExport() exportStopwatch.reset() } @@ -322,13 +325,54 @@ class CompositionPreviewViewModel(application: Application) : AndroidViewModel(a DebugTraceUtil.enableTracing = enable } - fun onFrameConsumerEnabledChanged(isEnabled: Boolean) { + fun onFrameProcessorEnabledChanged(isEnabled: Boolean) { _uiState.update { it.copy( - outputSettingsState = it.outputSettingsState.copy(frameConsumerEnabled = isEnabled), + outputSettingsState = it.outputSettingsState.copy(frameProcessorEnabled = isEnabled), isCompositionSet = false, ) } + playerPrepared = false + } + + fun onGmsVideoEnhancementEnabledChanged(isEnabled: Boolean) { + _uiState.update { + it.copy( + outputSettingsState = it.outputSettingsState.copy(gmsVideoEnhancementEnabled = isEnabled), + isCompositionSet = false, + ) + } + playerPrepared = false + } + + fun onGmsTonemappingEnabledChanged(isEnabled: Boolean) { + _uiState.update { + it.copy( + outputSettingsState = it.outputSettingsState.copy(gmsTonemappingEnabled = isEnabled), + isCompositionSet = false, + ) + } + playerPrepared = false + } + + fun onGmsDeblurAndDenoiseEnabledChanged(isEnabled: Boolean) { + _uiState.update { + it.copy( + outputSettingsState = it.outputSettingsState.copy(gmsDeblurAndDenoiseEnabled = isEnabled), + isCompositionSet = false, + ) + } + playerPrepared = false + } + + fun onGmsUpscaleVideoEnabledChanged(isEnabled: Boolean) { + _uiState.update { + it.copy( + outputSettingsState = it.outputSettingsState.copy(gmsUpscaleVideoEnabled = isEnabled), + isCompositionSet = false, + ) + } + playerPrepared = false } fun onIncludeBackgroundAudioChanged(isEnabled: Boolean) { @@ -385,6 +429,15 @@ class CompositionPreviewViewModel(application: Application) : AndroidViewModel(a } } + fun onFrameAggregationFpsChanged(fps: String) { + _uiState.update { + it.copy( + outputSettingsState = it.outputSettingsState.copy(frameAggregationFps = fps), + isCompositionSet = false, + ) + } + } + fun onRenderSizeChanged(newSize: geometrySize) { _uiState.update { it.copy( @@ -564,13 +617,19 @@ class CompositionPreviewViewModel(application: Application) : AndroidViewModel(a } fun setComposition() { - val isFrameConsumerEnabled = - frameConsumerEnabled && uiState.value.outputSettingsState.frameConsumerEnabled - // Recreate the player if it isn't prepared or if the FrameProcessor is disabled. - if (!playerPrepared || !isFrameConsumerEnabled) { + val composition = prepareComposition() ?: return + val settings = uiState.value.outputSettingsState + // CompositionPlayer only supports updating the Composition on a prepared player when + // FrameProcessor is enabled. Also recreate the player when GMS Video Enhancement is enabled + // because EnhancementSessionDecorator locks in the input frame format on the first frame. + val canReusePlayer = + playerPrepared && + frameProcessorEnabled && + settings.frameProcessorEnabled && + !settings.gmsVideoEnhancementEnabled + if (!canReusePlayer) { releaseAndRecreatePlayer() } - val composition = prepareComposition() preparedComposition = composition // Maintain the current position when updating the Composition. compositionPlayer.setComposition( @@ -593,6 +652,9 @@ class CompositionPreviewViewModel(application: Application) : AndroidViewModel(a val composition = if (uiState.value.isCompositionSet) preparedComposition ?: prepareComposition() else prepareComposition() + if (composition == null) { + return + } val settings = uiState.value.outputSettingsState try { @@ -613,22 +675,18 @@ class CompositionPreviewViewModel(application: Application) : AndroidViewModel(a val filePath = outputFile!!.absolutePath val transformerBuilder = - if (uiState.value.outputSettingsState.frameConsumerEnabled) { - if (SDK_INT < 33) { - _uiState.update { - it.copy(snackbarMessage = API_33_REQUIRED_MESSAGE, isCompositionSet = false) - } - return + if (settings.frameProcessorEnabled) { + if (SDK_INT < 28) { + throw UnsupportedOperationException(API_28_REQUIRED_MESSAGE) + } + if (settings.gmsVideoEnhancementEnabled) { + // Google Play services only allows one active EnhancementSession at a time + // (MAX_SESSIONS_REACHED). Release any prepared preview player before starting export. + releaseAndRecreatePlayer() + _uiState.update { it.copy(isCompositionSet = false) } } Transformer.Builder(getApplication()) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setFrameProcessorFactory( - SimpleGlFrameProcessor.Factory( - getApplication(), - hardwareBufferJniWrapper = HardwareBufferJni.INSTANCE, - overlaySettingsProvider = CompositionPreviewViewModel::getOverlaySettings, - ) - ) + .setFrameProcessorFactory(createFrameProcessorFactory()) } else { Transformer.Builder(/* context= */ getApplication()) } @@ -742,7 +800,7 @@ class CompositionPreviewViewModel(application: Application) : AndroidViewModel(a _uiState.update { it.copy(exportState = ExportState()) } } - private fun prepareComposition(): Composition { + private fun prepareComposition(): Composition? { val settings = uiState.value.outputSettingsState var compositionHasVideo = false @@ -833,6 +891,30 @@ class CompositionPreviewViewModel(application: Application) : AndroidViewModel(a } val compositionBuilder = Composition.Builder(sequences).setHdrMode(settings.hdrMode) + if (settings.frameProcessorEnabled) { + val frameAggregationFpsStr = settings.frameAggregationFps.trim() + if (frameAggregationFpsStr.isNotEmpty()) { + val frameAggregationFps = frameAggregationFpsStr.toDoubleOrNull() + val numerator = frameAggregationFps?.let { (it * 1000).roundToInt() } + if (numerator != null && numerator > 0) { + compositionBuilder.setVideoFrameAggregationParameters( + VideoFrameAggregationParameters.Builder() + .setFrameRate(Rational(numerator, 1000)) + .build() + ) + } else { + _uiState.update { + it.copy( + snackbarMessage = + getApplication() + .resources + .getString(R.string.error_invalid_frame_aggregation_fps) + ) + } + return null + } + } + } if (compositionHasVideo) { compositionBuilder @@ -861,8 +943,14 @@ class CompositionPreviewViewModel(application: Application) : AndroidViewModel(a val totalVideoSlots = videoEnabledUIIndices.size if (totalVideoSlots >= 3) { return object : VideoCompositorSettings { + var outputSize: Size? = null + override fun getOutputSize(inputSizes: List): Size { - return inputSizes[0] + // Use a constant compositing size to avoid sudden layout shifts. + if (outputSize == null) { + outputSize = inputSizes[0] + } + return outputSize!! } override fun getOverlaySettings(inputId: Int, presentationTimeUs: Long): OverlaySettings { @@ -922,18 +1010,14 @@ class CompositionPreviewViewModel(application: Application) : AndroidViewModel(a private fun createCompositionPlayer(): CompositionPlayer { val playerBuilder: CompositionPlayer.Builder - frameConsumerEnabled = uiState.value.outputSettingsState.frameConsumerEnabled - if (uiState.value.outputSettingsState.frameConsumerEnabled && SDK_INT >= 28) { + frameProcessorEnabled = uiState.value.outputSettingsState.frameProcessorEnabled + if (uiState.value.outputSettingsState.frameProcessorEnabled) { + if (SDK_INT < 28) { + throw UnsupportedOperationException(API_28_REQUIRED_MESSAGE) + } playerBuilder = CompositionPlayer.Builder(getApplication()) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setFrameProcessorFactory( - SimpleGlFrameProcessor.Factory( - getApplication(), - hardwareBufferJniWrapper = HardwareBufferJni.INSTANCE, - overlaySettingsProvider = CompositionPreviewViewModel::getOverlaySettings, - ) - ) + .setFrameProcessorFactory(createFrameProcessorFactory()) } else { playerBuilder = CompositionPlayer.Builder(getApplication()) if (uiState.value.sequenceTrackTypes.size > 1) { @@ -984,9 +1068,35 @@ class CompositionPreviewViewModel(application: Application) : AndroidViewModel(a return player } - private fun releaseAndRecreatePlayer() { + @RequiresApi(28) + private fun createFrameProcessorFactory(): FrameProcessor.Factory { + val baseGlFactory = + DefaultGlFrameProcessor.Factory.Builder( + /* context= */ getApplication(), + /* hardwareBufferJniWrapper= */ HardwareBufferJni.INSTANCE, + ) + .build() + val settings = uiState.value.outputSettingsState + if (!settings.gmsVideoEnhancementEnabled) { + return baseGlFactory + } + if (SDK_INT < 33) { + throw UnsupportedOperationException(API_33_REQUIRED_GMS_VIDEO_ENHANCEMENT_MESSAGE) + } + return EnhancementSessionDecorator.Builder(getApplication(), baseGlFactory) + .setTonemappingEnabled(settings.gmsTonemappingEnabled) + .setDeblurAndDenoiseVideoEnabled(settings.gmsDeblurAndDenoiseEnabled) + .setUpscaleVideoEnabled(settings.gmsUpscaleVideoEnabled) + .build() + } + + private fun releasePlayer() { compositionPlayer.stop() compositionPlayer.release() + } + + private fun releaseAndRecreatePlayer() { + releasePlayer() compositionPlayer = createCompositionPlayer() playerPrepared = false } @@ -1010,6 +1120,7 @@ class CompositionPreviewViewModel(application: Application) : AndroidViewModel(a } companion object { + const val UNSET_OPTION = "" const val SAME_AS_INPUT_OPTION = "same as input" private const val TAG = "CompPreviewVM" private const val AUDIO_URI = "https://storage.googleapis.com/exoplayer-test-media-0/play.mp3" @@ -1036,6 +1147,22 @@ class CompositionPreviewViewModel(application: Application) : AndroidViewModel(a listOf(SAME_AS_INPUT_OPTION, "144", "240", "360", "480", "720", "1080", "1440", "2160") val MUXER_OPTIONS = listOf("Use Platform MediaMuxer", "Use Media3 Mp4Muxer", "Use Media3 FragmentedMp4Muxer") + val FRAME_AGGREGATION_FPS_OPTIONS = + listOf( + UNSET_OPTION, + "0.5", + "1", + "15", + "23.976", + "24", + "25", + "29.97", + "30", + "48", + "50", + "59.94", + "60", + ) fun getAudioBackgroundSequence(): EditedMediaItemSequence { val audioMediaItem: MediaItem = MediaItem.Builder().setUri(AUDIO_URI).build() @@ -1045,15 +1172,5 @@ class CompositionPreviewViewModel(application: Application) : AndroidViewModel(a .setIsLooping(true) .build() } - - fun getOverlaySettings(frame: HardwareBufferFrame): OverlaySettings { - val metadata = - frame.metadata[CompositionFrameMetadata.KEY_COMPOSITION_FRAME_METADATA] - as CompositionFrameMetadata - return metadata.composition.videoCompositorSettings.getOverlaySettings( - metadata.sequenceIndex, - frame.contentTimeUs, - ) - } } } diff --git a/demos/composition/src/main/java/androidx/media3/demo/composition/data/CompositionState.kt b/demos/composition/src/main/java/androidx/media3/demo/composition/data/CompositionState.kt index 68eccd9749a..2bc2954a436 100644 --- a/demos/composition/src/main/java/androidx/media3/demo/composition/data/CompositionState.kt +++ b/demos/composition/src/main/java/androidx/media3/demo/composition/data/CompositionState.kt @@ -122,12 +122,17 @@ data class Gap(override val durationUs: Long) : Item { */ @OptIn(UnstableApi::class) data class OutputSettingsState( - val frameConsumerEnabled: Boolean = false, + val frameProcessorEnabled: Boolean = false, + val gmsVideoEnhancementEnabled: Boolean = false, + val gmsTonemappingEnabled: Boolean = true, + val gmsDeblurAndDenoiseEnabled: Boolean = false, + val gmsUpscaleVideoEnabled: Boolean = false, val includeBackgroundAudio: Boolean = false, val resolutionHeight: String, val hdrMode: Int, val audioMimeType: String, val videoMimeType: String, + val frameAggregationFps: String, val muxerOption: String, val renderSize: Size = Size.Zero, ) diff --git a/demos/composition/src/main/java/androidx/media3/demo/composition/ui/CompositionPreviewPane.kt b/demos/composition/src/main/java/androidx/media3/demo/composition/ui/CompositionPreviewPane.kt index 338ea46a02e..2d1bf975c30 100644 --- a/demos/composition/src/main/java/androidx/media3/demo/composition/ui/CompositionPreviewPane.kt +++ b/demos/composition/src/main/java/androidx/media3/demo/composition/ui/CompositionPreviewPane.kt @@ -15,11 +15,12 @@ */ package androidx.media3.demo.composition.ui -import android.view.SurfaceView +import android.os.Build.VERSION.SDK_INT import androidx.annotation.OptIn import androidx.compose.foundation.background +import androidx.compose.foundation.gestures.detectTapGestures import androidx.compose.foundation.layout.Arrangement -import androidx.compose.foundation.layout.BoxWithConstraints +import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.fillMaxSize @@ -38,18 +39,22 @@ import androidx.compose.material3.Switch import androidx.compose.material3.Tab import androidx.compose.material3.Text import androidx.compose.runtime.Composable +import androidx.compose.runtime.LaunchedEffect import androidx.compose.runtime.getValue import androidx.compose.runtime.mutableStateOf import androidx.compose.runtime.remember +import androidx.compose.runtime.rememberCoroutineScope +import androidx.compose.runtime.saveable.rememberSaveable import androidx.compose.runtime.setValue import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier import androidx.compose.ui.graphics.Color +import androidx.compose.ui.input.pointer.PointerEventType +import androidx.compose.ui.input.pointer.pointerInput import androidx.compose.ui.res.painterResource import androidx.compose.ui.res.stringResource import androidx.compose.ui.text.font.FontWeight import androidx.compose.ui.unit.dp -import androidx.compose.ui.viewinterop.AndroidView import androidx.media3.common.util.UnstableApi import androidx.media3.demo.composition.CompositionPreviewViewModel import androidx.media3.demo.composition.R @@ -57,7 +62,14 @@ import androidx.media3.demo.composition.data.CompositionPreviewState import androidx.media3.demo.composition.data.Preset import androidx.media3.demo.composition.ui.theme.spacing import androidx.media3.demo.composition.ui.theme.textPadding -import androidx.media3.ui.PlayerView +import androidx.media3.ui.compose.material3.Player +import kotlinx.coroutines.Job +import kotlinx.coroutines.delay +import kotlinx.coroutines.launch + +private const val CONTROLS_VISIBILITY_TIMEOUT_MS = 3000L + +private class JobHolder(var job: Job? = null) @OptIn(UnstableApi::class) @Composable @@ -67,9 +79,32 @@ internal fun CompositionPreviewPane( uiState: CompositionPreviewState, modifier: Modifier = Modifier, ) { + val scope = rememberCoroutineScope() val scrollState = rememberScrollState() var isLayoutDropdownExpanded by remember { mutableStateOf(false) } + var showControls by rememberSaveable { mutableStateOf(true) } + var anyPointerDown by remember { mutableStateOf(false) } + val hideJobHolder = remember { JobHolder() } + + fun scheduleHideControls() { + hideJobHolder.job?.cancel() + if (!anyPointerDown) { + hideJobHolder.job = scope.launch { + delay(CONTROLS_VISIBILITY_TIMEOUT_MS) + showControls = false + } + } + } + + LaunchedEffect(showControls, anyPointerDown) { + if (showControls && !anyPointerDown) { + scheduleHideControls() + } else { + hideJobHolder.job?.cancel() + } + } + Column(modifier = modifier.fillMaxSize()) { Text( text = @@ -77,22 +112,24 @@ internal fun CompositionPreviewPane( fontWeight = FontWeight.Bold, ) - @Suppress("UnusedBoxWithConstraintsScope") - BoxWithConstraints( + Box( modifier = Modifier.fillMaxWidth().heightIn(max = 250.dp), contentAlignment = Alignment.Center, ) { // Video Player - AndroidView( - factory = { context -> PlayerView(context) }, - update = { playerView -> - playerView.player = viewModel.compositionPlayer - playerView.setTimeBarScrubbingEnabled(true) - playerView.setUseController(true) - // TODO: b/449957627 - Remove once internal pipeline is migrated to FrameConsumer. - viewModel.surfaceView = playerView.videoSurfaceView as SurfaceView - }, - modifier = Modifier.fillMaxSize(), + Player( + player = viewModel.compositionPlayer, + showControls = showControls, + modifier = + Modifier.fillMaxSize() + .playerGestures( + onPointerDownChange = { anyPointerDown = it }, + onPointerMove = { + showControls = true + scheduleHideControls() + }, + onToggleControls = { showControls = !showControls }, + ), ) // FPS Tracker Overlay @@ -191,16 +228,75 @@ internal fun CompositionPreviewPane( horizontalArrangement = Arrangement.SpaceBetween, modifier = Modifier.fillMaxWidth(), ) { - Text( - text = stringResource(R.string.frame_consumer_enabled), - modifier = Modifier.textPadding(), - ) + Column { + Text( + text = stringResource(R.string.frame_processor_enabled), + modifier = Modifier.textPadding(), + ) + if (SDK_INT < 28) { + Text( + text = stringResource(R.string.api_28_required_frame_processor), + style = MaterialTheme.typography.labelSmall, + color = MaterialTheme.colorScheme.outline, + modifier = Modifier.textPadding(), + ) + } + } Switch( - checked = uiState.outputSettingsState.frameConsumerEnabled, - onCheckedChange = { isEnabled -> viewModel.onFrameConsumerEnabledChanged(isEnabled) }, + checked = uiState.outputSettingsState.frameProcessorEnabled, + onCheckedChange = { isEnabled -> viewModel.onFrameProcessorEnabledChanged(isEnabled) }, + enabled = SDK_INT >= 28, ) } + if (uiState.outputSettingsState.frameProcessorEnabled) { + Row( + verticalAlignment = Alignment.CenterVertically, + horizontalArrangement = Arrangement.SpaceBetween, + modifier = Modifier.fillMaxWidth(), + ) { + Column { + Text( + text = stringResource(R.string.enable_gms_video_enhancement), + modifier = Modifier.textPadding(), + ) + if (SDK_INT < 33) { + Text( + text = stringResource(R.string.api_33_required_gms_video_enhancement), + style = MaterialTheme.typography.labelSmall, + color = MaterialTheme.colorScheme.outline, + modifier = Modifier.textPadding(), + ) + } + } + Switch( + checked = uiState.outputSettingsState.gmsVideoEnhancementEnabled, + onCheckedChange = { isEnabled -> + viewModel.onGmsVideoEnhancementEnabledChanged(isEnabled) + }, + enabled = SDK_INT >= 33, + ) + } + + if (uiState.outputSettingsState.gmsVideoEnhancementEnabled && SDK_INT >= 33) { + IndentedSwitchRow( + text = stringResource(R.string.enable_tonemapping), + checked = uiState.outputSettingsState.gmsTonemappingEnabled, + onCheckedChange = viewModel::onGmsTonemappingEnabledChanged, + ) + IndentedSwitchRow( + text = stringResource(R.string.enable_deblur_and_denoise), + checked = uiState.outputSettingsState.gmsDeblurAndDenoiseEnabled, + onCheckedChange = viewModel::onGmsDeblurAndDenoiseEnabledChanged, + ) + IndentedSwitchRow( + text = stringResource(R.string.enable_upscale), + checked = uiState.outputSettingsState.gmsUpscaleVideoEnabled, + onCheckedChange = viewModel::onGmsUpscaleVideoEnabledChanged, + ) + } + } + Row( verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.SpaceBetween, @@ -220,6 +316,7 @@ internal fun CompositionPreviewPane( outputSettings = uiState.outputSettingsState, onResolutionChanged = viewModel::onOutputResolutionChanged, onHdrModeChanged = viewModel::onHdrModeChanged, + onFrameAggregationFpsChanged = viewModel::onFrameAggregationFpsChanged, ) } @@ -245,6 +342,18 @@ internal fun CompositionPreviewPane( } } +@Composable +private fun IndentedSwitchRow(text: String, checked: Boolean, onCheckedChange: (Boolean) -> Unit) { + Row( + verticalAlignment = Alignment.CenterVertically, + horizontalArrangement = Arrangement.SpaceBetween, + modifier = Modifier.fillMaxWidth().padding(start = MaterialTheme.spacing.standard), + ) { + Text(text = text, modifier = Modifier.textPadding()) + Switch(checked = checked, onCheckedChange = onCheckedChange) + } +} + @Composable private fun presetToString(preset: Preset): String { return when (preset) { @@ -254,3 +363,22 @@ private fun presetToString(preset: Preset): String { Preset.CUSTOM -> stringResource(R.string.preset_custom) } } + +private fun Modifier.playerGestures( + onPointerDownChange: (Boolean) -> Unit, + onPointerMove: () -> Unit, + onToggleControls: () -> Unit, +): Modifier = + this.pointerInput(Unit) { + awaitPointerEventScope { + while (true) { + val event = awaitPointerEvent() + val isAnyPressed = event.changes.any { it.pressed } + onPointerDownChange(isAnyPressed) + if (event.type == PointerEventType.Move) { + onPointerMove() + } + } + } + } + .pointerInput(Unit) { detectTapGestures(onTap = { onToggleControls() }) } diff --git a/demos/composition/src/main/java/androidx/media3/demo/composition/ui/OutputSettings.kt b/demos/composition/src/main/java/androidx/media3/demo/composition/ui/OutputSettings.kt index 227af443810..a5bfd3606ee 100644 --- a/demos/composition/src/main/java/androidx/media3/demo/composition/ui/OutputSettings.kt +++ b/demos/composition/src/main/java/androidx/media3/demo/composition/ui/OutputSettings.kt @@ -20,6 +20,7 @@ import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.text.KeyboardOptions import androidx.compose.material3.MaterialTheme import androidx.compose.material3.Text import androidx.compose.runtime.Composable @@ -30,9 +31,12 @@ import androidx.compose.runtime.setValue import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier import androidx.compose.ui.res.stringResource +import androidx.compose.ui.text.input.KeyboardType import androidx.media3.common.util.UnstableApi +import androidx.media3.demo.composition.CompositionPreviewViewModel.Companion.FRAME_AGGREGATION_FPS_OPTIONS import androidx.media3.demo.composition.CompositionPreviewViewModel.Companion.HDR_MODE_DESCRIPTIONS import androidx.media3.demo.composition.CompositionPreviewViewModel.Companion.RESOLUTION_HEIGHTS +import androidx.media3.demo.composition.CompositionPreviewViewModel.Companion.UNSET_OPTION import androidx.media3.demo.composition.R import androidx.media3.demo.composition.data.OutputSettingsState import androidx.media3.demo.composition.ui.theme.spacing @@ -45,9 +49,11 @@ internal fun OutputSettings( outputSettings: OutputSettingsState, onResolutionChanged: (String) -> Unit, onHdrModeChanged: (Int) -> Unit, + onFrameAggregationFpsChanged: (String) -> Unit, ) { var resolutionExpanded by remember { mutableStateOf(false) } var hdrExpanded by remember { mutableStateOf(false) } + var fpsExpanded by remember { mutableStateOf(false) } val selectedHdrKey = HDR_MODE_DESCRIPTIONS.entries.find { it.value == outputSettings.hdrMode }?.key @@ -86,5 +92,27 @@ internal fun OutputSettings( }, ) } + if (outputSettings.frameProcessorEnabled) { + Row( + verticalAlignment = Alignment.CenterVertically, + horizontalArrangement = Arrangement.SpaceBetween, + modifier = Modifier.fillMaxWidth(), + ) { + Text( + text = stringResource(R.string.frame_aggregation_fps), + modifier = Modifier.textPadding(), + ) + EditableDropDownSpinner( + isDropDownOpen = fpsExpanded, + currentValue = outputSettings.frameAggregationFps, + dropDownOptions = FRAME_AGGREGATION_FPS_OPTIONS, + changeDropDownOpen = { fpsExpanded = it }, + onValueChange = onFrameAggregationFpsChanged, + labelProvider = { if (it == UNSET_OPTION) stringResource(R.string.unset) else it }, + keyboardOptions = KeyboardOptions(keyboardType = KeyboardType.Decimal), + placeholder = { Text(stringResource(R.string.unset)) }, + ) + } + } } } diff --git a/demos/composition/src/main/java/androidx/media3/demo/composition/ui/UiComponents.kt b/demos/composition/src/main/java/androidx/media3/demo/composition/ui/UiComponents.kt index 6b4be06b3fe..8f5defc60a4 100644 --- a/demos/composition/src/main/java/androidx/media3/demo/composition/ui/UiComponents.kt +++ b/demos/composition/src/main/java/androidx/media3/demo/composition/ui/UiComponents.kt @@ -21,15 +21,19 @@ import androidx.compose.foundation.gestures.waitForUpOrCancellation import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.fillMaxWidth +import androidx.compose.foundation.text.KeyboardOptions import androidx.compose.material3.DropdownMenu import androidx.compose.material3.DropdownMenuItem import androidx.compose.material3.Icon +import androidx.compose.material3.IconButton import androidx.compose.material3.OutlinedTextField import androidx.compose.material3.Text import androidx.compose.runtime.Composable import androidx.compose.ui.Modifier import androidx.compose.ui.input.pointer.PointerEventPass import androidx.compose.ui.input.pointer.pointerInput +import androidx.compose.ui.platform.LocalFocusManager +import androidx.compose.ui.platform.LocalSoftwareKeyboardController import androidx.compose.ui.res.painterResource import androidx.media3.demo.composition.R @@ -75,3 +79,49 @@ fun DropDownSpinner( } } } + +@Composable +fun EditableDropDownSpinner( + isDropDownOpen: Boolean, + currentValue: String, + dropDownOptions: List, + changeDropDownOpen: (Boolean) -> Unit, + onValueChange: (String) -> Unit, + modifier: Modifier = Modifier, + labelProvider: @Composable (String) -> String = { it }, + keyboardOptions: KeyboardOptions = KeyboardOptions.Default, + placeholder: @Composable (() -> Unit)? = null, +) { + val keyboardController = LocalSoftwareKeyboardController.current + val focusManager = LocalFocusManager.current + Column(modifier = modifier) { + Box { + OutlinedTextField( + value = currentValue, + onValueChange = onValueChange, + trailingIcon = { + IconButton(onClick = { changeDropDownOpen(!isDropDownOpen) }) { + Icon(painterResource(R.drawable.arrow_drop_down), null) + } + }, + modifier = Modifier.fillMaxWidth(), + keyboardOptions = keyboardOptions, + singleLine = true, + placeholder = placeholder, + ) + DropdownMenu(expanded = isDropDownOpen, onDismissRequest = { changeDropDownOpen(false) }) { + dropDownOptions.forEach { option -> + DropdownMenuItem( + text = { Text(text = labelProvider(option)) }, + onClick = { + changeDropDownOpen(false) + onValueChange(option) + keyboardController?.hide() + focusManager.clearFocus() + }, + ) + } + } + } + } +} diff --git a/demos/composition/src/main/java/androidx/media3/demo/composition/ui/VideoSequenceDialog.kt b/demos/composition/src/main/java/androidx/media3/demo/composition/ui/VideoSequenceDialog.kt index 016fe797ef0..9bef41f6b41 100644 --- a/demos/composition/src/main/java/androidx/media3/demo/composition/ui/VideoSequenceDialog.kt +++ b/demos/composition/src/main/java/androidx/media3/demo/composition/ui/VideoSequenceDialog.kt @@ -15,7 +15,6 @@ */ package androidx.media3.demo.composition.ui -import android.os.Build.VERSION.SDK_INT import android.os.LocaleList import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Column @@ -94,7 +93,7 @@ internal fun VideoSequenceDialog( val duration = item.durationUs.toDuration(DurationUnit.MICROSECONDS) val durationString = String.format( - getLocaleWithSdk(), + getDefaultLocale(), "%02d:%02d", duration.inWholeMinutes, duration.inWholeSeconds % 60, @@ -114,10 +113,6 @@ internal fun VideoSequenceDialog( } } -private fun getLocaleWithSdk(): Locale { - return if (SDK_INT >= 24) { - LocaleList.getDefault().get(0) ?: Locale.getDefault() - } else { - Locale.getDefault() - } +private fun getDefaultLocale(): Locale { + return LocaleList.getDefault().get(0) ?: Locale.getDefault() } diff --git a/demos/composition/src/main/res/mipmap-hdpi/ic_launcher.png b/demos/composition/src/main/res/mipmap-hdpi/ic_launcher.png index adaa93220eb..e5a228fb765 100644 Binary files a/demos/composition/src/main/res/mipmap-hdpi/ic_launcher.png and b/demos/composition/src/main/res/mipmap-hdpi/ic_launcher.png differ diff --git a/demos/composition/src/main/res/mipmap-mdpi/ic_launcher.png b/demos/composition/src/main/res/mipmap-mdpi/ic_launcher.png index 9b6f7d5e806..a058cced802 100644 Binary files a/demos/composition/src/main/res/mipmap-mdpi/ic_launcher.png and b/demos/composition/src/main/res/mipmap-mdpi/ic_launcher.png differ diff --git a/demos/composition/src/main/res/mipmap-xhdpi/ic_launcher.png b/demos/composition/src/main/res/mipmap-xhdpi/ic_launcher.png index 2101026c9fe..b2e07ad4ff8 100644 Binary files a/demos/composition/src/main/res/mipmap-xhdpi/ic_launcher.png and b/demos/composition/src/main/res/mipmap-xhdpi/ic_launcher.png differ diff --git a/demos/composition/src/main/res/mipmap-xxhdpi/ic_launcher.png b/demos/composition/src/main/res/mipmap-xxhdpi/ic_launcher.png index 223ec8bd113..c40bc4e6d37 100644 Binary files a/demos/composition/src/main/res/mipmap-xxhdpi/ic_launcher.png and b/demos/composition/src/main/res/mipmap-xxhdpi/ic_launcher.png differ diff --git a/demos/composition/src/main/res/mipmap-xxxhdpi/ic_launcher.png b/demos/composition/src/main/res/mipmap-xxxhdpi/ic_launcher.png index 698ed68c429..b5340fa2881 100644 Binary files a/demos/composition/src/main/res/mipmap-xxxhdpi/ic_launcher.png and b/demos/composition/src/main/res/mipmap-xxxhdpi/ic_launcher.png differ diff --git a/demos/composition/src/main/res/values/strings.xml b/demos/composition/src/main/res/values/strings.xml index ccff162458c..2ecc4a7b0c5 100644 --- a/demos/composition/src/main/res/values/strings.xml +++ b/demos/composition/src/main/res/values/strings.xml @@ -27,7 +27,11 @@ Export completed in %.3f seconds.\nOutput: %s Export error Export started - Use FrameConsumer + Use FrameProcessor + GMS Video Enhancement + Tone Mapping + Deblur and Denoise + Upscale Add background audio Output video resolution HDR mode @@ -35,12 +39,15 @@ Select Select Effects Cancel + Unset + Invalid frame aggregation FPS Export Output audio MIME type Output video MIME type Enable debug tracing Use Media3 Mp4Muxer Use Media3 FragmentedMp4Muxer + Frame Aggregation FPS Choose a sequence layout preset MainActivity Grayscale @@ -62,7 +69,8 @@ Failed to load media item. Failed to get duration. Aborting export! Unable to create output file: %s - API 33+ required to export with PacketConsumer + API 28+ required for using FrameProcessor + API 33+ required for using GMS Video Enhancement Export error: %s Preview error: %s Sequential diff --git a/demos/effect/build.gradle.kts b/demos/effect/build.gradle.kts index a532bac691f..cd653d7c36b 100644 --- a/demos/effect/build.gradle.kts +++ b/demos/effect/build.gradle.kts @@ -20,6 +20,13 @@ plugins { android { namespace = "androidx.media3.demo.effect" + // TODO: b/502167525 - Remove this temporary JVM 11 override once commonConfig.kt + // is migrated to Java 11 and the alpha28 dependency is reverted back to the BOM. + compileOptions { + sourceCompatibility = JavaVersion.VERSION_11 + targetCompatibility = JavaVersion.VERSION_11 + } + buildTypes { getByName("release") { isShrinkResources = true @@ -35,15 +42,14 @@ android { } dependencies { - implementation(platform(libs.androidx.compose.bom)) - implementation(libs.androidx.compose.foundation) - implementation(libs.androidx.compose.material3) - implementation(libs.androidx.activity.compose) implementation(libs.material) implementation(project(":lib-exoplayer")) - implementation(project(":lib-ui")) + implementation(project(":lib-ui-compose-material3")) + // TODO(b/555710150): Revert this once our Gradle BOM is updated to a stable version that + // includes the M3 API changes (ExposedDropdownMenu). + implementation("androidx.compose.material3:material3:1.5.0-alpha28") implementation(project(":lib-effect")) implementation(project(":lib-effect-lottie")) diff --git a/demos/effect/src/main/java/androidx/media3/demo/effect/EffectActivity.kt b/demos/effect/src/main/java/androidx/media3/demo/effect/EffectActivity.kt index 18170e294c8..e08b0f8f501 100644 --- a/demos/effect/src/main/java/androidx/media3/demo/effect/EffectActivity.kt +++ b/demos/effect/src/main/java/androidx/media3/demo/effect/EffectActivity.kt @@ -16,17 +16,17 @@ package androidx.media3.demo.effect import android.os.Bundle -import android.text.Spannable -import android.text.SpannableString -import android.text.style.ForegroundColorSpan import androidx.activity.ComponentActivity import androidx.activity.compose.setContent +import androidx.activity.viewModels import androidx.annotation.OptIn import androidx.compose.animation.animateContentSize import androidx.compose.animation.core.LinearEasing import androidx.compose.animation.core.tween +import androidx.compose.foundation.background import androidx.compose.foundation.clickable import androidx.compose.foundation.layout.Arrangement +import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.fillMaxSize @@ -34,71 +34,68 @@ import androidx.compose.foundation.layout.fillMaxWidth import androidx.compose.foundation.layout.height import androidx.compose.foundation.layout.padding import androidx.compose.foundation.lazy.LazyColumn +import androidx.compose.foundation.shape.RoundedCornerShape import androidx.compose.material3.Button import androidx.compose.material3.Card import androidx.compose.material3.Checkbox +import androidx.compose.material3.CircularProgressIndicator import androidx.compose.material3.MaterialTheme import androidx.compose.material3.OutlinedTextField import androidx.compose.material3.Scaffold import androidx.compose.material3.Slider +import androidx.compose.material3.SliderState import androidx.compose.material3.SnackbarHost import androidx.compose.material3.SnackbarHostState import androidx.compose.material3.Text import androidx.compose.runtime.Composable +import androidx.compose.runtime.LaunchedEffect +import androidx.compose.runtime.collectAsState import androidx.compose.runtime.getValue import androidx.compose.runtime.mutableStateOf import androidx.compose.runtime.remember import androidx.compose.runtime.rememberCoroutineScope -import androidx.compose.runtime.saveable.rememberSaveable import androidx.compose.runtime.setValue import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier +import androidx.compose.ui.draw.clip import androidx.compose.ui.graphics.Color -import androidx.compose.ui.graphics.toArgb -import androidx.compose.ui.platform.LocalContext +import androidx.compose.ui.input.pointer.PointerEventPass +import androidx.compose.ui.input.pointer.pointerInput import androidx.compose.ui.res.dimensionResource import androidx.compose.ui.res.stringResource -import androidx.compose.ui.viewinterop.AndroidView -import androidx.lifecycle.lifecycleScope -import androidx.media3.common.Effect -import androidx.media3.common.util.UnstableApi -import androidx.media3.demo.effect.ui.COLORS +import androidx.compose.ui.unit.dp +import androidx.media3.common.Player +import androidx.media3.common.util.ExperimentalApi import androidx.media3.demo.effect.ui.ColorsDropDownMenu -import androidx.media3.demo.effect.ui.DropdownControlItem +import androidx.media3.demo.effect.ui.GenericExposedDropdownMenu import androidx.media3.demo.effect.ui.InputSelector -import androidx.media3.effect.Contrast -import androidx.media3.effect.OverlayEffect -import androidx.media3.effect.StaticOverlaySettings -import androidx.media3.effect.TextOverlay -import androidx.media3.effect.TextureOverlay -import androidx.media3.exoplayer.ExoPlayer -import androidx.media3.ui.PlayerView -import com.google.common.collect.ImmutableList +import androidx.media3.ui.compose.material3.Player import java.util.Locale +import kotlinx.coroutines.delay import kotlinx.coroutines.launch class EffectActivity : ComponentActivity() { + private val viewModel: EffectViewModel by viewModels() + override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) - val playlistHolderList = mutableStateOf>(emptyList()) - lifecycleScope.launch { - playlistHolderList.value = - loadPlaylistsFromJson(JSON_FILENAME, this@EffectActivity, "EffectActivity") - } - setContent { EffectDemo(playlistHolderList.value) } + setContent { EffectDemo(viewModel) } } - @OptIn(UnstableApi::class) @Composable - private fun EffectDemo(playlistHolderList: List) { + private fun EffectDemo(viewModel: EffectViewModel) { val snackbarHostState = remember { SnackbarHostState() } val coroutineScope = rememberCoroutineScope() - val context = LocalContext.current - val exoPlayer by remember { - mutableStateOf(ExoPlayer.Builder(context).build().apply { playWhenReady = true }) + val playlistHolderList by viewModel.playlistHolderList.collectAsState() + val uiState by viewModel.uiState.collectAsState() + + LaunchedEffect(uiState.errorMessage) { + uiState.errorMessage?.let { message -> + snackbarHostState.showSnackbar(message) + viewModel.clearErrorMessage() + } } - var effectsEnabled by remember { mutableStateOf(false) } Scaffold( modifier = Modifier.fillMaxSize(), @@ -115,135 +112,104 @@ class EffectActivity : ComponentActivity() { coroutineScope.launch { snackbarHostState.showSnackbar(message) } }, ) { mediaItems -> - effectsEnabled = true - exoPlayer.apply { - setMediaItems(mediaItems) - setVideoEffects(emptyList()) - prepare() - } + viewModel.selectMediaItems(mediaItems) } - PlayerScreen(exoPlayer) - EffectControls( - effectsEnabled, - onApplyEffectsClicked = { videoEffects -> - exoPlayer.apply { - setVideoEffects(videoEffects) - prepare() - } - }, - ) + PlayerScreen(viewModel.exoPlayer) + EffectControls(viewModel, uiState) } } } + @OptIn(ExperimentalApi::class) @Composable - private fun PlayerScreen(exoPlayer: ExoPlayer) { - val context = LocalContext.current - AndroidView( - factory = { PlayerView(context).apply { player = exoPlayer } }, - modifier = - Modifier.height(dimensionResource(id = R.dimen.android_view_height)) - .padding(all = dimensionResource(id = R.dimen.regular_padding)), - ) - } + private fun PlayerScreen(player: Player?) { + var showControls by remember { mutableStateOf(true) } + var interactionCount by remember { mutableStateOf(0) } - @OptIn(UnstableApi::class) - @Composable - private fun EffectControls(enabled: Boolean, onApplyEffectsClicked: (List) -> Unit) { - val initialLottieOverlayName = stringResource(R.string.lottie_effect_name_counter) - var effectControlsState by remember { - mutableStateOf(EffectControlsState(lottieOverlayName = initialLottieOverlayName)) + fun resetControlsTimer() { + interactionCount++ } - Button( - enabled = enabled && effectControlsState.effectsChanged, - onClick = { - val effectsList = mutableListOf() - - if (effectControlsState.contrastValue != 0f) { - effectsList += Contrast(effectControlsState.contrastValue) - } - - val overlaysBuilder = ImmutableList.builder() - if (effectControlsState.confettiOverlayChecked) { - overlaysBuilder.add(ConfettiOverlay()) - } - - if (effectControlsState.clockOverlayChecked) { - overlaysBuilder.add(ClockOverlay()) - } - - if (effectControlsState.lottieOverlayChecked) { - val lottieEffect = lottieOverlayOptions[effectControlsState.lottieOverlayName] - lottieEffect?.let { effectsList += lottieEffect } - } + LaunchedEffect(interactionCount, player) { + if (player != null) { + showControls = true + delay(CONTROLS_VISIBILITY_TIMEOUT_MS) + showControls = false + } + } - val textOverlayText = effectControlsState.textOverlayText - if (effectControlsState.textOverlayChecked && textOverlayText != null) { - val spannableOverlayText = SpannableString(textOverlayText) - spannableOverlayText.setSpan( - ForegroundColorSpan(effectControlsState.textOverlayColor.toArgb()), - /* start= */ 0, - textOverlayText.length, - Spannable.SPAN_EXCLUSIVE_EXCLUSIVE, - ) - val staticOverlaySettings = - StaticOverlaySettings.Builder() - .setAlphaScale(effectControlsState.textOverlayAlpha) - .build() - overlaysBuilder.add( - TextOverlay.createStaticTextOverlay(spannableOverlayText, staticOverlaySettings) - ) - } - effectsList += OverlayEffect(overlaysBuilder.build()) + Box( + modifier = + Modifier.fillMaxWidth() + .height(dimensionResource(id = R.dimen.android_view_height)) + .padding(all = dimensionResource(id = R.dimen.regular_padding)) + .clip(RoundedCornerShape(12.dp)) + .background(Color.Black), + contentAlignment = Alignment.Center, + ) { + if (player != null) { + Player( + player = player, + showControls = showControls, + modifier = + Modifier.pointerInput(Unit) { + awaitPointerEventScope { + while (true) { + // Using PointerEventPass.Initial is correct for a global "reset timer" behavior + // as it allows this component to see pointer events even if child components + // (like buttons in the Player UI) consume them. + awaitPointerEvent(PointerEventPass.Initial) + resetControlsTimer() + } + } + }, + // Ensure that the internal Player composable doesn't have any gestures that might + // conflict with this early interception. + ) + } else { + CircularProgressIndicator(color = MaterialTheme.colorScheme.onSurface) + } + } + } - onApplyEffectsClicked(effectsList) - effectControlsState = effectControlsState.copy(effectsChanged = false) - }, + @Composable + private fun EffectControls(viewModel: EffectViewModel, uiState: EffectUiState) { + Button( + enabled = uiState.effectsEnabled && uiState.effectsChanged, + onClick = { viewModel.applyEffects() }, ) { Text(text = stringResource(id = R.string.apply_effects)) } - EffectControlsList(enabled, effectControlsState) { newEffectControlsState -> - effectControlsState = newEffectControlsState - } + EffectControlsList(viewModel, uiState) } @Composable - private fun EffectControlsList( - enabled: Boolean, - effectControlsState: EffectControlsState, - onEffectControlsStateChange: (EffectControlsState) -> Unit, - ) { + private fun EffectControlsList(viewModel: EffectViewModel, uiState: EffectUiState) { LazyColumn(Modifier.padding(vertical = dimensionResource(id = R.dimen.small_padding))) { item { EffectItem( name = stringResource(id = R.string.contrast), - enabled = enabled, - onCheckedChange = { - onEffectControlsStateChange( - effectControlsState.copy(effectsChanged = true, contrastValue = 0f) - ) - }, + enabled = uiState.effectsEnabled, + checked = uiState.contrastChecked, + onCheckedChange = { checked -> viewModel.updateContrastChecked(checked) }, ) { Row { Text( - text = "%.2f".format(effectControlsState.contrastValue), + text = "%.2f".format(uiState.contrastValue), style = MaterialTheme.typography.bodyLarge, modifier = Modifier.padding(dimensionResource(id = R.dimen.large_padding)).weight(1f), ) + val sliderState = remember { + SliderState(value = uiState.contrastValue, trackRange = -1f..1f) + } + sliderState.value = uiState.contrastValue Slider( - value = effectControlsState.contrastValue, - onValueChange = { newContrastValue -> - val newRoundedContrastValue = "%.2f".format(Locale.ROOT, newContrastValue).toFloat() - onEffectControlsStateChange( - effectControlsState.copy( - effectsChanged = true, - contrastValue = newRoundedContrastValue, - ) - ) + state = sliderState, + onValueChange = { + viewModel.updateContrast(it) + sliderState.value = it }, - valueRange = -1f..1f, modifier = Modifier.weight(4f), ) } @@ -252,47 +218,36 @@ class EffectActivity : ComponentActivity() { item { EffectItem( name = stringResource(R.string.confetti_overlay), - enabled = enabled, - onCheckedChange = { checked -> - onEffectControlsStateChange( - effectControlsState.copy(effectsChanged = true, confettiOverlayChecked = checked) - ) - }, + enabled = uiState.effectsEnabled, + checked = uiState.confettiOverlayChecked, + onCheckedChange = { checked -> viewModel.updateConfetti(checked) }, ) } item { EffectItem( name = stringResource(R.string.clock_overlay), - enabled = enabled, - onCheckedChange = { checked -> - onEffectControlsStateChange( - effectControlsState.copy(effectsChanged = true, clockOverlayChecked = checked) - ) - }, + enabled = uiState.effectsEnabled, + checked = uiState.clockOverlayChecked, + onCheckedChange = { checked -> viewModel.updateClock(checked) }, ) } item { EffectItem( name = stringResource(R.string.lottie_overlay), - enabled = enabled, - onCheckedChange = { checked -> - onEffectControlsStateChange( - effectControlsState.copy(effectsChanged = true, lottieOverlayChecked = checked) - ) - }, + enabled = uiState.effectsEnabled && uiState.lottieEffectsLoaded, + checked = uiState.lottieOverlayChecked, + onCheckedChange = { checked -> viewModel.updateLottieChecked(checked) }, ) { Column { Row { - DropdownControlItem( - title = stringResource(R.string.lottie_asset), - value = - effectControlsState.lottieOverlayName ?: lottieOverlayOptions.keys.toList()[0], - options = lottieOverlayOptions.keys.toList(), - onValueChange = { value -> - onEffectControlsStateChange( - effectControlsState.copy(effectsChanged = true, lottieOverlayName = value) - ) - }, + GenericExposedDropdownMenu( + label = stringResource(R.string.lottie_asset), + selectedValue = + uiState.lottieOverlayName ?: uiState.lottieOverlayOptions.firstOrNull() ?: "", + options = uiState.lottieOverlayOptions, + onOptionSelected = { viewModel.updateLottieName(it) }, + modifier = + Modifier.fillMaxWidth().padding(bottom = dimensionResource(R.dimen.large_padding)), ) } } @@ -301,60 +256,42 @@ class EffectActivity : ComponentActivity() { item { EffectItem( name = stringResource(R.string.custom_text_overlay), - enabled = enabled, - onCheckedChange = { checked -> - onEffectControlsStateChange( - effectControlsState.copy(effectsChanged = !checked, textOverlayChecked = checked) - ) - }, + enabled = uiState.effectsEnabled, + checked = uiState.textOverlayChecked, + onCheckedChange = { checked -> viewModel.updateTextChecked(checked) }, ) { Column { OutlinedTextField( - value = effectControlsState.textOverlayText ?: "", - onValueChange = { newTextOverlayText -> - onEffectControlsStateChange( - effectControlsState.copy( - effectsChanged = true, - textOverlayText = newTextOverlayText.ifEmpty { null }, - ) - ) - }, + value = uiState.textOverlayText ?: "", + onValueChange = { viewModel.updateText(it.ifEmpty { null }) }, label = { Text(stringResource(R.string.text)) }, singleLine = true, modifier = Modifier.fillMaxWidth().padding(bottom = dimensionResource(R.dimen.large_padding)), ) Row { - ColorsDropDownMenu(effectControlsState.textOverlayColor) { color -> - onEffectControlsStateChange( - effectControlsState.copy( - effectsChanged = effectControlsState.textOverlayText != null, - textOverlayColor = color, - ) - ) + ColorsDropDownMenu(uiState.textOverlayColor) { color -> + viewModel.updateTextColor(color) } } Row { Text( - text = - stringResource(R.string.alpha) + - " = %.2f".format(effectControlsState.textOverlayAlpha), + text = stringResource(R.string.alpha) + " = %.2f".format(uiState.textOverlayAlpha), style = MaterialTheme.typography.bodyLarge, modifier = Modifier.padding(dimensionResource(id = R.dimen.large_padding)).weight(1f), ) + val sliderState = remember { + SliderState(value = uiState.textOverlayAlpha, trackRange = 0f..1f) + } + sliderState.value = uiState.textOverlayAlpha Slider( - value = effectControlsState.textOverlayAlpha, + state = sliderState, onValueChange = { newAlphaValue -> val newRoundedAlphaValue = "%.2f".format(Locale.ROOT, newAlphaValue).toFloat() - onEffectControlsStateChange( - effectControlsState.copy( - effectsChanged = effectControlsState.textOverlayText != null, - textOverlayAlpha = newRoundedAlphaValue, - ) - ) + viewModel.updateTextAlpha(newRoundedAlphaValue) + sliderState.value = newRoundedAlphaValue }, - valueRange = 0f..1f, modifier = Modifier.weight(2f), ) } @@ -364,32 +301,21 @@ class EffectActivity : ComponentActivity() { } } - private val lottieOverlayOptions: Map by lazy { - buildMap { - LottieEffectFactory.buildAvailableEffects(application).forEach { (name, effect) -> - put(name, effect) - } - } - } - @Composable fun EffectItem( name: String, enabled: Boolean, - onCheckedChange: (Boolean) -> Unit = {}, + checked: Boolean, + onCheckedChange: (Boolean) -> Unit, content: @Composable () -> Unit = {}, ) { - var checked by rememberSaveable { mutableStateOf(false) } Card( modifier = Modifier.padding( vertical = dimensionResource(id = R.dimen.small_padding), horizontal = dimensionResource(id = R.dimen.regular_padding), ) - .clickable(enabled = enabled && !checked) { - checked = !checked - onCheckedChange(checked) - } + .clickable(enabled = enabled && !checked) { onCheckedChange(!checked) } ) { Column( Modifier.padding(dimensionResource(id = R.dimen.large_padding)) @@ -399,14 +325,7 @@ class EffectActivity : ComponentActivity() { Column(Modifier.weight(1f).padding(dimensionResource(id = R.dimen.large_padding))) { Text(text = name, style = MaterialTheme.typography.bodyLarge) } - Checkbox( - enabled = enabled, - checked = checked, - onCheckedChange = { - checked = !checked - onCheckedChange(checked) - }, - ) + Checkbox(enabled = enabled, checked = checked, onCheckedChange = onCheckedChange) } if (checked) { content() @@ -414,21 +333,6 @@ class EffectActivity : ComponentActivity() { } } } - - private data class EffectControlsState( - val effectsChanged: Boolean = false, - val contrastValue: Float = 0f, - val confettiOverlayChecked: Boolean = false, - val textOverlayChecked: Boolean = false, - val clockOverlayChecked: Boolean = false, - val lottieOverlayChecked: Boolean = false, - val textOverlayText: String? = null, - val textOverlayColor: Color = COLORS[0], - val textOverlayAlpha: Float = 1f, - val lottieOverlayName: String? = null, - ) - - private companion object { - const val JSON_FILENAME = "media.playlist.json" - } } + +private const val CONTROLS_VISIBILITY_TIMEOUT_MS = 3000L diff --git a/demos/effect/src/main/java/androidx/media3/demo/effect/EffectUiState.kt b/demos/effect/src/main/java/androidx/media3/demo/effect/EffectUiState.kt new file mode 100644 index 00000000000..aca73a88c79 --- /dev/null +++ b/demos/effect/src/main/java/androidx/media3/demo/effect/EffectUiState.kt @@ -0,0 +1,39 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.demo.effect + +import androidx.compose.ui.graphics.Color +import androidx.media3.demo.effect.ui.COLORS +import com.google.common.collect.ImmutableList + +/** UI state for the effect demo application. */ +internal data class EffectUiState( + val effectsEnabled: Boolean = false, + val effectsChanged: Boolean = false, + val contrastChecked: Boolean = false, + val contrastValue: Float = 0f, + val confettiOverlayChecked: Boolean = false, + val textOverlayChecked: Boolean = false, + val clockOverlayChecked: Boolean = false, + val lottieOverlayChecked: Boolean = false, + val textOverlayText: String? = null, + val textOverlayColor: Color = COLORS[0], + val textOverlayAlpha: Float = 1f, + val lottieOverlayName: String? = null, + val lottieEffectsLoaded: Boolean = false, + val lottieOverlayOptions: ImmutableList = ImmutableList.of(), + val errorMessage: String? = null, +) diff --git a/demos/effect/src/main/java/androidx/media3/demo/effect/EffectViewModel.kt b/demos/effect/src/main/java/androidx/media3/demo/effect/EffectViewModel.kt new file mode 100644 index 00000000000..0e6d9314220 --- /dev/null +++ b/demos/effect/src/main/java/androidx/media3/demo/effect/EffectViewModel.kt @@ -0,0 +1,310 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.demo.effect + +import android.app.Application +import android.text.Spannable +import android.text.SpannableString +import android.text.style.ForegroundColorSpan +import androidx.annotation.OptIn +import androidx.compose.ui.graphics.Color +import androidx.compose.ui.graphics.toArgb +import androidx.lifecycle.AndroidViewModel +import androidx.lifecycle.viewModelScope +import androidx.media3.common.Effect +import androidx.media3.common.MediaItem +import androidx.media3.common.util.UnstableApi +import androidx.media3.effect.Contrast +import androidx.media3.effect.OverlayEffect +import androidx.media3.effect.StaticOverlaySettings +import androidx.media3.effect.TextOverlay +import androidx.media3.effect.TextureOverlay +import androidx.media3.exoplayer.ExoPlayer +import com.google.common.collect.ImmutableList +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.flow.MutableStateFlow +import kotlinx.coroutines.flow.StateFlow +import kotlinx.coroutines.flow.asStateFlow +import kotlinx.coroutines.flow.update +import kotlinx.coroutines.launch +import kotlinx.coroutines.withContext + +/** + * [AndroidViewModel] for the effect demo application. + * + * This ViewModel manages the [ExoPlayer] instance, loads playlists, and provides methods to update + * and apply video effects based on the current [EffectUiState]. + */ +@OptIn(UnstableApi::class) +internal class EffectViewModel(application: Application) : AndroidViewModel(application) { + + private val _playlistHolderList = MutableStateFlow>(emptyList()) + /** A [StateFlow] emitting the list of available playlists loaded from JSON assets. */ + val playlistHolderList: StateFlow> = _playlistHolderList.asStateFlow() + + private val _uiState = MutableStateFlow(EffectUiState()) + /** A [StateFlow] emitting the current UI state representing effect controls and settings. */ + val uiState: StateFlow = _uiState.asStateFlow() + + /** + * The [ExoPlayer] instance used for media playback, initialized lazily and automatically released + * when this ViewModel is cleared. + */ + val exoPlayer: ExoPlayer by lazy { + ExoPlayer.Builder(application).build().apply { playWhenReady = true } + } + + private var lottieOverlayOptions: Map = emptyMap() + + init { + loadPlaylists() + loadLottieEffects() + } + + private fun loadPlaylists() { + viewModelScope.launch { + try { + val playlists = + withContext(Dispatchers.IO) { + loadPlaylistsFromJson(JSON_FILENAME, getApplication(), "EffectViewModel") + } + _playlistHolderList.value = playlists + if (playlists.isNotEmpty()) { + selectMediaItems(playlists.first().mediaItems) + } else { + _uiState.update { + it.copy( + errorMessage = + getApplication().getString(R.string.no_loaded_playlists_error) + ) + } + } + } catch (e: Exception) { + _uiState.update { + it.copy( + errorMessage = + getApplication() + .getString(R.string.playlist_loading_error, JSON_FILENAME, e) + ) + } + } + } + } + + private fun loadLottieEffects() { + viewModelScope.launch { + val options = + withContext(Dispatchers.IO) { + buildMap { + LottieEffectFactory.buildAvailableEffects(getApplication()).forEach { (name, effect) -> + put(name, effect) + } + } + } + lottieOverlayOptions = options + _uiState.update { + it.copy( + lottieOverlayOptions = ImmutableList.copyOf(options.keys), + lottieOverlayName = + getApplication().getString(R.string.lottie_effect_name_counter), + lottieEffectsLoaded = true, + ) + } + } + } + + /** + * Updates the player with a new list of [MediaItem]s to play, clearing any active video effects + * and enabling effect controls in the UI. + * + * @param mediaItems The list of media items to play. + */ + fun selectMediaItems(mediaItems: List) { + exoPlayer.apply { + setMediaItems(mediaItems) + setVideoEffects(emptyList()) + prepare() + } + _uiState.update { it.copy(effectsEnabled = true, effectsChanged = false) } + } + + /** + * Toggles whether the contrast effect is enabled in the UI controls. + * + * @param checked Whether the contrast effect checkbox is checked. + */ + fun updateContrastChecked(checked: Boolean) { + _uiState.update { + val value = if (checked) it.contrastValue else 0f + it.copy(contrastChecked = checked, contrastValue = value, effectsChanged = true) + } + } + + /** + * Updates the contrast effect value in the UI controls. + * + * @param value The contrast value in the range of -1f to 1f. + */ + fun updateContrast(value: Float) { + _uiState.update { it.copy(contrastValue = value, effectsChanged = true) } + } + + /** + * Toggles whether the confetti overlay is enabled in the UI controls. + * + * @param checked Whether the confetti overlay checkbox is checked. + */ + fun updateConfetti(checked: Boolean) { + _uiState.update { it.copy(confettiOverlayChecked = checked, effectsChanged = true) } + } + + /** + * Toggles whether the clock overlay is enabled in the UI controls. + * + * @param checked Whether the clock overlay checkbox is checked. + */ + fun updateClock(checked: Boolean) { + _uiState.update { it.copy(clockOverlayChecked = checked, effectsChanged = true) } + } + + /** + * Toggles whether the Lottie animation overlay is enabled in the UI controls. + * + * @param checked Whether the Lottie overlay checkbox is checked. + */ + fun updateLottieChecked(checked: Boolean) { + _uiState.update { it.copy(lottieOverlayChecked = checked, effectsChanged = true) } + } + + /** + * Updates the selected Lottie asset name in the UI controls. + * + * @param name The display name of the Lottie effect to apply. + */ + fun updateLottieName(name: String) { + _uiState.update { it.copy(lottieOverlayName = name, effectsChanged = true) } + } + + /** + * Toggles whether the custom text overlay is enabled in the UI controls. + * + * @param checked Whether the custom text overlay checkbox is checked. + */ + fun updateTextChecked(checked: Boolean) { + _uiState.update { + val effectsChanged = !checked // Replicating original logic + it.copy(textOverlayChecked = checked, effectsChanged = effectsChanged) + } + } + + /** + * Updates the text content for the custom text overlay in the UI controls. + * + * @param text The text string to display, or null if empty. + */ + fun updateText(text: String?) { + _uiState.update { it.copy(textOverlayText = text, effectsChanged = true) } + } + + /** + * Updates the font color for the custom text overlay in the UI controls. + * + * @param color The [Color] to apply to the overlay text. + */ + fun updateTextColor(color: Color) { + _uiState.update { + it.copy(textOverlayColor = color, effectsChanged = it.textOverlayText != null) + } + } + + /** + * Updates the alpha scale (transparency) for the custom text overlay in the UI controls. + * + * @param alpha The alpha scale from 0f (transparent) to 1f (opaque). + */ + fun updateTextAlpha(alpha: Float) { + _uiState.update { + it.copy(textOverlayAlpha = alpha, effectsChanged = it.textOverlayText != null) + } + } + + /** + * Builds the video effects list based on the current [EffectUiState] and applies them to the + * underlying [ExoPlayer]. + */ + fun applyEffects() { + val currentState = _uiState.value + val listBuilder = ImmutableList.builder() + + if (currentState.contrastChecked && currentState.contrastValue != 0f) { + listBuilder.add(Contrast(currentState.contrastValue)) + } + + val overlaysBuilder = ImmutableList.builder() + if (currentState.confettiOverlayChecked) { + overlaysBuilder.add(ConfettiOverlay()) + } + + if (currentState.clockOverlayChecked) { + overlaysBuilder.add(ClockOverlay()) + } + + if (currentState.lottieOverlayChecked) { + val lottieEffect = lottieOverlayOptions[currentState.lottieOverlayName] + lottieEffect?.let { listBuilder.add(it) } + } + + val textOverlayText = currentState.textOverlayText + if (currentState.textOverlayChecked && textOverlayText != null) { + val spannableOverlayText = SpannableString(textOverlayText) + spannableOverlayText.setSpan( + ForegroundColorSpan(currentState.textOverlayColor.toArgb()), + /* start= */ 0, + textOverlayText.length, + Spannable.SPAN_EXCLUSIVE_EXCLUSIVE, + ) + val staticOverlaySettings = + StaticOverlaySettings.Builder().setAlphaScale(currentState.textOverlayAlpha).build() + overlaysBuilder.add( + TextOverlay.createStaticTextOverlay(spannableOverlayText, staticOverlaySettings) + ) + } + val overlays = overlaysBuilder.build() + if (overlays.isNotEmpty()) { + listBuilder.add(OverlayEffect(overlays)) + } + + exoPlayer.apply { + setVideoEffects(listBuilder.build()) + prepare() + } + _uiState.update { it.copy(effectsChanged = false) } + } + + /** Clears any active error message in the [EffectUiState]. */ + fun clearErrorMessage() { + _uiState.update { it.copy(errorMessage = null) } + } + + override fun onCleared() { + super.onCleared() + exoPlayer.release() + } + + private companion object { + private const val JSON_FILENAME = "media.playlist.json" + } +} diff --git a/demos/effect/src/main/java/androidx/media3/demo/effect/ui/UiComponents.kt b/demos/effect/src/main/java/androidx/media3/demo/effect/ui/UiComponents.kt index 47fc68a804c..b38e1751b29 100644 --- a/demos/effect/src/main/java/androidx/media3/demo/effect/ui/UiComponents.kt +++ b/demos/effect/src/main/java/androidx/media3/demo/effect/ui/UiComponents.kt @@ -5,7 +5,7 @@ * you may not use this file except in compliance with the License. * You may obtain a copy of the License at * - * http://www.apache.org/licenses/LICENSE-2.0 + * https://www.apache.org/licenses/LICENSE-2.0 * * Unless required by applicable law or agreed to in writing, software * distributed under the License is distributed on an "AS IS" BASIS, @@ -16,41 +16,29 @@ package androidx.media3.demo.effect.ui import androidx.compose.foundation.background -import androidx.compose.foundation.clickable import androidx.compose.foundation.layout.Box -import androidx.compose.foundation.layout.Column -import androidx.compose.foundation.layout.Row -import androidx.compose.foundation.layout.Spacer import androidx.compose.foundation.layout.fillMaxWidth -import androidx.compose.foundation.layout.height import androidx.compose.foundation.layout.padding import androidx.compose.foundation.layout.size -import androidx.compose.foundation.layout.width import androidx.compose.foundation.shape.CircleShape -import androidx.compose.material3.DropdownMenu import androidx.compose.material3.DropdownMenuItem import androidx.compose.material3.ExperimentalMaterial3Api +import androidx.compose.material3.ExposedDropdownMenu import androidx.compose.material3.ExposedDropdownMenuAnchorType import androidx.compose.material3.ExposedDropdownMenuBox import androidx.compose.material3.ExposedDropdownMenuDefaults -import androidx.compose.material3.Icon import androidx.compose.material3.MaterialTheme import androidx.compose.material3.OutlinedTextField -import androidx.compose.material3.Surface import androidx.compose.material3.Text import androidx.compose.runtime.Composable import androidx.compose.runtime.getValue import androidx.compose.runtime.mutableStateOf import androidx.compose.runtime.remember import androidx.compose.runtime.setValue -import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier import androidx.compose.ui.graphics.Color import androidx.compose.ui.res.dimensionResource -import androidx.compose.ui.res.painterResource import androidx.compose.ui.res.stringResource -import androidx.compose.ui.text.font.FontWeight -import androidx.compose.ui.unit.dp import androidx.media3.demo.effect.R internal val COLORS = @@ -68,7 +56,7 @@ internal val COLORS = Color.Magenta, ) -internal val COLOR_NAME_RES_IDS = +private val COLOR_NAME_RES_IDS = mapOf( Color.Black to R.string.color_black, Color.DarkGray to R.string.color_dark_gray, @@ -83,107 +71,78 @@ internal val COLOR_NAME_RES_IDS = Color.Magenta to R.string.color_magenta, ) -@Composable -fun DropdownControlItem( - title: String, - value: String, - options: List = emptyList(), - onValueChange: ((String) -> Unit)? = null, -) { - var expanded by remember { mutableStateOf(false) } - - Row( - modifier = - Modifier.fillMaxWidth() - .clickable { if (options.isNotEmpty()) expanded = true } - .padding(16.dp), - verticalAlignment = Alignment.CenterVertically, - ) { - Column(modifier = Modifier.weight(1f)) { - Text(text = title, style = MaterialTheme.typography.bodyLarge) - } - - Box { - Surface( - modifier = Modifier.height(36.dp).clickable { if (options.isNotEmpty()) expanded = true } - ) { - Row( - verticalAlignment = Alignment.CenterVertically, - modifier = Modifier.padding(horizontal = 12.dp), - ) { - Text( - text = value, - style = MaterialTheme.typography.bodyMedium, - fontWeight = FontWeight.Bold, - ) - Spacer(modifier = Modifier.width(8.dp)) - Icon( - painter = painterResource(R.drawable.keyboard_arrow_down), - contentDescription = null, - modifier = Modifier.size(16.dp), - ) - } - } - - DropdownMenu(expanded = expanded, onDismissRequest = { expanded = false }) { - options.forEach { option -> - DropdownMenuItem( - text = { Text(text = option) }, - onClick = { - onValueChange?.invoke(option) - expanded = false - }, - ) - } - } - } - } -} - -@Composable -private fun getColorName(color: Color): String { - return COLOR_NAME_RES_IDS[color]?.let { stringResource(it) } - ?: stringResource(R.string.unknown_color) -} - @OptIn(ExperimentalMaterial3Api::class) @Composable -internal fun ColorsDropDownMenu(color: Color, onItemSelected: (Color) -> Unit) { +internal fun GenericExposedDropdownMenu( + label: String, + selectedValue: T, + options: List, + onOptionSelected: (T) -> Unit, + modifier: Modifier = Modifier, + itemLabelProvider: @Composable (T) -> String = { it.toString() }, + leadingIconProvider: @Composable ((T) -> Unit)? = null, +) { var expanded by remember { mutableStateOf(false) } ExposedDropdownMenuBox( expanded = expanded, onExpandedChange = { expanded = it }, - modifier = Modifier.fillMaxWidth().padding(bottom = dimensionResource(R.dimen.large_padding)), + modifier = modifier, ) { OutlinedTextField( modifier = Modifier.fillMaxWidth().menuAnchor(ExposedDropdownMenuAnchorType.PrimaryNotEditable), - value = getColorName(color), + value = itemLabelProvider(selectedValue), onValueChange = {}, readOnly = true, singleLine = true, - label = { Text(stringResource(R.string.text_color)) }, + label = { Text(label) }, trailingIcon = { ExposedDropdownMenuDefaults.TrailingIcon(expanded = expanded) }, colors = ExposedDropdownMenuDefaults.textFieldColors(), ) ExposedDropdownMenu(expanded = expanded, onDismissRequest = { expanded = false }) { - for (color in COLORS) { + options.forEach { option -> DropdownMenuItem( - text = { Text(getColorName(color), style = MaterialTheme.typography.bodyLarge) }, + text = { Text(itemLabelProvider(option), style = MaterialTheme.typography.bodyLarge) }, onClick = { - onItemSelected(color) + onOptionSelected(option) expanded = false }, contentPadding = ExposedDropdownMenuDefaults.ItemContentPadding, - leadingIcon = { - Box( - modifier = - Modifier.size(dimensionResource(R.dimen.color_circle_size)) - .background(color, CircleShape) - ) - }, + leadingIcon = leadingIconProvider?.let { { it(option) } }, ) } } } } + +@Composable +private fun getColorName(color: Color): String { + return COLOR_NAME_RES_IDS[color]?.let { stringResource(it) } + ?: stringResource(R.string.unknown_color) +} + +@Composable +internal fun ColorsDropDownMenu( + color: Color, + modifier: Modifier = + Modifier.fillMaxWidth().padding(bottom = dimensionResource(R.dimen.large_padding)), + onItemSelected: (Color) -> Unit, +) { + GenericExposedDropdownMenu( + label = stringResource(R.string.text_color), + selectedValue = color, + options = COLORS, + onOptionSelected = onItemSelected, + itemLabelProvider = { getColorName(it) }, + leadingIconProvider = { ColorCircle(it) }, + modifier = modifier, + ) +} + +@Composable +private fun ColorCircle(color: Color) { + Box( + modifier = + Modifier.size(dimensionResource(R.dimen.color_circle_size)).background(color, CircleShape) + ) +} diff --git a/demos/effect/src/main/res/mipmap-hdpi/ic_launcher.png b/demos/effect/src/main/res/mipmap-hdpi/ic_launcher.png index adaa93220eb..e5a228fb765 100644 Binary files a/demos/effect/src/main/res/mipmap-hdpi/ic_launcher.png and b/demos/effect/src/main/res/mipmap-hdpi/ic_launcher.png differ diff --git a/demos/effect/src/main/res/mipmap-mdpi/ic_launcher.png b/demos/effect/src/main/res/mipmap-mdpi/ic_launcher.png index 9b6f7d5e806..a058cced802 100644 Binary files a/demos/effect/src/main/res/mipmap-mdpi/ic_launcher.png and b/demos/effect/src/main/res/mipmap-mdpi/ic_launcher.png differ diff --git a/demos/effect/src/main/res/mipmap-xhdpi/ic_launcher.png b/demos/effect/src/main/res/mipmap-xhdpi/ic_launcher.png index 2101026c9fe..b2e07ad4ff8 100644 Binary files a/demos/effect/src/main/res/mipmap-xhdpi/ic_launcher.png and b/demos/effect/src/main/res/mipmap-xhdpi/ic_launcher.png differ diff --git a/demos/effect/src/main/res/mipmap-xxhdpi/ic_launcher.png b/demos/effect/src/main/res/mipmap-xxhdpi/ic_launcher.png index 223ec8bd113..c40bc4e6d37 100644 Binary files a/demos/effect/src/main/res/mipmap-xxhdpi/ic_launcher.png and b/demos/effect/src/main/res/mipmap-xxhdpi/ic_launcher.png differ diff --git a/demos/effect/src/main/res/mipmap-xxxhdpi/ic_launcher.png b/demos/effect/src/main/res/mipmap-xxxhdpi/ic_launcher.png index 698ed68c429..b5340fa2881 100644 Binary files a/demos/effect/src/main/res/mipmap-xxxhdpi/ic_launcher.png and b/demos/effect/src/main/res/mipmap-xxxhdpi/ic_launcher.png differ diff --git a/demos/gl/src/main/java/androidx/media3/demo/gl/MainActivity.java b/demos/gl/src/main/java/androidx/media3/demo/gl/MainActivity.java index 66754efbaeb..1538711edf4 100644 --- a/demos/gl/src/main/java/androidx/media3/demo/gl/MainActivity.java +++ b/demos/gl/src/main/java/androidx/media3/demo/gl/MainActivity.java @@ -15,7 +15,6 @@ */ package androidx.media3.demo.gl; -import static android.os.Build.VERSION.SDK_INT; import static com.google.common.base.Preconditions.checkNotNull; import android.app.Activity; @@ -101,18 +100,16 @@ protected void onCreate(@Nullable Bundle savedInstanceState) { @Override public void onStart() { super.onStart(); - if (SDK_INT > 23) { - initializePlayer(); - if (playerView != null) { - playerView.onResume(); - } + initializePlayer(); + if (playerView != null) { + playerView.onResume(); } } @Override public void onResume() { super.onResume(); - if (SDK_INT == 23 || player == null) { + if (player == null) { initializePlayer(); if (playerView != null) { playerView.onResume(); @@ -120,26 +117,13 @@ public void onResume() { } } - @Override - public void onPause() { - super.onPause(); - if (SDK_INT == 23) { - if (playerView != null) { - playerView.onPause(); - } - releasePlayer(); - } - } - @Override public void onStop() { super.onStop(); - if (SDK_INT > 23) { - if (playerView != null) { - playerView.onPause(); - } - releasePlayer(); + if (playerView != null) { + playerView.onPause(); } + releasePlayer(); } private void initializePlayer() { diff --git a/demos/gl/src/main/res/mipmap-hdpi/ic_launcher.png b/demos/gl/src/main/res/mipmap-hdpi/ic_launcher.png index adaa93220eb..e5a228fb765 100644 Binary files a/demos/gl/src/main/res/mipmap-hdpi/ic_launcher.png and b/demos/gl/src/main/res/mipmap-hdpi/ic_launcher.png differ diff --git a/demos/gl/src/main/res/mipmap-mdpi/ic_launcher.png b/demos/gl/src/main/res/mipmap-mdpi/ic_launcher.png index 9b6f7d5e806..a058cced802 100644 Binary files a/demos/gl/src/main/res/mipmap-mdpi/ic_launcher.png and b/demos/gl/src/main/res/mipmap-mdpi/ic_launcher.png differ diff --git a/demos/gl/src/main/res/mipmap-xhdpi/ic_launcher.png b/demos/gl/src/main/res/mipmap-xhdpi/ic_launcher.png index 2101026c9fe..b2e07ad4ff8 100644 Binary files a/demos/gl/src/main/res/mipmap-xhdpi/ic_launcher.png and b/demos/gl/src/main/res/mipmap-xhdpi/ic_launcher.png differ diff --git a/demos/gl/src/main/res/mipmap-xxhdpi/ic_launcher.png b/demos/gl/src/main/res/mipmap-xxhdpi/ic_launcher.png index 223ec8bd113..c40bc4e6d37 100644 Binary files a/demos/gl/src/main/res/mipmap-xxhdpi/ic_launcher.png and b/demos/gl/src/main/res/mipmap-xxhdpi/ic_launcher.png differ diff --git a/demos/gl/src/main/res/mipmap-xxxhdpi/ic_launcher.png b/demos/gl/src/main/res/mipmap-xxxhdpi/ic_launcher.png index 698ed68c429..b5340fa2881 100644 Binary files a/demos/gl/src/main/res/mipmap-xxxhdpi/ic_launcher.png and b/demos/gl/src/main/res/mipmap-xxxhdpi/ic_launcher.png differ diff --git a/demos/main/src/main/assets/media.exolist.json b/demos/main/src/main/assets/media.exolist.json index 19b696d049f..d42a08e7e08 100644 --- a/demos/main/src/main/assets/media.exolist.json +++ b/demos/main/src/main/assets/media.exolist.json @@ -570,6 +570,30 @@ } ] }, + { + "name": "CBCS - CENC - CBCS", + "playlist": [ + { + "uri": "https://storage.googleapis.com/wvmedia/cbcs/h264/tears/tears_aes_cbcs.mpd", + "clip_end_position_ms": 10000, + "drm_scheme": "widevine", + "drm_license_uri": "https://proxy.uat.widevine.com/proxy?provider=widevine_test" + }, + { + "uri": "https://storage.googleapis.com/wvmedia/cenc/h264/tears/tears.mpd", + "clip_start_position_ms": 10000, + "clip_end_position_ms": 15000, + "drm_scheme": "widevine", + "drm_license_uri": "https://proxy.uat.widevine.com/proxy?provider=widevine_test" + }, + { + "uri": "https://storage.googleapis.com/wvmedia/cbcs/h264/tears/tears_aes_cbcs.mpd", + "clip_start_position_ms": 15000, + "drm_scheme": "widevine", + "drm_license_uri": "https://proxy.uat.widevine.com/proxy?provider=widevine_test" + } + ] + }, { "name": "Manual ad insertion", "playlist": [ @@ -854,6 +878,10 @@ "name": "AV1 Rooftop (Everyframe T35)", "uri": "https://storage.googleapis.com/exoplayer-test-media-1/hagc/rooftop_av1_t35track_everyframe_v0.mp4" }, + { + "name": "HEVC Rooftop (Everyframe T35)", + "uri": "https://storage.googleapis.com/exoplayer-test-media-1/hagc/rooftop_hevc_t35track_everyframe_v0.mp4" + }, { "name": "HEVC Lights (SDR Base, Sparse T35)", "uri": "https://storage.googleapis.com/exoplayer-test-media-1/hagc/lights_hevc_sdrbase_t35track_v0.mp4" diff --git a/demos/main/src/main/java/androidx/media3/demo/main/PlayerActivity.java b/demos/main/src/main/java/androidx/media3/demo/main/PlayerActivity.java index a2a2aff576b..6c05ef80b11 100644 --- a/demos/main/src/main/java/androidx/media3/demo/main/PlayerActivity.java +++ b/demos/main/src/main/java/androidx/media3/demo/main/PlayerActivity.java @@ -18,7 +18,6 @@ import android.content.ContentResolver; import android.content.Intent; import android.content.pm.PackageManager; -import android.os.Build; import android.os.Bundle; import android.text.TextUtils; import android.util.Pair; @@ -145,18 +144,16 @@ public void onNewIntent(Intent intent) { @Override public void onStart() { super.onStart(); - if (Build.VERSION.SDK_INT > 23) { - initializePlayer(); - if (playerView != null) { - playerView.onResume(); - } + initializePlayer(); + if (playerView != null) { + playerView.onResume(); } } @Override public void onResume() { super.onResume(); - if (Build.VERSION.SDK_INT == 23 || player == null) { + if (player == null) { initializePlayer(); if (playerView != null) { playerView.onResume(); @@ -164,26 +161,13 @@ public void onResume() { } } - @Override - public void onPause() { - super.onPause(); - if (Build.VERSION.SDK_INT == 23) { - if (playerView != null) { - playerView.onPause(); - } - releasePlayer(); - } - } - @Override public void onStop() { super.onStop(); - if (Build.VERSION.SDK_INT > 23) { - if (playerView != null) { - playerView.onPause(); - } - releasePlayer(); + if (playerView != null) { + playerView.onPause(); } + releasePlayer(); } @Override diff --git a/demos/main/src/main/res/mipmap-hdpi/ic_launcher.png b/demos/main/src/main/res/mipmap-hdpi/ic_launcher.png index adaa93220eb..e5a228fb765 100644 Binary files a/demos/main/src/main/res/mipmap-hdpi/ic_launcher.png and b/demos/main/src/main/res/mipmap-hdpi/ic_launcher.png differ diff --git a/demos/main/src/main/res/mipmap-mdpi/ic_launcher.png b/demos/main/src/main/res/mipmap-mdpi/ic_launcher.png index 9b6f7d5e806..a058cced802 100644 Binary files a/demos/main/src/main/res/mipmap-mdpi/ic_launcher.png and b/demos/main/src/main/res/mipmap-mdpi/ic_launcher.png differ diff --git a/demos/main/src/main/res/mipmap-xhdpi/ic_launcher.png b/demos/main/src/main/res/mipmap-xhdpi/ic_launcher.png index 2101026c9fe..b2e07ad4ff8 100644 Binary files a/demos/main/src/main/res/mipmap-xhdpi/ic_launcher.png and b/demos/main/src/main/res/mipmap-xhdpi/ic_launcher.png differ diff --git a/demos/main/src/main/res/mipmap-xxhdpi/ic_launcher.png b/demos/main/src/main/res/mipmap-xxhdpi/ic_launcher.png index 223ec8bd113..c40bc4e6d37 100644 Binary files a/demos/main/src/main/res/mipmap-xxhdpi/ic_launcher.png and b/demos/main/src/main/res/mipmap-xxhdpi/ic_launcher.png differ diff --git a/demos/main/src/main/res/mipmap-xxxhdpi/ic_launcher.png b/demos/main/src/main/res/mipmap-xxxhdpi/ic_launcher.png index 698ed68c429..b5340fa2881 100644 Binary files a/demos/main/src/main/res/mipmap-xxxhdpi/ic_launcher.png and b/demos/main/src/main/res/mipmap-xxxhdpi/ic_launcher.png differ diff --git a/demos/session/src/main/AndroidManifest.xml b/demos/session/src/main/AndroidManifest.xml index 90819f51fec..91d5053aa3a 100644 --- a/demos/session/src/main/AndroidManifest.xml +++ b/demos/session/src/main/AndroidManifest.xml @@ -32,9 +32,6 @@ android:name="com.google.android.gms.car.application" android:resource="@xml/auto_app_desc" /> - - - private val browser: MediaBrowser? - get() = if (browserFuture.isDone && !browserFuture.isCancelled) browserFuture.get() else null + private var browser: MediaBrowser? = null private lateinit var mediaListAdapter: FolderMediaItemArrayAdapter private lateinit var mediaListView: ListView @@ -55,6 +61,16 @@ class MainActivity : AppCompatActivity() { override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) + lifecycleScope.launch { + lifecycle.repeatOnLifecycle(Lifecycle.State.STARTED) { + try { + initializeBrowser() + awaitCancellation() + } finally { + releaseBrowser() + } + } + } // setting up the layout setContentView(R.layout.activity_main) mediaListView = findViewById(R.id.media_list_view) @@ -63,15 +79,13 @@ class MainActivity : AppCompatActivity() { // setting up on click. When user click on an item, try to display it mediaListView.setOnItemClickListener { _, _, position, _ -> - run { - val selectedMediaItem = mediaListAdapter.getItem(position)!! - // TODO(b/192235359): handle the case where the item is playable but it is not a folder - if (selectedMediaItem.mediaMetadata.isPlayable == true) { - val intent = PlayableFolderActivity.createIntent(this, selectedMediaItem.mediaId) - startActivity(intent) - } else { - pushPathStack(selectedMediaItem) - } + val selectedMediaItem = checkNotNull(mediaListAdapter.getItem(position)) + // TODO(b/192235359): handle the case where the item is playable but it is not a folder + if (selectedMediaItem.mediaMetadata.isPlayable == true) { + val intent = PlayableFolderActivity.createIntent(this, selectedMediaItem.mediaId) + startActivity(intent) + } else { + lifecycleScope.launch { pushPathStack(selectedMediaItem) } } } @@ -85,7 +99,7 @@ class MainActivity : AppCompatActivity() { onBackPressedDispatcher.addCallback( object : OnBackPressedCallback(true) { override fun handleOnBackPressed() { - popPathStack() + lifecycleScope.launch { popPathStack() } } } ) @@ -115,16 +129,6 @@ class MainActivity : AppCompatActivity() { return super.onOptionsItemSelected(item) } - override fun onStart() { - super.onStart() - initializeBrowser() - } - - override fun onStop() { - releaseBrowser() - super.onStop() - } - override fun onRequestPermissionsResult( requestCode: Int, permissions: Array, @@ -142,50 +146,59 @@ class MainActivity : AppCompatActivity() { } } - private fun initializeBrowser() { - browserFuture = - MediaBrowser.Builder( - this, - SessionToken(this, ComponentName(this, PlaybackService::class.java)), - ) - .buildAsync() - browserFuture.addListener({ pushRoot() }, ContextCompat.getMainExecutor(this)) + private suspend fun initializeBrowser() { + val browser = + try { + MediaBrowser.Builder( + this, + SessionToken(this, ComponentName(this, PlaybackService::class.java)), + ) + .buildAsync() + .await() + } catch (e: CancellationException) { + throw e + } catch (e: Exception) { + Log.w(TAG, "Failed to connect to MediaBrowser", e) + return + } + this.browser = browser + pushRoot(browser) } private fun releaseBrowser() { - MediaBrowser.releaseFuture(browserFuture) + browser?.release() + browser = null } - private fun displayChildrenList(mediaItem: MediaItem) { + private suspend fun displayChildrenList(mediaItem: MediaItem) { val browser = this.browser ?: return supportActionBar!!.setDisplayHomeAsUpEnabled(treePathStack.size != 1) - val childrenFuture = - browser.getChildren( - mediaItem.mediaId, - /* page= */ 0, - /* pageSize= */ Int.MAX_VALUE, - /* params= */ null, - ) + val result = + browser + .getChildren( + mediaItem.mediaId, + /* page= */ 0, + /* pageSize= */ Int.MAX_VALUE, + /* params= */ null, + ) + .await() subItemMediaList.clear() - childrenFuture.addListener( - { - val result = childrenFuture.get()!! - val children = result.value!! - subItemMediaList.addAll(children) - mediaListAdapter.notifyDataSetChanged() - }, - ContextCompat.getMainExecutor(this), - ) + if (result.resultCode == LibraryResult.RESULT_SUCCESS) { + subItemMediaList.addAll(checkNotNull(result.value)) + } else { + Log.w(TAG, "Failed to get children for ${mediaItem.mediaId}: ${result.resultCode}") + } + mediaListAdapter.notifyDataSetChanged() } - private fun pushPathStack(mediaItem: MediaItem) { + private suspend fun pushPathStack(mediaItem: MediaItem) { treePathStack.addLast(mediaItem) displayChildrenList(treePathStack.last()) } - private fun popPathStack() { + private suspend fun popPathStack() { treePathStack.removeLast() if (treePathStack.isEmpty()) { finish() @@ -195,22 +208,19 @@ class MainActivity : AppCompatActivity() { displayChildrenList(treePathStack.last()) } - private fun pushRoot() { + private suspend fun pushRoot(browser: MediaBrowser) { // browser can be initialized many times // only push root at the first initialization if (!treePathStack.isEmpty()) { return } - val browser = this.browser ?: return - val rootFuture = browser.getLibraryRoot(/* params= */ null) - rootFuture.addListener( - { - val result: LibraryResult = rootFuture.get()!! - val root: MediaItem = result.value!! - pushPathStack(root) - }, - ContextCompat.getMainExecutor(this), - ) + val result = browser.getLibraryRoot(/* params= */ null).await() + if (result.resultCode == LibraryResult.RESULT_SUCCESS) { + val root = checkNotNull(result.value) + pushPathStack(root) + } else { + Log.w(TAG, "Failed to get library root: ${result.resultCode}") + } } private class FolderMediaItemArrayAdapter( diff --git a/demos/session/src/main/java/androidx/media3/demo/session/PlayableFolderActivity.kt b/demos/session/src/main/java/androidx/media3/demo/session/PlayableFolderActivity.kt index 8d9edff8731..a53eebd0b00 100644 --- a/demos/session/src/main/java/androidx/media3/demo/session/PlayableFolderActivity.kt +++ b/demos/session/src/main/java/androidx/media3/demo/session/PlayableFolderActivity.kt @@ -19,6 +19,7 @@ import android.content.ComponentName import android.content.Context import android.content.Intent import android.os.Bundle +import android.util.Log import android.view.LayoutInflater import android.view.MenuItem import android.view.View @@ -30,24 +31,30 @@ import android.widget.ListView import android.widget.TextView import androidx.annotation.OptIn import androidx.appcompat.app.AppCompatActivity -import androidx.core.content.ContextCompat +import androidx.lifecycle.Lifecycle +import androidx.lifecycle.lifecycleScope +import androidx.lifecycle.repeatOnLifecycle import androidx.media3.cast.MediaRouteButtonFactory import androidx.media3.common.C import androidx.media3.common.MediaItem import androidx.media3.common.Player import androidx.media3.common.util.UnstableApi +import androidx.media3.session.LibraryResult import androidx.media3.session.MediaBrowser import androidx.media3.session.SessionToken import androidx.mediarouter.app.MediaRouteButton import com.google.android.material.floatingactionbutton.ExtendedFloatingActionButton import com.google.android.material.snackbar.BaseTransientBottomBar import com.google.android.material.snackbar.Snackbar -import com.google.common.util.concurrent.ListenableFuture +import kotlinx.coroutines.CancellationException +import kotlinx.coroutines.awaitCancellation +import kotlinx.coroutines.guava.await +import kotlinx.coroutines.launch + +private const val TAG = "PlayableFolderActivity" class PlayableFolderActivity : AppCompatActivity() { - private lateinit var browserFuture: ListenableFuture - private val browser: MediaBrowser? - get() = if (browserFuture.isDone && !browserFuture.isCancelled) browserFuture.get() else null + private var browser: MediaBrowser? = null private lateinit var mediaList: ListView private lateinit var mediaListAdapter: PlayableMediaItemArrayAdapter @@ -65,6 +72,16 @@ class PlayableFolderActivity : AppCompatActivity() { override fun onCreate(savedInstanceState: Bundle?) { super.onCreate(savedInstanceState) + lifecycleScope.launch { + lifecycle.repeatOnLifecycle(Lifecycle.State.STARTED) { + try { + initializeBrowser() + awaitCancellation() + } finally { + releaseBrowser() + } + } + } setContentView(R.layout.activity_playable_folder) supportActionBar!!.setDisplayHomeAsUpEnabled(true) mediaList = findViewById(R.id.media_list_view) @@ -116,16 +133,6 @@ class PlayableFolderActivity : AppCompatActivity() { } } - override fun onStart() { - super.onStart() - initializeBrowser() - } - - override fun onStop() { - super.onStop() - releaseBrowser() - } - override fun onOptionsItemSelected(item: MenuItem): Boolean { if (item.itemId == android.R.id.home) { onBackPressedDispatcher.onBackPressed() @@ -134,45 +141,51 @@ class PlayableFolderActivity : AppCompatActivity() { return super.onOptionsItemSelected(item) } - private fun initializeBrowser() { - browserFuture = - MediaBrowser.Builder( - this, - SessionToken(this, ComponentName(this, PlaybackService::class.java)), - ) - .buildAsync() - browserFuture.addListener({ displayFolder() }, ContextCompat.getMainExecutor(this)) + private suspend fun initializeBrowser() { + val browser = + try { + MediaBrowser.Builder( + this, + SessionToken(this, ComponentName(this, PlaybackService::class.java)), + ) + .buildAsync() + .await() + } catch (e: CancellationException) { + throw e + } catch (e: Exception) { + Log.w(TAG, "Failed to connect to MediaBrowser", e) + return + } + this.browser = browser + displayFolder(browser) } private fun releaseBrowser() { - MediaBrowser.releaseFuture(browserFuture) + browser?.release() + browser = null } - private fun displayFolder() { - val browser = this.browser ?: return - val id: String = intent.getStringExtra(MEDIA_ITEM_ID_KEY)!! - val mediaItemFuture = browser.getItem(id) - val childrenFuture = - browser.getChildren(id, /* page= */ 0, /* pageSize= */ Int.MAX_VALUE, /* params= */ null) - mediaItemFuture.addListener( - { - val title: TextView = findViewById(R.id.folder_description) - val result = mediaItemFuture.get()!! - title.text = result.value!!.mediaMetadata.title - }, - ContextCompat.getMainExecutor(this), - ) - childrenFuture.addListener( - { - val result = childrenFuture.get()!! - val children = result.value!! - - subItemMediaList.clear() - subItemMediaList.addAll(children) - mediaListAdapter.notifyDataSetChanged() - }, - ContextCompat.getMainExecutor(this), - ) + private suspend fun displayFolder(browser: MediaBrowser) { + val id: String = checkNotNull(intent.getStringExtra(MEDIA_ITEM_ID_KEY)) + val itemResult = browser.getItem(id).await() + if (itemResult.resultCode == LibraryResult.RESULT_SUCCESS) { + val title: TextView = findViewById(R.id.folder_description) + title.text = checkNotNull(itemResult.value).mediaMetadata.title + } else { + Log.w(TAG, "Failed to get item $id: ${itemResult.resultCode}") + } + + val childrenResult = + browser + .getChildren(id, /* page= */ 0, /* pageSize= */ Int.MAX_VALUE, /* params= */ null) + .await() + subItemMediaList.clear() + if (childrenResult.resultCode == LibraryResult.RESULT_SUCCESS) { + subItemMediaList.addAll(checkNotNull(childrenResult.value)) + } else { + Log.w(TAG, "Failed to get children for $id: ${childrenResult.resultCode}") + } + mediaListAdapter.notifyDataSetChanged() } private inner class PlayableMediaItemArrayAdapter( diff --git a/demos/session/src/main/res/mipmap-hdpi/ic_launcher.png b/demos/session/src/main/res/mipmap-hdpi/ic_launcher.png index adaa93220eb..e5a228fb765 100644 Binary files a/demos/session/src/main/res/mipmap-hdpi/ic_launcher.png and b/demos/session/src/main/res/mipmap-hdpi/ic_launcher.png differ diff --git a/demos/session/src/main/res/mipmap-mdpi/ic_launcher.png b/demos/session/src/main/res/mipmap-mdpi/ic_launcher.png index 9b6f7d5e806..a058cced802 100644 Binary files a/demos/session/src/main/res/mipmap-mdpi/ic_launcher.png and b/demos/session/src/main/res/mipmap-mdpi/ic_launcher.png differ diff --git a/demos/session/src/main/res/mipmap-xhdpi/ic_launcher.png b/demos/session/src/main/res/mipmap-xhdpi/ic_launcher.png index 2101026c9fe..b2e07ad4ff8 100644 Binary files a/demos/session/src/main/res/mipmap-xhdpi/ic_launcher.png and b/demos/session/src/main/res/mipmap-xhdpi/ic_launcher.png differ diff --git a/demos/session/src/main/res/mipmap-xxhdpi/ic_launcher.png b/demos/session/src/main/res/mipmap-xxhdpi/ic_launcher.png index 223ec8bd113..c40bc4e6d37 100644 Binary files a/demos/session/src/main/res/mipmap-xxhdpi/ic_launcher.png and b/demos/session/src/main/res/mipmap-xxhdpi/ic_launcher.png differ diff --git a/demos/session/src/main/res/mipmap-xxxhdpi/ic_launcher.png b/demos/session/src/main/res/mipmap-xxxhdpi/ic_launcher.png index 698ed68c429..b5340fa2881 100644 Binary files a/demos/session/src/main/res/mipmap-xxxhdpi/ic_launcher.png and b/demos/session/src/main/res/mipmap-xxxhdpi/ic_launcher.png differ diff --git a/demos/session_automotive/src/main/java/androidx/media3/demo/session/automotive/BitmapContentProvider.kt b/demos/session_automotive/src/main/java/androidx/media3/demo/session/automotive/BitmapContentProvider.kt index b95cd9490fe..0eee57ca41d 100644 --- a/demos/session_automotive/src/main/java/androidx/media3/demo/session/automotive/BitmapContentProvider.kt +++ b/demos/session_automotive/src/main/java/androidx/media3/demo/session/automotive/BitmapContentProvider.kt @@ -38,28 +38,21 @@ class BitmapContentProvider : ContentProvider() { override fun onCreate() = true override fun openFile(uri: Uri, mode: String): ParcelFileDescriptor? { - context?.let { ctx -> - getAssetPath(uri)?.let { - return ParcelFileDescriptor.open( - copyAssetFileToCacheDirectory(ctx, it), - ParcelFileDescriptor.MODE_READ_ONLY, - ) - } - } - return super.openFile(uri, mode) - } + val ctx = context ?: return super.openFile(uri, mode) + val assetPath = uri.path?.drop(1) ?: return super.openFile(uri, mode) - private fun getAssetPath(contentUri: Uri): String? { - contentUri.path?.let { - return it.substring(1) - } - return null + val file = copyAssetFileToCacheDirectory(ctx, assetPath) + return ParcelFileDescriptor.open(file, ParcelFileDescriptor.MODE_READ_ONLY) } private fun copyAssetFileToCacheDirectory(context: Context, assetPath: String): File { val publicFile = File(context.cacheDir, assetPath.replace("/", "_")) - if (!publicFile.exists()) { - context.assets.open(assetPath).copyTo(publicFile.outputStream()) + synchronized(this) { + if (!publicFile.exists()) { + context.assets.open(assetPath).use { input -> + publicFile.outputStream().use { output -> input.copyTo(output) } + } + } } return publicFile } diff --git a/demos/session_automotive/src/main/res/mipmap-hdpi/ic_launcher.png b/demos/session_automotive/src/main/res/mipmap-hdpi/ic_launcher.png index adaa93220eb..e5a228fb765 100644 Binary files a/demos/session_automotive/src/main/res/mipmap-hdpi/ic_launcher.png and b/demos/session_automotive/src/main/res/mipmap-hdpi/ic_launcher.png differ diff --git a/demos/session_automotive/src/main/res/mipmap-mdpi/ic_launcher.png b/demos/session_automotive/src/main/res/mipmap-mdpi/ic_launcher.png index 9b6f7d5e806..a058cced802 100644 Binary files a/demos/session_automotive/src/main/res/mipmap-mdpi/ic_launcher.png and b/demos/session_automotive/src/main/res/mipmap-mdpi/ic_launcher.png differ diff --git a/demos/session_automotive/src/main/res/mipmap-xhdpi/ic_launcher.png b/demos/session_automotive/src/main/res/mipmap-xhdpi/ic_launcher.png index 2101026c9fe..b2e07ad4ff8 100644 Binary files a/demos/session_automotive/src/main/res/mipmap-xhdpi/ic_launcher.png and b/demos/session_automotive/src/main/res/mipmap-xhdpi/ic_launcher.png differ diff --git a/demos/session_automotive/src/main/res/mipmap-xxhdpi/ic_launcher.png b/demos/session_automotive/src/main/res/mipmap-xxhdpi/ic_launcher.png index 223ec8bd113..c40bc4e6d37 100644 Binary files a/demos/session_automotive/src/main/res/mipmap-xxhdpi/ic_launcher.png and b/demos/session_automotive/src/main/res/mipmap-xxhdpi/ic_launcher.png differ diff --git a/demos/session_automotive/src/main/res/mipmap-xxxhdpi/ic_launcher.png b/demos/session_automotive/src/main/res/mipmap-xxxhdpi/ic_launcher.png index 698ed68c429..b5340fa2881 100644 Binary files a/demos/session_automotive/src/main/res/mipmap-xxxhdpi/ic_launcher.png and b/demos/session_automotive/src/main/res/mipmap-xxxhdpi/ic_launcher.png differ diff --git a/demos/session_service/build.gradle.kts b/demos/session_service/build.gradle.kts index 8294f168212..70edc512066 100644 --- a/demos/session_service/build.gradle.kts +++ b/demos/session_service/build.gradle.kts @@ -30,7 +30,7 @@ android { protobuf { protoc { artifact = libs.protobuf.protoc.get().toString() } - generateProtoTasks { all().forEach { task -> task.plugins { create("java") } } } + generateProtoTasks { all().forEach { task -> task.builtins { create("java") } } } } dependencies { diff --git a/demos/shortform/src/main/res/mipmap-hdpi/ic_launcher.png b/demos/shortform/src/main/res/mipmap-hdpi/ic_launcher.png index adaa93220eb..e5a228fb765 100644 Binary files a/demos/shortform/src/main/res/mipmap-hdpi/ic_launcher.png and b/demos/shortform/src/main/res/mipmap-hdpi/ic_launcher.png differ diff --git a/demos/shortform/src/main/res/mipmap-mdpi/ic_launcher.png b/demos/shortform/src/main/res/mipmap-mdpi/ic_launcher.png index 9b6f7d5e806..a058cced802 100644 Binary files a/demos/shortform/src/main/res/mipmap-mdpi/ic_launcher.png and b/demos/shortform/src/main/res/mipmap-mdpi/ic_launcher.png differ diff --git a/demos/shortform/src/main/res/mipmap-xhdpi/ic_launcher.png b/demos/shortform/src/main/res/mipmap-xhdpi/ic_launcher.png index 2101026c9fe..b2e07ad4ff8 100644 Binary files a/demos/shortform/src/main/res/mipmap-xhdpi/ic_launcher.png and b/demos/shortform/src/main/res/mipmap-xhdpi/ic_launcher.png differ diff --git a/demos/shortform/src/main/res/mipmap-xxhdpi/ic_launcher.png b/demos/shortform/src/main/res/mipmap-xxhdpi/ic_launcher.png index 223ec8bd113..c40bc4e6d37 100644 Binary files a/demos/shortform/src/main/res/mipmap-xxhdpi/ic_launcher.png and b/demos/shortform/src/main/res/mipmap-xxhdpi/ic_launcher.png differ diff --git a/demos/shortform/src/main/res/mipmap-xxxhdpi/ic_launcher.png b/demos/shortform/src/main/res/mipmap-xxxhdpi/ic_launcher.png index 698ed68c429..b5340fa2881 100644 Binary files a/demos/shortform/src/main/res/mipmap-xxxhdpi/ic_launcher.png and b/demos/shortform/src/main/res/mipmap-xxxhdpi/ic_launcher.png differ diff --git a/demos/surface/src/main/res/mipmap-hdpi/ic_launcher.png b/demos/surface/src/main/res/mipmap-hdpi/ic_launcher.png index adaa93220eb..e5a228fb765 100644 Binary files a/demos/surface/src/main/res/mipmap-hdpi/ic_launcher.png and b/demos/surface/src/main/res/mipmap-hdpi/ic_launcher.png differ diff --git a/demos/surface/src/main/res/mipmap-mdpi/ic_launcher.png b/demos/surface/src/main/res/mipmap-mdpi/ic_launcher.png index 9b6f7d5e806..a058cced802 100644 Binary files a/demos/surface/src/main/res/mipmap-mdpi/ic_launcher.png and b/demos/surface/src/main/res/mipmap-mdpi/ic_launcher.png differ diff --git a/demos/surface/src/main/res/mipmap-xhdpi/ic_launcher.png b/demos/surface/src/main/res/mipmap-xhdpi/ic_launcher.png index 2101026c9fe..b2e07ad4ff8 100644 Binary files a/demos/surface/src/main/res/mipmap-xhdpi/ic_launcher.png and b/demos/surface/src/main/res/mipmap-xhdpi/ic_launcher.png differ diff --git a/demos/surface/src/main/res/mipmap-xxhdpi/ic_launcher.png b/demos/surface/src/main/res/mipmap-xxhdpi/ic_launcher.png index 223ec8bd113..c40bc4e6d37 100644 Binary files a/demos/surface/src/main/res/mipmap-xxhdpi/ic_launcher.png and b/demos/surface/src/main/res/mipmap-xxhdpi/ic_launcher.png differ diff --git a/demos/surface/src/main/res/mipmap-xxxhdpi/ic_launcher.png b/demos/surface/src/main/res/mipmap-xxxhdpi/ic_launcher.png index 698ed68c429..b5340fa2881 100644 Binary files a/demos/surface/src/main/res/mipmap-xxxhdpi/ic_launcher.png and b/demos/surface/src/main/res/mipmap-xxxhdpi/ic_launcher.png differ diff --git a/demos/transformer/build.gradle.kts b/demos/transformer/build.gradle.kts index 04872658131..b6e7a0ae6e2 100644 --- a/demos/transformer/build.gradle.kts +++ b/demos/transformer/build.gradle.kts @@ -40,7 +40,7 @@ dependencies { implementation(libs.androidx.window) implementation(libs.material) implementation(project(":lib-effect")) - implementation(project(":lib-effect-ndk")) + implementation(project(":lib-effect-play-services")) implementation(project(":lib-exoplayer")) implementation(project(":lib-exoplayer-dash")) implementation(project(":lib-transformer")) diff --git a/demos/transformer/src/main/java/androidx/media3/demo/transformer/ConfigurationActivity.java b/demos/transformer/src/main/java/androidx/media3/demo/transformer/ConfigurationActivity.java index bfe725f5a4d..8d1e63e1989 100644 --- a/demos/transformer/src/main/java/androidx/media3/demo/transformer/ConfigurationActivity.java +++ b/demos/transformer/src/main/java/androidx/media3/demo/transformer/ConfigurationActivity.java @@ -104,7 +104,12 @@ public final class ConfigurationActivity extends AppCompatActivity { public static final String TEXT_OVERLAY_TEXT = "text_overlay_text"; public static final String TEXT_OVERLAY_TEXT_COLOR = "text_overlay_text_color"; public static final String TEXT_OVERLAY_ALPHA = "text_overlay_alpha"; - public static final String ENABLE_PACKET_PROCESSOR = "enable_packet_processor"; + public static final String ENABLE_FRAME_PROCESSOR = "enable_frame_processor"; + public static final String ENABLE_GMS_VIDEO_ENHANCEMENT = "enable_gms_video_enhancement"; + public static final String ENABLE_GMS_TONEMAPPING = "enable_gms_tonemapping"; + public static final String ENABLE_GMS_DEBLUR_DENOISE = "enable_gms_deblur_denoise"; + public static final String ENABLE_GMS_UPSCALE = "enable_gms_upscale"; + public static final String FRAME_AGGREGATION_FPS = "frame_aggregation_fps"; // Video effect selections. public static final int DIZZY_CROP_INDEX = 0; @@ -178,6 +183,7 @@ public final class ConfigurationActivity extends AppCompatActivity { private Spinner resolutionHeightSpinner; private Spinner scaleSpinner; private Spinner rotateSpinner; + private EditText frameAggregationFpsEditText; private CheckBox trimCheckBox; private CheckBox enableFallbackCheckBox; private CheckBox enableAnalyzerModeCheckBox; @@ -187,7 +193,11 @@ public final class ConfigurationActivity extends AppCompatActivity { private CheckBox produceFragmentedMp4; private Spinner hdrModeSpinner; private CheckBox enableTrimOptimization; - private CheckBox enablePacketProcessorCheckBox; + private CheckBox enableFrameProcessorCheckBox; + private CheckBox enableGmsVideoEnhancementCheckBox; + private CheckBox enableGmsTonemappingCheckBox; + private CheckBox enableGmsDeblurDenoiseCheckBox; + private CheckBox enableGmsUpscaleCheckBox; private CheckBox enableMp4EditListTrimming; private CheckBox enableCodecDbLite; private Button selectAudioEffectsButton; @@ -303,6 +313,8 @@ protected void onCreate(@Nullable Bundle savedInstanceState) { rotateSpinner.setAdapter(rotateAdapter); rotateAdapter.addAll(SAME_AS_INPUT_OPTION, "0", "10", "45", "60", "90", "180"); + frameAggregationFpsEditText = findViewById(R.id.frame_aggregation_fps_edit_text); + trimCheckBox = findViewById(R.id.trim_checkbox); trimCheckBox.setOnCheckedChangeListener((view, isChecked) -> selectTrimBounds(isChecked)); trimStartMs = C.TIME_UNSET; @@ -326,7 +338,46 @@ protected void onCreate(@Nullable Bundle savedInstanceState) { enableMp4EditListTrimming.setChecked(false); } }); - enablePacketProcessorCheckBox = findViewById(R.id.enable_packet_processor); + enableFrameProcessorCheckBox = findViewById(R.id.enable_frame_processor); + enableGmsVideoEnhancementCheckBox = findViewById(R.id.enable_gms_video_enhancement); + enableGmsTonemappingCheckBox = findViewById(R.id.enable_gms_tonemapping); + enableGmsDeblurDenoiseCheckBox = findViewById(R.id.enable_gms_deblur_denoise); + enableGmsUpscaleCheckBox = findViewById(R.id.enable_gms_upscale); + View frameAggregationFpsRow = findViewById(R.id.frame_aggregation_fps_row); + View gmsVideoEnhancementRow = findViewById(R.id.gms_video_enhancement_row); + View gmsTonemappingRow = findViewById(R.id.gms_tonemapping_row); + View gmsDeblurDenoiseRow = findViewById(R.id.gms_deblur_denoise_row); + View gmsUpscaleRow = findViewById(R.id.gms_upscale_row); + Runnable updateGmsSubRowsVisibility = + () -> { + int visibility = + enableFrameProcessorCheckBox.isChecked() + && enableGmsVideoEnhancementCheckBox.isChecked() + ? View.VISIBLE + : View.GONE; + gmsTonemappingRow.setVisibility(visibility); + gmsDeblurDenoiseRow.setVisibility(visibility); + gmsUpscaleRow.setVisibility(visibility); + }; + enableFrameProcessorCheckBox.setOnCheckedChangeListener( + (buttonView, isChecked) -> { + frameAggregationFpsRow.setVisibility(isChecked ? View.VISIBLE : View.GONE); + gmsVideoEnhancementRow.setVisibility(isChecked ? View.VISIBLE : View.GONE); + updateGmsSubRowsVisibility.run(); + }); + if (SDK_INT < 28) { + enableFrameProcessorCheckBox.setEnabled(false); + findViewById(R.id.enable_frame_processor_hint).setVisibility(View.VISIBLE); + } + enableGmsVideoEnhancementCheckBox.setOnCheckedChangeListener( + (buttonView, isChecked) -> updateGmsSubRowsVisibility.run()); + if (SDK_INT < 33) { + enableGmsVideoEnhancementCheckBox.setEnabled(false); + enableGmsTonemappingCheckBox.setEnabled(false); + enableGmsDeblurDenoiseCheckBox.setEnabled(false); + enableGmsUpscaleCheckBox.setEnabled(false); + findViewById(R.id.enable_gms_video_enhancement_hint).setVisibility(View.VISIBLE); + } enableMp4EditListTrimming.setOnCheckedChangeListener( (buttonView, isChecked) -> { if (isChecked) { @@ -432,7 +483,11 @@ private void startExport() { bundle.putBoolean(ABORT_SLOW_EXPORT, abortSlowExportCheckBox.isChecked()); bundle.putBoolean(PRODUCE_FRAGMENTED_MP4, produceFragmentedMp4.isChecked()); bundle.putBoolean(ENABLE_TRIM_OPTIMIZATION, enableTrimOptimization.isChecked()); - bundle.putBoolean(ENABLE_PACKET_PROCESSOR, enablePacketProcessorCheckBox.isChecked()); + bundle.putBoolean(ENABLE_FRAME_PROCESSOR, enableFrameProcessorCheckBox.isChecked()); + bundle.putBoolean(ENABLE_GMS_VIDEO_ENHANCEMENT, enableGmsVideoEnhancementCheckBox.isChecked()); + bundle.putBoolean(ENABLE_GMS_TONEMAPPING, enableGmsTonemappingCheckBox.isChecked()); + bundle.putBoolean(ENABLE_GMS_DEBLUR_DENOISE, enableGmsDeblurDenoiseCheckBox.isChecked()); + bundle.putBoolean(ENABLE_GMS_UPSCALE, enableGmsUpscaleCheckBox.isChecked()); bundle.putBoolean(ENABLE_MP4_EDIT_LIST_TRIMMING, enableMp4EditListTrimming.isChecked()); bundle.putBoolean(ENABLE_CODECDB_LITE, enableCodecDbLite.isChecked()); String selectedHdrMode = String.valueOf(hdrModeSpinner.getSelectedItem()); @@ -456,6 +511,31 @@ private void startExport() { bundle.putString(TEXT_OVERLAY_TEXT, textOverlayText); bundle.putInt(TEXT_OVERLAY_TEXT_COLOR, textOverlayTextColor); bundle.putFloat(TEXT_OVERLAY_ALPHA, textOverlayAlpha); + if (enableFrameProcessorCheckBox.isChecked()) { + String frameAggregationFps = frameAggregationFpsEditText.getText().toString().trim(); + if (!frameAggregationFps.isEmpty()) { + try { + float fps = Float.parseFloat(frameAggregationFps); + if (Math.round(fps * 1000f) > 0) { + bundle.putFloat(FRAME_AGGREGATION_FPS, fps); + } else { + Toast.makeText( + getApplicationContext(), + R.string.error_invalid_frame_aggregation_fps_value, + Toast.LENGTH_SHORT) + .show(); + return; + } + } catch (NumberFormatException e) { + Toast.makeText( + getApplicationContext(), + R.string.error_invalid_frame_aggregation_fps_format, + Toast.LENGTH_SHORT) + .show(); + return; + } + } + } transformerIntent.putExtras(bundle); @Nullable Uri intentUri; @@ -785,5 +865,7 @@ private void enableTrackSpecificOptions(boolean isAudioEnabled, boolean isVideoE findViewById(R.id.scale).setEnabled(isVideoEnabled); findViewById(R.id.rotate).setEnabled(isVideoEnabled); findViewById(R.id.hdr_mode).setEnabled(isVideoEnabled); + findViewById(R.id.frame_aggregation_fps_text_view).setEnabled(isVideoEnabled); + frameAggregationFpsEditText.setEnabled(isVideoEnabled); } } diff --git a/demos/transformer/src/main/java/androidx/media3/demo/transformer/TransformerActivity.java b/demos/transformer/src/main/java/androidx/media3/demo/transformer/TransformerActivity.java index 8260d970a08..03380a4c706 100644 --- a/demos/transformer/src/main/java/androidx/media3/demo/transformer/TransformerActivity.java +++ b/demos/transformer/src/main/java/androidx/media3/demo/transformer/TransformerActivity.java @@ -44,6 +44,7 @@ import android.text.Spannable; import android.text.SpannableString; import android.text.style.ForegroundColorSpan; +import android.util.Rational; import android.view.SurfaceHolder; import android.view.SurfaceView; import android.view.View; @@ -64,7 +65,6 @@ import androidx.media3.common.DebugViewProvider; import androidx.media3.common.Effect; import androidx.media3.common.MediaItem; -import androidx.media3.common.OverlaySettings; import androidx.media3.common.audio.AudioProcessor; import androidx.media3.common.audio.ChannelMixingAudioProcessor; import androidx.media3.common.audio.ChannelMixingMatrix; @@ -75,13 +75,15 @@ import androidx.media3.common.util.ExperimentalApi; import androidx.media3.common.util.Log; import androidx.media3.common.util.Util; -import androidx.media3.common.video.HardwareBufferFrame; +import androidx.media3.common.video.FrameProcessor; import androidx.media3.datasource.DataSourceBitmapLoader; import androidx.media3.effect.BitmapOverlay; import androidx.media3.effect.Contrast; import androidx.media3.effect.DebugTraceUtil; +import androidx.media3.effect.DefaultGlFrameProcessor; import androidx.media3.effect.DrawableOverlay; import androidx.media3.effect.GlEffect; +import androidx.media3.effect.HardwareBufferJni; import androidx.media3.effect.HslAdjustment; import androidx.media3.effect.LanczosResample; import androidx.media3.effect.OverlayEffect; @@ -90,17 +92,15 @@ import androidx.media3.effect.RgbFilter; import androidx.media3.effect.RgbMatrix; import androidx.media3.effect.ScaleAndRotateTransformation; -import androidx.media3.effect.SimpleGlFrameProcessor; import androidx.media3.effect.SingleColorLut; import androidx.media3.effect.StaticOverlaySettings; import androidx.media3.effect.TextOverlay; import androidx.media3.effect.TextureOverlay; -import androidx.media3.effect.ndk.HardwareBufferJni; +import androidx.media3.effect.playservices.EnhancementSessionDecorator; import androidx.media3.exoplayer.ExoPlayer; import androidx.media3.exoplayer.audio.SilenceSkippingAudioProcessor; import androidx.media3.exoplayer.util.DebugTextViewHelper; import androidx.media3.transformer.Composition; -import androidx.media3.transformer.CompositionFrameMetadata; import androidx.media3.transformer.DefaultEncoderFactory; import androidx.media3.transformer.EditedMediaItem; import androidx.media3.transformer.EditedMediaItemSequence; @@ -114,6 +114,7 @@ import androidx.media3.transformer.ProgressHolder; import androidx.media3.transformer.Transformer; import androidx.media3.transformer.VideoEncoderSettings; +import androidx.media3.transformer.VideoFrameAggregationParameters; import androidx.media3.ui.AspectRatioFrameLayout; import androidx.media3.ui.PlayerView; import androidx.window.layout.WindowMetricsCalculator; @@ -291,7 +292,7 @@ public void onReceive(Context context, Intent intent) { MediaItem mediaItem = createMediaItem(bundle, inputUri); Util.maybeRequestReadStoragePermission(/* activity= */ this, mediaItem); Composition composition = createComposition(mediaItem, bundle); - Transformer transformer = createTransformer(bundle, composition, inputUri, outputFilePath); + Transformer transformer = createTransformer(bundle, inputUri, outputFilePath); exportStopwatch.reset(); exportStopwatch.start(); if (oldOutputFile == null) { @@ -356,24 +357,45 @@ private MediaItem createMediaItem(@Nullable Bundle bundle, Uri uri) { return mediaItemBuilder.build(); } - @OptIn(markerClass = androidx.media3.common.util.ExperimentalApi.class) - private Transformer createTransformer( - @Nullable Bundle bundle, Composition composition, Uri inputUri, String filePath) { + @OptIn(markerClass = ExperimentalApi.class) + private Transformer createTransformer(@Nullable Bundle bundle, Uri inputUri, String filePath) { Transformer.Builder transformerBuilder; - if (bundle != null && bundle.getBoolean(ConfigurationActivity.ENABLE_PACKET_PROCESSOR)) { - // We cannot use checkState(SDK_INT < 28); because the compiler is not smart enough. + if (bundle != null && bundle.getBoolean(ConfigurationActivity.ENABLE_FRAME_PROCESSOR)) { if (SDK_INT < 28) { - throw new IllegalStateException("API version 28+ required to export with PacketProcessor"); + throw new UnsupportedOperationException( + getString(R.string.api_28_required_frame_processor)); } + FrameProcessor.Factory baseGlFactory = + new DefaultGlFrameProcessor.Factory.Builder( + /* context= */ this, HardwareBufferJni.INSTANCE) + .build(); + FrameProcessor.Factory frameProcessorFactory; + if (bundle.getBoolean(ConfigurationActivity.ENABLE_GMS_VIDEO_ENHANCEMENT)) { + if (SDK_INT < 33) { + throw new UnsupportedOperationException( + getString(R.string.api_33_required_gms_video_enhancement)); + } + boolean isTonemappingEnabled = + bundle.getBoolean(ConfigurationActivity.ENABLE_GMS_TONEMAPPING, true); + boolean isDeblurAndDenoiseVideoEnabled = + bundle.getBoolean(ConfigurationActivity.ENABLE_GMS_DEBLUR_DENOISE, false); + boolean isUpscaleVideoEnabled = + bundle.getBoolean(ConfigurationActivity.ENABLE_GMS_UPSCALE, false); + + frameProcessorFactory = + new EnhancementSessionDecorator.Builder(/* context= */ this, baseGlFactory) + .setTonemappingEnabled(isTonemappingEnabled) + .setDeblurAndDenoiseVideoEnabled(isDeblurAndDenoiseVideoEnabled) + .setUpscaleVideoEnabled(isUpscaleVideoEnabled) + .build(); + } else { + frameProcessorFactory = baseGlFactory; + } + transformerBuilder = new Transformer.Builder(/* context= */ this) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setFrameProcessorFactory( - new SimpleGlFrameProcessor.Factory( - /* context= */ this, - HardwareBufferJni.INSTANCE, - /* overlaySettingsProvider= */ TransformerActivity::getOverlaySettings)); + .setFrameProcessorFactory(frameProcessorFactory); } else { transformerBuilder = new Transformer.Builder(/* context= */ this); } @@ -469,6 +491,7 @@ private File createExternalCacheFile(String fileName) throws IOException { return file; } + @OptIn(markerClass = ExperimentalApi.class) private Composition createComposition(MediaItem mediaItem, @Nullable Bundle bundle) { EditedMediaItem.Builder editedMediaItemBuilder = new EditedMediaItem.Builder(mediaItem); // Required for image inputs. For video inputs, it sets the target FPS. @@ -498,6 +521,16 @@ private Composition createComposition(MediaItem mediaItem, @Nullable Bundle bund new Composition.Builder(editedMediaItemSequenceBuilder.build()); if (bundle != null) { compositionBuilder.setHdrMode(bundle.getInt(ConfigurationActivity.HDR_MODE)); + if (bundle.containsKey(ConfigurationActivity.FRAME_AGGREGATION_FPS)) { + float fps = bundle.getFloat(ConfigurationActivity.FRAME_AGGREGATION_FPS); + int numerator = Math.round(fps * 1000f); + if (numerator > 0) { + compositionBuilder.setVideoFrameAggregationParameters( + new VideoFrameAggregationParameters.Builder() + .setFrameRate(new Rational(numerator, 1000)) + .build()); + } + } } return compositionBuilder.build(); } @@ -889,6 +922,7 @@ private void cleanUpExport() { transformer.cancel(); transformer = null; } + if (outputFile != null) { outputFile.delete(); outputFile = null; @@ -931,16 +965,6 @@ private void pauseExport() { oldOutputFile = outputFile; } - @OptIn(markerClass = ExperimentalApi.class) - private static OverlaySettings getOverlaySettings(HardwareBufferFrame frame) { - CompositionFrameMetadata metadata = - checkNotNull( - (CompositionFrameMetadata) - frame.getMetadata().get(CompositionFrameMetadata.KEY_COMPOSITION_FRAME_METADATA)); - return metadata.composition.videoCompositorSettings.getOverlaySettings( - metadata.sequenceIndex, frame.getContentTimeUs()); - } - private boolean isUsingMediaProjection() { return mediaProjection != null; } diff --git a/demos/transformer/src/main/res/layout/configuration_activity.xml b/demos/transformer/src/main/res/layout/configuration_activity.xml index c05097412e4..933a1e8138f 100644 --- a/demos/transformer/src/main/res/layout/configuration_activity.xml +++ b/demos/transformer/src/main/res/layout/configuration_activity.xml @@ -275,13 +275,109 @@ + + + + + + + + android:id="@+id/frame_aggregation_fps_text_view" + android:text="@string/frame_aggregation_fps" /> + + + + + + + + + + + + + + + + + + + Trim HDR mode Enable trim optimization - Enable Packet Processor + Enable FrameProcessor Enable MP4 edit list trimming Enable CodecDB Lite At least one track must be enabled to start export. + API 28+ required for using FrameProcessor Add audio effects Add video effects Periodic vignette options + Frame Aggregation FPS Export Debug preview: Pause @@ -93,4 +95,12 @@ Single sequence items: Edit Average FPS: %1$.2f + Frame aggregation FPS must be positive + Invalid frame aggregation FPS format + e.g. 23.976 + GMS Video Enhancement + API 33+ required for using GMS Video Enhancement + Tone Mapping + Deblur and Denoise + Upscale diff --git a/docsamples/build.gradle.kts b/docsamples/build.gradle.kts index ba4e6ce0d94..fcaefecdbff 100644 --- a/docsamples/build.gradle.kts +++ b/docsamples/build.gradle.kts @@ -35,6 +35,7 @@ dependencies { implementation(libs.glide.concurrent.integration) implementation(libs.kotlinx.coroutines.guava) implementation(project(":lib-cast")) + implementation(project(":lib-common-ktx")) implementation(project(":lib-container")) implementation(project(":lib-datasource-cronet")) implementation(project(":lib-effect")) diff --git a/docsamples/src/main/java/androidx/media3/docsamples/exoplayer/ListeningToPlayerEvents.kt b/docsamples/src/main/java/androidx/media3/docsamples/exoplayer/ListeningToPlayerEvents.kt index ccabab62233..b0217a6b58a 100644 --- a/docsamples/src/main/java/androidx/media3/docsamples/exoplayer/ListeningToPlayerEvents.kt +++ b/docsamples/src/main/java/androidx/media3/docsamples/exoplayer/ListeningToPlayerEvents.kt @@ -27,11 +27,15 @@ import android.os.Looper import androidx.annotation.OptIn import androidx.media3.common.PlaybackException import androidx.media3.common.Player +import androidx.media3.common.listen +import androidx.media3.common.listenTo import androidx.media3.common.util.UnstableApi import androidx.media3.datasource.HttpDataSource.HttpDataSourceException import androidx.media3.datasource.HttpDataSource.InvalidResponseCodeException import androidx.media3.exoplayer.ExoPlayer import androidx.media3.exoplayer.util.EventLogger +import kotlinx.coroutines.CoroutineScope +import kotlinx.coroutines.launch /** Code snippets for listening to player events. */ @Suppress("unused", "CheckReturnValue") @@ -125,4 +129,99 @@ object ListeningToPlayerEventsKt { .send() // [END execute_player_message] } + + fun updateUi(playbackState: Int) {} + + fun handleError(error: PlaybackException?) {} + + fun updateUiAndHandleError(playbackState: Int, error: PlaybackException?) {} + + @OptIn(UnstableApi::class) + fun listenToPlaybackState(player: Player, coroutineScope: CoroutineScope) { + // [START listen_to_playback_state] + coroutineScope.launch { + player.listenTo(Player.EVENT_IS_PLAYING_CHANGED) { + // `Player` is a receiver scope for this trailing lambda + if (isPlaying) { + // Active playback. + } else { + // Not playing. + } + } + } + // [END listen_to_playback_state] + } + + @OptIn(UnstableApi::class) + fun listenToPlayerError(player: Player, coroutineScope: CoroutineScope) { + // [START listen_to_player_error] + coroutineScope.launch { + player.listenTo(Player.EVENT_PLAYER_ERROR) { + val error = playerError ?: return@listenTo + val cause = error.cause + if (cause is HttpDataSourceException) { + // An HTTP error occurred. + if (cause is InvalidResponseCodeException) { + // Retrieve the response code, message and headers + } else { + // Try calling cause.cause to retrieve the underlying cause + } + } + } + } + // [END listen_to_player_error] + } + + @OptIn(UnstableApi::class) + fun listenOnEvents(player: Player, coroutineScope: CoroutineScope) { + // [START listen_on_events] + coroutineScope.launch { + player.listen { events -> + // `Player` is a receiver scope for this trailing lambda + if (events.contains(Player.EVENT_PLAYBACK_STATE_CHANGED)) { + // Access the player state directly from the receiver + updateUi(playbackState) + } + + if (events.contains(Player.EVENT_PLAYER_ERROR)) { + // Access the error directly from the player + handleError(playerError) + } + } + } + // [END listen_on_events] + } + + @OptIn(UnstableApi::class) + fun listenToOnEvents(player: Player, coroutineScope: CoroutineScope) { + // [START listen_to_on_events] + coroutineScope.launch { + player.listenTo(Player.EVENT_PLAYBACK_STATE_CHANGED, Player.EVENT_PLAYER_ERROR) { events -> + // `Player` is a receiver scope for this trailing lambda + if (events.contains(Player.EVENT_PLAYBACK_STATE_CHANGED)) { + // Access the player state directly from the receiver + updateUi(playbackState) + } + + if (events.contains(Player.EVENT_PLAYER_ERROR)) { + // Access the error directly from the player + handleError(playerError) + } + } + } + // [END listen_to_on_events] + } + + @OptIn(UnstableApi::class) + fun listenToMultipleEvents(player: Player, coroutineScope: CoroutineScope) { + // [START listen_to_multiple_events] + coroutineScope.launch { + player.listenTo(Player.EVENT_PLAYBACK_STATE_CHANGED, Player.EVENT_PLAYER_ERROR) { events -> + // Unclear which event got triggered without querying `events` parameter + // The following function will fire whenever either one is caught + updateUiAndHandleError(playbackState, playerError) + } + } + // [END listen_to_multiple_events] + } } diff --git a/docsamples/src/main/java/androidx/media3/docsamples/exoplayer/preloadmanager/PreloadManagerCreate.kt b/docsamples/src/main/java/androidx/media3/docsamples/exoplayer/preloadmanager/PreloadManagerCreate.kt index f775366cb2d..ff0fa2ae9d1 100644 --- a/docsamples/src/main/java/androidx/media3/docsamples/exoplayer/preloadmanager/PreloadManagerCreate.kt +++ b/docsamples/src/main/java/androidx/media3/docsamples/exoplayer/preloadmanager/PreloadManagerCreate.kt @@ -20,6 +20,8 @@ package androidx.media3.docsamples.exoplayer.preloadmanager import android.content.Context import androidx.annotation.OptIn import androidx.media3.common.util.UnstableApi +import androidx.media3.exoplayer.DefaultLoadControl +import androidx.media3.exoplayer.ExoPlayer import androidx.media3.exoplayer.source.preload.DefaultPreloadManager import androidx.media3.exoplayer.source.preload.TargetPreloadStatusControl import kotlin.math.abs @@ -63,4 +65,34 @@ object PreloadManagerCreateKt { val preloadManager = preloadManagerBuilder.build() // [END android_defaultpreloadmanager_createPLM] } + + @OptIn(UnstableApi::class) + private fun createWithLoadControl(context: Context) { + // [START android_defaultpreloadmanager_createPLMWithLoadControl] + val targetPreloadStatusControl = MyTargetPreloadStatusControl() + val preloadManagerBuilder = DefaultPreloadManager.Builder(context, targetPreloadStatusControl) + + preloadManagerBuilder.setLoadControl( + DefaultLoadControl.Builder() + .setPlayerTargetBufferBytes("preload", 128 * 1024 * 1024) // 128 MiB + .build() + ) + val preloadManager = preloadManagerBuilder.build() + // [END android_defaultpreloadmanager_createPLMWithLoadControl] + } + + @OptIn(UnstableApi::class) + private fun buildExoPlayer( + context: Context, + preloadManagerBuilder: DefaultPreloadManager.Builder, + ) { + // [START android_defaultpreloadmanager_buildExoPlayer] + // Direct creation + val exoPlayer = preloadManagerBuilder.buildExoPlayer() + + // Creation with custom playback specific configurations + val skipSilenceExoPlayerBuilder = ExoPlayer.Builder(context).setSkipSilenceEnabled(true) + val skipSilenceExoPlayer = preloadManagerBuilder.buildExoPlayer(skipSilenceExoPlayerBuilder) + // [END android_defaultpreloadmanager_buildExoPlayer] + } } diff --git a/docsamples/src/main/java/androidx/media3/docsamples/exoplayer/preloadmanager/PreloadManagerManagePlay.kt b/docsamples/src/main/java/androidx/media3/docsamples/exoplayer/preloadmanager/PreloadManagerManagePlay.kt index 0ecbbee665d..f871b8a2069 100644 --- a/docsamples/src/main/java/androidx/media3/docsamples/exoplayer/preloadmanager/PreloadManagerManagePlay.kt +++ b/docsamples/src/main/java/androidx/media3/docsamples/exoplayer/preloadmanager/PreloadManagerManagePlay.kt @@ -32,14 +32,12 @@ object PreloadManagerManagePlayKt { } @OptIn(UnstableApi::class) - private fun addMedia(preloadManager: DefaultPreloadManager) { - // [START android_defaultpreloadmanager_addMedia] + private fun addMediaBatch(preloadManager: DefaultPreloadManager) { + // [START android_defaultpreloadmanager_addMediaBatch] val initialMediaItems = pullMediaItemsFromService(count = 20) - for (index in 0 until initialMediaItems.size) { - preloadManager.add(initialMediaItems[index], /* rankingData= */ index) - } - // items aren't actually loaded yet! need to call invalidate() after this - // [END android_defaultpreloadmanager_addMedia] + val rankingDataList = initialMediaItems.indices.toList() + preloadManager.addMediaItems(initialMediaItems, rankingDataList) + // [END android_defaultpreloadmanager_addMediaBatch] } @OptIn(UnstableApi::class) @@ -62,8 +60,8 @@ object PreloadManagerManagePlayKt { if (mediaSource != null) { player.setMediaSource(mediaSource) } else { - // If mediaSource is null, that mediaItem hasn't been added to the preload manager - // yet. So, send it directly to the player when it's about to play + // If the mediaSource is null, its mediaItem hasn't been added to the preload + // manager yet. Send it directly to the player when it's about to play. player.setMediaItem(mediaItem) } player.prepare() @@ -78,10 +76,40 @@ object PreloadManagerManagePlayKt { } @OptIn(UnstableApi::class) - private fun removeMedia(mediaItem: MediaItem, preloadManager: DefaultPreloadManager) { - // [START android_defaultpreloadmanager_removeItem] - preloadManager.remove(mediaItem) - // [END android_defaultpreloadmanager_removeItem] + private fun getAndPlayMediaAndUpdateIndex( + preloadManager: DefaultPreloadManager, + mediaItem: MediaItem, + player: ExoPlayer, + currentIndex: Int, + ) { + // [START android_defaultpreloadmanager_getAndPlayMediaAndUpdateIndex] + // When a media item is about to be displayed on the screen + val mediaSource = preloadManager.getMediaSource(mediaItem) + if (mediaSource != null) { + player.setMediaSource(mediaSource) + } else { + // If the mediaSource is null, its mediaItem hasn't been added to the preload + // manager yet. Send it directly to the player when it's about to play. + player.setMediaItem(mediaItem) + } + player.prepare() + + // When the media item is being displayed at the center of the screen ("in focus") + player.play() + // Update the current playing index to let the preload manager know where the user + // is in the carousel/pagination/list. + preloadManager.setCurrentPlayingIndex(currentIndex) + // [END android_defaultpreloadmanager_getAndPlayMediaAndUpdateIndex] + } + + @OptIn(UnstableApi::class) + private fun removeMediaBatch( + mediaItemsToRemove: List, + preloadManager: DefaultPreloadManager, + ) { + // [START android_defaultpreloadmanager_removeMediaBatch] + preloadManager.removeMediaItems(mediaItemsToRemove) + // [END android_defaultpreloadmanager_removeMediaBatch] } @OptIn(UnstableApi::class) diff --git a/docsamples/src/main/java/androidx/media3/docsamples/transformer/Skills.kt b/docsamples/src/main/java/androidx/media3/docsamples/transformer/Skills.kt new file mode 100644 index 00000000000..1ed0a8299b7 --- /dev/null +++ b/docsamples/src/main/java/androidx/media3/docsamples/transformer/Skills.kt @@ -0,0 +1,179 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.docsamples.transformer + +import android.content.Context +import androidx.annotation.OptIn +import androidx.media3.common.C +import androidx.media3.common.Effect +import androidx.media3.common.MediaItem +import androidx.media3.common.util.ExperimentalApi +import androidx.media3.common.util.UnstableApi +import androidx.media3.effect.LanczosResample +import androidx.media3.effect.MultipleInputVideoGraph +import androidx.media3.effect.RgbFilter +import androidx.media3.effect.ScaleAndRotateTransformation +import androidx.media3.transformer.CompositionPlayer +import androidx.media3.transformer.EditedMediaItem +import androidx.media3.transformer.EditedMediaItemSequence + +/** Code snippets for media editing agent skill. */ +@OptIn(UnstableApi::class, ExperimentalApi::class) +class Skills { + + /** Adds background audio that loops for an entire composition. */ + fun robustBackgroundAudio() { + val audioUri = "dummy_uri" + val audioDurationUs = 59_000_000L + val sequences = mutableListOf() + + // [START background_audio] + // CORRECT: Rely on the framework to loop the audio indefinitely + val audioMediaItem = MediaItem.Builder().setUri(audioUri).build() + val audioItem = EditedMediaItem.Builder(audioMediaItem).setDurationUs(audioDurationUs).build() + + val audioSequence = + EditedMediaItemSequence.Builder(setOf(C.TRACK_TYPE_AUDIO)) + .addItem(audioItem) + .setIsLooping(true) // Native API handles continuous playback safely + .build() + + sequences.add(audioSequence) + // [END background_audio] + } + + /** + * Performs frame dropping by setting a target frame rate. + * + * @param mediaItem The [MediaItem] to which frame dropping will be applied. + */ + fun frameDropping(mediaItem: MediaItem) { + // [START frame_dropping] + // CORRECT: Set the target frame rate on the EditedMediaItem + val targetFps = 15 // Example: 15 FPS + + val itemBuilder = EditedMediaItem.Builder(mediaItem).setFrameRate(targetFps) + // ... apply other effects + + val editedMediaItem = itemBuilder.build() + // [END frame_dropping] + } + + /** + * Applies a grayscale color filter to a video. + * + * @param itemVideoEffects The list of all video effects that will be applied to a media item. + */ + fun grayscaleColorFilter(itemVideoEffects: MutableList) { + // [START grayscale_color_filter] + // CORRECT: Apply a grayscale filter + val grayscaleEffect = RgbFilter.createGrayscaleFilter() + itemVideoEffects.add(grayscaleEffect) + // [END grayscale_color_filter] + } + + /** + * Configures [CompositionPlayer] for multi-track composition previews. + * + * @param context The application [Context]. + * @param sequences The list of [EditedMediaItemSequence] instances in the composition. + */ + fun multiTrackCompositionPreviews(context: Context, sequences: List) { + // [START multi_track_preview] + // CORRECT: Attach the MultipleInputVideoGraph for multi-track support + val playerBuilder = CompositionPlayer.Builder(context) + + if (sequences.size > 1) { // Or check track types + playerBuilder.setVideoGraphFactory(MultipleInputVideoGraph.Factory()) + } + + val player = playerBuilder.build() + // [END multi_track_preview] + } + + /** + * Mutes the audio of a [MediaItem]. + * + * @param mediaItem The [MediaItem] whose audio should be muted. + */ + fun muteAudio(mediaItem: MediaItem) { + // [START mute_audio] + // CORRECT: Remove the audio track from the media item + val itemBuilder = EditedMediaItem.Builder(mediaItem).setRemoveAudio(true) + // ... apply other effects + + val editedMediaItem = itemBuilder.build() + // [END mute_audio] + } + + /** + * Scales the output video resolution to a specific height. + * + * @param globalVideoEffects The list of global video effects to which scaling effects are added. + */ + fun outputResolutionScaling(globalVideoEffects: MutableList) { + // [START output_resolution_scaling] + // CORRECT: Safe and high-quality downscaling + val resolutionHeight = 720 // Example 720p + + globalVideoEffects.add( + // We use an arbitrarily high width (e.g., 10000) as a non-binding constraint. + // This ensures scaling is driven entirely by the target height while + // maintaining a flexible orientation (handling both landscape and portrait). + LanczosResample.scaleToFitWithFlexibleOrientation(10000, resolutionHeight) + ) + // [END output_resolution_scaling] + } + + /** + * Rotates a video frame by 90 degrees. + * + * @param globalVideoEffects The list of global video effects to which the rotation is added. + */ + fun rotateVideoFrame(globalVideoEffects: MutableList) { + // [START rotate_video_frame] + // CORRECT: Rotate the video by 90 degrees + val rotationEffect = ScaleAndRotateTransformation.Builder().setRotationDegrees(90f).build() + + globalVideoEffects.add(rotationEffect) + // [END rotate_video_frame] + } + + /** + * Trims a video using start and end times. + * + * @param item The original [MediaItem]. + * @param startTimeMs The start time of the trim in milliseconds. Must be >= 0, and if [endTimeMs] + * is not [C.TIME_END_OF_SOURCE], must be <= [endTimeMs]. + * @param endTimeMs The end time of the trim in milliseconds. Must be >= [startTimeMs] and <= + * duration of the media item, or [C.TIME_END_OF_SOURCE] to play to the end of the media. + */ + fun videoTrimming(item: MediaItem, startTimeMs: Long, endTimeMs: Long) { + // [START video_trimming] + // CORRECT: Use ClippingConfiguration to trim media + val clippingConfig = + MediaItem.ClippingConfiguration.Builder() + .setStartPositionMs(startTimeMs) + .setEndPositionMs(endTimeMs) + .build() + + val mediaItem = item.buildUpon().setClippingConfiguration(clippingConfig).build() + + val editedMediaItem = EditedMediaItem.Builder(mediaItem).build() + // [END video_trimming] + } +} diff --git a/docsamples/src/main/java/androidx/media3/docsamples/ui/ComposeCustomization.kt b/docsamples/src/main/java/androidx/media3/docsamples/ui/ComposeCustomization.kt index 5e7cfbc3ad1..67cc0df7674 100644 --- a/docsamples/src/main/java/androidx/media3/docsamples/ui/ComposeCustomization.kt +++ b/docsamples/src/main/java/androidx/media3/docsamples/ui/ComposeCustomization.kt @@ -18,16 +18,19 @@ package androidx.media3.docsamples.ui import androidx.annotation.OptIn +import androidx.compose.animation.AnimatedVisibility import androidx.compose.foundation.background import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.fillMaxSize +import androidx.compose.foundation.layout.size import androidx.compose.material.icons.Icons import androidx.compose.material.icons.filled.Pause import androidx.compose.material.icons.filled.PlayArrow import androidx.compose.material3.FilledTonalButton import androidx.compose.material3.Icon import androidx.compose.material3.IconButton +import androidx.compose.material3.Text import androidx.compose.runtime.Composable import androidx.compose.runtime.getValue import androidx.compose.runtime.mutableStateOf @@ -36,6 +39,7 @@ import androidx.compose.ui.Modifier import androidx.compose.ui.graphics.Color import androidx.compose.ui.layout.ContentScale import androidx.compose.ui.res.stringResource +import androidx.compose.ui.unit.dp import androidx.media3.common.Player import androidx.media3.common.Player.EVENT_AVAILABLE_COMMANDS_CHANGED import androidx.media3.common.listen @@ -45,8 +49,11 @@ import androidx.media3.ui.compose.PlayerSurface import androidx.media3.ui.compose.SURFACE_TYPE_SURFACE_VIEW import androidx.media3.ui.compose.SurfaceType import androidx.media3.ui.compose.buttons.PlayPauseButton +import androidx.media3.ui.compose.material3.Player +import androidx.media3.ui.compose.material3.PlayerDefaults import androidx.media3.ui.compose.material3.R import androidx.media3.ui.compose.material3.buttons.NextButton +import androidx.media3.ui.compose.material3.buttons.PlayPauseButton as Material3PlayPauseButton import androidx.media3.ui.compose.material3.buttons.PreviousButton import androidx.media3.ui.compose.modifiers.resizeWithContentScale import androidx.media3.ui.compose.state.rememberPlayPauseButtonState @@ -118,7 +125,7 @@ class ComposeCustomization { ) { val presentationState = rememberPresentationState(player, keepContentOnReset) val scaledModifier = - modifier.resizeWithContentScale(contentScale, presentationState.videoSizeDp) + modifier.resizeWithContentScale(contentScale, presentationState.videoAspectRatio) // Always leave PlayerSurface to be part of the Compose tree because it will be initialized in // the process. If this composable is guarded by some condition, it might never become visible @@ -160,4 +167,30 @@ class ComposeCustomization { } // [END android_compose_mixed_player_controls] } + + // [START android_compose_custom_player_slots] + @Composable + fun CustomPlayerSlots(player: Player, modifier: Modifier = Modifier) { + Player( + player = player, + modifier = modifier, + topControls = { p, visible -> + // Fully custom top controls + AnimatedVisibility(visible) { Text("My custom title") } + }, + centerControls = { p, visible -> + // Use default CenterControls but override the central button + PlayerDefaults.CenterControls( + player = p, + visible = visible, + central = { + // A custom play/pause button + Material3PlayPauseButton(it, modifier = Modifier.size(64.dp)) + }, + ) + }, + // bottomControls are left as default + ) + } + // [END android_compose_custom_player_slots] } diff --git a/gradle/libs.versions.toml b/gradle/libs.versions.toml index 9b9c58bd054..fea36f6e08b 100644 --- a/gradle/libs.versions.toml +++ b/gradle/libs.versions.toml @@ -1,16 +1,16 @@ [versions] -releaseVersion = "1.11.0-alpha01" -releaseVersionCode = "1011000001" +releaseVersion = "1.11.1" +releaseVersionCode = "1011001300" # SDK -compileSdkVersion = "36" appTargetSdkVersion = "34" -minSdkVersion = "23" +minSdkVersion = "24" libTestTargetSdkVersion = "36" +compileSdkVersion = "37" # See https://developer.android.com/training/cars/media/automotive-os#automotive-module automotiveMinSdkVersion = "28" # Gradle Plugins -androidGradlePlugin = "9.0.1" +androidGradlePlugin = "9.1.1" kotlinComposeCompiler = "2.2.0" kotlinGradlePlugin = "2.2.0" protobufGradlePlugin = "0.10.0" @@ -49,6 +49,7 @@ animalSnifferAnnotationsVersion = "1.21" # Keep this in sync with Google's internal Checker Framework version. checkerframeworkVersion = "3.13.0" cronetVersion = "143.7445.0" +coilComposeVersion = "2.7.0" desugarJdkLibsVersion = "2.1.5" dexmakerVersion = "2.28.3" # Keep in sync with gms.strict-version-matcher-plugin @@ -72,9 +73,10 @@ lottieVersion = "6.6.0" mockitoVersion = "3.12.4" okhttpVersion = "4.12.0" playServicesCronetVersion = "18.0.1" +playServicesMediaEffectEnhancementVersion = "16.0.0-beta08" playServicesTfliteVersion = "16.1.0" protobufVersion = "4.31.1" -robolectricVersion = "4.16" +robolectricVersion = "4.16.1" testParameterInjectorVersion = "1.22" truthVersion = "1.4.0" workRuntimeVersion = "2.8.1" @@ -129,6 +131,7 @@ animal-sniffer-annotations = { group = "org.codehaus.mojo", name = "animal-sniff checkerframework-qual = { group = "org.checkerframework", name = "checker-qual", version.ref = "checkerframeworkVersion" } cronet-api = { group = "org.chromium.net", name = "cronet-api", version.ref = "cronetVersion" } cronet-embedded = { group = "org.chromium.net", name = "cronet-embedded", version.ref = "cronetVersion" } +coil-compose = { group = "io.coil-kt", name = "coil-compose", version.ref = "coilComposeVersion" } desugar-jdk-libs = { group = "com.android.tools", name = "desugar_jdk_libs", version.ref = "desugarJdkLibsVersion" } dexmaker-mockito = { group = "com.linkedin.dexmaker", name = "dexmaker-mockito", version.ref = "dexmakerVersion" } errorprone-annotations = { group = "com.google.errorprone", name = "error_prone_annotations", version.ref = "errorProneVersion" } @@ -158,6 +161,7 @@ okhttp = { group = "com.squareup.okhttp3", name = "okhttp", version.ref = "okhtt okhttp-urlconnection = { group = "com.squareup.okhttp3", name = "okhttp-urlconnection", version.ref = "okhttpVersion" } okhttp-mockwebserver = { group = "com.squareup.okhttp3", name = "mockwebserver", version.ref = "okhttpVersion" } play-services-cronet = { group = "com.google.android.gms", name = "play-services-cronet", version.ref = "playServicesCronetVersion" } +play-services-media-effect-enhancement = { group = "com.google.android.gms", name = "play-services-media-effect-enhancement", version.ref = "playServicesMediaEffectEnhancementVersion" } play-services-tflite-java = { group = "com.google.android.gms", name = "play-services-tflite-java", version.ref = "playServicesTfliteVersion" } play-services-tflite-support = { group = "com.google.android.gms", name = "play-services-tflite-support", version.ref = "playServicesTfliteVersion" } protobuf-java = { group = "com.google.protobuf", name = "protobuf-java", version.ref = "protobufVersion" } @@ -169,5 +173,8 @@ truth = { group = "com.google.truth", name = "truth", version.ref = "truthVersio androidx-runner = { group = "androidx.test", name = "runner", version.ref = "runnerVersion" } [plugins] +android-application = { id = "com.android.application", version.ref = "androidGradlePlugin" } +android-library = { id = "com.android.library", version.ref = "androidGradlePlugin" } +kotlin-android = { id = "org.jetbrains.kotlin.android", version.ref = "kotlinGradlePlugin" } kotlin-compose-compiler = { id = "org.jetbrains.kotlin.plugin.compose", version.ref = "kotlinComposeCompiler" } protobuf = { id = "com.google.protobuf", version.ref = "protobufGradlePlugin" } diff --git a/gradle/wrapper/gradle-wrapper.properties b/gradle/wrapper/gradle-wrapper.properties index 7b0b78c686d..aaf5ccaef70 100644 --- a/gradle/wrapper/gradle-wrapper.properties +++ b/gradle/wrapper/gradle-wrapper.properties @@ -1,5 +1,5 @@ distributionBase=GRADLE_USER_HOME distributionPath=wrapper/dists -distributionUrl=https\://services.gradle.org/distributions/gradle-9.1.0-all.zip +distributionUrl=https\://services.gradle.org/distributions/gradle-9.6.1-all.zip zipStoreBase=GRADLE_USER_HOME zipStorePath=wrapper/dists diff --git a/libraries/cast/build.gradle.kts b/libraries/cast/build.gradle.kts index 2299ba3e47d..dcd0022e567 100644 --- a/libraries/cast/build.gradle.kts +++ b/libraries/cast/build.gradle.kts @@ -37,4 +37,5 @@ dependencies { testImplementation(libs.material) testImplementation(project(":test-utils")) testImplementation(libs.robolectric) + testImplementation(libs.test.parameter.injector) } diff --git a/libraries/cast/src/main/java/androidx/media3/cast/Cast.java b/libraries/cast/src/main/java/androidx/media3/cast/Cast.java index 1b8b1eb32b4..3599264142a 100644 --- a/libraries/cast/src/main/java/androidx/media3/cast/Cast.java +++ b/libraries/cast/src/main/java/androidx/media3/cast/Cast.java @@ -328,6 +328,11 @@ public void removeSessionManagerListener(SessionManagerListener lis pendingMediaRouteSelectorListeners.remove(listener); } + @VisibleForTesting + /* package */ boolean hasPendingMediaRouteSelectorListeners() { + return !pendingMediaRouteSelectorListeners.isEmpty(); + } + /** * Returns the ongoing Cast session, or null if there's no ongoing Cast session, or there's no * Cast context available. diff --git a/libraries/cast/src/main/java/androidx/media3/cast/CastPlayer.java b/libraries/cast/src/main/java/androidx/media3/cast/CastPlayer.java index e386a66c460..4195db504a7 100644 --- a/libraries/cast/src/main/java/androidx/media3/cast/CastPlayer.java +++ b/libraries/cast/src/main/java/androidx/media3/cast/CastPlayer.java @@ -61,9 +61,7 @@ public final class CastPlayer extends ForwardingPlayer { */ public interface TransferCallback { - TransferCallback DEFAULT = - (sourcePlayer, targetPlayer) -> - PlayerTransferState.fromPlayer(sourcePlayer).setToPlayer(targetPlayer); + TransferCallback DEFAULT = new DefaultCastPlayerTransferCallback(); /** * Called immediately before changing the active {@link Player}, with the intended use of diff --git a/libraries/cast/src/main/java/androidx/media3/cast/CastPlayerImpl.java b/libraries/cast/src/main/java/androidx/media3/cast/CastPlayerImpl.java index 0ba3dcf09b5..87399cb3a56 100644 --- a/libraries/cast/src/main/java/androidx/media3/cast/CastPlayerImpl.java +++ b/libraries/cast/src/main/java/androidx/media3/cast/CastPlayerImpl.java @@ -49,13 +49,16 @@ public CastPlayerImpl( remotePlayer.setInternalSessionAvailabilityListener(sessionAvailabilityListener); } - private void updateActivePlayer() { + private void updateActivePlayer(boolean shouldPauseOnTransfer) { Player previousPlayer = getPlayer(); Player newPlayer = remotePlayer.isCastSessionAvailable() ? remotePlayer : localPlayer; if (previousPlayer == newPlayer) { return; } transferCallback.transferState(previousPlayer, newPlayer); + if (shouldPauseOnTransfer) { + newPlayer.setPlayWhenReady(false); + } if (previousPlayer.getPlaybackState() != STATE_IDLE) { newPlayer.prepare(); } @@ -95,12 +98,21 @@ private class SessionAvailabilityListenerImpl implements SessionAvailabilityList @Override public void onCastSessionAvailable() { - updateActivePlayer(); + updateActivePlayer(/* shouldPauseOnTransfer= */ false); } @Override public void onCastSessionUnavailable() { - updateActivePlayer(); + // This should never be called as the overload that takes a reason is implemented. + throw new IllegalStateException( + "onCastSessionUnavailable() called when overload which takes reason is implemented."); + } + + @Override + public void onCastSessionUnavailable( + @SessionAvailabilityListener.SessionUnavailableReason int reason) { + updateActivePlayer( + /* shouldPauseOnTransfer= */ reason != SESSION_UNAVAILABLE_REASON_ROUTE_CHANGED); } } } diff --git a/libraries/cast/src/main/java/androidx/media3/cast/CastQueueOrchestrator.java b/libraries/cast/src/main/java/androidx/media3/cast/CastQueueOrchestrator.java new file mode 100644 index 00000000000..7c20be42bb3 --- /dev/null +++ b/libraries/cast/src/main/java/androidx/media3/cast/CastQueueOrchestrator.java @@ -0,0 +1,262 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.cast; + +import androidx.annotation.Nullable; +import androidx.media3.cast.CastTimeline.ItemUid; +import androidx.media3.cast.QueuedOperation.QueueSnapshot; +import androidx.media3.common.C; +import androidx.media3.common.Player; +import androidx.media3.common.Player.TimelineChangeReason; +import androidx.media3.common.PlayerTransferState; +import androidx.media3.common.util.Log; +import com.google.android.gms.cast.CastStatusCodes; +import com.google.android.gms.cast.MediaQueueItem; +import com.google.android.gms.cast.framework.media.RemoteMediaClient; +import com.google.android.gms.cast.framework.media.RemoteMediaClient.MediaChannelResult; +import com.google.android.gms.common.api.Status; +import com.google.common.base.Supplier; +import java.util.ArrayDeque; + +/** + * Serializes Cast queue mutations and computes optimistic masked {@link QueueSnapshot + * QueueSnapshots} on top of the receiver state as reported by the Cast SDK. + */ +/* package */ final class CastQueueOrchestrator { + + /** Listener for changes to the exposed {@link QueueSnapshot}. */ + public interface StateChangeListener { + /** + * Called when the current {@link QueueSnapshot} changes. + * + * @param snapshot The new {@link QueueSnapshot}. + * @param timelineChangeReason The {@link TimelineChangeReason} if listeners should be notified + * of the change, or {@code null} when resetting the snapshot as a result of {@link #reset} + * or if the timeline doesn't change. + */ + void onQueueSnapshotChanged( + QueueSnapshot snapshot, @Nullable @TimelineChangeReason Integer timelineChangeReason); + } + + private static final String TAG = "CastQueueOrchestrator"; + + /** + * Used to fetch the current {@link PlayerTransferState} when executing a {@link QueuedOperation}. + */ + private final Supplier playerTransferStateSupplier; + + private final CastTimelineTracker timelineTracker; + private final StateChangeListener onStateChangedListener; + private final ArrayDeque queuedOperations; + @Nullable private QueuedOperation activeOperation; + + /** The {@link QueueSnapshot} that represents the receiver state as reported by the Cast SDK. */ + private QueueSnapshot baseSnapshot; + + /** + * The {@link QueueSnapshot} obtained after applying the masking from all queued operations to + * {@link #baseSnapshot}. + */ + private QueueSnapshot maskedSnapshot; + + public CastQueueOrchestrator( + Supplier playerTransferStateSupplier, + MediaItemConverter mediaItemConverter, + StateChangeListener onStateChangedListener) { + this.playerTransferStateSupplier = playerTransferStateSupplier; + this.timelineTracker = new CastTimelineTracker(mediaItemConverter); + this.onStateChangedListener = onStateChangedListener; + this.queuedOperations = new ArrayDeque<>(); + this.baseSnapshot = QueueSnapshot.EMPTY; + this.maskedSnapshot = QueueSnapshot.EMPTY; + } + + /** Returns the {@link ItemUid} for the given receiver item ID. */ + @Nullable + public ItemUid getItemUid(int receiverItemId) { + return timelineTracker.getItemUid(receiverItemId); + } + + /** + * Clears all pending and active operations and resets the orchestrator state. + * + * @param clearSnapshots {@code true} to reset the current {@link QueueSnapshot} to {@link + * QueueSnapshot#EMPTY} (for example, when connecting to a new Cast session so state from a + * previous session is not carried over), or {@code false} to retain the last known {@link + * QueueSnapshot} (for example, when a Cast session ends and the player preserves its final + * timeline and playback state). + */ + public void reset(boolean clearSnapshots) { + queuedOperations.clear(); + activeOperation = null; + if (clearSnapshots) { + baseSnapshot = QueueSnapshot.EMPTY; + maskedSnapshot = QueueSnapshot.EMPTY; + } else { + baseSnapshot = maskedSnapshot; + } + timelineTracker.reset(); + onStateChangedListener.onQueueSnapshotChanged(maskedSnapshot, /* timelineChangeReason= */ null); + } + + /** + * Enqueues a {@link QueuedOperation}, updates the masked {@link QueueSnapshot}, and dispatches + * the next eligible operation to {@code remoteMediaClient} if none is currently active. + */ + public void enqueue(QueuedOperation operation, RemoteMediaClient remoteMediaClient) { + QueueSnapshot previousSnapshot = maskedSnapshot; + if (operation.replacesPreviousOperations()) { + queuedOperations.clear(); + activeOperation = null; + timelineTracker.reset(); + } + queuedOperations.addLast(operation); + maskedSnapshot = operation.createMaskedSnapshot(maskedSnapshot, timelineTracker); + dispatchNextOperationIfPossible(remoteMediaClient); + if (!maskedSnapshot.equals(previousSnapshot)) { + onStateChangedListener.onQueueSnapshotChanged( + maskedSnapshot, Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); + } + } + + /** + * Updates the receiver {@link #baseSnapshot} from {@code remoteMediaClient}, recomputes {@link + * #maskedSnapshot}, dispatches any unblocked queued operations, and notifies the state change + * listener if {@link #maskedSnapshot} has changed. + * + *

This method should be called when the receiver queue state changes. For example, upon {@code + * onStatusUpdated()} or {@code mediaQueueChanged()}. + */ + public void onReceiverStateUpdated(@Nullable RemoteMediaClient remoteMediaClient) { + CastTimeline receiverTimeline = + remoteMediaClient != null && remoteMediaClient.getMediaStatus() != null + ? timelineTracker.getCastTimeline(remoteMediaClient) + : CastTimeline.EMPTY_CAST_TIMELINE; + int receiverWindowIndex = + remoteMediaClient != null + ? fetchReceiverWindowIndex(remoteMediaClient, timelineTracker, receiverTimeline) + : 0; + QueueSnapshot previousSnapshot = maskedSnapshot; + boolean isReceiverTimelineComplete = !timelineTracker.hasPendingQueueFetches(); + if (isReceiverTimelineComplete + || (queuedOperations.isEmpty() && maskedSnapshot.timeline.isEmpty())) { + baseSnapshot = new QueueSnapshot(receiverTimeline, receiverWindowIndex, C.TIME_UNSET); + recomputeMaskedSnapshot(); + } + if (remoteMediaClient != null) { + dispatchNextOperationIfPossible(remoteMediaClient); + } + if (!maskedSnapshot.equals(previousSnapshot)) { + onStateChangedListener.onQueueSnapshotChanged( + maskedSnapshot, Player.TIMELINE_CHANGE_REASON_SOURCE_UPDATE); + } + } + + private void dispatchNextOperationIfPossible(RemoteMediaClient remoteMediaClient) { + if (activeOperation != null || queuedOperations.isEmpty()) { + return; + } + // Wait for pending MediaQueue item fetches to complete before dispatching incremental + // operations, so timelineTracker has mapped all synthetic UIDs from the preceding operation + // to their receiver item IDs. Operations that replace the entire queue (such as + // SetMediaItemsOperation) do not depend on prior receiver item IDs and can execute + // immediately. + if (!queuedOperations.peekFirst().replacesPreviousOperations() + && timelineTracker.hasPendingQueueFetches()) { + return; + } + // When the receiver timeline is fully fetched and no local operation is in flight, any + // target item UID from a previously completed operation has already been mapped to a + // receiver item ID. If the next queued operation still cannot execute, its target item no + // longer exists on the receiver (for example, because another sender removed it), so drop + // it to avoid permanently blocking subsequent operations. + boolean droppedOperation = false; + while (!queuedOperations.isEmpty() + && !queuedOperations.peekFirst().canExecute(timelineTracker, remoteMediaClient)) { + QueuedOperation dropped = queuedOperations.removeFirst(); + droppedOperation = true; + Log.w( + TAG, + "Dropping queued operation because its target item is not present on the receiver: " + + dropped.getName()); + } + if (droppedOperation) { + recomputeMaskedSnapshot(); + } + if (queuedOperations.isEmpty()) { + return; + } + QueuedOperation head = queuedOperations.peekFirst(); + activeOperation = head; + head.execute(remoteMediaClient, timelineTracker, playerTransferStateSupplier.get()) + .setResultCallback( + mediaChannelResult -> onOperationResult(head, mediaChannelResult, remoteMediaClient)); + } + + private void onOperationResult( + QueuedOperation operation, + MediaChannelResult mediaChannelResult, + RemoteMediaClient remoteMediaClient) { + // Reference check: if an operation that replaces previous operations (such as + // SetMediaItemsOperation) or reset() cleared or replaced activeOperation while this operation + // was in flight, ignore the stale callback. + if (operation != activeOperation) { + return; + } + activeOperation = null; + @Nullable Status status = mediaChannelResult != null ? mediaChannelResult.getStatus() : null; + boolean isError = + status != null && !status.isSuccess() && status.getStatusCode() != CastStatusCodes.REPLACED; + if (isError) { + CastUtils.logOperationFailedIfStatusError(TAG, operation.getName(), mediaChannelResult); + queuedOperations.clear(); + maskedSnapshot = baseSnapshot; + onStateChangedListener.onQueueSnapshotChanged( + maskedSnapshot, Player.TIMELINE_CHANGE_REASON_SOURCE_UPDATE); + return; + } + if (queuedOperations.peekFirst() == operation) { + queuedOperations.removeFirst(); + } + onReceiverStateUpdated(remoteMediaClient); + } + + private void recomputeMaskedSnapshot() { + QueueSnapshot current = baseSnapshot; + for (QueuedOperation operation : queuedOperations) { + current = operation.createMaskedSnapshot(current, timelineTracker); + } + maskedSnapshot = current; + } + + private static int fetchReceiverWindowIndex( + RemoteMediaClient remoteMediaClient, + CastTimelineTracker timelineTracker, + CastTimeline timeline) { + int itemId = CastUtils.getCurrentOrLoadingItemId(remoteMediaClient); + int windowIndex = C.INDEX_UNSET; + if (itemId != MediaQueueItem.INVALID_ITEM_ID) { + ItemUid itemUid = timelineTracker.getItemUid(itemId); + if (itemUid != null) { + windowIndex = timeline.getIndexOfPeriod(itemUid); + } + } + if (windowIndex == C.INDEX_UNSET) { + windowIndex = 0; + } + return windowIndex; + } +} diff --git a/libraries/cast/src/main/java/androidx/media3/cast/CastTimeline.java b/libraries/cast/src/main/java/androidx/media3/cast/CastTimeline.java index f963dbe883a..08177fe0d50 100644 --- a/libraries/cast/src/main/java/androidx/media3/cast/CastTimeline.java +++ b/libraries/cast/src/main/java/androidx/media3/cast/CastTimeline.java @@ -15,19 +15,63 @@ */ package androidx.media3.cast; -import android.util.SparseArray; -import android.util.SparseIntArray; +import static com.google.common.base.Preconditions.checkNotNull; + import androidx.annotation.Nullable; import androidx.media3.common.AdPlaybackState; import androidx.media3.common.C; import androidx.media3.common.MediaItem; import androidx.media3.common.Timeline; import com.google.android.gms.cast.MediaInfo; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; import java.util.Arrays; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.UUID; /** A {@link Timeline} for Cast media queues. */ /* package */ final class CastTimeline extends Timeline { + /** An identifier for a period or window in a {@link CastTimeline}. */ + public static final class ItemUid { + private final String value; + + private ItemUid(String value) { + this.value = value; + } + + public static ItemUid generateItemUid() { + return new ItemUid(UUID.randomUUID().toString()); + } + + public static ItemUid of(String value) { + return new ItemUid(checkNotNull(value)); + } + + @Override + public boolean equals(@Nullable Object other) { + if (this == other) { + return true; + } + if (!(other instanceof ItemUid)) { + return false; + } + return value.equals(((ItemUid) other).value); + } + + @Override + public int hashCode() { + return value.hashCode(); + } + + @Override + public String toString() { + return value; + } + } + /** Holds {@link Timeline} related data for a Cast media item. */ public static final class ItemData { @@ -110,11 +154,13 @@ public ItemData copyWithNewValues( /** {@link Timeline} for a cast queue that has no items. */ public static final CastTimeline EMPTY_CAST_TIMELINE = - new CastTimeline(new int[0], new SparseArray<>()); + new CastTimeline(ImmutableList.of(), ImmutableMap.of()); - private final SparseIntArray idsToIndex; - private final MediaItem[] mediaItems; - private final int[] ids; + // In CastTimeline, periods and windows have a 1:1 correspondence. Period IDs and window UIDs are + // used interchangeably in this class. + private final Map periodUidsToIndex; + private final ImmutableList mediaItems; + private final ImmutableList periodUids; private final long[] durationsUs; private final long[] defaultPositionsUs; private final boolean[] isLive; @@ -125,30 +171,35 @@ public ItemData copyWithNewValues( * @param itemIds The ids of the items in the timeline. * @param itemIdToData Maps item ids to {@link ItemData}. */ - public CastTimeline(int[] itemIds, SparseArray itemIdToData) { - int itemCount = itemIds.length; - idsToIndex = new SparseIntArray(itemCount); - ids = Arrays.copyOf(itemIds, itemCount); + public CastTimeline(List itemIds, Map itemIdToData) { + int itemCount = itemIds.size(); + periodUids = ImmutableList.copyOf(itemIds); + periodUidsToIndex = new HashMap<>(itemCount); durationsUs = new long[itemCount]; defaultPositionsUs = new long[itemCount]; isLive = new boolean[itemCount]; - mediaItems = new MediaItem[itemCount]; - for (int i = 0; i < ids.length; i++) { - int id = ids[i]; - idsToIndex.put(id, i); - ItemData data = itemIdToData.get(id, ItemData.EMPTY); - mediaItems[i] = data.mediaItem; + ImmutableList.Builder mediaItemsBuilder = + ImmutableList.builderWithExpectedSize(itemCount); + for (int i = 0; i < periodUids.size(); i++) { + ItemUid id = periodUids.get(i); + periodUidsToIndex.put(id, i); + ItemData data = itemIdToData.get(id); + if (data == null) { + data = ItemData.EMPTY; + } + mediaItemsBuilder.add(data.mediaItem); durationsUs[i] = data.durationUs; defaultPositionsUs[i] = data.defaultPositionUs == C.TIME_UNSET ? 0 : data.defaultPositionUs; isLive[i] = data.isLive; } + mediaItems = mediaItemsBuilder.build(); } // Timeline implementation. @Override public int getWindowCount() { - return ids.length; + return periodUids.size(); } @Override @@ -156,15 +207,15 @@ public Window getWindow(int windowIndex, Window window, long defaultPositionProj long durationUs = durationsUs[windowIndex]; boolean isDynamic = durationUs == C.TIME_UNSET; return window.set( - /* uid= */ ids[windowIndex], - /* mediaItem= */ mediaItems[windowIndex], + /* uid= */ periodUids.get(windowIndex), + /* mediaItem= */ mediaItems.get(windowIndex), /* manifest= */ null, /* presentationStartTimeMs= */ C.TIME_UNSET, /* windowStartTimeMs= */ C.TIME_UNSET, /* elapsedRealtimeEpochOffsetMs= */ C.TIME_UNSET, /* isSeekable= */ !isDynamic, isDynamic, - isLive[windowIndex] ? mediaItems[windowIndex].liveConfiguration : null, + isLive[windowIndex] ? mediaItems.get(windowIndex).liveConfiguration : null, defaultPositionsUs[windowIndex], durationUs, /* firstPeriodIndex= */ windowIndex, @@ -174,14 +225,14 @@ public Window getWindow(int windowIndex, Window window, long defaultPositionProj @Override public int getPeriodCount() { - return ids.length; + return periodUids.size(); } @Override public Period getPeriod(int periodIndex, Period period, boolean setIds) { - int id = ids[periodIndex]; + ItemUid id = periodUids.get(periodIndex); return period.set( - id, + /* id= */ id, /* uid= */ id, periodIndex, durationsUs[periodIndex], @@ -192,12 +243,16 @@ public Period getPeriod(int periodIndex, Period period, boolean setIds) { @Override public int getIndexOfPeriod(Object uid) { - return uid instanceof Integer ? idsToIndex.get((int) uid, C.INDEX_UNSET) : C.INDEX_UNSET; + if (!(uid instanceof ItemUid)) { + return C.INDEX_UNSET; + } + Integer index = periodUidsToIndex.get(uid); + return index != null ? index : C.INDEX_UNSET; } @Override - public Integer getUidOfPeriod(int periodIndex) { - return ids[periodIndex]; + public Object getUidOfPeriod(int periodIndex) { + return periodUids.get(periodIndex); } // equals and hashCode implementations. @@ -210,18 +265,20 @@ public boolean equals(@Nullable Object other) { return false; } CastTimeline that = (CastTimeline) other; - return Arrays.equals(ids, that.ids) + return periodUids.equals(that.periodUids) && Arrays.equals(durationsUs, that.durationsUs) && Arrays.equals(defaultPositionsUs, that.defaultPositionsUs) - && Arrays.equals(isLive, that.isLive); + && Arrays.equals(isLive, that.isLive) + && mediaItems.equals(that.mediaItems); } @Override public int hashCode() { - int result = Arrays.hashCode(ids); + int result = periodUids.hashCode(); result = 31 * result + Arrays.hashCode(durationsUs); result = 31 * result + Arrays.hashCode(defaultPositionsUs); result = 31 * result + Arrays.hashCode(isLive); + result = 31 * result + mediaItems.hashCode(); return result; } } diff --git a/libraries/cast/src/main/java/androidx/media3/cast/CastTimelineTracker.java b/libraries/cast/src/main/java/androidx/media3/cast/CastTimelineTracker.java index b0b75f1c20c..a0f0ba4655c 100644 --- a/libraries/cast/src/main/java/androidx/media3/cast/CastTimelineTracker.java +++ b/libraries/cast/src/main/java/androidx/media3/cast/CastTimelineTracker.java @@ -21,16 +21,24 @@ import android.util.SparseArray; import androidx.annotation.Nullable; import androidx.annotation.VisibleForTesting; +import androidx.media3.cast.CastTimeline.ItemData; +import androidx.media3.cast.CastTimeline.ItemUid; import androidx.media3.common.C; import androidx.media3.common.MediaItem; -import androidx.media3.common.Player; +import androidx.media3.common.util.Log; +import androidx.media3.common.util.Util; import com.google.android.gms.cast.MediaInfo; import com.google.android.gms.cast.MediaQueueItem; import com.google.android.gms.cast.MediaStatus; +import com.google.android.gms.cast.framework.media.MediaQueue; import com.google.android.gms.cast.framework.media.RemoteMediaClient; +import com.google.common.collect.ImmutableList; import java.util.HashMap; import java.util.HashSet; import java.util.List; +import java.util.Map; +import org.json.JSONException; +import org.json.JSONObject; /** * Creates {@link CastTimeline CastTimelines} from cast receiver app status updates. @@ -39,9 +47,24 @@ * durations in the media queue items [See internal: b/65152553]. */ /* package */ final class CastTimelineTracker { + private static final String TAG = "CastTlTracker"; + /* package */ static final String KEY_SYNTHETIC_ID = "m3-syntheticId"; - private final SparseArray itemIdToData; + // Maximum number of queue item IDs fetched from the remote client per pass. This is also the + // cache size configured on MediaQueue to prevent cache evictions during multi-pass fetching. + @VisibleForTesting /* package */ static final int MAX_FETCH_COUNT = 20; + + private final Map itemIdToData; + // TODO: b/553993066 - Evict entries when items are removed from the timeline to avoid unbounded + // growth during long-running sessions. + private final Map mediaItemsBySyntheticId; + // Maps the receiver assigned id to the synthetic id for media items. The synthetic id is used as + // the stable uid of the media item in CastTimeline and abstracts away the receiver assigned id + // from consumers. + private final SparseArray receiverItemIdToUid; + private final Map uidToReceiverItemId; private final MediaItemConverter mediaItemConverter; + private boolean hasPendingQueueFetches; @VisibleForTesting /* package */ final HashMap mediaItemsByContentId; /** @@ -52,36 +75,81 @@ */ public CastTimelineTracker(MediaItemConverter mediaItemConverter) { this.mediaItemConverter = mediaItemConverter; - itemIdToData = new SparseArray<>(); + itemIdToData = new HashMap<>(); + mediaItemsBySyntheticId = new HashMap<>(); + receiverItemIdToUid = new SparseArray<>(); + uidToReceiverItemId = new HashMap<>(); mediaItemsByContentId = new HashMap<>(); } /** - * Called when media items {@linkplain Player#setMediaItems have been set to the playlist} and are - * sent to the cast playback queue. A future queue update of the {@link RemoteMediaClient} will - * reflect this addition. - * - * @param mediaItems The media items that have been set. - * @param mediaQueueItems The corresponding media queue items. + * Returns the {@link ItemUid} associated with {@code receiverItemId}, or {@code null} if not + * found. */ - public void onMediaItemsSet(List mediaItems, MediaQueueItem[] mediaQueueItems) { + @Nullable + public ItemUid getItemUid(int receiverItemId) { + return receiverItemIdToUid.get(receiverItemId); + } + + /** + * Returns the receiver item ID for {@code uid}, or {@link MediaQueueItem#INVALID_ITEM_ID} if not + * found. + */ + public int getReceiverItemId(@Nullable Object uid) { + if (uid == null || !(uid instanceof ItemUid)) { + return MediaQueueItem.INVALID_ITEM_ID; + } + Integer receiverId = uidToReceiverItemId.get(uid); + return receiverId != null ? receiverId : MediaQueueItem.INVALID_ITEM_ID; + } + + /** Resets all item data and UID mappings. */ + public void reset() { + itemIdToData.clear(); + mediaItemsBySyntheticId.clear(); + receiverItemIdToUid.clear(); + uidToReceiverItemId.clear(); mediaItemsByContentId.clear(); - onMediaItemsAdded(mediaItems, mediaQueueItems); + hasPendingQueueFetches = false; + } + + /** Returns whether any items in the receiver's {@link MediaQueue} are still being fetched. */ + public boolean hasPendingQueueFetches() { + return hasPendingQueueFetches; } /** - * Called when media items {@linkplain Player#addMediaItems(List) have been added} and are sent to - * the cast playback queue. A future queue update of the {@link RemoteMediaClient} will reflect - * this addition. - * - * @param mediaItems The media items that have been added. - * @param mediaQueueItems The corresponding media queue items. + * Holds the registered {@link MediaQueueItem MediaQueueItems} and their corresponding synthetic + * {@link ItemUid ItemUids}. + */ + public static final class RegisteredMediaItems { + public final MediaQueueItem[] queueItems; + public final ImmutableList itemUids; + + public RegisteredMediaItems(MediaQueueItem[] queueItems, ImmutableList itemUids) { + this.queueItems = queueItems; + this.itemUids = itemUids; + } + } + + /** + * Registers the given {@link MediaItem MediaItems} with synthetic {@link ItemUid ItemUids} and + * returns the registered {@link MediaQueueItem MediaQueueItems} and {@link ItemUid ItemUids}. */ - public void onMediaItemsAdded(List mediaItems, MediaQueueItem[] mediaQueueItems) { + public RegisteredMediaItems registerMediaItems(List mediaItems) { + ImmutableList.Builder itemUids = + ImmutableList.builderWithExpectedSize(mediaItems.size()); + MediaQueueItem[] mediaQueueItems = new MediaQueueItem[mediaItems.size()]; for (int i = 0; i < mediaItems.size(); i++) { - mediaItemsByContentId.put( - checkNotNull(mediaQueueItems[i].getMedia()).getContentId(), mediaItems.get(i)); + MediaItem mediaItem = mediaItems.get(i); + ItemUid itemUid = ItemUid.generateItemUid(); + itemUids.add(itemUid); + MediaQueueItem queueItem = mediaItemConverter.toMediaQueueItem(mediaItem); + MediaQueueItem updatedMediaQueueItem = attachSyntheticId(queueItem, itemUid); + mediaQueueItems[i] = updatedMediaQueueItem; + registerMediaItem(itemUid, mediaItem, updatedMediaQueueItem); } + return new RegisteredMediaItems(mediaQueueItems, itemUids.build()); } /** @@ -95,7 +163,8 @@ public void onMediaItemsAdded(List mediaItems, MediaQueueItem[] media * @return A {@link CastTimeline} that represents the given {@code remoteMediaClient} status. */ public CastTimeline getCastTimeline(RemoteMediaClient remoteMediaClient) { - int[] itemIds = remoteMediaClient.getMediaQueue().getItemIds(); + MediaQueue mediaQueue = remoteMediaClient.getMediaQueue(); + int[] itemIds = mediaQueue.getItemIds(); if (itemIds.length > 0) { // Only remove unused items when there is something in the queue to avoid removing all entries // if the remote media client clears the queue temporarily. See [Internal ref: b/128825216]. @@ -105,41 +174,127 @@ public CastTimeline getCastTimeline(RemoteMediaClient remoteMediaClient) { // TODO: Reset state when the app instance changes [Internal ref: b/129672468]. MediaStatus mediaStatus = remoteMediaClient.getMediaStatus(); if (mediaStatus == null || mediaStatus.getMediaInfo() == null) { + hasPendingQueueFetches = false; return CastTimeline.EMPTY_CAST_TIMELINE; } - int currentItemId = mediaStatus.getCurrentItemId(); - String currentContentId = checkNotNull(mediaStatus.getMediaInfo()).getContentId(); - MediaItem mediaItem = mediaItemsByContentId.get(currentContentId); + for (MediaQueueItem queueItem : mediaStatus.getQueueItems()) { + updateItemDataFromQueueItem(queueItem); + } + + int currentItemId = CastUtils.getCurrentOrLoadingItemId(remoteMediaClient); + int currentItemIndex = Util.linearSearch(itemIds, currentItemId); + if (currentItemIndex == C.INDEX_UNSET) { + // This is not expected to happen, but prevents us from running out of bounds in the following + // loop. + currentItemIndex = 0; + } + + // MediaStatus can report an active item or queue items before MediaQueue finishes fetching + // the full item ID list (itemIds.length == 0). Mark fetches as pending so callers do not + // treat the resulting empty timeline as a complete receiver state. + hasPendingQueueFetches = + itemIds.length == 0 + && (currentItemId != MediaQueueItem.INVALID_ITEM_ID + || !mediaStatus.getQueueItems().isEmpty()); + + // Fetch missing item metadata starting from the current playback index forward. + // To prevent silent evictions in the MediaQueue fetch buffer (which is hard-capped at + // MediaQueue's internal DEFAULT_MAX_FETCH_COUNT = 20 items), we cap network fetch triggers per + // pass to MAX_FETCH_COUNT and fall back to cache-only lookups (fetchIfNeeded = false) for + // remaining items. + int fetchCount = 0; + for (int step = 0; step < itemIds.length; step++) { + int i = (currentItemIndex + step) % itemIds.length; + int itemId = itemIds[i]; + ItemUid uid = receiverItemIdToUid.get(itemId); + ItemData itemData = uid != null ? itemIdToData.get(uid) : null; + if (itemData == null || itemData.mediaItem == MediaItem.EMPTY) { + boolean fetchIfNeeded = fetchCount < MAX_FETCH_COUNT; + MediaQueueItem queueItem = mediaQueue.getItemAtIndex(i, fetchIfNeeded); + if (queueItem != null) { + updateItemDataFromQueueItem(queueItem); + } else { + if (itemId != currentItemId) { + hasPendingQueueFetches = true; + } + if (fetchIfNeeded) { + fetchCount++; + } + } + } + } + + // Process mediaStatus.getMediaInfo() after mediaStatus.getQueueItems()[...].getMedia(). Static + // queue items in the Cast SDK do not receive dynamic runtime updates (such as transitioning + // from buffered to live upon manifest loading). Updating from mediaStatus.getMediaInfo() second + // ensures that active runtime playback state is preserved and not overwritten by stale queue + // item metadata. + MediaInfo currentMediaInfo = checkNotNull(mediaStatus.getMediaInfo()); + ItemUid currentItemUid = getOrCreateItemUid(currentItemId, currentMediaInfo); + @Nullable MediaItem mediaItem = getMediaItem(currentItemUid, currentMediaInfo); updateItemData( - currentItemId, + currentItemUid, mediaItem != null ? mediaItem : MediaItem.EMPTY, - mediaStatus.getMediaInfo(), - currentContentId, + currentMediaInfo, + currentMediaInfo.getContentId(), /* defaultPositionUs= */ C.TIME_UNSET); - for (MediaQueueItem queueItem : mediaStatus.getQueueItems()) { - long defaultPositionUs = (long) (queueItem.getStartTime() * C.MICROS_PER_SECOND); - @Nullable MediaInfo mediaInfo = queueItem.getMedia(); - String contentId = mediaInfo != null ? mediaInfo.getContentId() : UNKNOWN_CONTENT_ID; - mediaItem = mediaItemsByContentId.get(contentId); - updateItemData( - queueItem.getItemId(), - mediaItem != null ? mediaItem : mediaItemConverter.toMediaItem(queueItem), - mediaInfo, - contentId, - defaultPositionUs); + ImmutableList.Builder uids = ImmutableList.builderWithExpectedSize(itemIds.length); + for (int itemId : itemIds) { + uids.add(getOrCreateItemUid(itemId)); + } + return new CastTimeline(uids.build(), itemIdToData); + } + + /** + * Registers a media item with its corresponding synthetic ID. + * + * @param itemUid The unique {@link ItemUid} generated for the item. + * @param mediaItem The {@link MediaItem}. + * @param queueItem The {@link MediaQueueItem} associated with the media item. + */ + private void registerMediaItem(ItemUid itemUid, MediaItem mediaItem, MediaQueueItem queueItem) { + mediaItemsBySyntheticId.put(itemUid, mediaItem); + @Nullable MediaInfo mediaInfo = queueItem.getMedia(); + if (mediaInfo != null && mediaInfo.getContentId() != null) { + mediaItemsByContentId.put(mediaInfo.getContentId(), mediaItem); + } + } + + private void updateItemDataFromQueueItem(MediaQueueItem queueItem) { + long defaultPositionUs = (long) (queueItem.getStartTime() * C.MICROS_PER_SECOND); + @Nullable MediaInfo mediaInfo = queueItem.getMedia(); + String contentId = mediaInfo != null ? mediaInfo.getContentId() : UNKNOWN_CONTENT_ID; + ItemUid itemUid = getOrCreateItemUid(queueItem.getItemId(), queueItem.getMedia()); + @Nullable MediaItem mediaItem = getMediaItem(itemUid, mediaInfo); + if (mediaItem == null || mediaItem == MediaItem.EMPTY) { + try { + mediaItem = mediaItemConverter.toMediaItem(queueItem); + } catch (Exception e) { + // TODO(b/524966241): Remove try/catch once the converter doesn't throw parsing + // exceptions. + Log.w(TAG, "Failed to convert MediaQueueItem to MediaItem"); + } } - return new CastTimeline(itemIds, itemIdToData); + updateItemData( + itemUid, + mediaItem != null ? mediaItem : MediaItem.EMPTY, + mediaInfo, + contentId, + defaultPositionUs); } private void updateItemData( - int itemId, + ItemUid uid, MediaItem mediaItem, @Nullable MediaInfo mediaInfo, String contentId, long defaultPositionUs) { - CastTimeline.ItemData previousData = itemIdToData.get(itemId, CastTimeline.ItemData.EMPTY); + ItemData previousData = itemIdToData.get(uid); + if (previousData == null) { + previousData = ItemData.EMPTY; + } long durationUs = CastUtils.getStreamDurationUs(mediaInfo); if (durationUs == C.TIME_UNSET) { durationUs = previousData.durationUs; @@ -151,27 +306,135 @@ private void updateItemData( if (defaultPositionUs == C.TIME_UNSET) { defaultPositionUs = previousData.defaultPositionUs; } + // The media item can be empty if it originates from another sender. We use previous data as + // itemIdToData is populated when all media items from the queue are updated. + if (mediaItem == MediaItem.EMPTY) { + mediaItem = previousData.mediaItem; + } itemIdToData.put( - itemId, + uid, previousData.copyWithNewValues( durationUs, defaultPositionUs, isLive, mediaItem, contentId)); } private void removeUnusedItemDataEntries(int[] itemIds) { - HashSet scratchItemIds = new HashSet<>(/* initialCapacity= */ itemIds.length * 2); + HashSet activeReceiverIds = new HashSet<>(/* initialCapacity= */ itemIds.length * 2); for (int id : itemIds) { - scratchItemIds.add(id); + activeReceiverIds.add(id); } - int index = 0; - while (index < itemIdToData.size()) { - if (!scratchItemIds.contains(itemIdToData.keyAt(index))) { - CastTimeline.ItemData itemData = itemIdToData.valueAt(index); - mediaItemsByContentId.remove(itemData.contentId); - itemIdToData.removeAt(index); - } else { - index++; + for (int i = receiverItemIdToUid.size() - 1; i >= 0; i--) { + int receiverId = receiverItemIdToUid.keyAt(i); + if (!activeReceiverIds.contains(receiverId)) { + ItemUid uid = receiverItemIdToUid.valueAt(i); + receiverItemIdToUid.removeAt(i); + uidToReceiverItemId.remove(uid); + mediaItemsBySyntheticId.remove(uid); + ItemData data = itemIdToData.remove(uid); + if (data != null) { + mediaItemsByContentId.remove(data.contentId); + } } } } + + /** + * Returns the {@link MediaItem} associated with the given {@link ItemUid} and {@link MediaInfo}. + * The method tries to look up the media item using the {@link ItemUid} first and then the {@link + * MediaInfo#getContentId()} if the {@link ItemUid} is not found. + * + * @param uid The {@link ItemUid} of the media item. + * @param mediaInfo The {@link MediaInfo} of the media item. + * @return The {@link MediaItem} associated with the given {@link ItemUid} and {@link MediaInfo}, + * or {@code null} if not found. + */ + @Nullable + private MediaItem getMediaItem(ItemUid uid, @Nullable MediaInfo mediaInfo) { + MediaItem mediaItem = mediaItemsBySyntheticId.get(uid); + if (mediaItem != null) { + return mediaItem; + } + ItemData itemData = itemIdToData.get(uid); + if (itemData != null && itemData.mediaItem != MediaItem.EMPTY) { + return itemData.mediaItem; + } + if (mediaInfo != null && mediaInfo.getContentId() != null) { + return mediaItemsByContentId.get(mediaInfo.getContentId()); + } + return null; + } + + /** + * Returns the existing {@link ItemUid} for {@code receiverItemId}, or creates and caches a new + * one if none exists. + */ + private ItemUid getOrCreateItemUid(int receiverItemId) { + return getOrCreateItemUid(receiverItemId, /* mediaInfo= */ null); + } + + /** + * Returns the existing {@link ItemUid} for {@code receiverItemId}, or creates and caches a new + * one using synthetic ID extracted from {@code mediaInfo} (or a randomly generated ID if not + * present). + */ + private ItemUid getOrCreateItemUid(int receiverItemId, @Nullable MediaInfo mediaInfo) { + @Nullable ItemUid syntheticUid = getSyntheticItemUid(mediaInfo); + @Nullable ItemUid uid = receiverItemIdToUid.get(receiverItemId); + if (uid == null) { + // First time seeing this receiverItemId: use syntheticUid from MediaInfo if available, or + // generate a random UID (either as a temporary placeholder while MediaInfo is still being + // fetched so CastTimeline can be populated with the full queue size and playing index, or + // for items from a sender that does not attach synthetic IDs). + uid = syntheticUid != null ? syntheticUid : ItemUid.generateItemUid(); + } else if (syntheticUid != null + && !syntheticUid.equals(uid) + && mediaItemsBySyntheticId.containsKey(syntheticUid)) { + // This receiverItemId was initially assigned a temporary placeholder UID on an earlier pass + // before MediaQueue finished fetching its MediaInfo. Because syntheticUid was registered by + // this sender, maskedSnapshot is already using syntheticUid; swap the temporary UID to + // match maskedSnapshot and prevent recomposition once fetches complete. + uidToReceiverItemId.remove(uid); + itemIdToData.remove(uid); + uid = syntheticUid; + } else { + return uid; + } + receiverItemIdToUid.put(receiverItemId, uid); + uidToReceiverItemId.put(uid, receiverItemId); + return uid; + } + + /** + * Enriches the given {@link MediaQueueItem} by embedding the synthetic ID into its {@link + * MediaInfo#getCustomData()} at the top-level under key {@code "m3-syntheticId"}. If {@code + * customData} is {@code null}, indicating a custom converter, it is left unchanged. + */ + private static MediaQueueItem attachSyntheticId(MediaQueueItem queueItem, ItemUid syntheticId) { + MediaInfo mediaInfo = queueItem.getMedia(); + if (mediaInfo == null) { + return queueItem; + } + JSONObject customData = mediaInfo.getCustomData(); + if (customData != null) { + try { + customData.put(KEY_SYNTHETIC_ID, syntheticId.toString()); + } catch (JSONException e) { + Log.w(TAG, "Failed to attach syntheticId to customData"); + } + } + return queueItem; + } + + @Nullable + private static ItemUid getSyntheticItemUid(@Nullable MediaInfo mediaInfo) { + if (mediaInfo == null) { + return null; + } + @Nullable JSONObject customData = mediaInfo.getCustomData(); + if (customData == null || customData.isNull(KEY_SYNTHETIC_ID)) { + return null; + } + String syntheticId = customData.optString(KEY_SYNTHETIC_ID); + return !syntheticId.isEmpty() ? ItemUid.of(syntheticId) : null; + } } diff --git a/libraries/cast/src/main/java/androidx/media3/cast/CastTrackSelector.java b/libraries/cast/src/main/java/androidx/media3/cast/CastTrackSelector.java index ecee92fca09..d02f537857e 100644 --- a/libraries/cast/src/main/java/androidx/media3/cast/CastTrackSelector.java +++ b/libraries/cast/src/main/java/androidx/media3/cast/CastTrackSelector.java @@ -15,7 +15,7 @@ */ package androidx.media3.cast; -import static androidx.core.util.Preconditions.checkNotNull; +import static com.google.common.base.Preconditions.checkNotNull; import static java.lang.annotation.ElementType.TYPE_USE; import androidx.annotation.IntDef; diff --git a/libraries/cast/src/main/java/androidx/media3/cast/CastUtils.java b/libraries/cast/src/main/java/androidx/media3/cast/CastUtils.java index 24eef9a0885..3b6c7d9b637 100644 --- a/libraries/cast/src/main/java/androidx/media3/cast/CastUtils.java +++ b/libraries/cast/src/main/java/androidx/media3/cast/CastUtils.java @@ -17,16 +17,22 @@ import android.os.Looper; import androidx.annotation.Nullable; -import androidx.core.util.Preconditions; import androidx.media3.common.C; import androidx.media3.common.C.TrackType; import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; import androidx.media3.common.TrackGroup; +import androidx.media3.common.util.Log; import androidx.media3.common.util.Util; import com.google.android.gms.cast.CastStatusCodes; import com.google.android.gms.cast.MediaInfo; +import com.google.android.gms.cast.MediaQueueItem; +import com.google.android.gms.cast.MediaStatus; import com.google.android.gms.cast.MediaTrack; +import com.google.android.gms.cast.framework.media.RemoteMediaClient; +import com.google.android.gms.cast.framework.media.RemoteMediaClient.MediaChannelResult; +import com.google.android.gms.common.api.Status; +import com.google.common.base.Preconditions; /** Utility methods for Cast integration. */ /* package */ final class CastUtils { @@ -55,6 +61,49 @@ public static long getStreamDurationUs(@Nullable MediaInfo mediaInfo) { return durationMs != MediaInfo.UNKNOWN_DURATION ? Util.msToUs(durationMs) : C.TIME_UNSET; } + /** + * Returns the receiver queue item ID of the item currently loading or playing in {@code + * remoteMediaClient}, or {@link MediaQueueItem#INVALID_ITEM_ID} if none is active. + * + *

Prefers {@link MediaStatus#getLoadingItemId()} over {@link MediaStatus#getCurrentItemId()} + * because during an initial load or item transition on the receiver, {@code loadingItemId} + * identifies the target item being prepared while {@code currentItemId} may still reference the + * previous item or be {@link MediaQueueItem#INVALID_ITEM_ID}. + */ + public static int getCurrentOrLoadingItemId(RemoteMediaClient remoteMediaClient) { + int itemId = MediaQueueItem.INVALID_ITEM_ID; + @Nullable MediaStatus mediaStatus = remoteMediaClient.getMediaStatus(); + if (mediaStatus != null) { + itemId = mediaStatus.getLoadingItemId(); + if (itemId == MediaQueueItem.INVALID_ITEM_ID) { + itemId = mediaStatus.getCurrentItemId(); + } + } + if (itemId == MediaQueueItem.INVALID_ITEM_ID) { + @Nullable MediaQueueItem currentItem = remoteMediaClient.getCurrentItem(); + if (currentItem != null) { + itemId = currentItem.getItemId(); + } + } + return itemId; + } + + /** + * Logs an error if {@code result} has a non-null status that is neither {@link + * CastStatusCodes#SUCCESS} nor {@link CastStatusCodes#REPLACED}. + */ + public static void logOperationFailedIfStatusError( + String tag, String operation, @Nullable MediaChannelResult result) { + @Nullable Status status = result != null ? result.getStatus() : null; + if (status != null) { + int statusCode = status.getStatusCode(); + if (statusCode != CastStatusCodes.SUCCESS && statusCode != CastStatusCodes.REPLACED) { + Log.e( + tag, operation + " failed. Error code " + statusCode + ": " + getLogString(statusCode)); + } + } + } + /** * Returns a descriptive log string for the given {@code statusCode}, or "Unknown." if not one of * {@link CastStatusCodes}. diff --git a/libraries/cast/src/main/java/androidx/media3/cast/DefaultCastPlayerTransferCallback.java b/libraries/cast/src/main/java/androidx/media3/cast/DefaultCastPlayerTransferCallback.java new file mode 100644 index 00000000000..a46b651891f --- /dev/null +++ b/libraries/cast/src/main/java/androidx/media3/cast/DefaultCastPlayerTransferCallback.java @@ -0,0 +1,86 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.cast; + +import android.net.Uri; +import androidx.media3.common.C; +import androidx.media3.common.DeviceInfo; +import androidx.media3.common.MediaItem; +import androidx.media3.common.Player; +import androidx.media3.common.PlayerTransferState; +import androidx.media3.common.util.UnstableApi; +import com.google.common.collect.ImmutableList; +import java.util.List; + +/** + * The default {@link CastPlayer.TransferCallback} implementation. + * + *

Filters out {@link MediaItem}s that do not have a playback URI and adjusts the current media + * item index accordingly. + */ +@UnstableApi +public final class DefaultCastPlayerTransferCallback implements CastPlayer.TransferCallback { + + @Override + public void transferState(Player sourcePlayer, Player targetPlayer) { + if (targetPlayer.getDeviceInfo().playbackType == DeviceInfo.PLAYBACK_TYPE_REMOTE + && sourcePlayer.getCurrentTimeline().isEmpty()) { + // If the source player is empty, we don't transfer the state to the remote player. This + // prevents the local state from overwriting the remote player state when joining an existing + // session. + return; + } + PlayerTransferState transferState = PlayerTransferState.fromPlayer(sourcePlayer); + List mediaItems = transferState.getMediaItems(); + ImmutableList.Builder playableMediaItemsBuilder = ImmutableList.builder(); + int currentIndex = transferState.getCurrentMediaItemIndex(); + int newCurrentIndex = currentIndex; + boolean currentIndexFiltered = false; + for (int i = 0; i < mediaItems.size(); i++) { + MediaItem mediaItem = mediaItems.get(i); + if (isPlayable(mediaItem)) { + playableMediaItemsBuilder.add(mediaItem); + } else if (i < currentIndex) { + newCurrentIndex--; + } else if (i == currentIndex) { + currentIndexFiltered = true; + } + } + List playableMediaItems = playableMediaItemsBuilder.build(); + if (playableMediaItems.size() < mediaItems.size()) { + if (playableMediaItems.isEmpty()) { + newCurrentIndex = C.INDEX_UNSET; + } else { + newCurrentIndex = Math.min(newCurrentIndex, playableMediaItems.size() - 1); + } + PlayerTransferState.Builder builder = + transferState + .buildUpon() + .setMediaItems(playableMediaItems) + .setCurrentMediaItemIndex(newCurrentIndex); + if (currentIndexFiltered) { + builder.setCurrentPosition(0); + } + transferState = builder.build(); + } + transferState.setToPlayer(targetPlayer); + } + + private static boolean isPlayable(MediaItem mediaItem) { + return mediaItem.localConfiguration != null + && !Uri.EMPTY.equals(mediaItem.localConfiguration.uri); + } +} diff --git a/libraries/cast/src/main/java/androidx/media3/cast/DefaultMediaItemConverter.java b/libraries/cast/src/main/java/androidx/media3/cast/DefaultMediaItemConverter.java index 136e196f175..2110ef1e672 100644 --- a/libraries/cast/src/main/java/androidx/media3/cast/DefaultMediaItemConverter.java +++ b/libraries/cast/src/main/java/androidx/media3/cast/DefaultMediaItemConverter.java @@ -28,6 +28,7 @@ import com.google.android.gms.cast.MediaMetadata; import com.google.android.gms.cast.MediaQueueItem; import com.google.android.gms.common.images.WebImage; +import java.util.Calendar; import java.util.HashMap; import java.util.Iterator; import java.util.UUID; @@ -50,6 +51,12 @@ public final class DefaultMediaItemConverter implements MediaItemConverter { private static final String KEY_UUID = "uuid"; private static final String KEY_LICENSE_URI = "licenseUri"; private static final String KEY_REQUEST_HEADERS = "requestHeaders"; + private static final String KEY_LIVE_CONFIGURATION = "m3-liveConfiguration"; + private static final String KEY_TARGET_OFFSET_MS = "m3-targetOffsetMs"; + private static final String KEY_MIN_OFFSET_MS = "m3-minOffsetMs"; + private static final String KEY_MAX_OFFSET_MS = "m3-maxOffsetMs"; + private static final String KEY_MIN_PLAYBACK_SPEED = "m3-minPlaybackSpeed"; + private static final String KEY_MAX_PLAYBACK_SPEED = "m3-maxPlaybackSpeed"; @Override public MediaItem toMediaItem(MediaQueueItem mediaQueueItem) { @@ -86,6 +93,15 @@ public MediaItem toMediaItem(MediaQueueItem mediaQueueItem) { if (metadata.containsKey(MediaMetadata.KEY_TRACK_NUMBER)) { metadataBuilder.setTrackNumber(metadata.getInt(MediaMetadata.KEY_TRACK_NUMBER)); } + if (metadata.containsKey(MediaMetadata.KEY_RELEASE_DATE)) { + @Nullable Calendar releaseDate = metadata.getDate(MediaMetadata.KEY_RELEASE_DATE); + if (releaseDate != null) { + metadataBuilder.setReleaseYear(releaseDate.get(Calendar.YEAR)); + // Month is 0-indexed in Calendar, but 1-indexed in MediaMetadata. + metadataBuilder.setReleaseMonth(releaseDate.get(Calendar.MONTH) + 1); + metadataBuilder.setReleaseDay(releaseDate.get(Calendar.DAY_OF_MONTH)); + } + } } // TODO: b/526548538 - Get rid of custom keys in media3 when equivalent CastSDK fields are // present. @@ -162,17 +178,35 @@ public MediaQueueItem toMediaQueueItem(MediaItem mediaItem) { if (mediaItem.mediaMetadata.trackNumber != null) { metadata.putInt(MediaMetadata.KEY_TRACK_NUMBER, mediaItem.mediaMetadata.trackNumber); } + if (mediaItem.mediaMetadata.releaseYear != null) { + Calendar calendar = Calendar.getInstance(); + calendar.clear(); + calendar.set( + mediaItem.mediaMetadata.releaseYear, + mediaItem.mediaMetadata.releaseMonth != null + // Month is 0-indexed in Calendar. + ? mediaItem.mediaMetadata.releaseMonth - 1 + : 0, + mediaItem.mediaMetadata.releaseDay != null ? mediaItem.mediaMetadata.releaseDay : 1); + metadata.putDate(MediaMetadata.KEY_RELEASE_DATE, calendar); + } String contentUrl = mediaItem.localConfiguration.uri.toString(); String contentId = mediaItem.mediaId.equals(MediaItem.DEFAULT_MEDIA_ID) ? contentUrl : mediaItem.mediaId; - MediaInfo mediaInfo = + // If the MediaItem has a LiveConfiguration, we set the stream type to STREAM_TYPE_LIVE. + // Otherwise, we leave it unset letting the receiver decide whether the item is live. + boolean isExplicitlyLive = + !mediaItem.liveConfiguration.equals(MediaItem.LiveConfiguration.UNSET); + MediaInfo.Builder mediaInfoBuilder = new MediaInfo.Builder(contentId) - .setStreamType(MediaInfo.STREAM_TYPE_BUFFERED) .setContentType(mediaItem.localConfiguration.mimeType) .setContentUrl(contentUrl) .setMetadata(metadata) - .setCustomData(getCustomData(mediaItem)) - .build(); + .setCustomData(getCustomData(mediaItem)); + if (isExplicitlyLive) { + mediaInfoBuilder.setStreamType(MediaInfo.STREAM_TYPE_LIVE); + } + MediaInfo mediaInfo = mediaInfoBuilder.build(); return new MediaQueueItem.Builder(mediaInfo).build(); } @@ -231,12 +265,37 @@ private static MediaItem getMediaItem( if (mediaItemJson.has(KEY_DRM_CONFIGURATION)) { populateDrmConfiguration(mediaItemJson.getJSONObject(KEY_DRM_CONFIGURATION), builder); } + if (mediaItemJson.has(KEY_LIVE_CONFIGURATION)) { + builder.setLiveConfiguration( + getLiveConfiguration(mediaItemJson.getJSONObject(KEY_LIVE_CONFIGURATION))); + } return builder.build(); } catch (JSONException e) { throw new RuntimeException(e); } } + private static MediaItem.LiveConfiguration getLiveConfiguration(JSONObject json) + throws JSONException { + MediaItem.LiveConfiguration.Builder builder = new MediaItem.LiveConfiguration.Builder(); + if (json.has(KEY_TARGET_OFFSET_MS)) { + builder.setTargetOffsetMs(json.getLong(KEY_TARGET_OFFSET_MS)); + } + if (json.has(KEY_MIN_OFFSET_MS)) { + builder.setMinOffsetMs(json.getLong(KEY_MIN_OFFSET_MS)); + } + if (json.has(KEY_MAX_OFFSET_MS)) { + builder.setMaxOffsetMs(json.getLong(KEY_MAX_OFFSET_MS)); + } + if (json.has(KEY_MIN_PLAYBACK_SPEED)) { + builder.setMinPlaybackSpeed((float) json.getDouble(KEY_MIN_PLAYBACK_SPEED)); + } + if (json.has(KEY_MAX_PLAYBACK_SPEED)) { + builder.setMaxPlaybackSpeed((float) json.getDouble(KEY_MAX_PLAYBACK_SPEED)); + } + return builder.build(); + } + private static void populateDrmConfiguration(JSONObject json, MediaItem.Builder mediaItem) throws JSONException { MediaItem.DrmConfiguration.Builder drmConfiguration = @@ -280,6 +339,31 @@ private static JSONObject getMediaItemJson(MediaItem mediaItem) throws JSONExcep KEY_DRM_CONFIGURATION, getDrmConfigurationJson(mediaItem.localConfiguration.drmConfiguration)); } + if (!mediaItem.liveConfiguration.equals(MediaItem.LiveConfiguration.UNSET)) { + JSONObject liveConfigurationJson = getLiveConfigurationJson(mediaItem.liveConfiguration); + json.put(KEY_LIVE_CONFIGURATION, liveConfigurationJson); + } + return json; + } + + private static JSONObject getLiveConfigurationJson(MediaItem.LiveConfiguration liveConfiguration) + throws JSONException { + JSONObject json = new JSONObject(); + if (liveConfiguration.targetOffsetMs != C.TIME_UNSET) { + json.put(KEY_TARGET_OFFSET_MS, liveConfiguration.targetOffsetMs); + } + if (liveConfiguration.minOffsetMs != C.TIME_UNSET) { + json.put(KEY_MIN_OFFSET_MS, liveConfiguration.minOffsetMs); + } + if (liveConfiguration.maxOffsetMs != C.TIME_UNSET) { + json.put(KEY_MAX_OFFSET_MS, liveConfiguration.maxOffsetMs); + } + if (liveConfiguration.minPlaybackSpeed != C.RATE_UNSET) { + json.put(KEY_MIN_PLAYBACK_SPEED, liveConfiguration.minPlaybackSpeed); + } + if (liveConfiguration.maxPlaybackSpeed != C.RATE_UNSET) { + json.put(KEY_MAX_PLAYBACK_SPEED, liveConfiguration.maxPlaybackSpeed); + } return json; } diff --git a/libraries/cast/src/main/java/androidx/media3/cast/MediaRouteButton.kt b/libraries/cast/src/main/java/androidx/media3/cast/MediaRouteButton.kt index 4594507fa79..d673ffb1c4d 100644 --- a/libraries/cast/src/main/java/androidx/media3/cast/MediaRouteButton.kt +++ b/libraries/cast/src/main/java/androidx/media3/cast/MediaRouteButton.kt @@ -27,7 +27,6 @@ import androidx.compose.material3.Icon import androidx.compose.material3.IconButton import androidx.compose.runtime.Composable import androidx.compose.runtime.DisposableEffect -import androidx.compose.runtime.LaunchedEffect import androidx.compose.runtime.getValue import androidx.compose.runtime.mutableStateOf import androidx.compose.runtime.remember @@ -174,7 +173,7 @@ private fun MediaRouterState.MediaRouteControllerDialog(onDismissRequest: () -> internal fun MediaRouteButtonContainer(content: @Composable MediaRouterState.() -> Unit) { val context = LocalContext.current var selector by remember { mutableStateOf(MediaRouteSelector.EMPTY) } - LaunchedEffect(context) { + DisposableEffect(context) { val cast = Cast.getSingletonInstance(context) cast.ensureInitialized(context) val mediaRouteSelectorListener: MediaRouteSelectorListener = @@ -187,6 +186,7 @@ internal fun MediaRouteButtonContainer(content: @Composable MediaRouterState.() if (currentSelector != null) { selector = currentSelector } + onDispose { cast.unregisterListener(mediaRouteSelectorListener) } } if (!selector.isEmpty) { rememberMediaRouterState(context, selector).content() diff --git a/libraries/cast/src/main/java/androidx/media3/cast/QueuedOperation.java b/libraries/cast/src/main/java/androidx/media3/cast/QueuedOperation.java new file mode 100644 index 00000000000..ba0a33de89e --- /dev/null +++ b/libraries/cast/src/main/java/androidx/media3/cast/QueuedOperation.java @@ -0,0 +1,671 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.cast; + +import static androidx.media3.cast.CastTimeline.ItemData.UNKNOWN_CONTENT_ID; +import static com.google.common.base.Preconditions.checkArgument; +import static java.lang.Math.max; +import static java.lang.Math.min; + +import androidx.annotation.Nullable; +import androidx.media3.cast.CastTimeline.ItemData; +import androidx.media3.cast.CastTimeline.ItemUid; +import androidx.media3.common.C; +import androidx.media3.common.MediaItem; +import androidx.media3.common.Player; +import androidx.media3.common.PlayerTransferState; +import androidx.media3.common.Timeline; +import com.google.android.gms.cast.MediaLoadRequestData; +import com.google.android.gms.cast.MediaQueueData; +import com.google.android.gms.cast.MediaQueueItem; +import com.google.android.gms.cast.MediaStatus; +import com.google.android.gms.cast.framework.media.RemoteMediaClient; +import com.google.android.gms.cast.framework.media.RemoteMediaClient.MediaChannelResult; +import com.google.android.gms.common.api.PendingResult; +import com.google.common.collect.ImmutableList; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +/** Encapsulates a mutating operation on the Cast media queue or playback position. */ +/* package */ abstract class QueuedOperation { + + /** + * Immutable snapshot of the queue state (timeline, current window index, and masked playback + * position). + */ + public static final class QueueSnapshot { + public static final QueueSnapshot EMPTY = + new QueueSnapshot( + CastTimeline.EMPTY_CAST_TIMELINE, + /* currentWindowIndex= */ 0, + /* maskedPositionMs= */ C.TIME_UNSET); + + public final CastTimeline timeline; + public final int currentWindowIndex; + + /** + * The optimistically masked playback position in milliseconds, or {@link C#TIME_UNSET} if no + * operation involved in the creation of this snapshot affects the playback position (in which + * case the receiver-reported playback position can be used). + */ + public final long maskedPositionMs; + + public QueueSnapshot(CastTimeline timeline, int currentWindowIndex, long maskedPositionMs) { + this.timeline = timeline; + this.currentWindowIndex = currentWindowIndex; + this.maskedPositionMs = maskedPositionMs; + } + + @Override + public boolean equals(@Nullable Object obj) { + if (this == obj) { + return true; + } + if (!(obj instanceof QueueSnapshot)) { + return false; + } + QueueSnapshot other = (QueueSnapshot) obj; + return timeline.equals(other.timeline) + && currentWindowIndex == other.currentWindowIndex + && maskedPositionMs == other.maskedPositionMs; + } + + @Override + public int hashCode() { + int result = timeline.hashCode(); + result = 31 * result + currentWindowIndex; + result = 31 * result + Long.hashCode(maskedPositionMs); + return result; + } + } + + /** Returns a human-readable label for logging. */ + public abstract String getName(); + + /** + * Returns whether this operation replaces all prior pending operations (for example, {@link + * SetMediaItemsOperation}). + */ + public boolean replacesPreviousOperations() { + return false; + } + + /** + * Applies this operation's optimistic changes onto {@code snapshot} and returns the resulting + * {@link QueueSnapshot}. + */ + public abstract QueueSnapshot createMaskedSnapshot( + QueueSnapshot snapshot, CastTimelineTracker timelineTracker); + + /** + * Returns whether the operation is ready to be executed. + * + *

For example, an operation is not ready to be executed if it targets an item whose synthetic + * UID has not yet been mapped to a receiver item ID (such as when a preceding {@link + * AddMediaItemsOperation} or {@link SetMediaItemsOperation} has not yet been confirmed by the + * receiver). + */ + public abstract boolean canExecute( + CastTimelineTracker timelineTracker, RemoteMediaClient remoteMediaClient); + + /** + * Executes the operation and returns the {@link PendingResult}. + * + * @param remoteMediaClient The {@link RemoteMediaClient} on which to execute the operation. + * @param timelineTracker The {@link CastTimelineTracker} used to resolve or register items. + * @param playerTransferState The {@link PlayerTransferState} at the moment of execution, for + * operations that require the current player state when dispatched. + */ + public abstract PendingResult execute( + RemoteMediaClient remoteMediaClient, + CastTimelineTracker timelineTracker, + PlayerTransferState playerTransferState); + + /** + * Base operation for commands that register and insert new {@link MediaItem MediaItems} into the + * queue. + */ + /* package */ abstract static class MediaItemsInsertionOperation extends QueuedOperation { + protected final ImmutableList mediaItems; + private ImmutableList itemUids; + protected MediaQueueItem[] queueItems; + + protected MediaItemsInsertionOperation(ImmutableList mediaItems) { + this.mediaItems = mediaItems; + this.itemUids = ImmutableList.of(); + this.queueItems = new MediaQueueItem[0]; + } + + protected final CastTimeline registerAndCreateMaskingTimeline( + CastTimeline currentTimeline, int index, CastTimelineTracker timelineTracker) { + if (itemUids.isEmpty() && !mediaItems.isEmpty()) { + CastTimelineTracker.RegisteredMediaItems registeredItems = + timelineTracker.registerMediaItems(mediaItems); + this.itemUids = registeredItems.itemUids; + this.queueItems = registeredItems.queueItems; + } + return createMaskingTimeline(currentTimeline, index, itemUids, mediaItems); + } + + protected final PendingResult executeLoad( + RemoteMediaClient remoteMediaClient, + PlayerTransferState playerTransferState, + int startIndex, + long startPositionMs) { + MediaQueueData mediaQueueData = + new MediaQueueData.Builder() + .setItems(Arrays.asList(queueItems)) + .setStartIndex(startIndex) + .setRepeatMode(toCastRepeatMode(playerTransferState.getRepeatMode())) + .setStartTime(startPositionMs) + .build(); + MediaLoadRequestData loadRequestData = + new MediaLoadRequestData.Builder() + .setAutoplay(playerTransferState.getPlayWhenReady()) + .setQueueData(mediaQueueData) + .setCurrentTime(startPositionMs) + .build(); + return remoteMediaClient.load(loadRequestData); + } + + private static int toCastRepeatMode(@Player.RepeatMode int repeatMode) { + switch (repeatMode) { + case Player.REPEAT_MODE_ONE: + return MediaStatus.REPEAT_MODE_REPEAT_SINGLE; + case Player.REPEAT_MODE_ALL: + return MediaStatus.REPEAT_MODE_REPEAT_ALL; + case Player.REPEAT_MODE_OFF: + return MediaStatus.REPEAT_MODE_REPEAT_OFF; + default: + throw new IllegalArgumentException(); + } + } + + private static CastTimeline createMaskingTimeline( + CastTimeline currentTimeline, + int index, + List itemUids, + List mediaItems) { + checkArgument(itemUids.size() == mediaItems.size()); + int currentWindowCount = currentTimeline.getWindowCount(); + int insertIndex = min(max(0, index), currentWindowCount); + int addedCount = mediaItems.size(); + int newWindowCount = currentWindowCount + addedCount; + if (newWindowCount == 0) { + return CastTimeline.EMPTY_CAST_TIMELINE; + } + + List newUids = new ArrayList<>(newWindowCount); + Map newItemIdToData = new HashMap<>(); + Timeline.Window window = new Timeline.Window(); + + for (int i = 0; i < currentWindowCount; i++) { + ItemUid uid = (ItemUid) currentTimeline.getWindow(i, window).uid; + newItemIdToData.put( + uid, + new ItemData( + window.durationUs, + window.defaultPositionUs, + window.isLive(), + window.mediaItem, + UNKNOWN_CONTENT_ID)); + } + + for (int i = 0; i < insertIndex; i++) { + newUids.add((ItemUid) currentTimeline.getWindow(i, window).uid); + } + + for (int i = 0; i < addedCount; i++) { + ItemUid uid = itemUids.get(i); + if (currentTimeline.getIndexOfPeriod(uid) != C.INDEX_UNSET) { + // Receiver status updates can arrive before the PendingResult callback for + // queueInsertItems. Skip items that are already present in currentTimeline to avoid + // inserting them a second time when recomputing the masked snapshot. + continue; + } + MediaItem mediaItem = mediaItems.get(i); + newUids.add(uid); + newItemIdToData.put( + uid, + new ItemData( + /* durationUs= */ C.TIME_UNSET, + /* defaultPositionUs= */ mediaItem.clippingConfiguration.startPositionUs, + /* isLive= */ false, + mediaItem, + UNKNOWN_CONTENT_ID)); + } + + for (int i = insertIndex; i < currentWindowCount; i++) { + newUids.add((ItemUid) currentTimeline.getWindow(i, window).uid); + } + + if (newUids.size() == currentWindowCount) { + return currentTimeline; + } + + return new CastTimeline(newUids, newItemIdToData); + } + } + + /** Command for {@code setMediaItems(...)}. */ + public static final class SetMediaItemsOperation extends MediaItemsInsertionOperation { + private final int startIndex; + private final long startPositionMs; + + public SetMediaItemsOperation( + ImmutableList mediaItems, int startIndex, long startPositionMs) { + super(mediaItems); + this.startIndex = mediaItems.isEmpty() ? 0 : min(startIndex, mediaItems.size() - 1); + this.startPositionMs = startPositionMs; + } + + @Override + public String getName() { + return "Load"; + } + + @Override + public boolean replacesPreviousOperations() { + return true; + } + + @Override + public QueueSnapshot createMaskedSnapshot( + QueueSnapshot snapshot, CastTimelineTracker timelineTracker) { + CastTimeline maskedTimeline = + registerAndCreateMaskingTimeline( + CastTimeline.EMPTY_CAST_TIMELINE, /* index= */ 0, timelineTracker); + return new QueueSnapshot(maskedTimeline, startIndex, startPositionMs); + } + + @Override + public boolean canExecute( + CastTimelineTracker timelineTracker, RemoteMediaClient remoteMediaClient) { + return true; + } + + @Override + public PendingResult execute( + RemoteMediaClient remoteMediaClient, + CastTimelineTracker timelineTracker, + PlayerTransferState playerTransferState) { + return executeLoad(remoteMediaClient, playerTransferState, startIndex, startPositionMs); + } + } + + /** Command for {@code addMediaItems(...)}. */ + public static final class AddMediaItemsOperation extends MediaItemsInsertionOperation { + @Nullable private final Object insertBeforePeriodUid; + + public AddMediaItemsOperation( + @Nullable Object insertBeforePeriodUid, ImmutableList mediaItems) { + super(mediaItems); + this.insertBeforePeriodUid = insertBeforePeriodUid; + } + + @Override + public String getName() { + return "Insert items"; + } + + @Override + public QueueSnapshot createMaskedSnapshot( + QueueSnapshot snapshot, CastTimelineTracker timelineTracker) { + CastTimeline currentTimeline = snapshot.timeline; + int effectiveIndex; + if (currentTimeline.isEmpty()) { + effectiveIndex = 0; + } else if (insertBeforePeriodUid != null) { + effectiveIndex = currentTimeline.getIndexOfPeriod(insertBeforePeriodUid); + if (effectiveIndex == C.INDEX_UNSET) { + return snapshot; + } + } else { + effectiveIndex = currentTimeline.getWindowCount(); + } + Timeline.Window window = new Timeline.Window(); + @Nullable + Object playingUid = + !currentTimeline.isEmpty() + ? currentTimeline.getWindow(snapshot.currentWindowIndex, window).uid + : null; + CastTimeline newTimeline = + registerAndCreateMaskingTimeline(currentTimeline, effectiveIndex, timelineTracker); + if (newTimeline.equals(currentTimeline)) { + return snapshot; + } + int newWindowIndex = 0; + if (playingUid != null) { + int idx = newTimeline.getIndexOfPeriod(playingUid); + newWindowIndex = idx != C.INDEX_UNSET ? idx : 0; + } + return new QueueSnapshot(newTimeline, newWindowIndex, snapshot.maskedPositionMs); + } + + @Override + public boolean canExecute( + CastTimelineTracker timelineTracker, RemoteMediaClient remoteMediaClient) { + if (isReceiverQueueEmpty(remoteMediaClient)) { + return true; + } + if (insertBeforePeriodUid != null) { + return timelineTracker.getReceiverItemId(insertBeforePeriodUid) + != MediaQueueItem.INVALID_ITEM_ID; + } + return true; + } + + @Override + public PendingResult execute( + RemoteMediaClient remoteMediaClient, + CastTimelineTracker timelineTracker, + PlayerTransferState playerTransferState) { + if (isReceiverQueueEmpty(remoteMediaClient)) { + return executeLoad( + remoteMediaClient, playerTransferState, /* startIndex= */ 0, /* startPositionMs= */ 0); + } + int insertBeforeItemId = + insertBeforePeriodUid != null + ? timelineTracker.getReceiverItemId(insertBeforePeriodUid) + : MediaQueueItem.INVALID_ITEM_ID; + return remoteMediaClient.queueInsertItems( + queueItems, insertBeforeItemId, /* customData= */ null); + } + + private static boolean isReceiverQueueEmpty(RemoteMediaClient remoteMediaClient) { + return remoteMediaClient.getMediaStatus() == null + || remoteMediaClient.getMediaQueue().getItemIds().length == 0; + } + } + + /** Command for {@code moveMediaItems(...)}. */ + public static final class MoveMediaItemsOperation extends QueuedOperation { + private final ImmutableList uidsToMove; + @Nullable private final Object insertBeforePeriodUid; + + public MoveMediaItemsOperation( + List uidsToMove, @Nullable Object insertBeforePeriodUid) { + this.uidsToMove = ImmutableList.copyOf(uidsToMove); + this.insertBeforePeriodUid = insertBeforePeriodUid; + } + + @Override + public String getName() { + return "Reorder items"; + } + + @Override + public QueueSnapshot createMaskedSnapshot( + QueueSnapshot snapshot, CastTimelineTracker timelineTracker) { + CastTimeline currentTimeline = snapshot.timeline; + int playlistSize = currentTimeline.getWindowCount(); + if (playlistSize == 0 || uidsToMove.isEmpty()) { + return snapshot; + } + if (insertBeforePeriodUid != null + && currentTimeline.getIndexOfPeriod(insertBeforePeriodUid) == C.INDEX_UNSET) { + return snapshot; + } + + List currentUids = new ArrayList<>(playlistSize); + List remainingUids = new ArrayList<>(playlistSize); + Map newItemIdToData = new HashMap<>(); + Timeline.Window window = new Timeline.Window(); + + for (int i = 0; i < playlistSize; i++) { + ItemUid uid = (ItemUid) currentTimeline.getWindow(i, window).uid; + currentUids.add(uid); + if (!uidsToMove.contains(uid)) { + remainingUids.add(uid); + } + newItemIdToData.put( + uid, + new ItemData( + window.durationUs, + window.defaultPositionUs, + window.isLive(), + window.mediaItem, + UNKNOWN_CONTENT_ID)); + } + + List movedUids = new ArrayList<>(uidsToMove.size()); + for (int i = 0; i < uidsToMove.size(); i++) { + Object uid = uidsToMove.get(i); + if (uid instanceof ItemUid && newItemIdToData.containsKey(uid)) { + movedUids.add((ItemUid) uid); + } + } + if (movedUids.isEmpty()) { + return snapshot; + } + + int insertPos = + insertBeforePeriodUid != null + ? remainingUids.indexOf(insertBeforePeriodUid) + : remainingUids.size(); + if (insertPos == -1) { + return snapshot; + } + remainingUids.addAll(insertPos, movedUids); + + if (remainingUids.equals(currentUids)) { + return snapshot; + } + + Object playingUid = currentUids.get(snapshot.currentWindowIndex); + CastTimeline newTimeline = new CastTimeline(remainingUids, newItemIdToData); + int idx = newTimeline.getIndexOfPeriod(playingUid); + int newWindowIndex = idx != C.INDEX_UNSET ? idx : 0; + return new QueueSnapshot(newTimeline, newWindowIndex, snapshot.maskedPositionMs); + } + + @Override + public boolean canExecute( + CastTimelineTracker timelineTracker, RemoteMediaClient remoteMediaClient) { + if (remoteMediaClient.getMediaStatus() == null) { + return false; + } + for (Object uid : uidsToMove) { + if (timelineTracker.getReceiverItemId(uid) == MediaQueueItem.INVALID_ITEM_ID) { + return false; + } + } + if (insertBeforePeriodUid != null + && timelineTracker.getReceiverItemId(insertBeforePeriodUid) + == MediaQueueItem.INVALID_ITEM_ID) { + return false; + } + return true; + } + + @Override + public PendingResult execute( + RemoteMediaClient remoteMediaClient, + CastTimelineTracker timelineTracker, + PlayerTransferState playerTransferState) { + int[] receiverUids = new int[uidsToMove.size()]; + for (int i = 0; i < uidsToMove.size(); i++) { + receiverUids[i] = timelineTracker.getReceiverItemId(uidsToMove.get(i)); + } + int receiverInsertBeforeId = + insertBeforePeriodUid != null + ? timelineTracker.getReceiverItemId(insertBeforePeriodUid) + : MediaQueueItem.INVALID_ITEM_ID; + return remoteMediaClient.queueReorderItems( + receiverUids, receiverInsertBeforeId, /* customData= */ null); + } + } + + /** Command for {@code removeMediaItems(...)}. */ + public static final class RemoveMediaItemsOperation extends QueuedOperation { + private final ImmutableList uidsToRemove; + + public RemoveMediaItemsOperation(List uidsToRemove) { + this.uidsToRemove = ImmutableList.copyOf(uidsToRemove); + } + + @Override + public String getName() { + return "Remove items"; + } + + @Override + public QueueSnapshot createMaskedSnapshot( + QueueSnapshot snapshot, CastTimelineTracker timelineTracker) { + CastTimeline currentTimeline = snapshot.timeline; + if (currentTimeline.isEmpty() || uidsToRemove.isEmpty()) { + return snapshot; + } + int currentWindowCount = currentTimeline.getWindowCount(); + int currentWindowIndex = snapshot.currentWindowIndex; + Timeline.Window window = new Timeline.Window(); + Object playingUid = currentTimeline.getWindow(currentWindowIndex, window).uid; + boolean playingPeriodRemoved = uidsToRemove.contains(playingUid); + + List newUids = new ArrayList<>(currentWindowCount); + Map newItemIdToData = new HashMap<>(); + int removedBefore = 0; + int removedCount = 0; + + for (int i = 0; i < currentWindowCount; i++) { + ItemUid uid = (ItemUid) currentTimeline.getWindow(i, window).uid; + if (uidsToRemove.contains(uid)) { + removedCount++; + if (i < currentWindowIndex) { + removedBefore++; + } + continue; + } + newUids.add(uid); + newItemIdToData.put( + uid, + new ItemData( + window.durationUs, + window.defaultPositionUs, + window.isLive(), + window.mediaItem, + UNKNOWN_CONTENT_ID)); + } + + if (removedCount == 0) { + return snapshot; + } + + CastTimeline newTimeline = + newUids.isEmpty() + ? CastTimeline.EMPTY_CAST_TIMELINE + : new CastTimeline(newUids, newItemIdToData); + int newWindowIndex = + !newTimeline.isEmpty() + ? min(currentWindowIndex - removedBefore, newTimeline.getWindowCount() - 1) + : 0; + long newMaskedPositionMs = + playingPeriodRemoved + ? (!newTimeline.isEmpty() + ? newTimeline.getWindow(newWindowIndex, window).getDefaultPositionMs() + : 0L) + : snapshot.maskedPositionMs; + return new QueueSnapshot(newTimeline, newWindowIndex, newMaskedPositionMs); + } + + @Override + public boolean canExecute( + CastTimelineTracker timelineTracker, RemoteMediaClient remoteMediaClient) { + if (remoteMediaClient.getMediaStatus() == null) { + return false; + } + for (Object uid : uidsToRemove) { + if (timelineTracker.getReceiverItemId(uid) == MediaQueueItem.INVALID_ITEM_ID) { + return false; + } + } + return true; + } + + @Override + public PendingResult execute( + RemoteMediaClient remoteMediaClient, + CastTimelineTracker timelineTracker, + PlayerTransferState playerTransferState) { + int[] receiverUids = new int[uidsToRemove.size()]; + for (int i = 0; i < uidsToRemove.size(); i++) { + receiverUids[i] = timelineTracker.getReceiverItemId(uidsToRemove.get(i)); + } + return remoteMediaClient.queueRemoveItems(receiverUids, /* customData= */ null); + } + } + + /** Command for {@code seekTo(...)}. */ + public static final class SeekOperation extends QueuedOperation { + private final long positionMs; + private final Object targetPeriodUid; + private final boolean isWindowChange; + + public SeekOperation(long positionMs, Object targetPeriodUid, boolean isWindowChange) { + this.positionMs = positionMs; + this.targetPeriodUid = targetPeriodUid; + this.isWindowChange = isWindowChange; + } + + @Override + public String getName() { + return "Seek"; + } + + @Override + public QueueSnapshot createMaskedSnapshot( + QueueSnapshot snapshot, CastTimelineTracker timelineTracker) { + CastTimeline timeline = snapshot.timeline; + if (timeline.isEmpty()) { + return new QueueSnapshot(timeline, /* currentWindowIndex= */ 0, positionMs); + } + int resolvedIndex = timeline.getIndexOfPeriod(targetPeriodUid); + if (resolvedIndex != C.INDEX_UNSET) { + return new QueueSnapshot(timeline, resolvedIndex, positionMs); + } + int fallbackIndex = min(snapshot.currentWindowIndex, timeline.getWindowCount() - 1); + return new QueueSnapshot(timeline, fallbackIndex, C.TIME_UNSET); + } + + @Override + public boolean canExecute( + CastTimelineTracker timelineTracker, RemoteMediaClient remoteMediaClient) { + if (remoteMediaClient.getMediaStatus() == null) { + return false; + } + if (isWindowChange) { + return timelineTracker.getReceiverItemId(targetPeriodUid) != MediaQueueItem.INVALID_ITEM_ID; + } + return true; + } + + @Override + public PendingResult execute( + RemoteMediaClient remoteMediaClient, + CastTimelineTracker timelineTracker, + PlayerTransferState playerTransferState) { + if (isWindowChange) { + int receiverItemId = timelineTracker.getReceiverItemId(targetPeriodUid); + return remoteMediaClient.queueJumpToItem( + receiverItemId, positionMs, /* customData= */ null); + } + return remoteMediaClient.seek(positionMs); + } + } +} diff --git a/libraries/cast/src/main/java/androidx/media3/cast/RemoteCastPlayer.java b/libraries/cast/src/main/java/androidx/media3/cast/RemoteCastPlayer.java index 17cb16f066e..21523edfe2c 100644 --- a/libraries/cast/src/main/java/androidx/media3/cast/RemoteCastPlayer.java +++ b/libraries/cast/src/main/java/androidx/media3/cast/RemoteCastPlayer.java @@ -19,7 +19,6 @@ import static androidx.media3.cast.CastTrackSelector.TRACK_SELECTION_REQUEST_REASON_INVALIDATION; import static androidx.media3.cast.CastTrackSelector.TRACK_SELECTION_REQUEST_REASON_PARAMETER_CHANGE; import static androidx.media3.cast.CastTrackSelector.TRACK_SELECTION_REQUEST_REASON_RECEIVER_UPDATE; -import static androidx.media3.common.util.Util.castNonNull; import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.base.Preconditions.checkState; @@ -31,6 +30,7 @@ import android.media.MediaRouter2.RoutingController; import android.media.MediaRouter2.TransferCallback; import android.media.RouteDiscoveryPreference; +import android.media.RoutingSessionInfo; import android.os.Handler; import android.os.Looper; import android.util.Range; @@ -38,6 +38,7 @@ import android.view.SurfaceHolder; import android.view.SurfaceView; import android.view.TextureView; +import androidx.annotation.DoNotInline; import androidx.annotation.IntRange; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; @@ -49,12 +50,14 @@ import androidx.media3.common.BasePlayer; import androidx.media3.common.C; import androidx.media3.common.DeviceInfo; +import androidx.media3.common.Flags; import androidx.media3.common.MediaItem; import androidx.media3.common.MediaLibraryInfo; import androidx.media3.common.MediaMetadata; import androidx.media3.common.PlaybackException; import androidx.media3.common.PlaybackParameters; import androidx.media3.common.Player; +import androidx.media3.common.PlayerTransferState; import androidx.media3.common.Timeline; import androidx.media3.common.TrackGroup; import androidx.media3.common.TrackSelectionParameters; @@ -69,8 +72,6 @@ import androidx.media3.common.util.Util; import com.google.android.gms.cast.CastStatusCodes; import com.google.android.gms.cast.MediaInfo; -import com.google.android.gms.cast.MediaLoadRequestData; -import com.google.android.gms.cast.MediaQueueData; import com.google.android.gms.cast.MediaQueueItem; import com.google.android.gms.cast.MediaStatus; import com.google.android.gms.cast.MediaTrack; @@ -89,7 +90,6 @@ import com.google.errorprone.annotations.InlineMe; import java.io.IOException; import java.util.ArrayList; -import java.util.Arrays; import java.util.List; import java.util.Objects; import org.checkerframework.checker.nullness.qual.RequiresNonNull; @@ -98,8 +98,9 @@ * {@link Player} implementation that communicates with a Cast receiver app. * *

The behavior of this class depends on the underlying Cast session, which is obtained from the - * injected {@link CastContext}. To keep track of the session, {@link #isCastSessionAvailable()} can - * be queried and {@link SessionAvailabilityListener} can be implemented and attached to the player. + * injected {@link com.google.android.gms.cast.framework.CastContext}. To keep track of the session, + * {@link #isCastSessionAvailable()} can be queried and {@link SessionAvailabilityListener} can be + * implemented and attached to the player. * *

If no session is available, the player state will remain unchanged and calls to methods that * alter it will be ignored. Querying the player state is possible even when no session is @@ -318,6 +319,25 @@ public RemoteCastPlayer build() { private static final long PROGRESS_REPORT_PERIOD_MS = 1000; private static final long[] EMPTY_TRACK_ID_ARRAY = new long[0]; + /** + * GMS error code 2055 corresponds to {@link CastStatusCodes#APPLICATION_STOPPED}. This value is + * not part of the public API but is used when the receiver application stops. + */ + @VisibleForTesting /* package */ static final int APPLICATION_STOPPED = 2055; + + /** + * GMS error code 2155 corresponds to {@link + * CastStatusCodes#SESSION_ENDED_WITH_REASON_ROUTE_CHANGED}. This value is not part of the public + * API but is used when the route playing the media is changed. + */ + @VisibleForTesting /* package */ static final int SESSION_END_REASON_ROUTE_CHANGE = 2155; + + /** + * GMS error code 2154 corresponds to {@link CastStatusCodes#SESSION_ENDED_WITH_REASON_STOPPED}. + * This value is not part of the public API but is used when the stop casting button is pressed. + */ + @VisibleForTesting /* package */ static final int SESSION_END_REASON_STOPPED = 2154; + private final Cast cast; private final MediaItemConverter mediaItemConverter; @Nullable private final CastTrackSelector trackSelector; @@ -325,8 +345,6 @@ public RemoteCastPlayer build() { private final long seekBackIncrementMs; private final long seekForwardIncrementMs; private final long maxSeekToPreviousPositionMs; - // TODO: Allow custom implementations of CastTimelineTracker. - private final CastTimelineTracker timelineTracker; private final Timeline.Period period; @Nullable private final Api30Impl api30Impl; @@ -335,7 +353,6 @@ public RemoteCastPlayer build() { private final StatusListener statusListener; private final MediaQueue.Callback mediaQueueCallback; - private final SeekResultCallback seekResultCallback; // Listeners and notification. private final ListenerSet listeners; @@ -346,22 +363,20 @@ public RemoteCastPlayer build() { private final StateHolder playWhenReady; private final StateHolder repeatMode; private boolean isMuted; + private boolean isSessionEnding; private int deviceVolume; private final StateHolder volume; + private float unmuteVolume; private final StateHolder playbackParameters; @Nullable private CastSession castSession; @Nullable private RemoteMediaClient remoteMediaClient; - private CastTimeline currentTimeline; + private final CastQueueOrchestrator queueOrchestrator; + private QueuedOperation.QueueSnapshot currentQueueSnapshot; private final StateHolder currentTracks; private final StateHolder trackSelectionParameters; private Commands availableCommands; private @Player.State int playbackState; - private int currentWindowIndex; private long lastReportedPositionMs; - private int pendingSeekCount; - private int pendingSeekWindowIndex; - private long pendingSeekPositionMs; - @Nullable private PositionInfo pendingMediaItemRemovalPosition; private MediaMetadata mediaMetadata; private MediaMetadata playlistMetadata; private DeviceInfo deviceInfo; @@ -402,19 +417,24 @@ private RemoteCastPlayer(Builder builder) { + MediaLibraryInfo.VERSION_SLASHY + "] [" + Util.DEVICE_DEBUG_INFO - + "]"); + + "] " + + Flags.getLogString()); this.cast = cast; this.mediaItemConverter = mediaItemConverter; this.trackSelector = trackSelector; this.seekBackIncrementMs = seekBackIncrementMs; this.seekForwardIncrementMs = seekForwardIncrementMs; this.maxSeekToPreviousPositionMs = maxSeekToPreviousPositionMs; - timelineTracker = new CastTimelineTracker(mediaItemConverter); + queueOrchestrator = + new CastQueueOrchestrator( + () -> PlayerTransferState.fromPlayer(this), + mediaItemConverter, + this::onQueueSnapshotChanged); + currentQueueSnapshot = QueuedOperation.QueueSnapshot.EMPTY; period = new Timeline.Period(); castListener = new CastListener(); statusListener = new StatusListener(); mediaQueueCallback = new MediaQueueCallback(); - seekResultCallback = new SeekResultCallback(); listeners = new ListenerSet<>( Looper.getMainLooper(), @@ -424,9 +444,9 @@ private RemoteCastPlayer(Builder builder) { repeatMode = new StateHolder<>(REPEAT_MODE_OFF); deviceVolume = MAX_VOLUME; volume = new StateHolder<>(1f); + unmuteVolume = 1f; playbackParameters = new StateHolder<>(PlaybackParameters.DEFAULT); playbackState = STATE_IDLE; - currentTimeline = CastTimeline.EMPTY_CAST_TIMELINE; mediaMetadata = MediaMetadata.EMPTY; playlistMetadata = MediaMetadata.EMPTY; currentTracks = new StateHolder<>(Tracks.EMPTY); @@ -436,8 +456,6 @@ private RemoteCastPlayer(Builder builder) { .addIf(COMMAND_SET_TRACK_SELECTION_PARAMETERS, trackSelector != null) .build(); trackSelectionParameters = new StateHolder<>(TrackSelectionParameters.DEFAULT); - pendingSeekWindowIndex = C.INDEX_UNSET; - pendingSeekPositionMs = C.TIME_UNSET; if (context != null) { cast.ensureInitialized(context); @@ -451,7 +469,7 @@ private RemoteCastPlayer(Builder builder) { deviceInfo = api30Impl.fetchDeviceInfo(); } else { api30Impl = null; - deviceInfo = DEVICE_INFO_REMOTE_EMPTY; + deviceInfo = fetchDeviceInfoFallback(); } if (trackSelector != null) { trackSelector.init(this::onTrackSelectionInvalidated); @@ -468,6 +486,7 @@ private void onTrackSelectionInvalidated() { listeners.flushEvents(); } + // TODO: b/549537517 - Remove this method once deprecation cycle is complete. /** * Returns the item that corresponds to the period with the given id, or null if no media queue or * period with id {@code periodId} exist. @@ -476,11 +495,17 @@ private void onTrackSelectionInvalidated() { * to get. * @return The item that corresponds to the period with the given id, or null if no media queue or * period with id {@code periodId} exist. + * @deprecated Use {@link #getCurrentTimeline()} to get {@link Timeline.Window#mediaItem} via + * {@link Timeline#getPeriodByUid(Object, Timeline.Period)} and {@link Timeline#getWindow(int, + * Timeline.Window)}. */ + @Deprecated @Nullable public MediaQueueItem getItem(int periodId) { MediaStatus mediaStatus = getMediaStatus(); - return mediaStatus != null && currentTimeline.getIndexOfPeriod(periodId) != C.INDEX_UNSET + return mediaStatus != null + && getCurrentTimeline().getIndexOfPeriod(queueOrchestrator.getItemUid(periodId)) + != C.INDEX_UNSET ? mediaStatus.getItemById(periodId) : null; } @@ -538,40 +563,77 @@ public void setMediaItems(List mediaItems, boolean resetPosition) { @Override public void setMediaItems(List mediaItems, int startIndex, long startPositionMs) { - setMediaItemsInternal(mediaItems, startIndex, startPositionMs, repeatMode.value); + if (!isCastSessionActive() || mediaItems.isEmpty()) { + return; + } + startPositionMs = startPositionMs == C.TIME_UNSET ? 0 : startPositionMs; + if (startIndex == C.INDEX_UNSET) { + startIndex = getCurrentMediaItemIndex(); + startPositionMs = getCurrentPosition(); + } + int newWindowIndex = min(startIndex, mediaItems.size() - 1); + QueuedOperation.SetMediaItemsOperation operation = + new QueuedOperation.SetMediaItemsOperation( + ImmutableList.copyOf(mediaItems), newWindowIndex, startPositionMs); + queueOrchestrator.enqueue(operation, remoteMediaClient); } @Override public void addMediaItems(int index, List mediaItems) { checkArgument(index >= 0); - int uid = MediaQueueItem.INVALID_ITEM_ID; - if (index < currentTimeline.getWindowCount()) { - uid = (int) currentTimeline.getWindow(/* windowIndex= */ index, window).uid; + if (!isCastSessionActive() || mediaItems.isEmpty()) { + return; } - addMediaItemsInternal(mediaItems, uid); + CastTimeline currentTimeline = currentQueueSnapshot.timeline; + int clampedIndex = min(index, currentTimeline.getWindowCount()); + @Nullable + Object insertBeforeUid = + clampedIndex < currentTimeline.getWindowCount() + ? currentTimeline.getWindow(clampedIndex, window).uid + : null; + QueuedOperation.AddMediaItemsOperation operation = + new QueuedOperation.AddMediaItemsOperation( + insertBeforeUid, ImmutableList.copyOf(mediaItems)); + queueOrchestrator.enqueue(operation, remoteMediaClient); } @Override public void moveMediaItems(int fromIndex, int toIndex, int newIndex) { checkArgument(fromIndex >= 0 && fromIndex <= toIndex && newIndex >= 0); + if (!isCastSessionActive()) { + return; + } + CastTimeline currentTimeline = currentQueueSnapshot.timeline; int playlistSize = currentTimeline.getWindowCount(); - toIndex = min(toIndex, playlistSize); - newIndex = min(newIndex, playlistSize - (toIndex - fromIndex)); - if (fromIndex >= playlistSize || fromIndex == toIndex || fromIndex == newIndex) { + int resolvedToIndex = min(toIndex, playlistSize); + int moveCount = resolvedToIndex - fromIndex; + int resolvedNewIndex = min(newIndex, playlistSize - moveCount); + if (fromIndex >= playlistSize + || fromIndex == resolvedToIndex + || fromIndex == resolvedNewIndex) { // Do nothing. return; } - int[] uids = new int[toIndex - fromIndex]; - for (int i = 0; i < uids.length; i++) { - uids[i] = (int) currentTimeline.getWindow(/* windowIndex= */ i + fromIndex, window).uid; + List uidsToMove = new ArrayList<>(moveCount); + for (int i = fromIndex; i < resolvedToIndex; i++) { + uidsToMove.add(currentTimeline.getWindow(i, window).uid); } - moveMediaItemsInternal(uids, fromIndex, newIndex); + int insertBeforeIndex = + fromIndex < resolvedNewIndex ? resolvedNewIndex + moveCount : resolvedNewIndex; + @Nullable + Object insertBeforeUid = + insertBeforeIndex < playlistSize + ? currentTimeline.getWindow(insertBeforeIndex, window).uid + : null; + QueuedOperation.MoveMediaItemsOperation operation = + new QueuedOperation.MoveMediaItemsOperation(uidsToMove, insertBeforeUid); + queueOrchestrator.enqueue(operation, remoteMediaClient); } @Override public void replaceMediaItems(int fromIndex, int toIndex, List mediaItems) { checkArgument(fromIndex >= 0 && fromIndex <= toIndex); - int playlistSize = currentTimeline.getWindowCount(); + int playlistSize = getCurrentTimeline().getWindowCount(); if (fromIndex > playlistSize) { return; } @@ -583,17 +645,23 @@ public void replaceMediaItems(int fromIndex, int toIndex, List mediaI @Override public void removeMediaItems(int fromIndex, int toIndex) { checkArgument(fromIndex >= 0 && toIndex >= fromIndex); + if (!isCastSessionActive()) { + return; + } + CastTimeline currentTimeline = currentQueueSnapshot.timeline; int playlistSize = currentTimeline.getWindowCount(); - toIndex = min(toIndex, playlistSize); - if (fromIndex >= playlistSize || fromIndex == toIndex) { + int resolvedToIndex = min(toIndex, playlistSize); + if (fromIndex >= playlistSize || fromIndex == resolvedToIndex) { // Do nothing. return; } - int[] uids = new int[toIndex - fromIndex]; - for (int i = 0; i < uids.length; i++) { - uids[i] = (int) currentTimeline.getWindow(/* windowIndex= */ i + fromIndex, window).uid; + List uidsToRemove = new ArrayList<>(resolvedToIndex - fromIndex); + for (int i = fromIndex; i < resolvedToIndex; i++) { + uidsToRemove.add(currentTimeline.getWindow(i, window).uid); } - removeMediaItemsInternal(uids); + QueuedOperation.RemoveMediaItemsOperation operation = + new QueuedOperation.RemoveMediaItemsOperation(uidsToRemove); + queueOrchestrator.enqueue(operation, remoteMediaClient); } @Override @@ -628,7 +696,7 @@ public PlaybackException getPlayerError() { @Override public void setPlayWhenReady(boolean playWhenReady) { - if (remoteMediaClient == null) { + if (!isCastSessionActive()) { return; } // We update the local state and send the message to the receiver app, which will cause the @@ -651,6 +719,7 @@ public void setPlayWhenReady(boolean playWhenReady) { new ResultCallback() { @Override public void onResult(MediaChannelResult mediaChannelResult) { + logOperationFailedIfStatusError("Set playWhenReady", mediaChannelResult); if (remoteMediaClient != null) { updatePlayerStateAndNotifyIfChanged(this); listeners.flushEvents(); @@ -679,26 +748,20 @@ protected void seekTo( return; } checkArgument(mediaItemIndex >= 0); - if (!currentTimeline.isEmpty() && mediaItemIndex >= currentTimeline.getWindowCount()) { + Timeline currentTimeline = getCurrentTimeline(); + if (currentTimeline.isEmpty() || mediaItemIndex >= currentTimeline.getWindowCount()) { return; } - MediaStatus mediaStatus = getMediaStatus(); // We assume the default position is 0. There is no support for seeking to the default position // in RemoteMediaClient. positionMs = positionMs != C.TIME_UNSET ? positionMs : 0; - if (mediaStatus != null) { - if (getCurrentMediaItemIndex() != mediaItemIndex) { - remoteMediaClient - .queueJumpToItem( - (int) currentTimeline.getPeriod(mediaItemIndex, period).uid, positionMs, null) - .setResultCallback(seekResultCallback); - } else { - remoteMediaClient.seek(positionMs).setResultCallback(seekResultCallback); - } + if (isCastSessionActive()) { + boolean isWindowChange = getCurrentMediaItemIndex() != mediaItemIndex; + Object targetPeriodUid = currentTimeline.getUidOfPeriod(mediaItemIndex); PositionInfo oldPosition = getCurrentPositionInfo(); - pendingSeekCount++; - pendingSeekWindowIndex = mediaItemIndex; - pendingSeekPositionMs = positionMs; + QueuedOperation.SeekOperation seekOperation = + new QueuedOperation.SeekOperation(positionMs, targetPeriodUid, isWindowChange); + queueOrchestrator.enqueue(seekOperation, remoteMediaClient); PositionInfo newPosition = getCurrentPositionInfo(); listeners.queueEvent( Player.EVENT_POSITION_DISCONTINUITY, @@ -750,7 +813,7 @@ public PlaybackParameters getPlaybackParameters() { @Override public void stop() { playbackState = STATE_IDLE; - if (remoteMediaClient != null) { + if (isCastSessionActive()) { // TODO(b/69792021): Support or emulate stop without position reset. remoteMediaClient.stop(); } @@ -779,7 +842,7 @@ public void release() { @Override public void setPlaybackParameters(PlaybackParameters playbackParameters) { - if (remoteMediaClient == null) { + if (!isCastSessionActive()) { return; } PlaybackParameters actualPlaybackParameters = @@ -794,6 +857,7 @@ public void setPlaybackParameters(PlaybackParameters playbackParameters) { new ResultCallback() { @Override public void onResult(MediaChannelResult mediaChannelResult) { + logOperationFailedIfStatusError("Set playback parameters", mediaChannelResult); if (remoteMediaClient != null) { updatePlaybackRateAndNotifyIfChanged(this); listeners.flushEvents(); @@ -805,7 +869,7 @@ public void onResult(MediaChannelResult mediaChannelResult) { @Override public void setRepeatMode(@RepeatMode int repeatMode) { - if (remoteMediaClient == null) { + if (!isCastSessionActive()) { return; } // We update the local state and send the message to the receiver app, which will cause the @@ -819,6 +883,7 @@ public void setRepeatMode(@RepeatMode int repeatMode) { new ResultCallback() { @Override public void onResult(MediaChannelResult mediaChannelResult) { + logOperationFailedIfStatusError("Set repeat mode", mediaChannelResult); if (remoteMediaClient != null) { updateRepeatModeAndNotifyIfChanged(this); listeners.flushEvents(); @@ -889,7 +954,7 @@ public void setPlaylistMetadata(MediaMetadata playlistMetadata) { @Override public Timeline getCurrentTimeline() { - return currentTimeline; + return currentQueueSnapshot.timeline; } @Override @@ -899,7 +964,7 @@ public int getCurrentPeriodIndex() { @Override public int getCurrentMediaItemIndex() { - return pendingSeekWindowIndex != C.INDEX_UNSET ? pendingSeekWindowIndex : currentWindowIndex; + return currentQueueSnapshot.currentWindowIndex; } // TODO: Fill the cast timeline information with ProgressListener's duration updates. @@ -911,11 +976,13 @@ public long getDuration() { @Override public long getCurrentPosition() { - return pendingSeekPositionMs != C.TIME_UNSET - ? pendingSeekPositionMs - : remoteMediaClient != null - ? remoteMediaClient.getApproximateStreamPosition() - : lastReportedPositionMs; + long maskedPositionMs = currentQueueSnapshot.maskedPositionMs; + if (maskedPositionMs != C.TIME_UNSET) { + return maskedPositionMs; + } + return isCastSessionActive() + ? remoteMediaClient.getApproximateStreamPosition() + : lastReportedPositionMs; } @Override @@ -970,7 +1037,7 @@ public AudioAttributes getAudioAttributes() { @Override public void setVolume(float volume) { - if (remoteMediaClient == null) { + if (!isCastSessionActive()) { return; } // We update the local state and send the message to the receiver app, which will cause the @@ -984,6 +1051,7 @@ public void setVolume(float volume) { new ResultCallback() { @Override public void onResult(MediaChannelResult result) { + logOperationFailedIfStatusError("Set volume", result); if (remoteMediaClient != null) { updateVolumeAndNotifyIfChanged(this); listeners.flushEvents(); @@ -998,13 +1066,46 @@ public float getVolume() { return volume.value; } - /** This method is not supported and does nothing. */ @Override - public void mute() {} + public void mute() { + if (volume.value != 0) { + setMute(true); + } + } - /** This method is not supported and does nothing. */ @Override - public void unmute() {} + public void unmute() { + if (volume.value == 0 && unmuteVolume != 0) { + setMute(false); + } + } + + private void setMute(boolean muted) { + if (!isCastSessionActive()) { + return; + } + float newVolume = muted ? 0f : unmuteVolume; + setVolumeAndNotifyIfChanged(newVolume); + listeners.flushEvents(); + PendingResult pendingResult = remoteMediaClient.setStreamMute(muted); + if (!muted) { + // Always restore stream volume when unmuting to handle both mute() and organic setVolume(0) + // calls. Reassign pendingResult so the result callback attaches to the final async call. + pendingResult = remoteMediaClient.setStreamVolume(unmuteVolume); + } + this.volume.pendingResultCallback = + new ResultCallback() { + @Override + public void onResult(MediaChannelResult result) { + logOperationFailedIfStatusError("Set mute", result); + if (remoteMediaClient != null) { + updateVolumeAndNotifyIfChanged(this); + listeners.flushEvents(); + } + } + }; + pendingResult.setResultCallback(this.volume.pendingResultCallback); + } /** This method is not supported and does nothing. */ @Override @@ -1169,85 +1270,44 @@ public void setAudioAttributes(AudioAttributes audioAttributes, boolean handleAu // Internal methods. - // Call deprecated callbacks. - @SuppressWarnings("deprecation") - private void updateInternalStateAndNotifyIfChanged() { - if (remoteMediaClient == null) { - // There is no session. We leave the state of the player as it is now. + private void onQueueSnapshotChanged( + QueuedOperation.QueueSnapshot newSnapshot, + @Nullable @TimelineChangeReason Integer timelineChangeReason) { + if (timelineChangeReason == null || !isCastSessionActive()) { + currentQueueSnapshot = newSnapshot; + return; + } + if (timelineChangeReason == Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED + && currentQueueSnapshot.timeline.equals(newSnapshot.timeline)) { + // Operations that do not modify the timeline (such as seekTo) handle their own position + // discontinuity and transition notifications. + currentQueueSnapshot = newSnapshot; return; } - int oldWindowIndex = this.currentWindowIndex; MediaMetadata oldMediaMetadata = mediaMetadata; - @Nullable - Object oldPeriodUid = - !getCurrentTimeline().isEmpty() - ? getCurrentTimeline().getPeriod(oldWindowIndex, period, /* setIds= */ true).uid - : null; + applyQueueSnapshotAndNotifyTimelineChanges(newSnapshot, timelineChangeReason); + mediaMetadata = getMediaMetadataInternal(); + if (!oldMediaMetadata.equals(mediaMetadata)) { + listeners.queueEvent( + Player.EVENT_MEDIA_METADATA_CHANGED, + listener -> listener.onMediaMetadataChanged(mediaMetadata)); + } + listeners.flushEvents(); + } + + private void updateInternalStateAndNotifyIfChanged() { + if (!isCastSessionActive()) { + // Leave the state of the player as it is when there is no session. + return; + } updatePlayerStateAndNotifyIfChanged(/* resultCallback= */ null); updateDeviceVolumeAndNotifyIfChanged(); updateRepeatModeAndNotifyIfChanged(/* resultCallback= */ null); updateVolumeAndNotifyIfChanged(/* resultCallback= */ null); updatePlaybackRateAndNotifyIfChanged(/* resultCallback= */ null); - boolean playingPeriodChangedByTimelineChange = updateTimelineAndNotifyIfChanged(); - Timeline currentTimeline = getCurrentTimeline(); - currentWindowIndex = fetchCurrentWindowIndex(remoteMediaClient, currentTimeline); - mediaMetadata = getMediaMetadataInternal(); - @Nullable - Object currentPeriodUid = - !currentTimeline.isEmpty() - ? currentTimeline.getPeriod(currentWindowIndex, period, /* setIds= */ true).uid - : null; - if (!playingPeriodChangedByTimelineChange - && !Objects.equals(oldPeriodUid, currentPeriodUid) - && pendingSeekCount == 0) { - // Report discontinuity and media item auto transition. - currentTimeline.getPeriod(oldWindowIndex, period, /* setIds= */ true); - currentTimeline.getWindow(oldWindowIndex, window); - long windowDurationMs = window.getDurationMs(); - PositionInfo oldPosition = - new PositionInfo( - window.uid, - period.windowIndex, - window.mediaItem, - period.uid, - period.windowIndex, - /* positionMs= */ windowDurationMs, - /* contentPositionMs= */ windowDurationMs, - /* adGroupIndex= */ C.INDEX_UNSET, - /* adIndexInAdGroup= */ C.INDEX_UNSET); - currentTimeline.getPeriod(currentWindowIndex, period, /* setIds= */ true); - currentTimeline.getWindow(currentWindowIndex, window); - PositionInfo newPosition = - new PositionInfo( - window.uid, - period.windowIndex, - window.mediaItem, - period.uid, - period.windowIndex, - /* positionMs= */ window.getDefaultPositionMs(), - /* contentPositionMs= */ window.getDefaultPositionMs(), - /* adGroupIndex= */ C.INDEX_UNSET, - /* adIndexInAdGroup= */ C.INDEX_UNSET); - listeners.queueEvent( - Player.EVENT_POSITION_DISCONTINUITY, - listener -> { - listener.onPositionDiscontinuity(DISCONTINUITY_REASON_AUTO_TRANSITION); - listener.onPositionDiscontinuity( - oldPosition, newPosition, DISCONTINUITY_REASON_AUTO_TRANSITION); - }); - listeners.queueEvent( - Player.EVENT_MEDIA_ITEM_TRANSITION, - listener -> - listener.onMediaItemTransition( - getCurrentMediaItem(), MEDIA_ITEM_TRANSITION_REASON_AUTO)); - } + updateTimelineAndNotifyIfChanged(); updateTracksAndNotifyIfChanged( /* resultCallback= */ null, TRACK_SELECTION_REQUEST_REASON_RECEIVER_UPDATE); - if (!oldMediaMetadata.equals(mediaMetadata)) { - listeners.queueEvent( - Player.EVENT_MEDIA_METADATA_CHANGED, - listener -> listener.onMediaMetadataChanged(mediaMetadata)); - } updateAvailableCommandsAndNotifyIfChanged(); listeners.flushEvents(); } @@ -1323,60 +1383,51 @@ private void updateRepeatModeAndNotifyIfChanged(@Nullable ResultCallback resu } } + /** Updates the timeline and notifies {@link Player.Listener event listeners} if required. */ + private void updateTimelineAndNotifyIfChanged() { + if (!isCastSessionActive()) { + // We ignore calls when the session is not active. + return; + } + queueOrchestrator.onReceiverStateUpdated(remoteMediaClient); + updateAvailableCommandsAndNotifyIfChanged(); + } + /** - * Updates the timeline and notifies {@link Player.Listener event listeners} if required. - * - * @return Whether the timeline change has caused a change of the period currently being played. + * Diffs {@code newSnapshot} against {@link #currentQueueSnapshot}, updates {@link + * #currentQueueSnapshot}, and queues timeline, discontinuity, and transition events. */ @SuppressWarnings("deprecation") // Calling deprecated listener method. - private boolean updateTimelineAndNotifyIfChanged() { - if (remoteMediaClient == null) { - // There is no session. We leave the state of the player as it is now. - return false; - } - Timeline oldTimeline = currentTimeline; - int oldWindowIndex = currentWindowIndex; - boolean playingPeriodChanged = false; - if (updateTimeline()) { - // TODO: Differentiate TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED and - // TIMELINE_CHANGE_REASON_SOURCE_UPDATE [see internal: b/65152553]. - Timeline timeline = currentTimeline; - // Call onTimelineChanged. + private void applyQueueSnapshotAndNotifyTimelineChanges( + QueuedOperation.QueueSnapshot newSnapshot, @TimelineChangeReason int timelineChangeReason) { + Timeline oldTimeline = currentQueueSnapshot.timeline; + int oldWindowIndex = currentQueueSnapshot.currentWindowIndex; + @Nullable + Object oldPeriodUid = + !oldTimeline.isEmpty() + ? oldTimeline.getPeriod(oldWindowIndex, period, /* setIds= */ true).uid + : null; + boolean playingPeriodRemoved = + oldPeriodUid != null + && newSnapshot.timeline.getIndexOfPeriod(oldPeriodUid) == C.INDEX_UNSET; + @Nullable PositionInfo oldPosition = playingPeriodRemoved ? getCurrentPositionInfo() : null; + + currentQueueSnapshot = newSnapshot; + + Timeline currentTimeline = newSnapshot.timeline; + int currentWindowIndex = newSnapshot.currentWindowIndex; + @Nullable + Object currentPeriodUid = + !currentTimeline.isEmpty() + ? currentTimeline.getPeriod(currentWindowIndex, period, /* setIds= */ true).uid + : null; + + if (!oldTimeline.equals(currentTimeline)) { listeners.queueEvent( Player.EVENT_TIMELINE_CHANGED, - listener -> - listener.onTimelineChanged(timeline, Player.TIMELINE_CHANGE_REASON_SOURCE_UPDATE)); - - // Call onPositionDiscontinuity if required. - Timeline currentTimeline = getCurrentTimeline(); - boolean playingPeriodRemoved = false; - if (!oldTimeline.isEmpty()) { - Object oldPeriodUid = - castNonNull(oldTimeline.getPeriod(oldWindowIndex, period, /* setIds= */ true).uid); - playingPeriodRemoved = currentTimeline.getIndexOfPeriod(oldPeriodUid) == C.INDEX_UNSET; - } - if (playingPeriodRemoved) { - PositionInfo oldPosition; - if (pendingMediaItemRemovalPosition != null) { - oldPosition = pendingMediaItemRemovalPosition; - pendingMediaItemRemovalPosition = null; - } else { - // If the media item has been removed by another client, we don't know the removal - // position. We use the current position as a fallback. - oldTimeline.getPeriod(oldWindowIndex, period, /* setIds= */ true); - oldTimeline.getWindow(period.windowIndex, window); - oldPosition = - new PositionInfo( - window.uid, - period.windowIndex, - window.mediaItem, - period.uid, - period.windowIndex, - getCurrentPosition(), - getContentPosition(), - /* adGroupIndex= */ C.INDEX_UNSET, - /* adIndexInAdGroup= */ C.INDEX_UNSET); - } + listener -> listener.onTimelineChanged(currentTimeline, timelineChangeReason)); + + if (playingPeriodRemoved && oldPosition != null) { PositionInfo newPosition = getCurrentPositionInfo(); listeners.queueEvent( Player.EVENT_POSITION_DISCONTINUITY, @@ -1387,8 +1438,7 @@ private boolean updateTimelineAndNotifyIfChanged() { }); } - // Call onMediaItemTransition if required. - playingPeriodChanged = + boolean playingPeriodChanged = currentTimeline.isEmpty() != oldTimeline.isEmpty() || playingPeriodRemoved; if (playingPeriodChanged) { listeners.queueEvent( @@ -1397,28 +1447,55 @@ private boolean updateTimelineAndNotifyIfChanged() { listener.onMediaItemTransition( getCurrentMediaItem(), MEDIA_ITEM_TRANSITION_REASON_PLAYLIST_CHANGED)); } - updateAvailableCommandsAndNotifyIfChanged(); } - return playingPeriodChanged; - } - /** - * Updates the current timeline. The current window index may change as a result. - * - * @return Whether the current timeline has changed. - */ - private boolean updateTimeline() { - CastTimeline oldTimeline = currentTimeline; - MediaStatus status = getMediaStatus(); - currentTimeline = - status != null - ? timelineTracker.getCastTimeline(remoteMediaClient) - : CastTimeline.EMPTY_CAST_TIMELINE; - boolean timelineChanged = !oldTimeline.equals(currentTimeline); - if (timelineChanged) { - currentWindowIndex = fetchCurrentWindowIndex(remoteMediaClient, currentTimeline); + if (timelineChangeReason == Player.TIMELINE_CHANGE_REASON_SOURCE_UPDATE + && oldPeriodUid != null + && !playingPeriodRemoved + && !Objects.equals(oldPeriodUid, currentPeriodUid)) { + // Report discontinuity and media item auto transition. + oldTimeline.getPeriod(oldWindowIndex, period, /* setIds= */ true); + oldTimeline.getWindow(oldWindowIndex, window); + long windowDurationMs = window.getDurationMs(); + PositionInfo autoTransitionOldPosition = + new PositionInfo( + window.uid, + period.windowIndex, + window.mediaItem, + period.uid, + period.windowIndex, + /* positionMs= */ windowDurationMs, + /* contentPositionMs= */ windowDurationMs, + /* adGroupIndex= */ C.INDEX_UNSET, + /* adIndexInAdGroup= */ C.INDEX_UNSET); + currentTimeline.getPeriod(currentWindowIndex, period, /* setIds= */ true); + currentTimeline.getWindow(currentWindowIndex, window); + PositionInfo autoTransitionNewPosition = + new PositionInfo( + window.uid, + period.windowIndex, + window.mediaItem, + period.uid, + period.windowIndex, + /* positionMs= */ window.getDefaultPositionMs(), + /* contentPositionMs= */ window.getDefaultPositionMs(), + /* adGroupIndex= */ C.INDEX_UNSET, + /* adIndexInAdGroup= */ C.INDEX_UNSET); + listeners.queueEvent( + Player.EVENT_POSITION_DISCONTINUITY, + listener -> { + listener.onPositionDiscontinuity(DISCONTINUITY_REASON_AUTO_TRANSITION); + listener.onPositionDiscontinuity( + autoTransitionOldPosition, + autoTransitionNewPosition, + DISCONTINUITY_REASON_AUTO_TRANSITION); + }); + listeners.queueEvent( + Player.EVENT_MEDIA_ITEM_TRANSITION, + listener -> + listener.onMediaItemTransition( + getCurrentMediaItem(), MEDIA_ITEM_TRANSITION_REASON_AUTO)); } - return timelineChanged; } /** @@ -1435,8 +1512,8 @@ private boolean updateTimeline() { private void updateTracksAndNotifyIfChanged( @Nullable ResultCallback resultCallback, @TrackSelectionRequestReason int selectionRequestReason) { - if (remoteMediaClient == null) { - // There is no session. We leave the state of the player as it is now. + if (!isCastSessionActive()) { + // We ignore calls when the session is not active. return; } if (currentTracks.acceptsUpdate(resultCallback)) { @@ -1504,6 +1581,7 @@ private void updateTracksAndNotifyIfChanged( new ResultCallback() { @Override public void onResult(MediaChannelResult result) { + logOperationFailedIfStatusError("Set active tracks", result); if (remoteMediaClient != null) { updateTracksAndNotifyIfChanged( this, TRACK_SELECTION_REQUEST_REASON_RECEIVER_UPDATE); @@ -1560,7 +1638,7 @@ private void updateAvailableCommandsAndNotifyIfChanged() { } private boolean isSetVolumeCommandAvailable() { - if (remoteMediaClient != null) { + if (isCastSessionActive()) { MediaStatus mediaStatus = remoteMediaClient.getMediaStatus(); if (mediaStatus != null) { return mediaStatus.isMediaCommandSupported(MediaStatus.COMMAND_SET_VOLUME); @@ -1569,85 +1647,6 @@ private boolean isSetVolumeCommandAvailable() { return false; } - private void setMediaItemsInternal( - List mediaItems, - int startIndex, - long startPositionMs, - @RepeatMode int repeatMode) { - if (remoteMediaClient == null || mediaItems.isEmpty()) { - return; - } - startPositionMs = startPositionMs == C.TIME_UNSET ? 0 : startPositionMs; - if (startIndex == C.INDEX_UNSET) { - startIndex = getCurrentMediaItemIndex(); - startPositionMs = getCurrentPosition(); - } - Timeline currentTimeline = getCurrentTimeline(); - if (!currentTimeline.isEmpty()) { - pendingMediaItemRemovalPosition = getCurrentPositionInfo(); - } - MediaQueueItem[] mediaQueueItems = toMediaQueueItems(mediaItems); - timelineTracker.onMediaItemsSet(mediaItems, mediaQueueItems); - MediaQueueData mediaQueueData = - new MediaQueueData.Builder() - .setItems(Arrays.asList(mediaQueueItems)) - .setStartIndex(min(startIndex, mediaItems.size() - 1)) - .setRepeatMode(getCastRepeatMode(repeatMode)) - .setStartTime(startPositionMs) - .build(); - // TODO: b/432716880 - Populate playback speed and repeat mode values. - // TODO: b/434761431 - Remove setCurrentTime call once setStartTime (above) is handled correctly - // by the Cast framework. - MediaLoadRequestData loadRequestData = - new MediaLoadRequestData.Builder() - .setAutoplay(getPlayWhenReady()) - .setQueueData(mediaQueueData) - .setCurrentTime(startPositionMs) - .build(); - // We don't use the pending result because the timeline tracker is taking care of the masking. - PendingResult unused = remoteMediaClient.load(loadRequestData); - } - - private void addMediaItemsInternal(List mediaItems, int uid) { - if (remoteMediaClient == null || getMediaStatus() == null) { - return; - } - MediaQueueItem[] itemsToInsert = toMediaQueueItems(mediaItems); - timelineTracker.onMediaItemsAdded(mediaItems, itemsToInsert); - remoteMediaClient.queueInsertItems(itemsToInsert, uid, /* customData= */ null); - } - - private void moveMediaItemsInternal(int[] uids, int fromIndex, int newIndex) { - if (remoteMediaClient == null || getMediaStatus() == null) { - return; - } - int insertBeforeIndex = fromIndex < newIndex ? newIndex + uids.length : newIndex; - int insertBeforeItemId = MediaQueueItem.INVALID_ITEM_ID; - if (insertBeforeIndex < currentTimeline.getWindowCount()) { - insertBeforeItemId = (int) currentTimeline.getWindow(insertBeforeIndex, window).uid; - } - remoteMediaClient.queueReorderItems(uids, insertBeforeItemId, /* customData= */ null); - } - - @Nullable - private PendingResult removeMediaItemsInternal(int[] uids) { - if (remoteMediaClient == null || getMediaStatus() == null) { - return null; - } - Timeline timeline = getCurrentTimeline(); - if (!timeline.isEmpty()) { - Object periodUid = - castNonNull(timeline.getPeriod(getCurrentPeriodIndex(), period, /* setIds= */ true).uid); - for (int uid : uids) { - if (periodUid.equals(uid)) { - pendingMediaItemRemovalPosition = getCurrentPositionInfo(); - break; - } - } - } - return remoteMediaClient.queueRemoveItems(uids, /* customData= */ null); - } - private PositionInfo getCurrentPositionInfo() { Timeline currentTimeline = getCurrentTimeline(); @Nullable Object newPeriodUid = null; @@ -1693,6 +1692,7 @@ private void setRepeatModeAndNotifyIfChanged(@Player.RepeatMode int repeatMode) private void setVolumeAndNotifyIfChanged(float volume) { if (this.volume.value != volume) { + unmuteVolume = volume != 0 ? volume : this.unmuteVolume; this.volume.value = volume; listeners.queueEvent( Player.EVENT_VOLUME_CHANGED, listener -> listener.onVolumeChanged(volume)); @@ -1764,6 +1764,14 @@ private void setTracksAndNotifyIfChanged(Tracks tracks) { } private void setCastSession(@Nullable CastSession castSession) { + setCastSession( + castSession, SessionAvailabilityListener.SESSION_UNAVAILABLE_REASON_DISCONNECTED); + } + + private void setCastSession( + @Nullable CastSession castSession, + @SessionAvailabilityListener.SessionUnavailableReason int sessionUnavailableReason) { + isSessionEnding = false; if (this.castSession != null) { this.castSession.removeCastListener(castListener); } @@ -1798,12 +1806,50 @@ private void setCastSession(@Nullable CastSession castSession) { updateInternalStateAndNotifyIfChanged(); } else { if (sessionAvailabilityListener != null) { - sessionAvailabilityListener.onCastSessionUnavailable(); + sessionAvailabilityListener.onCastSessionUnavailable(sessionUnavailableReason); } if (internalSessionAvailabilityListener != null) { - internalSessionAvailabilityListener.onCastSessionUnavailable(); + internalSessionAvailabilityListener.onCastSessionUnavailable(sessionUnavailableReason); } } + updateDeviceInfo(); + } + + private void updateDeviceInfo() { + DeviceInfo oldDeviceInfo = deviceInfo; + DeviceInfo newDeviceInfo = + SDK_INT >= 30 && api30Impl != null + ? api30Impl.fetchDeviceInfo() + : fetchDeviceInfoFallback(); + deviceInfo = newDeviceInfo; + if (!deviceInfo.equals(oldDeviceInfo)) { + listeners.sendEvent( + EVENT_DEVICE_INFO_CHANGED, listener -> listener.onDeviceInfoChanged(newDeviceInfo)); + } + } + + private DeviceInfo fetchDeviceInfoFallback() { + if (castSession != null && castSession.getCastDevice() != null) { + return new DeviceInfo.Builder(DeviceInfo.PLAYBACK_TYPE_REMOTE) + .setMaxVolume(MAX_VOLUME) + .setRoutingControllerName(castSession.getCastDevice().getFriendlyName()) + .build(); + } + return DEVICE_INFO_REMOTE_EMPTY; + } + + private static @SessionAvailabilityListener.SessionUnavailableReason int mapDisconnectionReason( + int error) { + switch (error) { + case SESSION_END_REASON_ROUTE_CHANGE: + return SessionAvailabilityListener.SESSION_UNAVAILABLE_REASON_ROUTE_CHANGED; + case SESSION_END_REASON_STOPPED: + return SessionAvailabilityListener.SESSION_UNAVAILABLE_REASON_STOPPED; + case APPLICATION_STOPPED: + return SessionAvailabilityListener.SESSION_UNAVAILABLE_REASON_APP_STOPPED; + default: + return SessionAvailabilityListener.SESSION_UNAVAILABLE_REASON_DISCONNECTED; + } } /** @@ -1821,15 +1867,16 @@ private void resetMaskingAndCacheState() { playbackParameters.clearPendingResultCallback(); currentTracks.clearPendingResultCallback(); trackSelectionParameters.clearPendingResultCallback(); - pendingSeekCount = 0; - pendingSeekWindowIndex = C.INDEX_UNSET; - pendingSeekPositionMs = C.TIME_UNSET; - pendingMediaItemRemovalPosition = null; + queueOrchestrator.reset(/* clearSnapshots= */ remoteMediaClient != null); } @Nullable private MediaStatus getMediaStatus() { - return remoteMediaClient != null ? remoteMediaClient.getMediaStatus() : null; + return isCastSessionActive() ? remoteMediaClient.getMediaStatus() : null; + } + + private boolean isCastSessionActive() { + return remoteMediaClient != null && !isSessionEnding; } /** @@ -1881,25 +1928,11 @@ private static float fetchVolume(RemoteMediaClient remoteMediaClient) { if (mediaStatus == null) { return 1f; } - return (float) mediaStatus.getStreamVolume(); - } - - private static int fetchCurrentWindowIndex( - @Nullable RemoteMediaClient remoteMediaClient, Timeline timeline) { - if (remoteMediaClient == null) { - return 0; - } - - int currentWindowIndex = C.INDEX_UNSET; - @Nullable MediaQueueItem currentItem = remoteMediaClient.getCurrentItem(); - if (currentItem != null) { - currentWindowIndex = timeline.getIndexOfPeriod(currentItem.getItemId()); - } - if (currentWindowIndex == C.INDEX_UNSET) { - // The timeline is empty. Fall back to index 0. - currentWindowIndex = 0; + // Mute check should come first because getStreamVolume stores the previous volume when muted. + if (mediaStatus.isMute()) { + return 0f; } - return currentWindowIndex; + return (float) mediaStatus.getStreamVolume(); } @SuppressWarnings("VisibleForTests") @@ -1916,12 +1949,8 @@ private static int getCastRepeatMode(@RepeatMode int repeatMode) { } } - private MediaQueueItem[] toMediaQueueItems(List mediaItems) { - MediaQueueItem[] mediaQueueItems = new MediaQueueItem[mediaItems.size()]; - for (int i = 0; i < mediaItems.size(); i++) { - mediaQueueItems[i] = mediaItemConverter.toMediaQueueItem(mediaItems.get(i)); - } - return mediaQueueItems; + private static void logOperationFailedIfStatusError(String operation, MediaChannelResult result) { + CastUtils.logOperationFailedIfStatusError(TAG, operation, result); } // Internal classes. @@ -1933,7 +1962,9 @@ private final class StatusListener extends RemoteMediaClient.Callback @Override public void onProgressUpdated(long progressMs, long unusedDurationMs) { - lastReportedPositionMs = progressMs; + if (isCastSessionActive()) { + lastReportedPositionMs = progressMs; + } } // RemoteMediaClient.Callback implementation. @@ -1974,13 +2005,13 @@ public void onSessionResumed(CastSession castSession, boolean b) { } @Override - public void onSessionEnded(CastSession castSession, int i) { - setCastSession(null); + public void onSessionEnded(CastSession castSession, int error) { + setCastSession(null, mapDisconnectionReason(error)); } @Override - public void onSessionSuspended(CastSession castSession, int i) { - setCastSession(null); + public void onSessionSuspended(CastSession castSession, int reason) { + setCastSession(null, SessionAvailabilityListener.SESSION_UNAVAILABLE_REASON_DISCONNECTED); } @Override @@ -2012,10 +2043,10 @@ public void onSessionStartFailed(CastSession castSession, int statusCode) { public void onSessionEnding(CastSession castSession) { // We don't wait until the onSessionEnded event because the media queue callback will send // updates before that (but after onSessionEnding) indicating the queue is being cleared. This - // complicates playback state transfer to local. As a result, we clear the active cast session - // here so that the player keeps reflecting the state at this point in time, before the queue - // is cleared. See [internal: b/479112029]. - setCastSession(null); + // complicates playback state transfer to local. As a result, we freeze the state here by + // setting isSessionEnding = true, and wait for onSessionEnded to call setCastSession(null) + // with the unselect reason. See [internal: b/479112029] and [internal: b/502159642]. + isSessionEnding = true; } @Override @@ -2024,24 +2055,6 @@ public void onSessionResuming(CastSession castSession, String s) { } } - private final class SeekResultCallback implements ResultCallback { - - @Override - public void onResult(MediaChannelResult result) { - int statusCode = result.getStatus().getStatusCode(); - if (statusCode != CastStatusCodes.SUCCESS && statusCode != CastStatusCodes.REPLACED) { - Log.e( - TAG, - "Seek failed. Error code " + statusCode + ": " + CastUtils.getLogString(statusCode)); - } - if (--pendingSeekCount == 0) { - currentWindowIndex = pendingSeekWindowIndex; - pendingSeekWindowIndex = C.INDEX_UNSET; - pendingSeekPositionMs = C.TIME_UNSET; - } - } - } - /** * Holds the value and the masking status of a specific part of the {@link RemoteCastPlayer} * state. @@ -2117,17 +2130,6 @@ public void release() { handler.removeCallbacksAndMessages(/* token= */ null); } - /** Updates the device info with an up-to-date value and notifies the listeners. */ - private void updateDeviceInfo() { - DeviceInfo oldDeviceInfo = deviceInfo; - DeviceInfo newDeviceInfo = fetchDeviceInfo(); - deviceInfo = newDeviceInfo; - if (!deviceInfo.equals(oldDeviceInfo)) { - listeners.sendEvent( - EVENT_DEVICE_INFO_CHANGED, listener -> listener.onDeviceInfoChanged(newDeviceInfo)); - } - } - /** * Returns a {@link DeviceInfo} with the {@link RoutingController#getId() id} that corresponds * to the Cast session, or {@link #DEVICE_INFO_REMOTE_EMPTY} if not available. @@ -2142,14 +2144,22 @@ public DeviceInfo fetchDeviceInfo() { // There's either no remote routing controller, or there's more than one. In either case we // don't populate the device info because either there's no Cast routing controller, or we // cannot safely identify the Cast routing controller. - return DEVICE_INFO_REMOTE_EMPTY; + return fetchDeviceInfoFallback(); } else { // There's only one remote routing controller. It's safe to assume it's the Cast routing // controller. RoutingController remoteController = controllers.get(1); + @Nullable String controllerName = null; + if (SDK_INT >= 34) { + controllerName = Api34Impl.getRoutingControllerName(remoteController); + } + if (controllerName == null && !remoteController.getSelectedRoutes().isEmpty()) { + controllerName = remoteController.getSelectedRoutes().get(0).getName().toString(); + } return new DeviceInfo.Builder(DeviceInfo.PLAYBACK_TYPE_REMOTE) .setMaxVolume(MAX_VOLUME) .setRoutingControllerId(remoteController.getId()) + .setRoutingControllerName(controllerName) .build(); } } @@ -2181,6 +2191,19 @@ public void onStop(RoutingController controller) { } } + @RequiresApi(34) + private static final class Api34Impl { + private Api34Impl() {} + + @Nullable + @DoNotInline + public static String getRoutingControllerName(RoutingController controller) { + RoutingSessionInfo sessionInfo = controller.getRoutingSessionInfo(); + CharSequence name = sessionInfo != null ? sessionInfo.getName() : null; + return name != null ? name.toString() : null; + } + } + private final class CastListener extends com.google.android.gms.cast.Cast.Listener { @Override diff --git a/libraries/cast/src/main/java/androidx/media3/cast/SessionAvailabilityListener.java b/libraries/cast/src/main/java/androidx/media3/cast/SessionAvailabilityListener.java index ddbeda78930..9aabdd3e62c 100644 --- a/libraries/cast/src/main/java/androidx/media3/cast/SessionAvailabilityListener.java +++ b/libraries/cast/src/main/java/androidx/media3/cast/SessionAvailabilityListener.java @@ -15,15 +15,57 @@ */ package androidx.media3.cast; +import androidx.annotation.IntDef; import androidx.media3.common.util.UnstableApi; +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; /** Listener of changes in the cast session availability. */ @UnstableApi public interface SessionAvailabilityListener { + /** Reasons for the Cast session becoming unavailable. */ + @IntDef({ + SESSION_UNAVAILABLE_REASON_ROUTE_CHANGED, + SESSION_UNAVAILABLE_REASON_STOPPED, + SESSION_UNAVAILABLE_REASON_APP_STOPPED, + SESSION_UNAVAILABLE_REASON_DISCONNECTED + }) + @Documented + @Retention(RetentionPolicy.SOURCE) + @Target(ElementType.TYPE_USE) + @interface SessionUnavailableReason {} + + /** Playback is transferring to the local device because the route was changed. */ + int SESSION_UNAVAILABLE_REASON_ROUTE_CHANGED = 1; + + /** The user explicitly stopped casting. */ + int SESSION_UNAVAILABLE_REASON_STOPPED = 2; + + /** The Cast receiver application was stopped. */ + int SESSION_UNAVAILABLE_REASON_APP_STOPPED = 3; + + /** The Cast session was disconnected. */ + int SESSION_UNAVAILABLE_REASON_DISCONNECTED = 4; + /** Called when a cast session becomes available to the player. */ void onCastSessionAvailable(); - /** Called when the cast session becomes unavailable. */ + /** + * Called when the cast session becomes unavailable. This only gets called if + * onCastSessionUnavailable(@CastPlayer.CastSessionChangeReason int reason) is not implemented + */ void onCastSessionUnavailable(); + + /** + * Called when the cast session becomes unavailable with the specified reason. The reason can + * affect the player state transfer. For example, a "Stop Casting" button press or a disconnection + * typically require the playback to pause. + */ + default void onCastSessionUnavailable(@SessionUnavailableReason int reason) { + onCastSessionUnavailable(); + } } diff --git a/libraries/cast/src/main/res/values-night/colors.xml b/libraries/cast/src/main/res/values-night/colors.xml index bf76ce471dc..5f8a6ebfca8 100644 --- a/libraries/cast/src/main/res/values-night/colors.xml +++ b/libraries/cast/src/main/res/values-night/colors.xml @@ -15,7 +15,6 @@ limitations under the License. --> - #000000 #636363 #d3d3d3 diff --git a/libraries/cast/src/test/java/androidx/media3/cast/CastPlayerTest.java b/libraries/cast/src/test/java/androidx/media3/cast/CastPlayerTest.java index 962e5d0250c..cff685adbf1 100644 --- a/libraries/cast/src/test/java/androidx/media3/cast/CastPlayerTest.java +++ b/libraries/cast/src/test/java/androidx/media3/cast/CastPlayerTest.java @@ -21,10 +21,13 @@ import static org.mockito.ArgumentMatchers.anyInt; import static org.mockito.ArgumentMatchers.eq; import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.never; import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; import static org.mockito.MockitoAnnotations.openMocks; +import static org.robolectric.Shadows.shadowOf; +import android.os.Looper; import androidx.media3.common.C; import androidx.media3.common.DeviceInfo; import androidx.media3.common.MediaItem; @@ -252,6 +255,15 @@ public void release_whenCastSessionIsAvailable_releasesBothPlayers() { public void playerTransfer_whenSourcePlayerIsNonIdle_callsPrepare() { // We need a non-empty timeline to be in a non-idle state, and check that the target player is // prepared as a result. + MediaInfo mediaInfo = + new MediaInfo.Builder("https://example.com/media.mp4") + .setStreamType(MediaInfo.STREAM_TYPE_BUFFERED) + .setContentType("video/mp4") + .build(); + MediaQueueItem queueItem = new MediaQueueItem.Builder(mediaInfo).setItemId(1).build(); + when(mockMediaStatus.getQueueItems()).thenReturn(ImmutableList.of(queueItem)); + when(mockMediaStatus.getCurrentItemId()).thenReturn(1); + when(mockMediaStatus.getMediaInfo()).thenReturn(mediaInfo); when(mockMediaQueue.getItemIds()).thenReturn(new int[] {1}); when(mockRemoteMediaClient.getPlayerState()).thenReturn(MediaStatus.PLAYER_STATE_PLAYING); castSessionListener.onSessionStarted(mockCastSession, /* sessionId= */ "ignored"); @@ -271,4 +283,154 @@ public void playerTransfer_whenSourcePlayerIsIdle_doesNotCallPrepare() { assertThat(localPlayer.getPlaybackState()).isEqualTo(Player.STATE_IDLE); } + + @Test + public void playerTransfer_whenSessionStopped_pausesLocalPlayer() { + castSessionListener.onSessionStarted(mockCastSession, /* sessionId= */ ""); + castPlayer = castPlayerBuilder.build(); + castPlayer.setPlayWhenReady(true); + + castSessionListener.onSessionEnded( + mockCastSession, RemoteCastPlayer.SESSION_END_REASON_STOPPED); + + assertThat(localPlayer.getPlayWhenReady()).isFalse(); + } + + @Test + public void playerTransfer_whenReceiverApplicationStopped_pausesLocalPlayer() { + castSessionListener.onSessionStarted(mockCastSession, /* sessionId= */ ""); + castPlayer = castPlayerBuilder.build(); + castPlayer.setPlayWhenReady(true); + + castSessionListener.onSessionEnded(mockCastSession, RemoteCastPlayer.APPLICATION_STOPPED); + + assertThat(localPlayer.getPlayWhenReady()).isFalse(); + } + + @Test + public void playerTransfer_whenRouteChanged_doesNotPauseLocalPlayer() { + castSessionListener.onSessionStarted(mockCastSession, /* sessionId= */ ""); + castPlayer = castPlayerBuilder.build(); + castPlayer.setPlayWhenReady(true); + + castSessionListener.onSessionEnded( + mockCastSession, RemoteCastPlayer.SESSION_END_REASON_ROUTE_CHANGE); + + assertThat(localPlayer.getPlayWhenReady()).isTrue(); + } + + @Test + public void playerTransfer_whenSessionSuspended_pausesLocalPlayer() { + castSessionListener.onSessionStarted(mockCastSession, /* sessionId= */ ""); + castPlayer = castPlayerBuilder.build(); + castPlayer.setPlayWhenReady(true); + + castSessionListener.onSessionSuspended(mockCastSession, /* reason= */ 0); + + assertThat(localPlayer.getPlayWhenReady()).isFalse(); + } + + @Test + public void playerTransfer_whenSessionStarted_doesNotPauseRemotePlayer() { + castPlayer = castPlayerBuilder.build(); + castPlayer.setMediaItem(MediaItem.fromUri("http://uri")); + castPlayer.setPlayWhenReady(true); + + castSessionListener.onSessionStarted(mockCastSession, /* sessionId= */ ""); + + // Verify the remote player inherited the playing state and was NOT explicitly paused + assertThat(castPlayer.getPlayWhenReady()).isTrue(); + // Verify that the command was actually forwarded to the Cast SDK + verify(mockRemoteMediaClient).play(); + } + + @Test + public void + playerTransfer_whenSessionStartedWithEmptyLocalPlayer_doesNotPausePlayingRemotePlayer() { + when(mockRemoteMediaClient.isPaused()).thenReturn(false); + when(mockRemoteMediaClient.getPlayerState()).thenReturn(MediaStatus.PLAYER_STATE_PLAYING); + castPlayer = castPlayerBuilder.build(); + castPlayer.clearMediaItems(); + castPlayer.setPlayWhenReady(false); + + castSessionListener.onSessionStarted(mockCastSession, /* sessionId= */ ""); + shadowOf(Looper.getMainLooper()).idle(); + + assertThat(castPlayer.getPlayWhenReady()).isTrue(); + verify(mockRemoteMediaClient, never()).pause(); + } + + @Test + public void + playerTransfer_whenSessionStartedWithEmptyLocalPlayer_doesNotPlayPausedRemotePlayer() { + when(mockRemoteMediaClient.isPaused()).thenReturn(true); + when(mockRemoteMediaClient.getPlayerState()).thenReturn(MediaStatus.PLAYER_STATE_PAUSED); + castPlayer = castPlayerBuilder.build(); + castPlayer.clearMediaItems(); + castPlayer.setPlayWhenReady(true); + + castSessionListener.onSessionStarted(mockCastSession, /* sessionId= */ ""); + shadowOf(Looper.getMainLooper()).idle(); + + assertThat(castPlayer.getPlayWhenReady()).isFalse(); + verify(mockRemoteMediaClient, never()).play(); + } + + @Test + public void + playerTransfer_whenSessionEndsWithUnconfirmedAddedItems_transfersAddedItemsToLocalPlayer() { + when(mockRemoteMediaClient.queueInsertItems(any(), anyInt(), any())) + .thenReturn(mockPendingResult); + castSessionListener.onSessionStarted(mockCastSession, /* sessionId= */ "session_1"); + castPlayer = castPlayerBuilder.build(); + MediaItem item1 = + new MediaItem.Builder() + .setMediaId("item_1") + .setUri("https://example.com/media1.mp4") + .setMimeType(MimeTypes.VIDEO_MP4) + .build(); + MediaItem item2 = + new MediaItem.Builder() + .setMediaId("item_2") + .setUri("https://example.com/media2.mp4") + .setMimeType(MimeTypes.VIDEO_MP4) + .build(); + castPlayer.setMediaItems(ImmutableList.of(item1)); + castPlayer.addMediaItems(ImmutableList.of(item2)); + + castSessionListener.onSessionEnded( + mockCastSession, RemoteCastPlayer.SESSION_END_REASON_STOPPED); + + assertThat(localPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(2); + assertThat(castPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(2); + } + + @Test + public void playerTransfer_castToLocalClearPlaylistAndBackToCast_clearsRemotePlayerSnapshots() { + castSessionListener.onSessionStarted(mockCastSession, /* sessionId= */ "session_1"); + castPlayer = castPlayerBuilder.build(); + MediaItem item1 = + new MediaItem.Builder() + .setMediaId("item_1") + .setUri("https://example.com/media1.mp4") + .setMimeType(MimeTypes.VIDEO_MP4) + .build(); + castPlayer.setMediaItems(ImmutableList.of(item1)); + + // Cast -> Local + castSessionListener.onSessionEnded( + mockCastSession, RemoteCastPlayer.SESSION_END_REASON_STOPPED); + assertThat(castPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(1); + + // Clear items while on Local + castPlayer.setMediaItems(ImmutableList.of()); + assertThat(castPlayer.getCurrentTimeline().isEmpty()).isTrue(); + + // Local -> Cast (new session with no media loaded on receiver) + when(mockRemoteMediaClient.getMediaStatus()).thenReturn(null); + castSessionListener.onSessionStarted(mockCastSession, /* sessionId= */ "session_2"); + + assertThat(castPlayer.getCurrentTimeline().isEmpty()).isTrue(); + assertThat(remoteCastPlayer.getCurrentTimeline().isEmpty()).isTrue(); + } } diff --git a/libraries/cast/src/test/java/androidx/media3/cast/CastTimelineTrackerTest.java b/libraries/cast/src/test/java/androidx/media3/cast/CastTimelineTrackerTest.java index 73637399874..c2d78864abc 100644 --- a/libraries/cast/src/test/java/androidx/media3/cast/CastTimelineTrackerTest.java +++ b/libraries/cast/src/test/java/androidx/media3/cast/CastTimelineTrackerTest.java @@ -17,31 +17,37 @@ import static com.google.common.truth.Truth.assertThat; import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; +import androidx.media3.cast.CastTimeline.ItemUid; import androidx.media3.common.C; import androidx.media3.common.MediaItem; import androidx.media3.common.MimeTypes; import androidx.media3.common.Timeline.Window; import androidx.media3.common.util.Util; import androidx.media3.test.utils.TimelineAsserts; -import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.android.gms.cast.MediaInfo; import com.google.android.gms.cast.MediaQueueItem; import com.google.android.gms.cast.MediaStatus; import com.google.android.gms.cast.framework.media.MediaQueue; import com.google.android.gms.cast.framework.media.RemoteMediaClient; import com.google.common.collect.ImmutableList; +import com.google.testing.junit.testparameterinjector.TestParameter; import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; import java.util.List; +import org.json.JSONException; +import org.json.JSONObject; import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; +import org.robolectric.RobolectricTestParameterInjector; /** Tests for {@link CastTimelineTracker}. */ -@RunWith(AndroidJUnit4.class) +@RunWith(RobolectricTestParameterInjector.class) public class CastTimelineTrackerTest { private static final long DURATION_2_MS = 2000; @@ -123,18 +129,19 @@ public void getCastTimelinePersistsDuration() { } @Test - public void getCastTimeline_onMediaItemsSet_correctMediaItemsInTimeline() { + public void getCastTimeline_registerMediaItemAndReset_correctMediaItemsInTimeline() { RemoteMediaClient mockRemoteMediaClient = mock(RemoteMediaClient.class); MediaQueue mockMediaQueue = mock(MediaQueue.class); MediaStatus mockMediaStatus = mock(MediaStatus.class); ImmutableList playlistMediaItems = ImmutableList.of(createMediaItem(0), createMediaItem(1)); - MediaQueueItem[] playlistMediaQueueItems = - new MediaQueueItem[] { - createMediaQueueItem(playlistMediaItems.get(0), 0), - createMediaQueueItem(playlistMediaItems.get(1), 1) - }; - castTimelineTracker.onMediaItemsSet(playlistMediaItems, playlistMediaQueueItems); + MediaQueueItem[] registeredQueueItems = + castTimelineTracker.registerMediaItems(playlistMediaItems).queueItems; + MediaQueueItem queueItem0 = + new MediaQueueItem.Builder(registeredQueueItems[0].getMedia()).setItemId(0).build(); + MediaQueueItem queueItem1 = + new MediaQueueItem.Builder(registeredQueueItems[1].getMedia()).setItemId(1).build(); + MediaQueueItem[] playlistMediaQueueItems = new MediaQueueItem[] {queueItem0, queueItem1}; // Mock remote media client state after adding two items. when(mockRemoteMediaClient.getMediaQueue()).thenReturn(mockMediaQueue); when(mockMediaQueue.getItemIds()).thenReturn(new int[] {0, 1}); @@ -152,9 +159,10 @@ public void getCastTimeline_onMediaItemsSet_correctMediaItemsInTimeline() { .isEqualTo(playlistMediaItems.get(1)); MediaItem thirdMediaItem = createMediaItem(2); - MediaQueueItem thirdMediaQueueItem = createMediaQueueItem(thirdMediaItem, 2); - castTimelineTracker.onMediaItemsSet( - ImmutableList.of(thirdMediaItem), new MediaQueueItem[] {thirdMediaQueueItem}); + MediaQueueItem[] newQueueItems = + castTimelineTracker.registerMediaItems(ImmutableList.of(thirdMediaItem)).queueItems; + MediaQueueItem thirdMediaQueueItem = + new MediaQueueItem.Builder(newQueueItems[0].getMedia()).setItemId(2).build(); // Mock remote media client state after a single item overrides the previous playlist. when(mockMediaQueue.getItemIds()).thenReturn(new int[] {2}); when(mockMediaStatus.getCurrentItemId()).thenReturn(2); @@ -169,23 +177,24 @@ public void getCastTimeline_onMediaItemsSet_correctMediaItemsInTimeline() { } @Test - public void getCastTimeline_onMediaItemsAdded_correctMediaItemsInTimeline() { + public void getCastTimeline_registerMediaItem_forAddition_correctMediaItemsInTimeline() { RemoteMediaClient mockRemoteMediaClient = mock(RemoteMediaClient.class); MediaQueue mockMediaQueue = mock(MediaQueue.class); MediaStatus mockMediaStatus = mock(MediaStatus.class); ImmutableList playlistMediaItems = ImmutableList.of(createMediaItem(0), createMediaItem(1)); - MediaQueueItem[] playlistQueueItems = - new MediaQueueItem[] { - createMediaQueueItem(playlistMediaItems.get(0), /* uid= */ 0), - createMediaQueueItem(playlistMediaItems.get(1), /* uid= */ 1) - }; + MediaQueueItem[] initialItems = + castTimelineTracker.registerMediaItems(playlistMediaItems).queueItems; + MediaQueueItem queueItem0 = + new MediaQueueItem.Builder(initialItems[0].getMedia()).setItemId(0).build(); + MediaQueueItem queueItem1 = + new MediaQueueItem.Builder(initialItems[1].getMedia()).setItemId(1).build(); + MediaQueueItem[] playlistQueueItems = new MediaQueueItem[] {queueItem0, queueItem1}; ImmutableList secondPlaylistMediaItems = new ImmutableList.Builder() .addAll(playlistMediaItems) .add(createMediaItem(2)) .build(); - castTimelineTracker.onMediaItemsAdded(playlistMediaItems, playlistQueueItems); when(mockRemoteMediaClient.getMediaQueue()).thenReturn(mockMediaQueue); when(mockRemoteMediaClient.getMediaStatus()).thenReturn(mockMediaStatus); // Mock remote media client state after two items have been added. @@ -203,11 +212,14 @@ public void getCastTimeline_onMediaItemsAdded_correctMediaItemsInTimeline() { .isEqualTo(playlistMediaItems.get(1)); // Mock remote media client state after adding a third item. + MediaQueueItem[] additionItems = + castTimelineTracker.registerMediaItems(ImmutableList.of(secondPlaylistMediaItems.get(2))) + .queueItems; + MediaQueueItem thirdQueueItem = + new MediaQueueItem.Builder(additionItems[0].getMedia()).setItemId(2).build(); List playlistThreeQueueItems = new ArrayList<>(Arrays.asList(playlistQueueItems)); - playlistThreeQueueItems.add(createMediaQueueItem(secondPlaylistMediaItems.get(2), 2)); - castTimelineTracker.onMediaItemsAdded( - secondPlaylistMediaItems, playlistThreeQueueItems.toArray(new MediaQueueItem[0])); + playlistThreeQueueItems.add(thirdQueueItem); when(mockMediaQueue.getItemIds()).thenReturn(new int[] {0, 1, 2}); when(mockMediaStatus.getQueueItems()).thenReturn(playlistThreeQueueItems); @@ -229,12 +241,13 @@ public void getCastTimeline_itemsRemoved_correctMediaItemsInTimelineAndMapCleane MediaStatus mockMediaStatus = mock(MediaStatus.class); ImmutableList playlistMediaItems = ImmutableList.of(createMediaItem(0), createMediaItem(1)); - MediaQueueItem[] initialPlaylistTwoQueueItems = - new MediaQueueItem[] { - createMediaQueueItem(playlistMediaItems.get(0), 0), - createMediaQueueItem(playlistMediaItems.get(1), 1) - }; - castTimelineTracker.onMediaItemsSet(playlistMediaItems, initialPlaylistTwoQueueItems); + MediaQueueItem[] registeredItems = + castTimelineTracker.registerMediaItems(playlistMediaItems).queueItems; + MediaQueueItem queueItem0 = + new MediaQueueItem.Builder(registeredItems[0].getMedia()).setItemId(0).build(); + MediaQueueItem queueItem1 = + new MediaQueueItem.Builder(registeredItems[1].getMedia()).setItemId(1).build(); + MediaQueueItem[] initialPlaylistTwoQueueItems = new MediaQueueItem[] {queueItem0, queueItem1}; when(mockRemoteMediaClient.getMediaQueue()).thenReturn(mockMediaQueue); when(mockRemoteMediaClient.getMediaStatus()).thenReturn(mockMediaStatus); // Mock remote media client state with two items in the queue. @@ -332,6 +345,797 @@ public void getCastTimeline_noCustomData_returnsFallbackMediaItems() { assertThat(window.mediaItem.localConfiguration.mimeType).isEqualTo(MimeTypes.AUDIO_MPEG); } + @Test + public void getCastTimeline_receiverReportsStreamTypeLive_windowIsLive() { + RemoteMediaClient mockRemoteMediaClient = mock(RemoteMediaClient.class); + MediaQueue mockMediaQueue = mock(MediaQueue.class); + MediaStatus mockMediaStatus = mock(MediaStatus.class); + MediaInfo mediaInfo = + new MediaInfo.Builder("https://example.com/live.m3u8") + .setContentType(MimeTypes.APPLICATION_M3U8) + .setStreamType(MediaInfo.STREAM_TYPE_LIVE) + .build(); + MediaQueueItem queueItem = new MediaQueueItem.Builder(mediaInfo).setItemId(1).build(); + when(mockRemoteMediaClient.getMediaQueue()).thenReturn(mockMediaQueue); + when(mockMediaQueue.getItemIds()).thenReturn(new int[] {1}); + when(mockRemoteMediaClient.getMediaStatus()).thenReturn(mockMediaStatus); + when(mockMediaStatus.getCurrentItemId()).thenReturn(1); + when(mockMediaStatus.getMediaInfo()).thenReturn(mediaInfo); + when(mockMediaStatus.getQueueItems()).thenReturn(Collections.singletonList(queueItem)); + + CastTimeline castTimeline = castTimelineTracker.getCastTimeline(mockRemoteMediaClient); + + assertThat(castTimeline.getWindowCount()).isEqualTo(1); + Window window = castTimeline.getWindow(/* windowIndex= */ 0, new Window()); + assertThat(window.isLive()).isTrue(); + assertThat(window.liveConfiguration).isEqualTo(MediaItem.LiveConfiguration.UNSET); + } + + @Test + public void getCastTimeline_activeMediaInfoLiveAndQueueItemStale_windowIsLive( + @TestParameter({"0" /* STREAM_TYPE_NONE */, "1" /* STREAM_TYPE_BUFFERED */}) + int staleStreamType) { + RemoteMediaClient mockRemoteMediaClient = mock(RemoteMediaClient.class); + MediaQueue mockMediaQueue = mock(MediaQueue.class); + MediaStatus mockMediaStatus = mock(MediaStatus.class); + MediaInfo staleQueueMediaInfo = + new MediaInfo.Builder("https://example.com/live.m3u8") + .setContentType(MimeTypes.APPLICATION_M3U8) + .setStreamType(staleStreamType) + .build(); + MediaQueueItem queueItem = new MediaQueueItem.Builder(staleQueueMediaInfo).setItemId(1).build(); + MediaInfo activeMediaInfo = + new MediaInfo.Builder("https://example.com/live.m3u8") + .setContentType(MimeTypes.APPLICATION_M3U8) + .setStreamType(MediaInfo.STREAM_TYPE_LIVE) + .build(); + when(mockRemoteMediaClient.getMediaQueue()).thenReturn(mockMediaQueue); + when(mockMediaQueue.getItemIds()).thenReturn(new int[] {1}); + when(mockRemoteMediaClient.getMediaStatus()).thenReturn(mockMediaStatus); + when(mockMediaStatus.getCurrentItemId()).thenReturn(1); + when(mockMediaStatus.getMediaInfo()).thenReturn(activeMediaInfo); + when(mockMediaStatus.getQueueItems()).thenReturn(Collections.singletonList(queueItem)); + + CastTimeline castTimeline = castTimelineTracker.getCastTimeline(mockRemoteMediaClient); + + assertThat(castTimeline.getWindowCount()).isEqualTo(1); + Window window = castTimeline.getWindow(/* windowIndex= */ 0, new Window()); + assertThat(window.isLive()).isTrue(); + assertThat(window.liveConfiguration).isEqualTo(MediaItem.LiveConfiguration.UNSET); + } + + @Test + public void getCastTimeline_unconfiguredLiveStreamResolvedByReceiver_windowIsLive() { + RemoteMediaClient mockRemoteMediaClient = mock(RemoteMediaClient.class); + MediaQueue mockMediaQueue = mock(MediaQueue.class); + MediaStatus mockMediaStatus = mock(MediaStatus.class); + MediaItem unconfiguredItem = + new MediaItem.Builder() + .setUri("https://example.com/live.m3u8") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build(); + MediaQueueItem[] registeredItems = + castTimelineTracker.registerMediaItems(ImmutableList.of(unconfiguredItem)).queueItems; + MediaInfo resolvedMediaInfo = + new MediaInfo.Builder(registeredItems[0].getMedia().getContentId()) + .setContentType(registeredItems[0].getMedia().getContentType()) + .setContentUrl(registeredItems[0].getMedia().getContentUrl()) + .setStreamType(MediaInfo.STREAM_TYPE_LIVE) + .setCustomData(registeredItems[0].getMedia().getCustomData()) + .build(); + MediaQueueItem activeQueueItem = + new MediaQueueItem.Builder(resolvedMediaInfo).setItemId(1).build(); + when(mockRemoteMediaClient.getMediaQueue()).thenReturn(mockMediaQueue); + when(mockMediaQueue.getItemIds()).thenReturn(new int[] {1}); + when(mockRemoteMediaClient.getMediaStatus()).thenReturn(mockMediaStatus); + when(mockMediaStatus.getCurrentItemId()).thenReturn(1); + when(mockMediaStatus.getMediaInfo()).thenReturn(resolvedMediaInfo); + when(mockMediaStatus.getQueueItems()).thenReturn(Collections.singletonList(activeQueueItem)); + + CastTimeline castTimeline = castTimelineTracker.getCastTimeline(mockRemoteMediaClient); + + assertThat(castTimeline.getWindowCount()).isEqualTo(1); + Window window = castTimeline.getWindow(/* windowIndex= */ 0, new Window()); + assertThat(window.isLive()).isTrue(); + assertThat(window.liveConfiguration).isEqualTo(MediaItem.LiveConfiguration.UNSET); + assertThat(window.mediaItem).isEqualTo(unconfiguredItem); + } + + @Test + public void getCastTimeline_slidingWindowQueueItems_fetchesOutofWindowItemsFromMediaQueue() { + RemoteMediaClient mockRemoteMediaClient = mock(RemoteMediaClient.class); + MediaQueue mockMediaQueue = mock(MediaQueue.class); + MediaStatus mockMediaStatus = mock(MediaStatus.class); + ImmutableList playlistMediaItems = + ImmutableList.of(createMediaItem(0), createMediaItem(1), createMediaItem(2)); + MediaQueueItem[] registeredItems = + castTimelineTracker.registerMediaItems(playlistMediaItems).queueItems; + MediaQueueItem queueItem0 = + new MediaQueueItem.Builder(registeredItems[0].getMedia()).setItemId(0).build(); + MediaQueueItem queueItem1 = + new MediaQueueItem.Builder(registeredItems[1].getMedia()).setItemId(1).build(); + MediaQueueItem queueItem2 = + new MediaQueueItem.Builder(registeredItems[2].getMedia()).setItemId(2).build(); + MediaQueueItem[] queueItems = new MediaQueueItem[] {queueItem0, queueItem1, queueItem2}; + when(mockRemoteMediaClient.getMediaQueue()).thenReturn(mockMediaQueue); + when(mockMediaQueue.getItemIds()).thenReturn(new int[] {0, 1, 2}); + when(mockRemoteMediaClient.getMediaStatus()).thenReturn(mockMediaStatus); + when(mockMediaStatus.getCurrentItemId()).thenReturn(0); + when(mockMediaStatus.getMediaInfo()).thenReturn(queueItems[0].getMedia()); + // Simulate sliding window: getQueueItems only returns the first 2 items (0 and 1) + when(mockMediaStatus.getQueueItems()).thenReturn(Arrays.asList(queueItems[0], queueItems[1])); + // Out-of-window item (index 2) is fetched on demand via mediaQueue.getItemAtIndex(2, true) + when(mockMediaQueue.getItemAtIndex(2, /* fetchIfNeeded= */ true)).thenReturn(queueItems[2]); + + CastTimeline castTimeline = castTimelineTracker.getCastTimeline(mockRemoteMediaClient); + + assertThat(castTimeline.getWindowCount()).isEqualTo(3); + assertThat(castTimeline.getWindow(/* windowIndex= */ 0, new Window()).mediaItem) + .isEqualTo(playlistMediaItems.get(0)); + assertThat(castTimeline.getWindow(/* windowIndex= */ 1, new Window()).mediaItem) + .isEqualTo(playlistMediaItems.get(1)); + assertThat(castTimeline.getWindow(/* windowIndex= */ 2, new Window()).mediaItem) + .isEqualTo(playlistMediaItems.get(2)); + } + + @Test + public void castTimeline_equals_validatesMediaItems() { + ImmutableList playlistMediaItems = + ImmutableList.of(createMediaItem(0), createMediaItem(1)); + // Setup tracker 1: Item at index 1 is MediaItem.EMPTY (not registered in placeholderTracker) + CastTimelineTracker placeholderTracker = new CastTimelineTracker(mediaItemConverter); + MediaQueueItem[] placeholderItems = + placeholderTracker.registerMediaItems(ImmutableList.of(playlistMediaItems.get(0))) + .queueItems; + MediaQueueItem queueItem0 = + new MediaQueueItem.Builder(placeholderItems[0].getMedia()).setItemId(0).build(); + MediaQueueItem queueItem1 = + new MediaQueueItem.Builder(mediaItemConverter.toMediaQueueItem(playlistMediaItems.get(1))) + .setItemId(1) + .build(); + MediaQueueItem[] queueItems = new MediaQueueItem[] {queueItem0, queueItem1}; + RemoteMediaClient mockPlaceholderClient = mock(RemoteMediaClient.class); + MediaQueue mockPlaceholderQueue = mock(MediaQueue.class); + MediaStatus mockPlaceholderStatus = mock(MediaStatus.class); + when(mockPlaceholderClient.getMediaQueue()).thenReturn(mockPlaceholderQueue); + when(mockPlaceholderQueue.getItemIds()).thenReturn(new int[] {0, 1}); + when(mockPlaceholderClient.getMediaStatus()).thenReturn(mockPlaceholderStatus); + when(mockPlaceholderStatus.getCurrentItemId()).thenReturn(0); + when(mockPlaceholderStatus.getMediaInfo()).thenReturn(queueItems[0].getMedia()); + when(mockPlaceholderStatus.getQueueItems()) + .thenReturn(Collections.singletonList(queueItems[0])); + + // Setup tracker 2: Item at index 1 is resolved to the real local MediaItem + CastTimelineTracker resolvedTracker = new CastTimelineTracker(mediaItemConverter); + MediaQueueItem[] resolvedItems = + resolvedTracker.registerMediaItems(playlistMediaItems).queueItems; + MediaQueueItem resolvedItem0 = + new MediaQueueItem.Builder(resolvedItems[0].getMedia()).setItemId(0).build(); + MediaQueueItem resolvedItem1 = + new MediaQueueItem.Builder(resolvedItems[1].getMedia()).setItemId(1).build(); + RemoteMediaClient mockResolvedClient = mock(RemoteMediaClient.class); + MediaQueue mockResolvedQueue = mock(MediaQueue.class); + MediaStatus mockResolvedStatus = mock(MediaStatus.class); + MediaQueueItem[] resolvedQueueItems = new MediaQueueItem[] {resolvedItem0, resolvedItem1}; + when(mockResolvedClient.getMediaQueue()).thenReturn(mockResolvedQueue); + when(mockResolvedQueue.getItemIds()).thenReturn(new int[] {0, 1}); + when(mockResolvedClient.getMediaStatus()).thenReturn(mockResolvedStatus); + when(mockResolvedStatus.getCurrentItemId()).thenReturn(0); + when(mockResolvedStatus.getMediaInfo()).thenReturn(resolvedQueueItems[0].getMedia()); + when(mockResolvedStatus.getQueueItems()) + .thenReturn(Collections.singletonList(resolvedQueueItems[0])); + when(mockResolvedQueue.getItemAtIndex(1, /* fetchIfNeeded= */ true)) + .thenReturn(resolvedQueueItems[1]); + + CastTimeline placeholderTimeline = placeholderTracker.getCastTimeline(mockPlaceholderClient); + CastTimeline resolvedTimeline = resolvedTracker.getCastTimeline(mockResolvedClient); + + assertThat(placeholderTimeline.equals(resolvedTimeline)).isFalse(); + assertThat(placeholderTimeline.hashCode()).isNotEqualTo(resolvedTimeline.hashCode()); + } + + @Test + public void getCastTimeline_itemFetchInFlight_returnsPlaceholderItem() { + RemoteMediaClient mockRemoteMediaClient = mock(RemoteMediaClient.class); + MediaQueue mockMediaQueue = mock(MediaQueue.class); + MediaStatus mockMediaStatus = mock(MediaStatus.class); + ImmutableList playlistMediaItems = + ImmutableList.of(createMediaItem(0), createMediaItem(1), createMediaItem(2)); + MediaQueueItem[] registeredItems = + castTimelineTracker.registerMediaItems(playlistMediaItems).queueItems; + MediaQueueItem queueItem0 = + new MediaQueueItem.Builder(registeredItems[0].getMedia()).setItemId(0).build(); + MediaQueueItem queueItem1 = + new MediaQueueItem.Builder(registeredItems[1].getMedia()).setItemId(1).build(); + MediaQueueItem queueItem2 = + new MediaQueueItem.Builder(registeredItems[2].getMedia()).setItemId(2).build(); + MediaQueueItem[] queueItems = new MediaQueueItem[] {queueItem0, queueItem1, queueItem2}; + when(mockRemoteMediaClient.getMediaQueue()).thenReturn(mockMediaQueue); + when(mockMediaQueue.getItemIds()).thenReturn(new int[] {0, 1, 2}); + when(mockRemoteMediaClient.getMediaStatus()).thenReturn(mockMediaStatus); + when(mockMediaStatus.getCurrentItemId()).thenReturn(0); + when(mockMediaStatus.getMediaInfo()).thenReturn(queueItems[0].getMedia()); + when(mockMediaStatus.getQueueItems()).thenReturn(Arrays.asList(queueItems[0], queueItems[1])); + when(mockMediaQueue.getItemAtIndex(2, /* fetchIfNeeded= */ true)).thenReturn(null); + + CastTimeline timeline = castTimelineTracker.getCastTimeline(mockRemoteMediaClient); + + assertThat(castTimelineTracker.hasPendingQueueFetches()).isTrue(); + assertThat(timeline.getWindowCount()).isEqualTo(3); + assertThat(timeline.getWindow(/* windowIndex= */ 2, new Window()).mediaItem) + .isEqualTo(MediaItem.EMPTY); + } + + @Test + public void getCastTimeline_inFlightFetchCompletes_updatesPlaceholderToRealItem() { + ImmutableList playlistMediaItems = + ImmutableList.of(createMediaItem(0), createMediaItem(1), createMediaItem(2)); + + // Setup in-flight state (index 2 is MediaItem.EMPTY) + CastTimelineTracker inFlightTracker = new CastTimelineTracker(mediaItemConverter); + MediaQueueItem[] inFlightRegistered = + inFlightTracker.registerMediaItems(playlistMediaItems).queueItems; + MediaQueueItem queueItem0 = + new MediaQueueItem.Builder(inFlightRegistered[0].getMedia()).setItemId(0).build(); + MediaQueueItem queueItem1 = + new MediaQueueItem.Builder(inFlightRegistered[1].getMedia()).setItemId(1).build(); + MediaQueueItem queueItem2 = + new MediaQueueItem.Builder(inFlightRegistered[2].getMedia()).setItemId(2).build(); + MediaQueueItem[] inFlightQueueItems = new MediaQueueItem[] {queueItem0, queueItem1, queueItem2}; + RemoteMediaClient mockInFlightClient = mock(RemoteMediaClient.class); + MediaQueue mockInFlightQueue = mock(MediaQueue.class); + MediaStatus mockInFlightStatus = mock(MediaStatus.class); + when(mockInFlightClient.getMediaQueue()).thenReturn(mockInFlightQueue); + when(mockInFlightQueue.getItemIds()).thenReturn(new int[] {0, 1, 2}); + when(mockInFlightClient.getMediaStatus()).thenReturn(mockInFlightStatus); + when(mockInFlightStatus.getCurrentItemId()).thenReturn(0); + when(mockInFlightStatus.getMediaInfo()).thenReturn(inFlightQueueItems[0].getMedia()); + when(mockInFlightStatus.getQueueItems()) + .thenReturn(Arrays.asList(inFlightQueueItems[0], inFlightQueueItems[1])); + when(mockInFlightQueue.getItemAtIndex(2, /* fetchIfNeeded= */ true)).thenReturn(null); + + // Setup completed state (index 2 resolved from MediaQueue cache) + CastTimelineTracker completedTracker = new CastTimelineTracker(mediaItemConverter); + MediaQueueItem[] completedRegistered = + completedTracker.registerMediaItems(playlistMediaItems).queueItems; + MediaQueueItem completed0 = + new MediaQueueItem.Builder(completedRegistered[0].getMedia()).setItemId(0).build(); + MediaQueueItem completed1 = + new MediaQueueItem.Builder(completedRegistered[1].getMedia()).setItemId(1).build(); + MediaQueueItem completed2 = + new MediaQueueItem.Builder(completedRegistered[2].getMedia()).setItemId(2).build(); + RemoteMediaClient mockCompletedClient = mock(RemoteMediaClient.class); + MediaQueue mockCompletedQueue = mock(MediaQueue.class); + MediaStatus mockCompletedStatus = mock(MediaStatus.class); + MediaQueueItem[] completedQueueItems = + new MediaQueueItem[] {completed0, completed1, completed2}; + when(mockCompletedClient.getMediaQueue()).thenReturn(mockCompletedQueue); + when(mockCompletedQueue.getItemIds()).thenReturn(new int[] {0, 1, 2}); + when(mockCompletedClient.getMediaStatus()).thenReturn(mockCompletedStatus); + when(mockCompletedStatus.getCurrentItemId()).thenReturn(0); + when(mockCompletedStatus.getMediaInfo()).thenReturn(completedQueueItems[0].getMedia()); + when(mockCompletedStatus.getQueueItems()) + .thenReturn(Arrays.asList(completedQueueItems[0], completedQueueItems[1])); + when(mockCompletedQueue.getItemAtIndex(2, /* fetchIfNeeded= */ true)) + .thenReturn(completedQueueItems[2]); + + CastTimeline inFlightTimeline = inFlightTracker.getCastTimeline(mockInFlightClient); + CastTimeline completedTimeline = completedTracker.getCastTimeline(mockCompletedClient); + + assertThat(inFlightTimeline.equals(completedTimeline)).isFalse(); + assertThat(completedTimeline.getWindow(/* windowIndex= */ 2, new Window()).mediaItem) + .isEqualTo(playlistMediaItems.get(2)); + } + + @Test + public void getCastTimeline_missingItemsExceedMaxFetchCount_capsNetworkFetchesAtMaxFetchCount() { + // Create a queue with more missing items than MAX_FETCH_COUNT + int overflowItems = 15; + int totalItems = CastTimelineTracker.MAX_FETCH_COUNT + overflowItems; + MediaQueueItem[] queueItems = new MediaQueueItem[totalItems]; + int[] itemIds = new int[totalItems]; + for (int i = 0; i < totalItems; i++) { + MediaItem mediaItem = createMediaItem(i); + queueItems[i] = createMediaQueueItem(mediaItem, ItemUid.generateItemUid(), i); + itemIds[i] = i; + } + RemoteMediaClient mockRemoteMediaClient = mock(RemoteMediaClient.class); + MediaQueue mockMediaQueue = mock(MediaQueue.class); + MediaStatus mockMediaStatus = mock(MediaStatus.class); + when(mockRemoteMediaClient.getMediaQueue()).thenReturn(mockMediaQueue); + when(mockMediaQueue.getItemIds()).thenReturn(itemIds); + when(mockRemoteMediaClient.getMediaStatus()).thenReturn(mockMediaStatus); + when(mockMediaStatus.getCurrentItemId()).thenReturn(0); + when(mockMediaStatus.getMediaInfo()).thenReturn(queueItems[0].getMedia()); + when(mockMediaStatus.getQueueItems()).thenReturn(Collections.singletonList(queueItems[0])); + + CastTimeline unused = castTimelineTracker.getCastTimeline(mockRemoteMediaClient); + + // First MAX_FETCH_COUNT missing items (indices 1..MAX_FETCH_COUNT) requested with fetchIfNeeded + // = true + for (int i = 1; i <= CastTimelineTracker.MAX_FETCH_COUNT; i++) { + verify(mockMediaQueue).getItemAtIndex(i, /* fetchIfNeeded= */ true); + } + // Overflow missing items beyond MAX_FETCH_COUNT requested with fetchIfNeeded = false + for (int i = CastTimelineTracker.MAX_FETCH_COUNT + 1; i < totalItems; i++) { + verify(mockMediaQueue).getItemAtIndex(i, /* fetchIfNeeded= */ false); + verify(mockMediaQueue, never()).getItemAtIndex(i, /* fetchIfNeeded= */ true); + } + } + + @Test + public void getCastTimeline_duplicateMediaItemsInQueue_distinguishesEachItem() { + RemoteMediaClient mockRemoteMediaClient = mock(RemoteMediaClient.class); + MediaQueue mockMediaQueue = mock(MediaQueue.class); + MediaStatus mockMediaStatus = mock(MediaStatus.class); + MediaItem duplicateItem1 = + new MediaItem.Builder() + .setUri("http://example.com/audio.mp3") + .setMimeType(MimeTypes.AUDIO_MPEG) + .setTag("first_instance") + .build(); + MediaItem duplicateItem2 = + new MediaItem.Builder() + .setUri("http://example.com/audio.mp3") + .setMimeType(MimeTypes.AUDIO_MPEG) + .setTag("second_instance") + .build(); + MediaQueueItem[] registeredItems = + castTimelineTracker.registerMediaItems(ImmutableList.of(duplicateItem1, duplicateItem2)) + .queueItems; + MediaQueueItem queueItem1 = + new MediaQueueItem.Builder(registeredItems[0].getMedia()).setItemId(101).build(); + MediaQueueItem queueItem2 = + new MediaQueueItem.Builder(registeredItems[1].getMedia()).setItemId(102).build(); + MediaQueueItem[] playlistMediaQueueItems = new MediaQueueItem[] {queueItem1, queueItem2}; + when(mockRemoteMediaClient.getMediaQueue()).thenReturn(mockMediaQueue); + when(mockMediaQueue.getItemIds()).thenReturn(new int[] {101, 102}); + when(mockRemoteMediaClient.getMediaStatus()).thenReturn(mockMediaStatus); + when(mockMediaStatus.getCurrentItemId()).thenReturn(101); + when(mockMediaStatus.getMediaInfo()).thenReturn(queueItem1.getMedia()); + when(mockMediaStatus.getQueueItems()).thenReturn(Arrays.asList(playlistMediaQueueItems)); + + CastTimeline castTimeline = castTimelineTracker.getCastTimeline(mockRemoteMediaClient); + + assertThat(castTimeline.getWindowCount()).isEqualTo(2); + assertThat(castTimeline.getWindow(/* windowIndex= */ 0, new Window()).mediaItem) + .isEqualTo(duplicateItem1); + assertThat(castTimeline.getWindow(/* windowIndex= */ 1, new Window()).mediaItem) + .isEqualTo(duplicateItem2); + } + + @Test + public void getCastTimeline_missingSyntheticId_fallsBackToConverter() { + RemoteMediaClient mockRemoteMediaClient = mock(RemoteMediaClient.class); + MediaQueue mockMediaQueue = mock(MediaQueue.class); + MediaStatus mockMediaStatus = mock(MediaStatus.class); + MediaInfo mediaInfo = + new MediaInfo.Builder("https://example.com/external.mp3") + .setContentType(MimeTypes.AUDIO_MPEG) + .setStreamType(MediaInfo.STREAM_TYPE_BUFFERED) + .build(); + MediaQueueItem queueItem = new MediaQueueItem.Builder(mediaInfo).setItemId(201).build(); + when(mockRemoteMediaClient.getMediaQueue()).thenReturn(mockMediaQueue); + when(mockMediaQueue.getItemIds()).thenReturn(new int[] {201}); + when(mockRemoteMediaClient.getMediaStatus()).thenReturn(mockMediaStatus); + when(mockMediaStatus.getCurrentItemId()).thenReturn(201); + when(mockMediaStatus.getMediaInfo()).thenReturn(mediaInfo); + when(mockMediaStatus.getQueueItems()).thenReturn(ImmutableList.of(queueItem)); + + CastTimeline castTimeline = castTimelineTracker.getCastTimeline(mockRemoteMediaClient); + + assertThat(castTimeline.getWindowCount()).isEqualTo(1); + Window window = castTimeline.getWindow(/* windowIndex= */ 0, new Window()); + assertThat(window.mediaItem.localConfiguration.uri.toString()) + .isEqualTo("https://example.com/external.mp3"); + } + + @Test + public void getCastTimeline_unknownSyntheticId_fallsBackToConverter() throws Exception { + RemoteMediaClient mockRemoteMediaClient = mock(RemoteMediaClient.class); + MediaQueue mockMediaQueue = mock(MediaQueue.class); + MediaStatus mockMediaStatus = mock(MediaStatus.class); + JSONObject customData = new JSONObject(); + JSONObject mediaItemJson = new JSONObject(); + mediaItemJson.put("uri", "https://example.com/foreign.mp3"); + mediaItemJson.put("mediaId", "foreign_media_id"); + customData.put("mediaItem", mediaItemJson); + customData.put("m3-syntheticId", "foreign-synthetic-id-999"); + MediaInfo mediaInfo = + new MediaInfo.Builder("https://example.com/foreign.mp3") + .setContentType(MimeTypes.AUDIO_MPEG) + .setStreamType(MediaInfo.STREAM_TYPE_BUFFERED) + .setCustomData(customData) + .build(); + MediaQueueItem queueItem = new MediaQueueItem.Builder(mediaInfo).setItemId(301).build(); + when(mockRemoteMediaClient.getMediaQueue()).thenReturn(mockMediaQueue); + when(mockMediaQueue.getItemIds()).thenReturn(new int[] {301}); + when(mockRemoteMediaClient.getMediaStatus()).thenReturn(mockMediaStatus); + when(mockMediaStatus.getCurrentItemId()).thenReturn(301); + when(mockMediaStatus.getMediaInfo()).thenReturn(mediaInfo); + when(mockMediaStatus.getQueueItems()).thenReturn(ImmutableList.of(queueItem)); + + CastTimeline castTimeline = castTimelineTracker.getCastTimeline(mockRemoteMediaClient); + + assertThat(castTimeline.getWindowCount()).isEqualTo(1); + Window window = castTimeline.getWindow(/* windowIndex= */ 0, new Window()); + assertThat(window.mediaItem.localConfiguration.uri.toString()) + .isEqualTo("https://example.com/foreign.mp3"); + } + + @Test + public void registerMediaItems_attachesSyntheticIdToCustomData() throws Exception { + MediaItem mediaItem1 = new MediaItem.Builder().setUri("http://example.com/1").build(); + MediaItem mediaItem2 = new MediaItem.Builder().setUri("http://example.com/2").build(); + + MediaQueueItem[] queueItems = + castTimelineTracker.registerMediaItems(ImmutableList.of(mediaItem1, mediaItem2)).queueItems; + + assertThat(queueItems).hasLength(2); + JSONObject customData1 = queueItems[0].getMedia().getCustomData(); + JSONObject customData2 = queueItems[1].getMedia().getCustomData(); + assertThat(customData1).isNotNull(); + assertThat(customData2).isNotNull(); + String syntheticId1 = customData1.getString("m3-syntheticId"); + String syntheticId2 = customData2.getString("m3-syntheticId"); + assertThat(syntheticId1).isNotEmpty(); + assertThat(syntheticId2).isNotEmpty(); + assertThat(syntheticId1).isNotEqualTo(syntheticId2); + } + + @Test + public void getCastTimeline_registeredMediaItems_populatesTimelineWithMediaItemsAndSyntheticUids() + throws Exception { + MediaItem mediaItem1 = + new MediaItem.Builder().setUri("http://example.com/1").setTag("tag1").build(); + MediaItem mediaItem2 = + new MediaItem.Builder().setUri("http://example.com/2").setTag("tag2").build(); + MediaQueueItem[] queueItems = + castTimelineTracker.registerMediaItems(ImmutableList.of(mediaItem1, mediaItem2)).queueItems; + String syntheticId1 = queueItems[0].getMedia().getCustomData().getString("m3-syntheticId"); + String syntheticId2 = queueItems[1].getMedia().getCustomData().getString("m3-syntheticId"); + MediaQueueItem queueItem1 = + new MediaQueueItem.Builder(queueItems[0].getMedia()).setItemId(101).build(); + MediaQueueItem queueItem2 = + new MediaQueueItem.Builder(queueItems[1].getMedia()).setItemId(102).build(); + RemoteMediaClient mockRemoteMediaClient = mock(RemoteMediaClient.class); + MediaQueue mockMediaQueue = mock(MediaQueue.class); + MediaStatus mockMediaStatus = mock(MediaStatus.class); + when(mockRemoteMediaClient.getMediaQueue()).thenReturn(mockMediaQueue); + when(mockMediaQueue.getItemIds()).thenReturn(new int[] {101, 102}); + when(mockRemoteMediaClient.getMediaStatus()).thenReturn(mockMediaStatus); + when(mockMediaStatus.getCurrentItemId()).thenReturn(101); + when(mockMediaStatus.getMediaInfo()).thenReturn(queueItem1.getMedia()); + when(mockMediaStatus.getQueueItems()).thenReturn(ImmutableList.of(queueItem1, queueItem2)); + + CastTimeline timeline = castTimelineTracker.getCastTimeline(mockRemoteMediaClient); + + assertThat(timeline.getWindowCount()).isEqualTo(2); + Window window1 = timeline.getWindow(/* windowIndex= */ 0, new Window()); + Window window2 = timeline.getWindow(/* windowIndex= */ 1, new Window()); + assertThat(window1.mediaItem).isEqualTo(mediaItem1); + assertThat(window2.mediaItem).isEqualTo(mediaItem2); + assertThat(window1.uid).isEqualTo(ItemUid.of(syntheticId1)); + assertThat(window2.uid).isEqualTo(ItemUid.of(syntheticId2)); + } + + @Test + public void registerMediaItems_multipleRegistrations_resolvesAllItemsInTimeline() + throws Exception { + MediaItem mediaItem1 = + new MediaItem.Builder().setUri("http://example.com/1").setTag("tag1").build(); + CastTimelineTracker.RegisteredMediaItems initialResult = + castTimelineTracker.registerMediaItems(ImmutableList.of(mediaItem1)); + MediaQueueItem[] initialQueueItems = initialResult.queueItems; + MediaItem mediaItem2 = + new MediaItem.Builder().setUri("http://example.com/2").setTag("tag2").build(); + CastTimelineTracker.RegisteredMediaItems additionalResult = + castTimelineTracker.registerMediaItems(ImmutableList.of(mediaItem2)); + MediaQueueItem[] additionalQueueItems = additionalResult.queueItems; + MediaQueueItem queueItem1 = + new MediaQueueItem.Builder(initialQueueItems[0].getMedia()).setItemId(101).build(); + MediaQueueItem queueItem2 = + new MediaQueueItem.Builder(additionalQueueItems[0].getMedia()).setItemId(102).build(); + RemoteMediaClient mockRemoteMediaClient = mock(RemoteMediaClient.class); + MediaQueue mockMediaQueue = mock(MediaQueue.class); + MediaStatus mockMediaStatus = mock(MediaStatus.class); + when(mockRemoteMediaClient.getMediaQueue()).thenReturn(mockMediaQueue); + when(mockMediaQueue.getItemIds()).thenReturn(new int[] {101, 102}); + when(mockRemoteMediaClient.getMediaStatus()).thenReturn(mockMediaStatus); + when(mockMediaStatus.getCurrentItemId()).thenReturn(101); + when(mockMediaStatus.getMediaInfo()).thenReturn(queueItem1.getMedia()); + when(mockMediaStatus.getQueueItems()).thenReturn(ImmutableList.of(queueItem1, queueItem2)); + + CastTimeline timeline = castTimelineTracker.getCastTimeline(mockRemoteMediaClient); + + assertThat(timeline.getWindowCount()).isEqualTo(2); + assertThat(timeline.getWindow(/* windowIndex= */ 0, new Window()).mediaItem) + .isEqualTo(mediaItem1); + assertThat(timeline.getWindow(/* windowIndex= */ 1, new Window()).mediaItem) + .isEqualTo(mediaItem2); + } + + @Test + public void reset_clearsPreviousRegistrationsInTracker() throws Exception { + MediaItem mediaItem1 = + new MediaItem.Builder().setUri("http://example.com/1").setTag("tag1").build(); + MediaQueueItem[] initialQueueItems = + castTimelineTracker.registerMediaItems(ImmutableList.of(mediaItem1)).queueItems; + + castTimelineTracker.reset(); + MediaItem mediaItem2 = + new MediaItem.Builder().setUri("http://example.com/2").setTag("tag2").build(); + MediaQueueItem[] newQueueItems = + castTimelineTracker.registerMediaItems(ImmutableList.of(mediaItem2)).queueItems; + MediaQueueItem oldQueueItem = + new MediaQueueItem.Builder(initialQueueItems[0].getMedia()).setItemId(101).build(); + MediaQueueItem newQueueItem = + new MediaQueueItem.Builder(newQueueItems[0].getMedia()).setItemId(102).build(); + RemoteMediaClient mockRemoteMediaClient = mock(RemoteMediaClient.class); + MediaQueue mockMediaQueue = mock(MediaQueue.class); + MediaStatus mockMediaStatus = mock(MediaStatus.class); + when(mockRemoteMediaClient.getMediaQueue()).thenReturn(mockMediaQueue); + when(mockMediaQueue.getItemIds()).thenReturn(new int[] {101, 102}); + when(mockRemoteMediaClient.getMediaStatus()).thenReturn(mockMediaStatus); + when(mockMediaStatus.getCurrentItemId()).thenReturn(102); + when(mockMediaStatus.getMediaInfo()).thenReturn(newQueueItem.getMedia()); + when(mockMediaStatus.getQueueItems()).thenReturn(ImmutableList.of(oldQueueItem, newQueueItem)); + + CastTimeline timeline = castTimelineTracker.getCastTimeline(mockRemoteMediaClient); + + assertThat(timeline.getWindowCount()).isEqualTo(2); + // Old item was cleared by reset(): sender-only custom tag is + // lost, fallback resolved from converter + assertThat( + timeline.getWindow(/* windowIndex= */ 0, new Window()).mediaItem.localConfiguration.tag) + .isNull(); + // New item registered after reset retains its custom sender tag + assertThat(timeline.getWindow(/* windowIndex= */ 1, new Window()).mediaItem) + .isEqualTo(mediaItem2); + } + + @Test + public void registerMediaItems_customDataIsNull_doesNotAttachCustomData() { + MediaItemConverter customConverter = + new MediaItemConverter() { + @Override + public MediaQueueItem toMediaQueueItem(MediaItem mediaItem) { + MediaInfo mediaInfo = + new MediaInfo.Builder(mediaItem.mediaId).setContentType("video/mp4").build(); + return new MediaQueueItem.Builder(mediaInfo).build(); + } + + @Override + public MediaItem toMediaItem(MediaQueueItem mediaQueueItem) { + return new MediaItem.Builder() + .setMediaId(mediaQueueItem.getMedia().getContentId()) + .build(); + } + }; + CastTimelineTracker customTracker = new CastTimelineTracker(customConverter); + + MediaItem mediaItem = new MediaItem.Builder().setMediaId("custom_media_id").build(); + MediaQueueItem[] queueItems = + customTracker.registerMediaItems(ImmutableList.of(mediaItem)).queueItems; + + assertThat(queueItems).hasLength(1); + assertThat(queueItems[0].getMedia().getCustomData()).isNull(); + } + + @Test + public void + getCastTimeline_inFlightItemAddition_preservesInFlightItemAcrossIntermediateUpdates() { + MediaItem item0 = + new MediaItem.Builder() + .setUri("http://example.com/0") + .setMediaId("item_0") + .setTag("custom_tag_0") + .build(); + CastTimelineTracker.RegisteredMediaItems initialResult = + castTimelineTracker.registerMediaItems(ImmutableList.of(item0)); + MediaQueueItem[] initialQueueItems = initialResult.queueItems; + MediaQueueItem queueItem0 = + new MediaQueueItem.Builder(initialQueueItems[0].getMedia()).setItemId(100).build(); + RemoteMediaClient mockClient = mock(RemoteMediaClient.class); + MediaQueue mockQueue = mock(MediaQueue.class); + MediaStatus mockStatus = mock(MediaStatus.class); + when(mockClient.getMediaQueue()).thenReturn(mockQueue); + when(mockClient.getMediaStatus()).thenReturn(mockStatus); + when(mockQueue.getItemIds()).thenReturn(new int[] {100}); + when(mockStatus.getCurrentItemId()).thenReturn(100); + when(mockStatus.getMediaInfo()).thenReturn(queueItem0.getMedia()); + when(mockStatus.getQueueItems()).thenReturn(ImmutableList.of(queueItem0)); + // Client initiates addition of Item 1 (in-flight) + MediaItem item1 = + new MediaItem.Builder() + .setUri("http://example.com/1") + .setMediaId("item_1") + .setTag("custom_tag_1") + .build(); + MediaQueueItem[] inFlightQueueItems = + castTimelineTracker.registerMediaItems(ImmutableList.of(item1)).queueItems; + // Intermediate getCastTimeline call while queue addition is still in-flight would trigger + // clean-up of unused items in the tracker. The in-flight item should be preserved. + CastTimeline unused = castTimelineTracker.getCastTimeline(mockClient); + MediaQueueItem queueItem1 = + new MediaQueueItem.Builder(inFlightQueueItems[0].getMedia()).setItemId(101).build(); + when(mockQueue.getItemIds()).thenReturn(new int[] {100, 101}); + when(mockStatus.getQueueItems()).thenReturn(ImmutableList.of(queueItem0, queueItem1)); + + CastTimeline confirmedTimeline = castTimelineTracker.getCastTimeline(mockClient); + + assertThat(confirmedTimeline.getWindowCount()).isEqualTo(2); + Window window1 = confirmedTimeline.getWindow(/* windowIndex= */ 1, new Window()); + assertThat(window1.mediaItem).isEqualTo(item1); + assertThat(window1.mediaItem.localConfiguration.tag).isEqualTo("custom_tag_1"); + } + + @Test + public void registerMediaItems_registersPendingItemsAndBindsUidOnConfirmation() { + MediaItem item = + new MediaItem.Builder() + .setUri("http://example.com/item") + .setMediaId("item_0") + .setClippingConfiguration( + new MediaItem.ClippingConfiguration.Builder().setStartPositionMs(5000).build()) + .build(); + + CastTimelineTracker.RegisteredMediaItems result = + castTimelineTracker.registerMediaItems(ImmutableList.of(item)); + + assertThat(result.itemUids).hasSize(1); + ItemUid registeredUid = result.itemUids.get(0); + assertThat(castTimelineTracker.getReceiverItemId(registeredUid)) + .isEqualTo(MediaQueueItem.INVALID_ITEM_ID); + + MediaQueueItem receiverQueueItem = + new MediaQueueItem.Builder(result.queueItems[0].getMedia()).setItemId(100).build(); + RemoteMediaClient mockClient = mock(RemoteMediaClient.class); + MediaQueue mockQueue = mock(MediaQueue.class); + MediaStatus mockStatus = mock(MediaStatus.class); + when(mockClient.getMediaQueue()).thenReturn(mockQueue); + when(mockClient.getMediaStatus()).thenReturn(mockStatus); + when(mockQueue.getItemIds()).thenReturn(new int[] {100}); + when(mockStatus.getCurrentItemId()).thenReturn(100); + when(mockStatus.getMediaInfo()).thenReturn(receiverQueueItem.getMedia()); + when(mockStatus.getQueueItems()).thenReturn(ImmutableList.of(receiverQueueItem)); + + CastTimeline timeline = castTimelineTracker.getCastTimeline(mockClient); + + assertThat(timeline.getWindowCount()).isEqualTo(1); + Window window = timeline.getWindow(/* windowIndex= */ 0, new Window()); + assertThat(window.uid).isEqualTo(registeredUid); + assertThat(window.mediaItem).isEqualTo(item); + assertThat(castTimelineTracker.getReceiverItemId(registeredUid)).isEqualTo(100); + } + + @Test + public void + registerMediaItems_multipleRegistrations_registersPendingItemsAndBindsUidOnConfirmation() { + MediaItem item0 = + new MediaItem.Builder().setUri("http://example.com/0").setMediaId("item_0").build(); + CastTimelineTracker.RegisteredMediaItems setResult = + castTimelineTracker.registerMediaItems(ImmutableList.of(item0)); + + MediaItem item1 = + new MediaItem.Builder() + .setUri("http://example.com/1") + .setMediaId("item_1") + .setClippingConfiguration( + new MediaItem.ClippingConfiguration.Builder().setStartPositionMs(3000).build()) + .build(); + CastTimelineTracker.RegisteredMediaItems addResult = + castTimelineTracker.registerMediaItems(ImmutableList.of(item1)); + + assertThat(addResult.itemUids).hasSize(1); + ItemUid addedUid = addResult.itemUids.get(0); + assertThat(castTimelineTracker.getReceiverItemId(addedUid)) + .isEqualTo(MediaQueueItem.INVALID_ITEM_ID); + + MediaQueueItem receiverQueueItem1 = + new MediaQueueItem.Builder(addResult.queueItems[0].getMedia()).setItemId(101).build(); + MediaQueueItem receiverQueueItem0 = + new MediaQueueItem.Builder(setResult.queueItems[0].getMedia()).setItemId(100).build(); + RemoteMediaClient mockClient = mock(RemoteMediaClient.class); + MediaQueue mockQueue = mock(MediaQueue.class); + MediaStatus mockStatus = mock(MediaStatus.class); + when(mockClient.getMediaQueue()).thenReturn(mockQueue); + when(mockClient.getMediaStatus()).thenReturn(mockStatus); + when(mockQueue.getItemIds()).thenReturn(new int[] {101, 100}); + when(mockStatus.getCurrentItemId()).thenReturn(101); + when(mockStatus.getMediaInfo()).thenReturn(receiverQueueItem1.getMedia()); + when(mockStatus.getQueueItems()) + .thenReturn(ImmutableList.of(receiverQueueItem1, receiverQueueItem0)); + + CastTimeline timeline = castTimelineTracker.getCastTimeline(mockClient); + + assertThat(timeline.getWindowCount()).isEqualTo(2); + Window confirmedWindow0 = timeline.getWindow(/* windowIndex= */ 0, new Window()); + assertThat(confirmedWindow0.uid).isEqualTo(addedUid); + assertThat(confirmedWindow0.mediaItem).isEqualTo(item1); + assertThat(castTimelineTracker.getReceiverItemId(addedUid)).isEqualTo(101); + } + + @Test + public void + getCastTimeline_lateJoiningSender_doesNotReplacePlaceholderUidWithForeignSyntheticId() { + MediaItem item0 = createMediaItem(0); + MediaItem item1 = createMediaItem(1); + MediaQueueItem queueItem0 = createMediaQueueItem(item0, ItemUid.of("foreign-uid-0"), 100); + MediaQueueItem queueItem1 = createMediaQueueItem(item1, ItemUid.of("foreign-uid-1"), 101); + + RemoteMediaClient mockClient = mock(RemoteMediaClient.class); + MediaQueue mockQueue = mock(MediaQueue.class); + MediaStatus mockStatus = mock(MediaStatus.class); + when(mockClient.getMediaQueue()).thenReturn(mockQueue); + when(mockClient.getMediaStatus()).thenReturn(mockStatus); + when(mockQueue.getItemIds()).thenReturn(new int[] {100, 101}); + when(mockStatus.getCurrentItemId()).thenReturn(100); + when(mockStatus.getMediaInfo()).thenReturn(queueItem0.getMedia()); + // Pass 1: item 100 is in MediaStatus, item 101 is outside the initial window (uncached). + when(mockStatus.getQueueItems()).thenReturn(ImmutableList.of(queueItem0)); + when(mockQueue.getItemAtIndex(1, /* fetchIfNeeded= */ true)).thenReturn(null); + + CastTimeline incompleteTimeline = castTimelineTracker.getCastTimeline(mockClient); + + assertThat(castTimelineTracker.hasPendingQueueFetches()).isTrue(); + assertThat(incompleteTimeline.getWindowCount()).isEqualTo(2); + // Item 100 was immediately present in MediaStatus, so it gets the foreign synthetic UID. + assertThat(incompleteTimeline.getWindow(0, new Window()).uid) + .isEqualTo(ItemUid.of("foreign-uid-0")); + // Item 101 was uncached on Pass 1, so it gets a generated placeholder UID. + Object placeholderUid1 = incompleteTimeline.getWindow(1, new Window()).uid; + assertThat(placeholderUid1).isNotEqualTo(ItemUid.of("foreign-uid-1")); + assertThat(incompleteTimeline.getWindow(1, new Window()).mediaItem).isEqualTo(MediaItem.EMPTY); + + // Pass 2: MediaQueue finishes fetching item 101 (which carries "foreign-uid-1" from another + // sender). Since "foreign-uid-1" was not registered in this sender's mediaItemsBySyntheticId, + // the tracker must preserve placeholderUid1 to avoid UID churn while populating its MediaItem. + when(mockQueue.getItemAtIndex(1, /* fetchIfNeeded= */ true)).thenReturn(queueItem1); + + CastTimeline completeTimeline = castTimelineTracker.getCastTimeline(mockClient); + + assertThat(castTimelineTracker.hasPendingQueueFetches()).isFalse(); + assertThat(completeTimeline.getWindow(1, new Window()).uid).isEqualTo(placeholderUid1); + assertThat(completeTimeline.getWindow(1, new Window()).mediaItem) + .isEqualTo(mediaItemConverter.toMediaItem(queueItem1)); + } + + @Test + public void + getCastTimeline_whenLoadingNewItemWhileCurrentItemIdIsStale_preservesDistinctItemUids() { + RemoteMediaClient mockClient = mock(RemoteMediaClient.class); + MediaQueue mockQueue = mock(MediaQueue.class); + MediaStatus mockStatus = mock(MediaStatus.class); + when(mockClient.getMediaQueue()).thenReturn(mockQueue); + when(mockClient.getMediaStatus()).thenReturn(mockStatus); + + ImmutableList playlist = ImmutableList.of(createMediaItem(0), createMediaItem(1)); + CastTimelineTracker.RegisteredMediaItems registered = + castTimelineTracker.registerMediaItems(playlist); + MediaQueueItem queueItem0 = + new MediaQueueItem.Builder(registered.queueItems[0].getMedia()).setItemId(10).build(); + MediaQueueItem queueItem1 = + new MediaQueueItem.Builder(registered.queueItems[1].getMedia()).setItemId(20).build(); + + // Receiver is loading item 20 after a seek, while currentItemId still reports item 10 and + // mediaStatus.getMediaInfo() already describes item 20. + when(mockQueue.getItemIds()).thenReturn(new int[] {10, 20}); + when(mockStatus.getCurrentItemId()).thenReturn(10); + when(mockStatus.getLoadingItemId()).thenReturn(20); + when(mockStatus.getMediaInfo()).thenReturn(queueItem1.getMedia()); + when(mockStatus.getQueueItems()).thenReturn(ImmutableList.of(queueItem0)); + when(mockQueue.getItemAtIndex(1, /* fetchIfNeeded= */ true)).thenReturn(queueItem1); + + CastTimeline timeline = castTimelineTracker.getCastTimeline(mockClient); + + assertThat(timeline.getWindowCount()).isEqualTo(2); + assertThat(timeline.getWindow(0, new Window()).uid).isEqualTo(registered.itemUids.get(0)); + assertThat(timeline.getWindow(1, new Window()).uid).isEqualTo(registered.itemUids.get(1)); + assertThat(castTimelineTracker.getItemUid(10)).isEqualTo(registered.itemUids.get(0)); + assertThat(castTimelineTracker.getItemUid(20)).isEqualTo(registered.itemUids.get(1)); + } + private MediaItem createMediaItem(int uid) { return new MediaItem.Builder() .setUri("http://www.google.com/" + uid) @@ -340,10 +1144,22 @@ private MediaItem createMediaItem(int uid) { .build(); } - private MediaQueueItem createMediaQueueItem(MediaItem mediaItem, int uid) { - return new MediaQueueItem.Builder(mediaItemConverter.toMediaQueueItem(mediaItem)) - .setItemId(uid) - .build(); + private static MediaQueueItem createMediaQueueItem( + MediaItem mediaItem, ItemUid itemUid, int uid) { + MediaQueueItem queueItem = + new MediaQueueItem.Builder(new DefaultMediaItemConverter().toMediaQueueItem(mediaItem)) + .setItemId(uid) + .build(); + JSONObject customData = + queueItem.getMedia() != null ? queueItem.getMedia().getCustomData() : null; + if (customData != null) { + try { + customData.put(CastTimelineTracker.KEY_SYNTHETIC_ID, itemUid.toString()); + } catch (JSONException e) { + throw new RuntimeException(e); + } + } + return queueItem; } private static RemoteMediaClient mockRemoteMediaClient( diff --git a/libraries/cast/src/test/java/androidx/media3/cast/DefaultCastPlayerTransferCallbackTest.java b/libraries/cast/src/test/java/androidx/media3/cast/DefaultCastPlayerTransferCallbackTest.java new file mode 100644 index 00000000000..c1f3065bbec --- /dev/null +++ b/libraries/cast/src/test/java/androidx/media3/cast/DefaultCastPlayerTransferCallbackTest.java @@ -0,0 +1,132 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.cast; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.media3.common.DeviceInfo; +import androidx.media3.common.MediaItem; +import androidx.media3.common.Player; +import androidx.media3.test.utils.FakePlayer; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link DefaultCastPlayerTransferCallback}. */ +@RunWith(AndroidJUnit4.class) +public final class DefaultCastPlayerTransferCallbackTest { + + private static final DeviceInfo DEVICE_INFO_REMOTE = + new DeviceInfo.Builder(DeviceInfo.PLAYBACK_TYPE_REMOTE).build(); + + private FakePlayer sourcePlayer; + private FakePlayer targetPlayer; + private DefaultCastPlayerTransferCallback transferCallback; + + @Before + public void setUp() { + sourcePlayer = new FakePlayer(); + targetPlayer = new FakePlayer(); + transferCallback = new DefaultCastPlayerTransferCallback(); + } + + @Test + public void transferState_filtersItemsWithNoPlaybackUri() { + MediaItem uriItem = new MediaItem.Builder().setUri("http://uri").build(); + MediaItem noUriItem = new MediaItem.Builder().setMediaId("no-uri").build(); + sourcePlayer.setMediaItems( + ImmutableList.of(uriItem, noUriItem), /* startIndex= */ 0, /* startPositionMs= */ 0); + + transferCallback.transferState(sourcePlayer, targetPlayer); + + assertThat(targetPlayer.getMediaItemCount()).isEqualTo(1); + assertThat(targetPlayer.getMediaItemAt(0)).isEqualTo(uriItem); + assertThat(targetPlayer.getCurrentMediaItemIndex()).isEqualTo(0); + } + + @Test + public void transferState_adjustsCurrentIndexWhenCurrentItemIsFiltered() { + MediaItem uriItem = new MediaItem.Builder().setUri("http://uri").build(); + MediaItem noUriItem = new MediaItem.Builder().setMediaId("no-uri").build(); + sourcePlayer.setMediaItems( + ImmutableList.of(uriItem, noUriItem), /* startIndex= */ 1, /* startPositionMs= */ 1000); + + transferCallback.transferState(sourcePlayer, targetPlayer); + + assertThat(targetPlayer.getMediaItemCount()).isEqualTo(1); + assertThat(targetPlayer.getMediaItemAt(0)).isEqualTo(uriItem); + assertThat(targetPlayer.getCurrentMediaItemIndex()).isEqualTo(0); + assertThat(targetPlayer.getCurrentPosition()).isEqualTo(0); + } + + @Test + public void transferState_whenCurrentItemIsNotFiltered_preservesPosition() { + MediaItem uriItem = new MediaItem.Builder().setUri("http://uri").build(); + MediaItem noUriItem = new MediaItem.Builder().setMediaId("no-uri").build(); + sourcePlayer.setMediaItems( + ImmutableList.of(uriItem, noUriItem), /* startIndex= */ 0, /* startPositionMs= */ 1000); + + transferCallback.transferState(sourcePlayer, targetPlayer); + + assertThat(targetPlayer.getMediaItemCount()).isEqualTo(1); + assertThat(targetPlayer.getMediaItemAt(0)).isEqualTo(uriItem); + assertThat(targetPlayer.getCurrentMediaItemIndex()).isEqualTo(0); + assertThat(targetPlayer.getCurrentPosition()).isEqualTo(1000); + } + + @Test + public void transferState_whenAllItemsAreFiltered_setsEmptyPlaylistAndIndexUnset() { + MediaItem noUriItem1 = new MediaItem.Builder().setMediaId("no-uri-1").build(); + MediaItem noUriItem2 = new MediaItem.Builder().setMediaId("no-uri-2").build(); + sourcePlayer.setMediaItems( + ImmutableList.of(noUriItem1, noUriItem2), /* startIndex= */ 0, /* startPositionMs= */ 0); + + transferCallback.transferState(sourcePlayer, targetPlayer); + + assertThat(targetPlayer.getMediaItemCount()).isEqualTo(0); + assertThat(targetPlayer.getCurrentMediaItemIndex()).isEqualTo(0); + } + + @Test + public void transferState_whenSourceTimelineIsEmptyAndTargetIsRemote_doesNotTransferState() { + targetPlayer.setDeviceInfo(DEVICE_INFO_REMOTE); + targetPlayer.setPlayWhenReady(true); + targetPlayer.setRepeatMode(Player.REPEAT_MODE_ONE); + + transferCallback.transferState(sourcePlayer, targetPlayer); + + assertThat(targetPlayer.getPlayWhenReady()).isTrue(); + assertThat(targetPlayer.getRepeatMode()).isEqualTo(Player.REPEAT_MODE_ONE); + } + + @Test + public void transferState_whenSourceTimelineIsEmptyAndTargetIsLocal_transfersState() { + sourcePlayer.setDeviceInfo(DEVICE_INFO_REMOTE); + sourcePlayer.setPlayWhenReady(true); + sourcePlayer.setRepeatMode(Player.REPEAT_MODE_ALL); + MediaItem uriItem = new MediaItem.Builder().setUri("http://uri").build(); + targetPlayer.setMediaItems( + ImmutableList.of(uriItem), /* startIndex= */ 0, /* startPositionMs= */ 0); + + transferCallback.transferState(sourcePlayer, targetPlayer); + + assertThat(targetPlayer.getMediaItemCount()).isEqualTo(0); + assertThat(targetPlayer.getPlayWhenReady()).isTrue(); + assertThat(targetPlayer.getRepeatMode()).isEqualTo(Player.REPEAT_MODE_ALL); + } +} diff --git a/libraries/cast/src/test/java/androidx/media3/cast/DefaultMediaItemConverterTest.java b/libraries/cast/src/test/java/androidx/media3/cast/DefaultMediaItemConverterTest.java index 5976b5824d5..5d884c258b6 100644 --- a/libraries/cast/src/test/java/androidx/media3/cast/DefaultMediaItemConverterTest.java +++ b/libraries/cast/src/test/java/androidx/media3/cast/DefaultMediaItemConverterTest.java @@ -27,6 +27,7 @@ import com.google.android.gms.cast.MediaInfo; import com.google.android.gms.cast.MediaQueueItem; import com.google.common.collect.ImmutableMap; +import java.util.Calendar; import org.junit.Test; import org.junit.runner.RunWith; @@ -64,6 +65,9 @@ public void serialize_deserialize_complete() { .setComposer("testComposer") .setDiscNumber(42) .setTrackNumber(23) + .setReleaseYear(2024) + .setReleaseMonth(5) + .setReleaseDay(27) .build()) .setMimeType(MimeTypes.APPLICATION_MPD) .setDrmConfiguration( @@ -228,4 +232,123 @@ public void toMediaItem_noCustomData_fallbackWithNonUriContentIdAndNoContentUrl( DefaultMediaItemConverter converter = new DefaultMediaItemConverter(); assertThrows(IllegalArgumentException.class, () -> converter.toMediaItem(queueItem)); } + + @Test + public void toMediaQueueItem_withLiveConfiguration_setsStreamTypeLive() { + DefaultMediaItemConverter converter = new DefaultMediaItemConverter(); + MediaItem mediaItem = + new MediaItem.Builder() + .setUri("http://example.com/live.m3u8") + .setLiveConfiguration( + new MediaItem.LiveConfiguration.Builder().setTargetOffsetMs(10000).build()) + .build(); + + MediaQueueItem queueItem = converter.toMediaQueueItem(mediaItem); + + assertThat(queueItem.getMedia().getStreamType()).isEqualTo(MediaInfo.STREAM_TYPE_LIVE); + } + + @Test + public void serialize_deserialize_liveConfiguration() { + MediaItem item = + new MediaItem.Builder() + .setUri("http://example.com/live.m3u8") + .setLiveConfiguration( + new MediaItem.LiveConfiguration.Builder() + .setTargetOffsetMs(10000) + .setMinOffsetMs(5000) + .setMaxOffsetMs(20000) + .setMinPlaybackSpeed(0.9f) + .setMaxPlaybackSpeed(1.1f) + .build()) + .build(); + + DefaultMediaItemConverter converter = new DefaultMediaItemConverter(); + MediaQueueItem queueItem = converter.toMediaQueueItem(item); + MediaItem reconstructedItem = converter.toMediaItem(queueItem); + assertThat(reconstructedItem.liveConfiguration).isEqualTo(item.liveConfiguration); + } + + @Test + public void toMediaQueueItem_withoutLiveConfiguration_setsStreamTypeInvalid() { + DefaultMediaItemConverter converter = new DefaultMediaItemConverter(); + MediaItem mediaItem = new MediaItem.Builder().setUri("http://example.com/live.m3u8").build(); + + MediaQueueItem queueItem = converter.toMediaQueueItem(mediaItem); + + assertThat(queueItem.getMedia().getStreamType()).isEqualTo(MediaInfo.STREAM_TYPE_INVALID); + } + + @Test + public void toMediaQueueItem_withReleaseDate_setsKeyReleaseDate() { + DefaultMediaItemConverter converter = new DefaultMediaItemConverter(); + MediaItem mediaItem = + new MediaItem.Builder() + .setUri("http://example.com") + .setMediaMetadata( + new MediaMetadata.Builder() + .setReleaseYear(2024) + .setReleaseMonth(5) + .setReleaseDay(27) + .build()) + .build(); + + MediaQueueItem queueItem = converter.toMediaQueueItem(mediaItem); + com.google.android.gms.cast.MediaMetadata metadata = queueItem.getMedia().getMetadata(); + + assertThat(metadata.containsKey(com.google.android.gms.cast.MediaMetadata.KEY_RELEASE_DATE)) + .isTrue(); + Calendar calendar = + metadata.getDate(com.google.android.gms.cast.MediaMetadata.KEY_RELEASE_DATE); + assertThat(calendar).isNotNull(); + assertThat(calendar.get(Calendar.YEAR)).isEqualTo(2024); + assertThat(calendar.get(Calendar.MONTH)).isEqualTo(Calendar.MAY); + assertThat(calendar.get(Calendar.DAY_OF_MONTH)).isEqualTo(27); + assertThat(metadata.getDateAsString(com.google.android.gms.cast.MediaMetadata.KEY_RELEASE_DATE)) + .isEqualTo("20240527"); + } + + @Test + public void toMediaQueueItem_withReleaseYearOnly_defaultsMonthAndDay() { + DefaultMediaItemConverter converter = new DefaultMediaItemConverter(); + MediaItem mediaItem = + new MediaItem.Builder() + .setUri("http://example.com") + .setMediaMetadata(new MediaMetadata.Builder().setReleaseYear(2024).build()) + .build(); + + MediaQueueItem queueItem = converter.toMediaQueueItem(mediaItem); + com.google.android.gms.cast.MediaMetadata metadata = queueItem.getMedia().getMetadata(); + + assertThat(metadata.containsKey(com.google.android.gms.cast.MediaMetadata.KEY_RELEASE_DATE)) + .isTrue(); + assertThat(metadata.getDateAsString(com.google.android.gms.cast.MediaMetadata.KEY_RELEASE_DATE)) + .isEqualTo("20240101"); + } + + @Test + public void toMediaItem_withReleaseDate_setsReleaseDate() { + com.google.android.gms.cast.MediaMetadata gmsMetadata = + new com.google.android.gms.cast.MediaMetadata( + com.google.android.gms.cast.MediaMetadata.MEDIA_TYPE_MOVIE); + Calendar calendar = Calendar.getInstance(); + calendar.clear(); + calendar.set(2024, Calendar.MAY, 27); + gmsMetadata.putDate(com.google.android.gms.cast.MediaMetadata.KEY_RELEASE_DATE, calendar); + MediaInfo mediaInfo = + new MediaInfo.Builder("contentId") + .setContentUrl("http://example.com/url") + .setContentType(MimeTypes.VIDEO_MP4) + .setStreamType(MediaInfo.STREAM_TYPE_BUFFERED) + .setMetadata(gmsMetadata) + .build(); + MediaQueueItem queueItem = new MediaQueueItem.Builder(mediaInfo).build(); + + DefaultMediaItemConverter converter = new DefaultMediaItemConverter(); + MediaItem mediaItem = converter.toMediaItem(queueItem); + + assertThat(mediaItem.mediaMetadata.releaseYear).isEqualTo(2024); + assertThat(mediaItem.mediaMetadata.releaseMonth).isEqualTo(5); + assertThat(mediaItem.mediaMetadata.releaseDay).isEqualTo(27); + } } diff --git a/libraries/cast/src/test/java/androidx/media3/cast/MediaRouteButtonTest.kt b/libraries/cast/src/test/java/androidx/media3/cast/MediaRouteButtonTest.kt index 752263f534a..c9fab7d5041 100644 --- a/libraries/cast/src/test/java/androidx/media3/cast/MediaRouteButtonTest.kt +++ b/libraries/cast/src/test/java/androidx/media3/cast/MediaRouteButtonTest.kt @@ -18,6 +18,9 @@ package androidx.media3.cast import android.content.Context import android.view.ContextThemeWrapper import androidx.compose.runtime.Composable +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.setValue import androidx.compose.ui.test.assertIsDisplayed import androidx.compose.ui.test.junit4.v2.createComposeRule import androidx.compose.ui.test.onNodeWithContentDescription @@ -100,6 +103,27 @@ class MediaRouteButtonTest { composeTestRule.onNodeWithContentDescription(buttonContentDescription).assertIsDisplayed() } + @Test + fun initializeMediaRouteButton_leavesCompositionBeforeCastInitialization_unregistersMediaRouteSelectorListener() = + runTest { + cast.initialize(mockCastContextInitializer) + var isComposed by mutableStateOf(true) + + composeTestRule.setContent { + if (isComposed) { + MediaRouteButtonContainer {} + } + } + composeTestRule.waitForIdle() + + assertThat(cast.hasPendingMediaRouteSelectorListeners()).isTrue() + + isComposed = false + composeTestRule.waitForIdle() + + assertThat(cast.hasPendingMediaRouteSelectorListeners()).isFalse() + } + @Test fun initializeMediaRouteButton_notInitialized_notThrowsException() = runTest { cast.initialize(mockCastContextInitializer) diff --git a/libraries/cast/src/test/java/androidx/media3/cast/QueuedOperationTest.java b/libraries/cast/src/test/java/androidx/media3/cast/QueuedOperationTest.java new file mode 100644 index 00000000000..6485190380f --- /dev/null +++ b/libraries/cast/src/test/java/androidx/media3/cast/QueuedOperationTest.java @@ -0,0 +1,510 @@ +/* + * Copyright (C) 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.cast; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.media3.cast.CastTimeline.ItemData; +import androidx.media3.cast.CastTimeline.ItemUid; +import androidx.media3.cast.QueuedOperation.AddMediaItemsOperation; +import androidx.media3.cast.QueuedOperation.MoveMediaItemsOperation; +import androidx.media3.cast.QueuedOperation.QueueSnapshot; +import androidx.media3.cast.QueuedOperation.RemoveMediaItemsOperation; +import androidx.media3.cast.QueuedOperation.SeekOperation; +import androidx.media3.cast.QueuedOperation.SetMediaItemsOperation; +import androidx.media3.common.C; +import androidx.media3.common.MediaItem; +import androidx.media3.common.MimeTypes; +import androidx.media3.common.Timeline; +import androidx.media3.common.Timeline.Window; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import java.util.HashMap; +import java.util.Map; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link QueuedOperation#createMaskedSnapshot}. */ +@RunWith(AndroidJUnit4.class) +public final class QueuedOperationTest { + + private CastTimelineTracker timelineTracker; + + @Before + public void setUp() { + timelineTracker = new CastTimelineTracker(new DefaultMediaItemConverter()); + } + + @Test + public void setMediaItems_createMaskedSnapshot_createsSnapshotWithItemsStartIndexAndPosition() { + MediaItem item1 = createMediaItem("id1"); + MediaItem item2 = createMediaItem("id2"); + SetMediaItemsOperation operation = + new SetMediaItemsOperation( + ImmutableList.of(item1, item2), /* startIndex= */ 1, /* startPositionMs= */ 12_345L); + + QueueSnapshot snapshot = operation.createMaskedSnapshot(QueueSnapshot.EMPTY, timelineTracker); + + assertThat(snapshot.currentWindowIndex).isEqualTo(1); + assertThat(snapshot.maskedPositionMs).isEqualTo(12_345L); + CastTimeline timeline = snapshot.timeline; + assertThat(timeline.getWindowCount()).isEqualTo(2); + Window window0 = timeline.getWindow(/* windowIndex= */ 0, new Window()); + Window window1 = timeline.getWindow(/* windowIndex= */ 1, new Window()); + assertThat(window0.mediaItem).isEqualTo(item1); + assertThat(window0.durationUs).isEqualTo(C.TIME_UNSET); + assertThat(window0.isDynamic).isTrue(); + assertThat(window1.mediaItem).isEqualTo(item2); + assertThat(window1.durationUs).isEqualTo(C.TIME_UNSET); + assertThat(window1.isDynamic).isTrue(); + assertThat(window0.uid).isNotEqualTo(window1.uid); + } + + @Test + public void setMediaItems_createMaskedSnapshot_withClippingConfiguration_setsDefaultPosition() { + MediaItem item1 = + createMediaItem("id1") + .buildUpon() + .setClippingConfiguration( + new MediaItem.ClippingConfiguration.Builder().setStartPositionMs(5_000L).build()) + .build(); + MediaItem item2 = createMediaItem("id2"); + SetMediaItemsOperation operation = + new SetMediaItemsOperation( + ImmutableList.of(item1, item2), /* startIndex= */ 0, /* startPositionMs= */ 0L); + + QueueSnapshot snapshot = operation.createMaskedSnapshot(QueueSnapshot.EMPTY, timelineTracker); + + assertThat(snapshot.timeline.getWindowCount()).isEqualTo(2); + Window window0 = snapshot.timeline.getWindow(/* windowIndex= */ 0, new Window()); + Window window1 = snapshot.timeline.getWindow(/* windowIndex= */ 1, new Window()); + assertThat(window0.defaultPositionUs).isEqualTo(5_000_000L); + assertThat(window1.defaultPositionUs).isEqualTo(0L); + } + + @Test + public void setMediaItems_createMaskedSnapshot_emptyList_createsEmptySnapshot() { + SetMediaItemsOperation operation = + new SetMediaItemsOperation( + ImmutableList.of(), /* startIndex= */ 0, /* startPositionMs= */ 0L); + + QueueSnapshot snapshot = operation.createMaskedSnapshot(QueueSnapshot.EMPTY, timelineTracker); + + assertThat(snapshot.timeline.isEmpty()).isTrue(); + assertThat(snapshot.timeline.getWindowCount()).isEqualTo(0); + assertThat(snapshot.currentWindowIndex).isEqualTo(0); + } + + @Test + public void addMediaItems_createMaskedSnapshot_prepend_insertsAtBeginningAndShiftsWindowIndex() { + QueueSnapshot initialSnapshot = createSampleSnapshot(/* currentWindowIndex= */ 1, "A", "B"); + Object uidA = initialSnapshot.timeline.getWindow(0, new Window()).uid; + Object uidB = initialSnapshot.timeline.getWindow(1, new Window()).uid; + MediaItem itemC = createMediaItem("C"); + AddMediaItemsOperation operation = + new AddMediaItemsOperation(/* insertBeforePeriodUid= */ uidA, ImmutableList.of(itemC)); + + QueueSnapshot updatedSnapshot = + operation.createMaskedSnapshot(initialSnapshot, timelineTracker); + + CastTimeline updatedTimeline = updatedSnapshot.timeline; + assertThat(updatedTimeline.getWindowCount()).isEqualTo(3); + assertThat(getMediaIdAt(updatedTimeline, 0)).isEqualTo("C"); + assertThat(getMediaIdAt(updatedTimeline, 1)).isEqualTo("A"); + assertThat(getMediaIdAt(updatedTimeline, 2)).isEqualTo("B"); + assertThat(updatedTimeline.getWindow(1, new Window()).uid).isEqualTo(uidA); + assertThat(updatedTimeline.getWindow(2, new Window()).uid).isEqualTo(uidB); + // Playing item "B" moved from index 1 to index 2. + assertThat(updatedSnapshot.currentWindowIndex).isEqualTo(2); + } + + @Test + public void addMediaItems_createMaskedSnapshot_middle_insertsAtGivenIndex() { + QueueSnapshot initialSnapshot = createSampleSnapshot(/* currentWindowIndex= */ 0, "A", "B"); + Object uidB = initialSnapshot.timeline.getWindow(1, new Window()).uid; + MediaItem itemC = createMediaItem("C"); + AddMediaItemsOperation operation = + new AddMediaItemsOperation(/* insertBeforePeriodUid= */ uidB, ImmutableList.of(itemC)); + + QueueSnapshot updatedSnapshot = + operation.createMaskedSnapshot(initialSnapshot, timelineTracker); + + assertThat(updatedSnapshot.timeline.getWindowCount()).isEqualTo(3); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 0)).isEqualTo("A"); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 1)).isEqualTo("C"); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 2)).isEqualTo("B"); + assertThat(updatedSnapshot.currentWindowIndex).isEqualTo(0); + } + + @Test + public void addMediaItems_createMaskedSnapshot_append_insertsAtEnd() { + QueueSnapshot initialSnapshot = createSampleSnapshot(/* currentWindowIndex= */ 0, "A", "B"); + MediaItem itemC = createMediaItem("C"); + AddMediaItemsOperation operation = + new AddMediaItemsOperation(/* insertBeforePeriodUid= */ null, ImmutableList.of(itemC)); + + QueueSnapshot updatedSnapshot = + operation.createMaskedSnapshot(initialSnapshot, timelineTracker); + + assertThat(updatedSnapshot.timeline.getWindowCount()).isEqualTo(3); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 0)).isEqualTo("A"); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 1)).isEqualTo("B"); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 2)).isEqualTo("C"); + } + + @Test + public void addMediaItems_createMaskedSnapshot_missingAnchorUid_returnsSameSnapshot() { + QueueSnapshot initialSnapshot = createSampleSnapshot(/* currentWindowIndex= */ 0, "A", "B"); + MediaItem itemC = createMediaItem("C"); + AddMediaItemsOperation operation = + new AddMediaItemsOperation( + /* insertBeforePeriodUid= */ ItemUid.generateItemUid(), ImmutableList.of(itemC)); + + QueueSnapshot updatedSnapshot = + operation.createMaskedSnapshot(initialSnapshot, timelineTracker); + + assertThat(updatedSnapshot).isSameInstanceAs(initialSnapshot); + } + + @Test + public void addMediaItems_createMaskedSnapshot_toEmptySnapshot_createsSnapshotWithItems() { + MediaItem itemA = createMediaItem("A"); + AddMediaItemsOperation operation = + new AddMediaItemsOperation(/* insertBeforePeriodUid= */ null, ImmutableList.of(itemA)); + + QueueSnapshot updatedSnapshot = + operation.createMaskedSnapshot(QueueSnapshot.EMPTY, timelineTracker); + + assertThat(updatedSnapshot.timeline.getWindowCount()).isEqualTo(1); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 0)).isEqualTo("A"); + assertThat(updatedSnapshot.currentWindowIndex).isEqualTo(0); + } + + @Test + public void removeMediaItems_createMaskedSnapshot_singleItem_removesItemAndShiftsWindowIndex() { + QueueSnapshot initialSnapshot = + createSampleSnapshot(/* currentWindowIndex= */ 2, "A", "B", "C"); + Object uidA = initialSnapshot.timeline.getWindow(0, new Window()).uid; + Object uidB = initialSnapshot.timeline.getWindow(1, new Window()).uid; + Object uidC = initialSnapshot.timeline.getWindow(2, new Window()).uid; + RemoveMediaItemsOperation operation = new RemoveMediaItemsOperation(ImmutableList.of(uidB)); + + QueueSnapshot updatedSnapshot = + operation.createMaskedSnapshot(initialSnapshot, timelineTracker); + + assertThat(updatedSnapshot.timeline.getWindowCount()).isEqualTo(2); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 0)).isEqualTo("A"); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 1)).isEqualTo("C"); + assertThat(updatedSnapshot.timeline.getWindow(0, new Window()).uid).isEqualTo(uidA); + assertThat(updatedSnapshot.timeline.getWindow(1, new Window()).uid).isEqualTo(uidC); + // Playing item "C" shifted from index 2 to index 1. + assertThat(updatedSnapshot.currentWindowIndex).isEqualTo(1); + } + + @Test + public void removeMediaItems_createMaskedSnapshot_range_removesItems() { + QueueSnapshot initialSnapshot = + createSampleSnapshot(/* currentWindowIndex= */ 0, "A", "B", "C", "D"); + Object uidB = initialSnapshot.timeline.getWindow(1, new Window()).uid; + Object uidC = initialSnapshot.timeline.getWindow(2, new Window()).uid; + RemoveMediaItemsOperation operation = + new RemoveMediaItemsOperation(ImmutableList.of(uidB, uidC)); + + QueueSnapshot updatedSnapshot = + operation.createMaskedSnapshot(initialSnapshot, timelineTracker); + + assertThat(updatedSnapshot.timeline.getWindowCount()).isEqualTo(2); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 0)).isEqualTo("A"); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 1)).isEqualTo("D"); + } + + @Test + public void removeMediaItems_createMaskedSnapshot_all_returnsEmptySnapshot() { + QueueSnapshot initialSnapshot = createSampleSnapshot(/* currentWindowIndex= */ 1, "A", "B"); + Object uidA = initialSnapshot.timeline.getWindow(0, new Window()).uid; + Object uidB = initialSnapshot.timeline.getWindow(1, new Window()).uid; + RemoveMediaItemsOperation operation = + new RemoveMediaItemsOperation(ImmutableList.of(uidA, uidB)); + + QueueSnapshot updatedSnapshot = + operation.createMaskedSnapshot(initialSnapshot, timelineTracker); + + assertThat(updatedSnapshot.timeline.isEmpty()).isTrue(); + assertThat(updatedSnapshot.currentWindowIndex).isEqualTo(0); + assertThat(updatedSnapshot.maskedPositionMs).isEqualTo(0L); + } + + @Test + public void removeMediaItems_createMaskedSnapshot_emptyRange_returnsSameSnapshot() { + QueueSnapshot initialSnapshot = createSampleSnapshot(/* currentWindowIndex= */ 0, "A", "B"); + RemoveMediaItemsOperation operation = new RemoveMediaItemsOperation(ImmutableList.of()); + + QueueSnapshot updatedSnapshot = + operation.createMaskedSnapshot(initialSnapshot, timelineTracker); + + assertThat(updatedSnapshot).isSameInstanceAs(initialSnapshot); + } + + @Test + public void moveMediaItems_createMaskedSnapshot_forward_movesItemsAndTracksPlayingWindow() { + QueueSnapshot initialSnapshot = + createSampleSnapshot(/* currentWindowIndex= */ 0, "A", "B", "C", "D"); + Object uidA = initialSnapshot.timeline.getWindow(0, new Window()).uid; + Object uidD = initialSnapshot.timeline.getWindow(3, new Window()).uid; + // Move "A" before "D" -> [B, C, A, D] + MoveMediaItemsOperation operation = + new MoveMediaItemsOperation(ImmutableList.of(uidA), /* insertBeforePeriodUid= */ uidD); + + QueueSnapshot updatedSnapshot = + operation.createMaskedSnapshot(initialSnapshot, timelineTracker); + + assertThat(updatedSnapshot.timeline.getWindowCount()).isEqualTo(4); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 0)).isEqualTo("B"); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 1)).isEqualTo("C"); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 2)).isEqualTo("A"); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 3)).isEqualTo("D"); + // Playing item "A" moved from index 0 to index 2. + assertThat(updatedSnapshot.currentWindowIndex).isEqualTo(2); + } + + @Test + public void moveMediaItems_createMaskedSnapshot_backward_movesItemsCorrectly() { + QueueSnapshot initialSnapshot = + createSampleSnapshot(/* currentWindowIndex= */ 0, "A", "B", "C", "D"); + Object uidB = initialSnapshot.timeline.getWindow(1, new Window()).uid; + Object uidD = initialSnapshot.timeline.getWindow(3, new Window()).uid; + // Move "D" before "B" -> [A, D, B, C] + MoveMediaItemsOperation operation = + new MoveMediaItemsOperation(ImmutableList.of(uidD), /* insertBeforePeriodUid= */ uidB); + + QueueSnapshot updatedSnapshot = + operation.createMaskedSnapshot(initialSnapshot, timelineTracker); + + assertThat(updatedSnapshot.timeline.getWindowCount()).isEqualTo(4); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 0)).isEqualTo("A"); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 1)).isEqualTo("D"); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 2)).isEqualTo("B"); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 3)).isEqualTo("C"); + } + + @Test + public void moveMediaItems_createMaskedSnapshot_slice_movesMultipleItems() { + QueueSnapshot initialSnapshot = + createSampleSnapshot(/* currentWindowIndex= */ 0, "A", "B", "C", "D"); + Object uidA = initialSnapshot.timeline.getWindow(0, new Window()).uid; + Object uidB = initialSnapshot.timeline.getWindow(1, new Window()).uid; + // Move "A", "B" to end -> [C, D, A, B] + MoveMediaItemsOperation operation = + new MoveMediaItemsOperation( + ImmutableList.of(uidA, uidB), /* insertBeforePeriodUid= */ null); + + QueueSnapshot updatedSnapshot = + operation.createMaskedSnapshot(initialSnapshot, timelineTracker); + + assertThat(updatedSnapshot.timeline.getWindowCount()).isEqualTo(4); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 0)).isEqualTo("C"); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 1)).isEqualTo("D"); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 2)).isEqualTo("A"); + assertThat(getMediaIdAt(updatedSnapshot.timeline, 3)).isEqualTo("B"); + } + + @Test + public void moveMediaItems_createMaskedSnapshot_samePosition_returnsSameSnapshot() { + QueueSnapshot initialSnapshot = + createSampleSnapshot(/* currentWindowIndex= */ 0, "A", "B", "C"); + Object uidB = initialSnapshot.timeline.getWindow(1, new Window()).uid; + Object uidC = initialSnapshot.timeline.getWindow(2, new Window()).uid; + MoveMediaItemsOperation operation = + new MoveMediaItemsOperation(ImmutableList.of(uidB), /* insertBeforePeriodUid= */ uidC); + + QueueSnapshot updatedSnapshot = + operation.createMaskedSnapshot(initialSnapshot, timelineTracker); + + assertThat(updatedSnapshot).isSameInstanceAs(initialSnapshot); + } + + @Test + public void moveMediaItems_createMaskedSnapshot_preservesItemDataAndUids() { + MediaItem itemA = createMediaItem("A"); + MediaItem itemB = createMediaItem("B"); + ItemUid uidA = ItemUid.generateItemUid(); + ItemUid uidB = ItemUid.generateItemUid(); + Map itemDataMap = new HashMap<>(); + itemDataMap.put( + uidA, + new ItemData( + /* durationUs= */ 100_000_000L, + /* defaultPositionUs= */ 5_000_000L, + /* isLive= */ false, + itemA, + "A")); + itemDataMap.put( + uidB, + new ItemData( + /* durationUs= */ 200_000_000L, + /* defaultPositionUs= */ 10_000_000L, + /* isLive= */ true, + itemB, + "B")); + CastTimeline initialTimeline = new CastTimeline(ImmutableList.of(uidA, uidB), itemDataMap); + QueueSnapshot initialSnapshot = + new QueueSnapshot(initialTimeline, /* currentWindowIndex= */ 0, C.TIME_UNSET); + MoveMediaItemsOperation operation = + new MoveMediaItemsOperation(ImmutableList.of(uidA), /* insertBeforePeriodUid= */ null); + + QueueSnapshot updatedSnapshot = + operation.createMaskedSnapshot(initialSnapshot, timelineTracker); + + assertThat(updatedSnapshot.timeline.getWindowCount()).isEqualTo(2); + Window window0 = updatedSnapshot.timeline.getWindow(0, new Window()); + Window window1 = updatedSnapshot.timeline.getWindow(1, new Window()); + assertThat(window0.uid).isEqualTo(uidB); + assertThat(window0.durationUs).isEqualTo(200_000_000L); + assertThat(window0.defaultPositionUs).isEqualTo(10_000_000L); + assertThat(window0.isLive()).isTrue(); + + assertThat(window1.uid).isEqualTo(uidA); + assertThat(window1.durationUs).isEqualTo(100_000_000L); + assertThat(window1.defaultPositionUs).isEqualTo(5_000_000L); + assertThat(window1.isLive()).isFalse(); + } + + @Test + public void seek_createMaskedSnapshot_updatesWindowIndexAndMaskedPosition() { + QueueSnapshot initialSnapshot = + createSampleSnapshot(/* currentWindowIndex= */ 0, "A", "B", "C"); + Object uidC = initialSnapshot.timeline.getWindow(2, new Window()).uid; + SeekOperation seekOperation = + new SeekOperation( + /* positionMs= */ 45_000L, /* targetPeriodUid= */ uidC, /* isWindowChange= */ true); + + QueueSnapshot updatedSnapshot = + seekOperation.createMaskedSnapshot(initialSnapshot, timelineTracker); + + assertThat(updatedSnapshot.timeline).isSameInstanceAs(initialSnapshot.timeline); + assertThat(updatedSnapshot.currentWindowIndex).isEqualTo(2); + assertThat(updatedSnapshot.maskedPositionMs).isEqualTo(45_000L); + } + + @Test + public void addMediaItems_createMaskedSnapshot_calledTwice_isIdempotent() { + QueueSnapshot initialSnapshot = createSampleSnapshot(/* currentWindowIndex= */ 0, "A", "B"); + Object uidB = initialSnapshot.timeline.getWindow(1, new Window()).uid; + MediaItem itemC = createMediaItem("C"); + AddMediaItemsOperation operation = + new AddMediaItemsOperation(/* insertBeforePeriodUid= */ uidB, ImmutableList.of(itemC)); + + QueueSnapshot firstMaskedSnapshot = + operation.createMaskedSnapshot(initialSnapshot, timelineTracker); + QueueSnapshot secondMaskedSnapshot = + operation.createMaskedSnapshot(firstMaskedSnapshot, timelineTracker); + + assertThat(secondMaskedSnapshot).isSameInstanceAs(firstMaskedSnapshot); + assertThat(secondMaskedSnapshot.timeline.getWindowCount()).isEqualTo(3); + } + + @Test + public void addMediaItems_createMaskedSnapshot_whenSomeItemsAlreadyPresent_insertsOnlyMissing() { + QueueSnapshot initialSnapshot = createSampleSnapshot(/* currentWindowIndex= */ 0, "A", "B"); + MediaItem itemC = createMediaItem("C"); + MediaItem itemD = createMediaItem("D"); + AddMediaItemsOperation operation = + new AddMediaItemsOperation( + /* insertBeforePeriodUid= */ null, ImmutableList.of(itemC, itemD)); + + // First call registers synthetic UIDs for C (index 2) and D (index 3). + QueueSnapshot firstMasked = operation.createMaskedSnapshot(initialSnapshot, timelineTracker); + Object uidC = firstMasked.timeline.getWindow(2, new Window()).uid; + + // Simulate a partial receiver snapshot where C is already present (for example, [A, B, C]) + // but D is not. + QueueSnapshot partialSnapshot = + new RemoveMediaItemsOperation( + ImmutableList.of(firstMasked.timeline.getWindow(3, new Window()).uid)) + .createMaskedSnapshot(firstMasked, timelineTracker); + + QueueSnapshot remasked = operation.createMaskedSnapshot(partialSnapshot, timelineTracker); + + assertThat(remasked.timeline.getWindowCount()).isEqualTo(4); + assertThat(remasked.timeline.getWindow(2, new Window()).uid).isEqualTo(uidC); + assertThat(getMediaIdAt(remasked.timeline, 2)).isEqualTo("C"); + assertThat(getMediaIdAt(remasked.timeline, 3)).isEqualTo("D"); + } + + @Test + public void removeMediaItems_createMaskedSnapshot_calledTwice_isIdempotent() { + QueueSnapshot initialSnapshot = + createSampleSnapshot(/* currentWindowIndex= */ 2, "A", "B", "C"); + Object uidB = initialSnapshot.timeline.getWindow(1, new Window()).uid; + RemoveMediaItemsOperation operation = new RemoveMediaItemsOperation(ImmutableList.of(uidB)); + + QueueSnapshot firstMaskedSnapshot = + operation.createMaskedSnapshot(initialSnapshot, timelineTracker); + QueueSnapshot secondMaskedSnapshot = + operation.createMaskedSnapshot(firstMaskedSnapshot, timelineTracker); + + assertThat(secondMaskedSnapshot).isSameInstanceAs(firstMaskedSnapshot); + assertThat(secondMaskedSnapshot.timeline.getWindowCount()).isEqualTo(2); + assertThat(secondMaskedSnapshot.currentWindowIndex).isEqualTo(1); + } + + @Test + public void moveMediaItems_createMaskedSnapshot_calledTwice_isIdempotent() { + QueueSnapshot initialSnapshot = + createSampleSnapshot(/* currentWindowIndex= */ 0, "A", "B", "C", "D"); + Object uidA = initialSnapshot.timeline.getWindow(0, new Window()).uid; + Object uidD = initialSnapshot.timeline.getWindow(3, new Window()).uid; + MoveMediaItemsOperation operation = + new MoveMediaItemsOperation(ImmutableList.of(uidA), /* insertBeforePeriodUid= */ uidD); + + QueueSnapshot firstMaskedSnapshot = + operation.createMaskedSnapshot(initialSnapshot, timelineTracker); + QueueSnapshot secondMaskedSnapshot = + operation.createMaskedSnapshot(firstMaskedSnapshot, timelineTracker); + + assertThat(secondMaskedSnapshot).isSameInstanceAs(firstMaskedSnapshot); + assertThat(secondMaskedSnapshot.timeline.getWindowCount()).isEqualTo(4); + assertThat(getMediaIdAt(secondMaskedSnapshot.timeline, 0)).isEqualTo("B"); + assertThat(getMediaIdAt(secondMaskedSnapshot.timeline, 1)).isEqualTo("C"); + assertThat(getMediaIdAt(secondMaskedSnapshot.timeline, 2)).isEqualTo("A"); + assertThat(getMediaIdAt(secondMaskedSnapshot.timeline, 3)).isEqualTo("D"); + assertThat(secondMaskedSnapshot.currentWindowIndex).isEqualTo(2); + } + + private QueueSnapshot createSampleSnapshot(int currentWindowIndex, String... mediaIds) { + ImmutableList.Builder items = ImmutableList.builder(); + for (String id : mediaIds) { + items.add(createMediaItem(id)); + } + SetMediaItemsOperation setOp = + new SetMediaItemsOperation( + items.build(), currentWindowIndex, /* startPositionMs= */ C.TIME_UNSET); + return setOp.createMaskedSnapshot(QueueSnapshot.EMPTY, timelineTracker); + } + + private static MediaItem createMediaItem(String id) { + return new MediaItem.Builder() + .setMediaId(id) + .setUri("http://example.com/" + id) + .setMimeType(MimeTypes.APPLICATION_MP4) + .build(); + } + + private static String getMediaIdAt(Timeline timeline, int windowIndex) { + return timeline.getWindow(windowIndex, new Window()).mediaItem.mediaId; + } +} diff --git a/libraries/cast/src/test/java/androidx/media3/cast/RemoteCastPlayerTest.java b/libraries/cast/src/test/java/androidx/media3/cast/RemoteCastPlayerTest.java index 661c1ba8cdc..2df139ba87c 100644 --- a/libraries/cast/src/test/java/androidx/media3/cast/RemoteCastPlayerTest.java +++ b/libraries/cast/src/test/java/androidx/media3/cast/RemoteCastPlayerTest.java @@ -58,6 +58,7 @@ import static com.google.common.truth.Truth.assertThat; import static org.junit.Assert.assertThrows; import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyBoolean; import static org.mockito.ArgumentMatchers.anyDouble; import static org.mockito.ArgumentMatchers.anyInt; import static org.mockito.ArgumentMatchers.anyLong; @@ -92,6 +93,7 @@ import androidx.media3.common.TrackSelectionParameters; import androidx.media3.common.Tracks; import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.gms.cast.CastStatusCodes; import com.google.android.gms.cast.MediaInfo; import com.google.android.gms.cast.MediaLoadRequestData; import com.google.android.gms.cast.MediaQueueData; @@ -104,8 +106,10 @@ import com.google.android.gms.cast.framework.SessionManagerListener; import com.google.android.gms.cast.framework.media.MediaQueue; import com.google.android.gms.cast.framework.media.RemoteMediaClient; +import com.google.android.gms.cast.framework.media.RemoteMediaClient.MediaChannelResult; import com.google.android.gms.common.api.PendingResult; import com.google.android.gms.common.api.ResultCallback; +import com.google.android.gms.common.api.Status; import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableSet; import com.google.common.primitives.Longs; @@ -113,7 +117,10 @@ import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; +import java.util.HashSet; import java.util.List; +import org.json.JSONException; +import org.json.JSONObject; import org.junit.After; import org.junit.Before; import org.junit.Rule; @@ -144,6 +151,7 @@ public class RemoteCastPlayerTest { private SessionManagerListener sessionManagerListener; private RemoteMediaClient.Callback remoteMediaClientCallback; private MediaQueue.Callback mediaQueueCallback; + private RemoteMediaClient.ProgressListener progressListener; @Mock private RemoteMediaClient mockRemoteMediaClient; @Mock private MediaStatus mockMediaStatus; @@ -153,6 +161,8 @@ public class RemoteCastPlayerTest { @Mock private CastSession mockCastSession; @Mock private Listener mockListener; @Mock private PendingResult mockPendingResult; + @Mock private PendingResult defaultPendingResult; + @Mock private SessionAvailabilityListener mockSessionAvailabilityListener; @Captor private ArgumentCaptor trackSelectionArgumentCaptor; @@ -175,8 +185,14 @@ public class RemoteCastPlayerTest { @Captor private ArgumentCaptor loadArgumentCaptor; @Captor private ArgumentCaptor queueItemsArgumentCaptor; @Captor private ArgumentCaptor mediaItemCaptor; + @Captor private ArgumentCaptor progressListenerArgumentCaptor; + + private final List> pendingResultCallbacks = new ArrayList<>(); + private final List sentQueueItems = new ArrayList<>(); + private MediaChannelResult defaultSuccessResult; @Before + @SuppressWarnings("deprecation") // Stubs deprecated RemoteMediaClient.seek(long). public void setUp() { when(mockCastContext.getSessionManager()).thenReturn(mockSessionManager); when(mockSessionManager.getCurrentCastSession()).thenReturn(mockCastSession); @@ -187,6 +203,42 @@ public void setUp() { when(mockMediaQueue.getItemIds()).thenReturn(new int[0]); // Make the remote media client present the same default values as ExoPlayer: when(mockRemoteMediaClient.isPaused()).thenReturn(true); + when(mockRemoteMediaClient.play()).thenReturn(mockPendingResult); + when(mockRemoteMediaClient.pause()).thenReturn(mockPendingResult); + defaultSuccessResult = mock(MediaChannelResult.class); + when(defaultSuccessResult.getStatus()).thenReturn(new Status(CastStatusCodes.SUCCESS)); + doAnswer( + invocation -> { + ResultCallback callback = invocation.getArgument(0); + pendingResultCallbacks.add(callback); + return null; + }) + .when(defaultPendingResult) + .setResultCallback(any()); + when(mockRemoteMediaClient.load(any(MediaLoadRequestData.class))) + .thenAnswer( + invocation -> { + MediaLoadRequestData requestData = invocation.getArgument(0); + MediaQueueData queueData = requestData.getQueueData(); + if (queueData != null && queueData.getItems() != null) { + sentQueueItems.addAll(queueData.getItems()); + } + return defaultPendingResult; + }); + when(mockRemoteMediaClient.queueInsertItems(any(MediaQueueItem[].class), anyInt(), any())) + .thenAnswer( + invocation -> { + MediaQueueItem[] items = invocation.getArgument(0); + sentQueueItems.addAll(Arrays.asList(items)); + return defaultPendingResult; + }); + when(mockRemoteMediaClient.queueReorderItems(any(int[].class), anyInt(), any())) + .thenReturn(defaultPendingResult); + when(mockRemoteMediaClient.queueRemoveItems(any(int[].class), any())) + .thenReturn(defaultPendingResult); + when(mockRemoteMediaClient.queueJumpToItem(anyInt(), anyLong(), any())) + .thenReturn(defaultPendingResult); + when(mockRemoteMediaClient.seek(anyLong())).thenReturn(defaultPendingResult); when(mockMediaStatus.getQueueRepeatMode()).thenReturn(MediaStatus.REPEAT_MODE_REPEAT_OFF); when(mockMediaStatus.getStreamVolume()).thenReturn(1.0); when(mockMediaStatus.getPlaybackRate()).thenReturn(1.0d); @@ -220,6 +272,9 @@ public void setUp() { remoteMediaClientCallback = remoteMediaClientCallbackArgumentCaptor.getValue(); verify(mockMediaQueue).registerCallback(mediaQueueCallbackArgumentCaptor.capture()); mediaQueueCallback = mediaQueueCallbackArgumentCaptor.getValue(); + verify(mockRemoteMediaClient) + .addProgressListener(progressListenerArgumentCaptor.capture(), anyLong()); + progressListener = progressListenerArgumentCaptor.getValue(); } @After @@ -412,7 +467,70 @@ public void mediaQueueChanged_updatesCurrentTimeline() { .onTimelineChanged(eq(newTimeline), eq(Player.TIMELINE_CHANGE_REASON_SOURCE_UPDATE)); assertThat(newTimeline.getPeriodCount()).isEqualTo(1); Timeline.Period period = new Timeline.Period(); - assertThat(newTimeline.getPeriod(/* periodIndex= */ 0, period).id).isEqualTo(firstItemId); + assertThat(newTimeline.getPeriod(/* periodIndex= */ 0, period).id).isNotNull(); + } + + @Test + public void mediaQueueChanged_whileSeekPendingToRemovedItem_updatesCurrentTimeline() { + when(mockRemoteMediaClient.queueJumpToItem(anyInt(), anyLong(), eq(null))) + .thenReturn(mockPendingResult); + MediaItem mediaItem1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem mediaItem2 = createMediaItem(/* mediaQueueItemId= */ 2); + ImmutableList mediaItems = ImmutableList.of(mediaItem1, mediaItem2); + int[] mediaQueueItemIds = new int[] {1, 2}; + addMediaItemsAndUpdateTimeline(mediaItems, mediaQueueItemIds); + + remoteCastPlayer.seekTo(/* mediaItemIndex= */ 1, /* positionMs= */ 1234); + // Receiver queue shrinks to 1 item. + when(mockMediaQueue.getItemIds()).thenReturn(new int[] {1}); + mediaQueueCallback.mediaQueueChanged(); + + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(1); + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(0); + } + + @Test + public void + mediaQueueChanged_mediaItemRemovedExternally_updatesTimelineAndCurrentMediaItemIndex() { + MediaItem mediaItem1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem mediaItem2 = createMediaItem(/* mediaQueueItemId= */ 2); + ImmutableList mediaItems = ImmutableList.of(mediaItem1, mediaItem2); + int[] mediaQueueItemIds = new int[] {1, 2}; + // Initialize 2-item queue and set currently playing item to item 2 (index 1) + remoteCastPlayer.addMediaItems(mediaItems); + updateTimeLine(mediaItems, mediaQueueItemIds, /* currentItemId= */ 2); + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(1); + + // Item 1 is removed externally by another source. Queue shrinks to [item 2]. + when(mockMediaQueue.getItemIds()).thenReturn(new int[] {2}); + mediaQueueCallback.mediaQueueChanged(); + + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(1); + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(0); + assertThat(remoteCastPlayer.isCurrentMediaItemSeekable()).isTrue(); + } + + @Test + public void mediaQueueChanged_whileSeekPendingToShiftedItem_updatesMaskedIndex() { + when(mockRemoteMediaClient.queueJumpToItem(anyInt(), anyLong(), eq(null))) + .thenReturn(mockPendingResult); + MediaItem mediaItem1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem mediaItem2 = createMediaItem(/* mediaQueueItemId= */ 2); + MediaItem mediaItem3 = createMediaItem(/* mediaQueueItemId= */ 3); + ImmutableList mediaItems = ImmutableList.of(mediaItem1, mediaItem2, mediaItem3); + int[] mediaQueueItemIds = new int[] {1, 2, 3}; + addMediaItemsAndUpdateTimeline(mediaItems, mediaQueueItemIds); + + // Seek to index 2 (item 3) + remoteCastPlayer.seekTo(/* mediaItemIndex= */ 2, /* positionMs= */ 1234); + // Item 1 is removed, so queue on receiver becomes [item 2, item 3] + when(mockMediaQueue.getItemIds()).thenReturn(new int[] {2, 3}); + mediaQueueCallback.mediaQueueChanged(); + + // Timeline length is 2, and target item 3 is now at index 1: + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(2); + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(1); + assertThat(remoteCastPlayer.isCurrentMediaItemSeekable()).isTrue(); } @Test @@ -584,6 +702,99 @@ public void volume_changesOnStatusUpdates() { assertThat(remoteCastPlayer.getVolume()).isEqualTo(0.75f); } + @Test + public void mute_withNonZeroVolume_setsVolumeToZero() { + when(mockRemoteMediaClient.setStreamMute(anyBoolean())).thenReturn(mockPendingResult); + assertThat(remoteCastPlayer.getVolume()).isEqualTo(1f); + + remoteCastPlayer.mute(); + + assertThat(remoteCastPlayer.getVolume()).isEqualTo(0f); + verify(mockListener).onVolumeChanged(0f); + verify(mockRemoteMediaClient).setStreamMute(true); + } + + @Test + public void mute_withZeroVolume_isNoOp() { + when(mockRemoteMediaClient.setStreamVolume(anyDouble())).thenReturn(mockPendingResult); + remoteCastPlayer.setVolume(0f); + reset(mockRemoteMediaClient, mockListener); + + remoteCastPlayer.mute(); + + assertThat(remoteCastPlayer.getVolume()).isEqualTo(0f); + verifyNoMoreInteractions(mockRemoteMediaClient, mockListener); + } + + @Test + public void unmute_withMutedVolume_restoresPreviousVolume() { + when(mockRemoteMediaClient.setStreamVolume(anyDouble())).thenReturn(mockPendingResult); + when(mockRemoteMediaClient.setStreamMute(anyBoolean())).thenReturn(mockPendingResult); + remoteCastPlayer.setVolume(0.5f); + remoteCastPlayer.mute(); + assertThat(remoteCastPlayer.getVolume()).isEqualTo(0f); + clearInvocations(mockRemoteMediaClient); + + remoteCastPlayer.unmute(); + + assertThat(remoteCastPlayer.getVolume()).isEqualTo(0.5f); + verify(mockRemoteMediaClient).setStreamMute(false); + verify(mockRemoteMediaClient).setStreamVolume(0.5); + } + + @Test + public void unmute_afterSetVolumeToZero_restoresPreviousVolume() { + when(mockRemoteMediaClient.setStreamVolume(anyDouble())).thenReturn(mockPendingResult); + when(mockRemoteMediaClient.setStreamMute(anyBoolean())).thenReturn(mockPendingResult); + remoteCastPlayer.setVolume(0.5f); + remoteCastPlayer.setVolume(0f); + when(mockMediaStatus.getStreamVolume()).thenReturn(0.0); + clearInvocations(mockRemoteMediaClient); + + remoteCastPlayer.unmute(); + + assertThat(remoteCastPlayer.getVolume()).isEqualTo(0.5f); + verify(mockRemoteMediaClient).setStreamMute(false); + verify(mockRemoteMediaClient).setStreamVolume(0.5); + } + + @Test + public void stressTest_rapidVolumeChangesAndMuteUnmute_stateStaysConsistent() { + when(mockRemoteMediaClient.setStreamVolume(anyDouble())).thenReturn(mockPendingResult); + when(mockRemoteMediaClient.setStreamMute(anyBoolean())).thenReturn(mockPendingResult); + + float[] targetVolumes = new float[] {0.8f, 0.3f, 0.0f, 0.6f, 0.0f, 1.0f, 0.2f}; + for (int i = 0; i < 50; i++) { + float targetVol = targetVolumes[i % targetVolumes.length]; + remoteCastPlayer.setVolume(targetVol); + + if (targetVol == 0f) { + assertThat(remoteCastPlayer.getVolume()).isEqualTo(0f); + remoteCastPlayer.unmute(); + assertThat(remoteCastPlayer.getVolume()).isGreaterThan(0f); + } else if (i % 3 == 0) { + remoteCastPlayer.mute(); + assertThat(remoteCastPlayer.getVolume()).isEqualTo(0f); + remoteCastPlayer.unmute(); + assertThat(remoteCastPlayer.getVolume()).isEqualTo(targetVol); + } else { + assertThat(remoteCastPlayer.getVolume()).isEqualTo(targetVol); + } + } + } + + @Test + public void unmute_withNonZeroVolume_isNoOp() { + when(mockRemoteMediaClient.setStreamVolume(anyDouble())).thenReturn(mockPendingResult); + remoteCastPlayer.setVolume(0.5f); + reset(mockRemoteMediaClient, mockListener); + + remoteCastPlayer.unmute(); + + assertThat(remoteCastPlayer.getVolume()).isEqualTo(0.5f); + verifyNoMoreInteractions(mockRemoteMediaClient, mockListener); + } + @Test public void onStatusUpdated_withMediaTracks_updatesPlayerTracks() { MediaTrack audioTrack = new MediaTrack.Builder(1, MediaTrack.TYPE_AUDIO).build(); @@ -675,6 +886,26 @@ public void setMediaItems_doNotReset_callsRemoteMediaClient() { assertThat(mediaQueueItems.get(1).getMedia().getContentId()).isEqualTo(uri2); } + @Test + public void setMediaItems_attachesSyntheticIdsToDispatchedQueueItems() throws Exception { + ImmutableList mediaItems = + ImmutableList.of( + new MediaItem.Builder().setUri("http://www.google.com/video1").build(), + new MediaItem.Builder().setUri("http://www.google.com/video2").build()); + + remoteCastPlayer.setMediaItems(mediaItems); + + verify(mockRemoteMediaClient).load(loadArgumentCaptor.capture()); + MediaLoadRequestData mediaLoadRequestData = loadArgumentCaptor.getValue(); + List queueItems = mediaLoadRequestData.getQueueData().getItems(); + assertThat(queueItems).hasSize(2); + for (MediaQueueItem queueItem : queueItems) { + JSONObject customData = queueItem.getMedia().getCustomData(); + assertThat(customData).isNotNull(); + assertThat(customData.getString("m3-syntheticId")).isNotEmpty(); + } + } + @Test public void setMediaItems_replaceExistingPlaylist_notifiesMediaItemTransition() { List firstPlaylist = new ArrayList<>(); @@ -742,6 +973,7 @@ public void setMediaItems_replaceExistingPlaylist_notifiesPositionDiscontinuity( /* positionMs= */ 2000L, /* notifyStatusUpdate= */ true); // Replacing existing playlist. + Object oldUid = remoteCastPlayer.getCurrentTimeline().getWindow(1, new Timeline.Window()).uid; remoteCastPlayer.setMediaItems( secondPlaylist, /* startIndex= */ 0, /* startPositionMs= */ 1000L); updateTimeLine( @@ -752,13 +984,14 @@ public void setMediaItems_replaceExistingPlaylist_notifiesPositionDiscontinuity( /* durationsMs= */ new long[] {20_000}, /* positionMs= */ 1000L, /* notifyStatusUpdate= */ true); + Object newUid = remoteCastPlayer.getCurrentTimeline().getWindow(0, new Timeline.Window()).uid; Player.PositionInfo oldPosition = new Player.PositionInfo( - /* windowUid= */ 2, + /* windowUid= */ oldUid, /* mediaItemIndex= */ 1, firstPlaylist.get(1), - /* periodUid= */ 2, + /* periodUid= */ oldUid, /* periodIndex= */ 1, /* positionMs= */ 2000, /* contentPositionMs= */ 2000, @@ -766,10 +999,10 @@ public void setMediaItems_replaceExistingPlaylist_notifiesPositionDiscontinuity( /* adIndexInAdGroup= */ C.INDEX_UNSET); Player.PositionInfo newPosition = new Player.PositionInfo( - /* windowUid= */ 3, + /* windowUid= */ newUid, /* mediaItemIndex= */ 0, secondPlaylist.get(0), - /* periodUid= */ 3, + /* periodUid= */ newUid, /* periodIndex= */ 0, /* positionMs= */ 1000, /* contentPositionMs= */ 1000, @@ -784,8 +1017,34 @@ public void setMediaItems_replaceExistingPlaylist_notifiesPositionDiscontinuity( inOrder.verify(mockListener, never()).onPositionDiscontinuity(any(), any(), anyInt()); } + @Test + public void addMediaItems_whenTimelineIsEmpty_callsLoadOnRemoteMediaClient() { + when(mockRemoteMediaClient.getMediaStatus()).thenReturn(null); + MediaItem.Builder builder = new MediaItem.Builder(); + List mediaItems = new ArrayList<>(); + String uri1 = "http://www.google.com/video1"; + String uri2 = "http://www.google.com/video2"; + mediaItems.add(builder.setUri(uri1).setMimeType(MimeTypes.APPLICATION_MPD).build()); + mediaItems.add(builder.setUri(uri2).setMimeType(MimeTypes.APPLICATION_MP4).build()); + + remoteCastPlayer.addMediaItems(mediaItems); + + verify(mockRemoteMediaClient).load(loadArgumentCaptor.capture()); + verify(mockRemoteMediaClient, never()).queueInsertItems(any(), anyInt(), any()); + MediaLoadRequestData mediaLoadRequestData = loadArgumentCaptor.getValue(); + MediaQueueData queueData = mediaLoadRequestData.getQueueData(); + assertThat(queueData.getStartIndex()).isEqualTo(0); + assertThat(queueData.getStartTime()).isEqualTo(0L); + List mediaQueueItems = queueData.getItems(); + assertThat(mediaQueueItems).hasSize(2); + assertThat(mediaQueueItems.get(0).getMedia().getContentId()).isEqualTo(uri1); + assertThat(mediaQueueItems.get(1).getMedia().getContentId()).isEqualTo(uri2); + } + @Test public void addMediaItems_callsRemoteMediaClient() { + int[] initialIds = createMediaQueueItemIds(/* numberOfIds= */ 1); + addMediaItemsAndUpdateTimeline(createMediaItems(initialIds), initialIds); MediaItem.Builder builder = new MediaItem.Builder(); List mediaItems = new ArrayList<>(); String uri1 = "http://www.google.com/video1"; @@ -819,15 +1078,15 @@ public void addMediaItems_insertAtIndex_callsRemoteMediaClient() { remoteCastPlayer.addMediaItems(index, newPlaylist); updateTimeLine(newPlaylist, /* mediaQueueItemIds= */ new int[] {123}, /* currentItemId= */ 1); - verify(mockRemoteMediaClient, times(2)) + verify(mockRemoteMediaClient) .queueInsertItems(queueItemsArgumentCaptor.capture(), anyInt(), any()); - assertThat(queueItemsArgumentCaptor.getAllValues().get(1)[0]) - .isEqualTo(mediaItemConverter.toMediaQueueItem(anotherMediaItem)); + MediaQueueItem insertedItem = queueItemsArgumentCaptor.getValue()[0]; + assertThat(mediaItemConverter.toMediaItem(insertedItem)).isEqualTo(anotherMediaItem); Timeline.Window currentWindow = remoteCastPlayer .getCurrentTimeline() .getWindow(remoteCastPlayer.getCurrentMediaItemIndex(), new Timeline.Window()); - assertThat(currentWindow.uid).isEqualTo(123); + assertThat(currentWindow.uid).isNotNull(); assertThat(currentWindow.mediaItem).isEqualTo(anotherMediaItem); } @@ -970,21 +1229,25 @@ public void replaceMediaItems_callsRemoteMediaClient() { // Replace item at position 1. remoteCastPlayer.replaceMediaItems( /* fromIndex= */ 1, /* toIndex= */ 2, ImmutableList.of(anotherMediaItem)); + updateTimeLine( + ImmutableList.of(mediaItems.get(0), mediaItems.get(1), anotherMediaItem), + /* mediaQueueItemIds= */ new int[] {mediaQueueItemIds[0], mediaQueueItemIds[1], 123}, + /* currentItemId= */ 123); updateTimeLine( newPlaylist, /* mediaQueueItemIds= */ new int[] {mediaQueueItemIds[0], 123}, /* currentItemId= */ 123); - verify(mockRemoteMediaClient, times(2)) + verify(mockRemoteMediaClient) .queueInsertItems(queueItemsArgumentCaptor.capture(), anyInt(), any()); verify(mockRemoteMediaClient).queueRemoveItems(new int[] {2}, /* customData= */ null); - assertThat(queueItemsArgumentCaptor.getAllValues().get(1)[0]) - .isEqualTo(mediaItemConverter.toMediaQueueItem(anotherMediaItem)); + MediaQueueItem insertedItem = queueItemsArgumentCaptor.getValue()[0]; + assertThat(mediaItemConverter.toMediaItem(insertedItem)).isEqualTo(anotherMediaItem); Timeline.Window currentWindow = remoteCastPlayer .getCurrentTimeline() .getWindow(remoteCastPlayer.getCurrentMediaItemIndex(), new Timeline.Window()); - assertThat(currentWindow.uid).isEqualTo(123); + assertThat(currentWindow.uid).isNotNull(); assertThat(currentWindow.mediaItem).isEqualTo(anotherMediaItem); } @@ -1061,6 +1324,7 @@ public void clearMediaItems_notifiesPositionDiscontinuity() { /* durationsMs= */ new long[] {20_000L, 30_000L}, /* positionMs= */ 1234, /* notifyStatusUpdate= */ true); + Object oldUid = remoteCastPlayer.getCurrentTimeline().getWindow(0, new Timeline.Window()).uid; remoteCastPlayer.clearMediaItems(); updateTimeLine( /* mediaItems= */ ImmutableList.of(), @@ -1073,10 +1337,10 @@ public void clearMediaItems_notifiesPositionDiscontinuity() { Player.PositionInfo oldPosition = new Player.PositionInfo( - /* windowUid= */ 1, + /* windowUid= */ oldUid, /* mediaItemIndex= */ 0, mediaItems.get(0), - /* periodUid= */ 1, + /* periodUid= */ oldUid, /* periodIndex= */ 0, /* positionMs= */ 1234, /* contentPositionMs= */ 1234, @@ -1146,6 +1410,8 @@ public void removeCurrentMediaItem_notifiesPositionDiscontinuity() { /* durationsMs= */ new long[] {20_000L, 30_000L}, /* positionMs= */ 1234, /* notifyStatusUpdate= */ true); + Object oldUid = remoteCastPlayer.getCurrentTimeline().getWindow(0, new Timeline.Window()).uid; + Object newUid = remoteCastPlayer.getCurrentTimeline().getWindow(1, new Timeline.Window()).uid; remoteCastPlayer.removeMediaItem(/* index= */ 0); // Update with the new timeline after removal. updateTimeLine( @@ -1159,10 +1425,10 @@ public void removeCurrentMediaItem_notifiesPositionDiscontinuity() { Player.PositionInfo oldPosition = new Player.PositionInfo( - /* windowUid= */ 1, + /* windowUid= */ oldUid, /* mediaItemIndex= */ 0, mediaItem1, - /* periodUid= */ 1, + /* periodUid= */ oldUid, /* periodIndex= */ 0, /* positionMs= */ 1234, /* contentPositionMs= */ 1234, @@ -1170,10 +1436,10 @@ public void removeCurrentMediaItem_notifiesPositionDiscontinuity() { /* adIndexInAdGroup= */ C.INDEX_UNSET); Player.PositionInfo newPosition = new Player.PositionInfo( - /* windowUid= */ 2, + /* windowUid= */ newUid, /* mediaItemIndex= */ 0, mediaItem2, - /* periodUid= */ 2, + /* periodUid= */ newUid, /* periodIndex= */ 0, /* positionMs= */ 0, /* contentPositionMs= */ 0, @@ -1230,6 +1496,8 @@ public void removeCurrentMediaItem_byRemoteClient_notifiesPositionDiscontinuity( /* durationsMs= */ new long[] {20_000L, 30_000L}, /* positionMs= */ 1234, /* notifyStatusUpdate= */ true); + Object oldUid = remoteCastPlayer.getCurrentTimeline().getWindow(0, new Timeline.Window()).uid; + Object newUid = remoteCastPlayer.getCurrentTimeline().getWindow(1, new Timeline.Window()).uid; // Update with the new timeline after removal on the device. updateTimeLine( ImmutableList.of(mediaItem2), @@ -1242,10 +1510,10 @@ public void removeCurrentMediaItem_byRemoteClient_notifiesPositionDiscontinuity( Player.PositionInfo oldPosition = new Player.PositionInfo( - /* windowUid= */ 1, + /* windowUid= */ oldUid, /* mediaItemIndex= */ 0, mediaItem1, - /* periodUid= */ 1, + /* periodUid= */ oldUid, /* periodIndex= */ 0, /* positionMs= */ 0, // position at which we receive the timeline change /* contentPositionMs= */ 0, // position at which we receive the timeline change @@ -1253,10 +1521,10 @@ public void removeCurrentMediaItem_byRemoteClient_notifiesPositionDiscontinuity( /* adIndexInAdGroup= */ C.INDEX_UNSET); Player.PositionInfo newPosition = new Player.PositionInfo( - /* windowUid= */ 2, + /* windowUid= */ newUid, /* mediaItemIndex= */ 0, mediaItem2, - /* periodUid= */ 2, + /* periodUid= */ newUid, /* periodIndex= */ 0, /* positionMs= */ 0, /* contentPositionMs= */ 0, @@ -1355,14 +1623,17 @@ public void seekTo_otherWindow_notifiesPositionDiscontinuity() { remoteCastPlayer.addMediaItems(mediaItems); updateTimeLine(mediaItems, mediaQueueItemIds, /* currentItemId= */ 1); + Timeline timeline = remoteCastPlayer.getCurrentTimeline(); + Object oldUid = timeline.getWindow(0, new Timeline.Window()).uid; + Object newUid = timeline.getWindow(1, new Timeline.Window()).uid; remoteCastPlayer.seekTo(/* mediaItemIndex= */ 1, /* positionMs= */ 1234); Player.PositionInfo oldPosition = new Player.PositionInfo( - /* windowUid= */ 1, + /* windowUid= */ oldUid, /* mediaItemIndex= */ 0, mediaItem1, - /* periodUid= */ 1, + /* periodUid= */ oldUid, /* periodIndex= */ 0, /* positionMs= */ 0, /* contentPositionMs= */ 0, @@ -1370,10 +1641,10 @@ public void seekTo_otherWindow_notifiesPositionDiscontinuity() { /* adIndexInAdGroup= */ C.INDEX_UNSET); Player.PositionInfo newPosition = new Player.PositionInfo( - /* windowUid= */ 2, + /* windowUid= */ newUid, /* mediaItemIndex= */ 1, mediaItem2, - /* periodUid= */ 2, + /* periodUid= */ newUid, /* periodIndex= */ 1, /* positionMs= */ 1234, /* contentPositionMs= */ 1234, @@ -1417,14 +1688,16 @@ public void seekTo_sameWindow_notifiesPositionDiscontinuity() { remoteCastPlayer.addMediaItems(mediaItems); updateTimeLine(mediaItems, mediaQueueItemIds, /* currentItemId= */ 1); + Timeline timeline = remoteCastPlayer.getCurrentTimeline(); + Object uid = timeline.getWindow(0, new Timeline.Window()).uid; remoteCastPlayer.seekTo(/* mediaItemIndex= */ 0, /* positionMs= */ 1234); Player.PositionInfo oldPosition = new Player.PositionInfo( - /* windowUid= */ 1, + /* windowUid= */ uid, /* mediaItemIndex= */ 0, mediaItems.get(0), - /* periodUid= */ 1, + /* periodUid= */ uid, /* periodIndex= */ 0, /* positionMs= */ 0, /* contentPositionMs= */ 0, @@ -1432,10 +1705,10 @@ public void seekTo_sameWindow_notifiesPositionDiscontinuity() { /* adIndexInAdGroup= */ C.INDEX_UNSET); Player.PositionInfo newPosition = new Player.PositionInfo( - /* windowUid= */ 1, + /* windowUid= */ uid, /* mediaItemIndex= */ 0, mediaItems.get(0), - /* periodUid= */ 1, + /* periodUid= */ uid, /* periodIndex= */ 0, /* positionMs= */ 1234, /* contentPositionMs= */ 1234, @@ -1493,6 +1766,9 @@ public void autoTransition_notifiesPositionDiscontinuity() { /* durationsMs= */ durationsFirstMs, /* positionMs= */ C.TIME_UNSET, /* notifyStatusUpdate= */ true); + Timeline timeline = remoteCastPlayer.getCurrentTimeline(); + Object oldUid = timeline.getWindow(0, new Timeline.Window()).uid; + Object newUid = timeline.getWindow(1, new Timeline.Window()).uid; updateTimeLine( mediaItems, mediaQueueItemIds, @@ -1504,10 +1780,10 @@ public void autoTransition_notifiesPositionDiscontinuity() { Player.PositionInfo oldPosition = new Player.PositionInfo( - /* windowUid= */ 1, + /* windowUid= */ oldUid, /* mediaItemIndex= */ 0, mediaItems.get(0), - /* periodUid= */ 1, + /* periodUid= */ oldUid, /* periodIndex= */ 0, /* positionMs= */ 12500, /* contentPositionMs= */ 12500, @@ -1515,10 +1791,10 @@ public void autoTransition_notifiesPositionDiscontinuity() { /* adIndexInAdGroup= */ C.INDEX_UNSET); Player.PositionInfo newPosition = new Player.PositionInfo( - /* windowUid= */ 2, + /* windowUid= */ newUid, /* mediaItemIndex= */ 1, mediaItems.get(1), - /* periodUid= */ 2, + /* periodUid= */ newUid, /* periodIndex= */ 1, /* positionMs= */ 0, /* contentPositionMs= */ 0, @@ -1556,14 +1832,16 @@ public void seekBack_notifiesPositionDiscontinuity() { durationsMs, positionMs, /* notifyStatusUpdate= */ true); + Timeline timeline = remoteCastPlayer.getCurrentTimeline(); + Object uid = timeline.getWindow(0, new Timeline.Window()).uid; remoteCastPlayer.seekBack(); Player.PositionInfo oldPosition = new Player.PositionInfo( - /* windowUid= */ 1, + /* windowUid= */ uid, /* mediaItemIndex= */ 0, mediaItems.get(0), - /* periodUid= */ 1, + /* periodUid= */ uid, /* periodIndex= */ 0, /* positionMs= */ 2 * C.DEFAULT_SEEK_BACK_INCREMENT_MS, /* contentPositionMs= */ 2 * C.DEFAULT_SEEK_BACK_INCREMENT_MS, @@ -1571,10 +1849,10 @@ public void seekBack_notifiesPositionDiscontinuity() { /* adIndexInAdGroup= */ C.INDEX_UNSET); Player.PositionInfo newPosition = new Player.PositionInfo( - /* windowUid= */ 1, + /* windowUid= */ uid, /* mediaItemIndex= */ 0, mediaItems.get(0), - /* periodUid= */ 1, + /* periodUid= */ uid, /* periodIndex= */ 0, /* positionMs= */ C.DEFAULT_SEEK_BACK_INCREMENT_MS, /* contentPositionMs= */ C.DEFAULT_SEEK_BACK_INCREMENT_MS, @@ -1610,14 +1888,16 @@ public void seekForward_notifiesPositionDiscontinuity() { durationsMs, positionMs, /* notifyStatusUpdate= */ true); + Timeline timeline = remoteCastPlayer.getCurrentTimeline(); + Object uid = timeline.getWindow(0, new Timeline.Window()).uid; remoteCastPlayer.seekForward(); Player.PositionInfo oldPosition = new Player.PositionInfo( - /* windowUid= */ 1, + /* windowUid= */ uid, /* mediaItemIndex= */ 0, mediaItems.get(0), - /* periodUid= */ 1, + /* periodUid= */ uid, /* periodIndex= */ 0, /* positionMs= */ 0, /* contentPositionMs= */ 0, @@ -1625,10 +1905,10 @@ public void seekForward_notifiesPositionDiscontinuity() { /* adIndexInAdGroup= */ C.INDEX_UNSET); Player.PositionInfo newPosition = new Player.PositionInfo( - /* windowUid= */ 1, + /* windowUid= */ uid, /* mediaItemIndex= */ 0, mediaItems.get(0), - /* periodUid= */ 1, + /* periodUid= */ uid, /* periodIndex= */ 0, /* positionMs= */ C.DEFAULT_SEEK_FORWARD_INCREMENT_MS, /* contentPositionMs= */ C.DEFAULT_SEEK_FORWARD_INCREMENT_MS, @@ -2757,97 +3037,1205 @@ public void mediaQueueChanged_afterOnSessionEnding_doesNotClearTheTimeline() { assertThat(initialTimeline).isEqualTo(timelineAfterSessionEnd); } - private int[] createMediaQueueItemIds(int numberOfIds) { - int[] mediaQueueItemIds = new int[numberOfIds]; - for (int i = 0; i < numberOfIds; i++) { - mediaQueueItemIds[i] = i + 1; - } - return mediaQueueItemIds; - } + @Test + public void onSessionEnded_withReasonStopped_notifiesUnavailableWithStoppedReason() { + remoteCastPlayer.setSessionAvailabilityListener(mockSessionAvailabilityListener); - private List createMediaItems(int[] mediaQueueItemIds) { - List mediaItems = new ArrayList<>(); - for (int mediaQueueItemId : mediaQueueItemIds) { - mediaItems.add(createMediaItem(mediaQueueItemId)); - } - return mediaItems; + sessionManagerListener.onSessionEnding(mockCastSession); + sessionManagerListener.onSessionEnded( + mockCastSession, RemoteCastPlayer.SESSION_END_REASON_STOPPED); + + verify(mockSessionAvailabilityListener) + .onCastSessionUnavailable(SessionAvailabilityListener.SESSION_UNAVAILABLE_REASON_STOPPED); } - private MediaItem createMediaItem(int mediaQueueItemId) { - return new MediaItem.Builder() - .setUri("http://www.google.com/video" + mediaQueueItemId) - .setMediaMetadata( - new MediaMetadata.Builder().setArtist("Foo Bar - " + mediaQueueItemId).build()) - .setMimeType(MimeTypes.APPLICATION_MPD) - .build(); + @Test + public void onSessionEnded_withReasonRouteChanged_notifiesUnavailableWithRouteChangedReason() { + remoteCastPlayer.setSessionAvailabilityListener(mockSessionAvailabilityListener); + + sessionManagerListener.onSessionEnding(mockCastSession); + sessionManagerListener.onSessionEnded( + mockCastSession, RemoteCastPlayer.SESSION_END_REASON_ROUTE_CHANGE); + + verify(mockSessionAvailabilityListener) + .onCastSessionUnavailable( + SessionAvailabilityListener.SESSION_UNAVAILABLE_REASON_ROUTE_CHANGED); } - private void addMediaItemsAndUpdateTimeline(List mediaItems, int[] mediaQueueItemIds) { - checkState(mediaItems.size() == mediaQueueItemIds.length); - remoteCastPlayer.addMediaItems(mediaItems); - updateTimeLine(mediaItems, mediaQueueItemIds, /* currentItemId= */ 1); + @Test + public void onSessionEnded_withReasonAppStopped_notifiesUnavailableWithAppStoppedReason() { + remoteCastPlayer.setSessionAvailabilityListener(mockSessionAvailabilityListener); + + sessionManagerListener.onSessionEnding(mockCastSession); + sessionManagerListener.onSessionEnded(mockCastSession, RemoteCastPlayer.APPLICATION_STOPPED); + + verify(mockSessionAvailabilityListener) + .onCastSessionUnavailable( + SessionAvailabilityListener.SESSION_UNAVAILABLE_REASON_APP_STOPPED); } - private void updateTimeLine( - List mediaItems, int[] mediaQueueItemIds, int currentItemId) { - int[] streamTypes = new int[mediaItems.size()]; - Arrays.fill(streamTypes, MediaInfo.STREAM_TYPE_BUFFERED); - long[] durationsMs = new long[mediaItems.size()]; - updateTimeLine( - mediaItems, - mediaQueueItemIds, - currentItemId, - streamTypes, - durationsMs, - /* positionMs= */ C.TIME_UNSET, - /* notifyStatusUpdate= */ true); + @Test + public void onSessionSuspended_notifiesUnavailableWithDisconnectedReason() { + remoteCastPlayer.setSessionAvailabilityListener(mockSessionAvailabilityListener); + + sessionManagerListener.onSessionSuspended(mockCastSession, 0); + + verify(mockSessionAvailabilityListener) + .onCastSessionUnavailable( + SessionAvailabilityListener.SESSION_UNAVAILABLE_REASON_DISCONNECTED); } - private void updateTimeLine( - List mediaItems, - int[] mediaQueueItemIds, - int currentItemId, - int[] streamTypes, - long[] durationsMs, - long positionMs, - boolean notifyStatusUpdate) { - // Set up mocks to allow the player to update the timeline. - List queueItems = new ArrayList<>(); - for (int i = 0; i < mediaQueueItemIds.length; i++) { - MediaItem mediaItem = mediaItems.get(i); - int mediaQueueItemId = mediaQueueItemIds[i]; - int streamType = streamTypes[i]; - long durationMs = durationsMs[i]; - String contentId = - mediaItem.mediaId.equals(MediaItem.DEFAULT_MEDIA_ID) - ? mediaItem.localConfiguration.uri.toString() - : mediaItem.mediaId; - MediaInfo.Builder mediaInfoBuilder = - new MediaInfo.Builder(contentId) - .setStreamType(streamType) - .setContentType(mediaItem.localConfiguration.mimeType); - if (durationMs != C.TIME_UNSET) { - mediaInfoBuilder.setStreamDuration(durationMs); - } - MediaInfo mediaInfo = mediaInfoBuilder.build(); - MediaQueueItem mediaQueueItem = mock(MediaQueueItem.class); - when(mediaQueueItem.getItemId()).thenReturn(mediaQueueItemId); - when(mediaQueueItem.getMedia()).thenReturn(mediaInfo); - queueItems.add(mediaQueueItem); - if (mediaQueueItemId == currentItemId) { - when(mockRemoteMediaClient.getCurrentItem()).thenReturn(mediaQueueItem); - when(mockMediaStatus.getMediaInfo()).thenReturn(mediaInfo); + @Test + public void onCastSessionUnavailable_notifiesLegacyListener() { + class LegacyListener implements SessionAvailabilityListener { + boolean onCastSessionUnavailableCalled = false; + + @Override + public void onCastSessionAvailable() {} + + @Override + public void onCastSessionUnavailable() { + onCastSessionUnavailableCalled = true; } } - if (positionMs != C.TIME_UNSET) { - when(mockRemoteMediaClient.getApproximateStreamPosition()).thenReturn(positionMs); - } - when(mockMediaQueue.getItemIds()).thenReturn(mediaQueueItemIds); - when(mockMediaStatus.getQueueItems()).thenReturn(queueItems); - when(mockMediaStatus.getCurrentItemId()) - .thenReturn(currentItemId == C.INDEX_UNSET ? 0 : currentItemId); + LegacyListener legacyListener = new LegacyListener(); + remoteCastPlayer.setSessionAvailabilityListener(legacyListener); - if (notifyStatusUpdate) { + sessionManagerListener.onSessionEnding(mockCastSession); + sessionManagerListener.onSessionEnded( + mockCastSession, RemoteCastPlayer.SESSION_END_REASON_STOPPED); + shadowOf(Looper.getMainLooper()).idle(); + + assertThat(legacyListener.onCastSessionUnavailableCalled).isTrue(); + } + + @Test + public void onSessionEnding_withReceiverUpdates_freezesTimelineAndTracks() { + // Set up an active playing session with playlist, tracks, and progress. + List mediaTracks = Collections.singletonList(FAKE_MEDIA_TRACK_AUDIO); + MediaInfo mediaInfo = new MediaInfo.Builder("contentId").setMediaTracks(mediaTracks).build(); + when(mockMediaStatus.getMediaInfo()).thenReturn(mediaInfo); + when(mockMediaStatus.getActiveTrackIds()) + .thenReturn(new long[] {FAKE_MEDIA_TRACK_AUDIO.getId()}); + int[] queueItemIds = new int[] {1}; + when(mockMediaQueue.getItemIds()).thenReturn(queueItemIds); + when(mockRemoteMediaClient.getPlayerState()).thenReturn(MediaStatus.PLAYER_STATE_PLAYING); + when(mockRemoteMediaClient.isPaused()).thenReturn(false); + when(mockRemoteMediaClient.getApproximateStreamPosition()).thenReturn(12345L); + // Trigger callbacks to update the player state. + mediaQueueCallback.mediaQueueChanged(); + remoteMediaClientCallback.onStatusUpdated(); + progressListener.onProgressUpdated(12345L, 0L); + shadowOf(Looper.getMainLooper()).idle(); + // Validate the initial player state. + Timeline originalTimeline = remoteCastPlayer.getCurrentTimeline(); + Tracks originalTracks = remoteCastPlayer.getCurrentTracks(); + boolean originalPlayWhenReady = remoteCastPlayer.getPlayWhenReady(); + long originalPosition = remoteCastPlayer.getCurrentPosition(); + assertThat(originalTimeline.getWindowCount()).isEqualTo(1); + assertThat(originalTracks.getGroups()).hasSize(1); + assertThat(originalPlayWhenReady).isTrue(); + assertThat(originalPosition).isEqualTo(12345L); + + // Trigger the session disconnect sequence and simulate background GMS updates. + sessionManagerListener.onSessionEnding(mockCastSession); + when(mockMediaQueue.getItemIds()).thenReturn(new int[0]); + when(mockMediaStatus.getMediaInfo()).thenReturn(new MediaInfo.Builder("contentId").build()); + when(mockRemoteMediaClient.getApproximateStreamPosition()).thenReturn(0L); + progressListener.onProgressUpdated(0L, 0L); + mediaQueueCallback.mediaQueueChanged(); + remoteMediaClientCallback.onStatusUpdated(); + shadowOf(Looper.getMainLooper()).idle(); + + // Verify that player state parameters remained frozen. + assertThat(remoteCastPlayer.getCurrentTimeline()).isEqualTo(originalTimeline); + assertThat(remoteCastPlayer.getCurrentTracks()).isEqualTo(originalTracks); + assertThat(remoteCastPlayer.getPlayWhenReady()).isEqualTo(originalPlayWhenReady); + assertThat(remoteCastPlayer.getCurrentPosition()).isEqualTo(originalPosition); + } + + @Test + public void onSessionEnding_withClientSideUpdates_freezesState() { + // Set up an active playing session + when(mockRemoteMediaClient.getPlayerState()).thenReturn(MediaStatus.PLAYER_STATE_PLAYING); + when(mockRemoteMediaClient.isPaused()).thenReturn(false); + when(mockMediaStatus.getPlaybackRate()).thenReturn(1.0); + when(mockMediaStatus.getQueueRepeatMode()).thenReturn(MediaStatus.REPEAT_MODE_REPEAT_OFF); + remoteMediaClientCallback.onStatusUpdated(); + + // Trigger the session disconnect sequence + sessionManagerListener.onSessionEnding(mockCastSession); + // Simulate client-side attempts to modify the player state + remoteCastPlayer.setPlayWhenReady(false); + remoteCastPlayer.setPlaybackParameters(new PlaybackParameters(2.0f)); + remoteCastPlayer.setRepeatMode(Player.REPEAT_MODE_ALL); + shadowOf(Looper.getMainLooper()).idle(); + + // Verify that the state was frozen and client-side updates were ignored + assertThat(remoteCastPlayer.getPlayWhenReady()).isTrue(); + assertThat(remoteCastPlayer.getPlaybackParameters().speed).isEqualTo(1.0f); + assertThat(remoteCastPlayer.getRepeatMode()).isEqualTo(Player.REPEAT_MODE_OFF); + } + + @Test + public void setMediaItems_synchronouslyMasksTimelineBeforeReceiverConfirmation() { + MediaItem item1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem item2 = createMediaItem(/* mediaQueueItemId= */ 2); + + remoteCastPlayer.setMediaItems( + ImmutableList.of(item1, item2), /* startIndex= */ 1, /* startPositionMs= */ 5000L); + // Simulate an intermediate status update while load is still in-flight; masked timeline must + // be preserved. + remoteMediaClientCallback.onStatusUpdated(); + + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(2); + assertThat(remoteCastPlayer.getCurrentMediaItem()).isEqualTo(item2); + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(1); + assertThat(remoteCastPlayer.getCurrentPosition()).isEqualTo(5000L); + verify(mockListener) + .onTimelineChanged( + eq(remoteCastPlayer.getCurrentTimeline()), + eq(Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED)); + } + + @Test + public void addMediaItems_synchronouslyMasksTimelineAndDefersWhileLoadPending() { + MediaItem item1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem item2 = createMediaItem(/* mediaQueueItemId= */ 2); + MediaItem item3 = createMediaItem(/* mediaQueueItemId= */ 3); + remoteCastPlayer.setMediaItems(ImmutableList.of(item1, item2)); + + // Chain addMediaItems before load completes + remoteCastPlayer.addMediaItems(/* index= */ 1, ImmutableList.of(item3)); + + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(3); + Timeline.Window window = new Timeline.Window(); + assertThat(remoteCastPlayer.getCurrentTimeline().getWindow(0, window).mediaItem) + .isEqualTo(item1); + assertThat(remoteCastPlayer.getCurrentTimeline().getWindow(1, window).mediaItem) + .isEqualTo(item3); + assertThat(remoteCastPlayer.getCurrentTimeline().getWindow(2, window).mediaItem) + .isEqualTo(item2); + verify(mockRemoteMediaClient, never()) + .queueInsertItems(any(MediaQueueItem[].class), anyInt(), any()); + + // Confirm receiver IDs for initial load and trigger load completion callback + updateTimeLine( + ImmutableList.of(item1, item2), + /* mediaQueueItemIds= */ new int[] {10, 20}, + /* currentItemId= */ 10); + + verify(mockRemoteMediaClient).queueInsertItems(any(MediaQueueItem[].class), eq(20), eq(null)); + } + + @Test + public void moveMediaItems_synchronouslyMasksTimelineAndDefersWhileQueueOperationPending() { + MediaItem item1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem item2 = createMediaItem(/* mediaQueueItemId= */ 2); + MediaItem item3 = createMediaItem(/* mediaQueueItemId= */ 3); + remoteCastPlayer.setMediaItems(ImmutableList.of(item1, item2, item3)); + + // Move item 0 to index 2 while load is pending + remoteCastPlayer.moveMediaItems(/* fromIndex= */ 0, /* toIndex= */ 1, /* newIndex= */ 2); + + Timeline.Window window = new Timeline.Window(); + assertThat(remoteCastPlayer.getCurrentTimeline().getWindow(0, window).mediaItem) + .isEqualTo(item2); + assertThat(remoteCastPlayer.getCurrentTimeline().getWindow(1, window).mediaItem) + .isEqualTo(item3); + assertThat(remoteCastPlayer.getCurrentTimeline().getWindow(2, window).mediaItem) + .isEqualTo(item1); + verify(mockRemoteMediaClient, never()).queueReorderItems(any(int[].class), anyInt(), any()); + + // Confirm receiver IDs and complete load + updateTimeLine( + ImmutableList.of(item1, item2, item3), + /* mediaQueueItemIds= */ new int[] {10, 20, 30}, + /* currentItemId= */ 10); + + verify(mockRemoteMediaClient) + .queueReorderItems(eq(new int[] {10}), eq(MediaQueueItem.INVALID_ITEM_ID), eq(null)); + } + + @Test + public void removeMediaItems_synchronouslyMasksTimelineAndDefersWhileQueueOperationPending() { + MediaItem item1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem item2 = createMediaItem(/* mediaQueueItemId= */ 2); + MediaItem item3 = createMediaItem(/* mediaQueueItemId= */ 3); + remoteCastPlayer.setMediaItems(ImmutableList.of(item1, item2, item3)); + + // Remove item 1 while load is pending + remoteCastPlayer.removeMediaItems(/* fromIndex= */ 1, /* toIndex= */ 2); + + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(2); + Timeline.Window window = new Timeline.Window(); + assertThat(remoteCastPlayer.getCurrentTimeline().getWindow(0, window).mediaItem) + .isEqualTo(item1); + assertThat(remoteCastPlayer.getCurrentTimeline().getWindow(1, window).mediaItem) + .isEqualTo(item3); + verify(mockRemoteMediaClient, never()).queueRemoveItems(any(int[].class), any()); + + // Confirm receiver IDs and complete load + updateTimeLine( + ImmutableList.of(item1, item2, item3), + /* mediaQueueItemIds= */ new int[] {10, 20, 30}, + /* currentItemId= */ 10); + + verify(mockRemoteMediaClient).queueRemoveItems(eq(new int[] {20}), eq(null)); + } + + @Test + public void seekTo_whileLoadPending_synchronouslyMasksAndDefersQueueJumpToItem() { + MediaItem item1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem item2 = createMediaItem(/* mediaQueueItemId= */ 2); + remoteCastPlayer.setMediaItems(ImmutableList.of(item1, item2)); + + // Seek to item 1 while load is pending + remoteCastPlayer.seekTo(/* mediaItemIndex= */ 1, /* positionMs= */ 3000L); + + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(1); + assertThat(remoteCastPlayer.getCurrentPosition()).isEqualTo(3000L); + verify(mockRemoteMediaClient, never()).queueJumpToItem(anyInt(), anyLong(), any()); + + // Confirm receiver IDs and complete load + updateTimeLine( + ImmutableList.of(item1, item2), + /* mediaQueueItemIds= */ new int[] {10, 20}, + /* currentItemId= */ 10); + + verify(mockRemoteMediaClient).queueJumpToItem(eq(20), eq(3000L), eq(null)); + } + + @Test + public void + addMediaItems_withConcurrentRemoteSenderInsertion_preservesMaskedStateAndResolvesShiftedPosition() { + MediaItem item1 = createMediaItem(/* mediaQueueItemId= */ 1); + updateTimeLine( + ImmutableList.of(item1), /* mediaQueueItemIds= */ new int[] {10}, /* currentItemId= */ 10); + + ArgumentCaptor insertedItemsCaptor = + ArgumentCaptor.forClass(MediaQueueItem[].class); + when(mockRemoteMediaClient.queueInsertItems(any(MediaQueueItem[].class), anyInt(), any())) + .thenReturn(mockPendingResult); + + MediaItem localItemA = + new MediaItem.Builder() + .setUri("http://www.google.com/localA") + .setTag("local_tag_A") + .setMimeType(MimeTypes.APPLICATION_MPD) + .build(); + + // Local sender adds localItemA at index 1 and immediately seeks to it + remoteCastPlayer.addMediaItems(/* index= */ 1, ImmutableList.of(localItemA)); + verify(mockRemoteMediaClient) + .queueInsertItems(insertedItemsCaptor.capture(), eq(MediaQueueItem.INVALID_ITEM_ID), any()); + verify(mockPendingResult).setResultCallback(setResultCallbackArgumentCaptor.capture()); + + Object optimisticUidA = + remoteCastPlayer + .getCurrentTimeline() + .getWindow(/* windowIndex= */ 1, new Timeline.Window()) + .uid; + remoteCastPlayer.seekTo(/* mediaItemIndex= */ 1, /* positionMs= */ 4000L); + + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(2); + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(1); + verify(mockRemoteMediaClient, never()).queueJumpToItem(anyInt(), anyLong(), any()); + + // Another sender (Sender B) concurrently inserts remoteItemB at index 0 before localItemA is + // confirmed by the receiver. + MediaItem remoteItemB = createMediaItem(/* mediaQueueItemId= */ 99); + updateTimeLine( + ImmutableList.of(remoteItemB, item1), + /* mediaQueueItemIds= */ new int[] {99, 10}, + /* currentItemId= */ 10); + + // While local operation is in flight, masked timeline replays onto the updated base snapshot + // so both remoteItemB (index 0) and localItemA (index 2) are present. + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(3); + assertThat( + remoteCastPlayer + .getCurrentTimeline() + .getWindow(/* windowIndex= */ 2, new Timeline.Window()) + .mediaItem) + .isEqualTo(localItemA); + + // Receiver now confirms localItemA at index 2 (shifted by remoteItemB at index 0) + MediaQueueItem confirmedItemA = + new MediaQueueItem.Builder(insertedItemsCaptor.getValue()[0].getMedia()) + .setItemId(20) + .build(); + MediaQueueItem queueItemB = mock(MediaQueueItem.class); + when(queueItemB.getItemId()).thenReturn(99); + when(queueItemB.getMedia()) + .thenReturn( + new MediaInfo.Builder("http://www.google.com/video99") + .setStreamType(MediaInfo.STREAM_TYPE_BUFFERED) + .setContentType(MimeTypes.APPLICATION_MPD) + .build()); + MediaQueueItem queueItem1 = mock(MediaQueueItem.class); + when(queueItem1.getItemId()).thenReturn(10); + when(queueItem1.getMedia()) + .thenReturn( + new MediaInfo.Builder("http://www.google.com/video1") + .setStreamType(MediaInfo.STREAM_TYPE_BUFFERED) + .setContentType(MimeTypes.APPLICATION_MPD) + .build()); + when(mockMediaQueue.getItemIds()).thenReturn(new int[] {99, 10, 20}); + when(mockMediaStatus.getQueueItems()) + .thenReturn(ImmutableList.of(queueItemB, queueItem1, confirmedItemA)); + when(mockMediaStatus.getItemById(99)).thenReturn(queueItemB); + when(mockMediaStatus.getItemById(10)).thenReturn(queueItem1); + when(mockMediaStatus.getItemById(20)).thenReturn(confirmedItemA); + when(mockMediaStatus.getCurrentItemId()).thenReturn(10); + + MediaChannelResult successResult = mock(MediaChannelResult.class); + when(successResult.getStatus()).thenReturn(new Status(CastStatusCodes.SUCCESS)); + setResultCallbackArgumentCaptor.getValue().onResult(successResult); + shadowOf(Looper.getMainLooper()).idle(); + + // Timeline unmasks to [remoteItemB, item1, localItemA] + Timeline resolvedTimeline = remoteCastPlayer.getCurrentTimeline(); + assertThat(resolvedTimeline.getWindowCount()).isEqualTo(3); + Timeline.Window resolvedWindowA = + resolvedTimeline.getWindow(/* windowIndex= */ 2, new Timeline.Window()); + // Despite shifting from index 1 to index 2, localItemA preserves its optimistic UID and tag + assertThat(resolvedWindowA.uid).isEqualTo(optimisticUidA); + assertThat(resolvedWindowA.mediaItem).isEqualTo(localItemA); + assertThat(resolvedWindowA.mediaItem.localConfiguration.tag).isEqualTo("local_tag_A"); + // Deferred seek targets receiver item ID 20 (index 2), not index 1 + verify(mockRemoteMediaClient).queueJumpToItem(eq(20), eq(4000L), eq(null)); + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(2); + } + + @Test + public void + setMediaItems_withDeferredMediaQueueFetches_holdsMaskingAndDrainsPendingSeekWhenQueueFetched() { + MediaItem item0 = + new MediaItem.Builder().setUri("http://www.google.com/video0").setTag("tag0").build(); + MediaItem item1 = + new MediaItem.Builder().setUri("http://www.google.com/video1").setTag("tag1").build(); + when(mockRemoteMediaClient.load(any(MediaLoadRequestData.class))).thenReturn(mockPendingResult); + when(mockRemoteMediaClient.queueJumpToItem(anyInt(), anyLong(), any())) + .thenReturn(mockPendingResult); + + remoteCastPlayer.setMediaItems( + ImmutableList.of(item0, item1), /* startIndex= */ 0, /* startPositionMs= */ 5000L); + Object optimisticUid1 = + remoteCastPlayer + .getCurrentTimeline() + .getWindow(/* windowIndex= */ 1, new Timeline.Window()) + .uid; + remoteCastPlayer.seekTo(/* mediaItemIndex= */ 1, /* positionMs= */ 12000L); + + verify(mockRemoteMediaClient).load(loadArgumentCaptor.capture()); + verify(mockPendingResult).setResultCallback(setResultCallbackArgumentCaptor.capture()); + ResultCallback loadCallback = setResultCallbackArgumentCaptor.getValue(); + + MediaQueueData queueData = loadArgumentCaptor.getValue().getQueueData(); + MediaQueueItem sentItem0 = queueData.getItems().get(0); + MediaQueueItem sentItem1 = queueData.getItems().get(1); + MediaQueueItem confirmedItem0 = + new MediaQueueItem.Builder(sentItem0.getMedia()).setItemId(100).build(); + MediaQueueItem confirmedItem1 = + new MediaQueueItem.Builder(sentItem1.getMedia()).setItemId(101).build(); + + // Receiver responds to load with itemIds [100, 101], but only item 100 is in MediaStatus + // initially; item 101 requires an async MediaQueue fetch (getItemAtIndex returns null). + when(mockMediaQueue.getItemIds()).thenReturn(new int[] {100, 101}); + when(mockMediaStatus.getCurrentItemId()).thenReturn(100); + when(mockMediaStatus.getMediaInfo()).thenReturn(confirmedItem0.getMedia()); + when(mockMediaStatus.getQueueItems()).thenReturn(ImmutableList.of(confirmedItem0)); + when(mockMediaQueue.getItemAtIndex(eq(1), anyBoolean())).thenReturn(null); + + MediaChannelResult successResult = mock(MediaChannelResult.class); + when(successResult.getStatus()).thenReturn(new Status(CastStatusCodes.SUCCESS)); + loadCallback.onResult(successResult); + shadowOf(Looper.getMainLooper()).idle(); + + // While item 101 fetch is pending in MediaQueue, timeline masking and pending seek are + // preserved and queueJumpToItem is not yet dispatched. + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(2); + assertThat( + remoteCastPlayer + .getCurrentTimeline() + .getWindow(/* windowIndex= */ 1, new Timeline.Window()) + .uid) + .isEqualTo(optimisticUid1); + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(1); + assertThat(remoteCastPlayer.getCurrentPosition()).isEqualTo(12000L); + verify(mockRemoteMediaClient, never()).queueJumpToItem(anyInt(), anyLong(), any()); + + // Simulate MediaQueue completing the background fetch for item 101 and firing + // mediaQueueChanged(). + when(mockMediaQueue.getItemAtIndex(eq(1), anyBoolean())).thenReturn(confirmedItem1); + mediaQueueCallback.mediaQueueChanged(); + shadowOf(Looper.getMainLooper()).idle(); + + // Pending seek is drained and dispatched to receiver item ID 101, and timeline unmasks + // preserving item 1's UID and metadata. + verify(mockRemoteMediaClient).queueJumpToItem(eq(101), eq(12000L), eq(null)); + Timeline finalTimeline = remoteCastPlayer.getCurrentTimeline(); + assertThat(finalTimeline.getWindowCount()).isEqualTo(2); + assertThat(finalTimeline.getWindow(/* windowIndex= */ 1, new Timeline.Window()).uid) + .isEqualTo(optimisticUid1); + assertThat(finalTimeline.getWindow(/* windowIndex= */ 1, new Timeline.Window()).mediaItem) + .isEqualTo(item1); + } + + @Test + public void addMediaItems_masksTimelineImmediately() { + MediaItem mediaItem1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem mediaItem2 = createMediaItem(/* mediaQueueItemId= */ 2); + ImmutableList mediaItems = ImmutableList.of(mediaItem1, mediaItem2); + + remoteCastPlayer.addMediaItems(mediaItems); + + assertThat(remoteCastPlayer.getMediaItemCount()).isEqualTo(2); + Timeline timeline = remoteCastPlayer.getCurrentTimeline(); + Timeline.Window window = new Timeline.Window(); + assertThat(timeline.getWindow(/* windowIndex= */ 0, window).mediaItem).isEqualTo(mediaItem1); + assertThat(timeline.getWindow(/* windowIndex= */ 1, window).mediaItem).isEqualTo(mediaItem2); + } + + @Test + public void removeMediaItems_masksTimelineImmediately() { + int[] mediaQueueItemIds = new int[] {1, 2, 3}; + List mediaItems = createMediaItems(mediaQueueItemIds); + addMediaItemsAndUpdateTimeline(mediaItems, mediaQueueItemIds); + + remoteCastPlayer.removeMediaItem(/* index= */ 1); + + assertThat(remoteCastPlayer.getMediaItemCount()).isEqualTo(2); + Timeline timeline = remoteCastPlayer.getCurrentTimeline(); + Timeline.Window window = new Timeline.Window(); + assertThat(timeline.getWindow(/* windowIndex= */ 0, window).mediaItem) + .isEqualTo(mediaItems.get(0)); + assertThat(timeline.getWindow(/* windowIndex= */ 1, window).mediaItem) + .isEqualTo(mediaItems.get(2)); + } + + @Test + public void moveMediaItem_masksTimelineImmediately() { + int[] mediaQueueItemIds = new int[] {1, 2, 3}; + List mediaItems = createMediaItems(mediaQueueItemIds); + addMediaItemsAndUpdateTimeline(mediaItems, mediaQueueItemIds); + + remoteCastPlayer.moveMediaItem(/* currentIndex= */ 0, /* newIndex= */ 2); + + assertThat(remoteCastPlayer.getMediaItemCount()).isEqualTo(3); + Timeline timeline = remoteCastPlayer.getCurrentTimeline(); + Timeline.Window window = new Timeline.Window(); + assertThat(timeline.getWindow(/* windowIndex= */ 0, window).mediaItem) + .isEqualTo(mediaItems.get(1)); + assertThat(timeline.getWindow(/* windowIndex= */ 1, window).mediaItem) + .isEqualTo(mediaItems.get(2)); + assertThat(timeline.getWindow(/* windowIndex= */ 2, window).mediaItem) + .isEqualTo(mediaItems.get(0)); + } + + @Test + public void replaceMediaItems_masksTimelineImmediately() { + int[] mediaQueueItemIds = new int[] {1, 2, 3}; + List mediaItems = createMediaItems(mediaQueueItemIds); + addMediaItemsAndUpdateTimeline(mediaItems, mediaQueueItemIds); + ImmutableList replacementItems = + ImmutableList.of( + createMediaItem(/* mediaQueueItemId= */ 4), createMediaItem(/* mediaQueueItemId= */ 5)); + + remoteCastPlayer.replaceMediaItems(/* fromIndex= */ 1, /* toIndex= */ 3, replacementItems); + + assertThat(remoteCastPlayer.getMediaItemCount()).isEqualTo(3); + Timeline timeline = remoteCastPlayer.getCurrentTimeline(); + Timeline.Window window = new Timeline.Window(); + assertThat(timeline.getWindow(/* windowIndex= */ 0, window).mediaItem) + .isEqualTo(mediaItems.get(0)); + assertThat(timeline.getWindow(/* windowIndex= */ 1, window).mediaItem) + .isEqualTo(replacementItems.get(0)); + assertThat(timeline.getWindow(/* windowIndex= */ 2, window).mediaItem) + .isEqualTo(replacementItems.get(1)); + } + + @Test + public void addAndRemoveMediaItems_masksTimelineCorrectly() { + MediaItem mediaItem1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem mediaItem2 = createMediaItem(/* mediaQueueItemId= */ 2); + ImmutableList mediaItems = ImmutableList.of(mediaItem1, mediaItem2); + + remoteCastPlayer.addMediaItems(mediaItems); + remoteCastPlayer.removeMediaItem(/* index= */ 0); + + assertThat(remoteCastPlayer.getMediaItemCount()).isEqualTo(1); + Timeline timeline = remoteCastPlayer.getCurrentTimeline(); + Timeline.Window window = new Timeline.Window(); + assertThat(timeline.getWindow(/* windowIndex= */ 0, window).mediaItem).isEqualTo(mediaItem2); + } + + @Test + public void addAndMoveMediaItems_masksTimelineCorrectly() { + MediaItem mediaItem1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem mediaItem2 = createMediaItem(/* mediaQueueItemId= */ 2); + ImmutableList mediaItems = ImmutableList.of(mediaItem1, mediaItem2); + + remoteCastPlayer.addMediaItems(mediaItems); + remoteCastPlayer.moveMediaItem(/* currentIndex= */ 0, /* newIndex= */ 1); + + assertThat(remoteCastPlayer.getMediaItemCount()).isEqualTo(2); + Timeline timeline = remoteCastPlayer.getCurrentTimeline(); + Timeline.Window window = new Timeline.Window(); + assertThat(timeline.getWindow(/* windowIndex= */ 0, window).mediaItem).isEqualTo(mediaItem2); + assertThat(timeline.getWindow(/* windowIndex= */ 1, window).mediaItem).isEqualTo(mediaItem1); + } + + @Test + public void addMediaItems_remoteUpdateConfirms_addMoreMediaItems() { + MediaItem mediaItem1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem mediaItem2 = createMediaItem(/* mediaQueueItemId= */ 2); + + remoteCastPlayer.addMediaItem(mediaItem1); + updateTimeLine(ImmutableList.of(mediaItem1), new int[] {1}, /* currentItemId= */ 1); + remoteCastPlayer.addMediaItem(mediaItem2); + + assertThat(remoteCastPlayer.getMediaItemCount()).isEqualTo(2); + Timeline timeline = remoteCastPlayer.getCurrentTimeline(); + Timeline.Window window = new Timeline.Window(); + assertThat(timeline.getWindow(/* windowIndex= */ 0, window).mediaItem).isEqualTo(mediaItem1); + assertThat(timeline.getWindow(/* windowIndex= */ 1, window).mediaItem).isEqualTo(mediaItem2); + } + + @Test + public void remoteUpdateWithUnexpectedItems_overridesMaskedTimeline() { + MediaItem mediaItem1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem mediaItem2 = createMediaItem(/* mediaQueueItemId= */ 2); + addMediaItemsAndUpdateTimeline(ImmutableList.of(mediaItem1, mediaItem2), new int[] {1, 2}); + MediaItem unexpectedMediaItem = + new MediaItem.Builder() + .setMediaId("http://www.google.com/video3") + .setUri("http://www.google.com/video3") + .setMimeType(MimeTypes.APPLICATION_MPD) + .build(); + + remoteCastPlayer.removeMediaItem(/* index= */ 0); + updateTimeLine( + ImmutableList.of(mediaItem2, unexpectedMediaItem), + new int[] {2, 3}, + /* currentItemId= */ 2); + shadowOf(Looper.getMainLooper()).idle(); + + assertThat(remoteCastPlayer.getMediaItemCount()).isEqualTo(2); + Timeline timeline = remoteCastPlayer.getCurrentTimeline(); + Timeline.Window window = new Timeline.Window(); + assertThat(timeline.getWindow(/* windowIndex= */ 0, window).mediaItem).isEqualTo(mediaItem2); + assertThat(timeline.getWindow(/* windowIndex= */ 1, window).mediaItem) + .isEqualTo(unexpectedMediaItem); + } + + @Test + public void multipleQueueOperations_areSerializedUntilEachResultCallbackCompletes() { + MediaItem item1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem item2 = createMediaItem(/* mediaQueueItemId= */ 2); + MediaItem item3 = createMediaItem(/* mediaQueueItemId= */ 3); + updateTimeLine( + ImmutableList.of(item1, item2, item3), + /* mediaQueueItemIds= */ new int[] {10, 20, 30}, + /* currentItemId= */ 10); + + when(mockRemoteMediaClient.queueRemoveItems(any(int[].class), any())) + .thenReturn(mockPendingResult); + when(mockRemoteMediaClient.queueReorderItems(any(int[].class), anyInt(), any())) + .thenReturn(mockPendingResult); + when(mockRemoteMediaClient.queueJumpToItem(anyInt(), anyLong(), any())) + .thenReturn(mockPendingResult); + + // Issue 3 queue operations in a single synchronous turn: + // 1. Remove item3 (index 2 -> receiver ID 30) + // 2. Move item2 from index 1 to index 0 (before item1 -> receiver ID 10) + // 3. Seek to item2 at index 0 (receiver ID 20) at position 5000ms + remoteCastPlayer.removeMediaItem(/* index= */ 2); + remoteCastPlayer.moveMediaItem(/* currentIndex= */ 1, /* newIndex= */ 0); + remoteCastPlayer.seekTo(/* mediaItemIndex= */ 0, /* positionMs= */ 5000L); + + // Optimistic masked state immediately reflects all 3 operations: [item2, item1], index 0, + // 5000ms + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(2); + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(0); + assertThat(remoteCastPlayer.getCurrentPosition()).isEqualTo(5000L); + + // Only the first queue RPC (queueRemoveItems) is sent immediately; reorder and jump wait + verify(mockRemoteMediaClient).queueRemoveItems(eq(new int[] {30}), eq(null)); + verify(mockPendingResult).setResultCallback(setResultCallbackArgumentCaptor.capture()); + verify(mockRemoteMediaClient, never()).queueReorderItems(any(int[].class), anyInt(), any()); + verify(mockRemoteMediaClient, never()).queueJumpToItem(anyInt(), anyLong(), any()); + + // Complete remove operation -> dispatches move operation (queueReorderItems), jump still waits + clearInvocations(mockPendingResult); + MediaChannelResult successResult = mock(MediaChannelResult.class); + when(successResult.getStatus()).thenReturn(new Status(CastStatusCodes.SUCCESS)); + setResultCallbackArgumentCaptor.getValue().onResult(successResult); + + verify(mockRemoteMediaClient).queueReorderItems(eq(new int[] {20}), eq(10), eq(null)); + verify(mockPendingResult).setResultCallback(setResultCallbackArgumentCaptor.capture()); + verify(mockRemoteMediaClient, never()).queueJumpToItem(anyInt(), anyLong(), any()); + + // Complete move operation -> dispatches seek operation (queueJumpToItem) + clearInvocations(mockPendingResult); + setResultCallbackArgumentCaptor.getValue().onResult(successResult); + + verify(mockRemoteMediaClient).queueJumpToItem(eq(20), eq(5000L), eq(null)); + verify(mockPendingResult).setResultCallback(setResultCallbackArgumentCaptor.capture()); + + // Complete seek operation with receiver status updated to [item2, item1] playing item 20 + updateTimeLine( + ImmutableList.of(item2, item1), + /* mediaQueueItemIds= */ new int[] {20, 10}, + /* currentItemId= */ 20); + setResultCallbackArgumentCaptor.getValue().onResult(successResult); + shadowOf(Looper.getMainLooper()).idle(); + + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(2); + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(0); + } + + @Test + public void + seekToWindow_whenReceiverReturnsError_revertsCurrentWindowIndexAndNotifiesListeners() { + MediaItem item1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem item2 = createMediaItem(/* mediaQueueItemId= */ 2); + updateTimeLine( + ImmutableList.of(item1, item2), + /* mediaQueueItemIds= */ new int[] {10, 20}, + /* currentItemId= */ 10); + MediaItem timelineItem1 = + remoteCastPlayer + .getCurrentTimeline() + .getWindow(/* windowIndex= */ 0, new Timeline.Window()) + .mediaItem; + MediaItem timelineItem2 = + remoteCastPlayer + .getCurrentTimeline() + .getWindow(/* windowIndex= */ 1, new Timeline.Window()) + .mediaItem; + + when(mockRemoteMediaClient.queueJumpToItem(anyInt(), anyLong(), any())) + .thenReturn(mockPendingResult); + + clearInvocations(mockListener); + + // Seek from item1 (index 0) to item2 (index 1) + remoteCastPlayer.seekTo(/* mediaItemIndex= */ 1, /* positionMs= */ 3000L); + shadowOf(Looper.getMainLooper()).idle(); + + // Optimistically reports index 1 + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(1); + verify(mockRemoteMediaClient).queueJumpToItem(eq(20), eq(3000L), eq(null)); + verify(mockPendingResult).setResultCallback(setResultCallbackArgumentCaptor.capture()); + verify(mockListener) + .onMediaItemTransition(eq(timelineItem2), eq(Player.MEDIA_ITEM_TRANSITION_REASON_SEEK)); + + clearInvocations(mockListener); + + // Receiver rejects queueJumpToItem with INVALID_REQUEST (2001) and remains on item 10 (index 0) + MediaChannelResult errorResult = mock(MediaChannelResult.class); + when(errorResult.getStatus()).thenReturn(new Status(CastStatusCodes.INVALID_REQUEST)); + setResultCallbackArgumentCaptor.getValue().onResult(errorResult); + shadowOf(Looper.getMainLooper()).idle(); + + // Player reverts currentMediaItemIndex back to 0 and notifies listeners of the adjustment + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(0); + verify(mockListener) + .onPositionDiscontinuity( + any(Player.PositionInfo.class), + any(Player.PositionInfo.class), + eq(Player.DISCONTINUITY_REASON_AUTO_TRANSITION)); + verify(mockListener) + .onMediaItemTransition(eq(timelineItem1), eq(Player.MEDIA_ITEM_TRANSITION_REASON_AUTO)); + } + + @Test + public void removeMediaItem_currentlyPlayingMediaItem_transitionsToNextItem() { + MediaItem item1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem item2 = createMediaItem(/* mediaQueueItemId= */ 2); + MediaItem item3 = createMediaItem(/* mediaQueueItemId= */ 3); + remoteCastPlayer.addMediaItems(ImmutableList.of(item1, item2, item3)); + updateTimeLine( + ImmutableList.of(item1, item2, item3), + /* mediaQueueItemIds= */ new int[] {10, 20, 30}, + /* currentItemId= */ 20); + + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(1); + assertThat(remoteCastPlayer.getCurrentMediaItem()).isEqualTo(item2); + clearInvocations(mockListener); + + // Remove the currently playing middle item (item2 at index 1) + remoteCastPlayer.removeMediaItem(/* index= */ 1); + shadowOf(Looper.getMainLooper()).idle(); + + // Optimistic masked state should transition to the NEXT item (item3, now at index 1), + // not the previous item (item1 at index 0) + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(2); + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(1); + assertThat(remoteCastPlayer.getCurrentMediaItem()).isEqualTo(item3); + verify(mockListener) + .onMediaItemTransition(eq(item3), eq(Player.MEDIA_ITEM_TRANSITION_REASON_PLAYLIST_CHANGED)); + } + + @Test + public void + seekTo_whenReceiverTransitionsThroughLoadingAndUncachedCurrentItem_doesNotBounceWindowIndexToZero() { + MediaItem item1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem item2 = createMediaItem(/* mediaQueueItemId= */ 2); + MediaItem item3 = createMediaItem(/* mediaQueueItemId= */ 3); + remoteCastPlayer.addMediaItems(ImmutableList.of(item1, item2, item3)); + updateTimeLine( + ImmutableList.of(item1, item2, item3), + /* mediaQueueItemIds= */ new int[] {10, 20, 30}, + /* currentItemId= */ 10); + + when(mockRemoteMediaClient.queueJumpToItem(anyInt(), anyLong(), any())) + .thenReturn(mockPendingResult); + + verify(mockRemoteMediaClient).load(loadArgumentCaptor.capture()); + ImmutableList loadedItems = + ImmutableList.copyOf(loadArgumentCaptor.getValue().getQueueData().getItems()); + MediaQueueItem queueItem10 = + new MediaQueueItem.Builder(loadedItems.get(0).getMedia()).setItemId(10).build(); + MediaQueueItem queueItem30 = + new MediaQueueItem.Builder(loadedItems.get(2).getMedia()).setItemId(30).build(); + when(mockMediaStatus.getItemById(10)).thenReturn(queueItem10); + when(mockMediaStatus.getItemById(30)).thenReturn(queueItem30); + when(mockMediaQueue.getItemAtIndex(eq(0), anyBoolean())).thenReturn(queueItem10); + when(mockMediaQueue.getItemAtIndex(eq(2), anyBoolean())).thenReturn(queueItem30); + + // Seek from index 0 (item 10) to index 2 (item 30). + remoteCastPlayer.seekTo(/* mediaItemIndex= */ 2, /* positionMs= */ 0L); + shadowOf(Looper.getMainLooper()).idle(); + verify(mockPendingResult).setResultCallback(setResultCallbackArgumentCaptor.capture()); + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(2); + clearInvocations(mockListener); + + // Phase 1: Seek RPC completes while receiver is in LOADING state for item 30 (loadingItemId=30, + // currentItemId still 10, getCurrentItem() still returns queueItem10, mediaInfo is item 30). + MediaInfo mediaInfo30 = queueItem30.getMedia(); + when(mockMediaStatus.getLoadingItemId()).thenReturn(30); + when(mockMediaStatus.getCurrentItemId()).thenReturn(10); + when(mockRemoteMediaClient.getCurrentItem()).thenReturn(queueItem10); + when(mockMediaStatus.getMediaInfo()).thenReturn(mediaInfo30); + when(mockMediaStatus.getQueueItems()).thenReturn(ImmutableList.of(queueItem10)); + + MediaChannelResult successResult = mock(MediaChannelResult.class); + when(successResult.getStatus()).thenReturn(new Status(CastStatusCodes.SUCCESS)); + setResultCallbackArgumentCaptor.getValue().onResult(successResult); + shadowOf(Looper.getMainLooper()).idle(); + + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(2); + verify(mockListener, never()).onPositionDiscontinuity(any(), any(), anyInt()); + verify(mockListener, never()).onMediaItemTransition(any(), anyInt()); + + // Phase 2: Receiver finishes loading item 30 (loadingItemId=0, currentItemId=30), but + // getCurrentItem() returns null because MediaStatus's sparse queueItems window still only + // contains queueItem10. + when(mockMediaStatus.getLoadingItemId()).thenReturn(MediaQueueItem.INVALID_ITEM_ID); + when(mockMediaStatus.getCurrentItemId()).thenReturn(30); + when(mockRemoteMediaClient.getCurrentItem()).thenReturn(null); + remoteMediaClientCallback.onStatusUpdated(); + shadowOf(Looper.getMainLooper()).idle(); + + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(2); + verify(mockListener, never()).onPositionDiscontinuity(any(), any(), anyInt()); + verify(mockListener, never()).onMediaItemTransition(any(), anyInt()); + } + + @Test + public void + setMediaItems_whenLoadResultArrivesBeforeMediaQueueFetchesItemIds_preservesMaskedTimeline() { + MediaItem item1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem item2 = createMediaItem(/* mediaQueueItemId= */ 2); + MediaItem item3 = createMediaItem(/* mediaQueueItemId= */ 3); + when(mockRemoteMediaClient.load(any(MediaLoadRequestData.class))).thenReturn(mockPendingResult); + + remoteCastPlayer.setMediaItems(ImmutableList.of(item1, item2, item3)); + shadowOf(Looper.getMainLooper()).idle(); + verify(mockPendingResult).setResultCallback(setResultCallbackArgumentCaptor.capture()); + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(3); + clearInvocations(mockListener); + + // Load RPC completes and MediaStatus reports active item 1, but MediaQueue has not yet + // fetched itemIds (itemIds.length == 0 -> hasPendingQueueFetches == true). + when(mockMediaQueue.getItemIds()).thenReturn(new int[0]); + when(mockMediaStatus.getCurrentItemId()).thenReturn(1); + when(mockMediaStatus.getQueueItems()).thenReturn(ImmutableList.of()); + MediaChannelResult successResult = mock(MediaChannelResult.class); + when(successResult.getStatus()).thenReturn(new Status(CastStatusCodes.SUCCESS)); + setResultCallbackArgumentCaptor.getValue().onResult(successResult); + shadowOf(Looper.getMainLooper()).idle(); + + // Masked 3-item timeline must be preserved rather than dropping to 0 items. + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(3); + verify(mockListener, never()).onTimelineChanged(any(), anyInt()); + } + + @Test + public void onSessionEnded_withUnconfirmedMaskedOperations_preservesMaskedTimeline() { + MediaItem item1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem item2 = createMediaItem(/* mediaQueueItemId= */ 2); + remoteCastPlayer.addMediaItems(ImmutableList.of(item1)); + updateTimeLine( + ImmutableList.of(item1), /* mediaQueueItemIds= */ new int[] {1}, /* currentItemId= */ 1); + + // Enqueue an optimistic addition without confirming it on the receiver before session ends. + remoteCastPlayer.addMediaItems(ImmutableList.of(item2)); + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(2); + + sessionManagerListener.onSessionEnding(mockCastSession); + sessionManagerListener.onSessionEnded( + mockCastSession, RemoteCastPlayer.SESSION_END_REASON_STOPPED); + shadowOf(Looper.getMainLooper()).idle(); + + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(2); + Timeline.Window window = new Timeline.Window(); + assertThat(remoteCastPlayer.getCurrentTimeline().getWindow(0, window).mediaItem) + .isEqualTo(item1); + assertThat(remoteCastPlayer.getCurrentTimeline().getWindow(1, window).mediaItem) + .isEqualTo(item2); + } + + @Test + public void onSessionStarted_afterPreviousSessionEnded_clearsSnapshots() { + MediaItem item1 = createMediaItem(/* mediaQueueItemId= */ 1); + remoteCastPlayer.addMediaItems(ImmutableList.of(item1)); + updateTimeLine( + ImmutableList.of(item1), /* mediaQueueItemIds= */ new int[] {1}, /* currentItemId= */ 1); + sessionManagerListener.onSessionEnding(mockCastSession); + sessionManagerListener.onSessionEnded( + mockCastSession, RemoteCastPlayer.SESSION_END_REASON_STOPPED); + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(1); + + // Join a new Cast session that has no active media status yet. + CastSession newCastSession = mock(CastSession.class); + RemoteMediaClient newRemoteMediaClient = mock(RemoteMediaClient.class); + MediaQueue newMediaQueue = mock(MediaQueue.class); + when(newCastSession.getRemoteMediaClient()).thenReturn(newRemoteMediaClient); + when(newRemoteMediaClient.getMediaQueue()).thenReturn(newMediaQueue); + when(newRemoteMediaClient.getMediaStatus()).thenReturn(null); + + sessionManagerListener.onSessionStarted(newCastSession, /* sessionId= */ "new_session"); + shadowOf(Looper.getMainLooper()).idle(); + + assertThat(remoteCastPlayer.getCurrentTimeline().isEmpty()).isTrue(); + } + + @Test + public void + queuedOperation_whenTargetItemRemovedOnReceiver_prunesOperationAndDispatchesNextOperation() { + MediaItem item1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem item2 = createMediaItem(/* mediaQueueItemId= */ 2); + MediaItem item3 = createMediaItem(/* mediaQueueItemId= */ 3); + MediaItem item4 = createMediaItem(/* mediaQueueItemId= */ 4); + updateTimeLine( + ImmutableList.of(item1, item2, item3, item4), + /* mediaQueueItemIds= */ new int[] {10, 20, 30, 40}, + /* currentItemId= */ 10); + when(mockRemoteMediaClient.queueRemoveItems(any(int[].class), any())) + .thenReturn(mockPendingResult); + when(mockRemoteMediaClient.queueReorderItems(any(int[].class), anyInt(), any())) + .thenReturn(mockPendingResult); + when(mockRemoteMediaClient.queueJumpToItem(anyInt(), anyLong(), any())) + .thenReturn(mockPendingResult); + // Enqueue 3 operations: + // 1. Remove item1 at index 0 (dispatched immediately). Masked timeline becomes [item2, item3, + // item4] playing index 0 (item2). + // 2. Move item2 from index 0 to index 2 (queued). Masked timeline becomes [item3, item4, + // item2] playing index 2 (item2). + // 3. Seek to item4 at index 1 at position 5000ms (queued). + remoteCastPlayer.removeMediaItem(/* index= */ 0); + remoteCastPlayer.moveMediaItem(/* currentIndex= */ 0, /* newIndex= */ 2); + remoteCastPlayer.seekTo(/* mediaItemIndex= */ 1, /* positionMs= */ 5000L); + verify(mockPendingResult).setResultCallback(setResultCallbackArgumentCaptor.capture()); + + // Simulate another sender concurrently removing item2 (receiver ID 20) on the receiver, so + // when the remove operation completes, the receiver timeline only contains [item3, item4]. + clearInvocations(mockPendingResult); + updateTimeLine( + ImmutableList.of(item3, item4), + /* mediaQueueItemIds= */ new int[] {30, 40}, + /* currentItemId= */ 30, + /* streamTypes= */ new int[] { + MediaInfo.STREAM_TYPE_BUFFERED, MediaInfo.STREAM_TYPE_BUFFERED + }, + /* durationsMs= */ new long[] {20_000, 20_000}, + /* positionMs= */ 0L, + /* notifyStatusUpdate= */ false); + MediaChannelResult successResult = mock(MediaChannelResult.class); + when(successResult.getStatus()).thenReturn(new Status(CastStatusCodes.SUCCESS)); + setResultCallbackArgumentCaptor.getValue().onResult(successResult); + shadowOf(Looper.getMainLooper()).idle(); + + // The move operation targeting missing item2 (ID 20) is pruned without calling + // queueReorderItems, and the subsequent seek operation targeting item4 (ID 40) is dispatched. + verify(mockRemoteMediaClient, never()).queueReorderItems(any(int[].class), anyInt(), any()); + verify(mockRemoteMediaClient).queueJumpToItem(eq(40), eq(5000L), eq(null)); + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(2); + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(1); + assertThat(remoteCastPlayer.getCurrentPosition()).isEqualTo(5000L); + } + + @Test + public void + addMediaItems_twoCallsInQuickSuccessionOnEmptyTimeline_callsLoadThenQueueInsertItems() { + when(mockRemoteMediaClient.getMediaStatus()).thenReturn(null); + MediaItem item1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem item2 = createMediaItem(/* mediaQueueItemId= */ 2); + + remoteCastPlayer.addMediaItems(ImmutableList.of(item1)); + remoteCastPlayer.addMediaItems(ImmutableList.of(item2)); + updateTimeLine( + ImmutableList.of(item1), /* mediaQueueItemIds= */ new int[] {10}, /* currentItemId= */ 10); + + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(2); + Timeline.Window window = new Timeline.Window(); + assertThat(remoteCastPlayer.getCurrentTimeline().getWindow(0, window).mediaItem) + .isEqualTo(item1); + assertThat(remoteCastPlayer.getCurrentTimeline().getWindow(1, window).mediaItem) + .isEqualTo(item2); + verify(mockRemoteMediaClient).load(loadArgumentCaptor.capture()); + assertThat(loadArgumentCaptor.getValue().getQueueData().getItems()).hasSize(1); + verify(mockRemoteMediaClient) + .queueInsertItems( + queueItemsArgumentCaptor.capture(), eq(MediaQueueItem.INVALID_ITEM_ID), eq(null)); + assertThat(queueItemsArgumentCaptor.getValue()).hasLength(1); + assertThat(queueItemsArgumentCaptor.getValue()[0].getMedia().getContentId()) + .isEqualTo(item2.localConfiguration.uri.toString()); + } + + @Test + public void addMediaItems_onEmptyTimelineFollowedBySeekTo_callsLoadThenSeek() { + when(mockRemoteMediaClient.getMediaStatus()).thenReturn(null); + MediaItem item1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem item2 = createMediaItem(/* mediaQueueItemId= */ 2); + + remoteCastPlayer.addMediaItems(ImmutableList.of(item1, item2)); + remoteCastPlayer.seekTo(/* mediaItemIndex= */ 1, /* positionMs= */ 5000L); + updateTimeLine( + ImmutableList.of(item1, item2), + /* mediaQueueItemIds= */ new int[] {10, 20}, + /* currentItemId= */ 10); + + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(2); + assertThat(remoteCastPlayer.getCurrentMediaItemIndex()).isEqualTo(1); + assertThat(remoteCastPlayer.getCurrentPosition()).isEqualTo(5000L); + verify(mockRemoteMediaClient).load(loadArgumentCaptor.capture()); + assertThat(loadArgumentCaptor.getValue().getQueueData().getItems()).hasSize(2); + verify(mockRemoteMediaClient).queueJumpToItem(eq(20), eq(5000L), eq(null)); + } + + @Test + public void addMediaItems_afterClearMediaItems_callsLoadOnRemoteMediaClient() { + MediaItem item1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem item2 = createMediaItem(/* mediaQueueItemId= */ 2); + updateTimeLine( + ImmutableList.of(item1), /* mediaQueueItemIds= */ new int[] {10}, /* currentItemId= */ 10); + + remoteCastPlayer.clearMediaItems(); + remoteCastPlayer.addMediaItems(ImmutableList.of(item2)); + updateTimeLine( + ImmutableList.of(), + /* mediaQueueItemIds= */ new int[0], + /* currentItemId= */ MediaQueueItem.INVALID_ITEM_ID); + + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(1); + Timeline.Window window = new Timeline.Window(); + assertThat(remoteCastPlayer.getCurrentTimeline().getWindow(0, window).mediaItem) + .isEqualTo(item2); + verify(mockRemoteMediaClient).queueRemoveItems(eq(new int[] {10}), eq(null)); + verify(mockRemoteMediaClient).load(loadArgumentCaptor.capture()); + assertThat(loadArgumentCaptor.getValue().getQueueData().getItems()).hasSize(1); + assertThat( + loadArgumentCaptor + .getValue() + .getQueueData() + .getItems() + .get(0) + .getMedia() + .getContentId()) + .isEqualTo(item2.localConfiguration.uri.toString()); + } + + @Test + public void addMediaItems_whenReceiverClearedWhileQueued_callsLoadWithCurrentPlayerState() { + when(mockRemoteMediaClient.pause()).thenReturn(mockPendingResult); + when(mockRemoteMediaClient.queueSetRepeatMode(anyInt(), eq(null))) + .thenReturn(mockPendingResult); + MediaItem item1 = createMediaItem(/* mediaQueueItemId= */ 1); + MediaItem item2 = createMediaItem(/* mediaQueueItemId= */ 2); + MediaItem item3 = createMediaItem(/* mediaQueueItemId= */ 3); + updateTimeLine( + ImmutableList.of(item1, item2), + /* mediaQueueItemIds= */ new int[] {10, 20}, + /* currentItemId= */ 10); + + remoteCastPlayer.removeMediaItem(/* index= */ 0); + remoteCastPlayer.addMediaItems(/* index= */ 0, ImmutableList.of(item3)); + remoteCastPlayer.setPlayWhenReady(false); + remoteCastPlayer.setRepeatMode(Player.REPEAT_MODE_ALL); + updateTimeLine( + ImmutableList.of(), + /* mediaQueueItemIds= */ new int[0], + /* currentItemId= */ MediaQueueItem.INVALID_ITEM_ID); + + assertThat(remoteCastPlayer.getCurrentTimeline().getWindowCount()).isEqualTo(1); + Timeline.Window window = new Timeline.Window(); + assertThat(remoteCastPlayer.getCurrentTimeline().getWindow(0, window).mediaItem) + .isEqualTo(item3); + verify(mockRemoteMediaClient).load(loadArgumentCaptor.capture()); + assertThat(loadArgumentCaptor.getValue().getAutoplay()).isFalse(); + assertThat(loadArgumentCaptor.getValue().getQueueData().getRepeatMode()) + .isEqualTo(MediaStatus.REPEAT_MODE_REPEAT_ALL); + assertThat(loadArgumentCaptor.getValue().getQueueData().getItems()).hasSize(1); + assertThat( + loadArgumentCaptor + .getValue() + .getQueueData() + .getItems() + .get(0) + .getMedia() + .getContentId()) + .isEqualTo(item3.localConfiguration.uri.toString()); + } + + private int[] createMediaQueueItemIds(int numberOfIds) { + int[] mediaQueueItemIds = new int[numberOfIds]; + for (int i = 0; i < numberOfIds; i++) { + mediaQueueItemIds[i] = i + 1; + } + return mediaQueueItemIds; + } + + private List createMediaItems(int[] mediaQueueItemIds) { + List mediaItems = new ArrayList<>(); + for (int mediaQueueItemId : mediaQueueItemIds) { + mediaItems.add(createMediaItem(mediaQueueItemId)); + } + return mediaItems; + } + + private MediaItem createMediaItem(int mediaQueueItemId) { + return new MediaItem.Builder() + .setUri("http://www.google.com/video" + mediaQueueItemId) + .setMediaMetadata( + new MediaMetadata.Builder().setArtist("Foo Bar - " + mediaQueueItemId).build()) + .setMimeType(MimeTypes.APPLICATION_MPD) + .build(); + } + + private void addMediaItemsAndUpdateTimeline(List mediaItems, int[] mediaQueueItemIds) { + checkState(mediaItems.size() == mediaQueueItemIds.length); + remoteCastPlayer.addMediaItems(mediaItems); + updateTimeLine(mediaItems, mediaQueueItemIds, /* currentItemId= */ 1); + } + + private void updateTimeLine( + List mediaItems, int[] mediaQueueItemIds, int currentItemId) { + int[] streamTypes = new int[mediaItems.size()]; + Arrays.fill(streamTypes, MediaInfo.STREAM_TYPE_BUFFERED); + long[] durationsMs = new long[mediaItems.size()]; + updateTimeLine( + mediaItems, + mediaQueueItemIds, + currentItemId, + streamTypes, + durationsMs, + /* positionMs= */ C.TIME_UNSET, + /* notifyStatusUpdate= */ true); + } + + private void updateTimeLine( + List mediaItems, + int[] mediaQueueItemIds, + int currentItemId, + int[] streamTypes, + long[] durationsMs, + long positionMs, + boolean notifyStatusUpdate) { + // Set up mocks to allow the player to update the timeline. + when(mockRemoteMediaClient.getCurrentItem()).thenReturn(null); + Timeline existingTimeline = remoteCastPlayer.getCurrentTimeline(); + Timeline.Window existingWindow = new Timeline.Window(); + HashSet matchedUids = new HashSet<>(); + List queueItems = new ArrayList<>(); + for (int i = 0; i < mediaQueueItemIds.length; i++) { + MediaItem mediaItem = mediaItems.get(i); + int mediaQueueItemId = mediaQueueItemIds[i]; + int streamType = streamTypes[i]; + long durationMs = durationsMs[i]; + String contentId = + mediaItem.mediaId.equals(MediaItem.DEFAULT_MEDIA_ID) + ? mediaItem.localConfiguration.uri.toString() + : mediaItem.mediaId; + MediaInfo.Builder mediaInfoBuilder = + new MediaInfo.Builder(contentId) + .setStreamType(streamType) + .setContentType(mediaItem.localConfiguration.mimeType); + if (durationMs != C.TIME_UNSET) { + mediaInfoBuilder.setStreamDuration(durationMs); + } + boolean attachedCustomData = false; + for (int s = sentQueueItems.size() - 1; s >= 0; s--) { + MediaInfo sentMedia = sentQueueItems.get(s).getMedia(); + if (sentMedia != null + && contentId.equals(sentMedia.getContentId()) + && sentMedia.getCustomData() != null + && matchedUids.add( + sentMedia.getCustomData().optString(CastTimelineTracker.KEY_SYNTHETIC_ID))) { + mediaInfoBuilder.setCustomData(sentMedia.getCustomData()); + attachedCustomData = true; + break; + } + } + if (!attachedCustomData) { + for (int w = 0; w < existingTimeline.getWindowCount(); w++) { + existingTimeline.getWindow(w, existingWindow); + if (existingWindow.mediaItem.equals(mediaItem) + && matchedUids.add(existingWindow.uid.toString())) { + try { + mediaInfoBuilder.setCustomData( + new JSONObject() + .put(CastTimelineTracker.KEY_SYNTHETIC_ID, existingWindow.uid.toString())); + } catch (JSONException e) { + throw new IllegalStateException(e); + } + break; + } + } + } + MediaInfo mediaInfo = mediaInfoBuilder.build(); + MediaQueueItem mediaQueueItem = mock(MediaQueueItem.class); + when(mediaQueueItem.getItemId()).thenReturn(mediaQueueItemId); + when(mediaQueueItem.getMedia()).thenReturn(mediaInfo); + when(mockMediaStatus.getItemById(mediaQueueItemId)).thenReturn(mediaQueueItem); + queueItems.add(mediaQueueItem); + if (mediaQueueItemId == currentItemId) { + when(mockRemoteMediaClient.getCurrentItem()).thenReturn(mediaQueueItem); + when(mockMediaStatus.getMediaInfo()).thenReturn(mediaInfo); + } + } + if (positionMs != C.TIME_UNSET) { + when(mockRemoteMediaClient.getApproximateStreamPosition()).thenReturn(positionMs); + } + if (mediaQueueItemIds.length > 0) { + when(mockRemoteMediaClient.getMediaStatus()).thenReturn(mockMediaStatus); + } + when(mockMediaQueue.getItemIds()).thenReturn(mediaQueueItemIds); + when(mockMediaStatus.getQueueItems()).thenReturn(queueItems); + when(mockMediaStatus.getCurrentItemId()) + .thenReturn(currentItemId == C.INDEX_UNSET ? 0 : currentItemId); + + if (notifyStatusUpdate) { + ImmutableList> callbacks = + ImmutableList.copyOf(pendingResultCallbacks); + pendingResultCallbacks.clear(); + for (ResultCallback callback : callbacks) { + callback.onResult(defaultSuccessResult); + } // Call listener to update the timeline of the player. remoteMediaClientCallback.onStatusUpdated(); } diff --git a/libraries/common/src/androidTest/AndroidManifest.xml b/libraries/common/src/androidTest/AndroidManifest.xml index 091e33b3eea..ea2b300e5b0 100644 --- a/libraries/common/src/androidTest/AndroidManifest.xml +++ b/libraries/common/src/androidTest/AndroidManifest.xml @@ -22,6 +22,8 @@ android:name="androidx.test.runner.AndroidJUnitRunner"/> + diff --git a/libraries/common/src/androidTest/java/androidx/media3/common/util/GlUtilAndroidTest.java b/libraries/common/src/androidTest/java/androidx/media3/common/util/GlUtilAndroidTest.java new file mode 100644 index 00000000000..6bb62bb31b2 --- /dev/null +++ b/libraries/common/src/androidTest/java/androidx/media3/common/util/GlUtilAndroidTest.java @@ -0,0 +1,75 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.common.util; + +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; +import static org.junit.Assume.assumeTrue; + +import android.opengl.EGL14; +import android.opengl.EGLContext; +import android.opengl.EGLDisplay; +import android.opengl.EGLSurface; +import androidx.media3.common.video.SyncFenceWrapper; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import androidx.test.filters.SdkSuppress; +import com.google.common.collect.ImmutableList; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Instrumentation tests for {@link GlUtil}. */ +@RunWith(AndroidJUnit4.class) +public final class GlUtilAndroidTest { + + private EGLDisplay eglDisplay; + private EGLContext eglContext; + private EGLSurface placeholderSurface; + + @Before + public void setUp() throws Exception { + eglDisplay = GlUtil.getDefaultEglDisplay(); + eglContext = GlUtil.createEglContext(eglDisplay); + placeholderSurface = GlUtil.createFocusedPlaceholderEglSurface(eglContext, eglDisplay); + } + + @After + public void tearDown() throws Exception { + GlUtil.destroyEglSurface(eglDisplay, placeholderSurface); + GlUtil.destroyEglContext(eglDisplay, eglContext); + } + + @Test + public void createSyncFences_nonPositiveCount_throwsIllegalArgumentException() throws Exception { + assertThrows(IllegalArgumentException.class, () -> GlUtil.createSyncFences(/* count= */ 0)); + assertThrows(IllegalArgumentException.class, () -> GlUtil.createSyncFences(/* count= */ -1)); + } + + @Test + @SdkSuppress(minSdkVersion = 33) + public void createSyncFences_onApi33OrHigher_returnsFencesIfSupported() throws Exception { + String extensions = EGL14.eglQueryString(eglDisplay, EGL14.EGL_EXTENSIONS); + assumeTrue(extensions != null && extensions.contains("EGL_ANDROID_native_fence_sync")); + + ImmutableList syncFences = GlUtil.createSyncFences(/* count= */ 2); + + assertThat(syncFences).hasSize(2); + for (SyncFenceWrapper syncFence : syncFences) { + assertThat(syncFence.asSyncFence().isValid()).isTrue(); + } + } +} diff --git a/libraries/common/src/androidTest/java/androidx/media3/common/video/SurfaceHolderFrameWriterAndroidTest.java b/libraries/common/src/androidTest/java/androidx/media3/common/video/SurfaceHolderFrameWriterAndroidTest.java index dbdfa0eac4e..eb1269de30f 100644 --- a/libraries/common/src/androidTest/java/androidx/media3/common/video/SurfaceHolderFrameWriterAndroidTest.java +++ b/libraries/common/src/androidTest/java/androidx/media3/common/video/SurfaceHolderFrameWriterAndroidTest.java @@ -15,10 +15,13 @@ */ package androidx.media3.common.video; +import static com.google.common.base.Preconditions.checkNotNull; +import static com.google.common.base.Preconditions.checkState; import static com.google.common.truth.Truth.assertThat; import static com.google.common.util.concurrent.MoreExecutors.directExecutor; import static java.util.concurrent.TimeUnit.MILLISECONDS; +import android.graphics.ImageFormat; import android.graphics.PixelFormat; import android.hardware.HardwareBuffer; import android.media.Image; @@ -36,6 +39,7 @@ import androidx.media3.test.utils.ImageReaderSurfaceHolder; import androidx.test.ext.junit.runners.AndroidJUnit4; import androidx.test.filters.SdkSuppress; +import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableMap; import java.util.concurrent.CountDownLatch; import java.util.concurrent.ExecutorService; @@ -48,7 +52,6 @@ // TODO: b/507446982 - Fix blocking dequeueInputFrame call and enable on API 28. /** Instrumentation tests for {@link SurfaceHolderFrameWriter}. */ -@SdkSuppress(minSdkVersion = 29) @RunWith(AndroidJUnit4.class) public final class SurfaceHolderFrameWriterAndroidTest { @@ -126,6 +129,22 @@ public void configureThenDequeue_notifiesWakeupListener() throws Exception { assertThat(wakeupLatch.await(TEST_TIMEOUT_MS, MILLISECONDS)).isTrue(); asyncFrame = asyncFrameRef.get(); assertThat(asyncFrame).isNotNull(); + } + + @Test + @SdkSuppress(minSdkVersion = 28) + public void dequeueInputFrame_onApi28AndAbove_returnsHardwareBufferMatchingFormatAndDimensions() + throws Exception { + Format format = + new Format.Builder() + .setWidth(WIDTH) + .setHeight(HEIGHT) + .setColorInfo(ColorInfo.SDR_BT709_LIMITED) + .build(); + frameWriter.configure(format, /* usage= */ Frame.USAGE_GPU_SAMPLED_IMAGE); + + AsyncFrame asyncFrame = dequeueInputFrameWithTimeout(frameWriter); + HardwareBufferFrame frame = (HardwareBufferFrame) asyncFrame.frame; assertThat(frame.getFormat()).isEqualTo(format); assertThat(frame.getHardwareBuffer().getHeight()).isEqualTo(HEIGHT); @@ -133,6 +152,38 @@ public void configureThenDequeue_notifiesWakeupListener() throws Exception { assertThat(frame.getHardwareBuffer().getFormat()).isEqualTo(HardwareBuffer.RGBA_8888); } + @Test + @SdkSuppress(maxSdkVersion = 27) + public void dequeueInputFrame_belowApi28_returnsImagePlanesMatchingFormatAndDimensions() + throws Exception { + Format format = + new Format.Builder() + .setWidth(WIDTH) + .setHeight(HEIGHT) + .setColorInfo(ColorInfo.SDR_BT709_LIMITED) + .build(); + frameWriter.configure(format, /* usage= */ 0); + + AsyncFrame asyncFrame = dequeueInputFrameWithTimeout(frameWriter); + + ImagePlanesFrame frame = (ImagePlanesFrame) asyncFrame.frame; + assertThat(frame.getFormat()).isEqualTo(format); + ImmutableList planes = frame.getPlanes(); + assertThat(planes).hasSize(3); + + ImagePlanesFrame.Plane yPlane = planes.get(0); + assertThat(yPlane.getBuffer()).isNotNull(); + assertThat(yPlane.getRowStride()).isAtLeast(WIDTH); + + ImagePlanesFrame.Plane uPlane = planes.get(1); + assertThat(uPlane.getBuffer()).isNotNull(); + assertThat(uPlane.getRowStride()).isAtLeast((WIDTH / 2) * uPlane.getPixelStride()); + + ImagePlanesFrame.Plane vPlane = planes.get(2); + assertThat(vPlane.getBuffer()).isNotNull(); + assertThat(vPlane.getRowStride()).isAtLeast((WIDTH / 2) * vPlane.getPixelStride()); + } + @Test public void dequeueThenQueue_notifiesListener() throws Exception { long contentTimeUs = 100_000L; @@ -163,14 +214,9 @@ public void dequeueThenQueue_notifiesListener() throws Exception { allowSurfaceHolderExecution.open(); assertThat(wakeupLatch.await(TEST_TIMEOUT_MS, MILLISECONDS)).isTrue(); - HardwareBufferFrame frame = (HardwareBufferFrame) asyncFrameRef.get().frame; + Frame frame = asyncFrameRef.get().frame; + Frame outputFrame = createOutputFrame(frame, displayTimeNs, contentTimeUs); - HardwareBufferFrame outputFrame = - frame - .buildUpon() - .setMetadata(ImmutableMap.of(Frame.KEY_DISPLAY_TIME_NS, displayTimeNs)) - .setContentTimeUs(contentTimeUs) - .build(); frameWriter.queueInputFrame(outputFrame, /* writeCompleteFence= */ null); assertThat(listener.renderedLatch.await(TEST_TIMEOUT_MS, MILLISECONDS)).isTrue(); @@ -180,6 +226,7 @@ public void dequeueThenQueue_notifiesListener() throws Exception { } @Test + @SdkSuppress(minSdkVersion = 29) // TODO(b/507446982): Enable on all APIs when issue is fixed. public void dequeueThenQueue_multipleFrames_notifiesListener() throws Exception { long contentTimeUs = 100_000L; long displayTimeNs = Clock.DEFAULT.nanoTime(); @@ -209,14 +256,9 @@ public void dequeueThenQueue_multipleFrames_notifiesListener() throws Exception allowSurfaceHolderExecution.open(); assertThat(wakeupLatch.await(TEST_TIMEOUT_MS, MILLISECONDS)).isTrue(); - HardwareBufferFrame frame = (HardwareBufferFrame) asyncFrameRef.get().frame; + Frame frame = asyncFrameRef.get().frame; + Frame outputFrame = createOutputFrame(frame, displayTimeNs, contentTimeUs); - HardwareBufferFrame outputFrame = - frame - .buildUpon() - .setMetadata(ImmutableMap.of(Frame.KEY_DISPLAY_TIME_NS, displayTimeNs)) - .setContentTimeUs(contentTimeUs) - .build(); frameWriter.queueInputFrame(outputFrame, /* writeCompleteFence= */ null); assertThat(listener.renderedLatch.await(TEST_TIMEOUT_MS, MILLISECONDS)).isTrue(); @@ -238,14 +280,8 @@ public void dequeueThenQueue_multipleFrames_notifiesListener() throws Exception }); assertThat(asyncFrame).isNotNull(); - frame = (HardwareBufferFrame) asyncFrame.frame; - - outputFrame = - frame - .buildUpon() - .setMetadata(ImmutableMap.of(Frame.KEY_DISPLAY_TIME_NS, displayTimeNs)) - .setContentTimeUs(contentTimeUs) - .build(); + frame = asyncFrame.frame; + outputFrame = createOutputFrame(frame, displayTimeNs, contentTimeUs); frameWriter.queueInputFrame(outputFrame, /* writeCompleteFence= */ null); assertThat(listener.renderedLatch.await(TEST_TIMEOUT_MS, MILLISECONDS)).isTrue(); @@ -258,6 +294,7 @@ public void dequeueThenQueue_multipleFrames_notifiesListener() throws Exception } @Test + @SdkSuppress(minSdkVersion = 29) // TODO(b/507446982): Enable on all APIs when issue is fixed. public void dequeueThenConfigure_returnsNewFormatOnNextDequeue() throws Exception { AtomicReference asyncFrameRef = new AtomicReference<>(); AtomicReference wakeupLatchRef = new AtomicReference<>(new CountDownLatch(1)); @@ -291,7 +328,7 @@ public void dequeueThenConfigure_returnsNewFormatOnNextDequeue() throws Exceptio allowSurfaceHolderExecution.open(); assertThat(wakeupLatchRef.get().await(TEST_TIMEOUT_MS, MILLISECONDS)).isTrue(); - HardwareBufferFrame frame1 = (HardwareBufferFrame) asyncFrameRef.get().frame; + Frame frame1 = asyncFrameRef.get().frame; assertThat(frame1.getFormat()).isEqualTo(format1); frameWriter.queueInputFrame(frame1, /* writeCompleteFence= */ null); @@ -325,7 +362,7 @@ public void dequeueThenConfigure_returnsNewFormatOnNextDequeue() throws Exceptio allowSurfaceHolderExecution.open(); assertThat(wakeupLatchRef.get().await(TEST_TIMEOUT_MS, MILLISECONDS)).isTrue(); - HardwareBufferFrame frame2 = (HardwareBufferFrame) asyncFrameRef.get().frame; + Frame frame2 = asyncFrameRef.get().frame; assertThat(frame2.getFormat()).isEqualTo(format2); frameWriter.queueInputFrame(frame2, /* writeCompleteFence= */ null); @@ -375,10 +412,8 @@ public void queueInputFrame_whenNativeCopyFails_reportsError() throws Exception } @Test - public void configure_propagatesPixelFormat() throws Exception { - AtomicReference asyncFrameRef = new AtomicReference<>(); - CountDownLatch wakeupLatch = new CountDownLatch(1); - ConditionVariable allowSurfaceHolderExecution = new ConditionVariable(); + @SdkSuppress(minSdkVersion = 28) + public void configure_onApi28AndAbove_propagatesPixelFormat() throws Exception { Format format = new Format.Builder() .setWidth(WIDTH) @@ -386,68 +421,80 @@ public void configure_propagatesPixelFormat() throws Exception { .setPixelFormat(HardwareBuffer.RGB_565) .setColorInfo(ColorInfo.SDR_BT709_LIMITED) .build(); - Future unused = - surfaceHolderExecutor.submit(() -> allowSurfaceHolderExecution.block(TEST_TIMEOUT_MS)); frameWriter.configure(format, /* usage= */ Frame.USAGE_GPU_SAMPLED_IMAGE); - AsyncFrame asyncFrame = - frameWriter.dequeueInputFrame( - /* wakeupExecutor= */ directExecutor(), - /* wakeupListener= */ () -> { - asyncFrameRef.set(frameWriter.dequeueInputFrame(directExecutor(), () -> {})); - wakeupLatch.countDown(); - }); - - assertThat(asyncFrame).isNull(); + AsyncFrame asyncFrame = dequeueInputFrameWithTimeout(frameWriter); - allowSurfaceHolderExecution.open(); - - assertThat(wakeupLatch.await(TEST_TIMEOUT_MS, MILLISECONDS)).isTrue(); - asyncFrame = asyncFrameRef.get(); - assertThat(asyncFrame).isNotNull(); HardwareBufferFrame frame = (HardwareBufferFrame) asyncFrame.frame; assertThat(frame.getFormat()).isEqualTo(format); assertThat(frame.getHardwareBuffer().getFormat()).isEqualTo(HardwareBuffer.RGB_565); } @Test - public void configure_withoutPixelFormat_propagatesDefaultSdrPixelFormat() throws Exception { - AtomicReference asyncFrameRef = new AtomicReference<>(); - CountDownLatch wakeupLatch = new CountDownLatch(1); - ConditionVariable allowSurfaceHolderExecution = new ConditionVariable(); + @SdkSuppress(maxSdkVersion = 27) + public void configure_belowApi28_propagatesPixelFormat() throws Exception { Format format = new Format.Builder() .setWidth(WIDTH) .setHeight(HEIGHT) + .setPixelFormat(ImageFormat.YV12) .setColorInfo(ColorInfo.SDR_BT709_LIMITED) .build(); - Future unused = - surfaceHolderExecutor.submit(() -> allowSurfaceHolderExecution.block(TEST_TIMEOUT_MS)); - frameWriter.configure(format, /* usage= */ Frame.USAGE_GPU_SAMPLED_IMAGE); - AsyncFrame asyncFrame = - frameWriter.dequeueInputFrame( - /* wakeupExecutor= */ directExecutor(), - /* wakeupListener= */ () -> { - asyncFrameRef.set(frameWriter.dequeueInputFrame(directExecutor(), () -> {})); - wakeupLatch.countDown(); - }); + frameWriter.configure(format, /* usage= */ 0); + AsyncFrame asyncFrame = dequeueInputFrameWithTimeout(frameWriter); - assertThat(asyncFrame).isNull(); + ImagePlanesFrame frame = (ImagePlanesFrame) asyncFrame.frame; + assertThat(frame.getFormat()).isEqualTo(format); + assertThat(checkNotNull(surfaceHolder.imageReader).getImageFormat()) + .isEqualTo(ImageFormat.YV12); + } - allowSurfaceHolderExecution.open(); + @Test + @SdkSuppress(minSdkVersion = 28) + public void + configure_withSdrFormatWithoutPixelFormatOnApi28AndAbove_propagatesDefaultSdrPixelFormat() + throws Exception { + Format format = + new Format.Builder() + .setWidth(WIDTH) + .setHeight(HEIGHT) + .setColorInfo(ColorInfo.SDR_BT709_LIMITED) + .build(); + frameWriter.configure(format, /* usage= */ Frame.USAGE_GPU_SAMPLED_IMAGE); - assertThat(wakeupLatch.await(TEST_TIMEOUT_MS, MILLISECONDS)).isTrue(); - asyncFrame = asyncFrameRef.get(); - assertThat(asyncFrame).isNotNull(); + AsyncFrame asyncFrame = dequeueInputFrameWithTimeout(frameWriter); + + assertThat(asyncFrame.frame.getFormat().pixelFormat).isEqualTo(Format.NO_VALUE); + assertThat(asyncFrame.frame).isInstanceOf(HardwareBufferFrame.class); HardwareBufferFrame frame = (HardwareBufferFrame) asyncFrame.frame; - assertThat(frame.getFormat().pixelFormat).isEqualTo(Format.NO_VALUE); assertThat(frame.getHardwareBuffer().getFormat()).isEqualTo(HardwareBuffer.RGBA_8888); } + @Test + @SdkSuppress(maxSdkVersion = 27) + public void configure_withSdrFormatWithoutPixelFormatBelowApi28_propagatesDefaultSdrPixelFormat() + throws Exception { + Format format = + new Format.Builder() + .setWidth(WIDTH) + .setHeight(HEIGHT) + .setColorInfo(ColorInfo.SDR_BT709_LIMITED) + .build(); + frameWriter.configure(format, /* usage= */ Frame.USAGE_GPU_SAMPLED_IMAGE); + + AsyncFrame asyncFrame = dequeueInputFrameWithTimeout(frameWriter); + + assertThat(asyncFrame.frame.getFormat().pixelFormat).isEqualTo(Format.NO_VALUE); + assertThat(asyncFrame.frame).isInstanceOf(ImagePlanesFrame.class); + assertThat(checkNotNull(surfaceHolder.imageReader).getImageFormat()) + .isEqualTo(ImageFormat.YV12); + } + @Test @SdkSuppress(minSdkVersion = 34) - public void configure_withoutPixelFormat_propagatesDefaultHdrPixelFormat() throws Exception { + public void configure_withHdrFormatWithoutPixelFormat_propagatesDefaultHdrPixelFormat() + throws Exception { AtomicReference asyncFrameRef = new AtomicReference<>(); CountDownLatch wakeupLatch = new CountDownLatch(1); ConditionVariable allowSurfaceHolderExecution = new ConditionVariable(); @@ -536,6 +583,43 @@ public void surfaceChanged_afterQueueInputFrame_rendersLastFrame() throws Except } } + @SuppressWarnings("PatternMatchingInstanceof") // TODO(b/561916275): Remove when issue is fixed. + private static Frame createOutputFrame(Frame frame, long displayTimeNs, long contentTimeUs) { + if (frame instanceof HardwareBufferFrame) { + return ((HardwareBufferFrame) frame) + .buildUpon() + .setMetadata(ImmutableMap.of(Frame.KEY_DISPLAY_TIME_NS, displayTimeNs)) + .setContentTimeUs(contentTimeUs) + .build(); + } else if (frame instanceof ImagePlanesFrame) { + return ((ImagePlanesFrame) frame) + .buildUpon() + .setMetadata(ImmutableMap.of(Frame.KEY_DISPLAY_TIME_NS, displayTimeNs)) + .setContentTimeUs(contentTimeUs) + .build(); + } + throw new IllegalArgumentException("Unsupported frame type: " + frame.getClass()); + } + + private AsyncFrame dequeueInputFrameWithTimeout(SurfaceHolderFrameWriter frameWriter) + throws InterruptedException { + AtomicReference asyncFrameRef = new AtomicReference<>(); + CountDownLatch wakeupLatch = new CountDownLatch(1); + AsyncFrame asyncFrame = + frameWriter.dequeueInputFrame( + /* wakeupExecutor= */ directExecutor(), + /* wakeupListener= */ () -> { + asyncFrameRef.set(frameWriter.dequeueInputFrame(directExecutor(), () -> {})); + wakeupLatch.countDown(); + }); + if (asyncFrame != null) { + return asyncFrame; + } + checkState( + wakeupLatch.await(TEST_TIMEOUT_MS, MILLISECONDS), "Timed out waiting for input frame."); + return checkNotNull(asyncFrameRef.get()); + } + private void waitHandlerIdle(Handler handler) throws InterruptedException { CountDownLatch latch = new CountDownLatch(1); handler.post(latch::countDown); diff --git a/libraries/common/src/androidTest/java/androidx/media3/common/video/SurfaceHolderFrameWriterSurfaceViewAndroidTest.java b/libraries/common/src/androidTest/java/androidx/media3/common/video/SurfaceHolderFrameWriterSurfaceViewAndroidTest.java new file mode 100644 index 00000000000..cb747f55c52 --- /dev/null +++ b/libraries/common/src/androidTest/java/androidx/media3/common/video/SurfaceHolderFrameWriterSurfaceViewAndroidTest.java @@ -0,0 +1,160 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.common.video; + +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.util.concurrent.MoreExecutors.directExecutor; +import static java.util.concurrent.TimeUnit.MILLISECONDS; + +import android.app.Activity; +import android.view.SurfaceHolder; +import android.view.SurfaceView; +import android.view.ViewGroup; +import androidx.annotation.Nullable; +import androidx.media3.common.ColorInfo; +import androidx.media3.common.Format; +import androidx.media3.common.VideoFrameProcessingException; +import androidx.media3.common.util.Size; +import androidx.media3.test.utils.FakeHardwareBufferNativeHelpers; +import androidx.test.ext.junit.rules.ActivityScenarioRule; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.util.concurrent.BlockingQueue; +import java.util.concurrent.LinkedBlockingQueue; +import org.junit.After; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** + * Instrumentation tests for {@link SurfaceHolderFrameWriter} attached to a real {@link + * SurfaceView}. + */ +@RunWith(AndroidJUnit4.class) +public final class SurfaceHolderFrameWriterSurfaceViewAndroidTest { + + private static final long TEST_TIMEOUT_MS = 10_000L; + private static final int WIDTH = 640; + private static final int HEIGHT = 480; + private static final int SECOND_WIDTH = 320; + private static final int SECOND_HEIGHT = 240; + + @Rule + public final ActivityScenarioRule activityRule = + new ActivityScenarioRule<>(Activity.class); + + private BlockingQueue surfaceSizes; + private SurfaceView surfaceView; + private SurfaceHolderFrameWriter frameWriter; + private FakeListener listener; + + @Before + public void setUp() throws Exception { + surfaceSizes = new LinkedBlockingQueue<>(); + activityRule + .getScenario() + .onActivity( + activity -> { + surfaceView = new SurfaceView(activity); + surfaceView + .getHolder() + .addCallback( + new SurfaceHolder.Callback() { + @Override + public void surfaceCreated(SurfaceHolder holder) {} + + @Override + public void surfaceChanged( + SurfaceHolder holder, int format, int width, int height) { + surfaceSizes.add(new Size(width, height)); + } + + @Override + public void surfaceDestroyed(SurfaceHolder holder) {} + }); + activity.setContentView(surfaceView, new ViewGroup.LayoutParams(100, 100)); + }); + assertThat(surfaceSizes.poll(TEST_TIMEOUT_MS, MILLISECONDS)).isEqualTo(new Size(100, 100)); + listener = new FakeListener(); + frameWriter = + SurfaceHolderFrameWriter.create( + surfaceView.getHolder(), + surfaceView::post, + listener, + directExecutor(), + new FakeHardwareBufferNativeHelpers()); + } + + @After + public void tearDown() { + if (frameWriter != null) { + frameWriter.close(); + } + activityRule.getScenario().close(); + assertThat(listener.lastException).isNull(); + } + + @Test + public void configure_withSurfaceViewHolder_resizesSurfaceToVideoSize() throws Exception { + frameWriter.configure(createFormat(WIDTH, HEIGHT), /* usage= */ 0); + + assertResized(WIDTH, HEIGHT); + } + + @Test + public void configure_withNewFormatSize_resizesSurfaceAgain() throws Exception { + frameWriter.configure(createFormat(WIDTH, HEIGHT), /* usage= */ 0); + assertResized(WIDTH, HEIGHT); + + frameWriter.configure(createFormat(SECOND_WIDTH, SECOND_HEIGHT), /* usage= */ 0); + + assertResized(SECOND_WIDTH, SECOND_HEIGHT); + } + + private static Format createFormat(int width, int height) { + return new Format.Builder() + .setWidth(width) + .setHeight(height) + .setColorInfo(ColorInfo.SDR_BT709_LIMITED) + .build(); + } + + private void assertResized(int width, int height) throws Exception { + // SurfaceHolderFrameWriter first sets the surface size to 1x1 to force a surfaceChanged + // callback before setting the target format size. + Size size = surfaceSizes.poll(TEST_TIMEOUT_MS, MILLISECONDS); + if (new Size(1, 1).equals(size)) { + size = surfaceSizes.poll(TEST_TIMEOUT_MS, MILLISECONDS); + } + assertThat(size).isEqualTo(new Size(width, height)); + } + + private static final class FakeListener implements SurfaceHolderFrameWriter.Listener { + @Nullable volatile VideoFrameProcessingException lastException; + + @Override + public void onFrameAboutToBeRendered( + long presentationTimeUs, long releaseTimeNs, Format format) {} + + @Override + public void onError(VideoFrameProcessingException videoFrameProcessingException) { + lastException = videoFrameProcessingException; + } + + @Override + public void onEnded() {} + } +} diff --git a/libraries/common/src/main/java/androidx/media3/common/AdPlaybackState.java b/libraries/common/src/main/java/androidx/media3/common/AdPlaybackState.java index 20dc7f81756..b9637bb6b8f 100644 --- a/libraries/common/src/main/java/androidx/media3/common/AdPlaybackState.java +++ b/libraries/common/src/main/java/androidx/media3/common/AdPlaybackState.java @@ -1795,6 +1795,8 @@ public String toString() { for (int i = 0; i < adGroups.length; i++) { sb.append("adGroup(timeUs="); sb.append(adGroups[i].timeUs); + sb.append(", contentResumeOffsetUs="); + sb.append(adGroups[i].contentResumeOffsetUs); sb.append(", ads=["); for (int j = 0; j < adGroups[i].states.length; j++) { sb.append("ad(state="); diff --git a/libraries/common/src/main/java/androidx/media3/common/BasePlayer.java b/libraries/common/src/main/java/androidx/media3/common/BasePlayer.java index c760bc8ee53..2d0e9eb4b66 100644 --- a/libraries/common/src/main/java/androidx/media3/common/BasePlayer.java +++ b/libraries/common/src/main/java/androidx/media3/common/BasePlayer.java @@ -366,9 +366,8 @@ public final long getCurrentLiveOffset() { if (timeline.isEmpty()) { return C.TIME_UNSET; } - long windowStartTimeMs = - timeline.getWindow(getCurrentMediaItemIndex(), window).windowStartTimeMs; - if (windowStartTimeMs == C.TIME_UNSET) { + timeline.getWindow(getCurrentMediaItemIndex(), window); + if (window.windowStartTimeMs == C.TIME_UNSET || !window.isLive()) { return C.TIME_UNSET; } return window.getCurrentUnixTimeMs() - window.windowStartTimeMs - getContentPosition(); diff --git a/libraries/common/src/main/java/androidx/media3/common/C.java b/libraries/common/src/main/java/androidx/media3/common/C.java index afd01e9872e..5ddfd552f5f 100644 --- a/libraries/common/src/main/java/androidx/media3/common/C.java +++ b/libraries/common/src/main/java/androidx/media3/common/C.java @@ -32,6 +32,7 @@ import android.net.Uri; import android.opengl.GLES20; import android.os.IBinder; +import android.os.Process; import android.view.Surface; import androidx.annotation.IntDef; import androidx.media3.common.util.UnstableApi; @@ -175,6 +176,16 @@ private C() {} @UnstableApi public static final int AUDIO_SESSION_ID_UNSET = AudioManager.AUDIO_SESSION_ID_GENERATE; + // LINT.IfChange(thread_priority_no_op) + /** + * Represents a no-op thread priority to inherit the spawning thread's priority without overriding + * it. {@link Process#setThreadPriority} supports int values between -20 and 19, so this value + * must be outside that range. + */ + @UnstableApi public static final int THREAD_PRIORITY_NO_OP = -99; + + // LINT.ThenChange() + /** * Represents an audio encoding, or an invalid or unset value. One of {@link Format#NO_VALUE}, * {@link #ENCODING_INVALID}, {@link #ENCODING_PCM_8BIT}, {@link #ENCODING_PCM_16BIT}, {@link @@ -685,6 +696,7 @@ private C() {} *
    *
  • {@link #VIDEO_CODEC_FLAG_H264} *
  • {@link #VIDEO_CODEC_FLAG_H265} + *
  • {@link #VIDEO_CODEC_FLAG_AV1} *
*/ @UnstableApi @@ -693,12 +705,18 @@ private C() {} @Target(TYPE_USE) @IntDef( flag = true, - value = {VIDEO_CODEC_FLAG_H264, VIDEO_CODEC_FLAG_H265}) + value = {VIDEO_CODEC_FLAG_H264, VIDEO_CODEC_FLAG_H265, VIDEO_CODEC_FLAG_AV1}) public @interface VideoCodecFlags {} + /** H.264 Video Coding Standard (AVC). */ @UnstableApi public static final int VIDEO_CODEC_FLAG_H264 = 1; + + /** H.265 Video Coding Standard (HEVC). */ @UnstableApi public static final int VIDEO_CODEC_FLAG_H265 = 2; + /** AV1 Video Coding Standard. */ + @UnstableApi public static final int VIDEO_CODEC_FLAG_AV1 = 4; + /** * Flags which can apply to a buffer containing a media sample. * @@ -1264,6 +1282,8 @@ private C() {} // LINT.ThenChange( // util/MediaFormatUtil.java:color_space, // ColorInfo.java:color_space, + // ../../../../../../../effect/src/main/res/raw/color_conversions_es2.glsl:color_space, + // ../../../../../../../effect/src/main/res/raw/color_conversions_es3.glsl:color_space, // ) // LINT.IfChange(color_transfer) @@ -1319,6 +1339,8 @@ private C() {} // ../../../../../../../effect/src/main/res/raw/fragment_shader_transformation_sdr_external_es2.glsl:color_transfer, // ../../../../../../../effect/src/main/res/raw/fragment_shader_transformation_external_yuv_es3.glsl:color_transfer, // ../../../../../../../effect/src/main/res/raw/fragment_shader_oetf_es3.glsl:color_transfer, + // ../../../../../../../effect/src/main/res/raw/color_conversions_es2.glsl:color_transfer, + // ../../../../../../../effect/src/main/res/raw/color_conversions_es3.glsl:color_transfer, // ) // LINT.IfChange(color_range) diff --git a/libraries/common/src/main/java/androidx/media3/common/ColorInfo.java b/libraries/common/src/main/java/androidx/media3/common/ColorInfo.java index 7ba562d86c9..21cd4dc2ab8 100644 --- a/libraries/common/src/main/java/androidx/media3/common/ColorInfo.java +++ b/libraries/common/src/main/java/androidx/media3/common/ColorInfo.java @@ -325,6 +325,11 @@ public static boolean isTransferHdr(@Nullable ColorInfo colorInfo) { || colorInfo.colorTransfer == C.COLOR_TRANSFER_ST2084); } + /** Returns whether the {@code colorInfo} is in a wide color gamut (BT.2020). */ + public static boolean isWideColorGamut(@Nullable ColorInfo colorInfo) { + return colorInfo != null && colorInfo.colorSpace == C.COLOR_SPACE_BT2020; + } + /** The {@link C.ColorSpace}, or {@link Format#NO_VALUE} if not set. */ public final @C.ColorSpace int colorSpace; diff --git a/libraries/common/src/main/java/androidx/media3/common/DeviceInfo.java b/libraries/common/src/main/java/androidx/media3/common/DeviceInfo.java index 518a5d5528e..930938cc51e 100644 --- a/libraries/common/src/main/java/androidx/media3/common/DeviceInfo.java +++ b/libraries/common/src/main/java/androidx/media3/common/DeviceInfo.java @@ -62,6 +62,7 @@ public static final class Builder { private int minVolume; private int maxVolume; @Nullable private String routingControllerId; + @Nullable private String routingControllerName; /** * Creates the builder. @@ -120,6 +121,24 @@ public Builder setRoutingControllerId(@Nullable String routingControllerId) { return this; } + /** + * Sets the display name of the routing controller or remote device (e.g. "Living Room TV"). + * + *

The set value must be null if {@link DeviceInfo#playbackType} is {@link + * #PLAYBACK_TYPE_LOCAL}. + * + * @param routingControllerName The display name of the routing controller, or null to leave it + * unspecified. + * @return This builder. + */ + @CanIgnoreReturnValue + @UnstableApi + public Builder setRoutingControllerName(@Nullable String routingControllerName) { + checkArgument(playbackType != PLAYBACK_TYPE_LOCAL || routingControllerName == null); + this.routingControllerName = routingControllerName; + return this; + } + /** Builds the {@link DeviceInfo}. */ public DeviceInfo build() { checkArgument(minVolume <= maxVolume); @@ -148,6 +167,12 @@ public DeviceInfo build() { */ @Nullable public final String routingControllerId; + /** + * The display name of the routing controller or remote device (e.g. "Living Room TV"), or null if + * unset or {@link #playbackType} is {@link #PLAYBACK_TYPE_LOCAL}. + */ + @UnstableApi @Nullable public final String routingControllerName; + /** * @deprecated Use {@link Builder} instead. */ @@ -165,6 +190,7 @@ private DeviceInfo(Builder builder) { this.minVolume = builder.minVolume; this.maxVolume = builder.maxVolume; this.routingControllerId = builder.routingControllerId; + this.routingControllerName = builder.routingControllerName; } @Override @@ -179,7 +205,8 @@ public boolean equals(@Nullable Object obj) { return playbackType == other.playbackType && minVolume == other.minVolume && maxVolume == other.maxVolume - && Objects.equals(routingControllerId, other.routingControllerId); + && Objects.equals(routingControllerId, other.routingControllerId) + && Objects.equals(routingControllerName, other.routingControllerName); } @Override @@ -189,6 +216,7 @@ public int hashCode() { result = 31 * result + minVolume; result = 31 * result + maxVolume; result = 31 * result + (routingControllerId == null ? 0 : routingControllerId.hashCode()); + result = 31 * result + (routingControllerName == null ? 0 : routingControllerName.hashCode()); return result; } @@ -196,6 +224,7 @@ public int hashCode() { private static final String FIELD_MIN_VOLUME = Util.intToStringMaxRadix(1); private static final String FIELD_MAX_VOLUME = Util.intToStringMaxRadix(2); private static final String FIELD_ROUTING_CONTROLLER_ID = Util.intToStringMaxRadix(3); + private static final String FIELD_ROUTING_CONTROLLER_NAME = Util.intToStringMaxRadix(4); @UnstableApi public Bundle toBundle() { @@ -212,6 +241,9 @@ public Bundle toBundle() { if (routingControllerId != null) { bundle.putString(FIELD_ROUTING_CONTROLLER_ID, routingControllerId); } + if (routingControllerName != null) { + bundle.putString(FIELD_ROUTING_CONTROLLER_NAME, routingControllerName); + } return bundle; } @@ -222,10 +254,12 @@ public static DeviceInfo fromBundle(Bundle bundle) { int minVolume = bundle.getInt(FIELD_MIN_VOLUME, /* defaultValue= */ 0); int maxVolume = bundle.getInt(FIELD_MAX_VOLUME, /* defaultValue= */ 0); @Nullable String routingControllerId = bundle.getString(FIELD_ROUTING_CONTROLLER_ID); + @Nullable String routingControllerName = bundle.getString(FIELD_ROUTING_CONTROLLER_NAME); return new DeviceInfo.Builder(playbackType) .setMinVolume(minVolume) .setMaxVolume(maxVolume) .setRoutingControllerId(routingControllerId) + .setRoutingControllerName(routingControllerName) .build(); } } diff --git a/libraries/common/src/main/java/androidx/media3/common/FileTypes.java b/libraries/common/src/main/java/androidx/media3/common/FileTypes.java index 38324d2f1ec..80364c1d0bb 100644 --- a/libraries/common/src/main/java/androidx/media3/common/FileTypes.java +++ b/libraries/common/src/main/java/androidx/media3/common/FileTypes.java @@ -23,6 +23,7 @@ import androidx.annotation.Nullable; import androidx.annotation.VisibleForTesting; import androidx.media3.common.util.UnstableApi; +import com.google.common.base.Ascii; import java.lang.annotation.Documented; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; @@ -267,7 +268,9 @@ private FileTypes() {} @Nullable String filename = uri.getLastPathSegment(); if (filename == null) { return FileTypes.UNKNOWN; - } else if (filename.endsWith(EXTENSION_AC3) || filename.endsWith(EXTENSION_EC3)) { + } + filename = Ascii.toLowerCase(filename); + if (filename.endsWith(EXTENSION_AC3) || filename.endsWith(EXTENSION_EC3)) { return FileTypes.AC3; } else if (filename.endsWith(EXTENSION_AC4)) { return FileTypes.AC4; diff --git a/libraries/common/src/main/java/androidx/media3/common/FlagSet.java b/libraries/common/src/main/java/androidx/media3/common/FlagSet.java index ee5309513ef..bc26979ea4e 100644 --- a/libraries/common/src/main/java/androidx/media3/common/FlagSet.java +++ b/libraries/common/src/main/java/androidx/media3/common/FlagSet.java @@ -15,7 +15,6 @@ */ package androidx.media3.common; -import static android.os.Build.VERSION.SDK_INT; import static com.google.common.base.Preconditions.checkElementIndex; import static com.google.common.base.Preconditions.checkState; @@ -237,33 +236,11 @@ public boolean equals(@Nullable Object o) { return false; } FlagSet that = (FlagSet) o; - if (SDK_INT < 24) { - // SparseBooleanArray.equals() is not implemented on API levels below 24. - if (size() != that.size()) { - return false; - } - for (int i = 0; i < size(); i++) { - if (get(i) != that.get(i)) { - return false; - } - } - return true; - } else { - return flags.equals(that.flags); - } + return flags.equals(that.flags); } @Override public int hashCode() { - if (SDK_INT < 24) { - // SparseBooleanArray.hashCode() is not implemented on API levels below 24. - int hashCode = size(); - for (int i = 0; i < size(); i++) { - hashCode = 31 * hashCode + get(i); - } - return hashCode; - } else { - return flags.hashCode(); - } + return flags.hashCode(); } } diff --git a/libraries/common/src/main/java/androidx/media3/common/Flags.java b/libraries/common/src/main/java/androidx/media3/common/Flags.java new file mode 100644 index 00000000000..2cccc023ecc --- /dev/null +++ b/libraries/common/src/main/java/androidx/media3/common/Flags.java @@ -0,0 +1,452 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.common; + +import static com.google.common.base.Preconditions.checkArgument; +import static java.lang.annotation.ElementType.TYPE_USE; + +import android.util.SparseBooleanArray; +import androidx.annotation.IntDef; +import androidx.annotation.VisibleForTesting; +import androidx.media3.common.util.ExperimentalApi; +import androidx.media3.common.util.ParsableByteArray; +import androidx.media3.common.util.UnstableApi; +import androidx.media3.common.util.Util; +import java.lang.annotation.Documented; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Central registry for Media3 experimental feature flags. + * + *

This class provides a mechanism to control experimental features in Media3 via global canary + * mode and individual flag overrides, enabling safe, gradual rollout of new library behaviors. + * + *

Canary mode + * + *

Canary mode is enabled by default. Only flags that are considered safe for production usage + * are included in canary mode. When canary mode is disabled, these flags are inactive. + * + *

Individual overrides + * + *

Individual flags can be explicitly {@linkplain #enableFlag enabled} or {@linkplain + * #disableFlag disabled}, regardless of the canary mode state. An explicit override always takes + * precedence over the canary mode default. + * + *

Threading + * + *

Flag configuration must be done on the application thread before creating any Media3 + * components (e.g., {@code ExoPlayer}, {@code Transformer}). Flag queries from other threads (e.g., + * the playback thread) rely on happens-before relationships established by {@link Thread#start()} + * and {@link android.os.Handler} message posting during component setup. + * + *

Testing + * + *

In unit tests, {@code androidx.media3.test.utils.Media3FlagsRule} is the preferred way to + * manage flag states and automatically reset all flag overrides and canary mode after each test. + */ +@UnstableApi +public final class Flags { + + // --------------------------------------------------------------------------------------------- + // Adding a new flag: + // 1. Allocate the next flag ID from NEXT_FLAG_ID and increment NEXT_FLAG_ID by 1. + // 2. Declare the public static final int constant, annotate with @ExperimentalApi, and add a + // TODO with a tracking issue to remove it (including earliest cleanup date if in canary mode). + // 3. Add the flag to the @Flag @IntDef list and update its javadoc. + // 4. Decide on the initial lifecycle stage (see Flag Lifecycle and Evolution below). + // + // Flag Lifecycle and Evolution: + // + // Feature flags in Media3 follow a defined progression through their development lifecycle: + // + // 1. Initial Development or Experiments (Statically Disabled): + // - When a new flag is introduced for an in-development feature or for experimental + // validation, register it in STATIC_FLAG_STATES with `false`. + // - This ensures the feature is inactive by default in production and excluded from canary + // mode rollout until implementation and validation are complete. + // - The feature can be tested via explicit `Flags.enableFlag(...)` in targeted unit tests or + // experiments. + // - This step is optional and can be skipped for flags that are ready for general rollout + // immediately. + // + // 2. Canary Rollout (Dynamic Flag): + // - When ready for general rollout, remove the flag from STATIC_FLAG_STATES so it is enabled + // by default in canary mode and disabled when canary mode is turned off. + // - Keep in canary mode for at least 2 months or until the next Media3 release (whichever is + // later). Note the earliest cleanup date in the TODO (e.g. `(earliest cleanup: 2026-11-24)`) + // + // 3. Fallback Mechanism (Statically Enabled, Optional): + // - If an opt-out mechanism is temporarily desired after graduating from canary mode, + // register the flag in STATIC_FLAG_STATES with `true`. + // - The feature remains active even if canary mode is turned off, while still allowing an + // opt-out fallback via `Flags.disableFlag(...)` if needed. + // - This step is optional and many simple changes can skip it entirely. + // + // 4. Permanent Cleanup: + // - Once the feature is permanently enabled and the fallback is no longer needed, remove the + // flag constant, its @IntDef entry, its STATIC_FLAG_STATES entry, and all conditional code + // branches guarding the feature. + // - Features requiring permanent configuration should add a dedicated API rather than using + // Flags in the long term. + // --------------------------------------------------------------------------------------------- + + /** + * Flags for experimental features in Media3. + * + *

Possible flag values are: + * + *

    + *
  • {@link #FLAG_PER_STREAM_MEDIA_PROGRESSION} + *
  • {@link #FLAG_PROCESSED_STREAM_CHANGED_AT_START} + *
  • {@link #FLAG_DYNAMIC_SCHEDULING} + *
  • {@link #FLAG_ENABLE_STUCK_PLAYING_DETECTION} + *
  • {@link #FLAG_ENABLE_CLIPPING_IN_MEDIA_PERIOD} + *
  • {@link #FLAG_ENABLE_MEDIACODEC_BUFFER_DECODE_ONLY} + *
  • {@link #FLAG_VIDEO_RENDERER_DURATION_TO_PROGRESS} + *
  • {@link #FLAG_ENFORCE_PARSABLE_BYTE_ARRAY_LIMIT} + *
  • {@link #FLAG_SKIP_AUDIO_TRACK_FLUSH_BEFORE_RELEASE} + *
+ */ + @Documented + @Retention(RetentionPolicy.SOURCE) + @Target(TYPE_USE) + @IntDef( + open = true, + value = { + FLAG_PER_STREAM_MEDIA_PROGRESSION, + FLAG_PROCESSED_STREAM_CHANGED_AT_START, + FLAG_DYNAMIC_SCHEDULING, + FLAG_ENABLE_STUCK_PLAYING_DETECTION, + FLAG_ENABLE_CLIPPING_IN_MEDIA_PERIOD, + FLAG_ENABLE_MEDIACODEC_BUFFER_DECODE_ONLY, + FLAG_VIDEO_RENDERER_DURATION_TO_PROGRESS, + FLAG_ENFORCE_PARSABLE_BYTE_ARRAY_LIMIT, + FLAG_SKIP_AUDIO_TRACK_FLUSH_BEFORE_RELEASE, + }) + public @interface Flag {} + + /** + * Flag to enable per-stream media progression in ExoPlayer. + * + *

When enabled, ExoPlayer may advance processing on a per-stream basis, potentially starting + * to process the next item in the playlist before finishing the current one. This can reduce + * startup latency between media items. + * + *

This flag is not yet covered by canary mode and defaults to {@code false}. + */ + @ExperimentalApi // TODO: b/510217604 - Remove this flag. + public static final int FLAG_PER_STREAM_MEDIA_PROGRESSION = 1; + + /** + * Flag to enable invoking {@code MediaCodecRenderer.onProcessedStreamChange()} on the first + * stream. + * + *

When enabled, {@code onProcessedStreamChange()} is invoked starting from the first stream. + * When disabled (the default), it is invoked from the second stream onwards. + * + *

This flag is not yet covered by canary mode and defaults to {@code false}. + */ + @ExperimentalApi // TODO: b/470373575 - Remove this flag. + public static final int FLAG_PROCESSED_STREAM_CHANGED_AT_START = 2; + + /** + * Flag to enable dynamic scheduling in ExoPlayer. + * + *

When enabled, ExoPlayer uses dynamic scheduling for its playback loop. + * + *

This flag is statically enabled and defaults to {@code true}. + */ + @ExperimentalApi // TODO: b/500985770 - Remove this flag. + public static final int FLAG_DYNAMIC_SCHEDULING = 3; + + /** + * Flag to enable stuck playing detection in ExoPlayer. + * + *

When enabled, ExoPlayer detects when playback is stuck without making progress or without + * ending, and triggers a playback error. + */ + @ExperimentalApi // TODO: b/443074686 - Remove this flag. + public static final int FLAG_ENABLE_STUCK_PLAYING_DETECTION = 4; + + /** + * Flag to enable delegating end-position clipping to {@code MediaPeriod} instances in ExoPlayer. + * + *

When enabled, clipping of the end position is delegated to the wrapped {@code MediaPeriod} + * rather than being handled at the media source level. + * + *

This flag is not yet covered by canary mode and defaults to {@code false}. + */ + @ExperimentalApi // TODO: b/474538573 - Remove this flag. + public static final int FLAG_ENABLE_CLIPPING_IN_MEDIA_PERIOD = 5; + + /** + * Flag to enable passing {@link android.media.MediaCodec#BUFFER_FLAG_DECODE_ONLY} to {@link + * android.media.MediaCodec} on API 34+. + * + *

When enabled on API 34+, buffers before the start time in non-tunneling mode are queued with + * {@code BUFFER_FLAG_DECODE_ONLY} rather than being dropped after decoding. + * + *

This flag is not yet covered by canary mode and defaults to {@code false}. + */ + @ExperimentalApi // TODO: b/470367414 - Remove this flag. + public static final int FLAG_ENABLE_MEDIACODEC_BUFFER_DECODE_ONLY = 6; + + /** + * Flag to enable calculating duration-to-progress based on next output buffer presentation time + * in {@code MediaCodecVideoRenderer}. + * + *

When enabled along with {@link #FLAG_DYNAMIC_SCHEDULING}, {@code MediaCodecVideoRenderer} + * returns duration-to-progress aligned with when its next output buffer can be rendered when in + * asynchronous mode. + * + *

This flag is not yet covered by canary mode and defaults to {@code false}. + */ + @ExperimentalApi // TODO: b/369523131 - Remove this flag. + public static final int FLAG_VIDEO_RENDERER_DURATION_TO_PROGRESS = 7; + + /** + * Flag to enable enforcing that read and peek methods in {@link ParsableByteArray} do not read + * beyond {@link ParsableByteArray#limit()}. + * + *

When enabled, reading or peeking beyond the limit throws an {@link + * IndexOutOfBoundsException} (or {@link java.nio.BufferOverflowException}) even if the read is + * within the bounds of the underlying byte array. + */ + @ExperimentalApi // TODO: b/147657250 - Remove this flag (earliest cleanup: 2026-12-01). + public static final int FLAG_ENFORCE_PARSABLE_BYTE_ARRAY_LIMIT = 8; + + /** + * Flag to skip calling {@link android.media.AudioTrack#flush()} before {@link + * android.media.AudioTrack#release()} on API 29+. + * + *

When enabled on API 29+, {@code AudioTrackAudioOutput} releases the underlying {@link + * android.media.AudioTrack} without calling {@code flush()} first. + */ + @ExperimentalApi // TODO: b/567523724 - Remove this flag (earliest cleanup: 2026-11-29). + public static final int FLAG_SKIP_AUDIO_TRACK_FLUSH_BEFORE_RELEASE = 9; + + @VisibleForTesting /* package */ static final int NEXT_FLAG_ID = 10; + + private static final SparseBooleanArray STATIC_FLAG_STATES = new SparseBooleanArray(); + + static { + // Statically disabled flags (not ready for general rollout). + STATIC_FLAG_STATES.put(FLAG_PER_STREAM_MEDIA_PROGRESSION, false); + STATIC_FLAG_STATES.put(FLAG_PROCESSED_STREAM_CHANGED_AT_START, false); + STATIC_FLAG_STATES.put(FLAG_ENABLE_CLIPPING_IN_MEDIA_PERIOD, false); + STATIC_FLAG_STATES.put(FLAG_ENABLE_MEDIACODEC_BUFFER_DECODE_ONLY, false); + STATIC_FLAG_STATES.put(FLAG_VIDEO_RENDERER_DURATION_TO_PROGRESS, false); + + // Statically enabled flags (kept as fallback for opt-out). + STATIC_FLAG_STATES.put(FLAG_DYNAMIC_SCHEDULING, true); + STATIC_FLAG_STATES.put(FLAG_ENABLE_STUCK_PLAYING_DETECTION, true); + } + + @SuppressWarnings("NonFinalStaticField") // Intentional statically shared mutable state. + private static boolean canaryModeEnabled = true; + + @SuppressWarnings("NonFinalStaticField") // Intentional statically shared mutable state. + private static SparseBooleanArray overrides = new SparseBooleanArray(); + + /** + * Sets whether canary mode is enabled globally. + * + *

Canary mode is enabled by default. + * + *

When enabled, all flags that are not statically excluded are active by default. Individual + * overrides set via {@link #enableFlag} or {@link #disableFlag} take precedence. + * + *

This must be called before creating any Media3 components. + * + * @param enabled Whether to enable canary mode. + */ + public static synchronized void setCanaryModeEnabled(boolean enabled) { + canaryModeEnabled = enabled; + } + + /** Returns whether canary mode is currently enabled. */ + public static boolean isCanaryModeEnabled() { + return canaryModeEnabled; + } + + /** + * Explicitly enables a flag, overriding the canary mode default. + * + *

This must be called before creating any Media3 components. + * + * @param flag The {@linkplain Flag flag identifier}. + */ + public static synchronized void enableFlag(@Flag int flag) { + checkArgument(flag > 0); + if (Util.contains(STATIC_FLAG_STATES, flag) && STATIC_FLAG_STATES.get(flag)) { + if (Util.contains(overrides, flag)) { + SparseBooleanArray newOverrides = overrides.clone(); + newOverrides.delete(flag); + overrides = newOverrides; + } + return; + } + SparseBooleanArray newOverrides = overrides.clone(); + newOverrides.put(flag, true); + overrides = newOverrides; + } + + /** + * Explicitly disables a flag, overriding the canary mode default. + * + *

This must be called before creating any Media3 components. + * + * @param flag The {@linkplain Flag flag identifier}. + */ + public static synchronized void disableFlag(@Flag int flag) { + checkArgument(flag > 0); + if (Util.contains(STATIC_FLAG_STATES, flag) && !STATIC_FLAG_STATES.get(flag)) { + if (Util.contains(overrides, flag)) { + SparseBooleanArray newOverrides = overrides.clone(); + newOverrides.delete(flag); + overrides = newOverrides; + } + return; + } + SparseBooleanArray newOverrides = overrides.clone(); + newOverrides.put(flag, false); + overrides = newOverrides; + } + + /** + * Checks if a specific flag is enabled. + * + *

The resolution order is: + * + *

    + *
  1. If the flag has an explicit override (via {@link #enableFlag} or {@link #disableFlag}), + * the override value is returned. + *
  2. If the flag has a static value not part of canary mode, that static value is returned. + *
  3. Otherwise, the flag is enabled if canary mode is enabled. + *
+ * + * @param flag The {@linkplain Flag flag identifier}. + * @return Whether the flag is enabled. + */ + public static boolean isEnabled(@Flag int flag) { + checkArgument(flag > 0); + SparseBooleanArray overrides = Flags.overrides; + // 1. Check explicit overrides first. + if (Util.contains(overrides, flag)) { + return overrides.get(flag); + } + // 2. Check static flag states. + if (Util.contains(STATIC_FLAG_STATES, flag)) { + return STATIC_FLAG_STATES.get(flag); + } + // 3. Default to canary mode state. + return canaryModeEnabled; + } + + /** + * Returns the flag identifiers that are explicitly enabled via {@link #enableFlag}, overriding + * the canary mode default. + */ + public static @Flag int[] getEnabledOverrides() { + return getOverrides(/* enabled= */ true); + } + + /** + * Returns the flag identifiers that are explicitly disabled via {@link #disableFlag}, overriding + * the canary mode default. + */ + public static @Flag int[] getDisabledOverrides() { + return getOverrides(/* enabled= */ false); + } + + /** + * Returns a log string describing the current canary mode and flag override state. + * + *

The format is {@code [Canary:, Overrides: enabled=[], disabled=[]]}, or + * {@code [Canary:]} if there are no overrides. Only deviations from the default state are + * listed. + */ + public static String getLogString() { + StringBuilder sb = new StringBuilder(); + sb.append("[Canary:").append(canaryModeEnabled ? "ON" : "OFF"); + + @Flag int[] enabledOverrides = getEnabledOverrides(); + @Flag int[] disabledOverrides = getDisabledOverrides(); + if (enabledOverrides.length > 0 || disabledOverrides.length > 0) { + sb.append(", Overrides:"); + if (enabledOverrides.length > 0) { + sb.append(" enabled="); + appendIds(sb, enabledOverrides); + } + if (enabledOverrides.length > 0 && disabledOverrides.length > 0) { + sb.append(","); + } + if (disabledOverrides.length > 0) { + sb.append(" disabled="); + appendIds(sb, disabledOverrides); + } + } + sb.append("]"); + return sb.toString(); + } + + private static @Flag int[] getOverrides(boolean enabled) { + SparseBooleanArray overrides = Flags.overrides; + int count = 0; + for (int i = 0; i < overrides.size(); i++) { + if (overrides.valueAt(i) == enabled) { + count++; + } + } + @Flag int[] result = new int[count]; + int index = 0; + for (int i = 0; i < overrides.size(); i++) { + if (overrides.valueAt(i) == enabled) { + result[index++] = overrides.keyAt(i); + } + } + return result; + } + + private static void appendIds(StringBuilder sb, @Flag int[] ids) { + sb.append("["); + for (int i = 0; i < ids.length; i++) { + if (i > 0) { + sb.append(","); + } + sb.append(ids[i]); + } + sb.append("]"); + } + + /** + * Resets the flag registry to its default configuration (all overrides cleared, canary mode + * enabled). + * + *

This is intended for use in tests only to avoid test pollution. In unit tests, {@code + * androidx.media3.test.utils.Media3FlagsRule} is the preferred alternative to calling this method + * directly. + */ + @VisibleForTesting + public static synchronized void resetForTesting() { + canaryModeEnabled = true; + overrides = new SparseBooleanArray(); + } + + private Flags() {} // Prevents instantiation. +} diff --git a/libraries/common/src/main/java/androidx/media3/common/Format.java b/libraries/common/src/main/java/androidx/media3/common/Format.java index 741805e55f7..16c29fa736e 100644 --- a/libraries/common/src/main/java/androidx/media3/common/Format.java +++ b/libraries/common/src/main/java/androidx/media3/common/Format.java @@ -177,7 +177,6 @@ public static final class Builder { @Nullable private List initializationData; @Nullable private DrmInitData drmInitData; private long subsampleOffsetUs; - private boolean hasPrerollSamples; // Video specific. @@ -282,7 +281,6 @@ private Builder(Format format) { this.initializationData = format.initializationData; this.drmInitData = format.drmInitData; this.subsampleOffsetUs = format.subsampleOffsetUs; - this.hasPrerollSamples = format.hasPrerollSamples; // Video specific. this.width = format.width; this.height = format.height; @@ -606,18 +604,6 @@ public Builder setSubsampleOffsetUs(long subsampleOffsetUs) { return this; } - /** - * Sets {@link Format#hasPrerollSamples}. The default value is {@code false}. - * - * @param hasPrerollSamples The {@link Format#hasPrerollSamples}. - * @return The builder. - */ - @CanIgnoreReturnValue - public Builder setHasPrerollSamples(boolean hasPrerollSamples) { - this.hasPrerollSamples = hasPrerollSamples; - return this; - } - // Video specific. /** @@ -1131,15 +1117,6 @@ public Format build() { */ @UnstableApi public final long subsampleOffsetUs; - /** - * Indicates whether the stream contains preroll samples. - * - *

When this field is set to {@code true}, it means that the stream includes decode-only - * samples that occur before the intended playback start position. These samples are necessary for - * decoding but are not meant to be rendered and should be skipped after decoding. - */ - @UnstableApi public final boolean hasPrerollSamples; - // Video specific. /** The width of the video in pixels, or {@link #NO_VALUE} if unknown or not applicable. */ @@ -1340,7 +1317,6 @@ private Format(Builder builder) { builder.initializationData == null ? Collections.emptyList() : builder.initializationData; drmInitData = builder.drmInitData; subsampleOffsetUs = builder.subsampleOffsetUs; - hasPrerollSamples = builder.hasPrerollSamples; // Video specific. width = builder.width; height = builder.height; diff --git a/libraries/common/src/main/java/androidx/media3/common/MediaItem.java b/libraries/common/src/main/java/androidx/media3/common/MediaItem.java index 6be88f395a3..cd046dd1142 100644 --- a/libraries/common/src/main/java/androidx/media3/common/MediaItem.java +++ b/libraries/common/src/main/java/androidx/media3/common/MediaItem.java @@ -1567,6 +1567,7 @@ public static final class Builder { private @C.RoleFlags int roleFlags; @Nullable private String label; @Nullable private String id; + private long timeOffsetUs; /** * Constructs an instance. @@ -1585,6 +1586,7 @@ private Builder(SubtitleConfiguration subtitleConfiguration) { this.roleFlags = subtitleConfiguration.roleFlags; this.label = subtitleConfiguration.label; this.id = subtitleConfiguration.id; + this.timeOffsetUs = subtitleConfiguration.timeOffsetUs; } /** Sets the {@link Uri} to the subtitle file. */ @@ -1636,6 +1638,37 @@ public Builder setId(@Nullable String id) { return this; } + /** + * Sets the offset that is added to the timestamps of the cues in this subtitle track, in + * microseconds. + * + *

A positive value shifts the cues to be displayed later relative to the media, a negative + * value shifts them to be displayed earlier. + * + *

ExoPlayer specific notes: + * + *

    + *
  • The offset can be changed during playback by passing an updated {@link MediaItem} to + * {@code Player.replaceMediaItem(int, MediaItem)}. If only the time offsets of the + * {@link SubtitleConfiguration} instances are changed, playback continues uninterrupted + * and the new offsets apply to cues that have not been read by the renderer yet. To + * also apply the new offset to the cues currently on screen, disable and re-enable the + * text track, for example with {@code + * TrackSelectionParameters.Builder.setTrackTypeDisabled(C.TRACK_TYPE_TEXT, boolean)}. + *
  • The offset only takes effect if the subtitles are parsed during extraction (the + * default behavior of {@code DefaultMediaSourceFactory}), and is ignored by the + * deprecated legacy subtitle decoding path. + *
+ * + *

The default value is {@code 0}. + */ + @CanIgnoreReturnValue + @UnstableApi + public Builder setTimeOffsetUs(long timeOffsetUs) { + this.timeOffsetUs = timeOffsetUs; + return this; + } + /** Creates a {@link SubtitleConfiguration} from the values of this builder. */ public SubtitleConfiguration build() { return new SubtitleConfiguration(this); @@ -1671,6 +1704,12 @@ private Subtitle buildSubtitle() { */ @Nullable public final String id; + /** + * The offset that is added to the timestamps of the cues in this subtitle track, in + * microseconds. See {@link Builder#setTimeOffsetUs(long)} for details. + */ + @UnstableApi public final long timeOffsetUs; + private SubtitleConfiguration( Uri uri, String mimeType, @@ -1686,6 +1725,7 @@ private SubtitleConfiguration( this.roleFlags = roleFlags; this.label = label; this.id = id; + this.timeOffsetUs = 0; } private SubtitleConfiguration(Builder builder) { @@ -1696,6 +1736,7 @@ private SubtitleConfiguration(Builder builder) { this.roleFlags = builder.roleFlags; this.label = builder.label; this.id = builder.id; + this.timeOffsetUs = builder.timeOffsetUs; } /** Returns a {@link Builder} initialized with the values of this instance. */ @@ -1720,7 +1761,8 @@ public boolean equals(@Nullable Object obj) { && selectionFlags == other.selectionFlags && roleFlags == other.roleFlags && Objects.equals(label, other.label) - && Objects.equals(id, other.id); + && Objects.equals(id, other.id) + && timeOffsetUs == other.timeOffsetUs; } @Override @@ -1732,6 +1774,7 @@ public int hashCode() { result = 31 * result + roleFlags; result = 31 * result + (label == null ? 0 : label.hashCode()); result = 31 * result + (id == null ? 0 : id.hashCode()); + result = 31 * result + (int) (timeOffsetUs ^ (timeOffsetUs >>> 32)); return result; } @@ -1742,6 +1785,7 @@ public int hashCode() { private static final String FIELD_ROLE_FLAGS = Util.intToStringMaxRadix(4); private static final String FIELD_LABEL = Util.intToStringMaxRadix(5); private static final String FIELD_ID = Util.intToStringMaxRadix(6); + private static final String FIELD_TIME_OFFSET_US = Util.intToStringMaxRadix(7); /** Restores a {@code SubtitleConfiguration} from a {@link Bundle}. */ @UnstableApi @@ -1753,6 +1797,7 @@ public static SubtitleConfiguration fromBundle(Bundle bundle) { @C.RoleFlags int roleFlags = bundle.getInt(FIELD_ROLE_FLAGS, 0); @Nullable String label = bundle.getString(FIELD_LABEL); @Nullable String id = bundle.getString(FIELD_ID); + long timeOffsetUs = bundle.getLong(FIELD_TIME_OFFSET_US, 0); SubtitleConfiguration.Builder builder = new SubtitleConfiguration.Builder(uri); return builder @@ -1762,6 +1807,7 @@ public static SubtitleConfiguration fromBundle(Bundle bundle) { .setRoleFlags(roleFlags) .setLabel(label) .setId(id) + .setTimeOffsetUs(timeOffsetUs) .build(); } @@ -1787,6 +1833,9 @@ public Bundle toBundle() { if (id != null) { bundle.putString(FIELD_ID, id); } + if (timeOffsetUs != 0) { + bundle.putLong(FIELD_TIME_OFFSET_US, timeOffsetUs); + } return bundle; } } diff --git a/libraries/common/src/main/java/androidx/media3/common/MediaLibraryInfo.java b/libraries/common/src/main/java/androidx/media3/common/MediaLibraryInfo.java index 5967d221a58..fac86ad5ef6 100644 --- a/libraries/common/src/main/java/androidx/media3/common/MediaLibraryInfo.java +++ b/libraries/common/src/main/java/androidx/media3/common/MediaLibraryInfo.java @@ -29,11 +29,11 @@ public final class MediaLibraryInfo { /** The version of the library expressed as a string, for example "1.2.3" or "1.2.0-beta01". */ // Intentionally hardcoded. Do not derive from other constants (e.g. VERSION_INT) or vice versa. - public static final String VERSION = "1.11.0-alpha01"; + public static final String VERSION = "1.11.1"; /** The version of the library expressed as {@code TAG + "/" + VERSION}. */ // Intentionally hardcoded. Do not derive from other constants (e.g. VERSION) or vice versa. - public static final String VERSION_SLASHY = "AndroidXMedia3/1.11.0-alpha01"; + public static final String VERSION_SLASHY = "AndroidXMedia3/1.11.1"; /** * The version of the library expressed as an integer, for example 1002003300. @@ -47,7 +47,7 @@ public final class MediaLibraryInfo { * (123-045-006-3-00). */ // Intentionally hardcoded. Do not derive from other constants (e.g. VERSION) or vice versa. - public static final int VERSION_INT = 1_011_000_0_01; + public static final int VERSION_INT = 1_011_001_3_00; /** Whether the library was compiled with {@link TraceUtil} trace enabled. */ public static final boolean TRACE_ENABLED = true; diff --git a/libraries/common/src/main/java/androidx/media3/common/MediaMetadata.java b/libraries/common/src/main/java/androidx/media3/common/MediaMetadata.java index a991bda8faa..f38830e130f 100644 --- a/libraries/common/src/main/java/androidx/media3/common/MediaMetadata.java +++ b/libraries/common/src/main/java/androidx/media3/common/MediaMetadata.java @@ -91,10 +91,13 @@ public static final class Builder { @Nullable private CharSequence compilation; @Nullable private CharSequence station; @Nullable private @MediaType Integer mediaType; + @Nullable private String playlistId; + private ImmutableList categories; @Nullable private Bundle extras; private ImmutableList supportedCommands; public Builder() { + categories = ImmutableList.of(); supportedCommands = ImmutableList.of(); } @@ -136,6 +139,8 @@ private Builder(MediaMetadata mediaMetadata) { this.compilation = mediaMetadata.compilation; this.station = mediaMetadata.station; this.mediaType = mediaMetadata.mediaType; + this.playlistId = mediaMetadata.playlistId; + this.categories = mediaMetadata.categories; this.supportedCommands = mediaMetadata.supportedCommands; this.extras = mediaMetadata.extras; } @@ -464,6 +469,27 @@ public Builder setMediaType(@Nullable @MediaType Integer mediaType) { return this; } + /** + * Sets the playlist ID. + * + * @throws IllegalArgumentException if {@code playlistId} is empty. + */ + @CanIgnoreReturnValue + @UnstableApi + public Builder setPlaylistId(@Nullable String playlistId) { + checkArgument(playlistId == null || !playlistId.isEmpty()); + this.playlistId = playlistId; + return this; + } + + /** Sets the list of categories or category tokens of the media. */ + @CanIgnoreReturnValue + @UnstableApi + public Builder setCategories(List categories) { + this.categories = ImmutableList.copyOf(categories); + return this; + } + /** Sets the extras {@link Bundle}. */ @CanIgnoreReturnValue public Builder setExtras(@Nullable Bundle extras) { @@ -638,6 +664,12 @@ public Builder populate(@Nullable MediaMetadata mediaMetadata) { if (mediaMetadata.mediaType != null) { setMediaType(mediaMetadata.mediaType); } + if (mediaMetadata.playlistId != null) { + setPlaylistId(mediaMetadata.playlistId); + } + if (!mediaMetadata.categories.isEmpty()) { + setCategories(mediaMetadata.categories); + } if (mediaMetadata.extras != null) { setExtras(mediaMetadata.extras); } @@ -1173,6 +1205,12 @@ public MediaMetadata build() { /** Optional {@link MediaType}. */ @Nullable public final @MediaType Integer mediaType; + /** Optional playlist ID. */ + @UnstableApi @Nullable public final String playlistId; + + /** The list of categories or category tokens of the media. */ + @UnstableApi public final ImmutableList categories; + /** * Optional extras {@link Bundle}. * @@ -1243,6 +1281,8 @@ private MediaMetadata(Builder builder) { this.compilation = builder.compilation; this.station = builder.station; this.mediaType = mediaType; + this.playlistId = builder.playlistId; + this.categories = builder.categories; this.supportedCommands = builder.supportedCommands; this.extras = builder.extras; } @@ -1297,6 +1337,8 @@ public boolean equals(@Nullable Object obj) { && TextUtils.equals(compilation, that.compilation) && TextUtils.equals(station, that.station) && Objects.equals(mediaType, that.mediaType) + && Objects.equals(playlistId, that.playlistId) + && Objects.equals(categories, that.categories) && Objects.equals(supportedCommands, that.supportedCommands) && ((extras == null) == (that.extras == null)); } @@ -1340,7 +1382,9 @@ public int hashCode() { station, mediaType, extras == null, - supportedCommands); + supportedCommands, + playlistId, + categories); } private static final String FIELD_TITLE = Util.intToStringMaxRadix(0); @@ -1379,6 +1423,8 @@ public int hashCode() { private static final String FIELD_DURATION_MS = Util.intToStringMaxRadix(33); private static final String FIELD_SUPPORTED_COMMANDS = Util.intToStringMaxRadix(34); private static final String FIELD_DISC_SUBTITLE = Util.intToStringMaxRadix(35); + private static final String FIELD_PLAYLIST_ID = Util.intToStringMaxRadix(36); + private static final String FIELD_CATEGORIES = Util.intToStringMaxRadix(37); private static final String FIELD_EXTRAS = Util.intToStringMaxRadix(1000); // Use a fairly lenient threshold for sending byte array to legacy processes that don't support @@ -1516,6 +1562,12 @@ public Bundle toBundle(int interfaceVersion) { if (mediaType != null) { bundle.putInt(FIELD_MEDIA_TYPE, mediaType); } + if (playlistId != null) { + bundle.putString(FIELD_PLAYLIST_ID, playlistId); + } + if (!categories.isEmpty()) { + bundle.putStringArrayList(FIELD_CATEGORIES, new ArrayList<>(categories)); + } if (!supportedCommands.isEmpty()) { bundle.putStringArrayList(FIELD_SUPPORTED_COMMANDS, new ArrayList<>(supportedCommands)); } @@ -1561,6 +1613,7 @@ public static MediaMetadata fromBundle(Bundle bundle, int interfaceVersion) { .setGenre(bundle.getCharSequence(FIELD_GENRE)) .setCompilation(bundle.getCharSequence(FIELD_COMPILATION)) .setStation(bundle.getCharSequence(FIELD_STATION)) + .setPlaylistId(bundle.getString(FIELD_PLAYLIST_ID)) .setExtras(convertToNullIfInvalid(bundle.getBundle(FIELD_EXTRAS))); if (bundle.containsKey(FIELD_ARTWORK_DATA)) { @@ -1637,6 +1690,10 @@ public static MediaMetadata fromBundle(Bundle bundle, int interfaceVersion) { if (bundle.containsKey(FIELD_MEDIA_TYPE)) { builder.setMediaType(bundle.getInt(FIELD_MEDIA_TYPE)); } + @Nullable ArrayList categories = bundle.getStringArrayList(FIELD_CATEGORIES); + if (categories != null) { + builder.setCategories(categories); + } @Nullable ArrayList supportedCommands = bundle.getStringArrayList(FIELD_SUPPORTED_COMMANDS); if (supportedCommands != null) { diff --git a/libraries/common/src/main/java/androidx/media3/common/MetricsProvider.java b/libraries/common/src/main/java/androidx/media3/common/MetricsProvider.java new file mode 100644 index 00000000000..ba106895f4f --- /dev/null +++ b/libraries/common/src/main/java/androidx/media3/common/MetricsProvider.java @@ -0,0 +1,170 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.common; + +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; +import static java.lang.annotation.ElementType.TYPE_USE; +import static java.lang.annotation.RetentionPolicy.SOURCE; + +import androidx.annotation.IntDef; +import androidx.annotation.RestrictTo; +import androidx.media3.common.util.UnstableApi; +import java.lang.annotation.Documented; +import java.lang.annotation.Retention; +import java.lang.annotation.Target; + +/** An internal provider of diagnostic metrics for Media3 components. */ +@UnstableApi +@RestrictTo(LIBRARY_GROUP) +public interface MetricsProvider { + + /** Consumer for quantitative and categorical diagnostic metrics. */ + interface MetricConsumer { + + /** + * Component categories for Media3 video effects and audio processors. One of: + * + *

    + *
  • {@link #CATEGORY_EFFECT_COLOR} + *
  • {@link #CATEGORY_EFFECT_SPATIAL} + *
  • {@link #CATEGORY_EFFECT_CONVOLUTION} + *
  • {@link #CATEGORY_EFFECT_OVERLAY} + *
  • {@link #CATEGORY_EFFECT_TEMPORAL} + *
  • {@link #CATEGORY_EFFECT_BUFFER} + *
  • {@link #CATEGORY_EFFECT_CUSTOM} + *
  • {@link #CATEGORY_AUDIO_SPEED_AND_PITCH} + *
  • {@link #CATEGORY_AUDIO_CHANNEL_MANIPULATION} + *
  • {@link #CATEGORY_AUDIO_TEMPORAL_TRIM} + *
  • {@link #CATEGORY_AUDIO_PROCESSOR_CUSTOM} + *
+ */ + @Documented + @Retention(SOURCE) + @Target(TYPE_USE) + @IntDef({ + CATEGORY_EFFECT_COLOR, + CATEGORY_EFFECT_SPATIAL, + CATEGORY_EFFECT_CONVOLUTION, + CATEGORY_EFFECT_OVERLAY, + CATEGORY_EFFECT_TEMPORAL, + CATEGORY_EFFECT_BUFFER, + CATEGORY_EFFECT_CUSTOM, + CATEGORY_AUDIO_SPEED_AND_PITCH, + CATEGORY_AUDIO_CHANNEL_MANIPULATION, + CATEGORY_AUDIO_TEMPORAL_TRIM, + CATEGORY_AUDIO_PROCESSOR_CUSTOM, + }) + @interface Category {} + + /** Color adjustment effect category (e.g., LUTs, RGB matrices, HSL). */ + int CATEGORY_EFFECT_COLOR = 1; + + /** Spatial transformation effect category (e.g., crop, scale, rotate, resample). */ + int CATEGORY_EFFECT_SPATIAL = 2; + + /** Convolution filter effect category (e.g., Gaussian blur, separable convolution). */ + int CATEGORY_EFFECT_CONVOLUTION = 3; + + /** Overlay compositing effect category. */ + int CATEGORY_EFFECT_OVERLAY = 4; + + /** Temporal frame manipulation effect category (e.g., frame drop). */ + int CATEGORY_EFFECT_TEMPORAL = 5; + + /** Frame caching or byte-buffer processing effect category. */ + int CATEGORY_EFFECT_BUFFER = 6; + + /** Custom or uncategorized video effect category. */ + int CATEGORY_EFFECT_CUSTOM = 7; + + /** Audio speed or pitch modification processor category. */ + int CATEGORY_AUDIO_SPEED_AND_PITCH = 8; + + /** Audio channel mixing, mapping, or gain modification processor category. */ + int CATEGORY_AUDIO_CHANNEL_MANIPULATION = 9; + + /** Audio temporal trimming or silence skipping processor category. */ + int CATEGORY_AUDIO_TEMPORAL_TRIM = 10; + + /** Custom or uncategorized audio processor category. */ + int CATEGORY_AUDIO_PROCESSOR_CUSTOM = 11; + + /** + * Quantitative metric keys for Media3 video effects and audio processors. One of: + * + *
    + *
  • {@link #METRIC_SPEED_MULTIPLIER} + *
  • {@link #METRIC_ROTATION_DEGREES} + *
  • {@link #METRIC_FILTER_SAMPLE_COUNT} + *
  • {@link #METRIC_OVERLAY_COUNT} + *
+ */ + @Documented + @Retention(SOURCE) + @Target(TYPE_USE) + @IntDef({ + METRIC_SPEED_MULTIPLIER, + METRIC_ROTATION_DEGREES, + METRIC_FILTER_SAMPLE_COUNT, + METRIC_OVERLAY_COUNT, + }) + @interface MetricKey {} + + /** + * Audio speed multiplier scaled by 10,000 (e.g., {@code 1.5x} speed is reported as {@code + * 15000}). + */ + int METRIC_SPEED_MULTIPLIER = 1; + + /** Counter-clockwise rotation angle in integer degrees. */ + int METRIC_ROTATION_DEGREES = 2; + + /** Convolution filter sample count (tap count). */ + int METRIC_FILTER_SAMPLE_COUNT = 3; + + /** Number of overlays applied in an overlay effect. */ + int METRIC_OVERLAY_COUNT = 4; + + /** + * Sets the component category for this component. + * + * @param category The {@link Category}. + */ + void setCategory(@Category int category); + + /** + * Adds a quantitative metric key and value. + * + *

Implementations should omit quantitative metrics when parameter values represent a no-op + * or identity state (for example, {@code speed == 1.0f} or {@code rotationDegrees == 0f}). + * + * @param metricKey The {@link MetricKey}. + * @param metricValue The quantitative metric value. + */ + void addMetric(@MetricKey int metricKey, long metricValue); + } + + /** + * Populates diagnostic category and quantitative metrics for this component. + * + *

Implementations should call {@link MetricConsumer#setCategory(int)} and optionally {@link + * MetricConsumer#addMetric(int, long)} for non-identity parameter values. + * + * @param consumer The {@link MetricConsumer} receiving diagnostic metrics. + */ + void populateMetrics(MetricConsumer consumer); +} diff --git a/libraries/common/src/main/java/androidx/media3/common/MimeTypes.java b/libraries/common/src/main/java/androidx/media3/common/MimeTypes.java index 134bafc85fd..9ef1db028c6 100644 --- a/libraries/common/src/main/java/androidx/media3/common/MimeTypes.java +++ b/libraries/common/src/main/java/androidx/media3/common/MimeTypes.java @@ -161,6 +161,8 @@ public final class MimeTypes { @UnstableApi public static final String APPLICATION_META = BASE_TYPE_APPLICATION + "/meta"; + @UnstableApi public static final String APPLICATION_RDF_XML = BASE_TYPE_APPLICATION + "/rdf+xml"; + @UnstableApi public static final String APPLICATION_ITUT_T35 = BASE_TYPE_APPLICATION + "/x-itut-t35"; diff --git a/libraries/common/src/main/java/androidx/media3/common/PlayerTransferState.java b/libraries/common/src/main/java/androidx/media3/common/PlayerTransferState.java index 2ae147efcd5..5882bfdd143 100644 --- a/libraries/common/src/main/java/androidx/media3/common/PlayerTransferState.java +++ b/libraries/common/src/main/java/androidx/media3/common/PlayerTransferState.java @@ -48,6 +48,7 @@ public static final class Builder { private ImmutableList mediaItems; private PlaybackParameters playbackParameters; private TrackSelectionParameters trackSelectionParameters; + private MediaMetadata playlistMetadata; /** * Constructs a new {@code Builder} with default values. @@ -63,6 +64,7 @@ public Builder() { this.mediaItems = ImmutableList.of(); this.playbackParameters = PlaybackParameters.DEFAULT; this.trackSelectionParameters = TrackSelectionParameters.DEFAULT; + this.playlistMetadata = MediaMetadata.EMPTY; } private Builder(PlayerTransferState state) { @@ -74,6 +76,7 @@ private Builder(PlayerTransferState state) { this.mediaItems = state.mediaItems; this.playbackParameters = state.playbackParameters; this.trackSelectionParameters = state.trackSelectionParameters; + this.playlistMetadata = state.playlistMetadata; } /** @@ -166,6 +169,18 @@ public Builder setTrackSelectionParameters(TrackSelectionParameters trackSelecti return this; } + /** + * Sets the {@link Player#getPlaylistMetadata() playlist metadata}. + * + * @param playlistMetadata The playlist metadata. + * @return This {@code Builder} instance for chaining. + */ + @CanIgnoreReturnValue + public Builder setPlaylistMetadata(MediaMetadata playlistMetadata) { + this.playlistMetadata = Objects.requireNonNull(playlistMetadata); + return this; + } + /** Builds a {@link PlayerTransferState} instance. */ public PlayerTransferState build() { return new PlayerTransferState(this); @@ -180,6 +195,7 @@ public PlayerTransferState build() { private final ImmutableList mediaItems; private final PlaybackParameters playbackParameters; private final TrackSelectionParameters trackSelectionParameters; + private final MediaMetadata playlistMetadata; private PlayerTransferState(Builder builder) { this.playWhenReady = builder.playWhenReady; @@ -190,6 +206,7 @@ private PlayerTransferState(Builder builder) { this.mediaItems = builder.mediaItems; this.playbackParameters = builder.playbackParameters; this.trackSelectionParameters = builder.trackSelectionParameters; + this.playlistMetadata = builder.playlistMetadata; } /** @@ -224,7 +241,8 @@ public static Builder builderFromPlayer(Player player) { .setCurrentPosition(player.getCurrentPosition()) .setMediaItems(mediaItems) .setPlaybackParameters(player.getPlaybackParameters()) - .setTrackSelectionParameters(player.getTrackSelectionParameters()); + .setTrackSelectionParameters(player.getTrackSelectionParameters()) + .setPlaylistMetadata(player.getPlaylistMetadata()); } /** @@ -254,6 +272,9 @@ public void setToPlayer(Player player) { if (player.getAvailableCommands().contains(Player.COMMAND_SET_TRACK_SELECTION_PARAMETERS)) { player.setTrackSelectionParameters(this.trackSelectionParameters); } + if (player.getAvailableCommands().contains(Player.COMMAND_SET_PLAYLIST_METADATA)) { + player.setPlaylistMetadata(this.playlistMetadata); + } } /** Returns a {@link Builder} pre-populated with the current state values. */ @@ -301,6 +322,11 @@ public TrackSelectionParameters getTrackSelectionParameters() { return trackSelectionParameters; } + /** Returns the {@link Player#getPlaylistMetadata() playlist metadata}. */ + public MediaMetadata getPlaylistMetadata() { + return playlistMetadata; + } + @Override public boolean equals(@Nullable Object o) { if (o == null || getClass() != o.getClass()) { @@ -314,7 +340,8 @@ public boolean equals(@Nullable Object o) { && currentPosition == that.currentPosition && Objects.equals(mediaItems, that.mediaItems) && Objects.equals(playbackParameters, that.playbackParameters) - && Objects.equals(trackSelectionParameters, that.trackSelectionParameters); + && Objects.equals(trackSelectionParameters, that.trackSelectionParameters) + && Objects.equals(playlistMetadata, that.playlistMetadata); } @Override @@ -327,6 +354,7 @@ public int hashCode() { currentPosition, mediaItems, playbackParameters, - trackSelectionParameters); + trackSelectionParameters, + playlistMetadata); } } diff --git a/libraries/common/src/main/java/androidx/media3/common/SimpleBasePlayer.java b/libraries/common/src/main/java/androidx/media3/common/SimpleBasePlayer.java index b409bf9f901..89a392aaad6 100644 --- a/libraries/common/src/main/java/androidx/media3/common/SimpleBasePlayer.java +++ b/libraries/common/src/main/java/androidx/media3/common/SimpleBasePlayer.java @@ -1525,13 +1525,10 @@ public Builder setLiveConfiguration(@Nullable MediaItem.LiveConfiguration liveCo } /** - * Sets the start time of the live presentation. + * Sets the start time of the presentation. * - *

This value can only be set to anything other than {@link C#TIME_UNSET} if the stream is - * {@linkplain #setLiveConfiguration live}. - * - * @param presentationStartTimeMs The start time of the live presentation, in milliseconds - * since the Unix epoch, or {@link C#TIME_UNSET} if unknown or not applicable. + * @param presentationStartTimeMs The start time of the presentation, in milliseconds since + * the Unix epoch, or {@link C#TIME_UNSET} if unknown or not applicable. * @return This builder. */ @CanIgnoreReturnValue @@ -1541,13 +1538,12 @@ public Builder setPresentationStartTimeMs(long presentationStartTimeMs) { } /** - * Sets the start time of the live window. + * Sets the start time of the window. * - *

This value can only be set to anything other than {@link C#TIME_UNSET} if the stream is - * {@linkplain #setLiveConfiguration live}. The value should also be greater or equal than the - * {@linkplain #setPresentationStartTimeMs presentation start time}, if set. + *

This value should be greater than or equal to the {@linkplain + * #setPresentationStartTimeMs presentation start time}, if set. * - * @param windowStartTimeMs The start time of the live window, in milliseconds since the Unix + * @param windowStartTimeMs The start time of the window, in milliseconds since the Unix * epoch, or {@link C#TIME_UNSET} if unknown or not applicable. * @return This builder. */ @@ -1723,14 +1719,14 @@ public MediaItemData build() { @Nullable public final MediaItem.LiveConfiguration liveConfiguration; /** - * The start time of the live presentation, in milliseconds since the Unix epoch, or {@link + * The start time of the presentation, in milliseconds since the Unix epoch, or {@link * C#TIME_UNSET} if unknown or not applicable. */ public final long presentationStartTimeMs; /** - * The start time of the live window, in milliseconds since the Unix epoch, or {@link - * C#TIME_UNSET} if unknown or not applicable. + * The start time of the window, in milliseconds since the Unix epoch, or {@link C#TIME_UNSET} + * if unknown or not applicable. */ public final long windowStartTimeMs; @@ -1778,16 +1774,11 @@ public MediaItemData build() { private MediaItemData(Builder builder) { if (builder.liveConfiguration == null) { - checkArgument( - builder.presentationStartTimeMs == C.TIME_UNSET, - "presentationStartTimeMs can only be set if liveConfiguration != null"); - checkArgument( - builder.windowStartTimeMs == C.TIME_UNSET, - "windowStartTimeMs can only be set if liveConfiguration != null"); checkArgument( builder.elapsedRealtimeEpochOffsetMs == C.TIME_UNSET, "elapsedRealtimeEpochOffsetMs can only be set if liveConfiguration != null"); - } else if (builder.presentationStartTimeMs != C.TIME_UNSET + } + if (builder.presentationStartTimeMs != C.TIME_UNSET && builder.windowStartTimeMs != C.TIME_UNSET) { checkArgument( builder.windowStartTimeMs >= builder.presentationStartTimeMs, @@ -1891,6 +1882,7 @@ private static MediaItemData buildFromState( new PeriodData.Builder(checkNotNull(period.uid)) .setAdPlaybackState(period.adPlaybackState) .setDurationUs(period.durationUs) + .setOriginalDurationUs(period.originalDurationUs) .setIsPlaceholder(period.isPlaceholder) .build()); } @@ -1944,7 +1936,8 @@ private Timeline.Period getPeriod( /* durationUs= */ positionInFirstPeriodUs + durationUs, /* positionInWindowUs= */ -positionInFirstPeriodUs, AdPlaybackState.NONE, - isPlaceholder); + isPlaceholder, + /* originalDurationUs= */ C.TIME_UNSET); } else { PeriodData periodData = periods.get(periodIndexInMediaItem); Object periodId = periodData.uid; @@ -1956,7 +1949,8 @@ private Timeline.Period getPeriod( periodData.durationUs, periodPositionInWindowUs[periodIndexInMediaItem], periodData.adPlaybackState, - periodData.isPlaceholder); + periodData.isPlaceholder, + periodData.originalDurationUs); } return period; } @@ -1978,6 +1972,7 @@ public static final class Builder { private Object uid; private long durationUs; + private long originalDurationUs; private AdPlaybackState adPlaybackState; private boolean isPlaceholder; @@ -1989,6 +1984,7 @@ public static final class Builder { public Builder(Object uid) { this.uid = uid; this.durationUs = 0; + this.originalDurationUs = C.TIME_UNSET; this.adPlaybackState = AdPlaybackState.NONE; this.isPlaceholder = false; } @@ -1996,6 +1992,7 @@ public Builder(Object uid) { private Builder(PeriodData periodData) { this.uid = periodData.uid; this.durationUs = periodData.durationUs; + this.originalDurationUs = periodData.originalDurationUs; this.adPlaybackState = periodData.adPlaybackState; this.isPlaceholder = periodData.isPlaceholder; } @@ -2028,6 +2025,21 @@ public Builder setDurationUs(long durationUs) { return this; } + /** + * Sets the original duration of the period before any modification of its end, in + * microseconds, or {@link C#TIME_UNSET} if unknown. + * + * @param originalDurationUs The original duration of the period before any modification of + * its end, in microseconds, or {@link C#TIME_UNSET} if unknown. + * @return This builder. + */ + @CanIgnoreReturnValue + public Builder setOriginalDurationUs(long originalDurationUs) { + checkArgument(originalDurationUs == C.TIME_UNSET || originalDurationUs >= 0); + this.originalDurationUs = originalDurationUs; + return this; + } + /** * Sets the {@link AdPlaybackState}. * @@ -2070,6 +2082,12 @@ public PeriodData build() { */ public final long durationUs; + /** + * The original duration of the period before any modification of its end, in microseconds, or + * {@link C#TIME_UNSET} if unknown. + */ + public final long originalDurationUs; + /** * The {@link AdPlaybackState} of the period, or {@link AdPlaybackState#NONE} if there are no * ads. @@ -2085,6 +2103,7 @@ public PeriodData build() { private PeriodData(Builder builder) { this.uid = builder.uid; this.durationUs = builder.durationUs; + this.originalDurationUs = builder.originalDurationUs; this.adPlaybackState = builder.adPlaybackState; this.isPlaceholder = builder.isPlaceholder; } @@ -2105,6 +2124,7 @@ public boolean equals(@Nullable Object o) { PeriodData periodData = (PeriodData) o; return this.uid.equals(periodData.uid) && this.durationUs == periodData.durationUs + && this.originalDurationUs == periodData.originalDurationUs && this.adPlaybackState.equals(periodData.adPlaybackState) && this.isPlaceholder == periodData.isPlaceholder; } @@ -2114,6 +2134,7 @@ public int hashCode() { int result = 7; result = 31 * result + uid.hashCode(); result = 31 * result + (int) (durationUs ^ (durationUs >>> 32)); + result = 31 * result + (int) (originalDurationUs ^ (originalDurationUs >>> 32)); result = 31 * result + adPlaybackState.hashCode(); result = 31 * result + (isPlaceholder ? 1 : 0); return result; diff --git a/libraries/common/src/main/java/androidx/media3/common/Timeline.java b/libraries/common/src/main/java/androidx/media3/common/Timeline.java index ddc1080ffda..62f4047c42c 100644 --- a/libraries/common/src/main/java/androidx/media3/common/Timeline.java +++ b/libraries/common/src/main/java/androidx/media3/common/Timeline.java @@ -608,9 +608,20 @@ public static final class Period { /** The index of the window to which this period belongs. */ public int windowIndex; - /** The duration of this period in microseconds, or {@link C#TIME_UNSET} if unknown. */ + /** + * The duration of this period in microseconds, or {@link C#TIME_UNSET} if unknown. + * + *

This is the playout duration of the period. It may be different from {@link + * #originalDurationUs} if the period end is modified. + */ @UnstableApi public long durationUs; + /** + * The original duration of this period in microseconds before any modification of the period + * end (for example by clipping), or {@link C#TIME_UNSET} if unknown. + */ + @UnstableApi public long originalDurationUs; + /** * The position of the start of this period relative to the start of the window to which it * belongs, in microseconds. May be negative if the start of the period is not within the @@ -633,7 +644,7 @@ public Period() { } /** - * @deprecated Use {@link #set(Object, Object, int, long, long, AdPlaybackState, boolean)} + * @deprecated Use {@link #set(Object, Object, int, long, long, AdPlaybackState, boolean, long)} * instead. */ @CanIgnoreReturnValue @@ -652,7 +663,8 @@ public Period set( durationUs, positionInWindowUs, AdPlaybackState.NONE, - /* isPlaceholder= */ false); + /* isPlaceholder= */ false, + /* originalDurationUs= */ C.TIME_UNSET); } /** @@ -673,8 +685,11 @@ public Period set( * @param isPlaceholder Whether this period contains placeholder information because the real * information has yet to be loaded. * @return This period, for convenience. + * @deprecated Use {@link #set(Object, Object, int, long, long, AdPlaybackState, boolean, long)} + * instead. */ @CanIgnoreReturnValue + @Deprecated @UnstableApi public Period set( @Nullable Object id, @@ -684,6 +699,50 @@ public Period set( long positionInWindowUs, AdPlaybackState adPlaybackState, boolean isPlaceholder) { + return set( + id, + uid, + windowIndex, + durationUs, + positionInWindowUs, + adPlaybackState, + isPlaceholder, + /* originalDurationUs= */ C.TIME_UNSET); + } + + /** + * Sets the data held by this period. + * + * @param id An identifier for the period. Not necessarily unique. May be null if the ids of the + * period are not required. + * @param uid A unique identifier for the period. May be null if the ids of the period are not + * required. + * @param windowIndex The index of the window to which this period belongs. + * @param durationUs The duration of this period in microseconds, or {@link C#TIME_UNSET} if + * unknown. + * @param positionInWindowUs The position of the start of this period relative to the start of + * the window to which it belongs, in microseconds. May be negative if the start of the + * period is not within the window. + * @param adPlaybackState The state of the period's ads, or {@link AdPlaybackState#NONE} if + * there are no ads. + * @param isPlaceholder Whether this period contains placeholder information because the real + * information has yet to be loaded. + * @param originalDurationUs The original duration of this period in microseconds before any + * modification of the period end (for example by clipping), or {@link C#TIME_UNSET} if + * unknown. + * @return This period, for convenience. + */ + @CanIgnoreReturnValue + @UnstableApi + public Period set( + @Nullable Object id, + @Nullable Object uid, + int windowIndex, + long durationUs, + long positionInWindowUs, + AdPlaybackState adPlaybackState, + boolean isPlaceholder, + long originalDurationUs) { this.id = id; this.uid = uid; this.windowIndex = windowIndex; @@ -691,6 +750,7 @@ public Period set( this.positionInWindowUs = positionInWindowUs; this.adPlaybackState = adPlaybackState; this.isPlaceholder = isPlaceholder; + this.originalDurationUs = originalDurationUs; return this; } @@ -704,6 +764,24 @@ public long getDurationUs() { return durationUs; } + /** + * Returns the original duration of the period in milliseconds before any modification of the + * period end, or {@link C#TIME_UNSET} if unknown. + */ + @UnstableApi + public long getOriginalDurationMs() { + return Util.usToMs(originalDurationUs); + } + + /** + * Returns the original duration of this period in microseconds before any modification of the + * period end, or {@link C#TIME_UNSET} if unknown. + */ + @UnstableApi + public long getOriginalDurationUs() { + return originalDurationUs; + } + /** * Returns the position of the start of this period relative to the start of the window to which * it belongs, in milliseconds. May be negative if the start of the period is not within the @@ -916,7 +994,8 @@ public boolean equals(@Nullable Object obj) { && durationUs == that.durationUs && positionInWindowUs == that.positionInWindowUs && isPlaceholder == that.isPlaceholder - && Objects.equals(adPlaybackState, that.adPlaybackState); + && Objects.equals(adPlaybackState, that.adPlaybackState) + && originalDurationUs == that.originalDurationUs; } @Override @@ -929,6 +1008,7 @@ public int hashCode() { result = 31 * result + (int) (positionInWindowUs ^ (positionInWindowUs >>> 32)); result = 31 * result + (isPlaceholder ? 1 : 0); result = 31 * result + adPlaybackState.hashCode(); + result = 31 * result + (int) (originalDurationUs ^ (originalDurationUs >>> 32)); return result; } @@ -939,6 +1019,7 @@ public int hashCode() { private static final String FIELD_AD_PLAYBACK_STATE = Util.intToStringMaxRadix(4); private static final String FIELD_ID = Util.intToStringMaxRadix(5); private static final String FIELD_UID = Util.intToStringMaxRadix(6); + private static final String FIELD_ORIGINAL_DURATION_US = Util.intToStringMaxRadix(7); /** * @deprecated Use {@link #toBundle(int)} instead. @@ -982,6 +1063,9 @@ public Bundle toBundle(int interfaceVersion) { if (uid instanceof String) { bundle.putString(FIELD_UID, (String) uid); } + if (originalDurationUs != C.TIME_UNSET) { + bundle.putLong(FIELD_ORIGINAL_DURATION_US, originalDurationUs); + } return bundle; } @@ -1006,6 +1090,8 @@ public static Period fromBundle(Bundle bundle, int interfaceVersion) { int windowIndex = bundle.getInt(FIELD_WINDOW_INDEX, /* defaultValue= */ 0); long durationUs = bundle.getLong(FIELD_DURATION_US, /* defaultValue= */ C.TIME_UNSET); long positionInWindowUs = bundle.getLong(FIELD_POSITION_IN_WINDOW_US, /* defaultValue= */ 0); + long originalDurationUs = + bundle.getLong(FIELD_ORIGINAL_DURATION_US, /* defaultValue= */ C.TIME_UNSET); boolean isPlaceholder = bundle.getBoolean(FIELD_PLACEHOLDER, /* defaultValue= */ false); @Nullable Bundle adPlaybackStateBundle = bundle.getBundle(FIELD_AD_PLAYBACK_STATE); AdPlaybackState adPlaybackState = @@ -1018,7 +1104,14 @@ public static Period fromBundle(Bundle bundle, int interfaceVersion) { Period period = new Period(); period.set( - id, uid, windowIndex, durationUs, positionInWindowUs, adPlaybackState, isPlaceholder); + id, + uid, + windowIndex, + durationUs, + positionInWindowUs, + adPlaybackState, + isPlaceholder, + originalDurationUs); return period; } } @@ -1632,7 +1725,8 @@ public RemotableTimeline( p.durationUs, p.positionInWindowUs, p.adPlaybackState, - p.isPlaceholder); + p.isPlaceholder, + p.originalDurationUs); p = newPeriod; } sanitizedPeriods.add(p); @@ -1737,7 +1831,8 @@ public Period getPeriod(int periodIndex, Period period, boolean setIds) { p.durationUs, p.positionInWindowUs, p.adPlaybackState, - p.isPlaceholder); + p.isPlaceholder, + p.originalDurationUs); return period; } diff --git a/libraries/common/src/main/java/androidx/media3/common/audio/ChannelMixingAudioProcessor.java b/libraries/common/src/main/java/androidx/media3/common/audio/ChannelMixingAudioProcessor.java index 781d984d25c..e0f14fecf9f 100644 --- a/libraries/common/src/main/java/androidx/media3/common/audio/ChannelMixingAudioProcessor.java +++ b/libraries/common/src/main/java/androidx/media3/common/audio/ChannelMixingAudioProcessor.java @@ -15,10 +15,13 @@ */ package androidx.media3.common.audio; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; import static com.google.common.base.Preconditions.checkNotNull; import android.util.SparseArray; import androidx.annotation.Nullable; +import androidx.annotation.RestrictTo; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.util.UnstableApi; import java.nio.ByteBuffer; @@ -28,7 +31,8 @@ * possible input channel count before using the audio processor. Input and output are 16-bit PCM. */ @UnstableApi -public final class ChannelMixingAudioProcessor extends BaseAudioProcessor { +public final class ChannelMixingAudioProcessor extends BaseAudioProcessor + implements MetricsProvider { private final SparseArray matrixByInputChannelCount; @@ -91,4 +95,10 @@ public void queueInput(ByteBuffer inputBuffer) { /* clipFloatOutput= */ true); outputBuffer.flip(); } + + @Override + @RestrictTo(LIBRARY_GROUP) + public void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_AUDIO_CHANNEL_MANIPULATION); + } } diff --git a/libraries/common/src/main/java/androidx/media3/common/audio/DefaultGainProvider.java b/libraries/common/src/main/java/androidx/media3/common/audio/DefaultGainProvider.java index 1bc50a83617..835a996ab49 100644 --- a/libraries/common/src/main/java/androidx/media3/common/audio/DefaultGainProvider.java +++ b/libraries/common/src/main/java/androidx/media3/common/audio/DefaultGainProvider.java @@ -28,6 +28,7 @@ import androidx.media3.common.util.UnstableApi; import com.google.common.base.Function; import com.google.common.collect.Range; +import com.google.common.collect.RangeMap; import com.google.common.collect.TreeRangeMap; import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.util.Map.Entry; diff --git a/libraries/common/src/main/java/androidx/media3/common/audio/GainProcessor.java b/libraries/common/src/main/java/androidx/media3/common/audio/GainProcessor.java index 6310864a45c..0a0d6606c4c 100644 --- a/libraries/common/src/main/java/androidx/media3/common/audio/GainProcessor.java +++ b/libraries/common/src/main/java/androidx/media3/common/audio/GainProcessor.java @@ -15,6 +15,7 @@ */ package androidx.media3.common.audio; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; import static androidx.media3.common.util.Util.durationUsToSampleCount; import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkNotNull; @@ -22,7 +23,9 @@ import static java.lang.Math.min; import androidx.annotation.IntRange; +import androidx.annotation.RestrictTo; import androidx.media3.common.C; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.util.UnstableApi; import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.nio.ByteBuffer; @@ -30,7 +33,7 @@ /** Applies {@linkplain GainProvider gain automation} over an audio stream. */ @UnstableApi -public final class GainProcessor extends BaseAudioProcessor { +public final class GainProcessor extends BaseAudioProcessor implements MetricsProvider { /** Interface that provides sample-level gain automation to be applied on an audio stream. */ public interface GainProvider { @@ -153,4 +156,10 @@ public void onFlush(StreamMetadata streamMetadata) { public void onReset() { readFrames = 0; } + + @Override + @RestrictTo(LIBRARY_GROUP) + public void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_AUDIO_CHANNEL_MANIPULATION); + } } diff --git a/libraries/common/src/main/java/androidx/media3/common/audio/SonicAudioProcessor.java b/libraries/common/src/main/java/androidx/media3/common/audio/SonicAudioProcessor.java index 350a2d9448c..0809a425992 100644 --- a/libraries/common/src/main/java/androidx/media3/common/audio/SonicAudioProcessor.java +++ b/libraries/common/src/main/java/androidx/media3/common/audio/SonicAudioProcessor.java @@ -15,14 +15,17 @@ */ package androidx.media3.common.audio; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkNotNull; import static java.lang.Math.abs; import androidx.annotation.FloatRange; import androidx.annotation.Nullable; +import androidx.annotation.RestrictTo; import androidx.media3.common.C; import androidx.media3.common.Format; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.util.UnstableApi; import androidx.media3.common.util.Util; import java.nio.ByteBuffer; @@ -32,7 +35,7 @@ * An {@link AudioProcessor} that uses the Sonic library to modify audio speed/pitch/sample rate. */ @UnstableApi -public final class SonicAudioProcessor implements AudioProcessor { +public final class SonicAudioProcessor implements AudioProcessor, MetricsProvider { /** Indicates that the output sample rate should be the same as the input. */ public static final int SAMPLE_RATE_NO_CHANGE = -1; @@ -311,4 +314,13 @@ public void reset() { outputBytes = 0; inputEnded = false; } + + @Override + @RestrictTo(LIBRARY_GROUP) + public void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_AUDIO_SPEED_AND_PITCH); + if (speed != 1.0f) { + consumer.addMetric(MetricConsumer.METRIC_SPEED_MULTIPLIER, Math.round(speed * 10000)); + } + } } diff --git a/libraries/common/src/main/java/androidx/media3/common/audio/SpeedChangingAudioProcessor.java b/libraries/common/src/main/java/androidx/media3/common/audio/SpeedChangingAudioProcessor.java index c2974531c2a..9a3eb593339 100644 --- a/libraries/common/src/main/java/androidx/media3/common/audio/SpeedChangingAudioProcessor.java +++ b/libraries/common/src/main/java/androidx/media3/common/audio/SpeedChangingAudioProcessor.java @@ -16,6 +16,7 @@ package androidx.media3.common.audio; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; import static androidx.media3.common.util.SpeedProviderUtil.getNextSpeedChangeSamplePosition; import static androidx.media3.common.util.SpeedProviderUtil.getSampleAlignedSpeed; import static androidx.media3.common.util.Util.durationUsToSampleCount; @@ -32,6 +33,7 @@ import androidx.annotation.VisibleForTesting; import androidx.media3.common.C; import androidx.media3.common.Format; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.util.LongArrayQueue; import androidx.media3.common.util.SpeedProviderUtil; import androidx.media3.common.util.SpeedProviderUtil.SpeedProviderMapper; @@ -47,7 +49,7 @@ * An {@link AudioProcessor} that changes the speed of audio samples depending on their timestamp. */ @UnstableApi -public final class SpeedChangingAudioProcessor implements AudioProcessor { +public final class SpeedChangingAudioProcessor implements AudioProcessor, MetricsProvider { private final Object lock; @@ -426,4 +428,10 @@ private void resetInternalState(boolean shouldResetSpeed) { framesRead = 0; endOfStreamQueuedToSonic = false; } + + @Override + @RestrictTo(LIBRARY_GROUP) + public void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_AUDIO_SPEED_AND_PITCH); + } } diff --git a/libraries/common/src/main/java/androidx/media3/common/text/CustomSpanBundler.java b/libraries/common/src/main/java/androidx/media3/common/text/CustomSpanBundler.java index 31292eeb143..199df6468f5 100644 --- a/libraries/common/src/main/java/androidx/media3/common/text/CustomSpanBundler.java +++ b/libraries/common/src/main/java/androidx/media3/common/text/CustomSpanBundler.java @@ -22,6 +22,9 @@ import android.os.Bundle; import android.text.Spannable; import android.text.Spanned; +import android.text.style.BackgroundColorSpan; +import android.text.style.StrikethroughSpan; +import android.text.style.UnderlineSpan; import androidx.annotation.IntDef; import androidx.annotation.Nullable; import androidx.media3.common.util.Util; diff --git a/libraries/common/src/main/java/androidx/media3/common/util/CodecSpecificDataUtil.java b/libraries/common/src/main/java/androidx/media3/common/util/CodecSpecificDataUtil.java index 6e4e83c4e81..04d99ce7e07 100644 --- a/libraries/common/src/main/java/androidx/media3/common/util/CodecSpecificDataUtil.java +++ b/libraries/common/src/main/java/androidx/media3/common/util/CodecSpecificDataUtil.java @@ -168,37 +168,6 @@ public boolean isSupportableByMediaCodec() { // IAMF private static final String CODEC_ID_IAMF = "iamf"; - // TODO(b/482032815): Replace VVC literal constants with MediaCodecInfo.CodecProfileLevel - // constants when compile SDK is updated to 37. - private static final int VVC_PROFILE_MAIN_8 = 0x01; - private static final int VVC_PROFILE_MAIN_10 = 0x02; - private static final int VVC_PROFILE_MAIN_10_STILL = 0x04; - private static final int VVC_PROFILE_MAIN_10_HDR10 = 0x1000; - - private static final int VVC_MAIN_TIER_LEVEL_1_0 = 0x01; - private static final int VVC_MAIN_TIER_LEVEL_2_0 = 0x02; - private static final int VVC_MAIN_TIER_LEVEL_2_1 = 0x04; - private static final int VVC_MAIN_TIER_LEVEL_3_0 = 0x08; - private static final int VVC_MAIN_TIER_LEVEL_3_1 = 0x10; - private static final int VVC_MAIN_TIER_LEVEL_4_0 = 0x20; - private static final int VVC_HIGH_TIER_LEVEL_4_0 = 0x40; - private static final int VVC_MAIN_TIER_LEVEL_4_1 = 0x80; - private static final int VVC_HIGH_TIER_LEVEL_4_1 = 0x100; - private static final int VVC_MAIN_TIER_LEVEL_5_0 = 0x200; - private static final int VVC_HIGH_TIER_LEVEL_5_0 = 0x400; - private static final int VVC_MAIN_TIER_LEVEL_5_1 = 0x800; - private static final int VVC_HIGH_TIER_LEVEL_5_1 = 0x1000; - private static final int VVC_MAIN_TIER_LEVEL_5_2 = 0x2000; - private static final int VVC_HIGH_TIER_LEVEL_5_2 = 0x4000; - private static final int VVC_MAIN_TIER_LEVEL_6_0 = 0x8000; - private static final int VVC_HIGH_TIER_LEVEL_6_0 = 0x10000; - private static final int VVC_MAIN_TIER_LEVEL_6_1 = 0x20000; - private static final int VVC_HIGH_TIER_LEVEL_6_1 = 0x40000; - private static final int VVC_MAIN_TIER_LEVEL_6_2 = 0x80000; - private static final int VVC_HIGH_TIER_LEVEL_6_2 = 0x100000; - private static final int VVC_MAIN_TIER_LEVEL_6_3 = 0x200000; - private static final int VVC_HIGH_TIER_LEVEL_6_3 = 0x400000; - private static final Pattern PROFILE_PATTERN = Pattern.compile("^\\D?(\\d+)$"); private static final String TAG = "CodecSpecificDataUtil"; @@ -940,14 +909,14 @@ private static MediaCodecProfileAndLevel getVvcProfileAndLevel( int profile; if (profileIdc == 1) { if (colorInfo != null && colorInfo.colorTransfer == C.COLOR_TRANSFER_ST2084) { - profile = VVC_PROFILE_MAIN_10_HDR10; + profile = CodecProfileLevel.VVCProfileMain10HDR10; } else if (colorInfo != null && colorInfo.lumaBitdepth == 8) { - profile = VVC_PROFILE_MAIN_8; + profile = CodecProfileLevel.VVCProfileMain8; } else { - profile = VVC_PROFILE_MAIN_10; + profile = CodecProfileLevel.VVCProfileMain10; } } else if (profileIdc == 65) { - profile = VVC_PROFILE_MAIN_10_STILL; + profile = CodecProfileLevel.VVCProfileMain10Still; } else { Log.w(TAG, "Unknown VVC profile IDC: " + parts[1]); return MediaCodecProfileAndLevel.UNSUPPORTABLE; @@ -977,51 +946,51 @@ private static MediaCodecProfileAndLevel getVvcProfileAndLevel( private static Integer vvcCodecStringToProfileLevel(String codecString) { switch (codecString) { case "L16": - return VVC_MAIN_TIER_LEVEL_1_0; + return CodecProfileLevel.VVCMainTierLevel10; case "L32": - return VVC_MAIN_TIER_LEVEL_2_0; + return CodecProfileLevel.VVCMainTierLevel20; case "L35": - return VVC_MAIN_TIER_LEVEL_2_1; + return CodecProfileLevel.VVCMainTierLevel21; case "L48": - return VVC_MAIN_TIER_LEVEL_3_0; + return CodecProfileLevel.VVCMainTierLevel30; case "L51": - return VVC_MAIN_TIER_LEVEL_3_1; + return CodecProfileLevel.VVCMainTierLevel31; case "L64": - return VVC_MAIN_TIER_LEVEL_4_0; + return CodecProfileLevel.VVCMainTierLevel40; case "H64": - return VVC_HIGH_TIER_LEVEL_4_0; + return CodecProfileLevel.VVCHighTierLevel40; case "L67": - return VVC_MAIN_TIER_LEVEL_4_1; + return CodecProfileLevel.VVCMainTierLevel41; case "H67": - return VVC_HIGH_TIER_LEVEL_4_1; + return CodecProfileLevel.VVCHighTierLevel41; case "L80": - return VVC_MAIN_TIER_LEVEL_5_0; + return CodecProfileLevel.VVCMainTierLevel50; case "H80": - return VVC_HIGH_TIER_LEVEL_5_0; + return CodecProfileLevel.VVCHighTierLevel50; case "L83": - return VVC_MAIN_TIER_LEVEL_5_1; + return CodecProfileLevel.VVCMainTierLevel51; case "H83": - return VVC_HIGH_TIER_LEVEL_5_1; + return CodecProfileLevel.VVCHighTierLevel51; case "L86": - return VVC_MAIN_TIER_LEVEL_5_2; + return CodecProfileLevel.VVCMainTierLevel52; case "H86": - return VVC_HIGH_TIER_LEVEL_5_2; + return CodecProfileLevel.VVCHighTierLevel52; case "L96": - return VVC_MAIN_TIER_LEVEL_6_0; + return CodecProfileLevel.VVCMainTierLevel60; case "H96": - return VVC_HIGH_TIER_LEVEL_6_0; + return CodecProfileLevel.VVCHighTierLevel60; case "L112": - return VVC_MAIN_TIER_LEVEL_6_1; + return CodecProfileLevel.VVCMainTierLevel61; case "H112": - return VVC_HIGH_TIER_LEVEL_6_1; + return CodecProfileLevel.VVCHighTierLevel61; case "L128": - return VVC_MAIN_TIER_LEVEL_6_2; + return CodecProfileLevel.VVCMainTierLevel62; case "H128": - return VVC_HIGH_TIER_LEVEL_6_2; + return CodecProfileLevel.VVCHighTierLevel62; case "L144": - return VVC_MAIN_TIER_LEVEL_6_3; + return CodecProfileLevel.VVCMainTierLevel63; case "H144": - return VVC_HIGH_TIER_LEVEL_6_3; + return CodecProfileLevel.VVCHighTierLevel63; default: return null; } @@ -2363,5 +2332,122 @@ public static int h263ConstToLevelNumber(int levelConst) { } } + // Expected magic bytes for HDR10+ metadata in AV1 OBU or HEVC SEI. Spec: + // https://aomediacodec.github.io/av1-hdr10plus/#hdr10plus-metadata + private static final byte[] expectedHdr10PlusPayloadPrefix = { + // itu_t_t35_country_code == 0xB5 + (byte) 0xB5, + // itu_t_t35_terminal_provider_code == 0x003C + (byte) 0x00, + (byte) 0x3C, + // itu_t_t35_terminal_provider_oriented_code == 0x0001 + (byte) 0x00, + (byte) 0x01, + // application_identifier == 0x04 + (byte) 0x04, + }; + + // Expected magic bytes for HAGC (ST 2094-50) metadata in AV1 OBU or HEVC SEI. + private static final byte[] expectedHagcPayloadPrefix = { + // itu_t_t35_country_code == 0xB5 + (byte) 0xB5, + // itu_t_t35_terminal_provider_code == 0x0090 + (byte) 0x00, + (byte) 0x90, + // itu_t_t35_terminal_provider_oriented_code == 0x0001 + (byte) 0x00, + (byte) 0x01, + }; + + /** + * The length in bytes of the ITU-T T.35 header that prefixes HAGC (ST 2094-50) metadata in AV1 + * OBUs, HEVC SEIs and container metadata. + */ + public static final int HAGC_T35_HEADER_LENGTH = expectedHagcPayloadPrefix.length; + + /** + * Returns whether the given {@link ByteBuffer} contains HDR10+ metadata at the current position. + * + * @param payload The {@link ByteBuffer} to check. + * @return Whether the payload contains HDR10+ metadata. + */ + public static boolean isHdr10PlusMetadata(ByteBuffer payload) { + return isHdr10PlusMetadata(payload, payload.position(), payload.limit()); + } + + /** + * Returns whether the given {@link ByteBuffer} contains HDR10+ metadata at the given offset. + * + * @param buffer The {@link ByteBuffer} to check. + * @param offset The offset in the buffer to start checking from. + * @param limit The limit in the buffer to stop checking at. + * @return Whether the payload contains HDR10+ metadata. + */ + public static boolean isHdr10PlusMetadata(ByteBuffer buffer, int offset, int limit) { + if (limit - offset < expectedHdr10PlusPayloadPrefix.length) { + return false; + } + for (int i = 0; i < expectedHdr10PlusPayloadPrefix.length; i++) { + if (buffer.get(offset + i) != expectedHdr10PlusPayloadPrefix[i]) { + return false; + } + } + return true; + } + + /** + * Returns whether the given {@code data} contains HAGC (ST 2094-50) ITU-T T.35 metadata. + * + * @param data The payload data. + * @param length The length of the payload data. + */ + public static boolean isHagcMetadata(byte[] data, int length) { + if (length < expectedHagcPayloadPrefix.length) { + return false; + } + for (int i = 0; i < expectedHagcPayloadPrefix.length; i++) { + if (data[i] != expectedHagcPayloadPrefix[i]) { + return false; + } + } + return true; + } + + /** + * Returns whether the given {@link ByteBuffer} contains HAGC (ST 2094-50) ITU-T T.35 metadata, + * without consuming any bytes. + * + * @param buffer The payload data. + */ + public static boolean isHagcMetadata(ByteBuffer buffer) { + if (buffer.remaining() < expectedHagcPayloadPrefix.length) { + return false; + } + int pos = buffer.position(); + for (int i = 0; i < expectedHagcPayloadPrefix.length; i++) { + if (buffer.get(pos + i) != expectedHagcPayloadPrefix[i]) { + return false; + } + } + return true; + } + + /** + * Returns whether the given {@link Format} represents an HAGC (ST 2094-50) metadata track. + * + * @param format The format of the track. + */ + public static boolean isHagcTrack(Format format) { + if (!Objects.equals(format.sampleMimeType, MimeTypes.APPLICATION_ITUT_T35)) { + return false; + } + for (byte[] initData : format.initializationData) { + if (isHagcMetadata(initData, initData.length)) { + return true; + } + } + return false; + } + private CodecSpecificDataUtil() {} } diff --git a/libraries/common/src/main/java/androidx/media3/common/util/GlProgram.java b/libraries/common/src/main/java/androidx/media3/common/util/GlProgram.java index 245bd3a3e4a..1030014d597 100644 --- a/libraries/common/src/main/java/androidx/media3/common/util/GlProgram.java +++ b/libraries/common/src/main/java/androidx/media3/common/util/GlProgram.java @@ -67,6 +67,32 @@ public GlProgram(Context context, int vertexShaderResId, int fragmentShaderResId Util.loadRawResource(context, fragmentShaderResId)); } + /** + * Compiles a GL shader program from a vertex shader resource and one or more fragment shader + * resource IDs. + * + *

When multiple fragment shader resource IDs are passed, their sources are concatenated in + * order. + * + * @param context The {@link Context}. + * @param vertexShaderResId The resource ID of a vertex shader program. + * @param firstFragmentShaderResId The resource ID of the primary fragment shader program. + * @param additionalFragmentShaderResIds Optional additional resource IDs of fragment shader + * programs. + * @throws IOException When failing to read shader files. + */ + public GlProgram( + Context context, + int vertexShaderResId, + int firstFragmentShaderResId, + int... additionalFragmentShaderResIds) + throws IOException, GlUtil.GlException { + this( + Util.loadRawResource(context, vertexShaderResId), + assembleFragmentShaderSources( + context, firstFragmentShaderResId, additionalFragmentShaderResIds)); + } + /** * Compiles a GL shader program from vertex and fragment shader GLSL GLES20 code. * @@ -128,6 +154,16 @@ public GlProgram(String vertexShaderGlsl, String fragmentShaderGlsl) throws GlUt GlUtil.checkGlError(); } + private static String assembleFragmentShaderSources( + Context context, int firstResId, int... additionalResIds) throws IOException { + StringBuilder fullShader = new StringBuilder(); + fullShader.append(Util.loadRawResource(context, firstResId)).append("\n"); + for (int resId : additionalResIds) { + fullShader.append(Util.loadRawResource(context, resId)).append("\n"); + } + return fullShader.toString(); + } + private static void addShader(int programId, int type, String glsl) throws GlUtil.GlException { int shader = GLES20.glCreateShader(type); GLES20.glShaderSource(shader, glsl); diff --git a/libraries/common/src/main/java/androidx/media3/common/util/GlUtil.java b/libraries/common/src/main/java/androidx/media3/common/util/GlUtil.java index 7a2d06c5537..ef370c3f6fd 100644 --- a/libraries/common/src/main/java/androidx/media3/common/util/GlUtil.java +++ b/libraries/common/src/main/java/androidx/media3/common/util/GlUtil.java @@ -25,11 +25,15 @@ import android.content.Context; import android.content.pm.PackageManager; import android.graphics.Bitmap; +import android.hardware.SyncFence; import android.opengl.EGL14; +import android.opengl.EGL15; import android.opengl.EGLConfig; import android.opengl.EGLContext; import android.opengl.EGLDisplay; +import android.opengl.EGLExt; import android.opengl.EGLSurface; +import android.opengl.EGLSync; import android.opengl.GLES11Ext; import android.opengl.GLES20; import android.opengl.GLES30; @@ -38,9 +42,9 @@ import android.os.Build; import androidx.annotation.IntRange; import androidx.annotation.Nullable; -import androidx.annotation.RequiresApi; import androidx.annotation.RestrictTo; import androidx.media3.common.C; +import androidx.media3.common.video.SyncFenceWrapper; import com.google.common.collect.ImmutableList; import java.nio.ByteBuffer; import java.nio.ByteOrder; @@ -197,9 +201,6 @@ public static float[] createVertexBuffer(List vertexList) { *

If {@code true}, the device supports a protected output path for DRM content when using GL. */ public static boolean isProtectedContentExtensionSupported(Context context) throws GlException { - if (SDK_INT < 24) { - return false; - } if (SDK_INT < 26 && ("samsung".equals(Build.MANUFACTURER) || "XT1650".equals(Build.MODEL))) { // Samsung devices running Nougat are known to be broken. See // https://github.com/google/ExoPlayer/issues/3373 and [Internal: b/37197802]. @@ -355,7 +356,8 @@ public static EGLContext createEglContext( * @param surface The surface to wrap; must be a surface, surface texture or surface holder. * @param colorTransfer The {@linkplain C.ColorTransfer color transfer characteristics} to which * the {@code surface} is configured. The only accepted values are {@link - * C#COLOR_TRANSFER_SDR}, {@link C#COLOR_TRANSFER_HLG}, and {@link C#COLOR_TRANSFER_ST2084}. + * C#COLOR_TRANSFER_SDR}, {@link C#COLOR_TRANSFER_GAMMA_2_2}, {@link C#COLOR_TRANSFER_SRGB}, + * {@link C#COLOR_TRANSFER_HLG}, and {@link C#COLOR_TRANSFER_ST2084}. * @param isEncoderInputSurface Whether the {@code surface} is the input surface of an encoder. */ public static EGLSurface createEglSurface( @@ -366,7 +368,9 @@ public static EGLSurface createEglSurface( throws GlException { int[] configAttributes; int[] windowAttributes; - if (colorTransfer == C.COLOR_TRANSFER_SDR || colorTransfer == C.COLOR_TRANSFER_GAMMA_2_2) { + if (colorTransfer == C.COLOR_TRANSFER_SDR + || colorTransfer == C.COLOR_TRANSFER_GAMMA_2_2 + || colorTransfer == C.COLOR_TRANSFER_SRGB) { configAttributes = EGL_CONFIG_ATTRIBUTES_RGBA_8888; windowAttributes = EGL_WINDOW_SURFACE_ATTRIBUTES_NONE; } else if (colorTransfer == C.COLOR_TRANSFER_HLG || colorTransfer == C.COLOR_TRANSFER_ST2084) { @@ -516,6 +520,93 @@ public static void deleteSyncObjectQuietly(long syncObject) { GLES30.glDeleteSync(syncObject); } + /** + * Creates the requested number of native sync fences from the current OpenGL context. + * + *

Inserts a sync fence command into the GL command stream. The returned sync fences will + * signal when the GPU reaches the fence command in the stream - meaning all OpenGL operations + * enqueued on the current context before this call have completed execution on the GPU. + * + *

Must be called on the thread that owns the current OpenGL context. + * + * @param count The number of fences to generate. + * @return A list of {@link SyncFenceWrapper} instances, or an empty list if native sync fences + * are unsupported on this device or the API level is less than 33. If an empty list is + * returned, {@code glFinish()} is called to ensure synchronization. + * @throws GlException If creating or duplicating the sync fence fails. + */ + public static ImmutableList createSyncFences(int count) throws GlException { + checkArgument(count > 0); + EGLDisplay eglDisplay = getDefaultEglDisplay(); + String extensions = EGL14.eglQueryString(eglDisplay, EGL14.EGL_EXTENSIONS); + if (SDK_INT < 33 + || extensions == null + || !extensions.contains("EGL_ANDROID_native_fence_sync")) { + GLES20.glFinish(); + checkGlError(); + return ImmutableList.of(); + } + EGLSync eglSync = + EGL15.eglCreateSync( + eglDisplay, + EGLExt.EGL_SYNC_NATIVE_FENCE_ANDROID, + /* attrib_list= */ new long[] {EGL14.EGL_NONE}, + /* offset= */ 0); + checkEglException("eglCreateSync failed"); + if (Objects.equals(eglSync, EGL15.EGL_NO_SYNC)) { + GLES20.glFinish(); + checkGlError(); + return ImmutableList.of(); + } + ImmutableList.Builder fences = ImmutableList.builderWithExpectedSize(count); + @Nullable Throwable pendingException = null; + try { + SyncFence syncFence = EGLExt.eglDupNativeFenceFDANDROID(eglDisplay, eglSync); + checkEglException("eglDupNativeFenceFDANDROID failed"); + if (!syncFence.isValid()) { + // Calling eglDupNativeFenceAndroid may produce an invalid fence the first time it + // is called. See b/18052459. + GLES20.glFlush(); + syncFence = EGLExt.eglDupNativeFenceFDANDROID(eglDisplay, eglSync); + checkEglException("eglDupNativeFenceFDANDROID failed after glFlush"); + } + if (!syncFence.isValid()) { + GLES20.glFinish(); + checkGlError(); + return ImmutableList.of(); + } + fences.add(SyncFenceWrapper.of(syncFence)); + for (int i = 0; i < count - 1; i++) { + SyncFence duplicatedFence = EGLExt.eglDupNativeFenceFDANDROID(eglDisplay, eglSync); + checkEglException("eglDupNativeFenceFDANDROID failed"); + checkState(duplicatedFence.isValid()); + fences.add(SyncFenceWrapper.of(duplicatedFence)); + } + return fences.build(); + } catch (Throwable t) { + pendingException = t; + for (SyncFenceWrapper fence : fences.build()) { + try { + fence.close(); + } catch (Throwable closeException) { + pendingException.addSuppressed(closeException); + } + } + throw t; + } finally { + EGL15.eglDestroySync(eglDisplay, eglSync); + try { + checkEglException("eglDestroySync failed"); + } catch (GlException e) { + if (pendingException != null) { + pendingException.addSuppressed(e); + } else { + throw e; + } + } + } + } + /** * Ensures that following commands on the current OpenGL context will not be executed until the * sync point has been reached. If {@code syncObject} equals {@code #GL_FENCE_SYNC_UNSET}, this is @@ -615,10 +706,18 @@ private static void assertValidTextureSize(int width, int height) throws GlExcep * #createFocusedPlaceholderEglSurface}. */ public static void clearFocusedBuffers() throws GlException { - GLES20.glClearColor(/* red= */ 0, /* green= */ 0, /* blue= */ 0, /* alpha= */ 0); - GLES20.glClearDepthf(1.0f); - GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT); - GlUtil.checkGlError(); + clearFocusedBuffersInternal(/* r= */ 0, /* g= */ 0, /* b= */ 0, /* a= */ 0); + } + + /** + * Fills the pixels in the current output render target buffers with (r=0, g=0, b=0, a=1). + * + *

Buffers can be focused using {@link #focusEglSurface} and {@link + * #focusFramebufferUsingCurrentContext}, {@link #focusFramebuffer}, and {@link + * #createFocusedPlaceholderEglSurface}. + */ + public static void clearFocusedBuffersOpaque() throws GlException { + clearFocusedBuffersInternal(/* r= */ 0, /* g= */ 0, /* b= */ 0, /* a= */ 1); } /** @@ -992,10 +1091,6 @@ public static int createPixelBufferObject(int size) throws GlException { * *

This a non-blocking call which reads the data asynchronously. * - *

Requires API 24: This method must call the version of {@link GLES30#glReadPixels(int, int, - * int, int, int, int, int)} which accepts an integer offset as the last parameter. This version - * of glReadPixels is not available in the Java {@link GLES30} wrapper until API 24. - * *

HDR support is not yet implemented. * * @param readFboId The framebuffer that holds pixel data. @@ -1003,7 +1098,6 @@ public static int createPixelBufferObject(int size) throws GlException { * @param height The image height. * @param bufferId The pixel buffer object to read into. */ - @RequiresApi(24) public static void schedulePixelBufferRead(int readFboId, int width, int height, int bufferId) throws GlException { focusFramebufferUsingCurrentContext(readFboId, width, height); @@ -1040,13 +1134,10 @@ public static void schedulePixelBufferRead(int readFboId, int width, int height, *

This call blocks until the pixel buffer data from the last {@link #schedulePixelBufferRead} * call is available. * - *

Requires API 24: see {@link #schedulePixelBufferRead}. - * * @param bufferId The pixel buffer object. * @param size The size of the pixel buffer object's data store to be mapped. * @return The {@link ByteBuffer} that holds pixel data. */ - @RequiresApi(24) public static ByteBuffer mapPixelBufferObject(int bufferId, int size) throws GlException { GLES20.glBindBuffer(GLES30.GL_PIXEL_PACK_BUFFER, bufferId); checkGlError(); @@ -1074,11 +1165,8 @@ public static ByteBuffer mapPixelBufferObject(int bufferId, int size) throws GlE *

When this method returns, the pixel buffer object {@code bufferId} can be reused by {@link * #schedulePixelBufferRead}. * - *

Requires API 24: see {@link #schedulePixelBufferRead}. - * * @param bufferId The pixel buffer object. */ - @RequiresApi(24) public static void unmapPixelBufferObject(int bufferId) throws GlException { GLES30.glBindBuffer(GLES30.GL_PIXEL_PACK_BUFFER, bufferId); GlUtil.checkGlError(); @@ -1151,4 +1239,12 @@ && getContextMajorVersion() >= 3) { checkGlError(); } } + + private static void clearFocusedBuffersInternal(float r, float g, float b, float a) + throws GlException { + GLES20.glClearColor(r, g, b, a); + GLES20.glClearDepthf(1.0f); + GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT | GLES20.GL_DEPTH_BUFFER_BIT); + GlUtil.checkGlError(); + } } diff --git a/libraries/common/src/main/java/androidx/media3/common/util/HandlerExecutor.java b/libraries/common/src/main/java/androidx/media3/common/util/HandlerExecutor.java new file mode 100644 index 00000000000..347bbf1a5bc --- /dev/null +++ b/libraries/common/src/main/java/androidx/media3/common/util/HandlerExecutor.java @@ -0,0 +1,77 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.common.util; + +import android.os.Handler; +import java.util.concurrent.Executor; + +/** + * An {@link Executor} that posts commands to a {@link Handler} or {@link HandlerWrapper} and + * forwards uncaught {@link RuntimeException runtime exceptions} to a {@link Listener}. + */ +@UnstableApi +public final class HandlerExecutor implements Executor { + + /** Listener for uncaught exceptions thrown during command execution. */ + public interface Listener { + /** + * Called when executing a {@link Runnable} throws an uncaught {@link RuntimeException}. + * + *

This callback is invoked on the {@link Handler} thread where the command ran. + * + * @param exception The uncaught {@link RuntimeException}. + */ + void onError(RuntimeException exception); + } + + private final HandlerWrapper handler; + private final Listener listener; + + /** + * Creates an instance that posts commands to the specified {@link HandlerWrapper}. + * + * @param handler The {@link HandlerWrapper} to post commands to. + * @param listener The {@link Listener} to notify of uncaught {@link RuntimeException runtime + * exceptions}. + */ + public HandlerExecutor(HandlerWrapper handler, Listener listener) { + this.handler = handler; + this.listener = listener; + } + + /** + * Creates an instance that posts commands to the specified {@link Handler}. + * + * @param handler The {@link Handler} to post commands to. + * @param listener The {@link Listener} to notify of uncaught {@link RuntimeException runtime + * exceptions}. + */ + public HandlerExecutor(Handler handler, Listener listener) { + this(new SystemHandlerWrapper(handler), listener); + } + + @Override + public void execute(Runnable command) { + handler.post( + () -> { + try { + command.run(); + } catch (RuntimeException e) { + listener.onError(e); + } + }); + } +} diff --git a/libraries/common/src/main/java/androidx/media3/common/util/MediaFormatUtil.java b/libraries/common/src/main/java/androidx/media3/common/util/MediaFormatUtil.java index 14e7a14e496..355fcc42af9 100644 --- a/libraries/common/src/main/java/androidx/media3/common/util/MediaFormatUtil.java +++ b/libraries/common/src/main/java/androidx/media3/common/util/MediaFormatUtil.java @@ -85,9 +85,7 @@ public static Format createFormatFromMediaFormat(MediaFormat mediaFormat) { getInteger( mediaFormat, MediaFormat.KEY_BIT_RATE, /* defaultValue= */ Format.NO_VALUE)) .setCodecs(getCodecString(mediaFormat)) - .setFrameRate( - getFloatFromIntOrFloat( - mediaFormat, MediaFormat.KEY_FRAME_RATE, /* defaultValue= */ Format.NO_VALUE)) + .setFrameRate(getFrameRate(mediaFormat)) .setWidth( getInteger(mediaFormat, MediaFormat.KEY_WIDTH, /* defaultValue= */ Format.NO_VALUE)) .setHeight( @@ -288,16 +286,9 @@ public static void maybeSetColorInfo(MediaFormat format, @Nullable ColorInfo col /** * Creates and returns a {@code ColorInfo}, if a valid instance is described in the {@link * MediaFormat}. - * - *

Under API 24, {@code null} will always be returned, because {@link MediaFormat} color keys - * like {@link MediaFormat#KEY_COLOR_STANDARD} were only added in API 24. */ @Nullable public static ColorInfo getColorInfo(MediaFormat mediaFormat) { - if (SDK_INT < 24) { - // MediaFormat KEY_COLOR_TRANSFER and other KEY_COLOR values available from API 24. - return null; - } int colorSpace = getInteger( mediaFormat, MediaFormat.KEY_COLOR_STANDARD, /* defaultValue= */ Format.NO_VALUE); @@ -435,6 +426,20 @@ private static String getCodecString(MediaFormat mediaFormat) { } } + /** Returns the frame rate from the given {@link MediaFormat}. */ + @SuppressLint("InlinedApi") // Inlined MediaFormat.KEY_FRAME_RATE. + private static float getFrameRate(MediaFormat mediaFormat) { + float frameRate = + getFloatFromIntOrFloat( + mediaFormat, MediaFormat.KEY_FRAME_RATE, /* defaultValue= */ Format.NO_VALUE); + // Some devices may produce non-positive frame rates from MediaFormat. + // See b/493371202 for more information. + if (frameRate <= 0f || Float.isNaN(frameRate)) { + frameRate = Format.NO_VALUE; + } + return frameRate; + } + /** Returns the ratio between a pixel's width and height for a {@link MediaFormat}. */ // Inlined MediaFormat.KEY_PIXEL_ASPECT_RATIO_WIDTH and MediaFormat.KEY_PIXEL_ASPECT_RATIO_HEIGHT. @SuppressLint("InlinedApi") diff --git a/libraries/common/src/main/java/androidx/media3/common/util/ParsableByteArray.java b/libraries/common/src/main/java/androidx/media3/common/util/ParsableByteArray.java index c936a14f9bc..b6a5e6375ec 100644 --- a/libraries/common/src/main/java/androidx/media3/common/util/ParsableByteArray.java +++ b/libraries/common/src/main/java/androidx/media3/common/util/ParsableByteArray.java @@ -16,14 +16,13 @@ package androidx.media3.common.util; import static com.google.common.base.Preconditions.checkArgument; +import static com.google.common.base.Preconditions.checkNotNull; import static java.nio.ByteOrder.BIG_ENDIAN; import static java.nio.ByteOrder.LITTLE_ENDIAN; -import android.app.ActivityManager; -import android.os.Build; import androidx.annotation.Nullable; import androidx.annotation.VisibleForTesting; -import com.google.common.base.Ascii; +import androidx.media3.common.Flags; import com.google.common.base.Function; import com.google.common.collect.ImmutableSet; import com.google.common.primitives.Chars; @@ -59,14 +58,10 @@ public final class ParsableByteArray { StandardCharsets.UTF_16BE, StandardCharsets.UTF_16LE); - // TODO: b/147657250 - Flip this to true + // TODO: b/147657250 - Remove once setShouldEnforceLimitOnLegacyMethods is removed. private static final AtomicReference<@NullableType Boolean> shouldEnforceLimitOnLegacyMethods = new AtomicReference<>(); - private static class IsRunningInTestHolder { - static final boolean IS_RUNNING_IN_TEST = isRunningInTest(); - } - private byte[] data; private int position; // TODO(internal b/147657250): Enforce this limit on all read methods. @@ -357,6 +352,14 @@ public int peekInt() { return result; } + /** Reads the next byte as a signed value. */ + public byte readByte() { + if (bytesLeft() < 1) { + throw new IndexOutOfBoundsException("position=" + position + ", limit=" + limit); + } + return data[position++]; + } + /** Reads the next byte as an unsigned value. */ public int readUnsignedByte() { maybeAssertAtLeastBytesLeftForLegacyMethod(1); @@ -783,6 +786,14 @@ public Charset readUtfCharsetFromBom() { return null; } + /** + * Returns whether the provided {@link Charset} is supported when passed to methods like {@link + * #peekChar(Charset)} and {@link #readLine(Charset)}. + */ + public static boolean isCharsetSupported(Charset charset) { + return SUPPORTED_CHARSETS_FOR_READLINE.contains(checkNotNull(charset)); + } + /** * Sets whether all read/peek methods should enforce that {@link #getPosition()} never exceeds * {@link #limit()}. @@ -795,7 +806,11 @@ public Charset readUtfCharsetFromBom() { * *

Defaults to {@code null}, which means a default value is used (which may change in a later * release). + * + * @deprecated Use {@link Flags#enableFlag(int)} or {@link Flags#disableFlag(int)} with {@link + * Flags#FLAG_ENFORCE_PARSABLE_BYTE_ARRAY_LIMIT} instead. */ + @Deprecated @VisibleForTesting public static void setShouldEnforceLimitOnLegacyMethods(@Nullable Boolean enforceLimit) { ParsableByteArray.shouldEnforceLimitOnLegacyMethods.set(enforceLimit); @@ -977,7 +992,7 @@ && isUtf8ContinuationByte(data[position + 3])) { /** * Enforces that {@link #bytesLeft()} is at least {@code bytesNeeded} if {@link - * #shouldEnforceLimitOnLegacyMethods} is set to {@code true}. + * Flags#FLAG_ENFORCE_PARSABLE_BYTE_ARRAY_LIMIT} is enabled. * *

This should only be called from methods that previously didn't enforce the limit. All new * methods added to this class should unconditionally enforce the limit. @@ -989,7 +1004,8 @@ private void maybeAssertAtLeastBytesLeftForLegacyMethod(int bytesNeeded) { private void maybeAssertAtLeastBytesLeftForLegacyMethod( int bytesNeeded, Function exceptionFactory) { Boolean override = shouldEnforceLimitOnLegacyMethods.get(); - boolean enforceLimit = override != null ? override : IsRunningInTestHolder.IS_RUNNING_IN_TEST; + boolean enforceLimit = + override != null ? override : Flags.isEnabled(Flags.FLAG_ENFORCE_PARSABLE_BYTE_ARRAY_LIMIT); if (enforceLimit) { if (bytesLeft() < bytesNeeded) { throw exceptionFactory.apply("bytesNeeded= " + bytesNeeded + ", bytesLeft=" + bytesLeft()); @@ -1008,17 +1024,4 @@ private static int decodeUtf8CodeUnit(int b1, int b2, int b3, int b4) { UnsignedBytes.checkedCast(((byte) b2 & 0xF) << 4 | ((byte) b3 & 0b0011_1100) >> 2), UnsignedBytes.checkedCast(((byte) b3 & 0x3) << 6 | ((byte) b4 & 0x3F))); } - - private static boolean isRunningInTest() { - if (Ascii.equalsIgnoreCase("robolectric", Build.FINGERPRINT)) { - return true; - } - @Nullable ClassLoader loader = ParsableByteArray.class.getClassLoader(); - return (loader != null - && (loader.getResource("androidx/test/espresso/Espresso.class") != null - || loader.getResource("org/junit/runner/Runner.class") != null - || loader.getResource("androidx/test/platform/app/InstrumentationRegistry.class") - != null)) - || ActivityManager.isRunningInTestHarness(); - } } diff --git a/libraries/common/src/main/java/androidx/media3/common/util/ThrowingRunnable.java b/libraries/common/src/main/java/androidx/media3/common/util/ThrowingRunnable.java new file mode 100644 index 00000000000..c1110fd08d6 --- /dev/null +++ b/libraries/common/src/main/java/androidx/media3/common/util/ThrowingRunnable.java @@ -0,0 +1,29 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.common.util; + +/** + * A {@link Runnable}-like interface that allows throwing checked exceptions. + * + * @param The type of exception that the operation may throw. + */ +@UnstableApi +@FunctionalInterface +public interface ThrowingRunnable { + + /** Runs the operation. */ + void run() throws E; +} diff --git a/libraries/common/src/main/java/androidx/media3/common/util/Util.java b/libraries/common/src/main/java/androidx/media3/common/util/Util.java index 1456fdef55e..88a79da67ee 100644 --- a/libraries/common/src/main/java/androidx/media3/common/util/Util.java +++ b/libraries/common/src/main/java/androidx/media3/common/util/Util.java @@ -234,6 +234,12 @@ public static byte[] toByteArray(InputStream inputStream) throws IOException { return ByteStreams.toByteArray(inputStream); } + /** Converts a single integer into an equivalent 4-byte array in big-endian order. */ + @UnstableApi + public static byte[] toByteArray(int value) { + return Ints.toByteArray(value); + } + /** * Converts an array of integers into an equivalent byte array. * @@ -334,19 +340,20 @@ public static void setForegroundServiceNotification( } /** - * Takes the service off the foreground by safely calling the correct variant of {@code - * Service.stopForeground} on the given API level of the device. - * - * @param service The service to be taken off the foreground. - * @param removeNotification Whether to remove the notification that was attached to the service. + * @deprecated Use {@link Service#stopForeground(int)} instead. */ @UnstableApi + @Deprecated + @InlineMe( + replacement = + "service.stopForeground(removeNotification ? STOP_FOREGROUND_REMOVE :" + + " STOP_FOREGROUND_DETACH)", + staticImports = { + "android.app.Service.STOP_FOREGROUND_DETACH", + "android.app.Service.STOP_FOREGROUND_REMOVE" + }) public static void stopForeground(Service service, boolean removeNotification) { - if (Build.VERSION.SDK_INT >= 24) { - Api24.stopForeground(service, removeNotification); - } else { - service.stopForeground(removeNotification); - } + service.stopForeground(removeNotification ? STOP_FOREGROUND_REMOVE : STOP_FOREGROUND_DETACH); } /** @@ -481,10 +488,6 @@ private static boolean isMediaStoreExternalContentUri(Uri uri) { * @return Whether it may be possible to load the URIs of the given media items. */ public static boolean checkCleartextTrafficPermitted(MediaItem... mediaItems) { - if (Build.VERSION.SDK_INT < 24) { - // We assume cleartext traffic is permitted. - return true; - } for (MediaItem mediaItem : mediaItems) { if (mediaItem.localConfiguration == null) { continue; @@ -2413,6 +2416,25 @@ public static String getCodecsWithoutType(@Nullable String codecs, @C.TrackType return builder.length() > 0 ? builder.toString() : null; } + /** + * Returns the {@link C.VideoCodecFlags} corresponding to the provided MIME type. + * + * @throws IllegalArgumentException If {@code mimeType} is not a recognized video codec. + */ + @UnstableApi + public static @C.VideoCodecFlags int getVideoCodecFlagForMime(String mimeType) { + switch (Ascii.toLowerCase(mimeType)) { + case MimeTypes.VIDEO_H264: + return C.VIDEO_CODEC_FLAG_H264; + case MimeTypes.VIDEO_H265: + return C.VIDEO_CODEC_FLAG_H265; + case MimeTypes.VIDEO_AV1: + return C.VIDEO_CODEC_FLAG_AV1; + default: + throw new IllegalArgumentException("Unsupported video codec MIME type: " + mimeType); + } + } + /** * Splits a codecs sequence string, as defined in RFC 6381, into individual codec strings. * @@ -2647,6 +2669,25 @@ public static boolean isEncodingHighResolutionPcm(@C.PcmEncoding int encoding) { || encoding == C.ENCODING_PCM_DOUBLE_BIG_ENDIAN; } + /** Returns whether {@code encoding} is a floating-point PCM encoding. */ + @UnstableApi + public static boolean isFloatPcmEncoding(@C.PcmEncoding int encoding) { + return encoding == C.ENCODING_PCM_FLOAT + || encoding == C.ENCODING_PCM_FLOAT_BIG_ENDIAN + || encoding == C.ENCODING_PCM_DOUBLE + || encoding == C.ENCODING_PCM_DOUBLE_BIG_ENDIAN; + } + + /** Returns whether {@code encoding} is a big-endian PCM encoding. */ + @UnstableApi + public static boolean isPcmEncodingBigEndian(@C.PcmEncoding int encoding) { + return encoding == C.ENCODING_PCM_16BIT_BIG_ENDIAN + || encoding == C.ENCODING_PCM_24BIT_BIG_ENDIAN + || encoding == C.ENCODING_PCM_32BIT_BIG_ENDIAN + || encoding == C.ENCODING_PCM_FLOAT_BIG_ENDIAN + || encoding == C.ENCODING_PCM_DOUBLE_BIG_ENDIAN; + } + /** * Returns the audio track channel configuration for the given {@link Format}, or {@link * AudioFormat#CHANNEL_INVALID} if output is not possible. @@ -3528,12 +3569,17 @@ public static String[] getSystemLanguageCodes() { return systemLocales; } - /** Returns the default {@link Locale.Category#DISPLAY DISPLAY} {@link Locale}. */ + /** + * @deprecated Use {@link Locale#getDefault(Locale.Category)} with {@link Locale.Category#DISPLAY} + * instead. + */ @UnstableApi + @Deprecated + @InlineMe( + replacement = "Locale.getDefault(Locale.Category.DISPLAY)", + imports = {"java.util.Locale"}) public static Locale getDefaultDisplayLocale() { - return Build.VERSION.SDK_INT >= 24 - ? Locale.getDefault(Locale.Category.DISPLAY) - : Locale.getDefault(); + return Locale.getDefault(Locale.Category.DISPLAY); } /** @@ -4033,7 +4079,9 @@ public static boolean isFrameDropAllowedOnSurfaceInput(Context context) { || context.getApplicationInfo().targetSdkVersion < 29 || ((Build.VERSION.SDK_INT == 30 && (Ascii.equalsIgnoreCase(Build.MODEL, "moto g(20)") + || Ascii.equalsIgnoreCase(Build.MODEL, "moto e20") || Ascii.equalsIgnoreCase(Build.MODEL, "rmx3231"))) + || (Ascii.equalsIgnoreCase(Build.MODEL, "sm-a032f")) || (Build.VERSION.SDK_INT == 34 && Ascii.equalsIgnoreCase(Build.MODEL, "sm-x200"))); } @@ -4369,13 +4417,6 @@ private static void getDisplaySize(Display display, Point outSize) { private static String[] getSystemLocales() { Configuration config = Resources.getSystem().getConfiguration(); - return Build.VERSION.SDK_INT >= 24 - ? getSystemLocalesV24(config) - : new String[] {getLocaleLanguageTag(config.locale)}; - } - - @RequiresApi(24) - private static String[] getSystemLocalesV24(Configuration config) { return split(config.getLocales().toLanguageTags(), ","); } @@ -4431,7 +4472,6 @@ private static boolean requestReadMediaPermissions(Activity activity) { return false; } - @RequiresApi(api = 24) private static boolean isTrafficRestricted(Uri uri) { return "http".equals(uri.getScheme()) && !NetworkSecurityPolicy.getInstance() @@ -4603,16 +4643,6 @@ public static void ignoreFuture(Future unused) { 0xF3 }; - @RequiresApi(24) - private static class Api24 { - - private static void stopForeground(Service service, boolean removeNotification) { - service.stopForeground(removeNotification ? STOP_FOREGROUND_REMOVE : STOP_FOREGROUND_DETACH); - } - - private Api24() {} - } - @RequiresApi(26) private static class Api26 { private static void createNotificationChannel( diff --git a/libraries/common/src/main/java/androidx/media3/common/video/DefaultHardwareBufferFrame.java b/libraries/common/src/main/java/androidx/media3/common/video/DefaultHardwareBufferFrame.java index 590dc4dc06c..f10a0d392b7 100644 --- a/libraries/common/src/main/java/androidx/media3/common/video/DefaultHardwareBufferFrame.java +++ b/libraries/common/src/main/java/androidx/media3/common/video/DefaultHardwareBufferFrame.java @@ -25,28 +25,50 @@ import com.google.common.collect.ImmutableMap; import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.util.Map; +import java.util.concurrent.Executor; /** Default implementation of {@link HardwareBufferFrame}. */ @RestrictTo(Scope.LIBRARY_GROUP) -public final class DefaultHardwareBufferFrame implements HardwareBufferFrame { +public final class DefaultHardwareBufferFrame implements HardwareBufferFrame, ReferenceCounter { /** Concrete implementation of {@link HardwareBufferFrame.Builder}. */ - public static final class Builder implements HardwareBufferFrame.Builder { + public static final class Builder + implements HardwareBufferFrame.Builder, ReferenceCounter.Builder { private final HardwareBuffer hardwareBuffer; + private final SharedStateReferenceCounter.Builder delegateCounterBuilder; private Format format; private ImmutableMap metadata; private long contentTimeUs; @Nullable private Object internalImage; /** - * Creates a builder for {@link HardwareBufferFrame} instances. + * Creates a builder for {@link HardwareBufferFrame} instances without lifecycle management. * * @param hardwareBuffer The {@link HardwareBuffer} that backs the frame. */ @RequiresApi(26) public Builder(HardwareBuffer hardwareBuffer) { this.hardwareBuffer = hardwareBuffer; + this.delegateCounterBuilder = new SharedStateReferenceCounter.Builder(); + this.format = new Format.Builder().build(); + this.metadata = ImmutableMap.of(); + this.contentTimeUs = C.TIME_UNSET; + } + + /** + * Creates a builder for reference-counted {@link HardwareBufferFrame} instances. + * + * @param hardwareBuffer The {@link HardwareBuffer} that backs the frame. + * @param releaseExecutor The {@link Executor} on which {@code releaseCallback} is called. + * @param releaseCallback The callback invoked when the frame is fully released. + */ + @RequiresApi(26) + public Builder( + HardwareBuffer hardwareBuffer, Executor releaseExecutor, ReleaseCallback releaseCallback) { + this.hardwareBuffer = hardwareBuffer; + this.delegateCounterBuilder = + new SharedStateReferenceCounter.Builder(releaseExecutor, releaseCallback); this.format = new Format.Builder().build(); this.metadata = ImmutableMap.of(); this.contentTimeUs = C.TIME_UNSET; @@ -54,12 +76,21 @@ public Builder(HardwareBuffer hardwareBuffer) { private Builder(DefaultHardwareBufferFrame frame) { this.hardwareBuffer = frame.hardwareBuffer; + this.delegateCounterBuilder = + new SharedStateReferenceCounter.Builder(frame.delegateCounter.getSharedState()); this.format = frame.format; this.metadata = frame.metadata; this.contentTimeUs = frame.contentTimeUs; this.internalImage = frame.internalImage; } + @CanIgnoreReturnValue + @Override + public DefaultHardwareBufferFrame.Builder shouldIncrementReferenceCount() { + delegateCounterBuilder.shouldIncrementReferenceCount(); + return this; + } + @Override public DefaultHardwareBufferFrame build() { return new DefaultHardwareBufferFrame(this); @@ -97,6 +128,7 @@ public DefaultHardwareBufferFrame.Builder setInternalImage(@Nullable Object inte private final ImmutableMap metadata; private final long contentTimeUs; @Nullable private final Object internalImage; + private final SharedStateReferenceCounter delegateCounter; /** Private constructor used by the builder. */ private DefaultHardwareBufferFrame(Builder builder) { @@ -105,6 +137,7 @@ private DefaultHardwareBufferFrame(Builder builder) { this.metadata = builder.metadata; this.contentTimeUs = builder.contentTimeUs; this.internalImage = builder.internalImage; + this.delegateCounter = builder.delegateCounterBuilder.build(); } @Override @@ -129,10 +162,17 @@ public HardwareBuffer getHardwareBuffer() { } @Override - public HardwareBufferFrame.Builder buildUpon() { + public DefaultHardwareBufferFrame.Builder buildUpon() { + delegateCounter.checkNotReleased( + "Cannot buildUpon a DefaultHardwareBufferFrame that has already been released."); return new Builder(this); } + @Override + public void release(@Nullable SyncFenceWrapper releaseFence) { + delegateCounter.release(releaseFence); + } + @Nullable public Object getInternalImage() { return internalImage; diff --git a/libraries/common/src/main/java/androidx/media3/common/video/DefaultImagePlanesFrame.java b/libraries/common/src/main/java/androidx/media3/common/video/DefaultImagePlanesFrame.java new file mode 100644 index 00000000000..707338d609f --- /dev/null +++ b/libraries/common/src/main/java/androidx/media3/common/video/DefaultImagePlanesFrame.java @@ -0,0 +1,222 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.common.video; + +import static com.google.common.base.Preconditions.checkArgument; +import static com.google.common.base.Preconditions.checkNotNull; + +import androidx.annotation.Nullable; +import androidx.annotation.RestrictTo; +import androidx.annotation.RestrictTo.Scope; +import androidx.media3.common.C; +import androidx.media3.common.Format; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import com.google.errorprone.annotations.CanIgnoreReturnValue; +import java.nio.ByteBuffer; +import java.util.List; +import java.util.Map; +import java.util.concurrent.Executor; + +/** + * Default implementation of {@link ImagePlanesFrame} with {@link ReferenceCounter} lifecycle + * management. + */ +@RestrictTo(Scope.LIBRARY_GROUP) +public final class DefaultImagePlanesFrame implements ImagePlanesFrame, ReferenceCounter { + + /** An implementation of {@link ImagePlanesFrame.Plane}. */ + public static final class DefaultPlane implements ImagePlanesFrame.Plane { + private final ByteBuffer buffer; + private final int rowStride; + private final int pixelStride; + + /** + * Creates an instance. + * + * @param buffer The {@linkplain ByteBuffer#isDirect() direct} {@link ByteBuffer} containing the + * plane data. + * @param rowStride The positive row stride of the plane, in bytes. + * @param pixelStride The positive pixel stride of the plane, in bytes. + */ + public DefaultPlane(ByteBuffer buffer, int rowStride, int pixelStride) { + this.buffer = checkNotNull(buffer); + checkArgument(buffer.isDirect(), "Plane buffer must be a direct ByteBuffer."); + checkArgument(rowStride > 0, "rowStride must be positive: %s", rowStride); + checkArgument(pixelStride > 0, "pixelStride must be positive: %s", pixelStride); + this.rowStride = rowStride; + this.pixelStride = pixelStride; + } + + @Override + public ByteBuffer getBuffer() { + return buffer; + } + + @Override + public int getRowStride() { + return rowStride; + } + + @Override + public int getPixelStride() { + return pixelStride; + } + } + + /** A builder for {@link DefaultImagePlanesFrame}. */ + public static final class Builder implements ImagePlanesFrame.Builder, ReferenceCounter.Builder { + + private final ImmutableList planes; + private final SharedStateReferenceCounter.Builder delegateCounterBuilder; + private Format format; + private ImmutableMap metadata; + private long contentTimeUs; + @Nullable private Object internalImage; + + /** + * Creates a builder for {@link ImagePlanesFrame} instances without lifecycle management. + * + * @param planes The {@link Plane} instances backing this frame. + */ + public Builder(List planes) { + checkArgument(!planes.isEmpty(), "planes cannot be empty"); + this.planes = ImmutableList.copyOf(planes); + this.delegateCounterBuilder = new SharedStateReferenceCounter.Builder(); + this.format = new Format.Builder().build(); + this.metadata = ImmutableMap.of(); + this.contentTimeUs = C.TIME_UNSET; + } + + /** + * Creates a builder for reference-counted {@link ImagePlanesFrame} instances. + * + * @param planes The {@link Plane} instances backing this frame. + * @param releaseExecutor The {@link Executor} on which {@code releaseCallback} is called. + * @param releaseCallback The callback invoked when the frame is fully released. + */ + public Builder(List planes, Executor releaseExecutor, ReleaseCallback releaseCallback) { + checkArgument(!planes.isEmpty(), "planes cannot be empty"); + this.planes = ImmutableList.copyOf(planes); + this.delegateCounterBuilder = + new SharedStateReferenceCounter.Builder(releaseExecutor, releaseCallback); + this.format = new Format.Builder().build(); + this.metadata = ImmutableMap.of(); + this.contentTimeUs = C.TIME_UNSET; + } + + private Builder(DefaultImagePlanesFrame frame) { + this.planes = frame.planes; + this.delegateCounterBuilder = + new SharedStateReferenceCounter.Builder(frame.delegateCounter.getSharedState()); + this.format = frame.format; + this.metadata = frame.metadata; + this.contentTimeUs = frame.contentTimeUs; + this.internalImage = frame.internalImage; + } + + @CanIgnoreReturnValue + @Override + public DefaultImagePlanesFrame.Builder shouldIncrementReferenceCount() { + delegateCounterBuilder.shouldIncrementReferenceCount(); + return this; + } + + @Override + public DefaultImagePlanesFrame build() { + return new DefaultImagePlanesFrame(this); + } + + @CanIgnoreReturnValue + @Override + public DefaultImagePlanesFrame.Builder setMetadata(Map metadata) { + this.metadata = ImmutableMap.copyOf(metadata); + return this; + } + + @CanIgnoreReturnValue + @Override + public DefaultImagePlanesFrame.Builder setContentTimeUs(long contentTimeUs) { + this.contentTimeUs = contentTimeUs; + return this; + } + + @CanIgnoreReturnValue + public DefaultImagePlanesFrame.Builder setFormat(Format format) { + this.format = format; + return this; + } + + @CanIgnoreReturnValue + public DefaultImagePlanesFrame.Builder setInternalImage(@Nullable Object internalImage) { + this.internalImage = internalImage; + return this; + } + } + + private final ImmutableList planes; + private final Format format; + private final ImmutableMap metadata; + private final long contentTimeUs; + @Nullable private final Object internalImage; + private final SharedStateReferenceCounter delegateCounter; + + private DefaultImagePlanesFrame(Builder builder) { + this.planes = builder.planes; + this.format = builder.format; + this.metadata = builder.metadata; + this.contentTimeUs = builder.contentTimeUs; + this.internalImage = builder.internalImage; + this.delegateCounter = builder.delegateCounterBuilder.build(); + } + + @Override + public Format getFormat() { + return format; + } + + @Override + public ImmutableMap getMetadata() { + return metadata; + } + + @Override + public long getContentTimeUs() { + return contentTimeUs; + } + + @Override + public ImmutableList getPlanes() { + return planes; + } + + @Override + public DefaultImagePlanesFrame.Builder buildUpon() { + delegateCounter.checkNotReleased( + "Cannot buildUpon a DefaultImagePlanesFrame that has already been released."); + return new Builder(this); + } + + @Override + public void release(@Nullable SyncFenceWrapper releaseFence) { + delegateCounter.release(releaseFence); + } + + @Nullable + public Object getInternalImage() { + return internalImage; + } +} diff --git a/libraries/common/src/main/java/androidx/media3/common/video/FrameSharedState.java b/libraries/common/src/main/java/androidx/media3/common/video/FrameSharedState.java new file mode 100644 index 00000000000..99f1c0af227 --- /dev/null +++ b/libraries/common/src/main/java/androidx/media3/common/video/FrameSharedState.java @@ -0,0 +1,134 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.common.video; + +import static com.google.common.base.Preconditions.checkState; + +import androidx.annotation.GuardedBy; +import androidx.annotation.Nullable; +import androidx.media3.common.util.Log; +import androidx.media3.common.util.Util; +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.Executor; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.RejectedExecutionException; + +/** + * Holds the shared reference count and release state across retained {@link ReferenceCounter} + * instances. + * + *

Separating the shared state from individual {@link ReferenceCounter} handles allows each + * instance to track its own released state locally, so that an individual instance can be safely + * released multiple times without decrementing the global reference count more than once. + */ +/* package */ final class FrameSharedState { + + private static final String TAG = "FrameSharedState"; + private static final long RELEASE_TIMEOUT_MS = 2000; + private static final ExecutorService FENCE_WAIT_EXECUTOR = + Util.newSingleThreadExecutor("FrameSharedState:FenceWaitThread"); + + private final Object lock; + private final ReleaseCallback callback; + private final Executor executor; + + @GuardedBy("lock") + private int refCount; + + @GuardedBy("lock") + private final List releaseFences; + + /* package */ FrameSharedState(ReleaseCallback callback, Executor executor) { + this.lock = new Object(); + this.callback = callback; + this.executor = executor; + this.refCount = 1; + this.releaseFences = new ArrayList<>(); + } + + /** Increments the reference count. */ + /* package */ void retain() { + synchronized (lock) { + checkState(refCount > 0, "retain() called on a released FrameSharedState."); + refCount++; + } + } + + /** + * Releases a reference to the shared state. + * + * @param releaseFence An optional {@link SyncFenceWrapper} that must signal before the underlying + * resources can be released. + */ + /* package */ void release(@Nullable SyncFenceWrapper releaseFence) { + List fencesToWaitOn; + + synchronized (lock) { + if (refCount <= 0) { + if (releaseFence != null) { + releaseFence.close(); + } + return; + } + + if (releaseFence != null) { + releaseFences.add(releaseFence); + } + if (--refCount != 0) { + return; + } + + fencesToWaitOn = new ArrayList<>(releaseFences); + releaseFences.clear(); + } + + if (fencesToWaitOn.isEmpty()) { + executor.execute(() -> callback.release(/* releaseFence= */ null)); + return; + } + + // TODO(b/540859379): Merge fences if possible and forward to frame owner for buffer reuse. + try { + FENCE_WAIT_EXECUTOR.execute( + () -> { + try { + for (int i = 0; i < fencesToWaitOn.size(); i++) { + if (!fencesToWaitOn.get(i).awaitMs(RELEASE_TIMEOUT_MS)) { + Log.w(TAG, "Timed out waiting for release fence to signal."); + } + } + } finally { + closeAll(fencesToWaitOn); + try { + executor.execute(() -> callback.release(/* releaseFence= */ null)); + } catch (RejectedExecutionException e) { + // Executor is shut down, ignore. + } + } + }); + } catch (RejectedExecutionException e) { + closeAll(fencesToWaitOn); + throw e; + } + } + + private static void closeAll(List fences) { + for (int i = 0; i < fences.size(); i++) { + fences.get(i).close(); + } + } +} diff --git a/libraries/common/src/main/java/androidx/media3/common/video/HardwareBufferNativeHelpers.java b/libraries/common/src/main/java/androidx/media3/common/video/HardwareBufferNativeHelpers.java index 4255616bbc5..bf9093cfd6c 100644 --- a/libraries/common/src/main/java/androidx/media3/common/video/HardwareBufferNativeHelpers.java +++ b/libraries/common/src/main/java/androidx/media3/common/video/HardwareBufferNativeHelpers.java @@ -21,7 +21,6 @@ import androidx.media3.common.util.ExperimentalApi; /** Wrapper for {@link HardwareBuffer} JNI methods. */ -@RequiresApi(26) @ExperimentalApi // TODO: b/498176910 - Remove once Frame is production ready. public interface HardwareBufferNativeHelpers { @@ -40,6 +39,7 @@ public interface HardwareBufferNativeHelpers { * @param hb The destination hardware buffer. * @return Whether the copy is successful. */ + @RequiresApi(26) boolean nativeCopyBitmapToHardwareBuffer(Bitmap bitmap, HardwareBuffer hb); /** @@ -56,5 +56,6 @@ public interface HardwareBufferNativeHelpers { * @param dstHb The destination hardware buffer. * @return Whether the copy is successful. */ + @RequiresApi(26) boolean nativeCopyHardwareBufferToHardwareBuffer(HardwareBuffer srcHb, HardwareBuffer dstHb); } diff --git a/libraries/common/src/main/java/androidx/media3/common/video/HardwareBufferPool.java b/libraries/common/src/main/java/androidx/media3/common/video/HardwareBufferPool.java index 2aa8a85ff86..52ac7d02dbd 100644 --- a/libraries/common/src/main/java/androidx/media3/common/video/HardwareBufferPool.java +++ b/libraries/common/src/main/java/androidx/media3/common/video/HardwareBufferPool.java @@ -15,6 +15,7 @@ */ package androidx.media3.common.video; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkState; @@ -22,9 +23,9 @@ import androidx.annotation.GuardedBy; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; +import androidx.annotation.RestrictTo; import androidx.media3.common.ColorInfo; import androidx.media3.common.Format; -import androidx.media3.common.util.ExperimentalApi; import java.util.ArrayDeque; import java.util.Queue; @@ -37,7 +38,7 @@ * *

Methods can be called from any thread. */ -@ExperimentalApi // TODO: b/498176910 - Remove once Frame is production ready. +@RestrictTo(LIBRARY_GROUP) @RequiresApi(26) public final class HardwareBufferPool { diff --git a/libraries/common/src/main/java/androidx/media3/common/video/ImagePlanesFrame.java b/libraries/common/src/main/java/androidx/media3/common/video/ImagePlanesFrame.java new file mode 100644 index 00000000000..e9bdf11c9bc --- /dev/null +++ b/libraries/common/src/main/java/androidx/media3/common/video/ImagePlanesFrame.java @@ -0,0 +1,85 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.common.video; + +import androidx.annotation.RestrictTo; +import androidx.annotation.RestrictTo.Scope; +import androidx.media3.common.C; +import androidx.media3.common.Format; +import com.google.common.collect.ImmutableList; +import com.google.errorprone.annotations.CanIgnoreReturnValue; +import java.nio.ByteBuffer; +import java.util.Map; + +/** + * A {@link Frame} backed by CPU image planes (e.g. planar/semi-planar YUV or single-plane RGB/RGBA + * from {@link android.media.Image} or {@link ByteBuffer} instances). + * + *

The pixel format of the planes is specified by {@link Format#pixelFormat} on the frame's + * {@linkplain #getFormat() format}. + */ +// TODO: b/531653682 - Remove LIBRARY_GROUP restriction when ImagePlanesFrame pipeline is ready. +@RestrictTo(Scope.LIBRARY_GROUP) +public interface ImagePlanesFrame extends Frame { + + /** Represents a single data plane of the frame. */ + interface Plane { + /** + * Returns the {@link ByteBuffer} containing the plane data. + * + *

The returned buffer is guaranteed to be a {@linkplain ByteBuffer#isDirect() direct} {@link + * ByteBuffer}. + */ + ByteBuffer getBuffer(); + + /** Returns the row stride of the plane, in bytes. */ + int getRowStride(); + + /** Returns the pixel stride of the plane, in bytes. */ + int getPixelStride(); + } + + /** A Builder for {@link ImagePlanesFrame} instances. */ + interface Builder { + /** + * Sets the {@linkplain Frame#getMetadata() metadata} associated with the frame. + * + * @param metadata The metadata to associate with this frame. + * @return This builder. + */ + @CanIgnoreReturnValue + Builder setMetadata(Map metadata); + + /** + * Sets the {@linkplain Frame#getContentTimeUs() content time} of the frame. + * + * @param contentTimeUs The time of the frame in the context of the input media, in + * microseconds, or {@link C#TIME_UNSET} if unset. + * @return This builder. + */ + @CanIgnoreReturnValue + Builder setContentTimeUs(long contentTimeUs); + + /** Builds an {@link ImagePlanesFrame} instance. */ + ImagePlanesFrame build(); + } + + /** Returns the planes backing this frame. */ + ImmutableList getPlanes(); + + /** Returns a {@link Builder} initialized with the values of this instance. */ + Builder buildUpon(); +} diff --git a/libraries/common/src/main/java/androidx/media3/common/video/ReferenceCounter.java b/libraries/common/src/main/java/androidx/media3/common/video/ReferenceCounter.java new file mode 100644 index 00000000000..5be8804a49a --- /dev/null +++ b/libraries/common/src/main/java/androidx/media3/common/video/ReferenceCounter.java @@ -0,0 +1,46 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.common.video; + +import androidx.annotation.Nullable; +import androidx.annotation.RestrictTo; +import androidx.annotation.RestrictTo.Scope; +import com.google.errorprone.annotations.CanIgnoreReturnValue; + +/** Interface with lifecycle management and reference counting. */ +@RestrictTo(Scope.LIBRARY_GROUP) +public interface ReferenceCounter { + + /** A builder for objects with lifecycle management and reference counting. */ + interface Builder { + + /** + * Increments the reference count of the underlying resources when the object is built. + * + * @return This builder. + */ + @CanIgnoreReturnValue + Builder shouldIncrementReferenceCount(); + } + + /** + * Releases this handle to the underlying storage. + * + * @param releaseFence A {@link SyncFenceWrapper} that must signal before the underlying resources + * can be fully released, or {@code null} if the resources can be released immediately. + */ + void release(@Nullable SyncFenceWrapper releaseFence); +} diff --git a/libraries/effect/src/main/java/androidx/media3/effect/HardwareBufferFrameProcessor.java b/libraries/common/src/main/java/androidx/media3/common/video/ReleaseCallback.java similarity index 50% rename from libraries/effect/src/main/java/androidx/media3/effect/HardwareBufferFrameProcessor.java rename to libraries/common/src/main/java/androidx/media3/common/video/ReleaseCallback.java index 883ce02dd00..7ecae301ecb 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/HardwareBufferFrameProcessor.java +++ b/libraries/common/src/main/java/androidx/media3/common/video/ReleaseCallback.java @@ -13,28 +13,21 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package androidx.media3.effect; +package androidx.media3.common.video; +import androidx.annotation.Nullable; import androidx.annotation.RestrictTo; import androidx.annotation.RestrictTo.Scope; -/** - * Processes {@link HardwareBufferFrame} instances. - * - *

This interface should only be used to improve backwards compatibility with older API versions, - * and is not intended for application use. - */ +/** A callback invoked when a reference-counted resource is fully released. */ @RestrictTo(Scope.LIBRARY_GROUP) -public interface HardwareBufferFrameProcessor extends AutoCloseable { +public interface ReleaseCallback { /** - * Converts the {@code inputFrame} to a new {@link HardwareBufferFrame}. - * - *

The implementation may return the {@code inputFrame} itself if no conversion is needed, or a - * new frame backed by a {@link android.hardware.HardwareBuffer}. + * Releases the underlying resources. * - * @param inputFrame The input {@link HardwareBufferFrame}. - * @return The processed {@link HardwareBufferFrame}. + * @param releaseFence A {@link SyncFenceWrapper} that must signal when resources are no longer + * used by hardware units, or {@code null} if the resources can be released immediately. */ - HardwareBufferFrame process(HardwareBufferFrame inputFrame); + void release(@Nullable SyncFenceWrapper releaseFence); } diff --git a/libraries/common/src/main/java/androidx/media3/common/video/SharedStateReferenceCounter.java b/libraries/common/src/main/java/androidx/media3/common/video/SharedStateReferenceCounter.java new file mode 100644 index 00000000000..162d7262dfc --- /dev/null +++ b/libraries/common/src/main/java/androidx/media3/common/video/SharedStateReferenceCounter.java @@ -0,0 +1,172 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.common.video; + +import static com.google.common.base.Preconditions.checkNotNull; +import static com.google.common.base.Preconditions.checkState; + +import androidx.annotation.GuardedBy; +import androidx.annotation.Nullable; +import com.google.errorprone.annotations.CanIgnoreReturnValue; +import java.util.concurrent.Executor; + +/** + * Implementation of {@link ReferenceCounter} that tracks the release state of an individual handle + * and delegates resource release to a {@link FrameSharedState}. + * + *

Instances of this class are thread-safe. + */ +/* package */ final class SharedStateReferenceCounter implements ReferenceCounter { + + /** A builder for {@link SharedStateReferenceCounter} instances. */ + /* package */ static final class Builder implements ReferenceCounter.Builder { + + @Nullable private final FrameSharedState sharedState; + private boolean shouldIncrementReferenceCount; + + /** + * Creates a builder for an unmanaged {@link SharedStateReferenceCounter}. + * + *

Unmanaged frames are necessary to avoid Garbage Collector (GC) churn. This constructor + * allows creating unmanaged frames, where {@link #shouldIncrementReferenceCount()} is not + * needed and calling it fails fast because there is no underlying pool of resources to retain. + */ + /* package */ Builder() { + this.sharedState = null; + } + + /** + * Creates a builder for a managed {@link SharedStateReferenceCounter}. + * + * @param releaseExecutor The {@link Executor} on which {@code releaseCallback} is called. + * @param releaseCallback The callback invoked when the frame is fully released. + */ + /* package */ Builder(Executor releaseExecutor, ReleaseCallback releaseCallback) { + this.sharedState = + new FrameSharedState(checkNotNull(releaseCallback), checkNotNull(releaseExecutor)); + } + + /** + * Creates a builder for copying a {@link SharedStateReferenceCounter}. + * + *

By default, the copied instance shares the underlying {@link FrameSharedState} without + * incrementing its reference count. Call {@link #shouldIncrementReferenceCount()} before {@link + * #build()} to increment the reference count for the newly built instance. + * + * @param sharedState The {@link FrameSharedState} to share, or {@code null} if unmanaged. + */ + /* package */ Builder(@Nullable FrameSharedState sharedState) { + this.sharedState = sharedState; + } + + /** + * {@inheritDoc} + * + *

Configures this builder so that the reference count of the underlying resource is + * incremented when {@link #build()} is invoked. + * + * @throws IllegalStateException If called on an unmanaged builder. + */ + @CanIgnoreReturnValue + @Override + public Builder shouldIncrementReferenceCount() { + checkState(sharedState != null, "Cannot increment reference count on an unmanaged frame."); + this.shouldIncrementReferenceCount = true; + return this; + } + + /** Builds the {@link SharedStateReferenceCounter}. */ + /* package */ SharedStateReferenceCounter build() { + if (shouldIncrementReferenceCount) { + checkNotNull(sharedState).retain(); + } + return new SharedStateReferenceCounter(this); + } + } + + private final Object lock; + @Nullable private final FrameSharedState sharedState; + + @GuardedBy("lock") + private boolean isReleased; + + private SharedStateReferenceCounter(Builder builder) { + this.lock = new Object(); + this.sharedState = builder.sharedState; + this.isReleased = false; + } + + /** + * Checks that the frame has not been released. + * + * @param errorMessage The exception message to use if the frame has been released. + * @throws IllegalStateException If the frame has been released. + */ + /* package */ void checkNotReleased(String errorMessage) { + synchronized (lock) { + checkState(!isReleased, errorMessage); + } + } + + /** Returns whether the frame has been released. */ + /* package */ boolean isReleased() { + synchronized (lock) { + return isReleased; + } + } + + /** + * Releases this frame handle. + * + *

If this handle has already been released, {@code releaseFence} is closed silently. + * Otherwise, the release is delegated to {@link FrameSharedState} (if present), or {@code + * releaseFence} is closed silently (if unmanaged). + * + * @param releaseFence An optional {@link SyncFenceWrapper} that must signal before the underlying + * resources can be fully released, or {@code null} if the resources can be released + * immediately. + */ + @Override + public void release(@Nullable SyncFenceWrapper releaseFence) { + boolean wasAlreadyReleased; + synchronized (lock) { + wasAlreadyReleased = isReleased; + isReleased = true; + } + if (wasAlreadyReleased) { + closeFenceSilently(releaseFence); + return; + } + if (sharedState != null) { + sharedState.release(releaseFence); + } else { + closeFenceSilently(releaseFence); + } + } + + /** Returns the {@link FrameSharedState}, or {@code null} if unmanaged. */ + @Nullable + /* package */ FrameSharedState getSharedState() { + return sharedState; + } + + private static void closeFenceSilently(@Nullable SyncFenceWrapper fence) { + if (fence == null) { + return; + } + fence.close(); + } +} diff --git a/libraries/common/src/main/java/androidx/media3/common/video/SurfaceHolderFrameWriter.java b/libraries/common/src/main/java/androidx/media3/common/video/SurfaceHolderFrameWriter.java index b2a894bcef6..f4f07811435 100644 --- a/libraries/common/src/main/java/androidx/media3/common/video/SurfaceHolderFrameWriter.java +++ b/libraries/common/src/main/java/androidx/media3/common/video/SurfaceHolderFrameWriter.java @@ -22,6 +22,7 @@ import android.graphics.Bitmap; import android.graphics.Bitmap.Config; +import android.graphics.ImageFormat; import android.graphics.PixelFormat; import android.hardware.DataSpace; import android.hardware.HardwareBuffer; @@ -37,13 +38,15 @@ import androidx.media3.common.Format; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.ExperimentalApi; +import androidx.media3.common.util.Log; +import androidx.media3.common.video.DefaultImagePlanesFrame.DefaultPlane; import androidx.media3.common.video.HardwareBufferPool.HardwareBufferWithFence; +import com.google.common.collect.ImmutableList; import java.io.IOException; import java.nio.ByteBuffer; import java.util.concurrent.Executor; /** A {@link FrameWriter} that outputs frames to a {@link SurfaceHolder}. */ -@RequiresApi(28) @ExperimentalApi // TODO: b/498176910 Remove once FrameWriter is production ready. public final class SurfaceHolderFrameWriter implements FrameWriter, SurfaceHolder.Callback { @@ -71,6 +74,8 @@ public interface Listener { void onEnded(); } + private static final String TAG = "SHFrameWriter"; + private static final int CAPACITY = 2; private final Object lock; @@ -114,12 +119,12 @@ public interface Listener { private final Executor listenerExecutor; - private final HardwareBufferPool hardwareBufferPool; + @Nullable private final HardwareBufferPool hardwareBufferPool; @Nullable private final HardwareBufferNativeHelpers hardwareBufferNativeHelpers; /** - * Creates a new instance that returns video frames that are directly dequeued from an {@link - * ImageWriter} linked to the output {@link SurfaceHolder#getSurface()}. + * Creates a new instance that returns {@link HardwareBufferFrame}s that are directly dequeued + * from an {@link ImageWriter} linked to the output {@link SurfaceHolder#getSurface()}. * * @param surfaceHolder The {@link SurfaceHolder} to which frames will be written, or {@code null} * if not yet available. @@ -143,9 +148,21 @@ public static SurfaceHolderFrameWriter create( } /** - * Creates a new instance, that will allocate intermediate {@link HardwareBufferFrame}s that are - * filled by the caller, and are then copied to {@link HardwareBuffer}s dequeued from an {@link - * ImageWriter} linked to the output {@link SurfaceHolder#getSurface()}. + * Creates a new instance that returns video frames that will be written to the output {@link + * SurfaceHolder#getSurface()}. + * + *

Below API 28, {@link #dequeueInputFrame} returns an {@link ImagePlanesFrame} wrapping an + * {@link ImageWriter} buffer without intermediate copying, and {@code + * hardwareBufferNativeHelpers} is ignored. + * + *

On API 28 to 32 (inclusive), {@link #dequeueInputFrame} returns a pre-allocated {@link + * HardwareBufferFrame}. It is copied to the {@link ImageWriter} (via {@link + * HardwareBufferNativeHelpers}) after the caller fills and {@linkplain #queueInputFrame queues} + * it. + * + *

On API 33 and above, {@link #dequeueInputFrame} returns a {@link HardwareBufferFrame} + * directly dequeued from the {@link ImageWriter} without intermediate copying, and {@code + * hardwareBufferNativeHelpers} is ignored. * * @param surfaceHolder The {@link SurfaceHolder} to which frames will be written, or {@code null} * if not yet available. @@ -155,7 +172,7 @@ public static SurfaceHolderFrameWriter create( * @param listenerExecutor The {@link Executor} on which the listener methods will be called. * @param hardwareBufferNativeHelpers The {@link HardwareBufferNativeHelpers} used to copy * intermediate {@link HardwareBuffer}s to {@link HardwareBuffer}s dequeued from {@link - * ImageWriter}. + * ImageWriter} on API 28 to 32; ignored on other API levels. */ public static SurfaceHolderFrameWriter create( @Nullable SurfaceHolder surfaceHolder, @@ -186,7 +203,7 @@ private SurfaceHolderFrameWriter( this.listener = listener; this.listenerExecutor = listenerExecutor; this.hardwareBufferNativeHelpers = hardwareBufferNativeHelpers; - this.hardwareBufferPool = new HardwareBufferPool(CAPACITY); + this.hardwareBufferPool = SDK_INT >= 28 ? new HardwareBufferPool(CAPACITY) : null; if (surfaceHolder != null) { surfaceHolder.addCallback(this); } @@ -203,7 +220,7 @@ public void setSurfaceHolder(@Nullable SurfaceHolder surfaceHolder) { this.surfaceHolder.removeCallback(this); } this.surfaceHolder = surfaceHolder; - if (lastQueuedHardwareBuffer != null) { + if (SDK_INT >= 28 && lastQueuedHardwareBuffer != null) { lastQueuedHardwareBuffer.close(); lastQueuedHardwareBuffer = null; } @@ -226,25 +243,43 @@ public void setSurfaceHolder(@Nullable SurfaceHolder surfaceHolder) { @Override public Info getInfo() { + // PQ SurfaceView output is supported from API 33, but HLG output is supported from API 34. + // By returning false for HLG on API < 34, we signal the pipeline to fall back (e.g., converting + // HLG to PQ) so that HLG input can still be displayed. return (format, usageFlags) -> format.colorInfo != null - && (!ColorInfo.isTransferHdr(format.colorInfo) || SDK_INT >= 33) && format.width > 0 - && format.height > 0; + && format.height > 0 + && isPixelFormatSupported(format.pixelFormat) + && (!ColorInfo.isTransferHdr(format.colorInfo) + || (format.colorInfo.colorTransfer == C.COLOR_TRANSFER_HLG + ? SDK_INT >= 34 + : SDK_INT >= 33)); } + /** + * {@inheritDoc} + * + *

If {@link Format#pixelFormat} is unset (i.e. {@link Format#NO_VALUE}), it defaults to {@link + * ImageFormat#YV12} below API 28, {@link HardwareBuffer#RGBA_1010102} for HDR on API 28+, or + * {@link HardwareBuffer#RGBA_8888} for SDR on API 28+. + */ @Override public void configure(Format format, @Frame.Usage long usage) { checkArgument(format.colorInfo != null); checkArgument(format.width > 0); checkArgument(format.height > 0); + checkArgument( + isPixelFormatSupported(format.pixelFormat), + "Only ImageFormat.YV12 or Format.NO_VALUE is supported below API 28, but got: %s", + format.pixelFormat); synchronized (lock) { if (format.equals(currentFormat) && usage == currentUsageFlags) { return; } currentFormat = format; currentUsageFlags = usage; - if (lastQueuedHardwareBuffer != null) { + if (SDK_INT >= 28 && lastQueuedHardwareBuffer != null) { lastQueuedHardwareBuffer.close(); lastQueuedHardwareBuffer = null; } @@ -276,45 +311,12 @@ public AsyncFrame dequeueInputFrame(Executor wakeupExecutor, Runnable wakeupList if (imageWriter != null && !isSurfaceChangeRequested) { try { - // On API 33+ the usage flags can be directly set on ImageWriter, so the dequeued image - // can be directly written to by the caller. - // On API < 33, if no specific usage flags are requested, or CPU usage only is requested, - // we can also use the dequeued buffer directly. - if (SDK_INT >= 33 - || currentUsageFlags == 0 - || currentUsageFlags == HardwareBuffer.USAGE_CPU_WRITE_OFTEN) { - InternalImage internalImage = dequeueInternalImage(/* needsCopy= */ false); - HardwareBuffer hardwareBuffer = checkNotNull(internalImage.image.getHardwareBuffer()); - DefaultHardwareBufferFrame frame = - new DefaultHardwareBufferFrame.Builder(hardwareBuffer) - .setInternalImage(internalImage) - .setFormat(currentFormat) - .build(); - return new AsyncFrame(frame, null); + if (SDK_INT < 28) { + return dequeueImagePlanesFrame(currentFormat); } - checkState(hardwareBufferNativeHelpers != null); - // On API <33, create an intermediate frame with usage equal to the requested usage, that - // can also be read by the CPU. This will be filled by the caller, then copied into the - // buffer dequeued from ImageWriter via the JNI. - @Nullable - HardwareBufferWithFence bufferWithFence = - hardwareBufferPool.get( - currentFormat, - currentUsageFlags | Frame.USAGE_CPU_READ_OFTEN, - () -> wakeupExecutor.execute(wakeupListener)); - // The pool is at capacity, wakeupListener will be invoked when there is an available - // buffer. - if (bufferWithFence == null) { - return null; - } - InternalImage internalImage = dequeueInternalImage(/* needsCopy= */ true); - DefaultHardwareBufferFrame frame = - new DefaultHardwareBufferFrame.Builder(bufferWithFence.hardwareBuffer) - .setInternalImage(internalImage) - .setFormat(currentFormat) - .build(); - return new AsyncFrame(frame, bufferWithFence.acquireFence); + return dequeueHardwareBufferFrame( + currentFormat, currentUsageFlags, wakeupExecutor, wakeupListener); } catch (IllegalStateException e) { listenerExecutor.execute(() -> listener.onError(new VideoFrameProcessingException(e))); return null; @@ -326,12 +328,95 @@ public AsyncFrame dequeueInputFrame(Executor wakeupExecutor, Runnable wakeupList } } + @RequiresApi(28) + @GuardedBy("lock") + @Nullable + private AsyncFrame dequeueHardwareBufferFrame( + Format currentFormat, + long currentUsageFlags, + Executor wakeupExecutor, + Runnable wakeupListener) { + // On API 33+ the usage flags can be directly set on ImageWriter, so the dequeued image + // can be directly written to by the caller. + // On API < 33, if no specific usage flags are requested, or CPU usage only is requested, + // we can also use the dequeued buffer directly. + if (SDK_INT >= 33 + || currentUsageFlags == 0 + || currentUsageFlags == HardwareBuffer.USAGE_CPU_WRITE_OFTEN) { + InternalImage internalImage = dequeueInternalImage(/* needsCopy= */ false); + HardwareBuffer hardwareBuffer = checkNotNull(internalImage.image.getHardwareBuffer()); + DefaultHardwareBufferFrame frame = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer) + .setInternalImage(internalImage) + .setFormat(currentFormat) + .build(); + return new AsyncFrame(frame, /* acquireFence= */ null); + } + + checkState(hardwareBufferNativeHelpers != null); + checkNotNull(hardwareBufferPool); + // On API <33, create an intermediate frame with usage equal to the requested usage, that + // can also be read by the CPU. This will be filled by the caller, then copied into the + // buffer dequeued from ImageWriter via the JNI. + @Nullable + HardwareBufferWithFence bufferWithFence = + hardwareBufferPool.get( + currentFormat, + currentUsageFlags | Frame.USAGE_CPU_READ_OFTEN, + () -> wakeupExecutor.execute(wakeupListener)); + // The pool is at capacity, wakeupListener will be invoked when there is an available + // buffer. + if (bufferWithFence == null) { + return null; + } + InternalImage internalImage = dequeueInternalImage(/* needsCopy= */ true); + DefaultHardwareBufferFrame frame = + new DefaultHardwareBufferFrame.Builder(bufferWithFence.hardwareBuffer) + .setInternalImage(internalImage) + .setFormat(currentFormat) + .build(); + return new AsyncFrame(frame, bufferWithFence.acquireFence); + } + + @GuardedBy("lock") + private AsyncFrame dequeueImagePlanesFrame(Format currentFormat) { + InternalImage internalImage = dequeueInternalImage(/* needsCopy= */ false); + Image image = internalImage.image; + Image.Plane[] planes = image.getPlanes(); + ImmutableList.Builder planeListBuilder = ImmutableList.builder(); + for (Image.Plane plane : planes) { + planeListBuilder.add( + new DefaultPlane(plane.getBuffer(), plane.getRowStride(), plane.getPixelStride())); + } + DefaultImagePlanesFrame frame = + new DefaultImagePlanesFrame.Builder(planeListBuilder.build()) + .setInternalImage(internalImage) + .setFormat(currentFormat) + .build(); + return new AsyncFrame(frame, /* acquireFence= */ null); + } + @Override + @SuppressWarnings("ReferenceEquality") // Checks if ImageWriter was recreated. public void queueInputFrame(Frame frame, @Nullable SyncFenceWrapper writeCompleteFence) { - checkArgument(frame instanceof DefaultHardwareBufferFrame); - DefaultHardwareBufferFrame hardwareBufferFrame = (DefaultHardwareBufferFrame) frame; - InternalImage internalImage = - (InternalImage) checkNotNull(hardwareBufferFrame.getInternalImage()); + InternalImage internalImage; + if (SDK_INT < 28) { + checkArgument( + frame instanceof DefaultImagePlanesFrame, + "Unsupported frame type %s for API level %s", + frame.getClass().getName(), + SDK_INT); + internalImage = + (InternalImage) checkNotNull(((DefaultImagePlanesFrame) frame).getInternalImage()); + } else { + checkArgument( + frame instanceof DefaultHardwareBufferFrame, + "Unsupported frame type %s for API level %s", + frame.getClass().getName(), + SDK_INT); + internalImage = + (InternalImage) checkNotNull(((DefaultHardwareBufferFrame) frame).getInternalImage()); + } Image image = internalImage.image; ImageWriter imageWriterFromFrame = internalImage.imageWriter; long releaseTimeNs; @@ -343,7 +428,7 @@ public void queueInputFrame(Frame frame, @Nullable SyncFenceWrapper writeComplet } catch (IOException e) { listenerExecutor.execute(() -> listener.onError(new VideoFrameProcessingException(e))); writeCompleteFence.close(); - releaseFrameResources(hardwareBufferFrame); + releaseFrameResources(frame); return; } } @@ -351,43 +436,18 @@ public void queueInputFrame(Frame frame, @Nullable SyncFenceWrapper writeComplet if (this.imageWriter != imageWriterFromFrame) { // The ImageWriter has changed since this frame was dequeued. The previous writer was // closed and this image can no longer be used. - releaseFrameResources(hardwareBufferFrame); + releaseFrameResources(frame); return; } - // Copy the input frame's content to the ImageWriter's HardwareBuffer if an intermediate - // pooled buffer was used. - if (internalImage.needsCopy) { - checkState(hardwareBufferNativeHelpers != null); - // Every call to image.getHardwareBuffer() returns a new reference that needs to be closed. - try (HardwareBuffer imageWriterHardwareBuffer = checkNotNull(image.getHardwareBuffer())) { - boolean copySuccess = - hardwareBufferNativeHelpers.nativeCopyHardwareBufferToHardwareBuffer( - hardwareBufferFrame.getHardwareBuffer(), imageWriterHardwareBuffer); - if (!copySuccess) { - listenerExecutor.execute( - () -> - listener.onError( - new VideoFrameProcessingException( - "Failed to copy HardwareBuffer via JNI."))); - releaseFrameResources(hardwareBufferFrame); - return; - } - // Recycle the pooled buffer. The ImageWriter's buffer is now filled. - hardwareBufferPool.recycle(hardwareBufferFrame.getHardwareBuffer(), /* fence= */ null); + if (SDK_INT >= 28) { + if (!tryPrepareHardwareBufferFrameForQueueing( + (DefaultHardwareBufferFrame) frame, internalImage, image)) { + return; } - } else { - // Only close the input buffer if it was directly created from an ImageWriter buffer, so - // needs to be closed separately. - hardwareBufferFrame.getHardwareBuffer().close(); } - if (lastQueuedHardwareBuffer != null) { - lastQueuedHardwareBuffer.close(); - lastQueuedHardwareBuffer = null; - } - lastQueuedHardwareBuffer = checkNotNull(image.getHardwareBuffer()); - Object value = hardwareBufferFrame.getMetadata().get(Frame.KEY_DISPLAY_TIME_NS); + Object value = frame.getMetadata().get(Frame.KEY_DISPLAY_TIME_NS); if (value instanceof Number) { releaseTimeNs = ((Number) value).longValue(); if (releaseTimeNs != C.TIME_UNSET) { @@ -402,20 +462,82 @@ public void queueInputFrame(Frame frame, @Nullable SyncFenceWrapper writeComplet listenerExecutor.execute(() -> listener.onError(new VideoFrameProcessingException(e))); // If queueInputImage fails, the Image is not consumed. We must close it. image.close(); + return; } } listenerExecutor.execute( () -> listener.onFrameAboutToBeRendered( - hardwareBufferFrame.getContentTimeUs(), - releaseTimeNs, - hardwareBufferFrame.getFormat())); + frame.getContentTimeUs(), releaseTimeNs, frame.getFormat())); } - private void releaseFrameResources(DefaultHardwareBufferFrame frame) { - frame.getHardwareBuffer().close(); - checkNotNull((InternalImage) frame.getInternalImage()).image.close(); + /** + * Attempts to prepare the hardware buffer resources before queueing the frame to the {@link + * ImageWriter}. + * + *

If an intermediate pooled buffer was used, copies its content via JNI into the {@link + * ImageWriter}'s buffer and recycles the pooled buffer. If drawn directly, closes the input + * frame's hardware buffer. In both cases, caches the image's hardware buffer as the backup frame. + * + * @return Whether the hardware buffer was prepared successfully. If {@code false}, error + * reporting and resource cleanup have already been performed. + */ + @RequiresApi(28) + @GuardedBy("lock") + private boolean tryPrepareHardwareBufferFrameForQueueing( + DefaultHardwareBufferFrame hardwareBufferFrame, InternalImage internalImage, Image image) { + if (internalImage.needsCopy) { + checkState(hardwareBufferNativeHelpers != null); + checkNotNull(hardwareBufferPool); + // Every call to image.getHardwareBuffer() returns a new reference that needs to be closed. + try (HardwareBuffer imageWriterHardwareBuffer = checkNotNull(image.getHardwareBuffer())) { + boolean copySuccess = + hardwareBufferNativeHelpers.nativeCopyHardwareBufferToHardwareBuffer( + hardwareBufferFrame.getHardwareBuffer(), imageWriterHardwareBuffer); + if (!copySuccess) { + listenerExecutor.execute( + () -> + listener.onError( + new VideoFrameProcessingException("Failed to copy HardwareBuffer via JNI."))); + releaseFrameResources(hardwareBufferFrame); + return false; + } + // Recycle the pooled buffer. The ImageWriter's buffer is now filled. + hardwareBufferPool.recycle(hardwareBufferFrame.getHardwareBuffer(), /* fence= */ null); + } + } else { + // Only close the input buffer if it was directly created from an ImageWriter buffer, so + // needs to be closed separately. + hardwareBufferFrame.getHardwareBuffer().close(); + } + if (lastQueuedHardwareBuffer != null) { + lastQueuedHardwareBuffer.close(); + lastQueuedHardwareBuffer = null; + } + lastQueuedHardwareBuffer = checkNotNull(image.getHardwareBuffer()); + return true; + } + + @GuardedBy("lock") + private void releaseFrameResources(Frame frame) { + if (SDK_INT < 28) { + DefaultImagePlanesFrame imagePlanesFrame = (DefaultImagePlanesFrame) frame; + InternalImage internalImage = + (InternalImage) checkNotNull(imagePlanesFrame.getInternalImage()); + internalImage.image.close(); + } else { + DefaultHardwareBufferFrame hardwareBufferFrame = (DefaultHardwareBufferFrame) frame; + InternalImage internalImage = + (InternalImage) checkNotNull(hardwareBufferFrame.getInternalImage()); + if (internalImage.needsCopy) { + checkNotNull(hardwareBufferPool) + .recycle(hardwareBufferFrame.getHardwareBuffer(), /* fence= */ null); + } else { + hardwareBufferFrame.getHardwareBuffer().close(); + } + internalImage.image.close(); + } } @Override @@ -426,7 +548,7 @@ public void signalEndOfStream() { @Override public void close() { synchronized (lock) { - if (lastQueuedHardwareBuffer != null) { + if (SDK_INT >= 28 && lastQueuedHardwareBuffer != null) { lastQueuedHardwareBuffer.close(); lastQueuedHardwareBuffer = null; } @@ -456,10 +578,13 @@ public void surfaceChanged(SurfaceHolder holder, int format, int width, int heig } int expectedPixelFormat = currentFormat.pixelFormat; if (expectedPixelFormat == Format.NO_VALUE) { - expectedPixelFormat = - ColorInfo.isTransferHdr(currentFormat.colorInfo) - ? HardwareBuffer.RGBA_1010102 - : HardwareBuffer.RGBA_8888; + if (SDK_INT < 28) { + expectedPixelFormat = ImageFormat.YV12; + } else if (ColorInfo.isTransferHdr(currentFormat.colorInfo)) { + expectedPixelFormat = HardwareBuffer.RGBA_1010102; + } else { + expectedPixelFormat = HardwareBuffer.RGBA_8888; + } } if (width != currentFormat.width || height != currentFormat.height @@ -500,7 +625,9 @@ public void surfaceChanged(SurfaceHolder holder, int format, int width, int heig } surface = holder.getSurface(); isSurfaceChangeRequested = false; - maybeRestoreBackupFrame(); + if (SDK_INT >= 28) { + maybeRestoreBackupHardwareBufferFrame(); + } Runnable wakeupListener = this.wakeupListener; this.wakeupListener = null; if (wakeupListener != null) { @@ -525,8 +652,9 @@ public void surfaceDestroyed(SurfaceHolder holder) { * surface. This is a best-effort attempt, which will be skipped if buffer dimensions do not * match. */ + @RequiresApi(28) @GuardedBy("lock") - private void maybeRestoreBackupFrame() { + private void maybeRestoreBackupHardwareBufferFrame() { ImageWriter imageWriter = this.imageWriter; if (imageWriter == null) { return; @@ -542,21 +670,26 @@ private void maybeRestoreBackupFrame() { image.close(); return; } - if (hardwareBufferNativeHelpers != null) { - checkState( - hardwareBufferNativeHelpers.nativeCopyHardwareBufferToHardwareBuffer( - lastQueuedHardwareBuffer, checkNotNull(image.getHardwareBuffer()))); - } else if (SDK_INT >= 29) { - Bitmap bitmap = - checkNotNull(Bitmap.wrapHardwareBuffer(lastQueuedHardwareBuffer, /* colorSpace= */ null)) - .copy(getBitmapConfig(lastQueuedHardwareBuffer.getFormat()), /* isMutable= */ false); - if (bitmap != null) { + try { + if (hardwareBufferNativeHelpers != null) { + checkState( + hardwareBufferNativeHelpers.nativeCopyHardwareBufferToHardwareBuffer( + lastQueuedHardwareBuffer, checkNotNull(image.getHardwareBuffer()))); + } else if (SDK_INT >= 29) { + Bitmap bitmap = + checkNotNull( + Bitmap.wrapHardwareBuffer(lastQueuedHardwareBuffer, /* colorSpace= */ null)) + .copy( + getBitmapConfig(lastQueuedHardwareBuffer.getFormat()), /* isMutable= */ false); ByteBuffer planeBuffer = image.getPlanes()[0].getBuffer(); planeBuffer.rewind(); - bitmap.copyPixelsToBuffer(planeBuffer); + checkNotNull(bitmap).copyPixelsToBuffer(planeBuffer); } + } catch (RuntimeException e) { + Log.w(TAG, "Failed to restore backup frame", e); + } finally { + imageWriter.queueInputImage(image); } - imageWriter.queueInputImage(image); } @GuardedBy("lock") @@ -631,10 +764,25 @@ private static int colorRangeToDataSpaceRange(@C.ColorRange int colorRange) { } } + @RequiresApi(26) private static Config getBitmapConfig(int imageFormat) { checkArgument(imageFormat == PixelFormat.RGBA_1010102 || imageFormat == PixelFormat.RGBA_8888); return imageFormat == PixelFormat.RGBA_1010102 && SDK_INT >= 33 ? Config.RGBA_1010102 : Config.ARGB_8888; } + + /** + * Returns whether {@code pixelFormat} is supported. + * + *

Below API 28, frames are written as {@link ImagePlanesFrame} through an {@link ImageWriter} + * configured with {@link ImageFormat#YV12}, so only that format, or an unset {@link + * Format#NO_VALUE}, is supported. + * + *

From API 28, the pixel format is forwarded to the {@link SurfaceHolder} and the dequeued + * {@link HardwareBuffer}, so it is not constrained here. + */ + private static boolean isPixelFormatSupported(int pixelFormat) { + return SDK_INT >= 28 || pixelFormat == Format.NO_VALUE || pixelFormat == ImageFormat.YV12; + } } diff --git a/libraries/common/src/main/java/androidx/media3/common/video/SyncFenceWrapper.java b/libraries/common/src/main/java/androidx/media3/common/video/SyncFenceWrapper.java index 92316b86150..6835828dd19 100644 --- a/libraries/common/src/main/java/androidx/media3/common/video/SyncFenceWrapper.java +++ b/libraries/common/src/main/java/androidx/media3/common/video/SyncFenceWrapper.java @@ -24,7 +24,7 @@ /** A wrapper for {@link SyncFence} that exists on all API levels. */ @ExperimentalApi // TODO: b/498176910 Remove once SyncFenceWrapper is production ready. -public final class SyncFenceWrapper { +public class SyncFenceWrapper { private final SyncFence syncFence; diff --git a/libraries/common/src/test/java/androidx/media3/common/BasePlayerTest.java b/libraries/common/src/test/java/androidx/media3/common/BasePlayerTest.java index cc2015acd4b..1e6a0b1131e 100644 --- a/libraries/common/src/test/java/androidx/media3/common/BasePlayerTest.java +++ b/libraries/common/src/test/java/androidx/media3/common/BasePlayerTest.java @@ -317,6 +317,87 @@ public void getBufferedPercentage_commandUnavailable() { assertThat(player.getBufferedPercentage()).isEqualTo(0); } + @Test + public void getCurrentLiveOffset_whenNotLiveWithWindowStartTime_returnsTimeUnset() { + Timeline timeline = + new Timeline() { + @Override + public int getWindowCount() { + return 1; + } + + @Override + public Window getWindow( + int windowIndex, Window window, long defaultPositionProjectionUs) { + return window.set( + /* uid= */ 0, + MediaItem.EMPTY, + /* manifest= */ null, + /* presentationStartTimeMs= */ 1_000_000L, + /* windowStartTimeMs= */ 1_000_000L, + /* elapsedRealtimeEpochOffsetMs= */ C.TIME_UNSET, + /* isSeekable= */ true, + /* isDynamic= */ false, + /* liveConfiguration= */ null, + /* defaultPositionUs= */ 0, + /* durationUs= */ 10_000_000L, + /* firstPeriodIndex= */ 0, + /* lastPeriodIndex= */ 0, + /* positionInFirstPeriodUs= */ 0); + } + + @Override + public int getPeriodCount() { + return 1; + } + + @Override + public Period getPeriod(int periodIndex, Period period, boolean setIds) { + return period.set( + /* id= */ 0, + /* uid= */ 0, + /* windowIndex= */ 0, + /* durationUs= */ 10_000_000L, + /* positionInWindowUs= */ 0, + AdPlaybackState.NONE, + /* isPlaceholder= */ false); + } + + @Override + public int getIndexOfPeriod(Object uid) { + return 0; + } + + @Override + public Object getUidOfPeriod(int periodIndex) { + return 0; + } + }; + TestBasePlayer player = + new TestBasePlayer() { + @Override + public Timeline getCurrentTimeline() { + return timeline; + } + + @Override + public int getCurrentMediaItemIndex() { + return 0; + } + + @Override + public long getContentPosition() { + return 2000; + } + }; + + boolean isLive = player.isCurrentMediaItemLive(); + long liveOffsetMs = player.getCurrentLiveOffset(); + + assertThat(isLive).isFalse(); + assertThat(liveOffsetMs).isEqualTo(C.TIME_UNSET); + } + private static class TestBasePlayer extends StubPlayer { private int mediaItemIndex; diff --git a/libraries/common/src/test/java/androidx/media3/common/DeviceInfoTest.java b/libraries/common/src/test/java/androidx/media3/common/DeviceInfoTest.java index a7e0b3e6722..086c715ac19 100644 --- a/libraries/common/src/test/java/androidx/media3/common/DeviceInfoTest.java +++ b/libraries/common/src/test/java/androidx/media3/common/DeviceInfoTest.java @@ -33,18 +33,22 @@ public void roundTripViaBundle_yieldsEqualInstance() { .setMinVolume(1) .setMaxVolume(9) .setRoutingControllerId("route") + .setRoutingControllerName("Living Room TV") .build(); assertThat(DeviceInfo.fromBundle(deviceInfo.toBundle())).isEqualTo(deviceInfo); } @Test - public void build_playbackTypeLocalWithRoutingId_throwsIllegalStateException() { + public void setRoutingControllerId_playbackTypeLocal_throwsIllegalArgumentException() { + DeviceInfo.Builder builder = new DeviceInfo.Builder(DeviceInfo.PLAYBACK_TYPE_LOCAL); + assertThrows(IllegalArgumentException.class, () -> builder.setRoutingControllerId("route")); + } + + @Test + public void setRoutingControllerName_playbackTypeLocal_throwsIllegalArgumentException() { + DeviceInfo.Builder builder = new DeviceInfo.Builder(DeviceInfo.PLAYBACK_TYPE_LOCAL); assertThrows( - IllegalArgumentException.class, - () -> - new DeviceInfo.Builder(DeviceInfo.PLAYBACK_TYPE_LOCAL) - .setRoutingControllerId("route") - .build()); + IllegalArgumentException.class, () -> builder.setRoutingControllerName("Living Room TV")); } } diff --git a/libraries/common/src/test/java/androidx/media3/common/FileTypesTest.java b/libraries/common/src/test/java/androidx/media3/common/FileTypesTest.java index f5439da50a1..dc17578c4fd 100644 --- a/libraries/common/src/test/java/androidx/media3/common/FileTypesTest.java +++ b/libraries/common/src/test/java/androidx/media3/common/FileTypesTest.java @@ -85,6 +85,16 @@ public void inferFileFormat_fromUriWithExtensionPrefix_returnsExpectedFormat() { assertThat(inferFileTypeFromUri(Uri.parse("filename.mka"))).isEqualTo(FileTypes.MATROSKA); } + @Test + public void inferFileFormat_fromUriWithMixedCaseExtension_returnsExpectedFormat() { + assertThat(inferFileTypeFromUri(Uri.parse("filename.JPG"))).isEqualTo(FileTypes.JPEG); + assertThat(inferFileTypeFromUri(Uri.parse("filename.Jpg"))).isEqualTo(FileTypes.JPEG); + assertThat( + inferFileTypeFromUri( + Uri.parse("http://www.example.com/FILENAME.MP3?Query=MyQuery#Fragment"))) + .isEqualTo(FileTypes.MP3); + } + @Test public void inferFileFormat_fromUriWithUnknownExtension_returnsUnknownFormat() { assertThat(inferFileTypeFromUri(Uri.parse("filename.unknown"))).isEqualTo(FileTypes.UNKNOWN); diff --git a/libraries/common/src/test/java/androidx/media3/common/FlagsTest.java b/libraries/common/src/test/java/androidx/media3/common/FlagsTest.java new file mode 100644 index 00000000000..22a32ff2fdc --- /dev/null +++ b/libraries/common/src/test/java/androidx/media3/common/FlagsTest.java @@ -0,0 +1,315 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.common; + +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.truth.Truth.assertWithMessage; +import static org.junit.Assert.assertThrows; + +import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.lang.reflect.Field; +import java.lang.reflect.Modifier; +import java.util.HashMap; +import java.util.Map; +import org.junit.After; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link Flags}. */ +@RunWith(AndroidJUnit4.class) +public class FlagsTest { + + private static final @Flags.Flag int TEST_FLAG_1 = 1_000_000; + private static final @Flags.Flag int TEST_FLAG_2 = 1_000_001; + + @After + public void tearDown() { + Flags.resetForTesting(); + } + + @Test + public void isEnabled_defaultState_returnsFalseForStaticallyDisabledFlags() { + assertThat(Flags.isEnabled(getStaticallyDisabledFlag())).isFalse(); + } + + @Test + public void isEnabled_defaultState_returnsTrueForStaticallyEnabledFlags() { + assertThat(Flags.isEnabled(getStaticallyEnabledFlag())).isTrue(); + } + + @Test + public void isEnabled_canaryModeEnabled_returnsFalseForStaticallyDisabledFlags() { + Flags.setCanaryModeEnabled(true); + + // Statically disabled flags remain disabled even with canary mode on. + assertThat(Flags.isEnabled(getStaticallyDisabledFlag())).isFalse(); + } + + @Test + public void isEnabled_canaryModeDisabled_returnsTrueForStaticallyEnabledFlags() { + Flags.setCanaryModeEnabled(false); + + // Statically enabled flags remain enabled even with canary mode off. + assertThat(Flags.isEnabled(getStaticallyEnabledFlag())).isTrue(); + } + + @Test + public void isEnabled_canaryModeEnabled_returnsTrueForDynamicFlags() { + Flags.setCanaryModeEnabled(true); + + assertThat(Flags.isEnabled(TEST_FLAG_1)).isTrue(); + } + + @Test + public void isEnabled_canaryModeDisabled_returnsFalseForDynamicFlags() { + Flags.setCanaryModeEnabled(false); + + assertThat(Flags.isEnabled(TEST_FLAG_1)).isFalse(); + } + + @Test + public void enableFlag_overridesStaticFalse() { + int flag = getStaticallyDisabledFlag(); + Flags.enableFlag(flag); + + assertThat(Flags.isEnabled(flag)).isTrue(); + assertThat(Flags.getEnabledOverrides()).asList().containsExactly(flag); + assertThat(Flags.getDisabledOverrides()).isEmpty(); + } + + @Test + public void disableFlag_overridesStaticTrue() { + int flag = getStaticallyEnabledFlag(); + Flags.disableFlag(flag); + + assertThat(Flags.isEnabled(flag)).isFalse(); + assertThat(Flags.getDisabledOverrides()).asList().containsExactly(flag); + assertThat(Flags.getEnabledOverrides()).isEmpty(); + } + + @Test + public void enableFlag_staticallyEnabledFlag_isIgnored() { + int flag = getStaticallyEnabledFlag(); + Flags.enableFlag(flag); + + assertThat(Flags.isEnabled(flag)).isTrue(); + assertThat(Flags.getEnabledOverrides()).isEmpty(); + assertThat(Flags.getDisabledOverrides()).isEmpty(); + } + + @Test + public void disableFlag_staticallyDisabledFlag_isIgnored() { + int flag = getStaticallyDisabledFlag(); + Flags.disableFlag(flag); + + assertThat(Flags.isEnabled(flag)).isFalse(); + assertThat(Flags.getEnabledOverrides()).isEmpty(); + assertThat(Flags.getDisabledOverrides()).isEmpty(); + } + + @Test + public void enableFlag_staticallyEnabledFlag_clearsPreviousDisable() { + int flag = getStaticallyEnabledFlag(); + Flags.disableFlag(flag); + assertThat(Flags.getDisabledOverrides()).asList().containsExactly(flag); + + Flags.enableFlag(flag); + + assertThat(Flags.isEnabled(flag)).isTrue(); + assertThat(Flags.getEnabledOverrides()).isEmpty(); + assertThat(Flags.getDisabledOverrides()).isEmpty(); + } + + @Test + public void disableFlag_staticallyDisabledFlag_clearsPreviousEnable() { + int flag = getStaticallyDisabledFlag(); + Flags.enableFlag(flag); + assertThat(Flags.getEnabledOverrides()).asList().containsExactly(flag); + + Flags.disableFlag(flag); + + assertThat(Flags.isEnabled(flag)).isFalse(); + assertThat(Flags.getEnabledOverrides()).isEmpty(); + assertThat(Flags.getDisabledOverrides()).isEmpty(); + } + + @Test + public void disableFlag_overridesCanaryDefault() { + Flags.disableFlag(TEST_FLAG_1); + + assertThat(Flags.isEnabled(TEST_FLAG_1)).isFalse(); + assertThat(Flags.getDisabledOverrides()).asList().containsExactly(TEST_FLAG_1); + } + + @Test + public void enableFlag_overridesDisableFlag() { + Flags.disableFlag(TEST_FLAG_1); + Flags.enableFlag(TEST_FLAG_1); + + assertThat(Flags.isEnabled(TEST_FLAG_1)).isTrue(); + assertThat(Flags.getEnabledOverrides()).asList().containsExactly(TEST_FLAG_1); + assertThat(Flags.getDisabledOverrides()).isEmpty(); + } + + @Test + public void disableFlag_overridesPreviousEnable() { + Flags.enableFlag(TEST_FLAG_1); + Flags.disableFlag(TEST_FLAG_1); + + assertThat(Flags.isEnabled(TEST_FLAG_1)).isFalse(); + assertThat(Flags.getDisabledOverrides()).asList().containsExactly(TEST_FLAG_1); + assertThat(Flags.getEnabledOverrides()).isEmpty(); + } + + @Test + public void enableFlag_withCanaryModeOff_stillEnablesFlag() { + Flags.setCanaryModeEnabled(false); + Flags.enableFlag(TEST_FLAG_1); + + assertThat(Flags.isEnabled(TEST_FLAG_1)).isTrue(); + } + + @Test + public void enableFlag_calledMultipleTimes_isIdempotent() { + Flags.enableFlag(TEST_FLAG_1); + Flags.enableFlag(TEST_FLAG_1); + + assertThat(Flags.isEnabled(TEST_FLAG_1)).isTrue(); + assertThat(Flags.getEnabledOverrides()).asList().containsExactly(TEST_FLAG_1); + assertThat(Flags.getDisabledOverrides()).isEmpty(); + } + + @Test + public void disableFlag_calledMultipleTimes_isIdempotent() { + Flags.disableFlag(TEST_FLAG_1); + Flags.disableFlag(TEST_FLAG_1); + + assertThat(Flags.isEnabled(TEST_FLAG_1)).isFalse(); + assertThat(Flags.getDisabledOverrides()).asList().containsExactly(TEST_FLAG_1); + assertThat(Flags.getEnabledOverrides()).isEmpty(); + } + + @Test + public void enableFlag_invalidFlag_throwsIllegalArgumentException() { + assertThrows(IllegalArgumentException.class, () -> Flags.enableFlag(0)); + assertThrows(IllegalArgumentException.class, () -> Flags.enableFlag(-1)); + } + + @Test + public void disableFlag_invalidFlag_throwsIllegalArgumentException() { + assertThrows(IllegalArgumentException.class, () -> Flags.disableFlag(0)); + assertThrows(IllegalArgumentException.class, () -> Flags.disableFlag(-1)); + } + + @Test + public void isEnabled_invalidFlag_throwsIllegalArgumentException() { + assertThrows(IllegalArgumentException.class, () -> Flags.isEnabled(0)); + assertThrows(IllegalArgumentException.class, () -> Flags.isEnabled(-1)); + } + + @Test + public void resetForTesting_restoresDefaults() { + int disabledFlag = getStaticallyDisabledFlag(); + int enabledFlag = getStaticallyEnabledFlag(); + + Flags.setCanaryModeEnabled(false); + Flags.enableFlag(disabledFlag); + Flags.disableFlag(enabledFlag); + Flags.enableFlag(TEST_FLAG_1); + + Flags.resetForTesting(); + + assertThat(Flags.isCanaryModeEnabled()).isTrue(); + assertThat(Flags.getEnabledOverrides()).isEmpty(); + assertThat(Flags.getDisabledOverrides()).isEmpty(); + assertThat(Flags.isEnabled(disabledFlag)).isFalse(); + assertThat(Flags.isEnabled(enabledFlag)).isTrue(); + assertThat(Flags.isEnabled(TEST_FLAG_1)).isTrue(); + } + + @Test + public void getLogString_noOverrides_canaryOff() { + Flags.setCanaryModeEnabled(false); + + assertThat(Flags.getLogString()).isEqualTo("[Canary:OFF]"); + } + + @Test + public void getLogString_noOverrides_canaryOn() { + Flags.setCanaryModeEnabled(true); + + assertThat(Flags.getLogString()).isEqualTo("[Canary:ON]"); + } + + @Test + public void getLogString_withOverrides() { + Flags.setCanaryModeEnabled(true); + Flags.enableFlag(TEST_FLAG_1); + Flags.disableFlag(TEST_FLAG_2); + + assertThat(Flags.getLogString()) + .isEqualTo( + "[Canary:ON, Overrides: enabled=[" + + TEST_FLAG_1 + + "], disabled=[" + + TEST_FLAG_2 + + "]]"); + } + + @Test + public void getLogString_staticallyConfiguredFlagsNotListedWhenMatchingStaticDefault() { + int disabledFlag = getStaticallyDisabledFlag(); + int enabledFlag = getStaticallyEnabledFlag(); + + Flags.setCanaryModeEnabled(true); + Flags.disableFlag(disabledFlag); + Flags.enableFlag(enabledFlag); + + assertThat(Flags.getLogString()).isEqualTo("[Canary:ON]"); + } + + @Test + public void declaredFlags_haveValidAndUniqueIdsLessThanNextFlagId() throws Exception { + Map seenIdsToFieldNames = new HashMap<>(); + for (Field field : Flags.class.getDeclaredFields()) { + if (field.getName().startsWith("FLAG_") + && Modifier.isPublic(field.getModifiers()) + && Modifier.isStatic(field.getModifiers()) + && field.getType() == int.class) { + int flagId = field.getInt(null); + assertThat(flagId).isGreaterThan(0); + assertThat(flagId).isLessThan(Flags.NEXT_FLAG_ID); + String previousField = seenIdsToFieldNames.put(flagId, field.getName()); + assertWithMessage( + "Duplicate flag ID %s found for fields '%s' and '%s'.", + flagId, previousField, field.getName()) + .that(previousField) + .isNull(); + } + } + assertThat(seenIdsToFieldNames).isNotEmpty(); + } + + private static @Flags.Flag int getStaticallyDisabledFlag() { + // Update to another statically disabled flag when the returned flag is removed. + return Flags.FLAG_PER_STREAM_MEDIA_PROGRESSION; + } + + private static @Flags.Flag int getStaticallyEnabledFlag() { + // Update to another statically enabled flag when the returned flag is removed. + return Flags.FLAG_DYNAMIC_SCHEDULING; + } +} diff --git a/libraries/common/src/test/java/androidx/media3/common/MediaItemTest.java b/libraries/common/src/test/java/androidx/media3/common/MediaItemTest.java index 60084362b52..24b7318fc55 100644 --- a/libraries/common/src/test/java/androidx/media3/common/MediaItemTest.java +++ b/libraries/common/src/test/java/androidx/media3/common/MediaItemTest.java @@ -328,6 +328,7 @@ public void builderSetSubtitleConfigurations() { .setRoleFlags(C.ROLE_FLAG_ALTERNATE) .setLabel("label") .setId("id") + .setTimeOffsetUs(500_000) .build()); MediaItem mediaItem = @@ -396,6 +397,7 @@ public void createSubtitleConfigurationInstance_roundTripViaBundle_yieldsEqualIn .setRoleFlags(C.ROLE_FLAG_ALTERNATE) .setLabel("label") .setId("id") + .setTimeOffsetUs(-300_000) .build(); MediaItem.SubtitleConfiguration subtitleConfigurationFromBundle = @@ -556,7 +558,7 @@ public void createClippingConfigurationInstance_setsEndPositionInMsAndUs_fieldsA .build(); assertThat(clippingConfiguration.endPositionMs).isEqualTo(C.TIME_END_OF_SOURCE); - assertThat(clippingConfiguration.endPositionMs).isEqualTo(C.TIME_END_OF_SOURCE); + assertThat(clippingConfiguration.endPositionUs).isEqualTo(C.TIME_END_OF_SOURCE); } @Test @@ -800,6 +802,7 @@ public void createLocalConfigurationInstance_roundTripViaBundle_yieldsEqualInsta .setRoleFlags(C.ROLE_FLAG_ALTERNATE) .setLabel("label") .setId("id") + .setTimeOffsetUs(250_000) .build())) .setDrmConfiguration( new MediaItem.DrmConfiguration.Builder(C.WIDEVINE_UUID) @@ -953,6 +956,7 @@ public void buildUpon_individualSetters_equalsToOriginal() { .setRoleFlags(C.ROLE_FLAG_ALTERNATE) .setLabel("label") .setId("id") + .setTimeOffsetUs(250_000) .build())) .setTag(new Object()) .build(); @@ -1010,6 +1014,7 @@ public void buildUpon_wholeObjectSetters_equalsToOriginal() { .setRoleFlags(C.ROLE_FLAG_ALTERNATE) .setLabel("label") .setId("id") + .setTimeOffsetUs(250_000) .build())) .setRequestMetadata( new RequestMetadata.Builder() diff --git a/libraries/common/src/test/java/androidx/media3/common/MediaMetadataTest.java b/libraries/common/src/test/java/androidx/media3/common/MediaMetadataTest.java index bde94dd9800..92ecd722209 100644 --- a/libraries/common/src/test/java/androidx/media3/common/MediaMetadataTest.java +++ b/libraries/common/src/test/java/androidx/media3/common/MediaMetadataTest.java @@ -16,6 +16,7 @@ package androidx.media3.common; import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; import android.net.Uri; import android.os.Bundle; @@ -70,6 +71,8 @@ public void builder_minimal_correctDefaults() { assertThat(mediaMetadata.compilation).isNull(); assertThat(mediaMetadata.station).isNull(); assertThat(mediaMetadata.mediaType).isNull(); + assertThat(mediaMetadata.playlistId).isNull(); + assertThat(mediaMetadata.categories).isEmpty(); assertThat(mediaMetadata.supportedCommands).isEmpty(); assertThat(mediaMetadata.extras).isNull(); } @@ -100,6 +103,31 @@ public void builderSetArtworkUri_setsArtworkUri() { assertThat(mediaMetadata.artworkUri).isEqualTo(uri); } + @Test + public void builderSetPlaylistId_withNonEmptyString_setsPlaylistId() { + String playlistId = "playlistId"; + + MediaMetadata mediaMetadata = new MediaMetadata.Builder().setPlaylistId(playlistId).build(); + + assertThat(mediaMetadata.playlistId).isEqualTo(playlistId); + } + + @Test + public void builderSetPlaylistId_withEmptyString_throwsIllegalArgumentException() { + MediaMetadata.Builder builder = new MediaMetadata.Builder(); + + assertThrows(IllegalArgumentException.class, () -> builder.setPlaylistId("")); + } + + @Test + public void builderSetCategories_setsCategories() { + ImmutableList categories = ImmutableList.of("category1", "category2"); + + MediaMetadata mediaMetadata = new MediaMetadata.Builder().setCategories(categories).build(); + + assertThat(mediaMetadata.categories).isEqualTo(categories); + } + @Test public void populate_populatesEveryField() { MediaMetadata mediaMetadata = getFullyPopulatedMediaMetadata(); @@ -335,6 +363,8 @@ private static MediaMetadata getFullyPopulatedMediaMetadata() { .setCompilation("Amazing songs.") .setStation("radio station") .setMediaType(MediaMetadata.MEDIA_TYPE_MIXED) + .setPlaylistId("playlist_id") + .setCategories(ImmutableList.of("category1", "category2")) .setSupportedCommands(ImmutableList.of("command1", "command2")) .setExtras(extras) .build(); diff --git a/libraries/common/src/test/java/androidx/media3/common/PlayerTransferStateTest.java b/libraries/common/src/test/java/androidx/media3/common/PlayerTransferStateTest.java index fe0ee1129a6..a8b854d8320 100644 --- a/libraries/common/src/test/java/androidx/media3/common/PlayerTransferStateTest.java +++ b/libraries/common/src/test/java/androidx/media3/common/PlayerTransferStateTest.java @@ -45,6 +45,8 @@ public void populateFrom_simpleBasePlayer_createsCorrectState() { TrackSelectionParameters trackSelectionParameters = new TrackSelectionParameters.Builder().setMaxVideoBitrate(500).build(); player.setTrackSelectionParameters(trackSelectionParameters); + MediaMetadata playlistMetadata = new MediaMetadata.Builder().setTitle("Test Playlist").build(); + player.setPlaylistMetadata(playlistMetadata); PlayerTransferState state = PlayerTransferState.fromPlayer(player); @@ -57,6 +59,7 @@ public void populateFrom_simpleBasePlayer_createsCorrectState() { assertThat(state.getPlaybackParameters().speed).isEqualTo(1.5f); assertThat(state.getPlaybackParameters().pitch).isEqualTo(0.8f); assertThat(state.getTrackSelectionParameters()).isEqualTo(trackSelectionParameters); + assertThat(state.getPlaylistMetadata()).isEqualTo(playlistMetadata); } @Test @@ -65,6 +68,8 @@ public void setToPlayer_simpleBasePlayer_appliesCorrectState() { MediaItem mediaItem2 = new MediaItem.Builder().setMediaId("id2").build(); TrackSelectionParameters trackSelectionParameters = new TrackSelectionParameters.Builder().setMaxVideoBitrate(1000).build(); + MediaMetadata playlistMetadata = + new MediaMetadata.Builder().setTitle("Target Playlist").build(); PlayerTransferState state = new PlayerTransferState.Builder() .setPlayWhenReady(false) @@ -75,6 +80,7 @@ public void setToPlayer_simpleBasePlayer_appliesCorrectState() { .setMediaItems(ImmutableList.of(mediaItem1, mediaItem2)) .setPlaybackParameters(new PlaybackParameters(/* speed= */ 0.5f, /* pitch= */ 1.2f)) .setTrackSelectionParameters(trackSelectionParameters) + .setPlaylistMetadata(playlistMetadata) .build(); SimpleBasePlayer player = new StateHolderPlayer(); // Set some initial state on the player to ensure it's overwritten. @@ -84,6 +90,7 @@ public void setToPlayer_simpleBasePlayer_appliesCorrectState() { player.seekTo(/* mediaItemIndex= */ 0, /* positionMs= */ 100); player.setPlaybackParameters(PlaybackParameters.DEFAULT); player.setTrackSelectionParameters(TrackSelectionParameters.DEFAULT); + player.setPlaylistMetadata(MediaMetadata.EMPTY); player.setMediaItems(ImmutableList.of(new MediaItem.Builder().setMediaId("id3").build())); state.setToPlayer(player); @@ -99,6 +106,7 @@ public void setToPlayer_simpleBasePlayer_appliesCorrectState() { assertThat(player.getPlaybackParameters().speed).isEqualTo(0.5f); assertThat(player.getPlaybackParameters().pitch).isEqualTo(1.2f); assertThat(player.getTrackSelectionParameters()).isEqualTo(trackSelectionParameters); + assertThat(player.getPlaylistMetadata()).isEqualTo(playlistMetadata); } @Test @@ -108,6 +116,8 @@ public void buildUpon_createsIdenticalCopy() { PlaybackParameters originalPlaybackParameters = new PlaybackParameters(1.1f, 0.9f); TrackSelectionParameters originalTrackSelectionParameters = new TrackSelectionParameters.Builder().setPreferredAudioLanguage("de").build(); + MediaMetadata originalPlaylistMetadata = + new MediaMetadata.Builder().setTitle("Original").build(); PlayerTransferState originalState = new PlayerTransferState.Builder() .setPlayWhenReady(true) @@ -118,6 +128,7 @@ public void buildUpon_createsIdenticalCopy() { .setMediaItems(ImmutableList.of(mediaItem1, mediaItem2)) .setPlaybackParameters(originalPlaybackParameters) .setTrackSelectionParameters(originalTrackSelectionParameters) + .setPlaylistMetadata(originalPlaylistMetadata) .build(); PlayerTransferState copiedState = originalState.buildUpon().build(); @@ -134,6 +145,7 @@ public void buildUpon_createsIdenticalCopy() { .isEqualTo(originalState.getPlaybackParameters()); assertThat(copiedState.getTrackSelectionParameters()) .isEqualTo(originalState.getTrackSelectionParameters()); + assertThat(copiedState.getPlaylistMetadata()).isEqualTo(originalState.getPlaylistMetadata()); } @Test @@ -141,6 +153,8 @@ public void buildUpon_allowsModifications() { MediaItem mediaItem1 = new MediaItem.Builder().setMediaId("original1").build(); MediaItem mediaItem2 = new MediaItem.Builder().setMediaId("original2").build(); PlaybackParameters originalPlaybackParameters = new PlaybackParameters(1.1f, 0.9f); + MediaMetadata originalPlaylistMetadata = + new MediaMetadata.Builder().setTitle("Original").build(); PlayerTransferState originalState = new PlayerTransferState.Builder() .setPlayWhenReady(true) @@ -151,11 +165,13 @@ public void buildUpon_allowsModifications() { .setMediaItems(ImmutableList.of(mediaItem1, mediaItem2)) .setPlaybackParameters(originalPlaybackParameters) .setTrackSelectionParameters(TrackSelectionParameters.DEFAULT) + .setPlaylistMetadata(originalPlaylistMetadata) .build(); MediaItem newMediaItem = new MediaItem.Builder().setMediaId("new").build(); PlaybackParameters newPlaybackParameters = new PlaybackParameters(2.0f); TrackSelectionParameters newTrackSelectionParameters = new TrackSelectionParameters.Builder().setForceHighestSupportedBitrate(true).build(); + MediaMetadata newPlaylistMetadata = new MediaMetadata.Builder().setTitle("Modified").build(); PlayerTransferState modifiedState = originalState @@ -165,6 +181,7 @@ public void buildUpon_allowsModifications() { .setMediaItems(ImmutableList.of(newMediaItem)) .setPlaybackParameters(newPlaybackParameters) .setTrackSelectionParameters(newTrackSelectionParameters) + .setPlaylistMetadata(newPlaylistMetadata) .build(); // The following are unchanged. @@ -178,6 +195,7 @@ public void buildUpon_allowsModifications() { assertThat(modifiedState.getPlaybackParameters()).isEqualTo(newPlaybackParameters); // Modified assertThat(modifiedState.getTrackSelectionParameters()) .isEqualTo(newTrackSelectionParameters); // Modified + assertThat(modifiedState.getPlaylistMetadata()).isEqualTo(newPlaylistMetadata); // Modified } /** A {@link Player} that holds state and supports its modification through setters. */ @@ -197,7 +215,9 @@ private static final class StateHolderPlayer extends SimpleBasePlayer { COMMAND_SET_MEDIA_ITEM, COMMAND_CHANGE_MEDIA_ITEMS, COMMAND_SEEK_TO_MEDIA_ITEM, - COMMAND_SET_TRACK_SELECTION_PARAMETERS) + COMMAND_SET_TRACK_SELECTION_PARAMETERS, + COMMAND_SET_PLAYLIST_METADATA, + COMMAND_GET_METADATA) .build(); private StateHolderPlayer() { @@ -247,6 +267,12 @@ protected ListenableFuture handleSetTrackSelectionParameters( return Futures.immediateVoidFuture(); } + @Override + protected ListenableFuture handleSetPlaylistMetadata(MediaMetadata playlistMetadata) { + state = state.buildUpon().setPlaylistMetadata(playlistMetadata).build(); + return Futures.immediateVoidFuture(); + } + @Override protected ListenableFuture handleSetMediaItems( List mediaItems, int startIndex, long startPositionMs) { diff --git a/libraries/common/src/test/java/androidx/media3/common/SimpleBasePlayerTest.java b/libraries/common/src/test/java/androidx/media3/common/SimpleBasePlayerTest.java index dff14a4a886..1666e57bb21 100644 --- a/libraries/common/src/test/java/androidx/media3/common/SimpleBasePlayerTest.java +++ b/libraries/common/src/test/java/androidx/media3/common/SimpleBasePlayerTest.java @@ -122,6 +122,8 @@ public void stateBuildUpon_build_isEqual() { .setPeriods( ImmutableList.of( new SimpleBasePlayer.PeriodData.Builder(/* uid= */ new Object()) + .setDurationUs(100_000) + .setOriginalDurationUs(200_000) .setAdPlaybackState( new AdPlaybackState( /* adsId= */ new Object(), @@ -249,6 +251,7 @@ public void periodDataBuildUpon_build_isEqual() { new SimpleBasePlayer.PeriodData.Builder(/* uid= */ new Object()) .setIsPlaceholder(true) .setDurationUs(600_000) + .setOriginalDurationUs(700_000) .setAdPlaybackState( new AdPlaybackState(/* adsId= */ new Object(), /* adGroupTimesUs...= */ 555, 666)) .build(); @@ -829,23 +832,16 @@ public void mediaItemDataBuilderBuild_setsCorrectValues() { } @Test - public void mediaItemDataBuilderBuild_presentationStartTimeIfNotLive_throwsException() { - assertThrows( - IllegalArgumentException.class, - () -> - new SimpleBasePlayer.MediaItemData.Builder(/* uid= */ new Object()) - .setPresentationStartTimeMs(12) - .build()); - } + public void mediaItemDataBuilderBuild_startTimeIfNotLive_setsValues() { + SimpleBasePlayer.MediaItemData mediaItemData = + new SimpleBasePlayer.MediaItemData.Builder(/* uid= */ new Object()) + .setPresentationStartTimeMs(12) + .setWindowStartTimeMs(23) + .build(); - @Test - public void mediaItemDataBuilderBuild_windowStartTimeIfNotLive_throwsException() { - assertThrows( - IllegalArgumentException.class, - () -> - new SimpleBasePlayer.MediaItemData.Builder(/* uid= */ new Object()) - .setWindowStartTimeMs(12) - .build()); + assertThat(mediaItemData.liveConfiguration).isNull(); + assertThat(mediaItemData.presentationStartTimeMs).isEqualTo(12); + assertThat(mediaItemData.windowStartTimeMs).isEqualTo(23); } @Test @@ -869,6 +865,13 @@ public void mediaItemDataBuilderBuild_elapsedEpochOffsetIfNotLive_throwsExceptio .setWindowStartTimeMs(12) .setPresentationStartTimeMs(13) .build()); + assertThrows( + IllegalArgumentException.class, + () -> + new SimpleBasePlayer.MediaItemData.Builder(/* uid= */ new Object()) + .setWindowStartTimeMs(12) + .setPresentationStartTimeMs(13) + .build()); } @Test @@ -907,12 +910,14 @@ public void periodDataBuilderBuild_setsCorrectValues() { new SimpleBasePlayer.PeriodData.Builder(uid) .setIsPlaceholder(true) .setDurationUs(600_000) + .setOriginalDurationUs(700_000) .setAdPlaybackState(adPlaybackState) .build(); assertThat(periodData.uid).isEqualTo(uid); assertThat(periodData.isPlaceholder).isTrue(); assertThat(periodData.durationUs).isEqualTo(600_000); + assertThat(periodData.originalDurationUs).isEqualTo(700_000); assertThat(periodData.adPlaybackState).isEqualTo(adPlaybackState); } @@ -981,6 +986,7 @@ public void getterMethods_noOtherMethodCalls_returnCurrentState() { new SimpleBasePlayer.PeriodData.Builder(periodUid) .setIsPlaceholder(true) .setDurationUs(600_000) + .setOriginalDurationUs(700_000) .setAdPlaybackState( new AdPlaybackState( /* adsId= */ new Object(), /* adGroupTimesUs...= */ 555, 666)) @@ -1105,12 +1111,14 @@ protected State getState() { Timeline.Period period = timeline.getPeriod(/* periodIndex= */ 0, new Timeline.Period(), /* setIds= */ true); assertThat(period.durationUs).isEqualTo(C.TIME_UNSET); + assertThat(period.originalDurationUs).isEqualTo(C.TIME_UNSET); assertThat(period.isPlaceholder).isFalse(); assertThat(period.positionInWindowUs).isEqualTo(0); assertThat(period.windowIndex).isEqualTo(0); assertThat(period.getAdGroupCount()).isEqualTo(0); period = timeline.getPeriod(/* periodIndex= */ 1, new Timeline.Period(), /* setIds= */ true); assertThat(period.durationUs).isEqualTo(600_000); + assertThat(period.originalDurationUs).isEqualTo(700_000); assertThat(period.isPlaceholder).isTrue(); assertThat(period.positionInWindowUs).isEqualTo(-100_000); assertThat(period.windowIndex).isEqualTo(1); @@ -1275,12 +1283,14 @@ protected State getState() { Timeline.Period period = timeline.getPeriod(/* periodIndex= */ 0, new Timeline.Period(), /* setIds= */ true); assertThat(period.durationUs).isEqualTo(C.TIME_UNSET); + assertThat(period.originalDurationUs).isEqualTo(C.TIME_UNSET); assertThat(period.isPlaceholder).isFalse(); assertThat(period.positionInWindowUs).isEqualTo(0); assertThat(period.windowIndex).isEqualTo(0); assertThat(period.getAdGroupCount()).isEqualTo(0); period = timeline.getPeriod(/* periodIndex= */ 1, new Timeline.Period(), /* setIds= */ true); assertThat(period.durationUs).isEqualTo(600_000); + assertThat(period.originalDurationUs).isEqualTo(C.TIME_UNSET); assertThat(period.isPlaceholder).isTrue(); assertThat(period.positionInWindowUs).isEqualTo(-100_000); assertThat(period.windowIndex).isEqualTo(1); diff --git a/libraries/common/src/test/java/androidx/media3/common/TimelineTest.java b/libraries/common/src/test/java/androidx/media3/common/TimelineTest.java index 9ebe8032d8e..7fce5cdc85a 100644 --- a/libraries/common/src/test/java/androidx/media3/common/TimelineTest.java +++ b/libraries/common/src/test/java/androidx/media3/common/TimelineTest.java @@ -224,6 +224,10 @@ public void periodEquals() { otherPeriod.durationUs = 11L; assertThat(period).isNotEqualTo(otherPeriod); + otherPeriod = new Timeline.Period(); + otherPeriod.originalDurationUs = 12L; + assertThat(period).isNotEqualTo(otherPeriod); + otherPeriod = new Timeline.Period(); AdPlaybackState adPlaybackState = new AdPlaybackState( @@ -240,6 +244,7 @@ public void periodEquals() { period.uid = new Object(); period.windowIndex = 1; period.durationUs = 123L; + period.originalDurationUs = 456L; period.adPlaybackState = adPlaybackState; period.isPlaceholder = true; @@ -251,7 +256,8 @@ public void periodEquals() { period.durationUs, /* positionInWindowUs= */ 0, period.adPlaybackState, - period.isPlaceholder); + period.isPlaceholder, + period.originalDurationUs); assertThat(period).isEqualTo(otherPeriod); } @@ -425,6 +431,7 @@ public void period_toBundleSkipsDefaultValues_fromBundleRestoresThem() { // Please refrain from altering these default values since doing so would cause issues with // backwards compatibility. period.durationUs = C.TIME_UNSET; + period.originalDurationUs = C.TIME_UNSET; Bundle periodBundle = period.toBundle(MediaLibraryInfo.INTERFACE_VERSION); @@ -447,6 +454,7 @@ public void roundTripViaBundle_ofPeriod_yieldsEqualInstanceExceptIds() { period.uid = new Object(); period.windowIndex = 1; period.durationUs = 123_000; + period.originalDurationUs = 456_000; period.positionInWindowUs = 4_000; period.isPlaceholder = true; diff --git a/libraries/common/src/test/java/androidx/media3/common/util/CodecSpecificDataUtilTest.java b/libraries/common/src/test/java/androidx/media3/common/util/CodecSpecificDataUtilTest.java index a46c045586e..ec81cb14379 100644 --- a/libraries/common/src/test/java/androidx/media3/common/util/CodecSpecificDataUtilTest.java +++ b/libraries/common/src/test/java/androidx/media3/common/util/CodecSpecificDataUtilTest.java @@ -28,6 +28,8 @@ import androidx.media3.common.MimeTypes; import androidx.media3.test.utils.TestUtil; import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import java.nio.ByteBuffer; import org.junit.Test; import org.junit.runner.RunWith; @@ -368,6 +370,90 @@ public void getMediaCodecProfileAndLevel_mvHevcWithNoMatchingMediaCodecConstant_ .isEqualTo(MimeTypes.VIDEO_AV1); } + @Test + public void isHdr10PlusMetadata_withHdr10PlusPrefix_returnsTrue() { + byte[] hdr10PlusBytes = + TestUtil.createByteArray(0xB5, 0x00, 0x3C, 0x00, 0x01, 0x04, 0x00, 0x01, 0x02, 0x03); + ByteBuffer buffer = ByteBuffer.wrap(hdr10PlusBytes); + assertThat(CodecSpecificDataUtil.isHdr10PlusMetadata(buffer)).isTrue(); + } + + @Test + public void isHdr10PlusMetadata_withNonHdr10PlusPrefix_returnsFalse() { + byte[] nonHdr10PlusBytes = + TestUtil.createByteArray(0xB5, 0x00, 0x90, 0x00, 0x01, 0x00, 0x00, 0x01, 0x02, 0x03); + ByteBuffer buffer = ByteBuffer.wrap(nonHdr10PlusBytes); + assertThat(CodecSpecificDataUtil.isHdr10PlusMetadata(buffer)).isFalse(); + } + + @Test + public void isHdr10PlusMetadata_withShortBuffer_returnsFalse() { + byte[] shortBytes = TestUtil.createByteArray(0xB5, 0x00, 0x3C); + ByteBuffer buffer = ByteBuffer.wrap(shortBytes); + assertThat(CodecSpecificDataUtil.isHdr10PlusMetadata(buffer)).isFalse(); + } + + @Test + public void isHdr10PlusMetadata_withOffset_returnsTrue() { + byte[] bytes = + TestUtil.createByteArray(0x00, 0x00, 0x00, 0x01, 0xB5, 0x00, 0x3C, 0x00, 0x01, 0x04, 0x00); + ByteBuffer buffer = ByteBuffer.wrap(bytes); + assertThat(CodecSpecificDataUtil.isHdr10PlusMetadata(buffer, 4, buffer.limit())).isTrue(); + } + + @Test + public void isHagcMetadata_withHagcPrefix_returnsTrue() { + byte[] hagcBytes = + TestUtil.createByteArray(0xB5, 0x00, 0x90, 0x00, 0x01, 0x00, 0x00, 0x01, 0x02, 0x03); + assertThat(CodecSpecificDataUtil.isHagcMetadata(hagcBytes, hagcBytes.length)).isTrue(); + + ByteBuffer buffer = ByteBuffer.wrap(hagcBytes); + assertThat(CodecSpecificDataUtil.isHagcMetadata(buffer)).isTrue(); + } + + @Test + public void isHagcMetadata_withShortBuffer_returnsFalse() { + byte[] shortBytes = TestUtil.createByteArray(0xB5, 0x00, 0x90); + assertThat(CodecSpecificDataUtil.isHagcMetadata(shortBytes, shortBytes.length)).isFalse(); + + ByteBuffer buffer = ByteBuffer.wrap(shortBytes); + assertThat(CodecSpecificDataUtil.isHagcMetadata(buffer)).isFalse(); + } + + @Test + public void isHagcTrack_withValidHagc_returnsTrue() { + byte[] hagcBytes = + TestUtil.createByteArray(0xB5, 0x00, 0x90, 0x00, 0x01, 0x00, 0x00, 0x01, 0x02, 0x03); + Format format = + new Format.Builder() + .setSampleMimeType(MimeTypes.APPLICATION_ITUT_T35) + .setInitializationData(ImmutableList.of(hagcBytes)) + .build(); + assertThat(CodecSpecificDataUtil.isHagcTrack(format)).isTrue(); + } + + @Test + public void isHagcTrack_withoutHagc_returnsFalse() { + byte[] badBytes = TestUtil.createByteArray(0x00, 0x01, 0x02); + Format formatWithBadBytes = + new Format.Builder() + .setSampleMimeType(MimeTypes.APPLICATION_ITUT_T35) + .setInitializationData(ImmutableList.of(badBytes)) + .build(); + assertThat(CodecSpecificDataUtil.isHagcTrack(formatWithBadBytes)).isFalse(); + + Format formatWithoutInitData = + new Format.Builder().setSampleMimeType(MimeTypes.APPLICATION_ITUT_T35).build(); + assertThat(CodecSpecificDataUtil.isHagcTrack(formatWithoutInitData)).isFalse(); + + Format formatWithWrongMimeType = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H265) + .setInitializationData(ImmutableList.of(badBytes)) + .build(); + assertThat(CodecSpecificDataUtil.isHagcTrack(formatWithWrongMimeType)).isFalse(); + } + private static void assertCodecProfileAndLevelForCodecsString( String sampleMimeType, String codecs, int profile, int level) { Format format = diff --git a/libraries/common/src/test/java/androidx/media3/common/util/GlUtilTest.java b/libraries/common/src/test/java/androidx/media3/common/util/GlUtilTest.java new file mode 100644 index 00000000000..f7b0665c63b --- /dev/null +++ b/libraries/common/src/test/java/androidx/media3/common/util/GlUtilTest.java @@ -0,0 +1,47 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.common.util; + +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; + +import androidx.media3.common.video.SyncFenceWrapper; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link GlUtil}. */ +@RunWith(AndroidJUnit4.class) +public final class GlUtilTest { + + @Test + public void createSyncFences_withoutNativeFenceSupport_returnsEmptyList() throws Exception { + // Returns an empty list in unit tests because: + // - On SDK < 33, native sync fences are unsupported by the platform. + // - On SDK >= 33, Robolectric's shadow EGL does not report the EGL_ANDROID_native_fence_sync + // extension. + // The successful creation path on supported hardware is verified in GlUtilAndroidTest. + ImmutableList syncFences = GlUtil.createSyncFences(/* count= */ 1); + + assertThat(syncFences).isEmpty(); + } + + @Test + public void createSyncFences_countZero_throwsIllegalArgumentException() throws Exception { + assertThrows(IllegalArgumentException.class, () -> GlUtil.createSyncFences(/* count= */ 0)); + } +} diff --git a/libraries/common/src/test/java/androidx/media3/common/util/HandlerExecutorTest.java b/libraries/common/src/test/java/androidx/media3/common/util/HandlerExecutorTest.java new file mode 100644 index 00000000000..a3d4a18aebc --- /dev/null +++ b/libraries/common/src/test/java/androidx/media3/common/util/HandlerExecutorTest.java @@ -0,0 +1,100 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.common.util; + +import static com.google.common.truth.Truth.assertThat; +import static org.robolectric.Shadows.shadowOf; + +import android.os.Handler; +import android.os.HandlerThread; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicReference; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link HandlerExecutor}. */ +@RunWith(AndroidJUnit4.class) +public final class HandlerExecutorTest { + + private HandlerThread handlerThread; + private Handler handler; + + @Before + public void setUp() { + handlerThread = new HandlerThread("HandlerExecutorTest"); + handlerThread.start(); + handler = new Handler(handlerThread.getLooper()); + } + + @After + public void tearDown() { + handlerThread.quit(); + } + + @Test + public void execute_withHandler_runsCommandOnHandlerThread() { + AtomicReference executionThread = new AtomicReference<>(); + AtomicReference caughtError = new AtomicReference<>(); + HandlerExecutor executor = new HandlerExecutor(handler, caughtError::set); + + executor.execute(() -> executionThread.set(Thread.currentThread())); + shadowOf(handlerThread.getLooper()).idle(); + + assertThat(executionThread.get()).isEqualTo(handlerThread); + assertThat(caughtError.get()).isNull(); + } + + @Test + public void execute_withHandlerWrapper_runsCommandOnHandlerThread() { + HandlerWrapper handlerWrapper = + Clock.DEFAULT.createHandler(handlerThread.getLooper(), /* callback= */ null); + AtomicBoolean ran = new AtomicBoolean(); + AtomicReference caughtError = new AtomicReference<>(); + HandlerExecutor executor = new HandlerExecutor(handlerWrapper, caughtError::set); + + executor.execute(() -> ran.set(true)); + shadowOf(handlerThread.getLooper()).idle(); + + assertThat(ran.get()).isTrue(); + assertThat(caughtError.get()).isNull(); + } + + @Test + public void execute_whenCommandThrowsRuntimeException_propagatesToListenerWithoutWrapping() { + IllegalStateException expectedException = new IllegalStateException("Test failure"); + AtomicReference caughtError = new AtomicReference<>(); + AtomicReference errorCallbackThread = new AtomicReference<>(); + HandlerExecutor executor = + new HandlerExecutor( + handler, + e -> { + caughtError.set(e); + errorCallbackThread.set(Thread.currentThread()); + }); + + executor.execute( + () -> { + throw expectedException; + }); + shadowOf(handlerThread.getLooper()).idle(); + + assertThat(caughtError.get()).isSameInstanceAs(expectedException); + assertThat(errorCallbackThread.get()).isEqualTo(handlerThread); + } +} diff --git a/libraries/common/src/test/java/androidx/media3/common/util/MediaFormatUtilTest.java b/libraries/common/src/test/java/androidx/media3/common/util/MediaFormatUtilTest.java index fb89fef13f5..da48b206c3a 100644 --- a/libraries/common/src/test/java/androidx/media3/common/util/MediaFormatUtilTest.java +++ b/libraries/common/src/test/java/androidx/media3/common/util/MediaFormatUtilTest.java @@ -58,6 +58,36 @@ public void createFormatFromMediaFormat_withEmptyMap_generatesExpectedFormat() { assertThat(format.initializationData).isEmpty(); } + @Test + public void createFormatFromMediaFormat_withZeroFrameRate_setsFrameRateToNoValue() { + MediaFormat mediaFormat = new MediaFormat(); + mediaFormat.setFloat(MediaFormat.KEY_FRAME_RATE, 0.0f); + + Format format = MediaFormatUtil.createFormatFromMediaFormat(mediaFormat); + + assertThat(format.frameRate).isEqualTo(Format.NO_VALUE); + } + + @Test + public void createFormatFromMediaFormat_withNegativeFrameRate_setsFrameRateToNoValue() { + MediaFormat mediaFormat = new MediaFormat(); + mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, -10); + + Format format = MediaFormatUtil.createFormatFromMediaFormat(mediaFormat); + + assertThat(format.frameRate).isEqualTo(Format.NO_VALUE); + } + + @Test + public void createFormatFromMediaFormat_withNanFrameRate_setsFrameRateToNoValue() { + MediaFormat mediaFormat = new MediaFormat(); + mediaFormat.setFloat(MediaFormat.KEY_FRAME_RATE, Float.NaN); + + Format format = MediaFormatUtil.createFormatFromMediaFormat(mediaFormat); + + assertThat(format.frameRate).isEqualTo(Format.NO_VALUE); + } + @Test public void createFormatFromMediaFormat_withContradictoryChannelMask_dropsChannelMask() { MediaFormat mediaFormat = new MediaFormat(); diff --git a/libraries/common/src/test/java/androidx/media3/common/util/ParsableByteArrayTest.java b/libraries/common/src/test/java/androidx/media3/common/util/ParsableByteArrayTest.java index a623ce27195..a324163669d 100644 --- a/libraries/common/src/test/java/androidx/media3/common/util/ParsableByteArrayTest.java +++ b/libraries/common/src/test/java/androidx/media3/common/util/ParsableByteArrayTest.java @@ -27,33 +27,29 @@ import static org.junit.Assume.assumeFalse; import static org.junit.Assume.assumeTrue; +import androidx.media3.common.Flags; +import androidx.media3.test.utils.BindFlag; +import androidx.media3.test.utils.Media3FlagsRule; import androidx.media3.test.utils.TestUtil; import com.google.common.primitives.Bytes; import com.google.common.primitives.Ints; import com.google.testing.junit.testparameterinjector.TestParameter; -import com.google.testing.junit.testparameterinjector.TestParameterInjector; import java.nio.ByteBuffer; import java.util.Arrays; -import org.junit.After; -import org.junit.Before; +import org.junit.Rule; import org.junit.Test; import org.junit.runner.RunWith; +import org.robolectric.RobolectricTestParameterInjector; /** Tests for {@link ParsableByteArray}. */ -@RunWith(TestParameterInjector.class) +@RunWith(RobolectricTestParameterInjector.class) public final class ParsableByteArrayTest { - @TestParameter private boolean enforceLimit; + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); - @Before - public void overrideLimitEnforcement() { - ParsableByteArray.setShouldEnforceLimitOnLegacyMethods(enforceLimit); - } - - @After - public void resetLimitEnforcement() { - ParsableByteArray.setShouldEnforceLimitOnLegacyMethods(null); - } + @TestParameter + @BindFlag(Flags.FLAG_ENFORCE_PARSABLE_BYTE_ARRAY_LIMIT) + private boolean enforceLimit; private static final byte[] TEST_DATA = new byte[] {0x0F, (byte) 0xFF, (byte) 0x42, (byte) 0x0F, 0x00, 0x00, 0x00, 0x00}; @@ -115,6 +111,34 @@ public void bytesLeft_positionExceedsLimit_returnsZero() { assertThat(array.bytesLeft()).isEqualTo(0); } + @Test + public void readByte() { + testReadByte((byte) -1); + testReadByte((byte) 0); + testReadByte((byte) 1); + testReadByte(Byte.MIN_VALUE); + testReadByte(Byte.MAX_VALUE); + } + + @Test + public void readByte_exceedsLimit_throwsException() { + ParsableByteArray testArray = new ParsableByteArray(0); + assertThrows(IndexOutOfBoundsException.class, testArray::readByte); + } + + private static void testReadByte(byte testValue) { + ParsableByteArray testArray = new ParsableByteArray(new byte[] {testValue}); + byte readValue = testArray.readByte(); + + assertThat(readValue).isEqualTo(testValue); + assertThat(testArray.getPosition()).isEqualTo(1); + + testArray.skipBytes(-1); + readValue = testArray.readByte(); + assertThat(readValue).isEqualTo(testValue); + assertThat(testArray.getPosition()).isEqualTo(1); + } + @Test public void readShort() { testReadShort((short) -1); diff --git a/libraries/common/src/test/java/androidx/media3/common/util/UtilTest.java b/libraries/common/src/test/java/androidx/media3/common/util/UtilTest.java index 09fcaed07be..eddd0f996ea 100644 --- a/libraries/common/src/test/java/androidx/media3/common/util/UtilTest.java +++ b/libraries/common/src/test/java/androidx/media3/common/util/UtilTest.java @@ -53,6 +53,7 @@ import android.util.SparseLongArray; import androidx.media3.common.C; import androidx.media3.common.Format; +import androidx.media3.common.MimeTypes; import androidx.media3.test.utils.TestUtil; import com.google.common.io.ByteStreams; import com.google.common.primitives.Bytes; @@ -1832,6 +1833,18 @@ public void getAudioTrackChannelConfig_withoutChannelMask_infersFromChannelCount assertThat(Util.getAudioTrackChannelConfig(format)).isEqualTo(AudioFormat.CHANNEL_OUT_5POINT1); } + @Test + public void getVideoCodecFlagForMime_returnsProperFlag() { + assertThat(Util.getVideoCodecFlagForMime(MimeTypes.VIDEO_H264)) + .isEqualTo(C.VIDEO_CODEC_FLAG_H264); + assertThat(Util.getVideoCodecFlagForMime(MimeTypes.VIDEO_H265)) + .isEqualTo(C.VIDEO_CODEC_FLAG_H265); + assertThat(Util.getVideoCodecFlagForMime(MimeTypes.VIDEO_AV1)) + .isEqualTo(C.VIDEO_CODEC_FLAG_AV1); + assertThrows( + IllegalArgumentException.class, () -> Util.getVideoCodecFlagForMime("video/unsupported")); + } + private static void assertEscapeUnescapeFileName(String fileName, String escapedFileName) { assertThat(escapeFileName(fileName)).isEqualTo(escapedFileName); assertThat(unescapeFileName(escapedFileName)).isEqualTo(fileName); diff --git a/libraries/common/src/test/java/androidx/media3/common/video/DefaultHardwareBufferFrameTest.java b/libraries/common/src/test/java/androidx/media3/common/video/DefaultHardwareBufferFrameTest.java new file mode 100644 index 00000000000..b6c5a277a78 --- /dev/null +++ b/libraries/common/src/test/java/androidx/media3/common/video/DefaultHardwareBufferFrameTest.java @@ -0,0 +1,378 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.common.video; + +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.truth.Truth.assertWithMessage; +import static com.google.common.util.concurrent.MoreExecutors.directExecutor; +import static java.util.concurrent.TimeUnit.MILLISECONDS; +import static org.junit.Assert.assertThrows; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyLong; +import static org.mockito.Mockito.doAnswer; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +import android.hardware.HardwareBuffer; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableMap; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.Executor; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.RejectedExecutionException; +import java.util.concurrent.atomic.AtomicReference; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.annotation.Config; + +/** Unit tests for {@link DefaultHardwareBufferFrame}. */ +@RunWith(AndroidJUnit4.class) +@Config(minSdk = 26) +public final class DefaultHardwareBufferFrameTest { + + private static final long TEST_TIMEOUT_MS = 5_000L; + + @Test + public void release_onDirectExecutor_immediatelyExecutesCallback() { + ReleaseCallback mockCallback = mock(ReleaseCallback.class); + try (HardwareBuffer hardwareBuffer = createHardwareBuffer()) { + DefaultHardwareBufferFrame frame = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer, directExecutor(), mockCallback) + .setContentTimeUs(1_000_000L) + .build(); + + frame.release(/* releaseFence= */ null); + + verify(mockCallback).release(null); + } + } + + @Test + public void release_withReleaseFence_awaitsAndClosesReleaseFence() throws Exception { + CountDownLatch callbackLatch = new CountDownLatch(1); + ReleaseCallback mockCallback = mock(ReleaseCallback.class); + doAnswer( + invocation -> { + callbackLatch.countDown(); + return null; + }) + .when(mockCallback) + .release(any()); + SyncFenceWrapper mockReleaseFence = mock(SyncFenceWrapper.class); + when(mockReleaseFence.awaitMs(anyLong())).thenReturn(true); + try (HardwareBuffer hardwareBuffer = createHardwareBuffer()) { + DefaultHardwareBufferFrame frame = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer, directExecutor(), mockCallback) + .setContentTimeUs(1_000_000L) + .build(); + + frame.release(mockReleaseFence); + + assertWithMessage("Release callback timed out") + .that(callbackLatch.await(TEST_TIMEOUT_MS, MILLISECONDS)) + .isTrue(); + verify(mockReleaseFence).awaitMs(2000); + verify(mockReleaseFence).close(); + verify(mockCallback).release(null); + } + } + + @Test + public void release_alreadyReleasedFrame_closesReleaseFenceSilently() { + SyncFenceWrapper mockReleaseFence = mock(SyncFenceWrapper.class); + try (HardwareBuffer hardwareBuffer = createHardwareBuffer()) { + DefaultHardwareBufferFrame frame = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer, directExecutor(), (fence) -> {}) + .setContentTimeUs(1_000_000L) + .build(); + frame.release(/* releaseFence= */ null); + + frame.release(mockReleaseFence); + + verify(mockReleaseFence).close(); + } + } + + @Test + public void buildUpon_buildWithoutModifications_returnsNewHandleWithSameProperties() { + try (HardwareBuffer hardwareBuffer = createHardwareBuffer()) { + DefaultHardwareBufferFrame frame1 = + new DefaultHardwareBufferFrame.Builder( + hardwareBuffer, directExecutor(), (SyncFenceWrapper fence) -> {}) + .setContentTimeUs(1_000_000L) + .build(); + + DefaultHardwareBufferFrame frame2 = frame1.buildUpon().build(); + + assertThat(frame2).isNotSameInstanceAs(frame1); + assertThat(frame2.getContentTimeUs()).isEqualTo(1_000_000L); + assertThat(frame2.getHardwareBuffer()).isSameInstanceAs(hardwareBuffer); + } + } + + @Test + public void buildUpon_withModifications_returnsNewHandleWithUpdatedProperties() { + try (HardwareBuffer hardwareBuffer = createHardwareBuffer()) { + DefaultHardwareBufferFrame frame1 = + new DefaultHardwareBufferFrame.Builder( + hardwareBuffer, directExecutor(), (SyncFenceWrapper fence) -> {}) + .setContentTimeUs(1_000_000L) + .build(); + + DefaultHardwareBufferFrame frame2 = + frame1.buildUpon().setMetadata(ImmutableMap.of("key", "value")).build(); + + assertThat(frame2).isNotSameInstanceAs(frame1); + assertThat(frame2.getMetadata()).containsEntry("key", "value"); + } + } + + @Test + public void buildUpon_incrementReferenceCountAndReleaseFirstHandle_doesNotReleaseCallback() { + ReleaseCallback mockCallback = mock(ReleaseCallback.class); + try (HardwareBuffer hardwareBuffer = createHardwareBuffer()) { + DefaultHardwareBufferFrame frame1 = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer, directExecutor(), mockCallback) + .setContentTimeUs(1_000_000L) + .build(); + DefaultHardwareBufferFrame unusedFrame2 = + frame1.buildUpon().shouldIncrementReferenceCount().build(); + + frame1.release(/* releaseFence= */ null); + + verify(mockCallback, never()).release(any()); + } + } + + @Test + public void buildUpon_incrementReferenceCountAndReleaseAllHandles_releasesCallback() { + ReleaseCallback mockCallback = mock(ReleaseCallback.class); + try (HardwareBuffer hardwareBuffer = createHardwareBuffer()) { + DefaultHardwareBufferFrame frame1 = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer, directExecutor(), mockCallback) + .setContentTimeUs(1_000_000L) + .build(); + DefaultHardwareBufferFrame frame2 = + frame1.buildUpon().shouldIncrementReferenceCount().build(); + frame1.release(/* releaseFence= */ null); + + frame2.release(/* releaseFence= */ null); + + verify(mockCallback).release(null); + } + } + + @Test + public void buildUpon_andReleaseMultipleReferences_awaitsAndClosesAllFences() throws Exception { + CountDownLatch callbackLatch = new CountDownLatch(1); + ReleaseCallback mockCallback = mock(ReleaseCallback.class); + doAnswer( + invocation -> { + callbackLatch.countDown(); + return null; + }) + .when(mockCallback) + .release(any()); + SyncFenceWrapper mockReleaseFence1 = mock(SyncFenceWrapper.class); + SyncFenceWrapper mockReleaseFence2 = mock(SyncFenceWrapper.class); + when(mockReleaseFence1.awaitMs(anyLong())).thenReturn(true); + when(mockReleaseFence2.awaitMs(anyLong())).thenReturn(true); + try (HardwareBuffer hardwareBuffer = createHardwareBuffer()) { + DefaultHardwareBufferFrame frame1 = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer, directExecutor(), mockCallback) + .setContentTimeUs(1_000_000L) + .build(); + DefaultHardwareBufferFrame frame2 = + frame1.buildUpon().shouldIncrementReferenceCount().build(); + + frame1.release(mockReleaseFence1); + verify(mockCallback, never()).release(any()); + verify(mockReleaseFence1, never()).close(); + + frame2.release(mockReleaseFence2); + assertWithMessage("Release callback timed out") + .that(callbackLatch.await(TEST_TIMEOUT_MS, MILLISECONDS)) + .isTrue(); + verify(mockReleaseFence1).awaitMs(2000); + verify(mockReleaseFence1).close(); + verify(mockReleaseFence2).awaitMs(2000); + verify(mockReleaseFence2).close(); + verify(mockCallback).release(null); + } + } + + @Test + public void buildUpon_withoutIncrementReferenceCount_doesNotIncrementRefCount() { + ReleaseCallback mockCallback = mock(ReleaseCallback.class); + try (HardwareBuffer hardwareBuffer = createHardwareBuffer()) { + DefaultHardwareBufferFrame frame1 = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer, directExecutor(), mockCallback) + .setContentTimeUs(1_000_000L) + .build(); + DefaultHardwareBufferFrame unusedFrame2 = + frame1.buildUpon().setMetadata(ImmutableMap.of("key", "value")).build(); + + frame1.release(/* releaseFence= */ null); + + verify(mockCallback).release(null); + } + } + + @Test + public void buildUpon_withoutIncrementReferenceCount_awaitsAndClosesFenceOnFirstRelease() + throws Exception { + CountDownLatch callbackLatch = new CountDownLatch(1); + ReleaseCallback mockCallback = mock(ReleaseCallback.class); + doAnswer( + invocation -> { + callbackLatch.countDown(); + return null; + }) + .when(mockCallback) + .release(any()); + SyncFenceWrapper mockReleaseFence = mock(SyncFenceWrapper.class); + when(mockReleaseFence.awaitMs(anyLong())).thenReturn(true); + try (HardwareBuffer hardwareBuffer = createHardwareBuffer()) { + DefaultHardwareBufferFrame frame1 = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer, directExecutor(), mockCallback) + .setContentTimeUs(1_000_000L) + .build(); + DefaultHardwareBufferFrame unusedFrame2 = + frame1.buildUpon().setMetadata(ImmutableMap.of("key", "value")).build(); + + frame1.release(mockReleaseFence); + + assertWithMessage("Release callback timed out") + .that(callbackLatch.await(TEST_TIMEOUT_MS, MILLISECONDS)) + .isTrue(); + verify(mockReleaseFence).awaitMs(2000); + verify(mockReleaseFence).close(); + verify(mockCallback).release(null); + } + } + + @Test + public void buildUpon_afterRelease_throwsIllegalStateException() { + try (HardwareBuffer hardwareBuffer = createHardwareBuffer()) { + DefaultHardwareBufferFrame frame = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer, directExecutor(), (fence) -> {}) + .setContentTimeUs(1_000_000L) + .build(); + frame.release(/* releaseFence= */ null); + + assertThrows(IllegalStateException.class, frame::buildUpon); + } + } + + @Test + public void incrementReferenceCount_withoutReferenceCounting_throwsIllegalStateException() { + try (HardwareBuffer hardwareBuffer = createHardwareBuffer()) { + DefaultHardwareBufferFrame.Builder builder = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer); + + assertThrows(IllegalStateException.class, builder::shouldIncrementReferenceCount); + } + } + + @Test + public void + buildUpon_incrementReferenceCountWithoutReferenceCounting_throwsIllegalStateException() { + try (HardwareBuffer hardwareBuffer = createHardwareBuffer()) { + DefaultHardwareBufferFrame frame = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer).build(); + + DefaultHardwareBufferFrame.Builder builder = frame.buildUpon(); + assertThrows(IllegalStateException.class, () -> builder.shouldIncrementReferenceCount()); + } + } + + @Test + public void release_multipleTimes_secondReleaseNoOp() { + ReleaseCallback mockCallback = mock(ReleaseCallback.class); + try (HardwareBuffer hardwareBuffer = createHardwareBuffer()) { + DefaultHardwareBufferFrame frame = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer, directExecutor(), mockCallback) + .setContentTimeUs(1_000_000L) + .build(); + frame.release(/* releaseFence= */ null); + + frame.release(/* releaseFence= */ null); + + verify(mockCallback).release(null); + } + } + + @Test + public void release_onBackgroundExecutor_executesCallbackOnExecutor() throws Exception { + ExecutorService executor = Executors.newSingleThreadExecutor(); + Thread executorThread = executor.submit(Thread::currentThread).get(); + CountDownLatch callbackLatch = new CountDownLatch(1); + AtomicReference callbackThread = new AtomicReference<>(); + + try (HardwareBuffer hardwareBuffer = createHardwareBuffer()) { + DefaultHardwareBufferFrame frame = + new DefaultHardwareBufferFrame.Builder( + hardwareBuffer, + executor, + (releaseFence) -> { + callbackThread.set(Thread.currentThread()); + callbackLatch.countDown(); + }) + .build(); + + frame.release(/* releaseFence= */ null); + + assertWithMessage("Release callback timed out") + .that(callbackLatch.await(TEST_TIMEOUT_MS, MILLISECONDS)) + .isTrue(); + assertThat(callbackThread.get()).isSameInstanceAs(executorThread); + } finally { + executor.shutdownNow(); + } + } + + @Test + public void release_whenExecutorThrows_throwsException() { + Executor failingExecutor = + r -> { + throw new RejectedExecutionException("Executor shutdown"); + }; + ReleaseCallback mockCallback = mock(ReleaseCallback.class); + try (HardwareBuffer hardwareBuffer = createHardwareBuffer()) { + DefaultHardwareBufferFrame frame = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer, failingExecutor, mockCallback) + .setContentTimeUs(1_000_000L) + .build(); + + assertThrows( + RejectedExecutionException.class, + () -> frame.release(/* releaseFence= */ (SyncFenceWrapper) null)); + verify(mockCallback, never()).release(any()); + } + } + + private static HardwareBuffer createHardwareBuffer() { + return HardwareBuffer.create( + /* width= */ 64, + /* height= */ 64, + HardwareBuffer.RGBA_8888, + /* layers= */ 1, + HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE); + } +} diff --git a/libraries/common/src/test/java/androidx/media3/common/video/DefaultImagePlanesFrameTest.java b/libraries/common/src/test/java/androidx/media3/common/video/DefaultImagePlanesFrameTest.java new file mode 100644 index 00000000000..ee41cbb4505 --- /dev/null +++ b/libraries/common/src/test/java/androidx/media3/common/video/DefaultImagePlanesFrameTest.java @@ -0,0 +1,177 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.common.video; + +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.util.concurrent.MoreExecutors.directExecutor; +import static org.junit.Assert.assertThrows; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; + +import androidx.media3.common.Format; +import androidx.media3.common.video.DefaultImagePlanesFrame.DefaultPlane; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import java.nio.ByteBuffer; +import java.util.concurrent.atomic.AtomicInteger; +import org.junit.Test; +import org.junit.runner.RunWith; + +@RunWith(AndroidJUnit4.class) +public final class DefaultImagePlanesFrameTest { + + @Test + public void defaultPlaneConstructor_withNonDirectBuffer_throwsIllegalArgumentException() { + ByteBuffer nonDirectBuffer = ByteBuffer.allocate(64); + + assertThrows( + IllegalArgumentException.class, + () -> new DefaultPlane(nonDirectBuffer, /* rowStride= */ 8, /* pixelStride= */ 1)); + } + + @Test + public void buildUpon_buildWithoutModifications_returnsNewHandleWithSameProperties() { + ImmutableList planes = createSamplePlanes(); + Format format = new Format.Builder().setWidth(128).setHeight(128).build(); + ImmutableMap metadata = ImmutableMap.of("key", "value"); + Object internalImage = new Object(); + DefaultImagePlanesFrame frame1 = + new DefaultImagePlanesFrame.Builder(planes) + .setFormat(format) + .setMetadata(metadata) + .setInternalImage(internalImage) + .setContentTimeUs(1_000_000L) + .build(); + + DefaultImagePlanesFrame frame2 = frame1.buildUpon().build(); + + assertThat(frame2).isNotSameInstanceAs(frame1); + assertThat(frame2.getPlanes()).isEqualTo(planes); + assertThat(frame2.getFormat()).isEqualTo(format); + assertThat(frame2.getMetadata()).isEqualTo(metadata); + assertThat(frame2.getContentTimeUs()).isEqualTo(1_000_000L); + assertThat(frame2.getInternalImage()).isSameInstanceAs(internalImage); + } + + @Test + public void buildUpon_withModifiedMetadata_updatesMetadataAndPreservesOtherProperties() { + ImmutableList planes = createSamplePlanes(); + Format format = new Format.Builder().setWidth(128).setHeight(128).build(); + ImmutableMap initialMetadata = ImmutableMap.of("initialKey", "initialValue"); + Object internalImage = new Object(); + DefaultImagePlanesFrame frame1 = + new DefaultImagePlanesFrame.Builder(planes) + .setFormat(format) + .setMetadata(initialMetadata) + .setInternalImage(internalImage) + .setContentTimeUs(1_000_000L) + .build(); + ImmutableMap newMetadata = ImmutableMap.of("newKey", "newValue"); + + DefaultImagePlanesFrame frame2 = frame1.buildUpon().setMetadata(newMetadata).build(); + + assertThat(frame2).isNotSameInstanceAs(frame1); + assertThat(frame2.getMetadata()).isEqualTo(newMetadata); + assertThat(frame2.getFormat()).isEqualTo(format); + assertThat(frame2.getPlanes()).isEqualTo(planes); + assertThat(frame2.getContentTimeUs()).isEqualTo(1_000_000L); + assertThat(frame2.getInternalImage()).isSameInstanceAs(internalImage); + } + + @Test + public void buildUpon_withIncrementReferenceCountAndOneHandleReleased_doesNotInvokeCallback() { + AtomicInteger callbackCount = new AtomicInteger(); + DefaultImagePlanesFrame frame1 = + new DefaultImagePlanesFrame.Builder( + createSamplePlanes(), directExecutor(), fence -> callbackCount.incrementAndGet()) + .build(); + DefaultImagePlanesFrame unusedFrame2 = + frame1.buildUpon().shouldIncrementReferenceCount().build(); + + frame1.release(/* releaseFence= */ null); + + assertThat(callbackCount.get()).isEqualTo(0); + } + + @Test + public void buildUpon_withIncrementReferenceCountAndAllHandlesReleased_invokesCallback() { + AtomicInteger callbackCount = new AtomicInteger(); + DefaultImagePlanesFrame frame1 = + new DefaultImagePlanesFrame.Builder( + createSamplePlanes(), directExecutor(), fence -> callbackCount.incrementAndGet()) + .build(); + DefaultImagePlanesFrame frame2 = frame1.buildUpon().shouldIncrementReferenceCount().build(); + + frame1.release(/* releaseFence= */ null); + frame2.release(/* releaseFence= */ null); + + assertThat(callbackCount.get()).isEqualTo(1); + } + + @Test + public void shouldIncrementReferenceCount_onUnmanagedFrame_throwsIllegalStateException() { + DefaultImagePlanesFrame frame = + new DefaultImagePlanesFrame.Builder(createSamplePlanes()).build(); + DefaultImagePlanesFrame.Builder builder = frame.buildUpon(); + + assertThrows(IllegalStateException.class, builder::shouldIncrementReferenceCount); + } + + @Test + public void buildUpon_afterRelease_throwsIllegalStateException() { + DefaultImagePlanesFrame frame = + new DefaultImagePlanesFrame.Builder(createSamplePlanes()).build(); + frame.release(/* releaseFence= */ null); + + assertThrows(IllegalStateException.class, frame::buildUpon); + } + + @Test + public void release_withoutLifecycleManagement_closesReleaseFence() { + DefaultImagePlanesFrame frame = + new DefaultImagePlanesFrame.Builder(createSamplePlanes()).build(); + SyncFenceWrapper mockReleaseFence = mock(SyncFenceWrapper.class); + + frame.release(mockReleaseFence); + + verify(mockReleaseFence).close(); + } + + @Test + public void release_multipleTimes_secondReleaseNoOp() { + AtomicInteger callbackCount = new AtomicInteger(); + DefaultImagePlanesFrame frame = + new DefaultImagePlanesFrame.Builder( + createSamplePlanes(), directExecutor(), fence -> callbackCount.incrementAndGet()) + .build(); + frame.release(/* releaseFence= */ null); + + frame.release(/* releaseFence= */ null); + + assertThat(callbackCount.get()).isEqualTo(1); + } + + private static ImmutableList createSamplePlanes() { + ByteBuffer yBuffer = ByteBuffer.allocateDirect(64 * 64); + ByteBuffer uBuffer = ByteBuffer.allocateDirect(32 * 32); + ByteBuffer vBuffer = ByteBuffer.allocateDirect(32 * 32); + return ImmutableList.of( + new DefaultPlane(yBuffer, /* rowStride= */ 64, /* pixelStride= */ 1), + new DefaultPlane(uBuffer, /* rowStride= */ 32, /* pixelStride= */ 1), + new DefaultPlane(vBuffer, /* rowStride= */ 32, /* pixelStride= */ 1)); + } +} diff --git a/libraries/common/src/test/java/androidx/media3/common/video/SharedStateReferenceCounterTest.java b/libraries/common/src/test/java/androidx/media3/common/video/SharedStateReferenceCounterTest.java new file mode 100644 index 00000000000..3be3535d061 --- /dev/null +++ b/libraries/common/src/test/java/androidx/media3/common/video/SharedStateReferenceCounterTest.java @@ -0,0 +1,253 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.common.video; + +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.truth.Truth.assertWithMessage; +import static com.google.common.util.concurrent.MoreExecutors.directExecutor; +import static java.util.concurrent.TimeUnit.MILLISECONDS; +import static org.junit.Assert.assertThrows; +import static org.mockito.ArgumentMatchers.anyLong; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +import androidx.annotation.Nullable; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.atomic.AtomicInteger; +import org.junit.Test; +import org.junit.runner.RunWith; + +@RunWith(AndroidJUnit4.class) +public final class SharedStateReferenceCounterTest { + + private static final long TEST_TIMEOUT_MS = 500L; + + @Test + public void checkNotReleased_afterRelease_throwsIllegalStateException() { + SharedStateReferenceCounter counter = new SharedStateReferenceCounter.Builder().build(); + counter.release(/* releaseFence= */ null); + + IllegalStateException exception = + assertThrows( + IllegalStateException.class, + () -> counter.checkNotReleased("Cannot use released frame.")); + assertThat(exception).hasMessageThat().isEqualTo("Cannot use released frame."); + } + + @Test + public void checkNotReleased_beforeRelease_doesNotThrow() { + SharedStateReferenceCounter counter = new SharedStateReferenceCounter.Builder().build(); + + counter.checkNotReleased("Cannot use released frame."); + } + + @Test + public void shouldIncrementReferenceCount_withoutReleaseCallback_throwsIllegalStateException() { + SharedStateReferenceCounter.Builder builder = new SharedStateReferenceCounter.Builder(); + + assertThrows(IllegalStateException.class, builder::shouldIncrementReferenceCount); + } + + @Test + public void release_withoutReleaseCallbackAndNullFence_marksReleased() { + SharedStateReferenceCounter counter = new SharedStateReferenceCounter.Builder().build(); + + counter.release(/* releaseFence= */ null); + + assertThat(counter.isReleased()).isTrue(); + } + + @Test + public void release_withoutReleaseCallbackAndWithFence_closesFence() { + SyncFenceWrapper mockReleaseFence = mock(SyncFenceWrapper.class); + SharedStateReferenceCounter counter = new SharedStateReferenceCounter.Builder().build(); + + counter.release(mockReleaseFence); + + assertThat(counter.isReleased()).isTrue(); + verify(mockReleaseFence).close(); + } + + @Test + public void release_withoutReleaseCallbackWhenAlreadyReleased_closesFence() { + SyncFenceWrapper mockReleaseFence = mock(SyncFenceWrapper.class); + SharedStateReferenceCounter counter = new SharedStateReferenceCounter.Builder().build(); + counter.release(/* releaseFence= */ null); + + counter.release(mockReleaseFence); + + assertThat(counter.isReleased()).isTrue(); + verify(mockReleaseFence).close(); + } + + @Test + public void release_withReleaseCallbackAndNullFence_delegatesToCallback() { + FakeReleaseCallback callback = new FakeReleaseCallback(); + SharedStateReferenceCounter counter = + new SharedStateReferenceCounter.Builder(directExecutor(), callback).build(); + + counter.release(/* releaseFence= */ null); + + assertThat(counter.isReleased()).isTrue(); + assertThat(callback.getReleaseCount()).isEqualTo(1); + } + + @Test + public void release_withReleaseCallbackAndWithFence_awaitsFenceAndDelegatesToCallback() + throws Exception { + CountDownLatch callbackLatch = new CountDownLatch(1); + FakeReleaseCallback fakeCallback = new FakeReleaseCallback(); + ReleaseCallback callback = + fence -> { + fakeCallback.release(fence); + callbackLatch.countDown(); + }; + SyncFenceWrapper mockReleaseFence = mock(SyncFenceWrapper.class); + when(mockReleaseFence.awaitMs(anyLong())).thenReturn(true); + SharedStateReferenceCounter counter = + new SharedStateReferenceCounter.Builder(directExecutor(), callback).build(); + + counter.release(mockReleaseFence); + + assertWithMessage("Release callback timed out") + .that(callbackLatch.await(TEST_TIMEOUT_MS, MILLISECONDS)) + .isTrue(); + assertThat(counter.isReleased()).isTrue(); + verify(mockReleaseFence).awaitMs(anyLong()); + verify(mockReleaseFence).close(); + assertThat(fakeCallback.getReleaseCount()).isEqualTo(1); + } + + @Test + public void + release_withReleaseCallbackAndFenceTimeout_awaitsAllFencesClosesAndDelegatesToCallback() + throws Exception { + CountDownLatch callbackLatch = new CountDownLatch(1); + FakeReleaseCallback fakeCallback = new FakeReleaseCallback(); + ReleaseCallback callback = + fence -> { + fakeCallback.release(fence); + callbackLatch.countDown(); + }; + SyncFenceWrapper mockTimedOutFence = mock(SyncFenceWrapper.class); + SyncFenceWrapper mockSignaledFence = mock(SyncFenceWrapper.class); + when(mockTimedOutFence.awaitMs(anyLong())).thenReturn(false); + when(mockSignaledFence.awaitMs(anyLong())).thenReturn(true); + SharedStateReferenceCounter counter1 = + new SharedStateReferenceCounter.Builder(directExecutor(), callback).build(); + SharedStateReferenceCounter counter2 = + new SharedStateReferenceCounter.Builder(counter1.getSharedState()) + .shouldIncrementReferenceCount() + .build(); + + counter1.release(mockTimedOutFence); + counter2.release(mockSignaledFence); + + assertWithMessage("Release callback timed out") + .that(callbackLatch.await(TEST_TIMEOUT_MS, MILLISECONDS)) + .isTrue(); + assertThat(counter1.isReleased()).isTrue(); + assertThat(counter2.isReleased()).isTrue(); + verify(mockTimedOutFence).awaitMs(anyLong()); + verify(mockTimedOutFence).close(); + verify(mockSignaledFence).awaitMs(anyLong()); + verify(mockSignaledFence).close(); + assertThat(fakeCallback.getReleaseCount()).isEqualTo(1); + } + + @Test + public void release_alreadyReleasedWithCallback_closesFenceWithoutInvokingCallback() { + FakeReleaseCallback callback = new FakeReleaseCallback(); + SyncFenceWrapper mockReleaseFence = mock(SyncFenceWrapper.class); + SharedStateReferenceCounter counter = + new SharedStateReferenceCounter.Builder(directExecutor(), callback).build(); + counter.release(/* releaseFence= */ null); + + counter.release(mockReleaseFence); + + assertThat(counter.isReleased()).isTrue(); + verify(mockReleaseFence).close(); + assertThat(callback.getReleaseCount()).isEqualTo(1); + } + + @Test + public void release_oneHandleWhileOtherRetained_doesNotInvokeCallback() { + FakeReleaseCallback callback = new FakeReleaseCallback(); + SharedStateReferenceCounter counter1 = + new SharedStateReferenceCounter.Builder(directExecutor(), callback).build(); + SharedStateReferenceCounter counter2 = + new SharedStateReferenceCounter.Builder(counter1.getSharedState()) + .shouldIncrementReferenceCount() + .build(); + + counter1.release(/* releaseFence= */ null); + + assertThat(counter1.isReleased()).isTrue(); + assertThat(counter2.isReleased()).isFalse(); + assertThat(callback.getReleaseCount()).isEqualTo(0); + } + + @Test + public void release_oneHandleMultipleTimesWhileOtherRetained_doesNotInvokeCallback() { + FakeReleaseCallback callback = new FakeReleaseCallback(); + SharedStateReferenceCounter counter1 = + new SharedStateReferenceCounter.Builder(directExecutor(), callback).build(); + SharedStateReferenceCounter counter2 = + new SharedStateReferenceCounter.Builder(counter1.getSharedState()) + .shouldIncrementReferenceCount() + .build(); + + counter1.release(/* releaseFence= */ null); + counter1.release(/* releaseFence= */ null); + + assertThat(counter1.isReleased()).isTrue(); + assertThat(counter2.isReleased()).isFalse(); + assertThat(callback.getReleaseCount()).isEqualTo(0); + } + + @Test + public void release_allHandlesSharingCallback_invokesCallback() { + FakeReleaseCallback callback = new FakeReleaseCallback(); + SharedStateReferenceCounter counter1 = + new SharedStateReferenceCounter.Builder(directExecutor(), callback).build(); + SharedStateReferenceCounter counter2 = + new SharedStateReferenceCounter.Builder(counter1.getSharedState()) + .shouldIncrementReferenceCount() + .build(); + + counter1.release(/* releaseFence= */ null); + counter2.release(/* releaseFence= */ null); + + assertThat(counter1.isReleased()).isTrue(); + assertThat(counter2.isReleased()).isTrue(); + assertThat(callback.getReleaseCount()).isEqualTo(1); + } + + private static final class FakeReleaseCallback implements ReleaseCallback { + private final AtomicInteger releaseCount = new AtomicInteger(); + + @Override + public void release(@Nullable SyncFenceWrapper releaseFence) { + releaseCount.incrementAndGet(); + } + + int getReleaseCount() { + return releaseCount.get(); + } + } +} diff --git a/libraries/common/src/test/java/androidx/media3/common/video/SurfaceHolderFrameWriterTest.java b/libraries/common/src/test/java/androidx/media3/common/video/SurfaceHolderFrameWriterTest.java index 54c84107baf..3c6282f4a08 100644 --- a/libraries/common/src/test/java/androidx/media3/common/video/SurfaceHolderFrameWriterTest.java +++ b/libraries/common/src/test/java/androidx/media3/common/video/SurfaceHolderFrameWriterTest.java @@ -20,6 +20,7 @@ import static org.junit.Assert.assertThrows; import static org.robolectric.Shadows.shadowOf; +import android.graphics.ImageFormat; import android.hardware.HardwareBuffer; import android.os.Handler; import android.os.HandlerThread; @@ -27,11 +28,14 @@ import androidx.media3.common.ColorInfo; import androidx.media3.common.Format; import androidx.media3.common.VideoFrameProcessingException; +import androidx.media3.common.video.DefaultImagePlanesFrame.DefaultPlane; import androidx.media3.common.video.SurfaceHolderFrameWriter.Listener; import androidx.media3.test.utils.FakeHardwareBufferNativeHelpers; import androidx.media3.test.utils.ImageReaderSurfaceHolder; import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableMap; +import java.nio.ByteBuffer; import java.util.concurrent.Executor; import java.util.concurrent.atomic.AtomicBoolean; import org.junit.After; @@ -40,19 +44,41 @@ import org.junit.runner.RunWith; import org.robolectric.annotation.Config; -/** Unit tests for {@link HardwareBufferPool}. */ +/** Unit tests for {@link SurfaceHolderFrameWriter}. */ @RunWith(AndroidJUnit4.class) -@Config(minSdk = 28) public final class SurfaceHolderFrameWriterTest { private static final int WIDTH = 640; private static final int HEIGHT = 480; + private static final ColorInfo PQ_COLOR_INFO = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorRange(C.COLOR_RANGE_LIMITED) + .setColorTransfer(C.COLOR_TRANSFER_ST2084) + .build(); + + private static final ColorInfo HLG_COLOR_INFO = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorRange(C.COLOR_RANGE_LIMITED) + .setColorTransfer(C.COLOR_TRANSFER_HLG) + .build(); + + private static final Format DEFAULT_FORMAT = + new Format.Builder() + .setWidth(WIDTH) + .setHeight(HEIGHT) + .setColorInfo(ColorInfo.SDR_BT709_LIMITED) + .build(); + private ImageReaderSurfaceHolder surfaceHolder; private SurfaceHolderFrameWriter frameWriter; private AtomicBoolean onEnded; private Executor executor; private HandlerThread callbackThread; + private Listener listener; + private FakeHardwareBufferNativeHelpers hardwareBufferNativeHelpers; @Before public void setUp() { @@ -60,11 +86,10 @@ public void setUp() { callbackThread.start(); Handler callbackHandler = new Handler(callbackThread.getLooper()); surfaceHolder = new ImageReaderSurfaceHolder(callbackHandler); - FakeHardwareBufferNativeHelpers hardwareBufferNativeHelpers = - new FakeHardwareBufferNativeHelpers(); + hardwareBufferNativeHelpers = new FakeHardwareBufferNativeHelpers(); onEnded = new AtomicBoolean(); executor = directExecutor(); - Listener listener = + listener = new Listener() { @Override public void onFrameAboutToBeRendered( @@ -139,18 +164,29 @@ public void configure_invalidColorInfo_throwsIllegalArgumentException() { } @Test - public void queueInputFrame_withInvalidFrameType_throwsIllegalArgumentException() { + @Config(maxSdk = 27) + public void configure_nonYv12PixelFormatBelowApi28_throwsIllegalArgumentException() { Format format = new Format.Builder() .setWidth(WIDTH) .setHeight(HEIGHT) + .setPixelFormat(HardwareBuffer.RGB_565) .setColorInfo(ColorInfo.SDR_BT709_LIMITED) .build(); + + assertThrows( + IllegalArgumentException.class, () -> frameWriter.configure(format, /* usage= */ 0)); + } + + @Test + public void queueInputFrame_withInvalidFrameType_throwsIllegalArgumentException() { + // SurfaceHolderFrameWriter requires DefaultImagePlanesFrame below API 28, or + // DefaultHardwareBufferFrame on API 28 and above. Frame invalidFrame = new Frame() { @Override public Format getFormat() { - return format; + return DEFAULT_FORMAT; } @Override @@ -163,13 +199,50 @@ public long getContentTimeUs() { return 0; } }; - frameWriter.configure(format, /* usage= */ 0); + frameWriter.configure(DEFAULT_FORMAT, /* usage= */ 0); assertThrows( IllegalArgumentException.class, () -> frameWriter.queueInputFrame(invalidFrame, /* writeCompleteFence= */ null)); } + @Test + @Config(minSdk = 28) + public void queueInputFrame_withImagePlanesFrameOnApi28AndAbove_throwsIllegalArgumentException() { + DefaultImagePlanesFrame imagePlanesFrame = + new DefaultImagePlanesFrame.Builder( + ImmutableList.of( + new DefaultPlane( + ByteBuffer.allocateDirect(1), /* rowStride= */ 1, /* pixelStride= */ 1))) + .setFormat(DEFAULT_FORMAT) + .build(); + frameWriter.configure(DEFAULT_FORMAT, /* usage= */ 0); + + assertThrows( + IllegalArgumentException.class, + () -> frameWriter.queueInputFrame(imagePlanesFrame, /* writeCompleteFence= */ null)); + } + + @Test + @Config(minSdk = 26, maxSdk = 27) + public void queueInputFrame_withHardwareBufferFrameBelowApi28_throwsIllegalArgumentException() { + try (HardwareBuffer hardwareBuffer = + HardwareBuffer.create( + WIDTH, + HEIGHT, + HardwareBuffer.RGBA_8888, + /* layers= */ 1, + HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE)) { + DefaultHardwareBufferFrame hardwareBufferFrame = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer).setFormat(DEFAULT_FORMAT).build(); + frameWriter.configure(DEFAULT_FORMAT, /* usage= */ 0); + + assertThrows( + IllegalArgumentException.class, + () -> frameWriter.queueInputFrame(hardwareBufferFrame, /* writeCompleteFence= */ null)); + } + } + @Test @Config(sdk = 33) public void configure_propagatesPixelFormat() { @@ -189,54 +262,101 @@ public void configure_propagatesPixelFormat() { } @Test - @Config(sdk = 32) - public void getInfo_hdrFormatOnApi32_returnsFalse() { + @Config(maxSdk = 32) + public void getInfo_hdrFormatBelowApi33_returnsFalse() { Format format = - new Format.Builder() - .setWidth(WIDTH) - .setHeight(HEIGHT) - .setColorInfo( - new ColorInfo.Builder() - .setColorSpace(C.COLOR_SPACE_BT2020) - .setColorRange(C.COLOR_RANGE_LIMITED) - .setColorTransfer(C.COLOR_TRANSFER_ST2084) - .build()) - .build(); + new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).setColorInfo(PQ_COLOR_INFO).build(); assertThat(frameWriter.getInfo().isSupported(format, /* usage= */ 0L)).isFalse(); } @Test @Config(sdk = 33) public void getInfo_hdrFormatOnApi33_returnsTrue() { + Format format = + new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).setColorInfo(PQ_COLOR_INFO).build(); + assertThat(frameWriter.getInfo().isSupported(format, /* usage= */ 0L)).isTrue(); + } + + @Test + @Config(sdk = 33) + public void getInfo_hlgFormatOnApi33_returnsFalse() { + Format format = + new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).setColorInfo(HLG_COLOR_INFO).build(); + assertThat(frameWriter.getInfo().isSupported(format, /* usage= */ 0L)).isFalse(); + } + + @Test + @Config(sdk = 34) + public void getInfo_hlgFormatOnApi34_returnsTrue() { + Format format = + new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).setColorInfo(HLG_COLOR_INFO).build(); + assertThat(frameWriter.getInfo().isSupported(format, /* usage= */ 0L)).isTrue(); + } + + @Test + public void getInfo_sdrFormat_returnsTrue() { + assertThat(frameWriter.getInfo().isSupported(DEFAULT_FORMAT, /* usage= */ 0L)).isTrue(); + } + + @Test + public void signalEndOfStream_notifiesListener() { + frameWriter.signalEndOfStream(); + + assertThat(onEnded.get()).isTrue(); + } + + @Test + @Config(maxSdk = 27) + public void getInfo_nonYv12PixelFormatBelowApi28_returnsFalse() { Format format = new Format.Builder() .setWidth(WIDTH) .setHeight(HEIGHT) - .setColorInfo( - new ColorInfo.Builder() - .setColorSpace(C.COLOR_SPACE_BT2020) - .setColorRange(C.COLOR_RANGE_LIMITED) - .setColorTransfer(C.COLOR_TRANSFER_ST2084) - .build()) + .setPixelFormat(HardwareBuffer.RGB_565) + .setColorInfo(ColorInfo.SDR_BT709_LIMITED) .build(); - assertThat(frameWriter.getInfo().isSupported(format, /* usage= */ 0L)).isTrue(); + + assertThat(frameWriter.getInfo().isSupported(format, /* usage= */ 0L)).isFalse(); } @Test - public void getInfo_sdrFormat_returnsTrue() { + @Config(maxSdk = 27) + public void getInfo_yv12PixelFormatBelowApi28_returnsTrue() { Format format = new Format.Builder() .setWidth(WIDTH) .setHeight(HEIGHT) + .setPixelFormat(ImageFormat.YV12) .setColorInfo(ColorInfo.SDR_BT709_LIMITED) .build(); + assertThat(frameWriter.getInfo().isSupported(format, /* usage= */ 0L)).isTrue(); } @Test - public void signalEndOfStream_notifiesListener() { - frameWriter.signalEndOfStream(); + @Config(maxSdk = 27) + public void create_withNativeHelpersBelowApi28_doesNotThrow() { + try (SurfaceHolderFrameWriter writer = + SurfaceHolderFrameWriter.create( + surfaceHolder, executor, listener, executor, hardwareBufferNativeHelpers)) { + assertThat(writer).isNotNull(); + } + } - assertThat(onEnded.get()).isTrue(); + @Test + @Config(maxSdk = 32) + public void create_withoutNativeHelpersBelowApi33_throwsIllegalStateException() { + assertThrows( + IllegalStateException.class, + () -> SurfaceHolderFrameWriter.create(surfaceHolder, executor, listener, executor)); + } + + @Test + @Config(minSdk = 33) + public void create_withoutNativeHelpersOnApi33AndAbove_doesNotThrow() { + try (SurfaceHolderFrameWriter writer = + SurfaceHolderFrameWriter.create(surfaceHolder, executor, listener, executor)) { + assertThat(writer).isNotNull(); + } } } diff --git a/libraries/container/src/main/java/androidx/media3/container/DolbyVisionConfig.java b/libraries/container/src/main/java/androidx/media3/container/DolbyVisionConfig.java index 7af4f691d8d..3ccf8c79f5a 100644 --- a/libraries/container/src/main/java/androidx/media3/container/DolbyVisionConfig.java +++ b/libraries/container/src/main/java/androidx/media3/container/DolbyVisionConfig.java @@ -28,31 +28,47 @@ public final class DolbyVisionConfig { * * @param data A {@link ParsableByteArray}, whose position is set to the start of the Dolby Vision * configuration data to parse. - * @return The {@link DolbyVisionConfig} corresponding to the configuration, or {@code null} if - * the configuration isn't supported. + * @return The {@link DolbyVisionConfig} corresponding to the configuration. */ - @Nullable public static DolbyVisionConfig parse(ParsableByteArray data) { data.skipBytes(2); // dv_version_major, dv_version_minor int profileData = data.readUnsignedByte(); int dvProfile = (profileData >> 1); int dvLevel = ((profileData & 0x1) << 5) | ((data.readUnsignedByte() >> 3) & 0x1F); - String codecsPrefix; - if (dvProfile == 4 || dvProfile == 5 || dvProfile == 7 || dvProfile == 8) { - codecsPrefix = "dvhe"; - } else if (dvProfile == 9) { - codecsPrefix = "dvav"; - } else if (dvProfile == 10) { - codecsPrefix = "dav1"; - } else { - return null; + @Nullable String codecsPrefix; + switch (dvProfile) { + case 4: + case 5: + case 7: + case 8: + codecsPrefix = "dvhe"; + break; + case 9: + codecsPrefix = "dvav"; + break; + case 10: + codecsPrefix = "dav1"; + break; + case 20: + codecsPrefix = "dvh1"; + break; + default: + // Unrecognized profile. Leave codecs unset so that the track is still reported as Dolby + // Vision, but with no codecs string: track/decoder selection then correctly treats it as + // unsupported instead of silently falling back to the base codec's MIME type. + // See MediaCodecInfo.isCodecProfileAndLevelSupported. + codecsPrefix = null; + break; } + @Nullable String codecs = - codecsPrefix - + (dvProfile < 10 ? ".0" : ".") - + dvProfile - + (dvLevel < 10 ? ".0" : ".") - + dvLevel; + codecsPrefix == null + ? null + : codecsPrefix + + (dvProfile < 10 ? ".0" : ".") + + dvProfile + + (dvLevel < 10 ? ".0" : ".") + + dvLevel; return new DolbyVisionConfig(dvProfile, dvLevel, codecs); } @@ -62,10 +78,13 @@ public static DolbyVisionConfig parse(ParsableByteArray data) { /** The level number. */ public final int level; - /** The RFC 6381 codecs string. */ - public final String codecs; + /** + * The RFC 6381 codecs string, or {@code null} if the profile isn't recognized and so a codecs + * string can't be constructed. + */ + @Nullable public final String codecs; - private DolbyVisionConfig(int profile, int level, String codecs) { + private DolbyVisionConfig(int profile, int level, @Nullable String codecs) { this.profile = profile; this.level = level; this.codecs = codecs; diff --git a/libraries/container/src/main/java/androidx/media3/container/Mp4Box.java b/libraries/container/src/main/java/androidx/media3/container/Mp4Box.java index a114fa7d6f3..9c6adab7bbd 100644 --- a/libraries/container/src/main/java/androidx/media3/container/Mp4Box.java +++ b/libraries/container/src/main/java/androidx/media3/container/Mp4Box.java @@ -407,6 +407,15 @@ public abstract class Mp4Box { @SuppressWarnings("ConstantCaseForConstants") public static final int TYPE_meta = 0x6d657461; + @SuppressWarnings({"ConstantCaseForConstants", "IdentifierName"}) + public static final int TYPE_iloc = 0x696c6f63; + + @SuppressWarnings({"ConstantCaseForConstants", "IdentifierName"}) + public static final int TYPE_iinf = 0x69696e66; + + @SuppressWarnings({"ConstantCaseForConstants", "IdentifierName"}) + public static final int TYPE_infe = 0x696e6665; + @SuppressWarnings("ConstantCaseForConstants") public static final int TYPE_xyz = 0xa978797a; diff --git a/libraries/container/src/main/java/androidx/media3/container/Mp4LocationData.java b/libraries/container/src/main/java/androidx/media3/container/Mp4LocationData.java index 1dede758c95..ed5485c43a0 100644 --- a/libraries/container/src/main/java/androidx/media3/container/Mp4LocationData.java +++ b/libraries/container/src/main/java/androidx/media3/container/Mp4LocationData.java @@ -21,7 +21,6 @@ import androidx.annotation.Nullable; import androidx.media3.common.Metadata; import androidx.media3.common.util.UnstableApi; -import com.google.common.primitives.Floats; /** * Stores MP4 location data. @@ -32,14 +31,22 @@ @UnstableApi public final class Mp4LocationData implements Metadata.Entry { + /** Represents an unset or unknown altitude. */ + public static final float ALTITUDE_UNSET = -Float.MAX_VALUE; + /** The latitude, in degrees. */ public final float latitude; /** The longitude, in degrees. */ public final float longitude; + /** The altitude, in meters, or {@link #ALTITUDE_UNSET} if unset. */ + public final float altitude; + /** - * Creates an instance. + * Creates an instance with latitude and longitude only. + * + *

The {@link #altitude} is set to {@link #ALTITUDE_UNSET}. * * @param latitude The latitude, in degrees. Its value must be in the range [-90, 90]. * @param longitude The longitude, in degrees. Its value must be in the range [-180, 180]. @@ -47,11 +54,26 @@ public final class Mp4LocationData implements Metadata.Entry { public Mp4LocationData( @FloatRange(from = -90.0, to = 90.0) float latitude, @FloatRange(from = -180.0, to = 180.0) float longitude) { + this(latitude, longitude, ALTITUDE_UNSET); + } + + /** + * Creates an instance with latitude, longitude, and altitude. + * + * @param latitude The latitude, in degrees. Its value must be in the range [-90, 90]. + * @param longitude The longitude, in degrees. Its value must be in the range [-180, 180]. + * @param altitude The altitude, in meters, or {@link #ALTITUDE_UNSET} if unset. + */ + public Mp4LocationData( + @FloatRange(from = -90.0, to = 90.0) float latitude, + @FloatRange(from = -180.0, to = 180.0) float longitude, + float altitude) { checkArgument( latitude >= -90.0f && latitude <= 90.0f && longitude >= -180.0f && longitude <= 180.0f, "Invalid latitude or longitude"); this.latitude = latitude; this.longitude = longitude; + this.altitude = altitude; } @Override @@ -63,19 +85,26 @@ public boolean equals(@Nullable Object obj) { return false; } Mp4LocationData other = (Mp4LocationData) obj; - return latitude == other.latitude && longitude == other.longitude; + return Float.compare(latitude, other.latitude) == 0 + && Float.compare(longitude, other.longitude) == 0 + && Float.compare(altitude, other.altitude) == 0; } @Override public int hashCode() { int result = 17; - result = 31 * result + Floats.hashCode(latitude); - result = 31 * result + Floats.hashCode(longitude); + result = 31 * result + Float.hashCode(latitude); + result = 31 * result + Float.hashCode(longitude); + result = 31 * result + Float.hashCode(altitude); return result; } @Override public String toString() { - return "xyz: latitude=" + latitude + ", longitude=" + longitude; + return "xyz: latitude=" + + latitude + + ", longitude=" + + longitude + + (altitude != ALTITUDE_UNSET ? ", altitude=" + altitude : ""); } } diff --git a/libraries/container/src/main/java/androidx/media3/container/NalUnitUtil.java b/libraries/container/src/main/java/androidx/media3/container/NalUnitUtil.java index e845bde811d..bbd201f8273 100644 --- a/libraries/container/src/main/java/androidx/media3/container/NalUnitUtil.java +++ b/libraries/container/src/main/java/androidx/media3/container/NalUnitUtil.java @@ -27,6 +27,7 @@ import androidx.media3.common.MimeTypes; import androidx.media3.common.util.CodecSpecificDataUtil; import androidx.media3.common.util.Log; +import androidx.media3.common.util.ParsableBitArray; import androidx.media3.common.util.UnstableApi; import com.google.common.collect.ImmutableList; import java.math.RoundingMode; @@ -1907,6 +1908,74 @@ public static H265Sei3dRefDisplayInfoData parseH265Sei3dRefDisplayInfo( return null; } + /** + * Parses the unescaped RBSP data of an H.264 or H.265 SEI NAL unit (excluding the NAL unit start + * code and NAL unit header) to extract the {@link C.StereoMode} from a {@code + * frame_packing_arrangement} SEI message (payload type 45), if present. + * + * @param seiData A buffer containing unescaped SEI RBSP data. + * @param offset The offset of the SEI RBSP payload (after the NAL unit header) in {@code + * seiData}. + * @param limit The limit (exclusive) of the SEI RBSP payload in {@code seiData}. + * @return The parsed {@link C.StereoMode}, or {@link Format#NO_VALUE} if no supported {@code + * frame_packing_arrangement} SEI message is present. + */ + public static @C.StereoMode int parseSeiStereoMode(byte[] seiData, int offset, int limit) { + int position = offset; + // Leave at least 1 byte for rbsp_trailing_bits. + while (position < limit - 1) { + int payloadType = 0; + int nextByte; + do { + if (position >= limit) { + return Format.NO_VALUE; + } + nextByte = seiData[position++] & 0xFF; + payloadType += nextByte; + } while (nextByte == 0xFF); + + int payloadSize = 0; + do { + if (position >= limit) { + return Format.NO_VALUE; + } + nextByte = seiData[position++] & 0xFF; + payloadSize += nextByte; + } while (nextByte == 0xFF); + + if (payloadSize < 0 || payloadSize > limit - position) { + return Format.NO_VALUE; + } + + if (payloadType == 45 && payloadSize > 0) { // frame_packing_arrangement + ParsableBitArray bitArray = new ParsableBitArray(seiData, position + payloadSize); + bitArray.setPosition(position * 8); + int leadingZeros = 0; + while (bitArray.bitsLeft() > 0 && !bitArray.readBit()) { + leadingZeros++; + } + if (leadingZeros <= 31 && bitArray.bitsLeft() >= leadingZeros + 1) { + bitArray.skipBits(leadingZeros); // Rest of frame_packing_arrangement_id + boolean framePackingArrangementCancelFlag = bitArray.readBit(); + if (framePackingArrangementCancelFlag) { + return C.STEREO_MODE_MONO; + } else if (bitArray.bitsLeft() >= 7) { + int framePackingArrangementType = bitArray.readBits(7); + if (framePackingArrangementType == 3) { + return C.STEREO_MODE_LEFT_RIGHT; + } else if (framePackingArrangementType == 4) { + return C.STEREO_MODE_TOP_BOTTOM; + } else if (framePackingArrangementType == 6) { + return C.STEREO_MODE_MONO; + } + } + } + } + position += payloadSize; + } + return Format.NO_VALUE; + } + /** * Finds the first NAL unit in {@code data}. * diff --git a/libraries/container/src/main/java/androidx/media3/container/XmpData.java b/libraries/container/src/main/java/androidx/media3/container/XmpData.java index e3656282b0a..c46130ea9c8 100644 --- a/libraries/container/src/main/java/androidx/media3/container/XmpData.java +++ b/libraries/container/src/main/java/androidx/media3/container/XmpData.java @@ -20,10 +20,15 @@ import androidx.media3.common.util.UnstableApi; import androidx.media3.common.util.Util; import java.util.Arrays; +import java.util.UUID; /** Stores XMP data. */ @UnstableApi public final class XmpData implements Metadata.Entry { + + /** The Adobe XMP UUID. */ + public static final UUID XMP_UUID = new UUID(0xBE7ACFCB97A942E8L, 0x9C71999491E3AFACL); + public final byte[] data; /** Creates an instance. */ diff --git a/libraries/container/src/test/java/androidx/media3/container/DolbyVisionConfigTest.java b/libraries/container/src/test/java/androidx/media3/container/DolbyVisionConfigTest.java new file mode 100644 index 00000000000..a0a78b02d1f --- /dev/null +++ b/libraries/container/src/test/java/androidx/media3/container/DolbyVisionConfigTest.java @@ -0,0 +1,89 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.container; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.media3.common.util.ParsableByteArray; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link DolbyVisionConfig}. */ +@RunWith(AndroidJUnit4.class) +public final class DolbyVisionConfigTest { + + @Test + public void parse_profile8_returnsExpectedConfig() { + ParsableByteArray data = buildDolbyVisionConfigData(/* profile= */ 8, /* level= */ 3); + + DolbyVisionConfig config = DolbyVisionConfig.parse(data); + + assertThat(config.profile).isEqualTo(8); + assertThat(config.level).isEqualTo(3); + assertThat(config.codecs).isEqualTo("dvhe.08.03"); + } + + @Test + public void parse_profile9_returnsExpectedConfig() { + ParsableByteArray data = buildDolbyVisionConfigData(/* profile= */ 9, /* level= */ 2); + + DolbyVisionConfig config = DolbyVisionConfig.parse(data); + + assertThat(config.profile).isEqualTo(9); + assertThat(config.level).isEqualTo(2); + assertThat(config.codecs).isEqualTo("dvav.09.02"); + } + + @Test + public void parse_profile10_returnsExpectedConfig() { + ParsableByteArray data = buildDolbyVisionConfigData(/* profile= */ 10, /* level= */ 9); + + DolbyVisionConfig config = DolbyVisionConfig.parse(data); + + assertThat(config.profile).isEqualTo(10); + assertThat(config.level).isEqualTo(9); + assertThat(config.codecs).isEqualTo("dav1.10.09"); + } + + @Test + public void parse_profile20_returnsExpectedConfig() { + ParsableByteArray data = buildDolbyVisionConfigData(/* profile= */ 20, /* level= */ 10); + + DolbyVisionConfig config = DolbyVisionConfig.parse(data); + + assertThat(config.profile).isEqualTo(20); + assertThat(config.level).isEqualTo(10); + assertThat(config.codecs).isEqualTo("dvh1.20.10"); + } + + @Test + public void parse_unrecognizedProfile_returnsConfigWithNullCodecs() { + ParsableByteArray data = buildDolbyVisionConfigData(/* profile= */ 12, /* level= */ 1); + + DolbyVisionConfig config = DolbyVisionConfig.parse(data); + + assertThat(config.profile).isEqualTo(12); + assertThat(config.level).isEqualTo(1); + assertThat(config.codecs).isNull(); + } + + private static ParsableByteArray buildDolbyVisionConfigData(int profile, int level) { + byte byte2 = (byte) ((profile << 1) | ((level >> 5) & 0x1)); + byte byte3 = (byte) ((level & 0x1F) << 3); + return new ParsableByteArray(new byte[] {1, 0, byte2, byte3}); + } +} diff --git a/libraries/container/src/test/java/androidx/media3/container/NalUnitUtilTest.java b/libraries/container/src/test/java/androidx/media3/container/NalUnitUtilTest.java index 289b15e020e..f38efabc9f5 100644 --- a/libraries/container/src/test/java/androidx/media3/container/NalUnitUtilTest.java +++ b/libraries/container/src/test/java/androidx/media3/container/NalUnitUtilTest.java @@ -23,6 +23,7 @@ import static androidx.media3.test.utils.TestUtil.createByteArray; import static com.google.common.truth.Truth.assertThat; +import androidx.media3.common.C; import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; import androidx.media3.common.util.Util; @@ -574,6 +575,86 @@ public void isNalUnitSei_vvc() { assertThat(isNalUnitSei(vvcFormat, suffixSei, /* offset= */ 0)).isFalse(); } + @Test + public void parseSeiStereoMode_sideBySide_returnsStereoModeLeftRight() { + // payloadType = 45 (0x2D), payloadSize = 3 (0x03) + // payload: id=0 (1), cancel_flag=0 (0), type=3 (0000011) -> 10000011 10000000 00000000 (0x81, + // 0x80, 0x00) + // rbsp_trailing_bits = 0x80 + byte[] seiData = createByteArray(0x2D, 0x03, 0x81, 0x80, 0x00, 0x80); + + @C.StereoMode + int stereoMode = NalUnitUtil.parseSeiStereoMode(seiData, /* offset= */ 0, seiData.length); + + assertThat(stereoMode).isEqualTo(C.STEREO_MODE_LEFT_RIGHT); + } + + @Test + public void parseSeiStereoMode_topBottom_returnsStereoModeTopBottom() { + // payloadType = 45 (0x2D), payloadSize = 3 (0x03) + // payload: id=0 (1), cancel_flag=0 (0), type=4 (0000100) -> 10000100 00000000 00000000 (0x82, + // 0x00, 0x00) + // rbsp_trailing_bits = 0x80 + byte[] seiData = createByteArray(0x2D, 0x03, 0x82, 0x00, 0x00, 0x80); + + @C.StereoMode + int stereoMode = NalUnitUtil.parseSeiStereoMode(seiData, /* offset= */ 0, seiData.length); + + assertThat(stereoMode).isEqualTo(C.STEREO_MODE_TOP_BOTTOM); + } + + @Test + public void parseSeiStereoMode_cancelFlagTrue_returnsStereoModeMono() { + // payloadType = 45 (0x2D), payloadSize = 1 (0x01) + // payload: id=0 (1), cancel_flag=1 (1) -> 11000000 (0xC0) + // rbsp_trailing_bits = 0x80 + byte[] seiData = createByteArray(0x2D, 0x01, 0xC0, 0x80); + + @C.StereoMode + int stereoMode = NalUnitUtil.parseSeiStereoMode(seiData, /* offset= */ 0, seiData.length); + + assertThat(stereoMode).isEqualTo(C.STEREO_MODE_MONO); + } + + @Test + public void parseSeiStereoMode_type2d_returnsStereoModeMono() { + // payloadType = 45 (0x2D), payloadSize = 3 (0x03) + // payload: id=0 (1), cancel_flag=0 (0), type=6 (0000110) -> 10000011 00000000 00000000 (0x83, + // 0x00, 0x00) + // rbsp_trailing_bits = 0x80 + byte[] seiData = createByteArray(0x2D, 0x03, 0x83, 0x00, 0x00, 0x80); + + @C.StereoMode + int stereoMode = NalUnitUtil.parseSeiStereoMode(seiData, /* offset= */ 0, seiData.length); + + assertThat(stereoMode).isEqualTo(C.STEREO_MODE_MONO); + } + + @Test + public void parseSeiStereoMode_afterOtherSeiPayload_returnsStereoModeLeftRight() { + // Message 1: payloadType = 1 (pic_timing), payloadSize = 2, payload = {0xAA, 0xBB} + // Message 2: payloadType = 45 (FPA), payloadSize = 3, payload = {0x81, 0x80, 0x00} (type=3) + // rbsp_trailing_bits = 0x80 + byte[] seiData = createByteArray(0x01, 0x02, 0xAA, 0xBB, 0x2D, 0x03, 0x81, 0x80, 0x00, 0x80); + + @C.StereoMode + int stereoMode = NalUnitUtil.parseSeiStereoMode(seiData, /* offset= */ 0, seiData.length); + + assertThat(stereoMode).isEqualTo(C.STEREO_MODE_LEFT_RIGHT); + } + + @Test + public void parseSeiStereoMode_withUnescapedZeroZeroThreeBytes_doesNotDoubleUnescape() { + // payloadType = 45 (0x2D), payloadSize = 7 (0x07), payload starts with 0x00, 0x00, 0x03 + // (id = 6291455), cancel_flag = 0, type = 3 (side-by-side). + byte[] seiData = createByteArray(0x2D, 0x07, 0x00, 0x00, 0x03, 0x00, 0x00, 0x00, 0x18, 0x80); + + @C.StereoMode + int stereoMode = NalUnitUtil.parseSeiStereoMode(seiData, /* offset= */ 0, seiData.length); + + assertThat(stereoMode).isEqualTo(C.STEREO_MODE_LEFT_RIGHT); + } + private static byte[] buildTestData() { byte[] data = new byte[20]; Arrays.fill(data, (byte) 0xFF); diff --git a/libraries/datasource/src/androidTest/java/androidx/media3/datasource/DataSourceBitmapLoaderTest.java b/libraries/datasource/src/androidTest/java/androidx/media3/datasource/DataSourceBitmapLoaderTest.java index 2090f7076cc..0a4e169057c 100644 --- a/libraries/datasource/src/androidTest/java/androidx/media3/datasource/DataSourceBitmapLoaderTest.java +++ b/libraries/datasource/src/androidTest/java/androidx/media3/datasource/DataSourceBitmapLoaderTest.java @@ -25,12 +25,12 @@ import android.graphics.BitmapFactory; import android.graphics.Matrix; import android.net.Uri; +import androidx.annotation.Nullable; import androidx.media3.common.MediaMetadata; import androidx.media3.common.ParserException; import androidx.media3.test.utils.TestUtil; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; -import androidx.test.filters.SdkSuppress; import com.google.common.io.Files; import com.google.common.util.concurrent.ListenableFuture; import java.io.File; @@ -38,6 +38,7 @@ import okhttp3.mockwebserver.MockResponse; import okhttp3.mockwebserver.MockWebServer; import okio.Buffer; +import org.junit.After; import org.junit.Before; import org.junit.Rule; import org.junit.Test; @@ -51,9 +52,6 @@ *

This test needs to run as an androidTest because robolectric's BitmapFactory is not fully * functional. */ -// The image data fails to decode on API 23 (b/429101350) so the only tests which pass are the -// "not found" or other failure ones, which don't seem worth running on their own. -@SdkSuppress(minSdkVersion = 24) @RunWith(AndroidJUnit4.class) public class DataSourceBitmapLoaderTest { @@ -65,6 +63,7 @@ public class DataSourceBitmapLoaderTest { private DataSource.Factory dataSourceFactory; private Context context; + @Nullable private MockWebServer mockWebServer; @Before public void setUp() { @@ -72,6 +71,13 @@ public void setUp() { dataSourceFactory = new DefaultDataSource.Factory(context); } + @After + public void tearDown() throws Exception { + if (mockWebServer != null) { + mockWebServer.shutdown(); + } + } + @Test public void decodeBitmap_withValidData_loadsCorrectData() throws Exception { DataSourceBitmapLoader bitmapLoader = @@ -140,12 +146,11 @@ public void loadBitmap_withHttpUri_loadsCorrectData() throws Exception { .build(); byte[] imageData = TestUtil.getByteArray(context, TEST_IMAGE_PATH); Buffer responseBody = new Buffer().write(imageData); - Bitmap bitmap; - try (MockWebServer mockWebServer = new MockWebServer()) { - mockWebServer.enqueue(new MockResponse().setResponseCode(200).setBody(responseBody)); + mockWebServer = new MockWebServer(); + mockWebServer.enqueue(new MockResponse().setResponseCode(200).setBody(responseBody)); - bitmap = bitmapLoader.loadBitmap(Uri.parse(mockWebServer.url("test_path").toString())).get(); - } + Bitmap bitmap = + bitmapLoader.loadBitmap(Uri.parse(mockWebServer.url("test_path").toString())).get(); assertThat( bitmap.sameAs( @@ -160,12 +165,11 @@ public void loadBitmap_httpUriAndServerError_throws() throws Exception { .setExecutorService(newDirectExecutorService()) .setDataSourceFactory(dataSourceFactory) .build(); - ListenableFuture future; - try (MockWebServer mockWebServer = new MockWebServer()) { - mockWebServer.enqueue(new MockResponse().setResponseCode(404)); + mockWebServer = new MockWebServer(); + mockWebServer.enqueue(new MockResponse().setResponseCode(404)); - future = bitmapLoader.loadBitmap(Uri.parse(mockWebServer.url("test_path").toString())); - } + ListenableFuture future = + bitmapLoader.loadBitmap(Uri.parse(mockWebServer.url("test_path").toString())); assertException( future::get, HttpDataSource.InvalidResponseCodeException.class, /* messagePart= */ "404"); @@ -306,22 +310,21 @@ public void loadBitmapFromMetadata_withArtworkDataAndArtworkUriSet_decodeFromArt .setExecutorService(newDirectExecutorService()) .setDataSourceFactory(dataSourceFactory) .build(); - try (MockWebServer mockWebServer = new MockWebServer()) { - Uri uri = Uri.parse(mockWebServer.url("test_path").toString()); - MediaMetadata metadata = - new MediaMetadata.Builder() - .setArtworkData(imageData, MediaMetadata.PICTURE_TYPE_FRONT_COVER) - .setArtworkUri(uri) - .build(); - - Bitmap bitmap = bitmapLoader.loadBitmapFromMetadata(metadata).get(); - - assertThat( - bitmap.sameAs( - BitmapFactory.decodeByteArray(imageData, /* offset= */ 0, imageData.length))) - .isTrue(); - assertThat(mockWebServer.getRequestCount()).isEqualTo(0); - } + mockWebServer = new MockWebServer(); + Uri uri = Uri.parse(mockWebServer.url("test_path").toString()); + MediaMetadata metadata = + new MediaMetadata.Builder() + .setArtworkData(imageData, MediaMetadata.PICTURE_TYPE_FRONT_COVER) + .setArtworkUri(uri) + .build(); + + Bitmap bitmap = bitmapLoader.loadBitmapFromMetadata(metadata).get(); + + assertThat( + bitmap.sameAs( + BitmapFactory.decodeByteArray(imageData, /* offset= */ 0, imageData.length))) + .isTrue(); + assertThat(mockWebServer.getRequestCount()).isEqualTo(0); } @Test @@ -333,19 +336,18 @@ public void loadBitmapFromMetadata_withArtworkUriSet_loadFromArtworkUri() throws .setExecutorService(newDirectExecutorService()) .setDataSourceFactory(dataSourceFactory) .build(); - try (MockWebServer mockWebServer = new MockWebServer()) { - mockWebServer.enqueue(new MockResponse().setResponseCode(200).setBody(responseBody)); - Uri uri = Uri.parse(mockWebServer.url("test_path").toString()); - MediaMetadata metadata = new MediaMetadata.Builder().setArtworkUri(uri).build(); - - Bitmap bitmap = bitmapLoader.loadBitmapFromMetadata(metadata).get(); - - assertThat( - bitmap.sameAs( - BitmapFactory.decodeByteArray(imageData, /* offset= */ 0, imageData.length))) - .isTrue(); - assertThat(mockWebServer.getRequestCount()).isEqualTo(1); - } + mockWebServer = new MockWebServer(); + mockWebServer.enqueue(new MockResponse().setResponseCode(200).setBody(responseBody)); + Uri uri = Uri.parse(mockWebServer.url("test_path").toString()); + MediaMetadata metadata = new MediaMetadata.Builder().setArtworkUri(uri).build(); + + Bitmap bitmap = bitmapLoader.loadBitmapFromMetadata(metadata).get(); + + assertThat( + bitmap.sameAs( + BitmapFactory.decodeByteArray(imageData, /* offset= */ 0, imageData.length))) + .isTrue(); + assertThat(mockWebServer.getRequestCount()).isEqualTo(1); } @Test @@ -381,7 +383,7 @@ public void loadBitmap_makeShared_returnsImmutableInstance() throws Exception { new DataSourceBitmapLoader.Builder(context).setMakeShared(true).build(); byte[] imageData = TestUtil.getByteArray(context, TEST_IMAGE_PATH); Buffer responseBody = new Buffer().write(imageData); - MockWebServer mockWebServer = new MockWebServer(); + mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse().setResponseCode(200).setBody(responseBody)); Uri uri = Uri.parse(mockWebServer.url("test_path").toString()); diff --git a/libraries/datasource/src/androidTest/java/androidx/media3/datasource/DefaultHttpDataSourceTest.java b/libraries/datasource/src/androidTest/java/androidx/media3/datasource/DefaultHttpDataSourceTest.java index ef9fa01f0dc..f6f98821f5a 100644 --- a/libraries/datasource/src/androidTest/java/androidx/media3/datasource/DefaultHttpDataSourceTest.java +++ b/libraries/datasource/src/androidTest/java/androidx/media3/datasource/DefaultHttpDataSourceTest.java @@ -31,6 +31,8 @@ import okhttp3.mockwebserver.MockWebServer; import okhttp3.mockwebserver.RecordedRequest; import okio.Buffer; +import org.junit.After; +import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; @@ -38,6 +40,18 @@ @RunWith(AndroidJUnit4.class) public class DefaultHttpDataSourceTest { + private MockWebServer mockWebServer; + + @Before + public void setUp() { + mockWebServer = new MockWebServer(); + } + + @After + public void tearDown() throws Exception { + mockWebServer.shutdown(); + } + /** * This test will set HTTP default request parameters (1) in the DefaultHttpDataSource, (2) via * DefaultHttpDataSource.setRequestProperty() and (3) in the DataSpec instance according to the @@ -57,7 +71,6 @@ public class DefaultHttpDataSourceTest { */ @Test public void open_withSpecifiedRequestParameters_usesCorrectParameters() throws Exception { - MockWebServer mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse()); String propertyFromFactory = "fromFactory"; @@ -111,7 +124,6 @@ public void open_invalidResponseCode() { .setReadTimeoutMs(1000) .createDataSource(); - MockWebServer mockWebServer = new MockWebServer(); mockWebServer.enqueue( new MockResponse() .setResponseCode(404) @@ -139,7 +151,6 @@ public void open_redirectCrossProtocol_shouldNotAllowCrossProtocol() throws Exce .setAllowCrossProtocolRedirects(false) .createDataSource(); - MockWebServer mockWebServer = new MockWebServer(); String newLocationUrl = mockWebServer.url("/redirect-path").toString(); String httpsUrl = newLocationUrl.replaceFirst("http", "https"); mockWebServer.enqueue( @@ -175,7 +186,6 @@ public void open_redirectCrossProtocol_shouldForceOriginalProtocol() .setCrossProtocolRedirectsForceOriginal(true) .createDataSource(); - MockWebServer mockWebServer = new MockWebServer(); String newLocationUrl = mockWebServer.url("/redirect-path").toString(); String httpsUrl = newLocationUrl.replaceFirst("http", "https"); mockWebServer.enqueue( @@ -217,7 +227,6 @@ public void open_redirectSameProtocolWithRelativeReference_shouldFollowRedirect( .setCrossProtocolRedirectsForceOriginal(true) .createDataSource(); - MockWebServer mockWebServer = new MockWebServer(); String newLocationUrl = "https/redirect-path"; mockWebServer.enqueue( new MockResponse() @@ -258,7 +267,6 @@ public void open_redirectChanges302PostToGet() .setAllowCrossProtocolRedirects(true) .createDataSource(); - MockWebServer mockWebServer = new MockWebServer(); String newLocationUrl = mockWebServer.url("/redirect-path").toString(); mockWebServer.enqueue( new MockResponse() @@ -297,7 +305,6 @@ public void open_redirectKeeps302Post() throws HttpDataSourceException, Interrup .setKeepPostFor302Redirects(true) .createDataSource(); - MockWebServer mockWebServer = new MockWebServer(); String newLocationUrl = mockWebServer.url("/redirect-path").toString(); mockWebServer.enqueue( new MockResponse() @@ -328,7 +335,6 @@ public void open_redirectKeeps302Post() throws HttpDataSourceException, Interrup @Test public void factory_setRequestPropertyAfterCreation_setsCorrectHeaders() throws Exception { - MockWebServer mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse()); DataSpec dataSpec = new DataSpec.Builder().setUri(mockWebServer.url("/test-path").toString()).build(); diff --git a/libraries/datasource/src/main/java/androidx/media3/datasource/DataSpec.java b/libraries/datasource/src/main/java/androidx/media3/datasource/DataSpec.java index 5d0e52cee49..75bff7e2813 100644 --- a/libraries/datasource/src/main/java/androidx/media3/datasource/DataSpec.java +++ b/libraries/datasource/src/main/java/androidx/media3/datasource/DataSpec.java @@ -51,6 +51,7 @@ public static final class Builder { @Nullable private Uri uri; private long uriPositionOffset; + @Nullable private String location; private @HttpMethod int httpMethod; @Nullable private byte[] httpBody; private Map httpRequestHeaders; @@ -74,6 +75,7 @@ public Builder() { */ private Builder(DataSpec dataSpec) { uri = dataSpec.uri; + location = dataSpec.location; uriPositionOffset = dataSpec.uriPositionOffset; httpMethod = dataSpec.httpMethod; httpBody = dataSpec.httpBody; @@ -121,6 +123,18 @@ public Builder setUriPositionOffset(long uriPositionOffset) { return this; } + /** + * Sets the {@link DataSpec#location}. The default value is {@code null}. + * + * @param location The {@link DataSpec#location}. + * @return The builder. + */ + @CanIgnoreReturnValue + public Builder setLocation(@Nullable String location) { + this.location = location; + return this; + } + /** * Sets {@link DataSpec#httpMethod}. The default value is {@link #HTTP_METHOD_GET}. * @@ -232,6 +246,7 @@ public DataSpec build() { return new DataSpec( uri, uriPositionOffset, + location, httpMethod, httpBody, httpRequestHeaders, @@ -346,6 +361,13 @@ public static String getStringForHttpMethod(@HttpMethod int httpMethod) { */ public final long uriPositionOffset; + /** + * A logical identifier of the location (e.g., a service location, pathway, CDN identifier etc.) + * from which the resource is loaded, or {@code null} if the location is not known or not + * applicable. + */ + @Nullable public final String location; + /** * The HTTP method to use when requesting the data. This value will be ignored by non-HTTP {@link * DataSource} implementations. @@ -446,6 +468,7 @@ public DataSpec(Uri uri, long position, long length, @Nullable String key) { this( uri, /* uriPositionOffset= */ 0, + /* location= */ null, HTTP_METHOD_GET, null, Collections.emptyMap(), @@ -459,6 +482,7 @@ public DataSpec(Uri uri, long position, long length, @Nullable String key) { private DataSpec( Uri uri, long uriPositionOffset, + @Nullable String location, @HttpMethod int httpMethod, @Nullable byte[] httpBody, Map httpRequestHeaders, @@ -473,6 +497,7 @@ private DataSpec( checkArgument(position >= 0); checkArgument(length > 0 || length == C.LENGTH_UNSET); this.uri = checkNotNull(uri); + this.location = location; this.uriPositionOffset = uriPositionOffset; this.httpMethod = httpMethod; this.httpBody = httpBody != null && httpBody.length != 0 ? httpBody : null; @@ -498,7 +523,7 @@ public boolean isFlagSet(@Flags int flag) { * Returns the uppercase HTTP method name (e.g., "GET", "POST", "HEAD") corresponding to the * {@link #httpMethod}. */ - public final String getHttpMethodString() { + public String getHttpMethodString() { return getStringForHttpMethod(httpMethod); } @@ -532,6 +557,7 @@ public DataSpec subrange(long offset, long length) { return new DataSpec( uri, uriPositionOffset, + location, httpMethod, httpBody, httpRequestHeaders, @@ -553,6 +579,7 @@ public DataSpec withUri(Uri uri) { return new DataSpec( uri, uriPositionOffset, + location, httpMethod, httpBody, httpRequestHeaders, @@ -574,6 +601,7 @@ public DataSpec withRequestHeaders(Map httpRequestHeaders) { return new DataSpec( uri, uriPositionOffset, + location, httpMethod, httpBody, httpRequestHeaders, @@ -598,6 +626,7 @@ public DataSpec withAdditionalHeaders(Map additionalHttpRequestH return new DataSpec( uri, uriPositionOffset, + location, httpMethod, httpBody, httpRequestHeaders, @@ -622,6 +651,7 @@ public String toString() { + key + ", " + flags + + (location != null ? ", location=" + location : "") + "]"; } } diff --git a/libraries/datasource/src/main/java/androidx/media3/datasource/DefaultHttpDataSource.java b/libraries/datasource/src/main/java/androidx/media3/datasource/DefaultHttpDataSource.java index 7f84d271707..26a3d66c9e5 100644 --- a/libraries/datasource/src/main/java/androidx/media3/datasource/DefaultHttpDataSource.java +++ b/libraries/datasource/src/main/java/androidx/media3/datasource/DefaultHttpDataSource.java @@ -411,10 +411,10 @@ public long open(DataSpec dataSpec) throws HttpDataSourceException { throw new InvalidContentTypeException(contentType, dataSpec); } - // If we requested a range starting from a non-zero position and received a 200 rather than a - // 206, then the server does not support partial requests. We'll need to manually skip to the - // requested position. - long bytesToSkip = responseCode == 200 && dataSpec.position != 0 ? dataSpec.position : 0; + // If we requested a range starting from a non-zero position and did not receive a 206, then + // the server does not support partial requests. We'll need to manually skip to the requested + // position. + long bytesToSkip = responseCode != 206 && dataSpec.position != 0 ? dataSpec.position : 0; // Determine the length of the data to be read, after skipping. boolean isCompressed = isCompressed(connection); diff --git a/libraries/datasource/src/main/java/androidx/media3/datasource/HttpEngineDataSource.java b/libraries/datasource/src/main/java/androidx/media3/datasource/HttpEngineDataSource.java index 07b96e9a101..47f763c32e8 100644 --- a/libraries/datasource/src/main/java/androidx/media3/datasource/HttpEngineDataSource.java +++ b/libraries/datasource/src/main/java/androidx/media3/datasource/HttpEngineDataSource.java @@ -540,10 +540,10 @@ public long open(DataSpec dataSpec) throws HttpDataSourceException { } } - // If we requested a range starting from a non-zero position and received a 200 rather than a - // 206, then the server does not support partial requests. We'll need to manually skip to the - // requested position. - long bytesToSkip = responseCode == 200 && dataSpec.position != 0 ? dataSpec.position : 0; + // If we requested a range starting from a non-zero position and did not receive a 206, then + // the server does not support partial requests. We'll need to manually skip to the requested + // position. + long bytesToSkip = responseCode != 206 && dataSpec.position != 0 ? dataSpec.position : 0; // Calculate the content length. if (!isCompressed(responseInfo)) { diff --git a/libraries/datasource_cronet/src/main/java/androidx/media3/datasource/cronet/CronetDataSource.java b/libraries/datasource_cronet/src/main/java/androidx/media3/datasource/cronet/CronetDataSource.java index 53f3bd0aacf..8e42bf053b4 100644 --- a/libraries/datasource_cronet/src/main/java/androidx/media3/datasource/cronet/CronetDataSource.java +++ b/libraries/datasource_cronet/src/main/java/androidx/media3/datasource/cronet/CronetDataSource.java @@ -670,10 +670,10 @@ public long open(DataSpec dataSpec) throws HttpDataSourceException { } } - // If we requested a range starting from a non-zero position and received a 200 rather than a - // 206, then the server does not support partial requests. We'll need to manually skip to the - // requested position. - long bytesToSkip = responseCode == 200 && dataSpec.position != 0 ? dataSpec.position : 0; + // If we requested a range starting from a non-zero position and did not receive a 206, then + // the server does not support partial requests. We'll need to manually skip to the requested + // position. + long bytesToSkip = responseCode != 206 && dataSpec.position != 0 ? dataSpec.position : 0; // Calculate the content length. if (!isCompressed(responseInfo)) { diff --git a/libraries/datasource_cronet/src/test/java/androidx/media3/datasource/cronet/CronetDataSourceTest.java b/libraries/datasource_cronet/src/test/java/androidx/media3/datasource/cronet/CronetDataSourceTest.java index ad7cc55b4c4..dc6214c47fa 100644 --- a/libraries/datasource_cronet/src/test/java/androidx/media3/datasource/cronet/CronetDataSourceTest.java +++ b/libraries/datasource_cronet/src/test/java/androidx/media3/datasource/cronet/CronetDataSourceTest.java @@ -35,6 +35,7 @@ import android.net.Uri; import android.os.ConditionVariable; import android.os.SystemClock; +import androidx.annotation.Nullable; import androidx.media3.common.C; import androidx.media3.common.util.Util; import androidx.media3.datasource.DataSpec; @@ -110,6 +111,7 @@ public final class CronetDataSourceTest { private ExecutorService executorService; private CronetDataSource dataSourceUnderTest; private boolean redirectCalled; + @Nullable private MockWebServer mockWebServer; @Before public void setUp() { @@ -141,8 +143,11 @@ public void setUp() { } @After - public void tearDown() { + public void tearDown() throws Exception { executorService.shutdown(); + if (mockWebServer != null) { + mockWebServer.shutdown(); + } } private UrlResponseInfo createUrlResponseInfo(int statusCode) { @@ -1474,7 +1479,7 @@ public void allowDirectExecutor() throws HttpDataSourceException { @Test public void factorySetFallbackHttpDataSourceFactory_cronetNotAvailable_usesFallbackFactory() throws HttpDataSourceException, InterruptedException { - MockWebServer mockWebServer = new MockWebServer(); + mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse()); CronetEngineWrapper cronetEngineWrapper = new CronetEngineWrapper((CronetEngine) null); DefaultHttpDataSource.Factory fallbackFactory = @@ -1496,7 +1501,7 @@ public void factorySetFallbackHttpDataSourceFactory_cronetNotAvailable_usesFallb public void factory_noFallbackFactoryCronetNotAvailable_delegateTransferListenerToInternalFallbackFactory() throws HttpDataSourceException { - MockWebServer mockWebServer = new MockWebServer(); + mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse()); CronetEngineWrapper cronetEngineWrapper = new CronetEngineWrapper((CronetEngine) null); HttpDataSource dataSourceUnderTest = @@ -1519,7 +1524,7 @@ public void factorySetFallbackHttpDataSourceFactory_cronetNotAvailable_usesFallb public void factory_noFallbackFactoryCronetNotAvailable_delegateDefaultRequestPropertiesToInternalFallbackFactory() throws HttpDataSourceException, InterruptedException { - MockWebServer mockWebServer = new MockWebServer(); + mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse()); CronetEngineWrapper cronetEngineWrapper = new CronetEngineWrapper((CronetEngine) null); Map defaultRequestProperties = new HashMap<>(); @@ -1542,7 +1547,7 @@ public void factorySetFallbackHttpDataSourceFactory_cronetNotAvailable_usesFallb public void factory_noFallbackFactoryCronetNotAvailable_delegateDefaultRequestPropertiesToInternalFallbackFactoryAfterCreation() throws HttpDataSourceException, InterruptedException { - MockWebServer mockWebServer = new MockWebServer(); + mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse()); CronetEngineWrapper cronetEngineWrapper = new CronetEngineWrapper((CronetEngine) null); Map defaultRequestProperties = new HashMap<>(); diff --git a/libraries/datasource_ktor/src/androidTest/java/androidx/media3/datasource/ktor/KtorDataSourceTest.kt b/libraries/datasource_ktor/src/androidTest/java/androidx/media3/datasource/ktor/KtorDataSourceTest.kt index 2908e4ac61b..bc604ff5c9a 100644 --- a/libraries/datasource_ktor/src/androidTest/java/androidx/media3/datasource/ktor/KtorDataSourceTest.kt +++ b/libraries/datasource_ktor/src/androidTest/java/androidx/media3/datasource/ktor/KtorDataSourceTest.kt @@ -15,15 +15,16 @@ */ package androidx.media3.datasource.ktor +import android.os.SystemClock import androidx.media3.datasource.DataSource import androidx.media3.datasource.DataSpec import androidx.media3.datasource.HttpDataSource import androidx.media3.datasource.TransferListener import com.google.common.net.HttpHeaders import com.google.common.truth.Truth.assertThat -import com.google.testing.junit.testparameterinjector.KotlinTestParameters.namedTestValues import com.google.testing.junit.testparameterinjector.TestParameter import com.google.testing.junit.testparameterinjector.TestParameterInjector +import com.google.testing.junit.testparameterinjector.TestParameterValuesProvider import io.ktor.client.HttpClient import io.ktor.client.HttpClientConfig import io.ktor.client.engine.HttpClientEngineFactory @@ -35,22 +36,38 @@ import java.nio.charset.StandardCharsets import java.util.concurrent.TimeUnit import okhttp3.mockwebserver.MockResponse import okhttp3.mockwebserver.MockWebServer +import org.junit.After import org.junit.Assert.assertThrows +import org.junit.Rule import org.junit.Test import org.junit.runner.RunWith @RunWith(TestParameterInjector::class) class KtorDataSourceTest( - @TestParameter - private val httpClientEngineFactory: HttpClientEngineFactory<*> = - namedTestValues("Android" to Android, "OkHttp" to OkHttp) + @TestParameter(valuesProvider = ClientEngineFactoryProvider::class) + private val httpClientEngineFactory: HttpClientEngineFactory<*> ) { + private class ClientEngineFactoryProvider : TestParameterValuesProvider() { + override fun provideValues(context: Context?): List<*>? { + return listOf(value(Android).withName("Android"), value(OkHttp).withName("OkHttp")) + } + } + + @get:Rule val mockWebServer = MockWebServer() + + private lateinit var httpClient: HttpClient + private lateinit var dataSource: KtorDataSource + + @After + fun tearDown() { + if (::dataSource.isInitialized) dataSource.close() + if (::httpClient.isInitialized) httpClient.close() + } + @Test - @Throws(Exception::class) fun open_setsCorrectHeaders() { - val httpClient = HttpClient(httpClientEngineFactory) { configureTimeout() } - val mockWebServer = MockWebServer() + httpClient = HttpClient(httpClientEngineFactory) { configureTimeout() } mockWebServer.enqueue(MockResponse()) val propertyFromFactory = "fromFactory" @@ -60,7 +77,7 @@ class KtorDataSourceTest( defaultRequestProperties["2"] = propertyFromFactory defaultRequestProperties["4"] = propertyFromFactory - val dataSource = + dataSource = KtorDataSource.Factory(httpClient) .setDefaultRequestProperties(defaultRequestProperties) .createDataSource() @@ -86,9 +103,9 @@ class KtorDataSourceTest( assertThat(dataSource.open(dataSpec)).isEqualTo(0) - val request = mockWebServer.takeRequest(10, TimeUnit.SECONDS) - assertThat(request).isNotNull() - val headers = request!!.headers + val headers = + checkNotNull(mockWebServer.takeRequest(10, TimeUnit.SECONDS)) { "No request received." } + .headers assertThat(headers["0"]).isEqualTo(propertyFromFactory) assertThat(headers["1"]).isEqualTo(propertyFromSetter) assertThat(headers["2"]).isEqualTo(propertyFromDataSpec) @@ -100,11 +117,10 @@ class KtorDataSourceTest( @Test fun open_invalidResponseCode() { - val httpClient = HttpClient(httpClientEngineFactory) { configureTimeout() } - val mockWebServer = MockWebServer() + httpClient = HttpClient(httpClientEngineFactory) { configureTimeout() } mockWebServer.enqueue(MockResponse().setResponseCode(404).setBody("failure msg")) - val dataSource = KtorDataSource.Factory(httpClient).createDataSource() + dataSource = KtorDataSource.Factory(httpClient).createDataSource() val dataSpec = DataSpec.Builder().setUri(mockWebServer.url("/test-path").toString()).build() @@ -118,31 +134,29 @@ class KtorDataSourceTest( } @Test - @Throws(Exception::class) fun factory_setRequestPropertyAfterCreation_setsCorrectHeaders() { - val httpClient = HttpClient(httpClientEngineFactory) { configureTimeout() } - val mockWebServer = MockWebServer() + httpClient = HttpClient(httpClientEngineFactory) { configureTimeout() } mockWebServer.enqueue(MockResponse()) val dataSpec = DataSpec.Builder().setUri(mockWebServer.url("/test-path").toString()).build() val factory = KtorDataSource.Factory(httpClient) - val dataSource = factory.createDataSource() + dataSource = factory.createDataSource() val defaultRequestProperties = HashMap() defaultRequestProperties["0"] = "afterCreation" factory.setDefaultRequestProperties(defaultRequestProperties) assertThat(dataSource.open(dataSpec)).isEqualTo(0) - val request = mockWebServer.takeRequest(10, TimeUnit.SECONDS) - assertThat(request).isNotNull() - val headers = request!!.headers + val headers = + checkNotNull(mockWebServer.takeRequest(10, TimeUnit.SECONDS)) { "No request received." } + .headers assertThat(headers["0"]).isEqualTo("afterCreation") } @Test fun open_malformedUrl_throwsException() { - val httpClient = HttpClient(httpClientEngineFactory) { configureTimeout() } - val dataSource = KtorDataSource.Factory(httpClient).createDataSource() + httpClient = HttpClient(httpClientEngineFactory) { configureTimeout() } + dataSource = KtorDataSource.Factory(httpClient).createDataSource() val dataSpec = DataSpec.Builder().setUri("not-a-valid-url").build() @@ -153,13 +167,11 @@ class KtorDataSourceTest( } @Test - @Throws(Exception::class) fun open_httpPost_sendsPostRequest() { - val httpClient = HttpClient(httpClientEngineFactory) { configureTimeout() } - val mockWebServer = MockWebServer() + httpClient = HttpClient(httpClientEngineFactory) { configureTimeout() } mockWebServer.enqueue(MockResponse()) - val dataSource = KtorDataSource.Factory(httpClient).createDataSource() + dataSource = KtorDataSource.Factory(httpClient).createDataSource() val dataSpec = DataSpec.Builder() @@ -170,101 +182,94 @@ class KtorDataSourceTest( assertThat(dataSource.open(dataSpec)).isEqualTo(0) - val request = mockWebServer.takeRequest(10, TimeUnit.SECONDS) - assertThat(request).isNotNull() - assertThat(request!!.method).isEqualTo("POST") + val request = + checkNotNull(mockWebServer.takeRequest(10, TimeUnit.SECONDS)) { "No request received." } + assertThat(request.method).isEqualTo("POST") assertThat(request.body.readUtf8()).isEqualTo("test body") } @Test - @Throws(java.lang.Exception::class) fun cookiesConfigured_cookiesPersistedBetweenRequests() { - val httpClient = + httpClient = HttpClient(httpClientEngineFactory) { configureTimeout() install(HttpCookies) } - MockWebServer().use { mockWebServer -> - mockWebServer.enqueue( - MockResponse().setHeader(HttpHeaders.SET_COOKIE, "cookie-name=cookie-val") - ) - mockWebServer.enqueue(MockResponse()) + mockWebServer.enqueue( + MockResponse().setHeader(HttpHeaders.SET_COOKIE, "cookie-name=cookie-val") + ) + mockWebServer.enqueue(MockResponse()) - val dataSpec = DataSpec.Builder().setUri(mockWebServer.url("foo").toString()).build() - val dataSource: KtorDataSource = KtorDataSource.Factory(httpClient).createDataSource() - assertThat(dataSource.open(dataSpec)).isEqualTo(0) - dataSource.close() - assertThat(dataSource.open(dataSpec)).isEqualTo(0) + val dataSpec = DataSpec.Builder().setUri(mockWebServer.url("foo").toString()).build() + dataSource = KtorDataSource.Factory(httpClient).createDataSource() + assertThat(dataSource.open(dataSpec)).isEqualTo(0) + dataSource.close() + assertThat(dataSource.open(dataSpec)).isEqualTo(0) - val firstRequest = mockWebServer.takeRequest() - assertThat(firstRequest.path).isEqualTo("/foo") - assertThat(firstRequest.getHeader(HttpHeaders.COOKIE)).isNull() + val firstRequest = + checkNotNull(mockWebServer.takeRequest(10, TimeUnit.SECONDS)) { "No request received." } + assertThat(firstRequest.path).isEqualTo("/foo") + assertThat(firstRequest.getHeader(HttpHeaders.COOKIE)).isNull() - val secondRequest = mockWebServer.takeRequest() - assertThat(secondRequest.path).isEqualTo("/foo") - assertThat(secondRequest.getHeader(HttpHeaders.COOKIE)).isEqualTo("cookie-name=cookie-val") - } + val secondRequest = + checkNotNull(mockWebServer.takeRequest(10, TimeUnit.SECONDS)) { "No request received." } + assertThat(secondRequest.path).isEqualTo("/foo") + assertThat(secondRequest.getHeader(HttpHeaders.COOKIE)).isEqualTo("cookie-name=cookie-val") } @Test - @Throws(java.lang.Exception::class) fun cookiesConfigured_cookiesForwardedOnRedirect() { - val httpClient = + httpClient = HttpClient(httpClientEngineFactory) { configureTimeout() install(HttpCookies) } MockWebServer().use { redirectWebServer -> - MockWebServer().use { originWebServer -> - val originUrl = originWebServer.url("bar").toString() - redirectWebServer.enqueue( - MockResponse() - .setResponseCode(302) - .setHeader(HttpHeaders.SET_COOKIE, "cookie-name=cookie-val; Path=/") - .setHeader(HttpHeaders.LOCATION, originUrl) - ) - originWebServer.enqueue(MockResponse()) + val originUrl = mockWebServer.url("bar").toString() + redirectWebServer.enqueue( + MockResponse() + .setResponseCode(302) + .setHeader(HttpHeaders.SET_COOKIE, "cookie-name=cookie-val; Path=/") + .setHeader(HttpHeaders.LOCATION, originUrl) + ) + mockWebServer.enqueue(MockResponse()) - val redirectUrl = redirectWebServer.url("foo").toString() - val dataSpec = DataSpec.Builder().setUri(redirectUrl).build() - val dataSource: KtorDataSource = KtorDataSource.Factory(httpClient).createDataSource() - assertThat(dataSource.open(dataSpec)).isEqualTo(0) + val redirectUrl = redirectWebServer.url("foo").toString() + val dataSpec = DataSpec.Builder().setUri(redirectUrl).build() + dataSource = KtorDataSource.Factory(httpClient).createDataSource() + assertThat(dataSource.open(dataSpec)).isEqualTo(0) - val originRequest = originWebServer.takeRequest() - assertThat(originRequest.path).isEqualTo("/bar") - assertThat(originRequest.getHeader(HttpHeaders.COOKIE)).isEqualTo("cookie-name=cookie-val") - } + val originRequest = + checkNotNull(mockWebServer.takeRequest(10, TimeUnit.SECONDS)) { "No request received." } + assertThat(originRequest.path).isEqualTo("/bar") + assertThat(originRequest.getHeader(HttpHeaders.COOKIE)).isEqualTo("cookie-name=cookie-val") } } @Test - @Throws(Exception::class) fun factory_setUserAgent_setsCorrectHeader() { - val httpClient = HttpClient(httpClientEngineFactory) { configureTimeout() } - val mockWebServer = MockWebServer() + httpClient = HttpClient(httpClientEngineFactory) { configureTimeout() } mockWebServer.enqueue(MockResponse()) val userAgent = "testUserAgent" - val dataSource = KtorDataSource.Factory(httpClient, userAgent = userAgent).createDataSource() + dataSource = KtorDataSource.Factory(httpClient, userAgent = userAgent).createDataSource() val dataSpec = DataSpec.Builder().setUri(mockWebServer.url("/test-path").toString()).build() assertThat(dataSource.open(dataSpec)).isEqualTo(0) - val request = mockWebServer.takeRequest(10, TimeUnit.SECONDS) - assertThat(request).isNotNull() - assertThat(request!!.getHeader("User-Agent")).isEqualTo(userAgent) + val request = + checkNotNull(mockWebServer.takeRequest(10, TimeUnit.SECONDS)) { "No request received." } + assertThat(request.getHeader("User-Agent")).isEqualTo(userAgent) } @Test - @Throws(Exception::class) fun factory_setContentTypePredicate_filtersContentType() { - val httpClient = HttpClient(httpClientEngineFactory) { configureTimeout() } - val mockWebServer = MockWebServer() + httpClient = HttpClient(httpClientEngineFactory) { configureTimeout() } mockWebServer.enqueue( MockResponse().setResponseCode(200).setHeader("Content-Type", "text/html") ) - val dataSource = + dataSource = KtorDataSource.Factory( httpClient, contentTypePredicate = { contentType -> contentType == "audio/mpeg" }, @@ -281,10 +286,8 @@ class KtorDataSourceTest( } @Test - @Throws(Exception::class) fun factory_setTransferListener_setsListener() { - val httpClient = HttpClient(httpClientEngineFactory) { configureTimeout() } - val mockWebServer = MockWebServer() + httpClient = HttpClient(httpClientEngineFactory) { configureTimeout() } mockWebServer.enqueue(MockResponse()) var transferInitializingCalled = false @@ -329,7 +332,7 @@ class KtorDataSourceTest( } } - val dataSource = + dataSource = KtorDataSource.Factory(httpClient, transferListener = transferListener).createDataSource() val dataSpec = DataSpec.Builder().setUri(mockWebServer.url("/test-path").toString()).build() @@ -341,6 +344,43 @@ class KtorDataSourceTest( assertThat(transferEndCalled).isTrue() } + @Test + fun open_doesNotWaitForResponseBody() { + val testData = "a".repeat(100) + val bodyDelayMs = 2000L + mockWebServer.enqueue( + MockResponse() + .setResponseCode(200) + .setHeader("Content-Length", testData.length) + .setBody(testData) + .setBodyDelay(bodyDelayMs, TimeUnit.MILLISECONDS) + ) + + httpClient = + HttpClient(httpClientEngineFactory) { + install(HttpTimeout) { + requestTimeoutMillis = 5000 + connectTimeoutMillis = 5000 + socketTimeoutMillis = 5000 + } + } + dataSource = KtorDataSource.Factory(httpClient).createDataSource() + val dataSpec = DataSpec.Builder().setUri(mockWebServer.url("/test-path").toString()).build() + + val startTimeMs = SystemClock.elapsedRealtime() + assertThat(dataSource.open(dataSpec)).isEqualTo(testData.length.toLong()) + val openDurationMs = SystemClock.elapsedRealtime() - startTimeMs + + // Verify that open() completed without waiting for the delayed response body. + assertThat(openDurationMs).isLessThan(1000L) + + val buffer = ByteArray(10) + val bytesRead = dataSource.read(buffer, 0, buffer.size) + val readDurationMs = SystemClock.elapsedRealtime() - startTimeMs + assertThat(bytesRead).isEqualTo(10) + assertThat(readDurationMs).isGreaterThan(bodyDelayMs) + } + // TODO: b/503301819 - Remove this when the OkHttp engine works without it. private fun HttpClientConfig<*>.configureTimeout() { install(HttpTimeout) { diff --git a/libraries/datasource_ktor/src/main/java/androidx/media3/datasource/ktor/KtorDataSource.kt b/libraries/datasource_ktor/src/main/java/androidx/media3/datasource/ktor/KtorDataSource.kt index 39da36b5484..85af82f0036 100644 --- a/libraries/datasource_ktor/src/main/java/androidx/media3/datasource/ktor/KtorDataSource.kt +++ b/libraries/datasource_ktor/src/main/java/androidx/media3/datasource/ktor/KtorDataSource.kt @@ -40,7 +40,6 @@ import io.ktor.client.statement.HttpResponse import io.ktor.client.statement.bodyAsChannel import io.ktor.client.statement.request import io.ktor.http.HttpMethod -import io.ktor.http.contentLength import io.ktor.http.contentType import io.ktor.utils.io.ByteReadChannel import io.ktor.utils.io.readAvailable @@ -154,7 +153,7 @@ private constructor( setBody(dataSpec.httpBody!!) } } - .execute() + .body() channel = httpResponse.bodyAsChannel() } this.response = httpResponse @@ -227,7 +226,7 @@ private constructor( throw HttpDataSource.InvalidContentTypeException(contentType, dataSpec) } - val bytesToSkip = if (responseCode == 200 && dataSpec.position != 0L) dataSpec.position else 0L + val bytesToSkip = if (responseCode != 206 && dataSpec.position != 0L) dataSpec.position else 0L if (dataSpec.length != C.LENGTH_UNSET.toLong()) { bytesToRead = dataSpec.length diff --git a/libraries/datasource_okhttp/src/androidTest/java/androidx/media3/datasource/okhttp/OkHttpDataSourceTest.java b/libraries/datasource_okhttp/src/androidTest/java/androidx/media3/datasource/okhttp/OkHttpDataSourceTest.java index e1babfceb77..be88efa9ddb 100644 --- a/libraries/datasource_okhttp/src/androidTest/java/androidx/media3/datasource/okhttp/OkHttpDataSourceTest.java +++ b/libraries/datasource_okhttp/src/androidTest/java/androidx/media3/datasource/okhttp/OkHttpDataSourceTest.java @@ -35,6 +35,8 @@ import okhttp3.mockwebserver.MockResponse; import okhttp3.mockwebserver.MockWebServer; import okhttp3.mockwebserver.RecordedRequest; +import org.junit.After; +import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; @@ -42,6 +44,18 @@ @RunWith(AndroidJUnit4.class) public class OkHttpDataSourceTest { + private MockWebServer mockWebServer; + + @Before + public void setUp() { + mockWebServer = new MockWebServer(); + } + + @After + public void tearDown() throws Exception { + mockWebServer.shutdown(); + } + /** * This test will set HTTP default request parameters (1) in the OkHttpDataSource, (2) via * OkHttpDataSource.setRequestProperty() and (3) in the DataSpec instance according to the table @@ -61,7 +75,6 @@ public class OkHttpDataSourceTest { */ @Test public void open_setsCorrectHeaders() throws Exception { - MockWebServer mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse()); String propertyFromFactory = "fromFactory"; @@ -108,7 +121,6 @@ public void open_setsCorrectHeaders() throws Exception { @Test public void open_invalidResponseCode() { - MockWebServer mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse().setResponseCode(404).setBody("failure msg")); HttpDataSource okHttpDataSource = @@ -129,44 +141,40 @@ public void open_invalidResponseCode() { @Test @SdkSuppress(minSdkVersion = 25) // Cookies aren't persisted on API <= 24: b/502906567 public void cookiejarConfigured_cookiesPersistedBetweenRequests() throws Exception { - try (MockWebServer mockWebServer = new MockWebServer()) { - mockWebServer.enqueue( - new MockResponse().setHeader(HttpHeaders.SET_COOKIE, "cookie-name=cookie-val")); - mockWebServer.enqueue(new MockResponse()); + mockWebServer.enqueue( + new MockResponse().setHeader(HttpHeaders.SET_COOKIE, "cookie-name=cookie-val")); + mockWebServer.enqueue(new MockResponse()); - DataSpec dataSpec = - new DataSpec.Builder().setUri(mockWebServer.url("foo").toString()).build(); - OkHttpDataSource dataSource = - new OkHttpDataSource.Factory( - new OkHttpClient.Builder() - .cookieJar(new JavaNetCookieJar(new CookieManager())) - .build()) - .createDataSource(); - dataSource.open(dataSpec); - dataSource.close(); - dataSource.open(dataSpec); + DataSpec dataSpec = new DataSpec.Builder().setUri(mockWebServer.url("foo").toString()).build(); + OkHttpDataSource dataSource = + new OkHttpDataSource.Factory( + new OkHttpClient.Builder() + .cookieJar(new JavaNetCookieJar(new CookieManager())) + .build()) + .createDataSource(); + dataSource.open(dataSpec); + dataSource.close(); + dataSource.open(dataSpec); - RecordedRequest firstRequest = mockWebServer.takeRequest(); - assertThat(firstRequest.getPath()).isEqualTo("/foo"); - assertThat(firstRequest.getHeader(HttpHeaders.COOKIE)).isNull(); + RecordedRequest firstRequest = mockWebServer.takeRequest(); + assertThat(firstRequest.getPath()).isEqualTo("/foo"); + assertThat(firstRequest.getHeader(HttpHeaders.COOKIE)).isNull(); - RecordedRequest secondRequest = mockWebServer.takeRequest(); - assertThat(secondRequest.getPath()).isEqualTo("/foo"); - assertThat(secondRequest.getHeader(HttpHeaders.COOKIE)).isEqualTo("cookie-name=cookie-val"); - } + RecordedRequest secondRequest = mockWebServer.takeRequest(); + assertThat(secondRequest.getPath()).isEqualTo("/foo"); + assertThat(secondRequest.getHeader(HttpHeaders.COOKIE)).isEqualTo("cookie-name=cookie-val"); } @Test @SdkSuppress(minSdkVersion = 25) // Cookies aren't persisted on API <= 24: b/502906567 public void cookiejarConfigured_cookiesForwardedOnRedirect() throws Exception { - try (MockWebServer redirectWebServer = new MockWebServer(); - MockWebServer originWebServer = new MockWebServer()) { + try (MockWebServer redirectWebServer = new MockWebServer()) { redirectWebServer.enqueue( new MockResponse() .setResponseCode(302) .setHeader(HttpHeaders.SET_COOKIE, "cookie-name=cookie-val") - .setHeader(HttpHeaders.LOCATION, originWebServer.url("bar").toString())); - originWebServer.enqueue(new MockResponse()); + .setHeader(HttpHeaders.LOCATION, mockWebServer.url("bar").toString())); + mockWebServer.enqueue(new MockResponse()); DataSpec dataSpec = new DataSpec.Builder().setUri(redirectWebServer.url("foo").toString()).build(); @@ -178,7 +186,7 @@ public void cookiejarConfigured_cookiesForwardedOnRedirect() throws Exception { .createDataSource(); dataSource.open(dataSpec); - RecordedRequest originRequest = originWebServer.takeRequest(); + RecordedRequest originRequest = mockWebServer.takeRequest(); assertThat(originRequest.getPath()).isEqualTo("/bar"); assertThat(originRequest.getHeader(HttpHeaders.COOKIE)).isEqualTo("cookie-name=cookie-val"); } @@ -186,7 +194,6 @@ public void cookiejarConfigured_cookiesForwardedOnRedirect() throws Exception { @Test public void factory_setRequestPropertyAfterCreation_setsCorrectHeaders() throws Exception { - MockWebServer mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse()); DataSpec dataSpec = new DataSpec.Builder().setUri(mockWebServer.url("/test-path").toString()).build(); diff --git a/libraries/datasource_okhttp/src/main/java/androidx/media3/datasource/okhttp/OkHttpDataSource.java b/libraries/datasource_okhttp/src/main/java/androidx/media3/datasource/okhttp/OkHttpDataSource.java index 1c24144c242..6b35144bef8 100644 --- a/libraries/datasource_okhttp/src/main/java/androidx/media3/datasource/okhttp/OkHttpDataSource.java +++ b/libraries/datasource_okhttp/src/main/java/androidx/media3/datasource/okhttp/OkHttpDataSource.java @@ -317,10 +317,10 @@ public long open(DataSpec dataSpec) throws HttpDataSourceException { throw new InvalidContentTypeException(contentType, dataSpec); } - // If we requested a range starting from a non-zero position and received a 200 rather than a - // 206, then the server does not support partial requests. We'll need to manually skip to the - // requested position. - long bytesToSkip = responseCode == 200 && dataSpec.position != 0 ? dataSpec.position : 0; + // If we requested a range starting from a non-zero position and did not receive a 206, then + // the server does not support partial requests. We'll need to manually skip to the requested + // position. + long bytesToSkip = responseCode != 206 && dataSpec.position != 0 ? dataSpec.position : 0; // Determine the length of the data to be read, after skipping. if (dataSpec.length != C.LENGTH_UNSET) { diff --git a/libraries/decoder/src/main/java/androidx/media3/decoder/Buffer.java b/libraries/decoder/src/main/java/androidx/media3/decoder/Buffer.java index b992c5d38f5..08114a1cb6d 100644 --- a/libraries/decoder/src/main/java/androidx/media3/decoder/Buffer.java +++ b/libraries/decoder/src/main/java/androidx/media3/decoder/Buffer.java @@ -18,6 +18,7 @@ import androidx.annotation.CallSuper; import androidx.media3.common.C; import androidx.media3.common.util.UnstableApi; +import org.checkerframework.dataflow.qual.Pure; /** Base class for buffers with flags. */ @UnstableApi @@ -32,6 +33,7 @@ public void clear() { } /** Returns whether the {@link C#BUFFER_FLAG_FIRST_SAMPLE} flag is set. */ + @Pure public final boolean isFirstSample() { return getFlag(C.BUFFER_FLAG_FIRST_SAMPLE); } @@ -41,26 +43,31 @@ public final boolean isFirstSample() { * *

If this is set, all other attributes of the buffer should be ignored. */ + @Pure public final boolean isEndOfStream() { return getFlag(C.BUFFER_FLAG_END_OF_STREAM); } /** Returns whether the {@link C#BUFFER_FLAG_KEY_FRAME} flag is set. */ + @Pure public final boolean isKeyFrame() { return getFlag(C.BUFFER_FLAG_KEY_FRAME); } /** Returns whether the {@link C#BUFFER_FLAG_LAST_SAMPLE} flag is set. */ + @Pure public final boolean isLastSample() { return getFlag(C.BUFFER_FLAG_LAST_SAMPLE); } /** Returns whether the {@link C#BUFFER_FLAG_HAS_SUPPLEMENTAL_DATA} flag is set. */ + @Pure public final boolean hasSupplementalData() { return getFlag(C.BUFFER_FLAG_HAS_SUPPLEMENTAL_DATA); } /** Returns whether the {@link C#BUFFER_FLAG_NOT_DEPENDED_ON} flag is set. */ + @Pure public final boolean notDependedOn() { return getFlag(C.BUFFER_FLAG_NOT_DEPENDED_ON); } @@ -100,6 +107,7 @@ public final void clearFlag(@C.BufferFlags int flag) { * @param flag The flag to check. * @return Whether the flag is set. */ + @Pure protected final boolean getFlag(@C.BufferFlags int flag) { return (flags & flag) == flag; } diff --git a/libraries/decoder/src/main/java/androidx/media3/decoder/CryptoInfo.java b/libraries/decoder/src/main/java/androidx/media3/decoder/CryptoInfo.java index 54b53d8dc35..a8660b15a57 100644 --- a/libraries/decoder/src/main/java/androidx/media3/decoder/CryptoInfo.java +++ b/libraries/decoder/src/main/java/androidx/media3/decoder/CryptoInfo.java @@ -15,11 +15,8 @@ */ package androidx.media3.decoder; -import static android.os.Build.VERSION.SDK_INT; -import static com.google.common.base.Preconditions.checkNotNull; - +import android.media.MediaCodec; import androidx.annotation.Nullable; -import androidx.annotation.RequiresApi; import androidx.media3.common.C; import androidx.media3.common.util.UnstableApi; @@ -86,12 +83,12 @@ public final class CryptoInfo { */ public int clearBlocks; - private final android.media.MediaCodec.CryptoInfo frameworkCryptoInfo; - @Nullable private final PatternHolderV24 patternHolder; + private final MediaCodec.CryptoInfo frameworkCryptoInfo; + private final MediaCodec.CryptoInfo.Pattern frameworkCryptoInfoPattern; public CryptoInfo() { - frameworkCryptoInfo = new android.media.MediaCodec.CryptoInfo(); - patternHolder = SDK_INT >= 24 ? new PatternHolderV24(frameworkCryptoInfo) : null; + frameworkCryptoInfo = new MediaCodec.CryptoInfo(); + frameworkCryptoInfoPattern = new MediaCodec.CryptoInfo.Pattern(0, 0); } /** @@ -122,9 +119,8 @@ public void set( frameworkCryptoInfo.key = key; frameworkCryptoInfo.iv = iv; frameworkCryptoInfo.mode = mode; - if (SDK_INT >= 24) { - checkNotNull(patternHolder).set(encryptedBlocks, clearBlocks); - } + frameworkCryptoInfoPattern.set(encryptedBlocks, clearBlocks); + frameworkCryptoInfo.setPattern(frameworkCryptoInfoPattern); } /** @@ -136,7 +132,7 @@ public void set( * * @return The equivalent {@link android.media.MediaCodec.CryptoInfo} instance. */ - public android.media.MediaCodec.CryptoInfo getFrameworkCryptoInfo() { + public MediaCodec.CryptoInfo getFrameworkCryptoInfo() { return frameworkCryptoInfo; } @@ -162,21 +158,4 @@ public void increaseClearDataFirstSubSampleBy(int count) { } numBytesOfClearData[0] += count; } - - @RequiresApi(24) - private static final class PatternHolderV24 { - - private final android.media.MediaCodec.CryptoInfo frameworkCryptoInfo; - private final android.media.MediaCodec.CryptoInfo.Pattern pattern; - - private PatternHolderV24(android.media.MediaCodec.CryptoInfo frameworkCryptoInfo) { - this.frameworkCryptoInfo = frameworkCryptoInfo; - pattern = new android.media.MediaCodec.CryptoInfo.Pattern(0, 0); - } - - private void set(int encryptedBlocks, int clearBlocks) { - pattern.set(encryptedBlocks, clearBlocks); - frameworkCryptoInfo.setPattern(pattern); - } - } } diff --git a/libraries/decoder_av1/README.md b/libraries/decoder_av1/README.md index f6cfd8206b4..ef9ae72c591 100644 --- a/libraries/decoder_av1/README.md +++ b/libraries/decoder_av1/README.md @@ -44,7 +44,7 @@ HOST_PLATFORM="linux-x86_64" ``` cd "${AV1_MODULE_PATH}/jni" && \ -git clone https://github.com/google/cpu_features +git clone https://github.com/google/cpu_features --branch v0.9.0 --depth=1 ``` * Install [Meson][] (0.49 or higher), [Ninja][], and, for x86* targets, @@ -54,7 +54,7 @@ git clone https://github.com/google/cpu_features ``` cd "${AV1_MODULE_PATH}/jni" && \ -git clone https://code.videolan.org/videolan/dav1d.git +git clone https://code.videolan.org/videolan/dav1d.git --branch 1.5.4 --depth=1 ``` * Execute `build_dav1d.sh` to build `libdav1d.a` for all supported diff --git a/libraries/decoder_av1/proguard-rules.txt b/libraries/decoder_av1/proguard-rules.txt index d639ccdd40d..9cef13cfdf9 100644 --- a/libraries/decoder_av1/proguard-rules.txt +++ b/libraries/decoder_av1/proguard-rules.txt @@ -1,7 +1,7 @@ # Proguard rules specific to the AV1 extension. # This prevents the names of native methods from being obfuscated. --keepclasseswithmembernames class * { +-keep,includedescriptorclasses class androidx.media3.decoder.av1.** { native ; } diff --git a/libraries/decoder_av1/src/androidTest/java/androidx/media3/decoder/av1/Av1PlaybackTest.kt b/libraries/decoder_av1/src/androidTest/java/androidx/media3/decoder/av1/Av1PlaybackTest.kt index 4eef80a01bf..0626dd33ba2 100644 --- a/libraries/decoder_av1/src/androidTest/java/androidx/media3/decoder/av1/Av1PlaybackTest.kt +++ b/libraries/decoder_av1/src/androidTest/java/androidx/media3/decoder/av1/Av1PlaybackTest.kt @@ -16,8 +16,6 @@ package androidx.media3.decoder.av1 import android.content.Context -import android.graphics.SurfaceTexture -import android.view.Surface import androidx.media3.common.MediaItem import androidx.media3.common.Player import androidx.media3.exoplayer.ExoPlayer @@ -25,89 +23,67 @@ import androidx.media3.exoplayer.RenderersFactory import androidx.media3.exoplayer.video.VideoDecoderGLSurfaceView import androidx.media3.test.utils.awaitPlaybackState import androidx.test.core.app.ApplicationProvider.getApplicationContext -import androidx.test.filters.SdkSuppress import com.google.common.truth.Truth.assertThat import com.google.common.truth.Truth.assertWithMessage -import com.google.testing.junit.testparameterinjector.KotlinTestParameters.namedTestValues -import com.google.testing.junit.testparameterinjector.TestParameter -import com.google.testing.junit.testparameterinjector.TestParameterInjector import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.runBlocking import org.junit.Before import org.junit.Test import org.junit.runner.RunWith +import org.junit.runners.Parameterized +import org.junit.runners.Parameterized.Parameter +import org.junit.runners.Parameterized.Parameters /** Playback tests using [Libdav1dVideoRenderer]. */ -@RunWith(TestParameterInjector::class) +@RunWith(Parameterized::class) class Av1PlaybackTest { + @Parameter(0) @JvmField var title: String = "" + @Parameter(1) @JvmField var uri: String = "" + @Before fun setUp() { assertWithMessage("Dav1d library not available").that(Dav1dLibrary.isAvailable()).isTrue() } @Test - fun playback_rendersToGlSurfaceView( - @TestParameter - videoUri: String = - namedTestValues("DEFAULT" to SAMPLE_AV1_URI, "MONOCHROME" to SAMPLE_AV1_MONO_URI) - ) = + fun playback() = runBlocking(Dispatchers.Main) { - runPlaybackTest(videoUri) { player -> - player.setVideoSurfaceView(VideoDecoderGLSurfaceView(getApplicationContext())) + val context = getApplicationContext() + val renderersFactory = RenderersFactory { eventHandler, videoRendererEventListener, _, _, _ -> + arrayOf( + Libdav1dVideoRenderer( + /* allowedJoiningTimeMs= */ 0, + eventHandler, + videoRendererEventListener, + /* maxDroppedFramesToNotify= */ -1, + ) + ) } - } + val player = ExoPlayer.Builder(context).setRenderersFactory(renderersFactory).build() + player.setVideoSurfaceView(VideoDecoderGLSurfaceView(context)) + player.setMediaItem(MediaItem.fromUri(uri)) + player.prepare() + player.play() - // Rendering using ANativeWindow_lock is only reliably supported on API 29+. - @Test - @SdkSuppress(minSdkVersion = 29) - fun playback_rendersToSurface( - @TestParameter - videoUri: String = - namedTestValues("DEFAULT" to SAMPLE_AV1_URI, "MONOCHROME" to SAMPLE_AV1_MONO_URI) - ) = - runBlocking(Dispatchers.Main) { - val surfaceTexture = SurfaceTexture(/* texName= */ 0) - val surface = Surface(surfaceTexture) - try { - runPlaybackTest(videoUri) { player -> player.setVideoSurface(surface) } - } finally { - surface.release() - surfaceTexture.release() - } - } + val renderedOutputBufferCount = + try { + player.awaitPlaybackState(Player.STATE_ENDED) + player.videoDecoderCounters?.renderedOutputBufferCount ?: 0 + } finally { + player.release() + } - private suspend fun runPlaybackTest(videoUri: String, setSurfaceAction: (ExoPlayer) -> Unit) { - val context = getApplicationContext() - val renderersFactory = RenderersFactory { eventHandler, videoRendererEventListener, _, _, _ -> - arrayOf( - Libdav1dVideoRenderer( - /* allowedJoiningTimeMs= */ 0, - eventHandler, - videoRendererEventListener, - /* maxDroppedFramesToNotify= */ -1, - ) - ) + assertThat(renderedOutputBufferCount).isAtLeast(1) } - val player = ExoPlayer.Builder(context).setRenderersFactory(renderersFactory).build() - setSurfaceAction(player) - player.setMediaItem(MediaItem.fromUri(videoUri)) - player.prepare() - player.play() - - val renderedOutputBufferCount = - try { - player.awaitPlaybackState(Player.STATE_ENDED) - player.videoDecoderCounters?.renderedOutputBufferCount ?: 0 - } finally { - player.release() - } - - assertThat(renderedOutputBufferCount).isAtLeast(1) - } companion object { - private const val SAMPLE_AV1_URI = "asset:///media/mp4/sample_av1.mp4" - private const val SAMPLE_AV1_MONO_URI = "asset:///media/mp4/sample_av1_mono.mp4" + @JvmStatic + @Parameters(name = "{0}") + fun params(): List> = + listOf( + arrayOf("8bit_color_sdr", "asset:///media/mp4/sample_av1.mp4"), + arrayOf("8bit_monochrome_sdr", "asset:///media/mp4/sample_av1_monochrome_sdr.mp4"), + ) } } diff --git a/libraries/decoder_av1/src/main/java/androidx/media3/decoder/av1/Dav1dDecoder.java b/libraries/decoder_av1/src/main/java/androidx/media3/decoder/av1/Dav1dDecoder.java index df803dbb64a..5f8bc5a92b5 100644 --- a/libraries/decoder_av1/src/main/java/androidx/media3/decoder/av1/Dav1dDecoder.java +++ b/libraries/decoder_av1/src/main/java/androidx/media3/decoder/av1/Dav1dDecoder.java @@ -18,6 +18,7 @@ import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkState; +import android.os.Process; import android.view.Surface; import androidx.annotation.GuardedBy; import androidx.annotation.Nullable; @@ -115,6 +116,11 @@ public final class Dav1dDecoder * default decoder behavior for setting the threads. * @param maxFrameDelay Maximum amount of frames libdav1d can be behind on. * @param useCustomAllocator Whether to use a custom allocator for the decoder. + * @param threadPriority The thread priority for the decode thread and native worker threads. + * {@link Process#setThreadPriority} expects a value between [-20, 19] or if the value is + * {@link C#THREAD_PRIORITY_NO_OP}, then the thread priority will not be set. Values beyond + * this range will be clamped to the nearest value in the range. Specific values can be found + * in {@link Process} with the prefix {@code THREAD_PRIORITY_*}. * @throws Dav1dDecoderException Thrown if an exception occurs when initializing the decoder. */ // Suppressing nulless for UnderInitialization and method.invocation. @@ -125,7 +131,8 @@ public Dav1dDecoder( int initialInputBufferSize, int threads, int maxFrameDelay, - boolean useCustomAllocator) + boolean useCustomAllocator, + int threadPriority) throws Dav1dDecoderException { if (!Dav1dLibrary.isAvailable()) { throw new Dav1dDecoderException("Failed to load decoder native library."); @@ -152,6 +159,18 @@ public Dav1dDecoder( new Thread("ExoPlayer:Dav1dDecoder") { @Override public void run() { + try { + if (threadPriority != C.THREAD_PRIORITY_NO_OP) { + Process.setThreadPriority(threadPriority); + } + } catch (RuntimeException e) { + synchronized (lock) { + Dav1dDecoder.this.exception = + new Dav1dDecoderException("Failed to set thread priority", e); + lock.notify(); + return; + } + } synchronized (lock) { Dav1dDecoder.this.dav1dDecoderContext = dav1dInit(threads, maxFrameDelay, useCustomAllocator, numInputBuffers); @@ -302,6 +321,10 @@ public void renderToSurface(VideoDecoderOutputBuffer outputBuffer, Surface surfa if (outputMode != C.VIDEO_OUTPUT_MODE_SURFACE_YUV) { throw new Dav1dDecoderException("Unsupported Output Mode."); } + if (surface == null || !surface.isValid()) { + throw new Dav1dDecoderException( + "Failed to render output buffer to surface: invalid surface."); + } int error = dav1dRenderFrame(dav1dDecoderContext, surface, outputBuffer); if (error != DAV1D_OK) { String errorMsg = dav1dGetErrorMessage(dav1dDecoderContext); diff --git a/libraries/decoder_av1/src/main/java/androidx/media3/decoder/av1/Libdav1dVideoRenderer.java b/libraries/decoder_av1/src/main/java/androidx/media3/decoder/av1/Libdav1dVideoRenderer.java index 24d5e03881c..d2866010de3 100644 --- a/libraries/decoder_av1/src/main/java/androidx/media3/decoder/av1/Libdav1dVideoRenderer.java +++ b/libraries/decoder_av1/src/main/java/androidx/media3/decoder/av1/Libdav1dVideoRenderer.java @@ -18,6 +18,7 @@ import static androidx.media3.exoplayer.DecoderReuseEvaluation.REUSE_RESULT_YES_WITHOUT_RECONFIGURATION; import android.os.Handler; +import android.os.Process; import android.view.Surface; import androidx.annotation.Nullable; import androidx.media3.common.C; @@ -71,6 +72,8 @@ public class Libdav1dVideoRenderer extends DecoderVideoRenderer { private final boolean useCustomAllocator; + private final int threadPriority; + @Nullable private Dav1dDecoder decoder; /** @@ -98,7 +101,8 @@ public Libdav1dVideoRenderer( DEFAULT_MAX_FRAME_DELAY, DEFAULT_NUM_OF_INPUT_BUFFERS, DEFAULT_NUM_OF_OUTPUT_BUFFERS, - /* useCustomAllocator= */ false); + /* useCustomAllocator= */ false, + C.THREAD_PRIORITY_NO_OP); } /** @@ -121,6 +125,11 @@ public Libdav1dVideoRenderer( * @param numInputBuffers Number of input buffers. * @param numOutputBuffers Number of output buffers. * @param useCustomAllocator Whether to use a custom allocator for libdav1d. + * @param threadPriority The thread priority for the decode thread and native worker threads. + * {@link Process#setThreadPriority} expects a value between [-20, 19] or if the value is + * {@link C#THREAD_PRIORITY_NO_OP}, then the thread priority will not be set. Values beyond + * this range will be clamped to the nearest value in the range. Specific values can be found + * in {@link Process} with the prefix {@code THREAD_PRIORITY_*}. */ public Libdav1dVideoRenderer( long allowedJoiningTimeMs, @@ -131,13 +140,15 @@ public Libdav1dVideoRenderer( int maxFrameDelay, int numInputBuffers, int numOutputBuffers, - boolean useCustomAllocator) { + boolean useCustomAllocator, + int threadPriority) { super(allowedJoiningTimeMs, eventHandler, eventListener, maxDroppedFramesToNotify); this.threads = threads; this.numInputBuffers = numInputBuffers; this.numOutputBuffers = numOutputBuffers; this.maxFrameDelay = maxFrameDelay; this.useCustomAllocator = useCustomAllocator; + this.threadPriority = threadPriority; } @Override @@ -171,7 +182,8 @@ protected final Dav1dDecoder createDecoder(Format format, @Nullable CryptoConfig initialInputBufferSize, threads, maxFrameDelay, - useCustomAllocator); + useCustomAllocator, + threadPriority); this.decoder = decoder; TraceUtil.endSection(); return decoder; diff --git a/libraries/decoder_av1/src/main/jni/CMakeLists.txt b/libraries/decoder_av1/src/main/jni/CMakeLists.txt index e894e24c005..49a91cd6086 100644 --- a/libraries/decoder_av1/src/main/jni/CMakeLists.txt +++ b/libraries/decoder_av1/src/main/jni/CMakeLists.txt @@ -55,6 +55,10 @@ add_library(dav1dJNI cpu_info.cc cpu_info.h) +set_target_properties(dav1dJNI PROPERTIES + CXX_VISIBILITY_PRESET hidden + VISIBILITY_INLINES_HIDDEN ON) + # Locate NDK log library. find_library(android_log_lib log) diff --git a/libraries/decoder_av1/src/main/jni/dav1d_jni.cc b/libraries/decoder_av1/src/main/jni/dav1d_jni.cc index f1899e80ec2..0d17ba1d7bb 100644 --- a/libraries/decoder_av1/src/main/jni/dav1d_jni.cc +++ b/libraries/decoder_av1/src/main/jni/dav1d_jni.cc @@ -14,9 +14,12 @@ * limitations under the License. */ +#include +#include #include #include #include +#include #include #include @@ -46,6 +49,10 @@ #include +#if defined(__ARM_NEON) || defined(__aarch64__) || defined(__ARM_ARCH_7A__) +#include +#endif + #include #include #include // NOLINT @@ -57,33 +64,6 @@ #define LOG_TAG "dav1d_jni" #define LOGE(...) \ ((void)__android_log_print(ANDROID_LOG_ERROR, LOG_TAG, __VA_ARGS__)) - -#define DECODER_FUNC(RETURN_TYPE, NAME, ...) \ - extern "C" { \ - JNIEXPORT RETURN_TYPE Java_androidx_media3_decoder_av1_Dav1dDecoder_##NAME( \ - JNIEnv* env, jobject thiz, ##__VA_ARGS__); \ - } \ - JNIEXPORT RETURN_TYPE Java_androidx_media3_decoder_av1_Dav1dDecoder_##NAME( \ - JNIEnv* env, jobject thiz, ##__VA_ARGS__) - -// If ANDROID_NATIVE_LIB_MERGING is set, rename JNI_OnLoad to -// JNI_OnLoad_libdav1dJNI. The suffix has to be the same as "lib" -// where is the name of the android_jni_library target. This lets -// apps merge native libraries and reduce the binary size. See go/android_onelib -// for details. -#ifdef ANDROID_NATIVE_LIB_MERGING -jint JNI_OnLoad_libdav1dJNI -#else -jint JNI_OnLoad -#endif - (JavaVM* vm, void* reserved) { - JNIEnv* env; - if (vm->GetEnv(reinterpret_cast(&env), JNI_VERSION_1_6) != JNI_OK) { - return -1; - } - return JNI_VERSION_1_6; -} - namespace { // YUV plane indices. @@ -92,10 +72,24 @@ const int kPlaneU = 1; const int kPlaneV = 2; const int kMaxPlanes = 3; +// Bit-shift for aligning 10-bit to the MSB of a 16-bit P010 word. +constexpr int kP010Shift = 6; + // Android YUV format. See: // https://developer.android.com/reference/android/graphics/ImageFormat.html#YV12. const int kImageFormatYV12 = 0x32315659; +// Android P010 format. See: +// https://developer.android.com/reference/android/hardware/HardwareBuffer#AHARDWAREBUFFER_FORMAT_YCbCr_P010. +const int kImageFormatP010 = 0x36; + +// Color range values for dav1d. +const int kColorRangeStudio = 0; +const int kColorRangeFull = 1; + +// Mid-gray pixel value for 8-bit YUV plane filling. +const uint8_t kMidGray = 128; + // LINT.IfChange // Output modes. const int kOutputModeYuv = 0; @@ -124,9 +118,6 @@ const int kDav1dThreadCountPerformanceCores = -1; const int kDav1dThreadCountExperimental = -2; // LINT.ThenChange(../java/androidx/media3/decoder/av1/Libdav1dVideoRenderer.java) -// Monochrome planes fill value -const int kMidGray = 128; - // Status codes specific to the JNI wrapper code. enum JniStatusCode { kJniStatusOk = 0, @@ -147,6 +138,8 @@ enum JniStatusCode { kJniStatusRenderNativeWindowNullError = -15, kJniStatusRenderPictureNullError = -16, kJniStatusRenderLockFailedError = -17, + kJniStatusInvalidBitDepthError = -18, + kJniStatusUnsupportedPixelLayoutError = -19, }; const int kLibdav1dDecoderStatusOk = 0; @@ -190,6 +183,10 @@ const char* GetJniErrorMessage(JniStatusCode error_code) { return "Picture unallocated null natively."; case kJniStatusRenderLockFailedError: return "ANativeWindow raw lock broke natively."; + case kJniStatusInvalidBitDepthError: + return "Invalid bit depth."; + case kJniStatusUnsupportedPixelLayoutError: + return "Unsupported pixel layout for 10-bit SurfaceYuv."; default: return "Unrecognized error code."; } @@ -266,6 +263,8 @@ struct JniContext { } native_window_width = 0; native_window_height = 0; + native_window_format = kImageFormatYV12; + native_window_dataspace = ADATASPACE_UNKNOWN; if (new_surface == nullptr) { LOGE("MaybeAcquireNativeWindow: new_surface is null."); jni_status_code = kJniStatusRenderMaybeAcquireFailedError; @@ -327,6 +326,8 @@ struct JniContext { jobject surface = nullptr; int native_window_width = 0; int native_window_height = 0; + int native_window_format = 0; + int native_window_dataspace = 0; JavaVM* jvm; }; @@ -340,20 +341,8 @@ void CopyFrameToDataBuffer(const Dav1dPicture* dav1d_picture, jbyte* data) { uint64_t planeHeight = (plane_index == kPlaneY) ? dav1d_picture->p.h : ((dav1d_picture->p.h + 1) / 2); - bool is_monochrome = dav1d_picture->p.layout == DAV1D_PIXEL_LAYOUT_I400; - int stride = dav1d_picture->stride[stride_index]; - if (is_monochrome && plane_index != kPlaneY) { - stride = (dav1d_picture->stride[kPlaneY] + 1) / 2; - } - uint64_t length = stride * planeHeight; - if (length > 0) { - // Monochrome plane pointers are NULL. Fill with a neutral value (128). - if (dav1d_picture->data[plane_index] != nullptr) { - memcpy(data, dav1d_picture->data[plane_index], length); - } else { - memset(data, kMidGray, length); - } - } + uint64_t length = dav1d_picture->stride[stride_index] * planeHeight; + memcpy(data, dav1d_picture->data[plane_index], length); data += length; } } @@ -369,8 +358,8 @@ void CopyPlane(const uint8_t* source, int source_stride, uint8_t* destination, } } -void FillPlane(const uint8_t value, uint8_t* destination, - int destination_stride, int width, int height) { +void FillPlane(uint8_t value, uint8_t* destination, int destination_stride, + int width, int height) { while (height--) { std::memset(destination, value, width); destination += destination_stride; @@ -389,6 +378,22 @@ constexpr int GetColorSpace(Dav1dColorPrimaries primary) { return kColorSpaceUnknown; } } + +int GetDataSpace(const Dav1dPicture* picture) { + if (picture->p.bpc == 8) { + return ADATASPACE_UNKNOWN; + } + bool full_range = picture->seq_hdr->color_range == kColorRangeFull; + switch (picture->seq_hdr->trc) { + case DAV1D_TRC_SMPTE2084: // PQ + return full_range ? ADATASPACE_BT2020_PQ : ADATASPACE_BT2020_ITU_PQ; + case DAV1D_TRC_HLG: // HLG + return full_range ? ADATASPACE_BT2020_HLG : ADATASPACE_BT2020_ITU_HLG; + default: + return ADATASPACE_UNKNOWN; + } +} + void Dav1dDataFreeCallback(const uint8_t* data, void* cookie) { Cookie* cookie_ptr = reinterpret_cast(cookie); JniContext* const context = @@ -522,10 +527,156 @@ void CleanUpAllocatorData(jlong jContext, JNIEnv* env) { context->unused_picture_allocator_data.pop_back(); } } + +void RenderFrame10Bit(const Dav1dPicture* dav1d_picture, + ANativeWindow_Buffer& native_window_buffer, + int32_t y_copy_width, int32_t y_copy_height) { + // Y plane + const uint16_t* __restrict src_y = + reinterpret_cast(dav1d_picture->data[kPlaneY]); + uint16_t* __restrict dst_y = + reinterpret_cast(native_window_buffer.bits); + + for (int h = 0; h < y_copy_height; h++) { + int w = 0; +#if defined(__aarch64__) + for (; w <= y_copy_width - 16; w += 16) { + uint16x8x2_t y = {vld1q_u16(src_y + w), vld1q_u16(src_y + w + 8)}; + y.val[0] = vshlq_n_u16(y.val[0], kP010Shift); + y.val[1] = vshlq_n_u16(y.val[1], kP010Shift); + vst1q_u16(dst_y + w, y.val[0]); + vst1q_u16(dst_y + w + 8, y.val[1]); + } +#elif defined(__ARM_ARCH_7A__) || defined(__ARM_NEON) + for (; w <= y_copy_width - 8; w += 8) { + uint16x8_t y = vld1q_u16(src_y + w); + y = vshlq_n_u16(y, kP010Shift); + vst1q_u16(dst_y + w, y); + } +#endif + for (; w < y_copy_width; w++) { + dst_y[w] = src_y[w] << kP010Shift; + } + src_y += dav1d_picture->stride[kPlaneY] / 2; + dst_y += native_window_buffer.stride; + } + + const int32_t y_plane_size = + native_window_buffer.stride * native_window_buffer.height; + uint16_t* dst_uv = + reinterpret_cast(native_window_buffer.bits) + y_plane_size; + const int32_t uv_copy_height = (y_copy_height + 1) / 2; + const int32_t uv_copy_width = (y_copy_width + 1) / 2; + + if (dav1d_picture->p.layout == DAV1D_PIXEL_LAYOUT_I400) { + for (int h = 0; h < uv_copy_height; h++) { + std::fill_n(dst_uv, uv_copy_width * 2, static_cast(0x8000)); + dst_uv += native_window_buffer.stride; + } + } else { + // interleave U and V Planes + const uint16_t* src_u = + reinterpret_cast(dav1d_picture->data[kPlaneU]); + const uint16_t* src_v = + reinterpret_cast(dav1d_picture->data[kPlaneV]); + + for (int h = 0; h < uv_copy_height; h++) { + int w = 0; +#if defined(__aarch64__) + for (; w <= uv_copy_width - 16; w += 16) { + // Process first 8 elements + uint16x8_t u0 = vld1q_u16(src_u + w); + uint16x8_t v0 = vld1q_u16(src_v + w); + u0 = vshlq_n_u16(u0, kP010Shift); + v0 = vshlq_n_u16(v0, kP010Shift); + uint16x8x2_t uv0 = {u0, v0}; + vst2q_u16(dst_uv + w * 2, uv0); + // Process next 8 elements + uint16x8_t u1 = vld1q_u16(src_u + w + 8); + uint16x8_t v1 = vld1q_u16(src_v + w + 8); + u1 = vshlq_n_u16(u1, kP010Shift); + v1 = vshlq_n_u16(v1, kP010Shift); + uint16x8x2_t uv1 = {u1, v1}; + vst2q_u16(dst_uv + w * 2 + 16, uv1); + } +#elif defined(__ARM_ARCH_7A__) || defined(__ARM_NEON) + for (; w <= uv_copy_width - 8; w += 8) { + uint16x8x2_t uv; + uv.val[0] = vld1q_u16(src_u + w); + uv.val[1] = vld1q_u16(src_v + w); + + uv.val[0] = vshlq_n_u16(uv.val[0], kP010Shift); + uv.val[1] = vshlq_n_u16(uv.val[1], kP010Shift); + vst2q_u16(dst_uv + w * 2, uv); + } +#endif + for (; w < uv_copy_width; w++) { + dst_uv[w * 2] = src_u[w] << kP010Shift; + dst_uv[w * 2 + 1] = src_v[w] << kP010Shift; + } + src_u += dav1d_picture->stride[kPlaneU] / 2; + src_v += dav1d_picture->stride[kPlaneU] / 2; + dst_uv += native_window_buffer.stride; + } + } +} + +void RenderFrame8Bit(const Dav1dPicture* dav1d_picture, + ANativeWindow_Buffer& native_window_buffer, + int32_t y_copy_width, int32_t y_copy_height) { + // 8-bit rendering (YV12 layout) + const int32_t y_plane_size = + native_window_buffer.stride * native_window_buffer.height; + const int32_t native_window_buffer_uv_height = + (native_window_buffer.height + 1) / 2; + const int32_t native_window_buffer_uv_stride = + AlignTo16(native_window_buffer.stride / 2); + const int32_t v_plane_size = + native_window_buffer_uv_height * native_window_buffer_uv_stride; + + uint8_t* uv_base_ptr = + reinterpret_cast(native_window_buffer.bits) + y_plane_size; + // In YV12 format, V plane comes before U plane. + uint8_t* v_dest_ptr = uv_base_ptr; + uint8_t* u_dest_ptr = uv_base_ptr + v_plane_size; + + const int32_t uv_copy_height = (y_copy_height + 1) / 2; + const int32_t uv_copy_width = (y_copy_width + 1) / 2; + + // Y plane + CopyPlane(reinterpret_cast(dav1d_picture->data[kPlaneY]), + dav1d_picture->stride[kPlaneY], + reinterpret_cast(native_window_buffer.bits), + native_window_buffer.stride, y_copy_width, y_copy_height); + + // Handle monochrome videos + const Dav1dSequenceHeader* header = dav1d_picture->seq_hdr; + if (header->monochrome) { + // V Plane + // Set to 128 to make it gray + FillPlane(kMidGray, v_dest_ptr, native_window_buffer_uv_stride, + uv_copy_width, uv_copy_height); + // U Plane + FillPlane(kMidGray, u_dest_ptr, native_window_buffer_uv_stride, + uv_copy_width, uv_copy_height); + } else { + // V plane + // Since the format for ANativeWindow is YV12, V plane is being processed + // before U plane. + CopyPlane(reinterpret_cast(dav1d_picture->data[kPlaneV]), + dav1d_picture->stride[kPlaneU], v_dest_ptr, + native_window_buffer_uv_stride, uv_copy_width, uv_copy_height); + // U plane + CopyPlane(reinterpret_cast(dav1d_picture->data[kPlaneU]), + dav1d_picture->stride[kPlaneU], u_dest_ptr, + native_window_buffer_uv_stride, uv_copy_width, uv_copy_height); + } +} + } // namespace -DECODER_FUNC(jlong, dav1dInit, jint threads, jint max_frame_delay, - jboolean use_custom_allocator, jint num_input_buffers) { +jlong dav1dInit(JNIEnv* env, jobject thiz, jint threads, jint max_frame_delay, + jboolean use_custom_allocator, jint num_input_buffers) { JniContext* context = new (std::nothrow) JniContext(); if (context == nullptr) { return kStatusError; @@ -689,7 +840,7 @@ DECODER_FUNC(jlong, dav1dInit, jint threads, jint max_frame_delay, return reinterpret_cast(context); } -DECODER_FUNC(void, dav1dClose, jlong jContext) { +void dav1dClose(JNIEnv* env, jobject thiz, jlong jContext) { if (jContext == kStatusError) { return; } @@ -729,9 +880,10 @@ DECODER_FUNC(void, dav1dClose, jlong jContext) { delete context; } -DECODER_FUNC(jint, dav1dDecode, jlong jContext, jobject jInputBuffer, - jint bufferIndex, jint offset, jint length, jboolean decodeOnly, - jint flags, jlong timeUs, jint outputMode) { +jint dav1dDecode(JNIEnv* env, jobject thiz, jlong jContext, + jobject jInputBuffer, jint bufferIndex, jint offset, + jint length, jboolean decodeOnly, jint flags, jlong timeUs, + jint outputMode) { if (jContext == kStatusError) { return kStatusError; } @@ -815,7 +967,8 @@ DECODER_FUNC(jint, dav1dDecode, jlong jContext, jobject jInputBuffer, return kStatusOk; } -DECODER_FUNC(jint, dav1dGetFrame, jlong jContext, jobject jOutputBuffer) { +jint dav1dGetFrame(JNIEnv* env, jobject thiz, jlong jContext, + jobject jOutputBuffer) { if (jContext == kStatusError) { return kStatusError; } @@ -862,7 +1015,24 @@ DECODER_FUNC(jint, dav1dGetFrame, jlong jContext, jobject jOutputBuffer) { context->jni_status_code = kJniStatusBufferInitError; return kStatusError; } - if (dav1d_picture->p.bpc != 8) { + int bpc = dav1d_picture->p.bpc; + if (bpc != 8 && bpc != 10) { + std::lock_guard lock( // NOLINT(build/c++11) + context->state_mutex); + context->jni_status_code = kJniStatusInvalidBitDepthError; + return kStatusError; + } + if (bpc == 10 && returned_user_data->output_mode == kOutputModeSurfaceYuv) { + // The current P010 rendering in dav1dRenderFrame assumes I420 or I400. + if (dav1d_picture->p.layout != DAV1D_PIXEL_LAYOUT_I420 && + dav1d_picture->p.layout != DAV1D_PIXEL_LAYOUT_I400) { + std::lock_guard lock( // NOLINT(build/c++11) + context->state_mutex); + context->jni_status_code = kJniStatusUnsupportedPixelLayoutError; + return kStatusError; + } + } + if (bpc != 8 && returned_user_data->output_mode == kOutputModeYuv) { std::lock_guard lock( // NOLINT(build/c++11) context->state_mutex); context->jni_status_code = kJniStatusHighBitDepthNotSupportedWithYuv; @@ -870,11 +1040,6 @@ DECODER_FUNC(jint, dav1dGetFrame, jlong jContext, jobject jOutputBuffer) { } if (returned_user_data->output_mode == kOutputModeYuv) { jboolean init_result; - int y_stride = dav1d_picture->stride[kPlaneY]; - int uv_stride = dav1d_picture->stride[kPlaneU]; - if (dav1d_picture->p.layout == DAV1D_PIXEL_LAYOUT_I400) { - uv_stride = (y_stride + 1) / 2; - } if (context->use_custom_allocator) { PictureAllocatorData* allocator_data = reinterpret_cast( @@ -884,13 +1049,15 @@ DECODER_FUNC(jint, dav1dGetFrame, jlong jContext, jobject jOutputBuffer) { init_result = env->CallBooleanMethod( jOutputBuffer, context->init_for_offset_frames_method, allocator_data->offset, dav1d_picture->p.w, dav1d_picture->p.h, - y_stride, uv_stride, GetColorSpace(dav1d_picture->seq_hdr->pri), + dav1d_picture->stride[kPlaneY], dav1d_picture->stride[kPlaneU], + GetColorSpace(dav1d_picture->seq_hdr->pri), allocator_data->aligned_height); CleanUpAllocatorData(jContext, env); } else { init_result = env->CallBooleanMethod( jOutputBuffer, context->init_for_yuv_frame_method, dav1d_picture->p.w, - dav1d_picture->p.h, y_stride, uv_stride, + dav1d_picture->p.h, dav1d_picture->stride[kPlaneY], + dav1d_picture->stride[kPlaneU], GetColorSpace(dav1d_picture->seq_hdr->pri)); } if (!init_result) { @@ -943,8 +1110,8 @@ DECODER_FUNC(jint, dav1dGetFrame, jlong jContext, jobject jOutputBuffer) { return kStatusOk; } -DECODER_FUNC(jint, dav1dRenderFrame, jlong jContext, jobject jSurface, - jobject jOutputBuffer) { +jint dav1dRenderFrame(JNIEnv* env, jobject thiz, jlong jContext, + jobject jSurface, jobject jOutputBuffer) { if (jContext == kStatusError) { LOGE("Failed to render frame. jContext is error."); return kStatusError; @@ -975,6 +1142,12 @@ DECODER_FUNC(jint, dav1dRenderFrame, jlong jContext, jobject jSurface, return kStatusError; } + if (dav1d_picture->p.bpc != 8 && dav1d_picture->p.bpc != 10) { + LOGE("Unsupported bit depth: %d", dav1d_picture->p.bpc); + context->jni_status_code = kJniStatusInvalidBitDepthError; + return kStatusError; + } + // Get the display width and height metadata from the output buffer. int32_t width = env->GetIntField(jOutputBuffer, context->display_width_field); if (ClearPendingException(env)) { @@ -987,16 +1160,43 @@ DECODER_FUNC(jint, dav1dRenderFrame, jlong jContext, jobject jSurface, context->jni_status_code = kJniStatusANativeWindowError; return kStatusError; } + + // Determine format and dataspace based on bit depth. + int32_t buffer_format = kImageFormatYV12; + int32_t dataspace = ADATASPACE_UNKNOWN; + if (dav1d_picture->p.bpc == 10) { + buffer_format = kImageFormatP010; + dataspace = GetDataSpace(dav1d_picture); + } + if (context->native_window_width != width || - context->native_window_height != height) { + context->native_window_height != height || + context->native_window_format != buffer_format) { if (ANativeWindow_setBuffersGeometry(context->native_window, width, height, - kImageFormatYV12)) { + buffer_format)) { context->jni_status_code = kJniStatusBufferResizeError; LOGE("Failed to set buffers geometry."); return kStatusError; } context->native_window_width = width; context->native_window_height = height; + context->native_window_format = buffer_format; + } + + if (context->native_window_dataspace != dataspace) { + if (__builtin_available(android 28, *)) { + typedef int32_t (*ANativeWindow_setBuffersDataSpaceFn)(ANativeWindow*, + int32_t); + static auto setBuffersDataSpace = + reinterpret_cast( + dlsym(RTLD_DEFAULT, "ANativeWindow_setBuffersDataSpace")); + if (setBuffersDataSpace != nullptr) { + if (setBuffersDataSpace(context->native_window, dataspace)) { + LOGE("Failed to set buffers dataspace."); + } + } + } + context->native_window_dataspace = dataspace; } ANativeWindow_Buffer native_window_buffer; @@ -1017,51 +1217,14 @@ DECODER_FUNC(jint, dav1dRenderFrame, jlong jContext, jobject jSurface, (int32_t)native_window_buffer.stride); y_copy_width = std::min(y_copy_width, (int32_t)width); - // Y plane - CopyPlane(reinterpret_cast(dav1d_picture->data[kPlaneY]), - dav1d_picture->stride[kPlaneY], - reinterpret_cast(native_window_buffer.bits), - native_window_buffer.stride, y_copy_width, y_copy_height); - - const int32_t y_plane_size = - native_window_buffer.stride * native_window_buffer.height; - const int32_t native_window_buffer_uv_stride = - AlignTo16(native_window_buffer.stride / 2); - const int32_t uv_plane_height = (native_window_buffer.height + 1) / 2; - const int32_t uv_plane_size = - uv_plane_height * native_window_buffer_uv_stride; - - uint8_t* uv_base_ptr = - reinterpret_cast(native_window_buffer.bits) + y_plane_size; - // In YV12 format, V plane comes before U plane. - uint8_t* v_dest_ptr = uv_base_ptr; - uint8_t* u_dest_ptr = uv_base_ptr + uv_plane_size; - - const int32_t uv_copy_height = (y_copy_height + 1) / 2; - const int32_t uv_copy_width = (y_copy_width + 1) / 2; - - // Handle monochrome videos - const Dav1dSequenceHeader* header = dav1d_picture->seq_hdr; - if (header->monochrome) { - // V Plane - // Set to 128 to make it gray - FillPlane(kMidGray, v_dest_ptr, native_window_buffer_uv_stride, - uv_copy_width, uv_copy_height); - // U Plane - FillPlane(kMidGray, u_dest_ptr, native_window_buffer_uv_stride, - uv_copy_width, uv_copy_height); + if (dav1d_picture->p.bpc == 10) { + RenderFrame10Bit(dav1d_picture, native_window_buffer, y_copy_width, + y_copy_height); } else { - // V plane - // Since the format for ANativeWindow is YV12, V plane is being processed - // before U plane. - CopyPlane(reinterpret_cast(dav1d_picture->data[kPlaneV]), - dav1d_picture->stride[kPlaneU], v_dest_ptr, - native_window_buffer_uv_stride, uv_copy_width, uv_copy_height); - // U plane - CopyPlane(reinterpret_cast(dav1d_picture->data[kPlaneU]), - dav1d_picture->stride[kPlaneU], u_dest_ptr, - native_window_buffer_uv_stride, uv_copy_width, uv_copy_height); + RenderFrame8Bit(dav1d_picture, native_window_buffer, y_copy_width, + y_copy_height); } + if (ANativeWindow_unlockAndPost(context->native_window)) { context->jni_status_code = kJniStatusANativeWindowError; LOGE("Failed to unlock and post native window."); @@ -1070,7 +1233,8 @@ DECODER_FUNC(jint, dav1dRenderFrame, jlong jContext, jobject jSurface, return kStatusOk; } -DECODER_FUNC(void, dav1dReleaseFrame, jlong jContext, jobject jOutputBuffer) { +void dav1dReleaseFrame(JNIEnv* env, jobject thiz, jlong jContext, + jobject jOutputBuffer) { if (jContext == kStatusError) { return; } @@ -1091,7 +1255,7 @@ DECODER_FUNC(void, dav1dReleaseFrame, jlong jContext, jobject jOutputBuffer) { } } -DECODER_FUNC(jstring, dav1dGetErrorMessage, jlong jContext) { +jstring dav1dGetErrorMessage(JNIEnv* env, jobject thiz, jlong jContext) { if (jContext == kStatusError) { return env->NewStringUTF("Failed to initialize JNI context."); } @@ -1118,7 +1282,7 @@ DECODER_FUNC(jstring, dav1dGetErrorMessage, jlong jContext) { return env->NewStringUTF("None."); } -DECODER_FUNC(jint, dav1dGetLastErrorJniStatusCode, jlong jContext) { +jint dav1dGetLastErrorJniStatusCode(JNIEnv* env, jobject thiz, jlong jContext) { if (jContext == kStatusError) { return kJniStatusOk; } @@ -1126,7 +1290,7 @@ DECODER_FUNC(jint, dav1dGetLastErrorJniStatusCode, jlong jContext) { return context->jni_status_code; } -DECODER_FUNC(jint, dav1dCheckError, jlong jContext) { +jint dav1dCheckError(JNIEnv* env, jobject thiz, jlong jContext) { if (jContext == kStatusError) { return kStatusError; } @@ -1139,7 +1303,7 @@ DECODER_FUNC(jint, dav1dCheckError, jlong jContext) { : kStatusOk; } -DECODER_FUNC(void, dav1dFlush, jlong jContext) { +void dav1dFlush(JNIEnv* env, jobject thiz, jlong jContext) { if (jContext == kStatusError) { return; } @@ -1150,7 +1314,8 @@ DECODER_FUNC(void, dav1dFlush, jlong jContext) { } } -DECODER_FUNC(void, releaseUnusedInputBuffers, jlong jContext, jobject decoder) { +void releaseUnusedInputBuffers(JNIEnv* env, jobject thiz, jlong jContext, + jobject decoder) { if (jContext == kStatusError) { return; } @@ -1178,3 +1343,48 @@ DECODER_FUNC(void, releaseUnusedInputBuffers, jlong jContext, jobject decoder) { CleanUpAllocatorData(jContext, env); } } + +extern "C" JNIEXPORT jint JNICALL JNI_OnLoad(JavaVM* vm, void* reserved) { + JNIEnv* env; + if (vm->GetEnv(reinterpret_cast(&env), JNI_VERSION_1_6) != JNI_OK) { + return -1; + } + + jclass clazz = env->FindClass("androidx/media3/decoder/av1/Dav1dDecoder"); + if (!clazz) { + LOGE("JNI_OnLoad: FindClass failed for Dav1dDecoder"); + return -1; + } + static const JNINativeMethod kMethods[] = { + {"dav1dInit", "(IIZI)J", reinterpret_cast(dav1dInit)}, + {"dav1dClose", "(J)V", reinterpret_cast(dav1dClose)}, + {"dav1dDecode", "(JLandroidx/media3/decoder/DecoderInputBuffer;IIIZIJI)I", + reinterpret_cast(dav1dDecode)}, + {"dav1dGetFrame", + "(JLandroidx/media3/decoder/VideoDecoderOutputBuffer;)I", + reinterpret_cast(dav1dGetFrame)}, + {"dav1dRenderFrame", + "(JLandroid/view/Surface;Landroidx/media3/decoder/" + "VideoDecoderOutputBuffer;)I", + reinterpret_cast(dav1dRenderFrame)}, + {"dav1dReleaseFrame", + "(JLandroidx/media3/decoder/VideoDecoderOutputBuffer;)V", + reinterpret_cast(dav1dReleaseFrame)}, + {"dav1dGetErrorMessage", "(J)Ljava/lang/String;", + reinterpret_cast(dav1dGetErrorMessage)}, + {"dav1dGetLastErrorJniStatusCode", "(J)I", + reinterpret_cast(dav1dGetLastErrorJniStatusCode)}, + {"dav1dCheckError", "(J)I", reinterpret_cast(dav1dCheckError)}, + {"dav1dFlush", "(J)V", reinterpret_cast(dav1dFlush)}, + {"releaseUnusedInputBuffers", + "(JLandroidx/media3/decoder/av1/Dav1dDecoder;)V", + reinterpret_cast(releaseUnusedInputBuffers)}, + }; + if (env->RegisterNatives(clazz, kMethods, + sizeof(kMethods) / sizeof(kMethods[0])) < 0) { + LOGE("JNI_OnLoad: RegisterNatives failed for Dav1dDecoder"); + return -1; + } + + return JNI_VERSION_1_6; +} diff --git a/libraries/decoder_ffmpeg/proguard-rules.txt b/libraries/decoder_ffmpeg/proguard-rules.txt index aaf9280fd9b..c53b832baae 100644 --- a/libraries/decoder_ffmpeg/proguard-rules.txt +++ b/libraries/decoder_ffmpeg/proguard-rules.txt @@ -1,7 +1,7 @@ # Proguard rules specific to the FFmpeg extension. # This prevents the names of native methods from being obfuscated. --keepclasseswithmembernames class * { +-keep,includedescriptorclasses class androidx.media3.decoder.ffmpeg.** { native ; } diff --git a/libraries/decoder_ffmpeg/src/main/jni/CMakeLists.txt b/libraries/decoder_ffmpeg/src/main/jni/CMakeLists.txt index ad345fe832c..e07963d740f 100644 --- a/libraries/decoder_ffmpeg/src/main/jni/CMakeLists.txt +++ b/libraries/decoder_ffmpeg/src/main/jni/CMakeLists.txt @@ -46,6 +46,10 @@ add_library(ffmpegJNI SHARED ffmpeg_jni.cc) +set_target_properties(ffmpegJNI PROPERTIES + CXX_VISIBILITY_PRESET hidden + VISIBILITY_INLINES_HIDDEN ON) + target_link_libraries(ffmpegJNI PRIVATE android PRIVATE swresample diff --git a/libraries/decoder_ffmpeg/src/main/jni/ffmpeg_jni.cc b/libraries/decoder_ffmpeg/src/main/jni/ffmpeg_jni.cc index 43ecf04edce..4dd42aaa309 100644 --- a/libraries/decoder_ffmpeg/src/main/jni/ffmpeg_jni.cc +++ b/libraries/decoder_ffmpeg/src/main/jni/ffmpeg_jni.cc @@ -38,27 +38,6 @@ extern "C" { #define LOGD(...) \ ((void)__android_log_print(ANDROID_LOG_DEBUG, LOG_TAG, __VA_ARGS__)) -#define LIBRARY_FUNC(RETURN_TYPE, NAME, ...) \ - extern "C" { \ - JNIEXPORT RETURN_TYPE \ - Java_androidx_media3_decoder_ffmpeg_FfmpegLibrary_##NAME(JNIEnv* env, \ - jobject thiz, \ - ##__VA_ARGS__); \ - } \ - JNIEXPORT RETURN_TYPE \ - Java_androidx_media3_decoder_ffmpeg_FfmpegLibrary_##NAME( \ - JNIEnv* env, jobject thiz, ##__VA_ARGS__) - -#define AUDIO_DECODER_FUNC(RETURN_TYPE, NAME, ...) \ - extern "C" { \ - JNIEXPORT RETURN_TYPE \ - Java_androidx_media3_decoder_ffmpeg_FfmpegAudioDecoder_##NAME( \ - JNIEnv* env, jobject thiz, ##__VA_ARGS__); \ - } \ - JNIEXPORT RETURN_TYPE \ - Java_androidx_media3_decoder_ffmpeg_FfmpegAudioDecoder_##NAME( \ - JNIEnv* env, jobject thiz, ##__VA_ARGS__) - #define ERROR_STRING_BUFFER_LENGTH 256 // Output format corresponding to AudioFormat.ENCODING_PCM_16BIT. @@ -119,44 +98,21 @@ void logError(const char* functionName, int errorNumber); */ void releaseContext(AVCodecContext* context); -jint JNI_OnLoad(JavaVM* vm, void* reserved) { - JNIEnv* env; - if (vm->GetEnv(reinterpret_cast(&env), JNI_VERSION_1_6) != JNI_OK) { - LOGE("JNI_OnLoad: GetEnv failed"); - return -1; - } - jclass clazz = - env->FindClass("androidx/media3/decoder/ffmpeg/FfmpegAudioDecoder"); - if (!clazz) { - LOGE("JNI_OnLoad: FindClass failed"); - return -1; - } - growOutputBufferMethod = - env->GetMethodID(clazz, "growOutputBuffer", - "(Landroidx/media3/decoder/" - "SimpleDecoderOutputBuffer;I)Ljava/nio/ByteBuffer;"); - if (!growOutputBufferMethod) { - LOGE("JNI_OnLoad: GetMethodID failed"); - return -1; - } - return JNI_VERSION_1_6; -} - -LIBRARY_FUNC(jstring, ffmpegGetVersion) { +jstring ffmpegGetVersion(JNIEnv* env, jobject thiz) { return env->NewStringUTF(LIBAVCODEC_IDENT); } -LIBRARY_FUNC(jint, ffmpegGetInputBufferPaddingSize) { +jint ffmpegGetInputBufferPaddingSize(JNIEnv* env, jobject thiz) { return (jint)AV_INPUT_BUFFER_PADDING_SIZE; } -LIBRARY_FUNC(jboolean, ffmpegHasDecoder, jstring codecName) { +jboolean ffmpegHasDecoder(JNIEnv* env, jobject thiz, jstring codecName) { return getCodecByName(env, codecName) != NULL; } -AUDIO_DECODER_FUNC(jlong, ffmpegInitialize, jstring codecName, - jbyteArray extraData, jboolean outputFloat, - jint rawSampleRate, jint rawChannelCount) { +jlong ffmpegInitialize(JNIEnv* env, jobject thiz, jstring codecName, + jbyteArray extraData, jboolean outputFloat, + jint rawSampleRate, jint rawChannelCount) { const AVCodec* codec = getCodecByName(env, codecName); if (!codec) { LOGE("Codec not found."); @@ -166,9 +122,9 @@ AUDIO_DECODER_FUNC(jlong, ffmpegInitialize, jstring codecName, rawChannelCount); } -AUDIO_DECODER_FUNC(jint, ffmpegDecode, jlong context, jobject inputData, - jint inputSize, jobject decoderOutputBuffer, - jobject outputData, jint outputSize) { +jint ffmpegDecode(JNIEnv* env, jobject thiz, jlong context, jobject inputData, + jint inputSize, jobject decoderOutputBuffer, + jobject outputData, jint outputSize) { if (!context) { LOGE("Context must be non-NULL."); return -1; @@ -212,7 +168,7 @@ uint8_t* GrowOutputBufferCallback::operator()(int requiredSize) const { return static_cast(env->GetDirectBufferAddress(newOutputData)); } -AUDIO_DECODER_FUNC(jint, ffmpegGetChannelCount, jlong context) { +jint ffmpegGetChannelCount(JNIEnv* env, jobject thiz, jlong context) { if (!context) { LOGE("Context must be non-NULL."); return -1; @@ -220,7 +176,7 @@ AUDIO_DECODER_FUNC(jint, ffmpegGetChannelCount, jlong context) { return ((AVCodecContext*)context)->ch_layout.nb_channels; } -AUDIO_DECODER_FUNC(jint, ffmpegGetSampleRate, jlong context) { +jint ffmpegGetSampleRate(JNIEnv* env, jobject thiz, jlong context) { if (!context) { LOGE("Context must be non-NULL."); return -1; @@ -228,7 +184,8 @@ AUDIO_DECODER_FUNC(jint, ffmpegGetSampleRate, jlong context) { return ((AVCodecContext*)context)->sample_rate; } -AUDIO_DECODER_FUNC(jlong, ffmpegReset, jlong jContext, jbyteArray extraData) { +jlong ffmpegReset(JNIEnv* env, jobject thiz, jlong jContext, + jbyteArray extraData) { AVCodecContext* context = (AVCodecContext*)jContext; if (!context) { LOGE("Tried to reset without a context."); @@ -256,7 +213,7 @@ AUDIO_DECODER_FUNC(jlong, ffmpegReset, jlong jContext, jbyteArray extraData) { return (jlong)context; } -AUDIO_DECODER_FUNC(void, ffmpegRelease, jlong context) { +void ffmpegRelease(JNIEnv* env, jobject thiz, jlong context) { if (context) { releaseContext((AVCodecContext*)context); } @@ -430,3 +387,68 @@ void releaseContext(AVCodecContext* context) { } avcodec_free_context(&context); } + +jint JNI_OnLoad(JavaVM* vm, void* reserved) { + JNIEnv* env; + if (vm->GetEnv(reinterpret_cast(&env), JNI_VERSION_1_6) != JNI_OK) { + LOGE("JNI_OnLoad: GetEnv failed"); + return -1; + } + jclass clazz = + env->FindClass("androidx/media3/decoder/ffmpeg/FfmpegAudioDecoder"); + if (!clazz) { + LOGE("JNI_OnLoad: FindClass failed"); + return -1; + } + growOutputBufferMethod = + env->GetMethodID(clazz, "growOutputBuffer", + "(Landroidx/media3/decoder/" + "SimpleDecoderOutputBuffer;I)Ljava/nio/ByteBuffer;"); + if (!growOutputBufferMethod) { + LOGE("JNI_OnLoad: GetMethodID failed"); + return -1; + } + static const JNINativeMethod kFfmpegAudioDecoderMethods[] = { + {"ffmpegInitialize", "(Ljava/lang/String;[BZII)J", + reinterpret_cast(ffmpegInitialize)}, + {"ffmpegDecode", + "(JLjava/nio/ByteBuffer;ILandroidx/media3/decoder/" + "SimpleDecoderOutputBuffer;Ljava/nio/ByteBuffer;I)I", + reinterpret_cast(ffmpegDecode)}, + {"ffmpegGetChannelCount", "(J)I", + reinterpret_cast(ffmpegGetChannelCount)}, + {"ffmpegGetSampleRate", "(J)I", + reinterpret_cast(ffmpegGetSampleRate)}, + {"ffmpegReset", "(J[B)J", reinterpret_cast(ffmpegReset)}, + {"ffmpegRelease", "(J)V", reinterpret_cast(ffmpegRelease)}, + }; + if (env->RegisterNatives(clazz, kFfmpegAudioDecoderMethods, + sizeof(kFfmpegAudioDecoderMethods) / + sizeof(kFfmpegAudioDecoderMethods[0])) < 0) { + LOGE("JNI_OnLoad: RegisterNatives failed for FfmpegAudioDecoder"); + return -1; + } + + jclass libraryClazz = + env->FindClass("androidx/media3/decoder/ffmpeg/FfmpegLibrary"); + if (!libraryClazz) { + LOGE("JNI_OnLoad: FindClass failed for FfmpegLibrary"); + return -1; + } + static const JNINativeMethod kFfmpegLibraryMethods[] = { + {"ffmpegGetVersion", "()Ljava/lang/String;", + reinterpret_cast(ffmpegGetVersion)}, + {"ffmpegGetInputBufferPaddingSize", "()I", + reinterpret_cast(ffmpegGetInputBufferPaddingSize)}, + {"ffmpegHasDecoder", "(Ljava/lang/String;)Z", + reinterpret_cast(ffmpegHasDecoder)}, + }; + if (env->RegisterNatives(libraryClazz, kFfmpegLibraryMethods, + sizeof(kFfmpegLibraryMethods) / + sizeof(kFfmpegLibraryMethods[0])) < 0) { + LOGE("JNI_OnLoad: RegisterNatives failed for FfmpegLibrary"); + return -1; + } + + return JNI_VERSION_1_6; +} diff --git a/libraries/decoder_flac/README.md b/libraries/decoder_flac/README.md index d09c4ffd4f2..c19ba1c406f 100644 --- a/libraries/decoder_flac/README.md +++ b/libraries/decoder_flac/README.md @@ -22,7 +22,7 @@ your media3 check-out): ```shell cd libraries/decoder_flac/src/main/jni && \ -git clone https://github.com/xiph/flac.git --depth=1 libflac +git clone https://github.com/xiph/flac.git --branch 1.5.0 --depth=1 libflac ``` * [Install CMake][] diff --git a/libraries/decoder_flac/src/main/jni/CMakeLists.txt b/libraries/decoder_flac/src/main/jni/CMakeLists.txt index 7d6ddbaa533..6a1a1eff4b0 100644 --- a/libraries/decoder_flac/src/main/jni/CMakeLists.txt +++ b/libraries/decoder_flac/src/main/jni/CMakeLists.txt @@ -23,7 +23,7 @@ set(CMAKE_CXX_STANDARD 11) project(libflacJNI C CXX) # Suppress pointer cast alignment warnings from upstream libflac source. -set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-cast-align") +set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} -Wno-cast-align -DHAVE_FSEEKO=") set(libflac_jni_root "${CMAKE_CURRENT_SOURCE_DIR}") diff --git a/libraries/decoder_iamf/proguard-rules.txt b/libraries/decoder_iamf/proguard-rules.txt index 7b4eaa02434..1804f3f7c26 100755 --- a/libraries/decoder_iamf/proguard-rules.txt +++ b/libraries/decoder_iamf/proguard-rules.txt @@ -1,6 +1,6 @@ # Proguard rules specific to the IAMF extension. # This prevents the names of native methods from being obfuscated. --keepclasseswithmembernames class * { +-keep,includedescriptorclasses class androidx.media3.decoder.iamf.** { native ; } diff --git a/libraries/decoder_iamf/src/main/jni/CMakeLists.txt b/libraries/decoder_iamf/src/main/jni/CMakeLists.txt index 3202944efa6..1892fa869c6 100644 --- a/libraries/decoder_iamf/src/main/jni/CMakeLists.txt +++ b/libraries/decoder_iamf/src/main/jni/CMakeLists.txt @@ -38,6 +38,10 @@ add_library(iamfJNI SHARED iamf_jni.cc) +set_target_properties(iamfJNI PROPERTIES + CXX_VISIBILITY_PRESET hidden + VISIBILITY_INLINES_HIDDEN ON) + # Locate NDK log library. find_library(android_log_lib log) diff --git a/libraries/decoder_iamf/src/main/jni/iamf_jni.cc b/libraries/decoder_iamf/src/main/jni/iamf_jni.cc index b3e2cf6b304..2484242b420 100644 --- a/libraries/decoder_iamf/src/main/jni/iamf_jni.cc +++ b/libraries/decoder_iamf/src/main/jni/iamf_jni.cc @@ -38,27 +38,11 @@ #include #include -#define DECODER_FUNC(RETURN_TYPE, NAME, ...) \ - extern "C" { \ - JNIEXPORT RETURN_TYPE Java_androidx_media3_decoder_iamf_IamfDecoder_##NAME( \ - JNIEnv* env, jobject thiz, ##__VA_ARGS__); \ - } \ - JNIEXPORT RETURN_TYPE Java_androidx_media3_decoder_iamf_IamfDecoder_##NAME( \ - JNIEnv* env, jobject thiz, ##__VA_ARGS__) - namespace { constexpr int ERROR = -1; constexpr int OK = 0; } // namespace -jint JNI_OnLoad(JavaVM* vm, void* reserved) { - JNIEnv* env; - if (vm->GetEnv(reinterpret_cast(&env), JNI_VERSION_1_6) != JNI_OK) { - return ERROR; - } - return JNI_VERSION_1_6; -} - namespace { struct IamfDecoderWrapper { std::unique_ptr decoder; @@ -235,17 +219,17 @@ iamf_tools::api::IamfDecoderFactory::Settings CreateSettings( } } // namespace -DECODER_FUNC(jlong, iamfOpen) { +jlong iamfOpen(JNIEnv* env, jobject thiz) { return reinterpret_cast(new IamfDecoderWrapper{nullptr}); } -DECODER_FUNC(void, iamfClose, jlong decoderRawPointer) { +void iamfClose(JNIEnv* env, jobject thiz, jlong decoderRawPointer) { delete reinterpret_cast(decoderRawPointer); } -DECODER_FUNC(jint, iamfCreate, jint requestedOutputLayout, - jlong requestedMixPresentationId, jint outputSampleType, - jint channelOrdering, jlong decoderRawPointer) { +jint iamfCreate(JNIEnv* env, jobject thiz, jint requestedOutputLayout, + jlong requestedMixPresentationId, jint outputSampleType, + jint channelOrdering, jlong decoderRawPointer) { IamfDecoderWrapper* wrapper = reinterpret_cast(decoderRawPointer); if (!wrapper) { @@ -263,10 +247,12 @@ DECODER_FUNC(jint, iamfCreate, jint requestedOutputLayout, return OK; } -DECODER_FUNC(jint, iamfCreateFromDescriptors, - jbyteArray initializationDataArray, jint requestedOutputLayout, - jlong requestedMixPresentationId, jint outputSampleType, - jint channelOrdering, jlong decoderRawPointer) { +jint iamfCreateFromDescriptors(JNIEnv* env, jobject thiz, + jbyteArray initializationDataArray, + jint requestedOutputLayout, + jlong requestedMixPresentationId, + jint outputSampleType, jint channelOrdering, + jlong decoderRawPointer) { IamfDecoderWrapper* wrapper = reinterpret_cast(decoderRawPointer); if (!wrapper) { @@ -292,8 +278,8 @@ DECODER_FUNC(jint, iamfCreateFromDescriptors, return OK; // Success } -DECODER_FUNC(jint, iamfDecode, jobject inputBuffer, jint inputSize, - jlong decoderRawPointer) { +jint iamfDecode(JNIEnv* env, jobject thiz, jobject inputBuffer, jint inputSize, + jlong decoderRawPointer) { IamfDecoderWrapper* wrapper = reinterpret_cast(decoderRawPointer); if (!wrapper || !wrapper->decoder) { @@ -321,8 +307,8 @@ DECODER_FUNC(jint, iamfDecode, jobject inputBuffer, jint inputSize, return OK; } -DECODER_FUNC(jint, iamfGetOutputTemporalUnit, jobject outputBuffer, - jint outputSize, jlong decoderRawPointer) { +jint iamfGetOutputTemporalUnit(JNIEnv* env, jobject thiz, jobject outputBuffer, + jint outputSize, jlong decoderRawPointer) { IamfDecoderWrapper* wrapper = reinterpret_cast(decoderRawPointer); if (!wrapper || !wrapper->decoder) { @@ -350,7 +336,8 @@ DECODER_FUNC(jint, iamfGetOutputTemporalUnit, jobject outputBuffer, return bytes_written; } -DECODER_FUNC(jboolean, iamfIsTemporalUnitAvailable, jlong decoderRawPointer) { +jboolean iamfIsTemporalUnitAvailable(JNIEnv* env, jobject thiz, + jlong decoderRawPointer) { IamfDecoderWrapper* wrapper = reinterpret_cast(decoderRawPointer); if (!wrapper || !wrapper->decoder) { @@ -360,8 +347,8 @@ DECODER_FUNC(jboolean, iamfIsTemporalUnitAvailable, jlong decoderRawPointer) { return wrapper->decoder->IsTemporalUnitAvailable(); } -DECODER_FUNC(jboolean, iamfIsDescriptorProcessingComplete, - jlong decoderRawPointer) { +jboolean iamfIsDescriptorProcessingComplete(JNIEnv* env, jobject thiz, + jlong decoderRawPointer) { IamfDecoderWrapper* wrapper = reinterpret_cast(decoderRawPointer); if (!wrapper || !wrapper->decoder) { @@ -371,7 +358,8 @@ DECODER_FUNC(jboolean, iamfIsDescriptorProcessingComplete, return wrapper->decoder->IsDescriptorProcessingComplete(); } -DECODER_FUNC(jint, iamfGetNumberOfOutputChannels, jlong decoderRawPointer) { +jint iamfGetNumberOfOutputChannels(JNIEnv* env, jobject thiz, + jlong decoderRawPointer) { IamfDecoderWrapper* wrapper = reinterpret_cast(decoderRawPointer); if (!wrapper || !wrapper->decoder) { @@ -389,7 +377,8 @@ DECODER_FUNC(jint, iamfGetNumberOfOutputChannels, jlong decoderRawPointer) { return channel_count; } -DECODER_FUNC(jint, iamfGetSelectedOutputLayout, jlong decoderRawPointer) { +jint iamfGetSelectedOutputLayout(JNIEnv* env, jobject thiz, + jlong decoderRawPointer) { IamfDecoderWrapper* wrapper = reinterpret_cast(decoderRawPointer); if (!wrapper || !wrapper->decoder) { @@ -406,7 +395,8 @@ DECODER_FUNC(jint, iamfGetSelectedOutputLayout, jlong decoderRawPointer) { return FromIamfToolsOutputLayout(selected_mix.output_layout); } -DECODER_FUNC(jlong, iamfGetSelectedMixPresentationId, jlong decoderRawPointer) { +jlong iamfGetSelectedMixPresentationId(JNIEnv* env, jobject thiz, + jlong decoderRawPointer) { IamfDecoderWrapper* wrapper = reinterpret_cast(decoderRawPointer); if (!wrapper || !wrapper->decoder) { @@ -423,7 +413,8 @@ DECODER_FUNC(jlong, iamfGetSelectedMixPresentationId, jlong decoderRawPointer) { return selected_mix.mix_presentation_id; } -DECODER_FUNC(jint, iamfGetOutputSampleType, jlong decoderRawPointer) { +jint iamfGetOutputSampleType(JNIEnv* env, jobject thiz, + jlong decoderRawPointer) { IamfDecoderWrapper* wrapper = reinterpret_cast(decoderRawPointer); if (!wrapper || !wrapper->decoder) { @@ -440,7 +431,7 @@ DECODER_FUNC(jint, iamfGetOutputSampleType, jlong decoderRawPointer) { return value; } -DECODER_FUNC(jint, iamfGetSampleRate, jlong decoderRawPointer) { +jint iamfGetSampleRate(JNIEnv* env, jobject thiz, jlong decoderRawPointer) { IamfDecoderWrapper* wrapper = reinterpret_cast(decoderRawPointer); if (!wrapper || !wrapper->decoder) { @@ -457,7 +448,7 @@ DECODER_FUNC(jint, iamfGetSampleRate, jlong decoderRawPointer) { return sample_rate; } -DECODER_FUNC(jint, iamfGetFrameSize, jlong decoderRawPointer) { +jint iamfGetFrameSize(JNIEnv* env, jobject thiz, jlong decoderRawPointer) { IamfDecoderWrapper* wrapper = reinterpret_cast(decoderRawPointer); if (!wrapper || !wrapper->decoder) { @@ -474,7 +465,7 @@ DECODER_FUNC(jint, iamfGetFrameSize, jlong decoderRawPointer) { return frame_size; } -DECODER_FUNC(jint, iamfReset, jlong decoderRawPointer) { +jint iamfReset(JNIEnv* env, jobject thiz, jlong decoderRawPointer) { IamfDecoderWrapper* wrapper = reinterpret_cast(decoderRawPointer); if (!wrapper || !wrapper->decoder) { @@ -489,8 +480,9 @@ DECODER_FUNC(jint, iamfReset, jlong decoderRawPointer) { return OK; } -DECODER_FUNC(jint, iamfResetWithNewMix, jint requestedOutputLayout, - jlong requestedMixPresentationId, jlong decoderRawPointer) { +jint iamfResetWithNewMix(JNIEnv* env, jobject thiz, jint requestedOutputLayout, + jlong requestedMixPresentationId, + jlong decoderRawPointer) { IamfDecoderWrapper* wrapper = reinterpret_cast(decoderRawPointer); if (!wrapper || !wrapper->decoder) { @@ -513,7 +505,8 @@ DECODER_FUNC(jint, iamfResetWithNewMix, jint requestedOutputLayout, return OK; } -DECODER_FUNC(jint, iamfSignalEndOfDecoding, jlong decoderRawPointer) { +jint iamfSignalEndOfDecoding(JNIEnv* env, jobject thiz, + jlong decoderRawPointer) { IamfDecoderWrapper* wrapper = reinterpret_cast(decoderRawPointer); if (!wrapper || !wrapper->decoder) { @@ -527,3 +520,55 @@ DECODER_FUNC(jint, iamfSignalEndOfDecoding, jlong decoderRawPointer) { } return OK; } + +extern "C" JNIEXPORT jint JNICALL JNI_OnLoad(JavaVM* vm, void* reserved) { + JNIEnv* env; + if (vm->GetEnv(reinterpret_cast(&env), JNI_VERSION_1_6) != JNI_OK) { + return ERROR; + } + + static const JNINativeMethod kIamfDecoderMethods[] = { + {"iamfOpen", "()J", reinterpret_cast(iamfOpen)}, + {"iamfClose", "(J)V", reinterpret_cast(iamfClose)}, + {"iamfCreate", "(IIIJ)I", reinterpret_cast(iamfCreate)}, + {"iamfCreateFromDescriptors", "([BIJIIJ)I", + reinterpret_cast(iamfCreateFromDescriptors)}, + {"iamfDecode", "(Ljava/nio/ByteBuffer;IJ)I", + reinterpret_cast(iamfDecode)}, + {"iamfGetOutputTemporalUnit", "(Ljava/nio/ByteBuffer;IJ)I", + reinterpret_cast(iamfGetOutputTemporalUnit)}, + {"iamfIsTemporalUnitAvailable", "(J)Z", + reinterpret_cast(iamfIsTemporalUnitAvailable)}, + {"iamfIsDescriptorProcessingComplete", "(J)Z", + reinterpret_cast(iamfIsDescriptorProcessingComplete)}, + {"iamfGetNumberOfOutputChannels", "(J)I", + reinterpret_cast(iamfGetNumberOfOutputChannels)}, + {"iamfGetSelectedOutputLayout", "(J)I", + reinterpret_cast(iamfGetSelectedOutputLayout)}, + {"iamfGetSelectedMixPresentationId", "(J)J", + reinterpret_cast(iamfGetSelectedMixPresentationId)}, + {"iamfGetOutputSampleType", "(J)I", + reinterpret_cast(iamfGetOutputSampleType)}, + {"iamfGetSampleRate", "(J)I", reinterpret_cast(iamfGetSampleRate)}, + {"iamfGetFrameSize", "(J)I", reinterpret_cast(iamfGetFrameSize)}, + {"iamfReset", "(J)I", reinterpret_cast(iamfReset)}, + {"iamfResetWithNewMix", "(IJJ)I", + reinterpret_cast(iamfResetWithNewMix)}, + {"iamfSignalEndOfDecoding", "(J)I", + reinterpret_cast(iamfSignalEndOfDecoding)}, + }; + + jclass clazz = env->FindClass("androidx/media3/decoder/iamf/IamfDecoder"); + if (!clazz) { + LOGE("JNI_OnLoad: FindClass failed for IamfDecoder"); + return ERROR; + } + if (env->RegisterNatives( + clazz, kIamfDecoderMethods, + sizeof(kIamfDecoderMethods) / sizeof(kIamfDecoderMethods[0])) < 0) { + LOGE("JNI_OnLoad: RegisterNatives failed for IamfDecoder"); + return ERROR; + } + + return JNI_VERSION_1_6; +} diff --git a/libraries/decoder_midi/src/main/java/androidx/media3/decoder/midi/MidiDecoder.java b/libraries/decoder_midi/src/main/java/androidx/media3/decoder/midi/MidiDecoder.java index 16252c61790..bc973f53ca3 100644 --- a/libraries/decoder_midi/src/main/java/androidx/media3/decoder/midi/MidiDecoder.java +++ b/libraries/decoder_midi/src/main/java/androidx/media3/decoder/midi/MidiDecoder.java @@ -23,6 +23,7 @@ import androidx.media3.common.C; import androidx.media3.common.Format; import androidx.media3.common.MediaLibraryInfo; +import androidx.media3.common.util.UnstableApi; import androidx.media3.common.util.Util; import androidx.media3.decoder.DecoderInputBuffer; import androidx.media3.decoder.SimpleDecoder; @@ -37,7 +38,8 @@ import org.checkerframework.checker.nullness.qual.EnsuresNonNull; /** Decodes MIDI commands into PCM. */ -/* package */ final class MidiDecoder +@UnstableApi +public final class MidiDecoder extends SimpleDecoder { static { diff --git a/libraries/decoder_mpegh/src/androidTest/java/androidx/media3/decoder/mpegh/MpeghPlaybackTest.java b/libraries/decoder_mpegh/src/androidTest/java/androidx/media3/decoder/mpegh/MpeghPlaybackTest.java index fdd7b6fb2a8..33de2c92c6a 100644 --- a/libraries/decoder_mpegh/src/androidTest/java/androidx/media3/decoder/mpegh/MpeghPlaybackTest.java +++ b/libraries/decoder_mpegh/src/androidTest/java/androidx/media3/decoder/mpegh/MpeghPlaybackTest.java @@ -60,6 +60,10 @@ public final class MpeghPlaybackTest { private static final String ACTION_EVENT_XML = ""; + private static final String ACTION_EVENT_NON_ASCII_XML = + ""; private static final int MAX_PERSISTENCE_STORAGE = 8096; @Nullable private ByteBuffer persistenceBuffer; @@ -73,19 +77,19 @@ public void setUp() { } @Test - public void testPlayback() throws Exception { + public void playback_matchesExpectedDump() throws Exception { playAndAssertAudioSinkInput( SAMPLE_MHM1_URI, /* command= */ null, /* persistenceBuffer= */ null, /* run= */ 1); } @Test - public void testPlaybackWithCommand() throws Exception { + public void playback_withCommand_matchesExpectedDump() throws Exception { playAndAssertAudioSinkInput( SAMPLE_MHM1_URI, ACTION_EVENT_XML, /* persistenceBuffer= */ null, /* run= */ 1); } @Test - public void testPlaybackWithCommandAndPersistence() throws Exception { + public void playback_withCommandAndPersistence_matchesExpectedDump() throws Exception { // First run with command to populate persistence playAndAssertAudioSinkInput(SAMPLE_MHM1_URI, ACTION_EVENT_XML, persistenceBuffer, /* run= */ 1); @@ -94,9 +98,30 @@ public void testPlaybackWithCommandAndPersistence() throws Exception { SAMPLE_MHM1_URI, /* command= */ null, persistenceBuffer, /* run= */ 2); } + @Test + public void playback_withNonAsciiCommand_matchesExpectedDump() throws Exception { + playAndAssertAudioSinkInput( + SAMPLE_MHM1_URI, + ACTION_EVENT_NON_ASCII_XML, + ".nonasciicmd", + /* persistenceBuffer= */ null, + /* run= */ 1); + } + private static void playAndAssertAudioSinkInput( String fileName, @Nullable String command, @Nullable ByteBuffer persistenceBuffer, int run) throws Exception { + playAndAssertAudioSinkInput( + fileName, command, command != null ? ".cmd" : "", persistenceBuffer, run); + } + + private static void playAndAssertAudioSinkInput( + String fileName, + @Nullable String command, + String commandSuffix, + @Nullable ByteBuffer persistenceBuffer, + int run) + throws Exception { CapturingAudioSink audioSink = CapturingAudioSink.create(); TestPlaybackRunnable testPlaybackRunnable = new TestPlaybackRunnable( @@ -113,8 +138,6 @@ private static void playAndAssertAudioSinkInput( if (testPlaybackRunnable.playbackException != null) { throw testPlaybackRunnable.playbackException; } - - String commandSuffix = command != null ? ".cmd" : ""; String persistenceSuffix = persistenceBuffer != null ? ".persist." + run : ""; DumpFileAsserts.assertOutput( diff --git a/libraries/decoder_mpegh/src/main/jni/mpeghuimanager_jni.cpp b/libraries/decoder_mpegh/src/main/jni/mpeghuimanager_jni.cpp index 11fe31b5c08..ffce8aaaab8 100644 --- a/libraries/decoder_mpegh/src/main/jni/mpeghuimanager_jni.cpp +++ b/libraries/decoder_mpegh/src/main/jni/mpeghuimanager_jni.cpp @@ -170,20 +170,32 @@ UIMANAGER_FUNC(jint, destroy, jobject persistenceBuffer, * will be called to pass the received MHAS frame to the UI manager */ UIMANAGER_FUNC(jboolean, command, jstring xmlAction) { + if (xmlAction == nullptr) { + return false; + } + UIMANAGER_CONTEXT* ctx = getContext_UI(env, obj); + if (ctx == nullptr || ctx->handle == nullptr) { + LOGW("MPEG-H UI manager is not initialized."); + return false; + } const char* str = env->GetStringUTFChars(xmlAction, 0); - uint32_t strlen = env->GetStringLength(xmlAction); + if (str == nullptr) { + return false; + } + uint32_t xmlActionLength = env->GetStringUTFLength(xmlAction); unsigned int flagsOut = 0; MPEGH_UI_ERROR result = - mpegh_UI_ApplyXmlAction(ctx->handle, str, strlen, &flagsOut); + mpegh_UI_ApplyXmlAction(ctx->handle, str, xmlActionLength, &flagsOut); if (result != MPEGH_UI_OK) { LOGW("Failed to apply XML action with result %d for command %s", result, str); - return false; } - return true; + + env->ReleaseStringUTFChars(xmlAction, str); + return result == MPEGH_UI_OK; } /* diff --git a/libraries/decoder_opus/README.md b/libraries/decoder_opus/README.md index 8fa5cc08d2b..c00c0007586 100644 --- a/libraries/decoder_opus/README.md +++ b/libraries/decoder_opus/README.md @@ -22,7 +22,10 @@ your media3 check-out): ```shell cd libraries/decoder_opus/src/main/jni && \ -git clone https://github.com/xiph/opus.git --depth=1 libopus +git clone https://github.com/xiph/opus.git \ + --branch v1.6.1 \ + --depth=1 \ + libopus ``` * [Install CMake][] diff --git a/libraries/decoder_opus/proguard-rules.txt b/libraries/decoder_opus/proguard-rules.txt index 39af1402447..e962c1006c4 100644 --- a/libraries/decoder_opus/proguard-rules.txt +++ b/libraries/decoder_opus/proguard-rules.txt @@ -1,7 +1,7 @@ # Proguard rules specific to the Opus extension. # This prevents the names of native methods from being obfuscated. --keepclasseswithmembernames class * { +-keep,includedescriptorclasses class androidx.media3.decoder.opus.** { native ; } diff --git a/libraries/decoder_opus/src/androidTest/java/androidx/media3/decoder/opus/OpusDecoderTest.java b/libraries/decoder_opus/src/androidTest/java/androidx/media3/decoder/opus/OpusDecoderTest.java index c66b2486a2c..53b49c1c018 100644 --- a/libraries/decoder_opus/src/androidTest/java/androidx/media3/decoder/opus/OpusDecoderTest.java +++ b/libraries/decoder_opus/src/androidTest/java/androidx/media3/decoder/opus/OpusDecoderTest.java @@ -144,6 +144,62 @@ public void decode_withPaddingInSupplementalData_removesPadding() throws OpusDec .isEqualTo(DECODED_DATA_SIZE - nanosecondsToBytes(DISCARD_PADDING_NANOS)); } + // b/528791354 + @Test + public void decode_concurrentDecoders_sizesOutputBufferPerInstance() throws OpusDecoderException { + assertThat(OpusLibrary.isAvailable()).isTrue(); + OpusDecoder decoderStereo = + new OpusDecoder( + /* numInputBuffers= */ 0, + /* numOutputBuffers= */ 0, + /* initialInputBufferSize= */ 0, + createInitializationData(/* preSkipNanos= */ 0), + /* cryptoConfig= */ null, + /* outputFloat= */ false); + byte[] headerMono = HEADER.clone(); + headerMono[9] = 1; + OpusDecoder decoderMono = + new OpusDecoder( + /* numInputBuffers= */ 0, + /* numOutputBuffers= */ 0, + /* initialInputBufferSize= */ 0, + ImmutableList.of(headerMono, CUSTOM_PRE_SKIP_BYTES, CUSTOM_SEEK_PRE_ROLL_BYTES), + /* cryptoConfig= */ null, + /* outputFloat= */ false); + + DecoderInputBuffer input = + createInputBuffer(decoderStereo, ENCODED_DATA, /* supplementalData= */ null); + SimpleDecoderOutputBuffer output = decoderStereo.createOutputBuffer(); + + assertThat(decoderStereo.decode(input, output, false)).isNull(); + assertThat(output.data.capacity()).isEqualTo(960 * 6 * 2 * 2); + decoderStereo.release(); + decoderMono.release(); + } + + @Test + public void decode_invalidInputData_returnsDecoderException() throws OpusDecoderException { + assertThat(OpusLibrary.isAvailable()).isTrue(); + OpusDecoder decoder = + new OpusDecoder( + /* numInputBuffers= */ 0, + /* numOutputBuffers= */ 0, + /* initialInputBufferSize= */ 0, + createInitializationData(/* preSkipNanos= */ 0), + /* cryptoConfig= */ null, + /* outputFloat= */ false); + // TOC byte 0x03 indicates code 3 (arbitrary frame count), which requires a second byte. + DecoderInputBuffer input = + createInputBuffer(decoder, new byte[] {0x03}, /* supplementalData= */ null); + SimpleDecoderOutputBuffer output = decoder.createOutputBuffer(); + + OpusDecoderException exception = decoder.decode(input, output, false); + + assertThat(exception).isNotNull(); + assertThat(exception).hasMessageThat().contains("Decode error:"); + decoder.release(); + } + private static long sampleCountToNanoseconds(long sampleCount) { return (sampleCount * C.NANOS_PER_SECOND) / OpusDecoder.SAMPLE_RATE; } diff --git a/libraries/decoder_opus/src/main/java/androidx/media3/decoder/opus/OpusDecoder.java b/libraries/decoder_opus/src/main/java/androidx/media3/decoder/opus/OpusDecoder.java index 051b594e4c5..b41d2b39a4d 100644 --- a/libraries/decoder_opus/src/main/java/androidx/media3/decoder/opus/OpusDecoder.java +++ b/libraries/decoder_opus/src/main/java/androidx/media3/decoder/opus/OpusDecoder.java @@ -139,7 +139,7 @@ public OpusDecoder( this.outputFloat = outputFloat; if (outputFloat) { - opusSetFloatOutput(); + opusSetFloatOutput(nativeDecoderContext); } } @@ -204,7 +204,8 @@ protected OpusDecoderException decode( new CryptoException(opusGetErrorCode(nativeDecoderContext), message); return new OpusDecoderException(message, cause); } else { - return new OpusDecoderException("Decode error: " + opusGetErrorMessage(result)); + return new OpusDecoderException( + "Decode error: " + opusGetErrorMessage(nativeDecoderContext)); } } @@ -351,5 +352,5 @@ private native int opusSecureDecode( private native String opusGetErrorMessage(long decoder); - private native void opusSetFloatOutput(); + private native void opusSetFloatOutput(long decoder); } diff --git a/libraries/decoder_opus/src/main/jni/CMakeLists.txt b/libraries/decoder_opus/src/main/jni/CMakeLists.txt index da055e7e2ea..eff49530bb9 100644 --- a/libraries/decoder_opus/src/main/jni/CMakeLists.txt +++ b/libraries/decoder_opus/src/main/jni/CMakeLists.txt @@ -28,6 +28,9 @@ set(libopus_jni_root "${CMAKE_CURRENT_SOURCE_DIR}") add_subdirectory("${libopus_jni_root}/libopus" EXCLUDE_FROM_ALL) +set_target_properties(opus PROPERTIES + C_VISIBILITY_PRESET hidden) + # Add the include directory from libopus. include_directories ("${libopus_jni_root}/libopus/include") @@ -36,6 +39,10 @@ add_library(opusV2JNI SHARED opus_jni.cc) +set_target_properties(opusV2JNI PROPERTIES + CXX_VISIBILITY_PRESET hidden + VISIBILITY_INLINES_HIDDEN ON) + # Locate NDK log library. find_library(android_log_lib log) diff --git a/libraries/decoder_opus/src/main/jni/opus_jni.cc b/libraries/decoder_opus/src/main/jni/opus_jni.cc index ce127d19ab2..af1cde08000 100644 --- a/libraries/decoder_opus/src/main/jni/opus_jni.cc +++ b/libraries/decoder_opus/src/main/jni/opus_jni.cc @@ -35,46 +35,24 @@ } while (0) #endif // __ANDROID__ -#define DECODER_FUNC(RETURN_TYPE, NAME, ...) \ - extern "C" { \ - JNIEXPORT RETURN_TYPE Java_androidx_media3_decoder_opus_OpusDecoder_##NAME( \ - JNIEnv* env, jobject thiz, ##__VA_ARGS__); \ - } \ - JNIEXPORT RETURN_TYPE Java_androidx_media3_decoder_opus_OpusDecoder_##NAME( \ - JNIEnv* env, jobject thiz, ##__VA_ARGS__) - -#define LIBRARY_FUNC(RETURN_TYPE, NAME, ...) \ - extern "C" { \ - JNIEXPORT RETURN_TYPE Java_androidx_media3_decoder_opus_OpusLibrary_##NAME( \ - JNIEnv* env, jobject thiz, ##__VA_ARGS__); \ - } \ - JNIEXPORT RETURN_TYPE Java_androidx_media3_decoder_opus_OpusLibrary_##NAME( \ - JNIEnv* env, jobject thiz, ##__VA_ARGS__) - // JNI references for SimpleOutputBuffer class. static jmethodID outputBufferInit; -jint JNI_OnLoad(JavaVM* vm, void* reserved) { - JNIEnv* env; - if (vm->GetEnv(reinterpret_cast(&env), JNI_VERSION_1_6) != JNI_OK) { - return -1; - } - return JNI_VERSION_1_6; -} - static const int kBytesPerIntPcmSample = 2; static const int kBytesPerFloatSample = 4; static const int kMaxOpusOutputPacketSizeSamples = 960 * 6; -static int channelCount; -static int errorCode; -static bool outputFloat = false; -DECODER_FUNC(jlong, opusInit, jint sampleRate, jint channelCount, - jint numStreams, jint numCoupled, jint gain, - jbyteArray jStreamMap) { +struct OpusDecoderContext { + OpusMSDecoder* decoder = nullptr; + int channelCount = 0; + bool outputFloat = false; + int errorCode = 0; +}; + +jlong opusInit(JNIEnv* env, jobject thiz, jint sampleRate, jint channelCount, + jint numStreams, jint numCoupled, jint gain, + jbyteArray jStreamMap) { int status = OPUS_INVALID_STATE; - ::channelCount = channelCount; - errorCode = 0; jbyte* streamMapBytes = env->GetByteArrayElements(jStreamMap, 0); uint8_t* streamMap = reinterpret_cast(streamMapBytes); OpusMSDecoder* decoder = opus_multistream_decoder_create( @@ -87,6 +65,7 @@ DECODER_FUNC(jlong, opusInit, jint sampleRate, jint channelCount, status = opus_multistream_decoder_ctl(decoder, OPUS_SET_GAIN(gain)); if (status != OPUS_OK) { LOGE("Failed to set Opus header gain; status=%s", opus_strerror(status)); + opus_multistream_decoder_destroy(decoder); return 0; } @@ -96,19 +75,26 @@ DECODER_FUNC(jlong, opusInit, jint sampleRate, jint channelCount, outputBufferInit = env->GetMethodID(outputBufferClass, "init", "(JI)Ljava/nio/ByteBuffer;"); - return reinterpret_cast(decoder); + OpusDecoderContext* context = new OpusDecoderContext(); + context->decoder = decoder; + context->channelCount = channelCount; + context->outputFloat = false; + context->errorCode = 0; + + return reinterpret_cast(context); } -DECODER_FUNC(jint, opusDecode, jlong jDecoder, jlong jTimeUs, - jobject jInputBuffer, jint inputSize, jobject jOutputBuffer) { - OpusMSDecoder* decoder = reinterpret_cast(jDecoder); +jint opusDecode(JNIEnv* env, jobject thiz, jlong jContext, jlong jTimeUs, + jobject jInputBuffer, jint inputSize, jobject jOutputBuffer) { + OpusDecoderContext* context = reinterpret_cast(jContext); + OpusMSDecoder* decoder = context->decoder; const uint8_t* inputBuffer = reinterpret_cast( env->GetDirectBufferAddress(jInputBuffer)); const int byteSizePerSample = - outputFloat ? kBytesPerFloatSample : kBytesPerIntPcmSample; - const jint outputSize = - kMaxOpusOutputPacketSizeSamples * byteSizePerSample * channelCount; + context->outputFloat ? kBytesPerFloatSample : kBytesPerIntPcmSample; + const jint outputSize = kMaxOpusOutputPacketSizeSamples * byteSizePerSample * + context->channelCount; env->CallObjectMethod(jOutputBuffer, outputBufferInit, jTimeUs, outputSize); if (env->ExceptionCheck()) { @@ -123,7 +109,7 @@ DECODER_FUNC(jint, opusDecode, jlong jDecoder, jlong jTimeUs, } int sampleCount; - if (outputFloat) { + if (context->outputFloat) { float* outputBufferData = reinterpret_cast( env->GetDirectBufferAddress(jOutputBufferData)); sampleCount = opus_multistream_decode_float( @@ -138,16 +124,19 @@ DECODER_FUNC(jint, opusDecode, jlong jDecoder, jlong jTimeUs, } // record error code - errorCode = (sampleCount < 0) ? sampleCount : 0; - return (sampleCount < 0) ? sampleCount - : sampleCount * byteSizePerSample * channelCount; + context->errorCode = (sampleCount < 0) ? sampleCount : 0; + return (sampleCount < 0) + ? sampleCount + : sampleCount * byteSizePerSample * context->channelCount; } -DECODER_FUNC(jint, opusSecureDecode, jlong jDecoder, jlong jTimeUs, - jobject jInputBuffer, jint inputSize, jobject jOutputBuffer, - jint sampleRate, jobject mediaCrypto, jint inputMode, - jbyteArray key, jbyteArray javaIv, jint inputNumSubSamples, - jintArray numBytesOfClearData, jintArray numBytesOfEncryptedData) { +jint opusSecureDecode(JNIEnv* env, jobject thiz, jlong jContext, jlong jTimeUs, + jobject jInputBuffer, jint inputSize, + jobject jOutputBuffer, jint sampleRate, + jobject mediaCrypto, jint inputMode, jbyteArray key, + jbyteArray javaIv, jint inputNumSubSamples, + jintArray numBytesOfClearData, + jintArray numBytesOfEncryptedData) { // Doesn't support // Java client should have checked vpxSupportSecureDecode // and avoid calling this @@ -155,29 +144,108 @@ DECODER_FUNC(jint, opusSecureDecode, jlong jDecoder, jlong jTimeUs, return -2; } -DECODER_FUNC(void, opusClose, jlong jDecoder) { - OpusMSDecoder* decoder = reinterpret_cast(jDecoder); - opus_multistream_decoder_destroy(decoder); +void opusClose(JNIEnv* env, jobject thiz, jlong jContext) { + OpusDecoderContext* context = reinterpret_cast(jContext); + if (context) { + if (context->decoder) { + opus_multistream_decoder_destroy(context->decoder); + } + delete context; + } } -DECODER_FUNC(void, opusReset, jlong jDecoder) { - OpusMSDecoder* decoder = reinterpret_cast(jDecoder); - opus_multistream_decoder_ctl(decoder, OPUS_RESET_STATE); +void opusReset(JNIEnv* env, jobject thiz, jlong jContext) { + OpusDecoderContext* context = reinterpret_cast(jContext); + if (context && context->decoder) { + opus_multistream_decoder_ctl(context->decoder, OPUS_RESET_STATE); + } } -DECODER_FUNC(jstring, opusGetErrorMessage, jlong jContext) { +jstring opusGetErrorMessage(JNIEnv* env, jobject thiz, jlong jContext) { + OpusDecoderContext* context = reinterpret_cast(jContext); + int errorCode = context ? context->errorCode : OPUS_INVALID_STATE; return env->NewStringUTF(opus_strerror(errorCode)); } -DECODER_FUNC(jint, opusGetErrorCode, jlong jContext) { return errorCode; } +jint opusGetErrorCode(JNIEnv* env, jobject thiz, jlong jContext) { + OpusDecoderContext* context = reinterpret_cast(jContext); + return context ? context->errorCode : OPUS_INVALID_STATE; +} -DECODER_FUNC(void, opusSetFloatOutput) { outputFloat = true; } +void opusSetFloatOutput(JNIEnv* env, jobject thiz, jlong jContext) { + OpusDecoderContext* context = reinterpret_cast(jContext); + if (context) { + context->outputFloat = true; + } +} -LIBRARY_FUNC(jstring, opusIsSecureDecodeSupported) { +jboolean opusIsSecureDecodeSupported(JNIEnv* env, jobject thiz) { // Doesn't support - return 0; + return JNI_FALSE; } -LIBRARY_FUNC(jstring, opusGetVersion) { +jstring opusGetVersion(JNIEnv* env, jobject thiz) { return env->NewStringUTF(opus_get_version_string()); } + +extern "C" JNIEXPORT jint JNICALL JNI_OnLoad(JavaVM* vm, void* reserved) { + JNIEnv* env; + if (vm->GetEnv(reinterpret_cast(&env), JNI_VERSION_1_6) != JNI_OK) { + return -1; + } + + static const JNINativeMethod kOpusDecoderMethods[] = { + {"opusInit", "(IIIII[B)J", reinterpret_cast(opusInit)}, + {"opusClose", "(J)V", reinterpret_cast(opusClose)}, + {"opusReset", "(J)V", reinterpret_cast(opusReset)}, + {"opusDecode", + "(JJLjava/nio/ByteBuffer;ILandroidx/media3/decoder/" + "SimpleDecoderOutputBuffer;)I", + reinterpret_cast(opusDecode)}, + {"opusSecureDecode", + "(JJLjava/nio/ByteBuffer;ILandroidx/media3/decoder/" + "SimpleDecoderOutputBuffer;ILandroidx/media3/decoder/" + "CryptoConfig;I[B[BI[I[I)I", + reinterpret_cast(opusSecureDecode)}, + {"opusGetErrorMessage", "(J)Ljava/lang/String;", + reinterpret_cast(opusGetErrorMessage)}, + {"opusGetErrorCode", "(J)I", reinterpret_cast(opusGetErrorCode)}, + {"opusSetFloatOutput", "(J)V", + reinterpret_cast(opusSetFloatOutput)}, + }; + + static const JNINativeMethod kOpusLibraryMethods[] = { + {"opusGetVersion", "()Ljava/lang/String;", + reinterpret_cast(opusGetVersion)}, + {"opusIsSecureDecodeSupported", "()Z", + reinterpret_cast(opusIsSecureDecodeSupported)}, + }; + + jclass decoderClazz = + env->FindClass("androidx/media3/decoder/opus/OpusDecoder"); + if (!decoderClazz) { + LOGE("JNI_OnLoad: FindClass failed for OpusDecoder"); + return -1; + } + if (env->RegisterNatives( + decoderClazz, kOpusDecoderMethods, + sizeof(kOpusDecoderMethods) / sizeof(kOpusDecoderMethods[0])) < 0) { + LOGE("JNI_OnLoad: RegisterNatives failed for OpusDecoder"); + return -1; + } + + jclass libraryClazz = + env->FindClass("androidx/media3/decoder/opus/OpusLibrary"); + if (!libraryClazz) { + LOGE("JNI_OnLoad: FindClass failed for OpusLibrary"); + return -1; + } + if (env->RegisterNatives( + libraryClazz, kOpusLibraryMethods, + sizeof(kOpusLibraryMethods) / sizeof(kOpusLibraryMethods[0])) < 0) { + LOGE("JNI_OnLoad: RegisterNatives failed for OpusLibrary"); + return -1; + } + + return JNI_VERSION_1_6; +} diff --git a/libraries/decoder_vp9/proguard-rules.txt b/libraries/decoder_vp9/proguard-rules.txt index 7e0800afdc9..128c5c5ebb5 100644 --- a/libraries/decoder_vp9/proguard-rules.txt +++ b/libraries/decoder_vp9/proguard-rules.txt @@ -1,7 +1,7 @@ # Proguard rules specific to the VP9 extension. # This prevents the names of native methods from being obfuscated. --keepclasseswithmembernames class * { +-keep,includedescriptorclasses class androidx.media3.decoder.vp9.** { native ; } diff --git a/libraries/decoder_vp9/src/main/jni/Android.mk b/libraries/decoder_vp9/src/main/jni/Android.mk index 6d9455e3ad4..563c4d8a858 100644 --- a/libraries/decoder_vp9/src/main/jni/Android.mk +++ b/libraries/decoder_vp9/src/main/jni/Android.mk @@ -32,6 +32,7 @@ LOCAL_SRC_FILES := vpx_jni.cc LOCAL_LDLIBS := -llog -lz -lm -landroid LOCAL_SHARED_LIBRARIES := libvpx LOCAL_STATIC_LIBRARIES := cpufeatures +LOCAL_CFLAGS += -fvisibility=hidden include $(BUILD_SHARED_LIBRARY) diff --git a/libraries/decoder_vp9/src/main/jni/vpx_jni.cc b/libraries/decoder_vp9/src/main/jni/vpx_jni.cc index e983e7ff8bb..670419679e5 100644 --- a/libraries/decoder_vp9/src/main/jni/vpx_jni.cc +++ b/libraries/decoder_vp9/src/main/jni/vpx_jni.cc @@ -38,22 +38,6 @@ #define LOGE(...) \ ((void)__android_log_print(ANDROID_LOG_ERROR, LOG_TAG, __VA_ARGS__)) -#define DECODER_FUNC(RETURN_TYPE, NAME, ...) \ - extern "C" { \ - JNIEXPORT RETURN_TYPE Java_androidx_media3_decoder_vp9_VpxDecoder_##NAME( \ - JNIEnv* env, jobject thiz, ##__VA_ARGS__); \ - } \ - JNIEXPORT RETURN_TYPE Java_androidx_media3_decoder_vp9_VpxDecoder_##NAME( \ - JNIEnv* env, jobject thiz, ##__VA_ARGS__) - -#define LIBRARY_FUNC(RETURN_TYPE, NAME, ...) \ - extern "C" { \ - JNIEXPORT RETURN_TYPE Java_androidx_media3_decoder_vp9_VpxLibrary_##NAME( \ - JNIEnv* env, jobject thiz, ##__VA_ARGS__); \ - } \ - JNIEXPORT RETURN_TYPE Java_androidx_media3_decoder_vp9_VpxLibrary_##NAME( \ - JNIEnv* env, jobject thiz, ##__VA_ARGS__) - // JNI references for VideoDecoderOutputBuffer class. static jmethodID initForYuvFrame; static jmethodID initForPrivateFrame; @@ -68,14 +52,6 @@ static const int kDecoderPrivateBase = 0x100; static int errorCode; -jint JNI_OnLoad(JavaVM* vm, void* reserved) { - JNIEnv* env; - if (vm->GetEnv(reinterpret_cast(&env), JNI_VERSION_1_6) != JNI_OK) { - return -1; - } - return JNI_VERSION_1_6; -} - #ifdef __ARM_NEON__ static int convert_16_to_8_neon(const vpx_image_t* const img, jbyte* const data, const int32_t uvHeight, const int32_t yLength, @@ -432,8 +408,8 @@ int vpx_release_frame_buffer(void* priv, vpx_codec_frame_buffer_t* fb) { return buffer_manager->release(*(int*)fb->priv); } -DECODER_FUNC(jlong, vpxInit, jboolean disableLoopFilter, - jboolean enableRowMultiThreadMode, jint threads) { +jlong vpxInit(JNIEnv* env, jobject thiz, jboolean disableLoopFilter, + jboolean enableRowMultiThreadMode, jint threads) { JniCtx* context = new JniCtx(); context->decoder = new vpx_codec_ctx_t(); vpx_codec_dec_cfg_t cfg = {0, 0, 0}; @@ -488,7 +464,8 @@ DECODER_FUNC(jlong, vpxInit, jboolean disableLoopFilter, return reinterpret_cast(context); } -DECODER_FUNC(jlong, vpxDecode, jlong jContext, jobject encoded, jint len) { +jlong vpxDecode(JNIEnv* env, jobject thiz, jlong jContext, jobject encoded, + jint len) { JniCtx* const context = reinterpret_cast(jContext); const uint8_t* const buffer = reinterpret_cast(env->GetDirectBufferAddress(encoded)); @@ -503,10 +480,11 @@ DECODER_FUNC(jlong, vpxDecode, jlong jContext, jobject encoded, jint len) { return 0; } -DECODER_FUNC(jlong, vpxSecureDecode, jlong jContext, jobject encoded, jint len, - jobject mediaCrypto, jint inputMode, jbyteArray&, jbyteArray&, - jint inputNumSubSamples, jintArray numBytesOfClearData, - jintArray numBytesOfEncryptedData) { +jlong vpxSecureDecode(JNIEnv* env, jobject thiz, jlong jContext, + jobject encoded, jint len, jobject mediaCrypto, + jint inputMode, jbyteArray&, jbyteArray&, + jint inputNumSubSamples, jintArray numBytesOfClearData, + jintArray numBytesOfEncryptedData) { // Doesn't support // Java client should have checked vpxSupportSecureDecode // and avoid calling this @@ -514,14 +492,15 @@ DECODER_FUNC(jlong, vpxSecureDecode, jlong jContext, jobject encoded, jint len, return -2; } -DECODER_FUNC(jlong, vpxClose, jlong jContext) { +jlong vpxClose(JNIEnv* env, jobject thiz, jlong jContext) { JniCtx* const context = reinterpret_cast(jContext); vpx_codec_destroy(context->decoder); delete context; return 0; } -DECODER_FUNC(jint, vpxGetFrame, jlong jContext, jobject jOutputBuffer) { +jint vpxGetFrame(JNIEnv* env, jobject thiz, jlong jContext, + jobject jOutputBuffer) { JniCtx* const context = reinterpret_cast(jContext); vpx_codec_iter_t iter = NULL; const vpx_image_t* const img = vpx_codec_get_frame(context->decoder, &iter); @@ -621,8 +600,8 @@ DECODER_FUNC(jint, vpxGetFrame, jlong jContext, jobject jOutputBuffer) { return 0; } -DECODER_FUNC(jint, vpxRenderFrame, jlong jContext, jobject jSurface, - jobject jOutputBuffer) { +jint vpxRenderFrame(JNIEnv* env, jobject thiz, jlong jContext, jobject jSurface, + jobject jOutputBuffer) { JniCtx* const context = reinterpret_cast(jContext); const int id = env->GetLongField(jOutputBuffer, decoderPrivateField) - kDecoderPrivateBase; @@ -677,30 +656,101 @@ DECODER_FUNC(jint, vpxRenderFrame, jlong jContext, jobject jSurface, return ANativeWindow_unlockAndPost(context->native_window); } -DECODER_FUNC(void, vpxReleaseFrame, jlong jContext, jobject jOutputBuffer) { +int vpxReleaseFrame(JNIEnv* env, jobject thiz, jlong jContext, + jobject jOutputBuffer) { JniCtx* const context = reinterpret_cast(jContext); const int id = env->GetLongField(jOutputBuffer, decoderPrivateField) - kDecoderPrivateBase; env->SetLongField(jOutputBuffer, decoderPrivateField, -1); context->buffer_manager->release(id); + return 0; } -DECODER_FUNC(jstring, vpxGetErrorMessage, jlong jContext) { +jstring vpxGetErrorMessage(JNIEnv* env, jobject thiz, jlong jContext) { JniCtx* const context = reinterpret_cast(jContext); return env->NewStringUTF(vpx_codec_error(context->decoder)); } -DECODER_FUNC(jint, vpxGetErrorCode, jlong jContext) { return errorCode; } +jint vpxGetErrorCode(JNIEnv* env, jobject thiz, jlong jContext) { + return errorCode; +} -LIBRARY_FUNC(jstring, vpxIsSecureDecodeSupported) { +jboolean vpxIsSecureDecodeSupported(JNIEnv* env, jobject thiz) { // Doesn't support - return 0; + return JNI_FALSE; } -LIBRARY_FUNC(jstring, vpxGetVersion) { +jstring vpxGetVersion(JNIEnv* env, jobject thiz) { return env->NewStringUTF(vpx_codec_version_str()); } -LIBRARY_FUNC(jstring, vpxGetBuildConfig) { +jstring vpxGetBuildConfig(JNIEnv* env, jobject thiz) { return env->NewStringUTF(vpx_codec_build_config()); } + +extern "C" JNIEXPORT jint JNICALL JNI_OnLoad(JavaVM* vm, void* reserved) { + JNIEnv* env; + if (vm->GetEnv(reinterpret_cast(&env), JNI_VERSION_1_6) != JNI_OK) { + return -1; + } + + static const JNINativeMethod kVpxDecoderMethods[] = { + {"vpxInit", "(ZZI)J", reinterpret_cast(vpxInit)}, + {"vpxClose", "(J)J", reinterpret_cast(vpxClose)}, + {"vpxDecode", "(JLjava/nio/ByteBuffer;I)J", + reinterpret_cast(vpxDecode)}, + {"vpxSecureDecode", + "(JLjava/nio/ByteBuffer;ILandroidx/media3/decoder/CryptoConfig;I[B[BI[I" + "[I)J", + reinterpret_cast(vpxSecureDecode)}, + {"vpxGetFrame", "(JLandroidx/media3/decoder/VideoDecoderOutputBuffer;)I", + reinterpret_cast(vpxGetFrame)}, + {"vpxRenderFrame", + "(JLandroid/view/Surface;Landroidx/media3/decoder/" + "VideoDecoderOutputBuffer;)I", + reinterpret_cast(vpxRenderFrame)}, + {"vpxReleaseFrame", + "(JLandroidx/media3/decoder/VideoDecoderOutputBuffer;)I", + reinterpret_cast(vpxReleaseFrame)}, + {"vpxGetErrorCode", "(J)I", reinterpret_cast(vpxGetErrorCode)}, + {"vpxGetErrorMessage", "(J)Ljava/lang/String;", + reinterpret_cast(vpxGetErrorMessage)}, + }; + + static const JNINativeMethod kVpxLibraryMethods[] = { + {"vpxGetVersion", "()Ljava/lang/String;", + reinterpret_cast(vpxGetVersion)}, + {"vpxGetBuildConfig", "()Ljava/lang/String;", + reinterpret_cast(vpxGetBuildConfig)}, + {"vpxIsSecureDecodeSupported", "()Z", + reinterpret_cast(vpxIsSecureDecodeSupported)}, + }; + + jclass decoderClazz = + env->FindClass("androidx/media3/decoder/vp9/VpxDecoder"); + if (!decoderClazz) { + LOGE("JNI_OnLoad: FindClass failed for VpxDecoder"); + return -1; + } + if (env->RegisterNatives( + decoderClazz, kVpxDecoderMethods, + sizeof(kVpxDecoderMethods) / sizeof(kVpxDecoderMethods[0])) < 0) { + LOGE("JNI_OnLoad: RegisterNatives failed for VpxDecoder"); + return -1; + } + + jclass libraryClazz = + env->FindClass("androidx/media3/decoder/vp9/VpxLibrary"); + if (!libraryClazz) { + LOGE("JNI_OnLoad: FindClass failed for VpxLibrary"); + return -1; + } + if (env->RegisterNatives( + libraryClazz, kVpxLibraryMethods, + sizeof(kVpxLibraryMethods) / sizeof(kVpxLibraryMethods[0])) < 0) { + LOGE("JNI_OnLoad: RegisterNatives failed for VpxLibrary"); + return -1; + } + + return JNI_VERSION_1_6; +} diff --git a/libraries/effect/build.gradle.kts b/libraries/effect/build.gradle.kts index 5700a2b0a74..a119796a7ef 100644 --- a/libraries/effect/build.gradle.kts +++ b/libraries/effect/build.gradle.kts @@ -12,6 +12,8 @@ // See the License for the specific language governing permissions and // limitations under the License. +import androidx.media3.buildlogic.StripGlslCommentsTask + plugins { id("media3.android-library") id("media3.publish") @@ -20,12 +22,40 @@ plugins { android { namespace = "androidx.media3.effect" + defaultConfig { + externalNativeBuild { + cmake { + arguments.add("-DANDROID_WEAK_API_DEFS=ON") + arguments.add("-Werror=unguarded-availability") + targets.add("media3EffectJNI") + } + } + } + externalNativeBuild { cmake { path = file("src/main/jni/CMakeLists.txt") } } + sourceSets { getByName("androidTest").assets.directories.add("../test_data/src/test/assets/") getByName("test").assets.directories.add("../test_data/src/test/assets/") } } +// Packages the GLSL shaders in src/main/res/raw without comments, to reduce APK size. +androidComponents { + onVariants { variant -> + val stripGlslComments = + tasks.register( + "strip${variant.name.replaceFirstChar { it.uppercase() }}GlslComments" + ) { + shaders.from(fileTree("src/main/res/raw") { include("*.glsl") }) + } + // Generated resources override the resources with the same name in src/main/res. + variant.sources.res?.addGeneratedSourceDirectory( + stripGlslComments, + StripGlslCommentsTask::outputDirectory, + ) + } +} + dependencies { implementation(libs.androidx.annotation) implementation(libs.androidx.concurrent.futures) @@ -40,7 +70,6 @@ dependencies { testImplementation(libs.kotlinx.coroutines.test) testImplementation(libs.androidx.runner) androidTestImplementation(project(":lib-exoplayer")) - androidTestImplementation(project(":lib-effect-ndk")) androidTestImplementation(libs.kotlinx.coroutines.android) androidTestImplementation(libs.test.parameter.injector) androidTestImplementation(libs.junit) diff --git a/libraries/effect/proguard-rules.txt b/libraries/effect/proguard-rules.txt new file mode 100644 index 00000000000..fbe5f87ee8b --- /dev/null +++ b/libraries/effect/proguard-rules.txt @@ -0,0 +1,9 @@ +# Proguard rules specific to the effect module. + +# This prevents the class name and native methods registered in JNI_OnLoad from +# being shrunk or obfuscated when HardwareBufferJni is used, while allowing +# HardwareBufferJni to be shrunk when unused. +-keepnames class androidx.media3.effect.HardwareBufferJni +-keepclassmembers,includedescriptorclasses class androidx.media3.effect.HardwareBufferJni { + native ; +} diff --git a/libraries/effect/src/androidTest/java/androidx/media3/effect/AlphaScaleShaderProgramPixelTest.java b/libraries/effect/src/androidTest/java/androidx/media3/effect/AlphaScaleShaderProgramPixelTest.java index 6c7bf7c6ac2..f36c422f98c 100644 --- a/libraries/effect/src/androidTest/java/androidx/media3/effect/AlphaScaleShaderProgramPixelTest.java +++ b/libraries/effect/src/androidTest/java/androidx/media3/effect/AlphaScaleShaderProgramPixelTest.java @@ -15,26 +15,33 @@ */ package androidx.media3.effect; +import static androidx.media3.effect.EffectsTestUtil.createFocusedEglContextWithFallback; import static androidx.media3.test.utils.BitmapPixelTestUtil.MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE; +import static androidx.media3.test.utils.BitmapPixelTestUtil.MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE_DIFFERENT_DEVICE_FP16; +import static androidx.media3.test.utils.BitmapPixelTestUtil.createFp16BitmapFromFocusedGlFramebuffer; import static androidx.media3.test.utils.BitmapPixelTestUtil.createGlTextureFromBitmap; import static androidx.media3.test.utils.BitmapPixelTestUtil.createUnpremultipliedArgb8888BitmapFromFocusedGlFramebuffer; import static androidx.media3.test.utils.BitmapPixelTestUtil.getBitmapAveragePixelAbsoluteDifferenceArgb8888; +import static androidx.media3.test.utils.BitmapPixelTestUtil.getBitmapAveragePixelAbsoluteDifferenceFp16; import static androidx.media3.test.utils.BitmapPixelTestUtil.maybeSaveTestBitmap; import static androidx.media3.test.utils.BitmapPixelTestUtil.readBitmapUnpremultipliedAlpha; import static androidx.test.core.app.ApplicationProvider.getApplicationContext; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assume.assumeTrue; import android.content.Context; import android.graphics.Bitmap; import android.opengl.EGLContext; import android.opengl.EGLDisplay; import android.opengl.EGLSurface; +import android.util.Pair; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.GlUtil; import androidx.media3.common.util.Size; import androidx.media3.test.utils.BitmapPixelTestUtil; import androidx.test.ext.junit.runners.AndroidJUnit4; +import androidx.test.filters.SdkSuppress; import java.io.IOException; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; import org.junit.After; @@ -65,6 +72,8 @@ public final class AlphaScaleShaderProgramPixelTest { "test-generated-goldens/sample_mp4_first_frame/electrical_colors/increase_alpha.png"; private static final String ZERO_ALPHA_PNG_ASSET_PATH = "test-generated-goldens/sample_mp4_first_frame/electrical_colors/zero_alpha.png"; + private static final String ORIGINAL_HLG10_PNG_ASSET_PATH = + "test-generated-goldens/sample_mp4_first_frame/electrical_colors/original_hlg10.png"; private final Context context = getApplicationContext(); @@ -80,8 +89,11 @@ public final class AlphaScaleShaderProgramPixelTest { @Before public void createGlObjects() throws IOException, GlUtil.GlException { eglDisplay = GlUtil.getDefaultEglDisplay(); - eglContext = GlUtil.createEglContext(eglDisplay); - placeholderEglSurface = GlUtil.createFocusedPlaceholderEglSurface(eglContext, eglDisplay); + Pair eglContextAndSurface = + createFocusedEglContextWithFallback( + checkNotNull(eglDisplay), GlUtil.EGL_CONFIG_ATTRIBUTES_RGBA_8888); + eglContext = eglContextAndSurface.first; + placeholderEglSurface = eglContextAndSurface.second; Bitmap inputBitmap = readBitmapUnpremultipliedAlpha(ORIGINAL_PNG_ASSET_PATH); inputWidth = inputBitmap.getWidth(); @@ -111,7 +123,12 @@ public void release() throws GlUtil.GlException, VideoFrameProcessingException { if (alphaScaleShaderProgram != null) { alphaScaleShaderProgram.release(); } - GlUtil.destroyEglContext(eglDisplay, eglContext); + if (eglDisplay != null && placeholderEglSurface != null) { + GlUtil.destroyEglSurface(eglDisplay, placeholderEglSurface); + } + if (eglDisplay != null && eglContext != null) { + GlUtil.destroyEglContext(eglDisplay, eglContext); + } } @Test @@ -194,4 +211,48 @@ public void increaseAlpha_matchesGoldenFile() throws Exception { getBitmapAveragePixelAbsoluteDifferenceArgb8888(expectedBitmap, actualBitmap, testId); assertThat(averagePixelAbsoluteDifference).isAtMost(MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE); } + + @Test + @SdkSuppress(minSdkVersion = 33) + public void noOpAlpha_withHdrHlgInput_matchesGoldenFile() throws Exception { + assumeTrue("Device does not support OpenGL ES 3.0", GlUtil.getContextMajorVersion() >= 3); + + Bitmap hlgBitmap = readBitmapUnpremultipliedAlpha(ORIGINAL_HLG10_PNG_ASSET_PATH); + int hlgWidth = hlgBitmap.getWidth(); + int hlgHeight = hlgBitmap.getHeight(); + int hlgTexId = createGlTextureFromBitmap(hlgBitmap); + + int outputTexId = + GlUtil.createTexture(hlgWidth, hlgHeight, /* useHighPrecisionColorComponents= */ true); + int frameBuffer = GlUtil.createFboForTexture(outputTexId); + GlUtil.focusFramebuffer( + checkNotNull(eglDisplay), + checkNotNull(eglContext), + checkNotNull(placeholderEglSurface), + frameBuffer, + hlgWidth, + hlgHeight); + GlUtil.clearFocusedBuffers(); + + alphaScaleShaderProgram = + (AlphaScaleShaderProgram) + new AlphaScale(/* alphaScale= */ 1.0f).toGlShaderProgram(context, /* useHdr= */ true); + Size outputSize = alphaScaleShaderProgram.configure(hlgWidth, hlgHeight); + + alphaScaleShaderProgram.drawFrame(hlgTexId, /* presentationTimeUs= */ 0); + + Bitmap actualBitmap = + createFp16BitmapFromFocusedGlFramebuffer(outputSize.getWidth(), outputSize.getHeight()); + actualBitmap.setColorSpace(checkNotNull(hlgBitmap.getColorSpace())); + + maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual", actualBitmap, /* path= */ null); + float averagePixelAbsoluteDifference = + getBitmapAveragePixelAbsoluteDifferenceFp16(hlgBitmap, actualBitmap); + assertThat(averagePixelAbsoluteDifference) + .isAtMost(MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE_DIFFERENT_DEVICE_FP16); + + GlUtil.deleteTexture(hlgTexId); + GlUtil.deleteTexture(outputTexId); + GlUtil.deleteFbo(frameBuffer); + } } diff --git a/libraries/effect/src/androidTest/java/androidx/media3/effect/BitmapToHardwareBufferProcessorTest.java b/libraries/effect/src/androidTest/java/androidx/media3/effect/BitmapToHardwareBufferProcessorTest.java deleted file mode 100644 index 7d1c94328c7..00000000000 --- a/libraries/effect/src/androidTest/java/androidx/media3/effect/BitmapToHardwareBufferProcessorTest.java +++ /dev/null @@ -1,406 +0,0 @@ -/* - * Copyright 2026 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package androidx.media3.effect; - -import static com.google.common.truth.Truth.assertThat; -import static com.google.common.util.concurrent.MoreExecutors.directExecutor; -import static java.util.concurrent.TimeUnit.MILLISECONDS; -import static org.junit.Assert.assertThrows; - -import android.graphics.Bitmap; -import android.graphics.Bitmap.Config; -import android.graphics.Color; -import android.graphics.ColorSpace; -import android.graphics.Matrix; -import android.hardware.HardwareBuffer; -import android.opengl.EGLContext; -import android.opengl.EGLDisplay; -import android.opengl.EGLSurface; -import android.opengl.GLES20; -import android.os.Build; -import androidx.media3.common.util.GlUtil; -import androidx.media3.effect.ndk.HardwareBufferJni; -import androidx.media3.test.utils.BitmapPixelTestUtil; -import androidx.test.filters.SdkSuppress; -import com.google.testing.junit.testparameterinjector.TestParameter; -import com.google.testing.junit.testparameterinjector.TestParameterInjector; -import java.io.IOException; -import java.util.ArrayList; -import java.util.List; -import java.util.concurrent.CountDownLatch; -import java.util.concurrent.ExecutionException; -import java.util.concurrent.ExecutorService; -import java.util.concurrent.Executors; -import java.util.concurrent.TimeoutException; -import org.checkerframework.checker.nullness.qual.MonotonicNonNull; -import org.junit.After; -import org.junit.Before; -import org.junit.Rule; -import org.junit.Test; -import org.junit.rules.TestName; -import org.junit.runner.RunWith; - -/** Instrumentation tests for {@link BitmapToHardwareBufferProcessor}. */ -@RunWith(TestParameterInjector.class) -@SdkSuppress(minSdkVersion = 26) -public final class BitmapToHardwareBufferProcessorTest { - - // TODO: b/475511702 - Update with HDR bitmap formats once supported. - private enum BitmapType { - HARDWARE, - ARGB_8888, - } - - private static final float MAX_AVG_PIXEL_DIFFERENCE = 1.0f; - private static final String INPUT_PATH = "media/png/first_frame_1920x1080.png"; - private static final long TEST_TIMEOUT_MS = 1000L; - - private @MonotonicNonNull ExecutorService executorService; - private @MonotonicNonNull BitmapToHardwareBufferProcessor processor; - private @MonotonicNonNull List resourcesToClose; - - @Rule public final TestName testName = new TestName(); - - @Before - public void setUp() { - executorService = Executors.newSingleThreadExecutor(); - processor = - new BitmapToHardwareBufferProcessor( - HardwareBufferJni.INSTANCE, - /* internalExecutor= */ executorService, - /* errorExecutor= */ directExecutor(), - /* errorCallback= */ (e) -> { - throw new AssertionError(e); - }); - resourcesToClose = new ArrayList<>(); - } - - @After - public void tearDown() throws Exception { - if (processor != null) { - processor.close(); - } - if (executorService != null) { - executorService.shutdown(); - } - for (AutoCloseable resource : resourcesToClose) { - resource.close(); - } - } - - @Test - public void process_validBitmap_copiesPixelsCorrectly(@TestParameter BitmapType bitmapType) - throws Exception { - Bitmap inputBitmap = readBitmap(bitmapType); - // The pixel comparison can only run on ARGB_8888 bitmaps, if the input is HARDWARE, copy it to - // a ARGB_8888 for the assertion. - Bitmap expectedBitmap = - bitmapType == BitmapType.ARGB_8888 - ? inputBitmap - : inputBitmap.copy(Config.ARGB_8888, /* isMutable= */ false); - HardwareBufferFrame inputFrame = - new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, directExecutor(), /* releaseCallback= */ (fence) -> {}) - .setInternalFrame(inputBitmap) - .build(); - - HardwareBufferFrame outputFrame = processor.process(inputFrame); - - assertThat(outputFrame.hardwareBuffer).isNotNull(); - HardwareBuffer hardwareBuffer = outputFrame.hardwareBuffer; - - assertThat(hardwareBuffer.getWidth()).isEqualTo(inputBitmap.getWidth()); - assertThat(hardwareBuffer.getHeight()).isEqualTo(inputBitmap.getHeight()); - - Bitmap outputBitmap = readBitmapFromHardwareBuffer(hardwareBuffer); - - assertThat( - BitmapPixelTestUtil.getBitmapAveragePixelAbsoluteDifferenceArgb8888( - expectedBitmap, outputBitmap, testName.getMethodName())) - .isLessThan(MAX_AVG_PIXEL_DIFFERENCE); - - outputFrame.release(/* releaseFence= */ null); - } - - @Test - public void process_recycledBitmap_throwsIllegalStateException() throws Exception { - Bitmap inputBitmap = BitmapPixelTestUtil.readBitmap(INPUT_PATH); - HardwareBufferFrame inputFrame = - new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, directExecutor(), /* releaseCallback= */ (fence) -> {}) - .setInternalFrame(inputBitmap) - .build(); - - inputBitmap.recycle(); - - assertThrows(IllegalStateException.class, () -> processor.process(inputFrame)); - } - - @Test - public void process_alreadyHasHardwareBuffer_returnsOriginalFrame() { - try (HardwareBuffer hardwareBuffer = createHardwareBuffer()) { - HardwareBufferFrame inputFrame = - new HardwareBufferFrame.Builder( - hardwareBuffer, directExecutor(), /* releaseCallback= */ (fence) -> {}) - .build(); - - HardwareBufferFrame outputFrame = processor.process(inputFrame); - - assertThat(outputFrame).isSameInstanceAs(inputFrame); - - outputFrame.release(/* releaseFence= */ null); - } - } - - @Test - public void process_notABitmap_returnsOriginalFrame() { - HardwareBufferFrame inputFrame = - new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, directExecutor(), /* releaseCallback= */ (fence) -> {}) - .setInternalFrame(new Object()) - .build(); - - HardwareBufferFrame outputFrame = processor.process(inputFrame); - - assertThat(outputFrame).isSameInstanceAs(inputFrame); - - outputFrame.release(/* releaseFence= */ null); - } - - @Test - public void process_repeatedBitmap_reusesSameBuffer(@TestParameter BitmapType bitmapType) - throws IOException { - Bitmap inputBitmap = readBitmap(bitmapType); - HardwareBufferFrame inputFrame1 = createBitmapFrame(inputBitmap); - HardwareBufferFrame inputFrame2 = createBitmapFrame(inputBitmap); - - HardwareBufferFrame outputFrame1 = processor.process(inputFrame1); - HardwareBufferFrame outputFrame2 = processor.process(inputFrame2); - - assertThat(outputFrame1.hardwareBuffer).isSameInstanceAs(outputFrame2.hardwareBuffer); - assertThat(outputFrame1.hardwareBuffer.isClosed()).isFalse(); - - outputFrame1.release(/* releaseFence= */ null); - outputFrame2.release(/* releaseFence= */ null); - } - - @Test - public void process_repeatedBitmapAfterRelease_reusesSameBuffer( - @TestParameter BitmapType bitmapType) throws IOException { - Bitmap inputBitmap = readBitmap(bitmapType); - HardwareBufferFrame inputFrame1 = createBitmapFrame(inputBitmap); - HardwareBufferFrame inputFrame2 = createBitmapFrame(inputBitmap); - - HardwareBufferFrame outputFrame1 = processor.process(inputFrame1); - outputFrame1.release(/* releaseFence= */ null); - HardwareBufferFrame outputFrame2 = processor.process(inputFrame2); - - assertThat(outputFrame1.hardwareBuffer).isSameInstanceAs(outputFrame2.hardwareBuffer); - assertThat(outputFrame1.hardwareBuffer.isClosed()).isFalse(); - - outputFrame2.release(/* releaseFence= */ null); - } - - @Test - public void process_differentBitmap_createsNewBufferAndRemovesReferenceToOldBuffer( - @TestParameter BitmapType bitmapType) - throws IOException, ExecutionException, InterruptedException, TimeoutException { - Bitmap bitmap1 = readBitmap(bitmapType); - Bitmap bitmap2 = readBitmap(bitmapType); - - HardwareBufferFrame outputFrame1 = processor.process(createBitmapFrame(bitmap1)); - HardwareBufferFrame outputFrame2 = processor.process(createBitmapFrame(bitmap2)); - - assertThat(outputFrame1.hardwareBuffer).isNotSameInstanceAs(outputFrame2.hardwareBuffer); - - // Processor released its hold on buffer1 when bitmap2 was converted. - // buffer1 is still held by outputFrame1. - assertThat(outputFrame1.hardwareBuffer.isClosed()).isFalse(); - - outputFrame1.release(/* releaseFence= */ null); - // Ensure there are no pending tasks left on the executor. - executorService.submit(() -> {}).get(TEST_TIMEOUT_MS, MILLISECONDS); - - assertThat(outputFrame1.hardwareBuffer.isClosed()).isTrue(); - - outputFrame2.release(/* releaseFence= */ null); - } - - @Test - public void process_differentGenerationId_createsNewBuffer() { - // Cannot be tested with Config.Hardware because hardware backed bitmaps are immutable. - Bitmap bitmap = Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888); - - HardwareBufferFrame outputFrame1 = processor.process(createBitmapFrame(bitmap)); - bitmap.eraseColor(Color.RED); - HardwareBufferFrame outputFrame2 = processor.process(createBitmapFrame(bitmap)); - - assertThat(outputFrame1.hardwareBuffer).isNotSameInstanceAs(outputFrame2.hardwareBuffer); - - outputFrame1.release(/* releaseFence= */ null); - outputFrame2.release(/* releaseFence= */ null); - } - - @Test - public void process_releasesInputFrame() throws Exception { - CountDownLatch releasedLatch = new CountDownLatch(1); - HardwareBufferFrame inputFrame = - new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, - directExecutor(), - /* releaseCallback= */ (fence) -> releasedLatch.countDown()) - .setInternalFrame(Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888)) - .build(); - - HardwareBufferFrame outputFrame = processor.process(inputFrame); - - assertThat(releasedLatch.await(TEST_TIMEOUT_MS, MILLISECONDS)).isTrue(); - - outputFrame.release(/* releaseFence= */ null); - } - - @Test - public void releaseOutputFrame_sharedBuffer_doesNotCloseSharedBuffer( - @TestParameter BitmapType bitmapType) throws Exception { - Bitmap inputBitmap = readBitmap(bitmapType); - HardwareBufferFrame inputFrame1 = createBitmapFrame(inputBitmap); - HardwareBufferFrame inputFrame2 = createBitmapFrame(inputBitmap); - - HardwareBufferFrame outputFrame1 = processor.process(inputFrame1); - HardwareBufferFrame outputFrame2 = processor.process(inputFrame2); - - outputFrame1.release(/* releaseFence= */ null); - // Ensure there are no pending tasks left on the executor. - executorService.submit(() -> {}).get(TEST_TIMEOUT_MS, MILLISECONDS); - - assertThat(outputFrame1.hardwareBuffer.isClosed()).isFalse(); - assertThat(outputFrame2.hardwareBuffer.isClosed()).isFalse(); - - outputFrame2.release(/* releaseFence= */ null); - // Ensure there are no pending tasks left on the executor. - executorService.submit(() -> {}).get(TEST_TIMEOUT_MS, MILLISECONDS); - - assertThat(outputFrame1.hardwareBuffer.isClosed()).isFalse(); - assertThat(outputFrame2.hardwareBuffer.isClosed()).isFalse(); - } - - @Test - public void close_shutsDownInternalExecutor() { - Bitmap bitmap = Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888); - HardwareBufferFrame outputFrame = processor.process(createBitmapFrame(bitmap)); - outputFrame.release(/* releaseFence= */ null); - - processor.close(); - - assertThat(executorService.isShutdown()).isTrue(); - } - - @Test - public void releaseOutputFrame_afterClose_closesBuffer() throws IOException { - Bitmap inputBitmap = BitmapPixelTestUtil.readBitmap(INPUT_PATH); - HardwareBufferFrame outputFrame = processor.process(createBitmapFrame(inputBitmap)); - HardwareBuffer hardwareBuffer = outputFrame.hardwareBuffer; - - processor.close(); - - // Processor released its reference, but the frame still holds one. - assertThat(hardwareBuffer.isClosed()).isFalse(); - - outputFrame.release(/* releaseFence= */ null); - - // This will run synchronously because the internal executor is shutdown. - assertThat(hardwareBuffer.isClosed()).isTrue(); - } - - @Test - public void process_afterClose_throwsIllegalStateException() { - Bitmap bitmap = Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888); - - processor.close(); - - assertThrows(IllegalStateException.class, () -> processor.process(createBitmapFrame(bitmap))); - } - - private Bitmap readBitmapFromHardwareBuffer(HardwareBuffer hardwareBuffer) throws Exception { - if (Build.VERSION.SDK_INT >= 29) { - Bitmap hardwareBitmap = - Bitmap.wrapHardwareBuffer(hardwareBuffer, ColorSpace.get(ColorSpace.Named.SRGB)); - assertThat(hardwareBitmap).isNotNull(); - return hardwareBitmap.copy(Bitmap.Config.ARGB_8888, false); - } - - int width = hardwareBuffer.getWidth(); - int height = hardwareBuffer.getHeight(); - EGLDisplay eglDisplay = GlUtil.getDefaultEglDisplay(); - EGLContext eglContext = GlUtil.createEglContext(eglDisplay); - EGLSurface eglSurface = GlUtil.createFocusedPlaceholderEglSurface(eglContext, eglDisplay); - - int texId = GlUtil.createTexture(width, height, false); - long eglImage = - HardwareBufferJni.INSTANCE.nativeCreateEglImageFromHardwareBuffer( - eglDisplay.getNativeHandle(), hardwareBuffer); - assertThat(eglImage).isNotEqualTo(0L); - - GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, texId); - assertThat(HardwareBufferJni.INSTANCE.nativeBindEGLImage(GLES20.GL_TEXTURE_2D, eglImage)) - .isTrue(); - - int fboId = GlUtil.createFboForTexture(texId); - GlUtil.focusFramebufferUsingCurrentContext(fboId, width, height); - Bitmap bitmap = BitmapPixelTestUtil.createArgb8888BitmapFromFocusedGlFramebuffer(width, height); - - GLES20.glDeleteTextures(1, new int[] {texId}, 0); - assertThat( - HardwareBufferJni.INSTANCE.nativeDestroyEGLImage( - eglDisplay.getNativeHandle(), eglImage)) - .isTrue(); - GlUtil.destroyEglSurface(eglDisplay, eglSurface); - GlUtil.destroyEglContext(eglDisplay, eglContext); - - // OpenGL returns an upside down bitmap due to coordinate system differences, flip it before - // returning. - Matrix matrix = new Matrix(); - matrix.postScale(1, -1, bitmap.getWidth() / 2f, bitmap.getHeight() / 2f); - return Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(), bitmap.getHeight(), matrix, true); - } - - private static Bitmap readBitmap(BitmapType bitmapType) throws IOException { - switch (bitmapType) { - case ARGB_8888: - return BitmapPixelTestUtil.readBitmap(INPUT_PATH); - case HARDWARE: - return BitmapPixelTestUtil.readBitmap(INPUT_PATH) - .copy(Config.HARDWARE, /* isMutable= */ false); - } - throw new IllegalArgumentException(); - } - - private static HardwareBufferFrame createBitmapFrame(Bitmap bitmap) { - return new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, directExecutor(), /* releaseCallback= */ (fence) -> {}) - .setInternalFrame(bitmap) - .build(); - } - - private static HardwareBuffer createHardwareBuffer() { - return HardwareBuffer.create( - 10, - 10, - /* format= */ HardwareBuffer.RGBA_8888, - /* layers= */ 1, - /* usage= */ HardwareBuffer.USAGE_CPU_WRITE_OFTEN | HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE); - } -} diff --git a/libraries/effect/src/androidTest/java/androidx/media3/effect/ColorConversionShaderProgramPixelTest.java b/libraries/effect/src/androidTest/java/androidx/media3/effect/ColorConversionShaderProgramPixelTest.java new file mode 100644 index 00000000000..3b17d0aced6 --- /dev/null +++ b/libraries/effect/src/androidTest/java/androidx/media3/effect/ColorConversionShaderProgramPixelTest.java @@ -0,0 +1,285 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect; + +import static androidx.media3.effect.DefaultGlFrameProcessor.BT2020_HLG; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT2020_LINEAR; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT709_LINEAR; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT709_SRGB; +import static androidx.media3.effect.EffectsTestUtil.createFocusedEglContextWithFallback; +import static androidx.media3.test.utils.BitmapPixelTestUtil.createArgb8888BitmapWithSolidColor; +import static androidx.media3.test.utils.BitmapPixelTestUtil.createGlTextureFromBitmap; +import static androidx.media3.test.utils.BitmapPixelTestUtil.createUnpremultipliedArgb8888BitmapFromFocusedGlFramebuffer; +import static androidx.media3.test.utils.TestUtil.PSNR_THRESHOLD; +import static androidx.media3.test.utils.TestUtil.assertBitmapsAreSimilar; +import static androidx.test.core.app.ApplicationProvider.getApplicationContext; +import static com.google.common.base.Preconditions.checkNotNull; +import static org.junit.Assume.assumeTrue; + +import android.content.Context; +import android.graphics.Bitmap; +import android.graphics.Color; +import android.opengl.EGLContext; +import android.opengl.EGLDisplay; +import android.opengl.EGLSurface; +import android.util.Pair; +import androidx.media3.common.C; +import androidx.media3.common.ColorInfo; +import androidx.media3.common.util.GlUtil; +import androidx.media3.common.util.Size; +import androidx.test.filters.SdkSuppress; +import com.google.common.collect.ImmutableList; +import com.google.testing.junit.testparameterinjector.TestParameter; +import com.google.testing.junit.testparameterinjector.TestParameterInjector; +import com.google.testing.junit.testparameterinjector.TestParameterValuesProvider; +import org.checkerframework.checker.nullness.qual.MonotonicNonNull; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Pixel tests for {@link ColorConversionShaderProgram}. */ +@RunWith(TestParameterInjector.class) +@SdkSuppress(minSdkVersion = 26) +public final class ColorConversionShaderProgramPixelTest { + + private static final int WIDTH = 16; + private static final int HEIGHT = 16; + + private static final ColorInfo HDR_PQ = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorTransfer(C.COLOR_TRANSFER_ST2084) + .build(); + + private static final ImmutableList CONVERSION_TEST_CASES = + ImmutableList.of( + // 1.0 in the linear HDR working space is the BT.2408 203-nit diffuse white reference, + // which BT.2408 places at the 75% HLG signal level: round(0.75 * 255) = 191. + new ConversionTestCase( + "hdrLinearInputAndHlgOutput_mapsDiffuseWhiteToBt2408SignalLevel", + /* inputColor= */ Color.WHITE, + /* inputColorInfo= */ BT2020_LINEAR, + /* outputColorInfo= */ BT2020_HLG, + /* expectedColor= */ Color.rgb(191, 191, 191)), + // SDR 1.0 is anchored at 203 nits, the same as HDR 1.0, so this matches + // hdrLinearInputAndHlgOutput_mapsDiffuseWhiteToBt2408SignalLevel. + new ConversionTestCase( + "sdrLinearInputAndHlgOutput_mapsReferenceWhiteToBt2408SignalLevel", + /* inputColor= */ Color.WHITE, + /* inputColorInfo= */ BT709_LINEAR, + /* outputColorInfo= */ BT2020_HLG, + /* expectedColor= */ Color.rgb(191, 191, 191)), + // srgbEotf(1.0) = 1.0, so this matches + // sdrLinearInputAndHlgOutput_mapsReferenceWhiteToBt2408SignalLevel. + new ConversionTestCase( + "srgbInputAndHlgOutput_mapsReferenceWhiteToBt2408SignalLevel", + /* inputColor= */ Color.WHITE, + /* inputColorInfo= */ BT709_SRGB, + /* outputColorInfo= */ BT2020_HLG, + /* expectedColor= */ Color.rgb(191, 191, 191)), + // Tone mapping maps the 1,000-nit HDR peak to the 203.15-nit SDR peak white, with the + // BT.2446 Method C inflection point at 118.9 nits (80% of the SDR electrical range). + // Diffuse white (203.1521 nits) is above the 118.9-nit inflection point and rolls off to + // 154.26 nits: 154.2593 / 203.1521 = 0.7593 of SDR display light, srgbOetf(0.759329) = + // 0.8857, and round(0.8857 * 255) = 226. + new ConversionTestCase( + "hdrLinearInputAndSrgbOutput_toneMapsDiffuseWhite", + /* inputColor= */ Color.WHITE, + /* inputColorInfo= */ BT2020_LINEAR, + /* outputColorInfo= */ BT709_SRGB, + /* expectedColor= */ Color.rgb(226, 226, 226)), + // 8-bit HLG 191 / 255 = 0.7490 decodes to 201.9393 nits (~1.0 in the linear HDR working + // space), which BT.2446 Method C rolls off to 154.00 nits: 153.9984 / 203.1521 = 0.7580, + // srgbOetf(0.758045) = 0.8850, and round(0.8850 * 255) = 226. + new ConversionTestCase( + "hlgInputAndSrgbOutput_toneMapsDiffuseWhite", + /* inputColor= */ Color.rgb(191, 191, 191), + /* inputColorInfo= */ BT2020_HLG, + /* outputColorInfo= */ BT709_SRGB, + /* expectedColor= */ Color.rgb(226, 226, 226)), + // 128 / 255 = 0.502 of linear light. srgbOetf(0.502) = 0.7366, and + // round(0.7366 * 255) = 188. An unconverted output would be 128. + new ConversionTestCase( + "sdrLinearInputAndSrgbOutput_appliesSrgbOetf", + /* inputColor= */ Color.rgb(128, 128, 128), + /* inputColorInfo= */ BT709_LINEAR, + /* outputColorInfo= */ BT709_SRGB, + /* expectedColor= */ Color.rgb(188, 188, 188)), + // An arbitrary color that is not a fixed point of any transfer function, so that a stray + // decode or encode would change it. + new ConversionTestCase( + "srgbInputAndSrgbOutput_passesThrough", + /* inputColor= */ Color.rgb(37, 128, 219), + /* inputColorInfo= */ BT709_SRGB, + /* outputColorInfo= */ BT709_SRGB, + /* expectedColor= */ Color.rgb(37, 128, 219)), + // A saturated primary, unlike the neutral colors above, is only correct if the gamut + // matrices are applied in the right order and orientation. BT.709 red is inside the + // BT.2020 gamut, so the conversion is not clipped. + // hlgOetf(hlgInverseOotf(BT709_TO_BT2020 * (1, 0, 0) * hlgEotf(0.75))) = + // (0.7087, 0.2666, 0.1299), scaled to 8 bits. + new ConversionTestCase( + "sdrLinearInputAndHlgOutput_convertsGamut", + /* inputColor= */ Color.RED, + /* inputColorInfo= */ BT709_LINEAR, + /* outputColorInfo= */ BT2020_HLG, + /* expectedColor= */ Color.rgb(181, 68, 33)), + // The transfer functions guard against non-positive inputs, so black also checks that + // those guards do not leak a NaN or a floor value into the output. + new ConversionTestCase( + "hdrLinearInputAndHlgOutput_preservesBlack", + /* inputColor= */ Color.BLACK, + /* inputColorInfo= */ BT2020_LINEAR, + /* outputColorInfo= */ BT2020_HLG, + /* expectedColor= */ Color.BLACK), + // An unsupported output color transfer outputs magenta as an error color. + new ConversionTestCase( + "unsupportedOutputTransfer_outputsErrorColor", + /* inputColor= */ Color.WHITE, + /* inputColorInfo= */ BT2020_LINEAR, + /* outputColorInfo= */ HDR_PQ, + /* expectedColor= */ Color.MAGENTA)); + + private final Context context = getApplicationContext(); + + private @MonotonicNonNull EGLDisplay eglDisplay; + private @MonotonicNonNull EGLContext eglContext; + private @MonotonicNonNull EGLSurface placeholderEglSurface; + private @MonotonicNonNull ColorConversionShaderProgram colorConversionShaderProgram; + private int inputTexId; + private int outputTexId; + private int frameBuffer; + + @Before + public void createGlObjects() throws Exception { + eglDisplay = GlUtil.getDefaultEglDisplay(); + Pair eglContextAndSurface = + createFocusedEglContextWithFallback( + checkNotNull(eglDisplay), GlUtil.EGL_CONFIG_ATTRIBUTES_RGBA_8888); + eglContext = eglContextAndSurface.first; + placeholderEglSurface = eglContextAndSurface.second; + + // All four conversions output electrical values in [0, 1], so an 8-bit output is enough. + outputTexId = GlUtil.createTexture(WIDTH, HEIGHT, /* useHighPrecisionColorComponents= */ false); + frameBuffer = GlUtil.createFboForTexture(outputTexId); + GlUtil.focusFramebuffer( + checkNotNull(eglDisplay), + checkNotNull(eglContext), + checkNotNull(placeholderEglSurface), + frameBuffer, + WIDTH, + HEIGHT); + GlUtil.clearFocusedBuffers(); + } + + @After + public void release() throws Exception { + if (colorConversionShaderProgram != null) { + colorConversionShaderProgram.release(); + } + if (inputTexId != 0) { + GlUtil.deleteTexture(inputTexId); + } + if (outputTexId != 0) { + GlUtil.deleteFbo(frameBuffer); + GlUtil.deleteTexture(outputTexId); + } + if (eglDisplay != null && placeholderEglSurface != null) { + GlUtil.destroyEglSurface(eglDisplay, placeholderEglSurface); + } + if (eglDisplay != null && eglContext != null) { + GlUtil.destroyEglContext(eglDisplay, eglContext); + } + } + + @Test + public void drawFrame_outputsExpectedColor( + @TestParameter(valuesProvider = ConversionTestCasesProvider.class) + ConversionTestCase testCase) + throws Exception { + assumeTrue("Device does not support OpenGL ES 3.0", GlUtil.getContextMajorVersion() >= 3); + + Bitmap actualBitmap = + drawFrame(testCase.inputColor, testCase.inputColorInfo, testCase.outputColorInfo); + + Bitmap expectedBitmap = + createArgb8888BitmapWithSolidColor(WIDTH, HEIGHT, testCase.expectedColor); + assertBitmapsAreSimilar(expectedBitmap, actualBitmap, PSNR_THRESHOLD); + } + + /** + * Runs the color conversion over a frame of a single solid color, and returns the output frame. + * + *

The bitmap only carries the 8-bit values. They are uploaded to the texture unchanged, and + * the shader interprets them as {@code inputColorInfo}. + * + * @param inputColor The color of every pixel of the input frame, scaled to 8 bits, expressed in + * {@code inputColorInfo}. + * @param inputColorInfo The working {@link ColorInfo} the input frame is in. + * @param outputColorInfo The {@link ColorInfo} to convert to. + * @return The output {@link Bitmap}. + * @throws Exception If an error occurs during frame processing. + */ + private Bitmap drawFrame(int inputColor, ColorInfo inputColorInfo, ColorInfo outputColorInfo) + throws Exception { + Bitmap inputBitmap = Bitmap.createBitmap(WIDTH, HEIGHT, Bitmap.Config.ARGB_8888); + inputBitmap.eraseColor(inputColor); + inputTexId = createGlTextureFromBitmap(inputBitmap); + + colorConversionShaderProgram = + new ColorConversionShaderProgram(context, inputColorInfo, outputColorInfo); + Size outputSize = colorConversionShaderProgram.configure(WIDTH, HEIGHT); + colorConversionShaderProgram.drawFrame(inputTexId, /* presentationTimeUs= */ 0); + + return createUnpremultipliedArgb8888BitmapFromFocusedGlFramebuffer( + outputSize.getWidth(), outputSize.getHeight()); + } + + private static final class ConversionTestCase { + private final String name; + private final int inputColor; + private final ColorInfo inputColorInfo; + private final ColorInfo outputColorInfo; + private final int expectedColor; + + private ConversionTestCase( + String name, + int inputColor, + ColorInfo inputColorInfo, + ColorInfo outputColorInfo, + int expectedColor) { + this.name = name; + this.inputColor = inputColor; + this.inputColorInfo = inputColorInfo; + this.outputColorInfo = outputColorInfo; + this.expectedColor = expectedColor; + } + + @Override + public String toString() { + return name; + } + } + + private static final class ConversionTestCasesProvider extends TestParameterValuesProvider { + @Override + protected ImmutableList provideValues( + TestParameterValuesProvider.Context context) { + return CONVERSION_TEST_CASES; + } + } +} diff --git a/libraries/effect/src/androidTest/java/androidx/media3/effect/DefaultGlTextureFrameCompositorPixelTest.java b/libraries/effect/src/androidTest/java/androidx/media3/effect/DefaultGlTextureFrameCompositorPixelTest.java new file mode 100644 index 00000000000..8b2b408edb7 --- /dev/null +++ b/libraries/effect/src/androidTest/java/androidx/media3/effect/DefaultGlTextureFrameCompositorPixelTest.java @@ -0,0 +1,405 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect; + +import static androidx.media3.effect.DefaultGlFrameProcessor.BT2020_HLG; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT709_SRGB; +import static androidx.media3.effect.DefaultGlFrameProcessor.KEY_COMPOSITION_SEQUENCE_INDEX; +import static androidx.media3.effect.DefaultGlFrameProcessor.KEY_COMPOSITOR_SETTINGS; +import static androidx.media3.effect.EffectsTestUtil.createFocusedEglContextWithFallback; +import static androidx.media3.test.utils.BitmapPixelTestUtil.createFp16BitmapFromFocusedGlFramebuffer; +import static androidx.media3.test.utils.BitmapPixelTestUtil.createGlTextureFromBitmap; +import static androidx.media3.test.utils.BitmapPixelTestUtil.createUnpremultipliedArgb8888BitmapFromFocusedGlFramebuffer; +import static androidx.media3.test.utils.BitmapPixelTestUtil.maybeSaveTestBitmap; +import static androidx.media3.test.utils.BitmapPixelTestUtil.readBitmapUnpremultipliedAlpha; +import static androidx.media3.test.utils.TestUtil.PSNR_THRESHOLD; +import static androidx.media3.test.utils.TestUtil.assertBitmapsAreSimilar; +import static androidx.test.core.app.ApplicationProvider.getApplicationContext; +import static com.google.common.base.Preconditions.checkNotNull; +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.util.concurrent.MoreExecutors.directExecutor; +import static org.junit.Assume.assumeTrue; + +import android.content.Context; +import android.graphics.Bitmap; +import android.opengl.EGLContext; +import android.opengl.EGLDisplay; +import android.opengl.EGLSurface; +import android.util.Pair; +import androidx.media3.common.C; +import androidx.media3.common.Format; +import androidx.media3.common.GlObjectsProvider; +import androidx.media3.common.GlTextureInfo; +import androidx.media3.common.OverlaySettings; +import androidx.media3.common.VideoCompositorSettings; +import androidx.media3.common.util.GlUtil; +import androidx.media3.common.util.Size; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import androidx.test.filters.SdkSuppress; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.Executor; +import java.util.concurrent.atomic.AtomicReference; +import org.checkerframework.checker.nullness.qual.MonotonicNonNull; +import org.junit.After; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TestName; +import org.junit.runner.RunWith; + +/** Pixel tests for {@link DefaultGlTextureFrameCompositor}. */ +@RunWith(AndroidJUnit4.class) +public final class DefaultGlTextureFrameCompositorPixelTest { + + private static final String ORIGINAL_HLG10_PNG_ASSET_PATH = + "test-generated-goldens/sample_mp4_first_frame/electrical_colors/original_hlg10.png"; + private static final String ORIGINAL_SDR_PNG_ASSET_PATH = + "test-generated-goldens/sample_mp4_first_frame/electrical_colors/original.png"; + + @Rule public final TestName testName = new TestName(); + + private final Context context = getApplicationContext(); + private final List texturesToDelete = new ArrayList<>(); + private final List frameBuffersToDelete = new ArrayList<>(); + + private String testId; + private @MonotonicNonNull EGLDisplay eglDisplay; + private @MonotonicNonNull EGLContext eglContext; + private @MonotonicNonNull EGLSurface placeholderEglSurface; + private @MonotonicNonNull GlObjectsProvider glObjectsProvider; + + @Before + public void setUp() throws Exception { + testId = testName.getMethodName(); + eglDisplay = GlUtil.getDefaultEglDisplay(); + glObjectsProvider = new DefaultGlObjectsProvider(); + Pair eglContextAndSurface = + createFocusedEglContextWithFallback( + checkNotNull(eglDisplay), GlUtil.EGL_CONFIG_ATTRIBUTES_RGBA_8888); + eglContext = eglContextAndSurface.first; + placeholderEglSurface = eglContextAndSurface.second; + } + + @After + public void tearDown() throws Exception { + for (int texId : texturesToDelete) { + GlUtil.deleteTexture(texId); + } + for (int frameBuffer : frameBuffersToDelete) { + GlUtil.deleteFbo(frameBuffer); + } + if (eglDisplay != null && placeholderEglSurface != null) { + GlUtil.destroyEglSurface(eglDisplay, placeholderEglSurface); + } + if (eglDisplay != null && eglContext != null) { + GlUtil.destroyEglContext(eglDisplay, eglContext); + } + } + + @Test + @SdkSuppress(minSdkVersion = 33) + public void composite_singleHdrSequence_passesThroughHdrFrame() throws Exception { + Format hlgFormat = new Format.Builder().setColorInfo(BT2020_HLG).build(); + assumeTrue("Device does not support OpenGL ES 3.0", GlUtil.getContextMajorVersion() >= 3); + + Bitmap hlgBitmap = readBitmapUnpremultipliedAlpha(ORIGINAL_HLG10_PNG_ASSET_PATH); + int width = hlgBitmap.getWidth(); + int height = hlgBitmap.getHeight(); + int texId = createGlTextureFromBitmap(hlgBitmap); + texturesToDelete.add(texId); + GlTextureInfo textureInfo = + new GlTextureInfo( + texId, /* fboId= */ C.INDEX_UNSET, /* rboId= */ C.INDEX_UNSET, width, height); + + GlTextureFrame inputFrame = + new GlTextureFrame.Builder( + textureInfo, directExecutor(), /* releaseTextureCallback= */ unused -> {}) + .setPresentationTimeUs(0) + .setFormat(hlgFormat) + .setMetadata( + ImmutableMap.of( + KEY_COMPOSITION_SEQUENCE_INDEX, + 0, + KEY_COMPOSITOR_SETTINGS, + VideoCompositorSettings.DEFAULT)) + .build(); + + AtomicReference outputFrameRef = new AtomicReference<>(); + GlTextureFrameConsumer downstreamConsumer = + new GlTextureFrameConsumer() { + @Override + public boolean queue( + GlTextureFrame frame, Executor listenerExecutor, Runnable wakeupListener) { + outputFrameRef.set(frame); + return true; + } + + @Override + public void signalEndOfStream() {} + + @Override + public void close() {} + }; + + DefaultGlTextureFrameCompositor.Factory factory = + new DefaultGlTextureFrameCompositor.Factory( + new DefaultCompositorGlProgram.Factory(context)); + GlTextureFrameCompositor compositor = + factory.create( + glObjectsProvider, + BT2020_HLG, + /* errorConsumer= */ exception -> { + throw new AssertionError(exception); + }, + directExecutor(), + downstreamConsumer); + + assertThat( + compositor.queue( + ImmutableList.of(inputFrame), directExecutor(), /* wakeupListener= */ () -> {})) + .isTrue(); + + GlTextureFrame outputFrame = checkNotNull(outputFrameRef.get()); + assertThat(outputFrame.format.colorInfo).isEqualTo(BT2020_HLG); + assertThat(outputFrame.glTextureInfo.texId).isEqualTo(texId); + + outputFrame.release(/* releaseFence= */ null); + compositor.close(); + } + + @Test + @SdkSuppress(minSdkVersion = 33) + public void composite_twoHdrSequences_withOpaqueOverlay_outputsCorrectTexture() throws Exception { + Format hlgFormat = new Format.Builder().setColorInfo(BT2020_HLG).build(); + assumeTrue("Device does not support OpenGL ES 3.0", GlUtil.getContextMajorVersion() >= 3); + + Bitmap hlgBitmap = readBitmapUnpremultipliedAlpha(ORIGINAL_HLG10_PNG_ASSET_PATH); + int width = hlgBitmap.getWidth(); + int height = hlgBitmap.getHeight(); + int texId0 = createGlTextureFromBitmap(hlgBitmap); + int texId1 = createGlTextureFromBitmap(hlgBitmap); + texturesToDelete.add(texId0); + texturesToDelete.add(texId1); + + VideoCompositorSettings compositorSettings = + new VideoCompositorSettings() { + @Override + public Size getOutputSize(List inputSizes) { + return inputSizes.get(0); + } + + @Override + public OverlaySettings getOverlaySettings(int inputId, long presentationTimeUs) { + return new StaticOverlaySettings.Builder().build(); + } + }; + + GlTextureFrame frame0 = + new GlTextureFrame.Builder( + new GlTextureInfo( + texId0, /* fboId= */ C.INDEX_UNSET, /* rboId= */ C.INDEX_UNSET, width, height), + directExecutor(), + /* releaseTextureCallback= */ unused -> {}) + .setPresentationTimeUs(0) + .setFormat(hlgFormat) + .setMetadata( + ImmutableMap.of( + KEY_COMPOSITION_SEQUENCE_INDEX, 0, KEY_COMPOSITOR_SETTINGS, compositorSettings)) + .build(); + + GlTextureFrame frame1 = + new GlTextureFrame.Builder( + new GlTextureInfo( + texId1, /* fboId= */ C.INDEX_UNSET, /* rboId= */ C.INDEX_UNSET, width, height), + directExecutor(), + /* releaseTextureCallback= */ unused -> {}) + .setPresentationTimeUs(0) + .setFormat(hlgFormat) + .setMetadata( + ImmutableMap.of( + KEY_COMPOSITION_SEQUENCE_INDEX, 1, KEY_COMPOSITOR_SETTINGS, compositorSettings)) + .build(); + + AtomicReference outputFrameRef = new AtomicReference<>(); + GlTextureFrameConsumer downstreamConsumer = + new GlTextureFrameConsumer() { + @Override + public boolean queue( + GlTextureFrame frame, Executor listenerExecutor, Runnable wakeupListener) { + outputFrameRef.set(frame); + return true; + } + + @Override + public void signalEndOfStream() {} + + @Override + public void close() {} + }; + + DefaultGlTextureFrameCompositor.Factory factory = + new DefaultGlTextureFrameCompositor.Factory( + new DefaultCompositorGlProgram.Factory(context)); + GlTextureFrameCompositor compositor = + factory.create( + glObjectsProvider, + BT2020_HLG, + /* errorConsumer= */ exception -> { + throw new AssertionError(exception); + }, + directExecutor(), + downstreamConsumer); + + assertThat( + compositor.queue( + ImmutableList.of(frame0, frame1), directExecutor(), /* wakeupListener= */ () -> {})) + .isTrue(); + + GlTextureFrame outputFrame = checkNotNull(outputFrameRef.get()); + assertThat(outputFrame.format.colorInfo).isEqualTo(BT2020_HLG); + + int outputFbo = GlUtil.createFboForTexture(outputFrame.glTextureInfo.texId); + frameBuffersToDelete.add(outputFbo); + GlUtil.focusFramebuffer( + checkNotNull(eglDisplay), + checkNotNull(eglContext), + checkNotNull(placeholderEglSurface), + outputFbo, + width, + height); + + Bitmap actualBitmap = createFp16BitmapFromFocusedGlFramebuffer(width, height); + actualBitmap.setColorSpace(checkNotNull(hlgBitmap.getColorSpace())); + + maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual", actualBitmap, /* path= */ null); + assertBitmapsAreSimilar(hlgBitmap, actualBitmap, PSNR_THRESHOLD); + + outputFrame.release(/* releaseFence= */ null); + compositor.close(); + } + + @Test + public void composite_twoSdrSequences_withOpaqueOverlay_outputsCorrectTexture() throws Exception { + Format sdrFormat = new Format.Builder().setColorInfo(BT709_SRGB).build(); + + Bitmap sdrBitmap = readBitmapUnpremultipliedAlpha(ORIGINAL_SDR_PNG_ASSET_PATH); + int width = sdrBitmap.getWidth(); + int height = sdrBitmap.getHeight(); + int texId0 = createGlTextureFromBitmap(sdrBitmap); + int texId1 = createGlTextureFromBitmap(sdrBitmap); + texturesToDelete.add(texId0); + texturesToDelete.add(texId1); + + VideoCompositorSettings compositorSettings = + new VideoCompositorSettings() { + @Override + public Size getOutputSize(List inputSizes) { + return inputSizes.get(0); + } + + @Override + public OverlaySettings getOverlaySettings(int inputId, long presentationTimeUs) { + return new StaticOverlaySettings.Builder().build(); + } + }; + + GlTextureFrame frame0 = + new GlTextureFrame.Builder( + new GlTextureInfo( + texId0, /* fboId= */ C.INDEX_UNSET, /* rboId= */ C.INDEX_UNSET, width, height), + directExecutor(), + /* releaseTextureCallback= */ unused -> {}) + .setPresentationTimeUs(0) + .setFormat(sdrFormat) + .setMetadata( + ImmutableMap.of( + KEY_COMPOSITION_SEQUENCE_INDEX, 0, KEY_COMPOSITOR_SETTINGS, compositorSettings)) + .build(); + + GlTextureFrame frame1 = + new GlTextureFrame.Builder( + new GlTextureInfo( + texId1, /* fboId= */ C.INDEX_UNSET, /* rboId= */ C.INDEX_UNSET, width, height), + directExecutor(), + /* releaseTextureCallback= */ unused -> {}) + .setPresentationTimeUs(0) + .setFormat(sdrFormat) + .setMetadata( + ImmutableMap.of( + KEY_COMPOSITION_SEQUENCE_INDEX, 1, KEY_COMPOSITOR_SETTINGS, compositorSettings)) + .build(); + + AtomicReference outputFrameRef = new AtomicReference<>(); + GlTextureFrameConsumer downstreamConsumer = + new GlTextureFrameConsumer() { + @Override + public boolean queue( + GlTextureFrame frame, Executor listenerExecutor, Runnable wakeupListener) { + outputFrameRef.set(frame); + return true; + } + + @Override + public void signalEndOfStream() {} + + @Override + public void close() {} + }; + + DefaultGlTextureFrameCompositor.Factory factory = + new DefaultGlTextureFrameCompositor.Factory( + new DefaultCompositorGlProgram.Factory(context)); + GlTextureFrameCompositor compositor = + factory.create( + glObjectsProvider, + BT709_SRGB, + /* errorConsumer= */ exception -> { + throw new AssertionError(exception); + }, + directExecutor(), + downstreamConsumer); + + assertThat( + compositor.queue( + ImmutableList.of(frame0, frame1), directExecutor(), /* wakeupListener= */ () -> {})) + .isTrue(); + + GlTextureFrame outputFrame = checkNotNull(outputFrameRef.get()); + assertThat(outputFrame.format.colorInfo).isEqualTo(BT709_SRGB); + + int outputFbo = GlUtil.createFboForTexture(outputFrame.glTextureInfo.texId); + frameBuffersToDelete.add(outputFbo); + GlUtil.focusFramebuffer( + checkNotNull(eglDisplay), + checkNotNull(eglContext), + checkNotNull(placeholderEglSurface), + outputFbo, + width, + height); + + Bitmap actualBitmap = + createUnpremultipliedArgb8888BitmapFromFocusedGlFramebuffer(width, height); + + maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual", actualBitmap, /* path= */ null); + assertBitmapsAreSimilar(sdrBitmap, actualBitmap, PSNR_THRESHOLD); + + outputFrame.release(/* releaseFence= */ null); + compositor.close(); + } +} diff --git a/libraries/effect/src/androidTest/java/androidx/media3/effect/DefaultVideoFrameProcessorPixelTest.java b/libraries/effect/src/androidTest/java/androidx/media3/effect/DefaultVideoFrameProcessorPixelTest.java index 257f0fd5aae..b23158a95c3 100644 --- a/libraries/effect/src/androidTest/java/androidx/media3/effect/DefaultVideoFrameProcessorPixelTest.java +++ b/libraries/effect/src/androidTest/java/androidx/media3/effect/DefaultVideoFrameProcessorPixelTest.java @@ -26,8 +26,10 @@ import android.content.Context; import android.graphics.Bitmap; +import android.graphics.Color; import android.graphics.Matrix; import androidx.media3.common.C; +import androidx.media3.common.ColorInfo; import androidx.media3.common.Effect; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.Size; @@ -86,6 +88,8 @@ public final class DefaultVideoFrameProcessorPixelTest { "test-generated-goldens/sample_mp4_first_frame/electrical_colors/increase_brightness.png"; private static final String GRAYSCALE_THEN_INCREASE_RED_CHANNEL_PNG_ASSET_PATH = "test-generated-goldens/sample_mp4_first_frame/electrical_colors/grayscale_then_increase_red_channel.png"; + private static final String TONE_MAP_PQ_REGISTER_INPUT_PNG_ASSET_PATH = + "test-generated-goldens/DefaultVideoFrameProcessorPixelTest/toneMap_pqRegisterInput.png"; /** Input video of which we only use the first frame. */ private static final String INPUT_SDR_MP4_ASSET_STRING = "media/mp4/sample.mp4"; @@ -159,6 +163,46 @@ public void noEffects_withDisabledColorTransfers_matchesGoldenFile() throws Exce assertThat(averagePixelAbsoluteDifference).isAtMost(MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE); } + @Test + public void toneMap_pqRegisterInput_matchesGoldenFile() throws Exception { + videoFrameProcessorTestRunner = getDefaultFrameProcessorTestRunnerBuilder(testId).build(); + + int width = 64; + int height = 64; + Bitmap inputBitmap = Bitmap.createBitmap(width, height, Bitmap.Config.ARGB_8888); + // Use saturated primary colors (pure green and pure blue). + // In wide-gamut BT.2020 PQ space, saturated primaries lie outside the BT.709 color gamut. + // When tone-mapping to SDR (BT.709), applying the OOTF to these colors yields negative RGB + // component values in linear BT.709 space. Without clamping to 0.0 before applying the Gamma + // 2.2 OETF (pow(x, 1.0 / 2.2)), GLSL produces NaNs for pow(negative, fraction), causing severe + // rendering artifacts or corrupted output bitmaps. + for (int y = 0; y < height; y++) { + for (int x = 0; x < width; x++) { + if (x < width / 2) { + inputBitmap.setPixel(x, y, Color.GREEN); + } else { + inputBitmap.setPixel(x, y, Color.BLUE); + } + } + } + + ColorInfo pqColorInfo = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorRange(C.COLOR_RANGE_FULL) + .setColorTransfer(C.COLOR_TRANSFER_ST2084) + .build(); + + videoFrameProcessorTestRunner.queueInputBitmap( + inputBitmap, C.MICROS_PER_SECOND, /* offsetToAddUs= */ 0L, /* frameRate= */ 1, pqColorInfo); + videoFrameProcessorTestRunner.endFrameProcessing(); + Bitmap actualBitmap = videoFrameProcessorTestRunner.getOutputBitmap(); + Bitmap expectedBitmap = readBitmap(TONE_MAP_PQ_REGISTER_INPUT_PNG_ASSET_PATH); + float averagePixelAbsoluteDifference = + getBitmapAveragePixelAbsoluteDifferenceArgb8888(expectedBitmap, actualBitmap, testId); + assertThat(averagePixelAbsoluteDifference).isAtMost(MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE); + } + @Test public void noEffects_withImageInput_matchesGoldenFile() throws Exception { videoFrameProcessorTestRunner = getDefaultFrameProcessorTestRunnerBuilder(testId).build(); diff --git a/libraries/effect/src/androidTest/java/androidx/media3/effect/EffectsTestUtil.java b/libraries/effect/src/androidTest/java/androidx/media3/effect/EffectsTestUtil.java index 0539bdf6cab..9c9a38cde52 100644 --- a/libraries/effect/src/androidTest/java/androidx/media3/effect/EffectsTestUtil.java +++ b/libraries/effect/src/androidTest/java/androidx/media3/effect/EffectsTestUtil.java @@ -25,7 +25,12 @@ import static com.google.common.truth.Truth.assertThat; import android.graphics.Bitmap; +import android.opengl.EGL14; +import android.opengl.EGLContext; +import android.opengl.EGLDisplay; +import android.opengl.EGLSurface; import android.text.SpannableString; +import android.util.Pair; import androidx.annotation.Nullable; import androidx.media3.common.ColorInfo; import androidx.media3.common.DebugViewProvider; @@ -34,6 +39,7 @@ import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.VideoFrameProcessor; import androidx.media3.common.util.Consumer; +import androidx.media3.common.util.GlUtil; import androidx.media3.common.util.NullableType; import androidx.media3.common.util.Util; import androidx.media3.test.utils.TextureBitmapReader; @@ -208,5 +214,34 @@ private static void checkNoVideoFrameProcessingExceptionIsThrown( } } + /** + * Creates an {@link EGLContext} and focused {@link EGLSurface}, falling back to OpenGL ES 2.0 if + * 3.0 fails. + */ + public static Pair createFocusedEglContextWithFallback( + EGLDisplay eglDisplay, int[] configAttributes) throws GlUtil.GlException { + try { + return createFocusedEglContext(eglDisplay, /* openGlVersion= */ 3, configAttributes); + } catch (GlUtil.GlException e) { + return createFocusedEglContext(eglDisplay, /* openGlVersion= */ 2, configAttributes); + } + } + + /** + * Creates an {@link EGLContext} and focused {@link EGLSurface} for the given OpenGL ES version. + */ + private static Pair createFocusedEglContext( + EGLDisplay eglDisplay, int openGlVersion, int[] configAttributes) throws GlUtil.GlException { + EGLContext eglContext = + GlUtil.createEglContext(EGL14.EGL_NO_CONTEXT, eglDisplay, openGlVersion, configAttributes); + try { + EGLSurface eglSurface = GlUtil.createFocusedPlaceholderEglSurface(eglContext, eglDisplay); + return Pair.create(eglContext, eglSurface); + } catch (GlUtil.GlException e) { + GlUtil.destroyEglContext(eglDisplay, eglContext); + throw e; + } + } + private EffectsTestUtil() {} } diff --git a/libraries/effect/src/androidTest/java/androidx/media3/effect/FrameWriterGlTextureFrameConsumerTest.java b/libraries/effect/src/androidTest/java/androidx/media3/effect/FrameWriterGlTextureFrameConsumerTest.java index 814a5fe033b..785dbc60cee 100644 --- a/libraries/effect/src/androidTest/java/androidx/media3/effect/FrameWriterGlTextureFrameConsumerTest.java +++ b/libraries/effect/src/androidTest/java/androidx/media3/effect/FrameWriterGlTextureFrameConsumerTest.java @@ -18,22 +18,29 @@ import static androidx.media3.effect.FrameProcessorUtils.releaseOpenGl; import static androidx.media3.effect.FrameProcessorUtils.setupOpenGl; import static androidx.media3.effect.FrameProcessorUtils.shutdownGlExecutorService; +import static androidx.media3.effect.FrameProcessorUtils.useHighPrecisionColorComponents; import static androidx.media3.test.utils.BitmapPixelTestUtil.MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE; import static androidx.test.core.app.ApplicationProvider.getApplicationContext; import static com.google.common.base.Preconditions.checkArgument; +import static com.google.common.base.Preconditions.checkNotNull; +import static com.google.common.base.Preconditions.checkState; import static com.google.common.truth.Truth.assertThat; import static com.google.common.util.concurrent.MoreExecutors.directExecutor; import static com.google.common.util.concurrent.MoreExecutors.listeningDecorator; import static java.util.concurrent.Executors.newSingleThreadExecutor; +import static java.util.concurrent.TimeUnit.MILLISECONDS; import android.content.Context; import android.graphics.Bitmap; import android.graphics.Canvas; import android.graphics.Color; import android.graphics.ColorSpace; +import android.graphics.Matrix; import android.hardware.HardwareBuffer; +import android.opengl.GLES20; import androidx.annotation.Nullable; import androidx.media3.common.C; +import androidx.media3.common.ColorInfo; import androidx.media3.common.Format; import androidx.media3.common.GlObjectsProvider; import androidx.media3.common.GlTextureInfo; @@ -46,7 +53,6 @@ import androidx.media3.common.video.FrameWriter; import androidx.media3.common.video.HardwareBufferFrame; import androidx.media3.common.video.SyncFenceWrapper; -import androidx.media3.effect.ndk.HardwareBufferJni; import androidx.media3.test.utils.BitmapPixelTestUtil; import androidx.test.ext.junit.runners.AndroidJUnit4; import androidx.test.filters.SdkSuppress; @@ -69,19 +75,39 @@ public final class FrameWriterGlTextureFrameConsumerTest { private static final String TEST_IMAGE_ASSET = "media/png/first_frame_1920x1080.png"; + + private static final long TIMEOUT_MS = 5_000; private static final int BITMAP_WIDTH = 1920; private static final int BITMAP_HEIGHT = 1080; - - @Rule public final TestName testName = new TestName(); - private final Context context = getApplicationContext(); + private static final int SOLID_FRAME_WIDTH = 16; + private static final int SOLID_FRAME_HEIGHT = 16; + // Diffuse white in the linear BT.2020 working space, which is anchored on BT.2408 diffuse white. + private static final float DIFFUSE_WHITE_BT2020_LINEAR = 1f; + // BT.2408 diffuse white HLG signal level. + private static final float DIFFUSE_WHITE_HLG = 0.75f; + // sRGB OETF applied to 0.5: 1.055 * 0.5^(1 / 2.4) - 0.055. + private static final float HALF_LINEAR_SRGB = 0.7354f; + private static final float PIXEL_TOLERANCE = 2f; + // FrameWriterGlTextureFrameConsumer configures the FrameWriter with limited range because + // encoders output limited range. Only the metadata changes, the written pixels are full range. + private static final ColorInfo ENCODER_OUTPUT_LIMITED_RANGE_HLG = + DefaultGlFrameProcessor.BT2020_HLG.buildUpon().setColorRange(C.COLOR_RANGE_LIMITED).build(); + + @Rule public final TestName testName; + private final Context context; private @MonotonicNonNull ListeningExecutorService glExecutorService; private @MonotonicNonNull GlObjectsProvider glObjectsProvider; - private @MonotonicNonNull HardwareBuffer frameWriterHardwareBuffer; private @MonotonicNonNull BitmapSavingFrameWriter frameWriter; - private @MonotonicNonNull FakeHardwareBufferJniWrapper fakeJniWrapper; + private @MonotonicNonNull ForwardingHardwareBufferJniWrapper fakeJniWrapper; private @MonotonicNonNull FrameWriterGlTextureFrameConsumer frameWriterGlTextureFrameConsumer; - private final List actualBitmaps = new CopyOnWriteArrayList<>(); + private final List actualBitmaps; + + public FrameWriterGlTextureFrameConsumerTest() { + testName = new TestName(); + context = getApplicationContext(); + actualBitmaps = new CopyOnWriteArrayList<>(); + } @Before public void setUp() throws Exception { @@ -90,23 +116,13 @@ public void setUp() throws Exception { glExecutorService .submit( () -> { - setupOpenGl(glObjectsProvider); + int unused = setupOpenGl(glObjectsProvider); return null; }) .get(); - frameWriterHardwareBuffer = - HardwareBuffer.create( - BITMAP_WIDTH, - BITMAP_HEIGHT, - HardwareBuffer.RGBA_8888, - /* layers= */ 1, - HardwareBuffer.USAGE_CPU_READ_OFTEN - | HardwareBuffer.USAGE_GPU_COLOR_OUTPUT - | HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE); - - fakeJniWrapper = new FakeHardwareBufferJniWrapper(); - frameWriter = new BitmapSavingFrameWriter(frameWriterHardwareBuffer, actualBitmaps); + fakeJniWrapper = new ForwardingHardwareBufferJniWrapper(); + frameWriter = new BitmapSavingFrameWriter(actualBitmaps); frameWriterGlTextureFrameConsumer = new FrameWriterGlTextureFrameConsumer(context, frameWriter, fakeJniWrapper); } @@ -117,9 +133,6 @@ public void tearDown() throws Exception { glExecutorService.submit(this::tearDownInternal).get(); shutdownGlExecutorService(glExecutorService); } - if (frameWriterHardwareBuffer != null) { - frameWriterHardwareBuffer.close(); - } } private void tearDownInternal() { @@ -127,6 +140,9 @@ private void tearDownInternal() { if (frameWriterGlTextureFrameConsumer != null) { frameWriterGlTextureFrameConsumer.close(); } + if (frameWriter != null) { + frameWriter.close(); + } if (glObjectsProvider != null) { releaseOpenGl(glObjectsProvider); } @@ -298,11 +314,354 @@ inputFrame1, directExecutor(), /* wakeupListener= */ () -> {})) .isLessThan(MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE); } + @Test + public void queue_portraitFrameWhenPortraitUnsupported_rotatesToLandscape() throws Exception { + frameWriter.supportsPortrait = false; + Bitmap landscapeBitmap = BitmapPixelTestUtil.readBitmap(TEST_IMAGE_ASSET); + Matrix matrix = new Matrix(); + matrix.postRotate(90); + Bitmap portraitInputBitmap = + Bitmap.createBitmap( + landscapeBitmap, + /* x= */ 0, + /* y= */ 0, + landscapeBitmap.getWidth(), + landscapeBitmap.getHeight(), + matrix, + /* filter= */ true); + Matrix expectedMatrix = new Matrix(); + expectedMatrix.postRotate(180); + // The input bitmap is rotated 90 clockwise, and the pipeline rotates it to landscape by another + // 90 degrees, so the output will be 180 degrees rotated. + Bitmap expectedBitmap = + Bitmap.createBitmap( + landscapeBitmap, + /* x= */ 0, + /* y= */ 0, + landscapeBitmap.getWidth(), + landscapeBitmap.getHeight(), + expectedMatrix, + /* filter= */ true); + + glExecutorService + .submit( + () -> { + try { + GlTextureFrame inputFrame = + createGlTextureFrame(portraitInputBitmap, /* presentationTimeUs= */ 1000); + assertThat( + frameWriterGlTextureFrameConsumer.queue( + inputFrame, directExecutor(), /* wakeupListener= */ () -> {})) + .isTrue(); + } catch (Exception e) { + throw new AssertionError(e); + } + }) + .get(TIMEOUT_MS, MILLISECONDS); + + Bitmap outputBitmap = actualBitmaps.get(0); + assertThat( + BitmapPixelTestUtil.getBitmapAveragePixelAbsoluteDifferenceArgb8888( + expectedBitmap, outputBitmap, testName.getMethodName())) + .isLessThan(MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE); + + Format configuredFrameWriterFormat = checkNotNull(frameWriter.configuredFormat); + assertThat(configuredFrameWriterFormat.width).isEqualTo(BITMAP_WIDTH); + assertThat(configuredFrameWriterFormat.height).isEqualTo(BITMAP_HEIGHT); + assertThat(configuredFrameWriterFormat.rotationDegrees).isEqualTo(270); + } + + @Test + public void queue_landscapeThenPortrait_outputsCorrectBitmaps() throws Exception { + Bitmap inputBitmap1 = BitmapPixelTestUtil.readBitmap(TEST_IMAGE_ASSET); // 1920x1080 + Matrix matrix = new Matrix(); + matrix.postRotate(90); + Bitmap inputBitmap2 = + Bitmap.createBitmap( + inputBitmap1, + /* x= */ 0, + /* y= */ 0, + inputBitmap1.getWidth(), + inputBitmap1.getHeight(), + matrix, + /* filter= */ true); // 1080x1920 + // Construct the expected pillarboxed bitmap. + // Scale 1080x1920 to fit 1920x1080: 608x1080, hence pillar box width = (1920 - 608) / 2 = 656. + Bitmap expectedBitmap2 = + Bitmap.createBitmap(/* width= */ 1920, /* height= */ 1080, Bitmap.Config.ARGB_8888); + Canvas canvas = new Canvas(expectedBitmap2); + canvas.drawColor(Color.BLACK); // black background + Bitmap scaledInput2 = Bitmap.createScaledBitmap(inputBitmap2, 608, 1080, /* filter= */ true); + canvas.drawBitmap(scaledInput2, /* left= */ 656, /* top= */ 0, /* paint= */ null); + + glExecutorService + .submit( + () -> { + try { + GlTextureFrame inputFrame1 = + createGlTextureFrame(inputBitmap1, /* presentationTimeUs= */ 1000); + assertThat( + frameWriterGlTextureFrameConsumer.queue( + inputFrame1, directExecutor(), /* wakeupListener= */ () -> {})) + .isTrue(); + + GlTextureFrame inputFrame2 = + createGlTextureFrame(inputBitmap2, /* presentationTimeUs= */ 2000); + assertThat( + frameWriterGlTextureFrameConsumer.queue( + inputFrame2, directExecutor(), /* wakeupListener= */ () -> {})) + .isTrue(); + } catch (Exception e) { + throw new AssertionError(e); + } + }) + .get(); + + // Frame 1: 1920x1080 + Bitmap outputBitmap1 = actualBitmaps.get(0); + assertThat( + BitmapPixelTestUtil.getBitmapAveragePixelAbsoluteDifferenceArgb8888( + inputBitmap1, outputBitmap1, testName.getMethodName() + "_frame1")) + .isLessThan(MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE); + + // Frame 2: Conformed to 1920x1080 (pillarboxed) + Bitmap outputBitmap2 = actualBitmaps.get(1); + assertThat( + BitmapPixelTestUtil.getBitmapAveragePixelAbsoluteDifferenceArgb8888( + expectedBitmap2, outputBitmap2, testName.getMethodName() + "_frame2")) + .isLessThan(MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE); + } + + @Test + public void queue_portraitThenLandscape_outputsCorrectBitmaps() throws Exception { + Bitmap originalBitmap = BitmapPixelTestUtil.readBitmap(TEST_IMAGE_ASSET); // 1920x1080 + Matrix matrix = new Matrix(); + matrix.postRotate(90); + Bitmap inputBitmap1 = + Bitmap.createBitmap( + originalBitmap, + /* x= */ 0, + /* y= */ 0, + originalBitmap.getWidth(), + originalBitmap.getHeight(), + matrix, + /* filter= */ true); // 1080x1920 + Bitmap inputBitmap2 = originalBitmap; // 1920x1080 (landscape) + // Construct the expected letterboxed bitmap. + // Scale 1920x1080 to fit 1080x1920: 1080x608, hence letter box height = (1920 - 608) / 2 = 656. + Bitmap expectedBitmap2 = + Bitmap.createBitmap(/* width= */ 1080, /* height= */ 1920, Bitmap.Config.ARGB_8888); + Canvas canvas = new Canvas(expectedBitmap2); + canvas.drawColor(Color.BLACK); // black background + Bitmap scaledInput2 = + Bitmap.createScaledBitmap( + inputBitmap2, /* dstWidth= */ 1080, /* dstHeight= */ 608, /* filter= */ true); + canvas.drawBitmap(scaledInput2, /* left= */ 0, /* top= */ 656, /* paint= */ null); + + glExecutorService + .submit( + () -> { + try { + GlTextureFrame inputFrame1 = + createGlTextureFrame(inputBitmap1, /* presentationTimeUs= */ 1000); + assertThat( + frameWriterGlTextureFrameConsumer.queue( + inputFrame1, directExecutor(), /* wakeupListener= */ () -> {})) + .isTrue(); + + GlTextureFrame inputFrame2 = + createGlTextureFrame(inputBitmap2, /* presentationTimeUs= */ 2000); + assertThat( + frameWriterGlTextureFrameConsumer.queue( + inputFrame2, directExecutor(), /* wakeupListener= */ () -> {})) + .isTrue(); + } catch (Exception e) { + throw new AssertionError(e); + } + }) + .get(); + + // Frame 1: 1080x1920 (portrait) + Bitmap outputBitmap1 = actualBitmaps.get(0); + assertThat( + BitmapPixelTestUtil.getBitmapAveragePixelAbsoluteDifferenceArgb8888( + inputBitmap1, outputBitmap1, testName.getMethodName() + "_frame1")) + .isLessThan(MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE); + + // Frame 2: Conformed to 1080x1920 (letterboxed) + Bitmap outputBitmap2 = actualBitmaps.get(1); + assertThat( + BitmapPixelTestUtil.getBitmapAveragePixelAbsoluteDifferenceArgb8888( + expectedBitmap2, outputBitmap2, testName.getMethodName() + "_frame2")) + .isLessThan(MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE); + } + + @Test + public void queue_hdrLinearInputAndUnsetOutput_convertsToHlg() throws Exception { + queueOnGlThread( + () -> + createSolidGrayGlTextureFrame( + /* value= */ DIFFUSE_WHITE_BT2020_LINEAR, DefaultGlFrameProcessor.BT2020_LINEAR)); + + assertThat(checkNotNull(frameWriter.configuredFormat).colorInfo) + .isEqualTo(ENCODER_OUTPUT_LIMITED_RANGE_HLG); + assertThat(actualBitmaps).hasSize(1); + assertBitmapIsSolidGray( + actualBitmaps.get(0), + /* expectedComponent= */ DIFFUSE_WHITE_HLG * 255, + testName.getMethodName()); + } + + @Test + public void queue_sdrLinearInputAndUnsetOutput_convertsToSrgb() throws Exception { + queueOnGlThread( + () -> + createSolidGrayGlTextureFrame(/* value= */ 0.5f, DefaultGlFrameProcessor.BT709_LINEAR)); + + assertThat(checkNotNull(frameWriter.configuredFormat).colorInfo) + .isEqualTo(ColorInfo.SDR_BT709_LIMITED); + assertThat(actualBitmaps).hasSize(1); + assertBitmapIsSolidGray( + actualBitmaps.get(0), + /* expectedComponent= */ HALF_LINEAR_SRGB * 255, + testName.getMethodName()); + } + + @Test + public void queue_srgbInputAndUnsetOutput_writesPixelsUnchanged() throws Exception { + queueOnGlThread( + () -> createSolidGrayGlTextureFrame(/* value= */ 0.5f, DefaultGlFrameProcessor.BT709_SRGB)); + + assertThat(checkNotNull(frameWriter.configuredFormat).colorInfo) + .isEqualTo(ColorInfo.SDR_BT709_LIMITED); + assertThat(actualBitmaps).hasSize(1); + assertBitmapIsSolidGray( + actualBitmaps.get(0), /* expectedComponent= */ 0.5f * 255, testName.getMethodName()); + } + + @Test + public void queue_hlgInputAndUnsetOutput_writesPixelsUnchanged() throws Exception { + queueOnGlThread( + () -> createSolidGrayGlTextureFrame(/* value= */ 0.5f, DefaultGlFrameProcessor.BT2020_HLG)); + + assertThat(checkNotNull(frameWriter.configuredFormat).colorInfo) + .isEqualTo(ENCODER_OUTPUT_LIMITED_RANGE_HLG); + assertThat(actualBitmaps).hasSize(1); + assertBitmapIsSolidGray( + actualBitmaps.get(0), /* expectedComponent= */ 0.5f * 255, testName.getMethodName()); + } + + @Test + public void queue_hdrLinearThenHlgInputAndUnsetOutput_convertsOnlyLinearFrame() throws Exception { + queueOnGlThread( + () -> + createSolidGrayGlTextureFrame( + /* value= */ DIFFUSE_WHITE_BT2020_LINEAR, DefaultGlFrameProcessor.BT2020_LINEAR)); + queueOnGlThread( + () -> createSolidGrayGlTextureFrame(/* value= */ 0.5f, DefaultGlFrameProcessor.BT2020_HLG)); + + assertThat(actualBitmaps).hasSize(2); + assertBitmapIsSolidGray( + actualBitmaps.get(0), + /* expectedComponent= */ DIFFUSE_WHITE_HLG * 255, + testName.getMethodName() + "_frame1"); + assertBitmapIsSolidGray( + actualBitmaps.get(1), + /* expectedComponent= */ 0.5f * 255, + testName.getMethodName() + "_frame2"); + } + + /** Creates a {@link GlTextureFrame} on the GL thread and queues it to the consumer under test. */ + private void queueOnGlThread(GlTextureFrameSupplier frameSupplier) throws Exception { + glExecutorService + .submit( + () -> { + try { + assertThat( + frameWriterGlTextureFrameConsumer.queue( + frameSupplier.get(), directExecutor(), /* wakeupListener= */ () -> {})) + .isTrue(); + } catch (Exception e) { + throw new AssertionError(e); + } + }) + .get(TIMEOUT_MS, MILLISECONDS); + } + + /** + * Asserts that every pixel of {@code bitmap} is the gray {@code expectedComponent}, within {@link + * #PIXEL_TOLERANCE} on average. + */ + private static void assertBitmapIsSolidGray( + Bitmap bitmap, float expectedComponent, String testId) { + int expectedValue = Math.round(expectedComponent); + Bitmap expectedBitmap = + BitmapPixelTestUtil.createArgb8888BitmapWithSolidColor( + bitmap.getWidth(), + bitmap.getHeight(), + Color.rgb(expectedValue, expectedValue, expectedValue)); + assertThat( + BitmapPixelTestUtil.getBitmapAveragePixelAbsoluteDifferenceArgb8888( + expectedBitmap, bitmap, testId)) + .isAtMost(PIXEL_TOLERANCE); + } + + /** + * Creates a {@link GlTextureFrame} filled with the gray {@code value}, whose format has the given + * {@link ColorInfo}. + */ + private static GlTextureFrame createSolidGrayGlTextureFrame(float value, ColorInfo colorInfo) + throws GlException { + int texId = + GlUtil.createTexture( + SOLID_FRAME_WIDTH, SOLID_FRAME_HEIGHT, useHighPrecisionColorComponents(colorInfo)); + int fboId = GlUtil.createFboForTexture(texId); + GlUtil.focusFramebufferUsingCurrentContext(fboId, SOLID_FRAME_WIDTH, SOLID_FRAME_HEIGHT); + GLES20.glClearColor(value, value, value, /* alpha= */ 1f); + GLES20.glClear(GLES20.GL_COLOR_BUFFER_BIT); + GlUtil.checkGlError(); + return new GlTextureFrame.Builder( + new GlTextureInfo( + texId, fboId, /* rboId= */ C.INDEX_UNSET, SOLID_FRAME_WIDTH, SOLID_FRAME_HEIGHT), + directExecutor(), + /* releaseTextureCallback= */ textureInfo -> { + try { + textureInfo.release(); + } catch (GlException e) { + // Ignore. + } + }) + .setFormat( + new Format.Builder() + .setWidth(SOLID_FRAME_WIDTH) + .setHeight(SOLID_FRAME_HEIGHT) + .setColorInfo(colorInfo) + .build()) + .build(); + } + + private interface GlTextureFrameSupplier { + GlTextureFrame get() throws GlException; + } + + /** + * Creates a {@link GlTextureFrame} that is flipped along the y-axis from the input {@link + * Bitmap}. + * + *

This method converts the input {@link Bitmap} to the OpenGL coordinate system that {@link + * FrameWriterGlTextureFrameConsumer} always receives. + */ private static GlTextureFrame createGlTextureFrame(Bitmap bitmap, long presentationTimeUs) throws GlException { int width = bitmap.getWidth(); int height = bitmap.getHeight(); - int texId = GlUtil.createTexture(bitmap); + Matrix matrix = new Matrix(); + matrix.postScale(/* sx= */ 1f, /* sy= */ -1f, width / 2f, height / 2f); + Bitmap flippedBitmap = + Bitmap.createBitmap( + bitmap, /* x= */ 0, /* y= */ 0, width, height, matrix, /* filter= */ true); + int texId = GlUtil.createTexture(flippedBitmap); + flippedBitmap.recycle(); return new GlTextureFrame.Builder( new GlTextureInfo( /* texId= */ texId, @@ -319,11 +678,17 @@ private static GlTextureFrame createGlTextureFrame(Bitmap bitmap, long presentat } }) .setPresentationTimeUs(presentationTimeUs) - .setFormat(new Format.Builder().setWidth(width).setHeight(height).build()) + .setFormat( + new Format.Builder() + .setWidth(width) + .setHeight(height) + .setColorInfo(ColorInfo.SRGB_BT709_FULL) + .build()) .build(); } - private static final class FakeHardwareBufferJniWrapper implements HardwareBufferJniWrapper { + private static final class ForwardingHardwareBufferJniWrapper + implements HardwareBufferJniWrapper { private final HardwareBufferJniWrapper delegate = HardwareBufferJni.INSTANCE; int destroyEglImageCount; @@ -357,15 +722,18 @@ public boolean nativeCopyHardwareBufferToHardwareBuffer( } private static final class BitmapSavingFrameWriter implements FrameWriter { - private final HardwareBuffer hardwareBuffer; + @Nullable private HardwareBuffer hardwareBuffer; private final List outputBitmaps; - private boolean hasCapacity = true; + private boolean hasCapacity; @Nullable private Runnable pendingWakeupListener; @Nullable private Executor pendingWakeupExecutor; + private boolean supportsPortrait; + @Nullable private Format configuredFormat; - BitmapSavingFrameWriter(HardwareBuffer hardwareBuffer, List outputBitmaps) { - this.hardwareBuffer = hardwareBuffer; + BitmapSavingFrameWriter(List outputBitmaps) { this.outputBitmaps = outputBitmaps; + supportsPortrait = true; + hasCapacity = true; } void setCapacity(boolean hasCapacity) { @@ -379,11 +747,32 @@ void setCapacity(boolean hasCapacity) { @Override public Info getInfo() { - return (format, usage) -> true; + // Rejects rotation degree if portrait is not supported, this mirrors + // DefaultEncoderFactory. + return (format, usage) -> + format.rotationDegrees == 0 && (supportsPortrait || format.width >= format.height); } @Override - public void configure(Format format, long usage) {} + public void configure(Format format, long usage) { + this.configuredFormat = format; + if (hardwareBuffer == null + || hardwareBuffer.getWidth() != format.width + || hardwareBuffer.getHeight() != format.height) { + if (hardwareBuffer != null) { + hardwareBuffer.close(); + } + hardwareBuffer = + HardwareBuffer.create( + format.width, + format.height, + HardwareBuffer.RGBA_8888, + /* layers= */ 1, + HardwareBuffer.USAGE_CPU_READ_OFTEN + | HardwareBuffer.USAGE_GPU_COLOR_OUTPUT + | HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE); + } + } @Override public AsyncFrame dequeueInputFrame(Executor wakeupExecutor, Runnable wakeupListener) { @@ -393,16 +782,15 @@ public AsyncFrame dequeueInputFrame(Executor wakeupExecutor, Runnable wakeupList return null; } return new AsyncFrame( - new DefaultHardwareBufferFrame.Builder(hardwareBuffer).build(), /* acquireFence= */ null); + new DefaultHardwareBufferFrame.Builder(checkNotNull(hardwareBuffer)).build(), + /* acquireFence= */ null); } @Override public void queueInputFrame(Frame frame, @Nullable SyncFenceWrapper writeCompleteFence) { try { if (writeCompleteFence != null) { - if (!writeCompleteFence.awaitMs(500)) { - throw new IllegalStateException("Fence wait timeout"); - } + checkState(writeCompleteFence.awaitMs(500), "Fence wait timeout"); writeCompleteFence.close(); } checkArgument(frame instanceof HardwareBufferFrame); @@ -410,7 +798,7 @@ public void queueInputFrame(Frame frame, @Nullable SyncFenceWrapper writeComplet Bitmap hardwareBitmap = Bitmap.wrapHardwareBuffer(hardwareBuffer, ColorSpace.get(ColorSpace.Named.SRGB)); outputBitmaps.add(hardwareBitmap.copy(Bitmap.Config.ARGB_8888, /* isMutable= */ false)); - } catch (Exception e) { + } catch (RuntimeException e) { throw new IllegalStateException(e); } } @@ -419,6 +807,10 @@ public void queueInputFrame(Frame frame, @Nullable SyncFenceWrapper writeComplet public void signalEndOfStream() {} @Override - public void close() {} + public void close() { + if (hardwareBuffer != null) { + hardwareBuffer.close(); + } + } } } diff --git a/libraries/effect/src/androidTest/java/androidx/media3/effect/GlslShaderCompilationTest.java b/libraries/effect/src/androidTest/java/androidx/media3/effect/GlslShaderCompilationTest.java new file mode 100644 index 00000000000..6c4fc82209c --- /dev/null +++ b/libraries/effect/src/androidTest/java/androidx/media3/effect/GlslShaderCompilationTest.java @@ -0,0 +1,245 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect; + +import static androidx.test.core.app.ApplicationProvider.getApplicationContext; +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.truth.Truth.assertWithMessage; + +import android.content.Context; +import android.opengl.EGL14; +import android.opengl.EGLContext; +import android.opengl.EGLDisplay; +import android.text.SpannableString; +import androidx.media3.common.util.GlProgram; +import androidx.media3.common.util.GlUtil; +import androidx.media3.common.util.Util; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import java.io.IOException; +import java.lang.reflect.Field; +import java.lang.reflect.Modifier; +import java.util.HashSet; +import java.util.Set; +import org.checkerframework.checker.nullness.qual.MonotonicNonNull; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** + * Instrumentation test verifying that all stripped GLSL raw shader resources in {@link R.raw} + * compile and link into valid OpenGL ES shader programs. + */ +@RunWith(AndroidJUnit4.class) +public final class GlslShaderCompilationTest { + + private final Context context = getApplicationContext(); + + private @MonotonicNonNull EGLDisplay eglDisplay; + private @MonotonicNonNull EGLContext eglContext; + + @Before + public void setUpEglContext() throws GlUtil.GlException { + eglDisplay = GlUtil.getDefaultEglDisplay(); + eglContext = + GlUtil.createEglContext( + EGL14.EGL_NO_CONTEXT, + eglDisplay, + /* openGlVersion= */ 3, + GlUtil.EGL_CONFIG_ATTRIBUTES_RGBA_8888); + GlUtil.createFocusedPlaceholderEglSurface(eglContext, eglDisplay); + } + + @After + public void tearDownEglContext() throws GlUtil.GlException { + if (eglContext != null && eglDisplay != null) { + GlUtil.destroyEglContext(eglDisplay, eglContext); + } + } + + @Test + public void compile_allRawGlslShaders_succeeds() throws Exception { + Set compiledResIds = new HashSet<>(); + + // Standalone ES2 vertex and fragment shaders. + compileGlProgram( + compiledResIds, + R.raw.vertex_shader_transformation_es2, + R.raw.fragment_shader_alpha_scale_es2); + compileGlProgram( + compiledResIds, R.raw.vertex_shader_transformation_es2, R.raw.fragment_shader_copy_es2); + compileGlProgram( + compiledResIds, R.raw.vertex_shader_thumbnail_strip_es2, R.raw.fragment_shader_copy_es2); + compileGlProgram( + compiledResIds, R.raw.vertex_shader_transformation_es2, R.raw.fragment_shader_hsl_es2); + compileGlProgram( + compiledResIds, R.raw.vertex_shader_transformation_es2, R.raw.fragment_shader_lut_es2); + compileGlProgram( + compiledResIds, + R.raw.vertex_shader_transformation_es2, + R.raw.fragment_shader_separable_convolution_es2); + compileGlProgram( + compiledResIds, + R.raw.vertex_shader_transformation_es2, + R.raw.fragment_shader_transformation_es2); + compileGlProgram( + compiledResIds, + R.raw.vertex_shader_transformation_es2, + R.raw.fragment_shader_transformation_sdr_external_es2); + compileGlProgram( + compiledResIds, + R.raw.vertex_shader_transformation_es2, + R.raw.fragment_shader_transformation_sdr_internal_es2); + compileGlProgram( + compiledResIds, + R.raw.vertex_shader_transformation_es2, + R.raw.fragment_shader_transformation_sdr_oetf_es2); + + // Concatenated ES2 color conversion shaders. + compileGlProgram( + compiledResIds, + R.raw.vertex_shader_transformation_es2, + R.raw.color_conversions_es2, + R.raw.fragment_shader_transformation_sdr_internal_color_conversion_es2); + compileGlProgram( + compiledResIds, + R.raw.vertex_shader_transformation_es2, + R.raw.color_conversions_es2, + R.raw.fragment_shader_transformation_sdr_external_color_conversion_es2); + + // Standalone ES3 vertex and fragment shaders. + compileGlProgram( + compiledResIds, + R.raw.vertex_shader_transformation_es3, + R.raw.fragment_shader_hdr_to_ultra_hdr_es3); + compileGlProgram( + compiledResIds, R.raw.vertex_shader_transformation_es3, R.raw.fragment_shader_oetf_es3); + compileGlProgram( + compiledResIds, + R.raw.vertex_shader_transformation_es3, + R.raw.fragment_shader_transformation_hdr_internal_es3); + compileGlProgram( + compiledResIds, + R.raw.vertex_shader_transformation_es3, + R.raw.fragment_shader_transformation_ultra_hdr_es3); + + // Concatenated ES3 color conversion shaders. + compileGlProgram( + compiledResIds, + R.raw.vertex_shader_transformation_es3, + R.raw.color_conversions_es3, + R.raw.fragment_shader_transformation_hdr_internal_color_conversion_es3); + compileGlProgram( + compiledResIds, + R.raw.vertex_shader_transformation_es3, + R.raw.color_conversions_es3, + R.raw.fragment_shader_transformation_ultra_hdr_color_conversion_es3); + compileGlProgram( + compiledResIds, + R.raw.vertex_shader_transformation_es3, + R.raw.color_conversions_es3, + R.raw.fragment_shader_color_conversion_es3); + + // ES3 external YUV shaders (require GL_EXT_YUV_target). + compileExternalYuvGlProgram( + compiledResIds, + R.raw.vertex_shader_transformation_es3, + R.raw.fragment_shader_transformation_external_yuv_es3); + compileExternalYuvGlProgram( + compiledResIds, + R.raw.vertex_shader_transformation_es3, + R.raw.color_conversions_es3, + R.raw.fragment_shader_transformation_external_yuv_color_conversion_es3); + + // Injected overlay fragment shader snippets (insert_overlay_fragment_shader_methods and + // insert_ultra_hdr). + OverlayShaderProgram overlayShaderProgram = + new OverlayShaderProgram( + context, + /* useHdr= */ true, + ImmutableList.of( + TextOverlay.createStaticTextOverlay(SpannableString.valueOf("overlay")))); + overlayShaderProgram.release(); + compiledResIds.add(R.raw.insert_overlay_fragment_shader_methods); + compiledResIds.add(R.raw.insert_ultra_hdr); + + // Verify that every R.raw.* resource in androidx.media3.effect.R.raw was compiled and has + // comments stripped. + Set allRawResIds = new HashSet<>(); + for (Field field : R.raw.class.getFields()) { + if (Modifier.isStatic(field.getModifiers()) && field.getType() == int.class) { + int resId = field.getInt(null); + allRawResIds.add(resId); + String shaderSource = Util.loadRawResource(context, resId); + assertWithMessage("Shader %s is empty", field.getName()).that(shaderSource).isNotEmpty(); + assertWithMessage("Shader %s still contains // comments", field.getName()) + .that(shaderSource) + .doesNotContain("//"); + assertWithMessage("Shader %s still contains /* comments", field.getName()) + .that(shaderSource) + .doesNotContain("/*"); + } + } + assertThat(compiledResIds).containsExactlyElementsIn(allRawResIds); + } + + private void compileGlProgram( + Set compiledResIds, + int vertexShaderResId, + int firstFragmentShaderResId, + int... additionalFragmentShaderResIds) + throws IOException, GlUtil.GlException { + compiledResIds.add(vertexShaderResId); + compiledResIds.add(firstFragmentShaderResId); + for (int resId : additionalFragmentShaderResIds) { + compiledResIds.add(resId); + } + GlProgram glProgram = + new GlProgram( + context, vertexShaderResId, firstFragmentShaderResId, additionalFragmentShaderResIds); + glProgram.delete(); + } + + private void compileExternalYuvGlProgram( + Set compiledResIds, int vertexShaderResId, int... fragmentShaderResIds) + throws IOException, GlUtil.GlException { + compiledResIds.add(vertexShaderResId); + StringBuilder fragmentShader = new StringBuilder(); + for (int resId : fragmentShaderResIds) { + compiledResIds.add(resId); + fragmentShader.append(Util.loadRawResource(context, resId)).append('\n'); + } + String fragmentShaderGlsl = fragmentShader.toString(); + if (!GlUtil.isYuvTargetExtensionSupported()) { + // On emulators without hardware GL_EXT_YUV_target support, substitute the external sampler + // extension and type so the rest of the stripped shader is still compiled and linked by the + // device's OpenGL ES 3.0 compiler. + fragmentShaderGlsl = + fragmentShaderGlsl + .replace( + "#extension GL_EXT_YUV_target : require", + "#extension GL_OES_EGL_image_external_essl3 : require") + .replace( + "#extension GL_EXT_YUV_target : enable", + "#extension GL_OES_EGL_image_external_essl3 : enable") + .replace("__samplerExternal2DY2YEXT", "samplerExternalOES"); + } + GlProgram glProgram = + new GlProgram(Util.loadRawResource(context, vertexShaderResId), fragmentShaderGlsl); + glProgram.delete(); + } +} diff --git a/libraries/effect_ndk/src/androidTest/java/androidx/media3/effect/ndk/HardwareBufferJniTest.java b/libraries/effect/src/androidTest/java/androidx/media3/effect/HardwareBufferJniTest.java similarity index 79% rename from libraries/effect_ndk/src/androidTest/java/androidx/media3/effect/ndk/HardwareBufferJniTest.java rename to libraries/effect/src/androidTest/java/androidx/media3/effect/HardwareBufferJniTest.java index c699bfdf1f3..43fb749cac2 100644 --- a/libraries/effect_ndk/src/androidTest/java/androidx/media3/effect/ndk/HardwareBufferJniTest.java +++ b/libraries/effect/src/androidTest/java/androidx/media3/effect/HardwareBufferJniTest.java @@ -13,7 +13,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package androidx.media3.effect.ndk; +package androidx.media3.effect; import static androidx.media3.test.utils.BitmapPixelTestUtil.getBitmapAveragePixelAbsoluteDifferenceArgb8888; import static com.google.common.truth.Truth.assertThat; @@ -70,7 +70,7 @@ public void nativeCopyBitmapToHardwareBuffer_sdr_matchesInputBitmap() throws Exc @Test @SdkSuppress(minSdkVersion = 33) - public void nativeCopyBitmapToHardwareBuffer_hdr_matchesInputBitmap() throws Exception { + public void nativeCopyBitmapToHardwareBuffer_rgba1010102_matchesInputBitmap() throws Exception { Bitmap inputBitmap = BitmapPixelTestUtil.readBitmap(FILE_PATH) .copy(Bitmap.Config.RGBA_1010102, /* isMutable= */ false); @@ -96,6 +96,35 @@ public void nativeCopyBitmapToHardwareBuffer_hdr_matchesInputBitmap() throws Exc } } + // TODO: b/545085046 - Investigate supporting RGBA_FP16 on API 30-32. + @Test + @SdkSuppress(minSdkVersion = 33) + public void nativeCopyBitmapToHardwareBuffer_f16_matchesInputBitmap() throws Exception { + Bitmap inputBitmap = + BitmapPixelTestUtil.readBitmap(FILE_PATH) + .copy(Bitmap.Config.RGBA_F16, /* isMutable= */ false); + try (HardwareBuffer hardwareBuffer = + HardwareBuffer.create( + inputBitmap.getWidth(), + inputBitmap.getHeight(), + HardwareBuffer.RGBA_FP16, + /* layers= */ 1, + HardwareBuffer.USAGE_CPU_WRITE_OFTEN | HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE)) { + assertThat( + HardwareBufferJni.INSTANCE.nativeCopyBitmapToHardwareBuffer( + inputBitmap, hardwareBuffer)) + .isTrue(); + Bitmap hardwareBitmap = + Bitmap.wrapHardwareBuffer(hardwareBuffer, ColorSpace.get(ColorSpace.Named.SRGB)); + assertThat(hardwareBitmap).isNotNull(); + Bitmap outputBitmap = hardwareBitmap.copy(Bitmap.Config.RGBA_F16, /* isMutable= */ false); + assertThat( + getBitmapAveragePixelAbsoluteDifferenceArgb8888( + inputBitmap, outputBitmap, testName.getMethodName())) + .isEqualTo(0); + } + } + @Test public void nativeCopyBitmapToHardwareBuffer_mismatchedDimensions_returnsFalse() throws Exception { @@ -188,8 +217,8 @@ public void nativeCopyHardwareBufferToHardwareBuffer_sdr_matchesInputBuffer() th } @Test - @SdkSuppress(maxSdkVersion = 32) - public void nativeCopyHardwareBufferToHardwareBuffer_hdrBelowApi33_returnsTrue() + @SdkSuppress(minSdkVersion = 28, maxSdkVersion = 32) + public void nativeCopyHardwareBufferToHardwareBuffer_rgba1010102BelowApi33_returnsTrue() throws Exception { // The methods to access the HDR bitmap do not exist below API 33, so just assert copying the // buffer formats succeeds. @@ -218,7 +247,8 @@ public void nativeCopyHardwareBufferToHardwareBuffer_hdrBelowApi33_returnsTrue() @Test @SdkSuppress(minSdkVersion = 33) - public void nativeCopyHardwareBufferToHardwareBuffer_hdr_matchesInputBuffer() throws Exception { + public void nativeCopyHardwareBufferToHardwareBuffer_rgba1010102_matchesInputBuffer() + throws Exception { Bitmap inputBitmap = BitmapPixelTestUtil.readBitmap(FILE_PATH) .copy(Bitmap.Config.RGBA_1010102, /* isMutable= */ false); @@ -256,6 +286,43 @@ public void nativeCopyHardwareBufferToHardwareBuffer_hdr_matchesInputBuffer() th } } + // TODO: b/545085046 - Investigate supporting RGBA_FP16 on API 30-32. + @Test + @SdkSuppress(minSdkVersion = 33) + public void nativeCopyHardwareBufferToHardwareBuffer_f16_matchesInputBuffer() throws Exception { + Bitmap inputBitmap = + BitmapPixelTestUtil.readBitmap(FILE_PATH) + .copy(Bitmap.Config.RGBA_F16, /* isMutable= */ false); + try (HardwareBuffer srcHb = + HardwareBuffer.create( + inputBitmap.getWidth(), + inputBitmap.getHeight(), + HardwareBuffer.RGBA_FP16, + /* layers= */ 1, + HardwareBuffer.USAGE_CPU_WRITE_OFTEN | HardwareBuffer.USAGE_CPU_READ_OFTEN); + HardwareBuffer dstHb = + HardwareBuffer.create( + inputBitmap.getWidth(), + inputBitmap.getHeight(), + HardwareBuffer.RGBA_FP16, + /* layers= */ 1, + HardwareBuffer.USAGE_CPU_WRITE_OFTEN | HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE)) { + assertThat(HardwareBufferJni.INSTANCE.nativeCopyBitmapToHardwareBuffer(inputBitmap, srcHb)) + .isTrue(); + assertThat(HardwareBufferJni.INSTANCE.nativeCopyHardwareBufferToHardwareBuffer(srcHb, dstHb)) + .isTrue(); + + Bitmap hardwareBitmap = + Bitmap.wrapHardwareBuffer(dstHb, ColorSpace.get(ColorSpace.Named.SRGB)); + assertThat(hardwareBitmap).isNotNull(); + Bitmap outputBitmap = hardwareBitmap.copy(Bitmap.Config.RGBA_F16, /* isMutable= */ false); + assertThat( + getBitmapAveragePixelAbsoluteDifferenceArgb8888( + inputBitmap, outputBitmap, testName.getMethodName())) + .isEqualTo(0); + } + } + @Test public void nativeCopyHardwareBufferToHardwareBuffer_sameSourceAndDestination_returnsTrue() throws Exception { diff --git a/libraries/effect/src/androidTest/java/androidx/media3/effect/HardwareBufferToGlTextureConverterTest.java b/libraries/effect/src/androidTest/java/androidx/media3/effect/HardwareBufferToGlTextureConverterTest.java index 016965a0b3d..c265d098d31 100644 --- a/libraries/effect/src/androidTest/java/androidx/media3/effect/HardwareBufferToGlTextureConverterTest.java +++ b/libraries/effect/src/androidTest/java/androidx/media3/effect/HardwareBufferToGlTextureConverterTest.java @@ -16,49 +16,72 @@ package androidx.media3.effect; import static androidx.media3.common.util.Util.isRunningOnEmulator; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT2020_HLG; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT2020_LINEAR; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT709_LINEAR; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT709_SRGB; import static androidx.media3.effect.FrameProcessorUtils.releaseOpenGl; import static androidx.media3.effect.FrameProcessorUtils.setupOpenGl; import static androidx.media3.effect.FrameProcessorUtils.shutdownGlExecutorService; +import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_COLOR_TEST_720P_STRIP_HLG; import static androidx.media3.test.utils.BitmapPixelTestUtil.createArgb8888BitmapFromFocusedGlFramebuffer; +import static androidx.media3.test.utils.BitmapPixelTestUtil.createArgb8888BitmapWithSolidColor; +import static androidx.media3.test.utils.BitmapPixelTestUtil.createFp16BitmapFromFocusedGlFramebuffer; import static androidx.media3.test.utils.BitmapPixelTestUtil.getBitmapAveragePixelAbsoluteDifferenceArgb8888; +import static androidx.media3.test.utils.BitmapPixelTestUtil.readBitmap; +import static androidx.media3.test.utils.HdrCapabilitiesUtil.assumeDeviceSupportsOpenGlToneMapping; +import static androidx.media3.test.utils.TestUtil.assertFp16BitmapsAreSimilar; import static androidx.test.core.app.ApplicationProvider.getApplicationContext; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.truth.Truth.assertThat; import static com.google.common.util.concurrent.MoreExecutors.directExecutor; import static com.google.common.util.concurrent.MoreExecutors.listeningDecorator; import static java.util.concurrent.TimeUnit.MILLISECONDS; +import static org.junit.Assume.assumeTrue; import android.content.Context; import android.graphics.Bitmap; +import android.graphics.Color; +import android.graphics.Gainmap; import android.graphics.ImageFormat; import android.graphics.Matrix; import android.hardware.HardwareBuffer; import android.media.Image; import android.media.ImageReader; import android.media.MediaFormat; +import android.opengl.GLES30; +import android.util.Half; import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; import androidx.media3.common.C; import androidx.media3.common.ColorInfo; import androidx.media3.common.Format; import androidx.media3.common.GlObjectsProvider; import androidx.media3.common.MediaItem; +import androidx.media3.common.MimeTypes; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.GlUtil; import androidx.media3.common.util.GlUtil.GlException; import androidx.media3.common.util.MediaFormatUtil; +import androidx.media3.common.util.Util; import androidx.media3.common.video.DefaultHardwareBufferFrame; import androidx.media3.common.video.Frame; import androidx.media3.common.video.FrameProcessor; import androidx.media3.common.video.HardwareBufferFrame; import androidx.media3.common.video.SyncFenceWrapper; -import androidx.media3.effect.ndk.HardwareBufferJni; import androidx.media3.test.utils.AssetInfo; import androidx.media3.test.utils.BitmapPixelTestUtil; import androidx.media3.test.utils.DecodeOneFrameUtil; -import androidx.test.ext.junit.runners.AndroidJUnit4; +import androidx.media3.test.utils.TestUtil; import androidx.test.filters.SdkSuppress; +import com.google.common.collect.ImmutableList; import com.google.common.util.concurrent.ListeningExecutorService; -import java.util.concurrent.CountDownLatch; +import com.google.testing.junit.testparameterinjector.TestParameter; +import com.google.testing.junit.testparameterinjector.TestParameterInjector; +import com.google.testing.junit.testparameterinjector.TestParameterValuesProvider; +import java.nio.ByteBuffer; +import java.nio.ByteOrder; +import java.util.Arrays; import java.util.concurrent.Executors; import java.util.concurrent.atomic.AtomicReference; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; @@ -70,15 +93,283 @@ import org.junit.runner.RunWith; /** Instrumentation tests for {@link HardwareBufferToGlTextureConverter}. */ -@RunWith(AndroidJUnit4.class) +@RunWith(TestParameterInjector.class) @SdkSuppress(minSdkVersion = 29) public final class HardwareBufferToGlTextureConverterTest { @Rule public final TestName testName = new TestName(); private static final AssetInfo TEST_VIDEO_ASSET = AssetInfo.MP4_ASSET_WITH_INCREASING_TIMESTAMPS; + private static final ColorInfo PQ_INPUT_COLOR = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorTransfer(C.COLOR_TRANSFER_ST2084) + .setColorRange(C.COLOR_RANGE_FULL) + .build(); + private static final ColorInfo HLG_INPUT_COLOR = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorTransfer(C.COLOR_TRANSFER_HLG) + .setColorRange(C.COLOR_RANGE_FULL) + .build(); private static final float MAX_PIXEL_DIFFERENCE = 10.f; private static final long TEST_TIMEOUT_MS = isRunningOnEmulator() ? 20_000L : 10_000L; private static final long FENCE_TIMEOUT_MS = 1_000L; + private static final int SOLID_COLOR_BITMAP_SIZE = 64; + private static final ImmutableList SOLID_COLOR_TEST_CASES = + ImmutableList.of( + new SolidColorTestCase( + /* name= */ "SRGB_BLACK", + /* inputSrgbColor= */ Color.BLACK, + /* expectedHlgRgb= */ Color.valueOf(0.0f, 0.0f, 0.0f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.0f, 0.0f, 0.0f)), + new SolidColorTestCase( + /* name= */ "SRGB_WHITE", + /* inputSrgbColor= */ Color.WHITE, + /* expectedHlgRgb= */ Color.valueOf(0.75f, 0.75f, 0.75f), + /* expectedBt2020LinearRgb= */ Color.valueOf(1.0f, 1.0f, 1.0f)), + new SolidColorTestCase( + /* name= */ "SRGB_GRAY_128", + /* inputSrgbColor= */ Color.rgb(128, 128, 128), + /* expectedHlgRgb= */ Color.valueOf(0.4707f, 0.4707f, 0.4707f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.2159f, 0.2159f, 0.2159f)), + new SolidColorTestCase( + /* name= */ "SRGB_RED", + /* inputSrgbColor= */ Color.RED, + /* expectedHlgRgb= */ Color.valueOf(0.7087f, 0.2666f, 0.1298f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.6274f, 0.0691f, 0.0164f)), + new SolidColorTestCase( + /* name= */ "SRGB_GREEN", + /* inputSrgbColor= */ Color.GREEN, + /* expectedHlgRgb= */ Color.valueOf(0.5250f, 0.7445f, 0.2720f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.3293f, 0.9195f, 0.0880f)), + new SolidColorTestCase( + /* name= */ "SRGB_BLUE", + /* inputSrgbColor= */ Color.BLUE, + /* expectedHlgRgb= */ Color.valueOf(0.2309f, 0.1183f, 0.8135f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.0433f, 0.0114f, 0.8956f))); + + private static final ImmutableList PQ_SOLID_COLOR_TEST_CASES = + ImmutableList.of( + new SolidColorTestCase( + /* name= */ "PQ_BLACK", + /* inputPqColor= */ Color.valueOf(0.0f, 0.0f, 0.0f), + /* expectedHlgRgb= */ Color.valueOf(0.0f, 0.0f, 0.0f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.0f, 0.0f, 0.0f), + /* expectedSdrLinearRgb= */ Color.valueOf(0.0f, 0.0f, 0.0f)), + // Diffuse White (203 nits): the linear working space is anchored here, so Bt2020 Linear + // is 1.0. HLG = 0.7501. For SDR Linear, 202.8 nits is above the 118.9-nit inflection + // point and rolls off to 154.18 nits -> 154.18 / 203.1521 = 0.7589. + new SolidColorTestCase( + /* name= */ "PQ_DIFFUSE_WHITE", + /* inputPqColor= */ Color.valueOf(0.5807f, 0.5807f, 0.5807f), + /* expectedHlgRgb= */ Color.valueOf(0.7501f, 0.7501f, 0.7501f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.9996f, 0.9996f, 0.9996f), + /* expectedSdrLinearRgb= */ Color.valueOf(0.7589f, 0.7589f, 0.7589f)), + // 500 nits (0.6766 in PQ -> 500.8 nits): in display-referred linear space (1.0 = 203.15 + // nits), 500.0 / 203.1521 = 2.4612. For SDR Linear, BT.2446 Method C compresses 500.8 + // nits (above the 118.9-nit inflection point) to 184.73 nits -> 184.73 / 203.1521 = + // 0.9093. + new SolidColorTestCase( + /* name= */ "PQ_FIVE_HUNDRED_NITS", + /* inputPqColor= */ Color.valueOf(0.6766f, 0.6766f, 0.6766f), + /* expectedHlgRgb= */ Color.valueOf(0.8935f, 0.8935f, 0.8935f), + /* expectedBt2020LinearRgb= */ Color.valueOf(2.4612f, 2.4612f, 2.4612f), + /* expectedSdrLinearRgb= */ Color.valueOf(0.9093f, 0.9093f, 0.9093f)), + new SolidColorTestCase( + /* name= */ "PQ_ONE_THOUSAND_NITS", + /* inputPqColor= */ Color.valueOf(0.7518f, 0.7518f, 0.7518f), + /* expectedHlgRgb= */ Color.valueOf(1.0f, 1.0f, 1.0f), + /* expectedBt2020LinearRgb= */ Color.valueOf(4.9216f, 4.9216f, 4.9216f), + /* expectedSdrLinearRgb= */ Color.valueOf(1.0f, 1.0f, 1.0f)), + new SolidColorTestCase( + /* name= */ "PQ_TEN_THOUSAND_NITS", + /* inputPqColor= */ Color.valueOf(1.0f, 1.0f, 1.0f), + /* expectedHlgRgb= */ Color.valueOf(1.0f, 1.0f, 1.0f), + /* expectedBt2020LinearRgb= */ Color.valueOf(49.2242f, 49.2242f, 49.2242f), + /* expectedSdrLinearRgb= */ Color.valueOf(1.0f, 1.0f, 1.0f)), + new SolidColorTestCase( + /* name= */ "PQ_PRIMARY_RED", + /* inputPqColor= */ Color.valueOf(1.0f, 0.0f, 0.0f), + /* expectedHlgRgb= */ Color.valueOf(1.0f, 0.0f, 0.0f), + /* expectedBt2020LinearRgb= */ Color.valueOf(49.2242f, 0.0f, 0.0f), + /* expectedSdrLinearRgb= */ Color.valueOf(1.0000f, 0.0122f, 0.0213f)), + new SolidColorTestCase( + /* name= */ "PQ_PRIMARY_GREEN", + /* inputPqColor= */ Color.valueOf(0.0f, 1.0f, 0.0f), + /* expectedHlgRgb= */ Color.valueOf(0.0f, 1.0f, 0.0f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.0f, 49.2242f, 0.0f), + /* expectedSdrLinearRgb= */ Color.valueOf(0.0005f, 1.0000f, 0.0110f)), + new SolidColorTestCase( + /* name= */ "PQ_PRIMARY_BLUE", + /* inputPqColor= */ Color.valueOf(0.0f, 0.0f, 1.0f), + /* expectedHlgRgb= */ Color.valueOf(0.0f, 0.0f, 1.0f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.0f, 0.0f, 49.2242f), + /* expectedSdrLinearRgb= */ Color.valueOf(0.0131f, 0.0221f, 1.0000f)), + // Tests gamut compression prior to tone mapping avoids clipping at 1.0: + // Converting 400-nit BT.2020 primary red to BT.709 coordinates multiplies the red channel + // by the matrix coefficient 1.6605 (400 * 1.6605 = 664.2 nits). Running ACES 1.3 soft + // gamut compression prior to tone mapping allows the tone curve to see this ~665.8-nit + // peak channel and smoothly roll it off to 192.57 nits (192.57 / 203.1521 = 0.9479) below + // the 203.15-nit ceiling, whereas tone mapping before gamut conversion would hard-clip at + // 1.0 (178.2 * 1.6605 = 295.9 nits > 203.15). + new SolidColorTestCase( + /* name= */ "PQ_PRIMARY_RED_400_NITS", + /* inputPqColor= */ Color.valueOf(0.6526f, 0.0f, 0.0f), + /* expectedHlgRgb= */ Color.valueOf(0.9002f, 0.0f, 0.0f), + /* expectedBt2020LinearRgb= */ Color.valueOf(1.9694f, 0.0f, 0.0f), + /* expectedSdrLinearRgb= */ Color.valueOf(0.9479f, 0.0116f, 0.0202f)), + // Tests 100% saturated BT.709 primary red inside a PQ container (203-nit diffuse white + // red) is softly compressed above the 95% boundary knee (d = 1.0 > 0.95). + new SolidColorTestCase( + /* name= */ "PQ_BT709_RED_PRIMARY_COMPRESSED", + /* inputPqColor= */ Color.valueOf(0.5325f, 0.3270f, 0.2201f), + /* expectedHlgRgb= */ Color.valueOf(0.7085f, 0.2665f, 0.1299f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.6267f, 0.0690f, 0.0164f), + /* expectedSdrLinearRgb= */ Color.valueOf(0.7596f, 0.0182f, 0.0182f)), + // Tests in-gamut BT.709 color within 95% saturation (BT.709 linear (1, 0.25, 0.25) at 203 + // nits, d = 0.75 <= 0.95) passes through gamut compression with zero distortion. + new SolidColorTestCase( + /* name= */ "PQ_BT709_IN_GAMUT_DESATURATED_RED", + /* inputPqColor= */ Color.valueOf(0.5467f, 0.4598f, 0.4463f), + /* expectedHlgRgb= */ Color.valueOf(0.7144f, 0.5263f, 0.4918f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.7199f, 0.3017f, 0.2622f), + /* expectedSdrLinearRgb= */ Color.valueOf(0.7596f, 0.1893f, 0.1895f)), + // Mid-gray (~100 nits, 0.5081 in PQ -> 100.22 nits): below the 118.9-nit BT.2446 Method C + // inflection point (k1 = 1.0), so it passes through at 1:1 nits with 203.15-nit SDR + // reference white (expectedSdrLinearRgb == expectedBt2020LinearRgb = 100.22 / 203.1521 = + // 0.4933). + new SolidColorTestCase( + /* name= */ "PQ_MID_GRAY_100_NITS", + /* inputPqColor= */ Color.valueOf(0.5081f, 0.5081f, 0.5081f), + /* expectedHlgRgb= */ Color.valueOf(0.6300f, 0.6300f, 0.6300f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.4933f, 0.4933f, 0.4933f), + /* expectedSdrLinearRgb= */ Color.valueOf(0.4933f, 0.4933f, 0.4933f))); + + private static final ImmutableList HLG_SOLID_COLOR_TEST_CASES = + ImmutableList.of( + new SolidColorTestCase( + /* name= */ "HLG_BLACK", + /* inputHlgColor= */ Color.valueOf(0.0f, 0.0f, 0.0f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.0f, 0.0f, 0.0f), + /* expectedSdrLinearRgb= */ Color.valueOf(0.0f, 0.0f, 0.0f)), + // Diffuse White (203 nits): 75% HLG signal represents 18% scene reflectance, mapping to + // 203.15 nits on a 1,000-nit reference display and 1.0 in diffuse-white linear working + // space. For SDR Linear, 203.15 nits is above the 118.9-nit inflection point and rolls + // off to 154.3 nits; dividing by the 203.1521-nit SDR peak yields 154.26 / 203.1521 = + // 0.7593. + new SolidColorTestCase( + /* name= */ "HLG_DIFFUSE_WHITE", + /* inputHlgColor= */ Color.valueOf(0.75f, 0.75f, 0.75f), + /* expectedBt2020LinearRgb= */ Color.valueOf(1.0f, 1.0f, 1.0f), + /* expectedSdrLinearRgb= */ Color.valueOf(0.7593f, 0.7593f, 0.7593f)), + // Peak White (1000 nits): 100% HLG signal maps to 1000 nits (4.9224 in diffuse-white + // working space), which BT.2446 Method C tone maps to 203.1521 nits peak SDR = 1.0. + new SolidColorTestCase( + /* name= */ "HLG_PEAK_WHITE", + /* inputHlgColor= */ Color.valueOf(1.0f, 1.0f, 1.0f), + /* expectedBt2020LinearRgb= */ Color.valueOf(4.9224f, 4.9224f, 4.9224f), + /* expectedSdrLinearRgb= */ Color.valueOf(1.0f, 1.0f, 1.0f)), + new SolidColorTestCase( + /* name= */ "HLG_PRIMARY_RED", + /* inputHlgColor= */ Color.valueOf(1.0f, 0.0f, 0.0f), + /* expectedBt2020LinearRgb= */ Color.valueOf(3.7677f, 0.0f, 0.0f), + /* expectedSdrLinearRgb= */ Color.valueOf(1.0000f, 0.0122f, 0.0213f)), + new SolidColorTestCase( + /* name= */ "HLG_PRIMARY_GREEN", + /* inputHlgColor= */ Color.valueOf(0.0f, 1.0f, 0.0f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.0f, 4.5543f, 0.0f), + /* expectedSdrLinearRgb= */ Color.valueOf(0.0005f, 1.0000f, 0.0110f)), + new SolidColorTestCase( + /* name= */ "HLG_PRIMARY_BLUE", + /* inputHlgColor= */ Color.valueOf(0.0f, 0.0f, 1.0f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.0f, 0.0f, 2.7976f), + /* expectedSdrLinearRgb= */ Color.valueOf(0.0124f, 0.0208f, 0.9418f)), + // Tests gamut compression prior to tone mapping avoids clipping at 1.0: + // Converting 400-nit BT.2020 primary red to BT.709 coordinates multiplies the red channel + // by the matrix coefficient 1.6605 (400 * 1.6605 = 664.2 nits). Running ACES 1.3 soft + // gamut compression prior to tone mapping allows the tone curve to see this ~665.2-nit + // peak channel and smoothly roll it off to 192.55 nits (192.55 / 203.1521 = 0.9478) below + // the 203.15-nit ceiling, whereas tone mapping before gamut conversion would hard-clip at + // 1.0 (178.2 * 1.6605 = 295.9 nits > 203.15). + new SolidColorTestCase( + /* name= */ "HLG_PRIMARY_RED_400_NITS", + /* inputHlgColor= */ Color.valueOf(0.9002f, 0.0f, 0.0f), + /* expectedBt2020LinearRgb= */ Color.valueOf(1.9697f, 0.0f, 0.0f), + /* expectedSdrLinearRgb= */ Color.valueOf(0.9478f, 0.0116f, 0.0202f)), + // Tests 100% saturated BT.709 primary red inside an HLG container (203-nit diffuse white + // red) is softly compressed above the 95% boundary knee (d = 1.0 > 0.95). + new SolidColorTestCase( + /* name= */ "HLG_BT709_RED_PRIMARY_COMPRESSED", + /* inputHlgColor= */ Color.valueOf(0.7085f, 0.2665f, 0.1299f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.6268f, 0.0690f, 0.0164f), + /* expectedSdrLinearRgb= */ Color.valueOf(0.7591f, 0.0182f, 0.0182f)), + // Tests in-gamut BT.709 color within 95% saturation (BT.709 linear (1, 0.25, 0.25) at 203 + // nits, d = 0.75 <= 0.95) passes through gamut compression with zero distortion. + new SolidColorTestCase( + /* name= */ "HLG_BT709_IN_GAMUT_DESATURATED_RED", + /* inputHlgColor= */ Color.valueOf(0.7144f, 0.5263f, 0.4918f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.7200f, 0.3017f, 0.2622f), + /* expectedSdrLinearRgb= */ Color.valueOf(0.7591f, 0.1900f, 0.1898f)), + // Mid-gray (~100 nits, 0.6296 in HLG -> 99.87 nits): below the 118.9-nit BT.2446 Method C + // inflection point (k1 = 1.0), so it passes through at 1:1 nits with 203.15-nit SDR + // reference white (expectedSdrLinearRgb == expectedBt2020LinearRgb = 99.87 / 203.1521 = + // 0.4916). + new SolidColorTestCase( + /* name= */ "HLG_MID_GRAY_100_NITS", + /* inputHlgColor= */ Color.valueOf(0.6296f, 0.6296f, 0.6296f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.4916f, 0.4916f, 0.4916f), + /* expectedSdrLinearRgb= */ Color.valueOf(0.4916f, 0.4916f, 0.4916f))); + + private static final ImmutableList ULTRA_HDR_SOLID_COLOR_TEST_CASES = + ImmutableList.of( + // Unboosted SDR diffuse white (gainmap alpha = 0 -> 1.0x gain = 203.15 nits): + // Maps to 1.0 in BT2020_LINEAR and 0.75 in BT2020_HLG. + new SolidColorTestCase( + /* name= */ "ULTRA_HDR_WHITE_UNBOOSTED", + /* inputSrgbColor= */ Color.WHITE, + /* gainmapAlpha= */ 0, + /* expectedHlgRgb= */ Color.valueOf(0.7500f, 0.7500f, 0.7500f), + /* expectedBt2020LinearRgb= */ Color.valueOf(1.0000f, 1.0000f, 1.0000f)), + // Mid-boosted SDR diffuse white (gainmap alpha = 128 -> g = 128/255 = 0.501961 -> 2.2256x + // gain): + new SolidColorTestCase( + /* name= */ "ULTRA_HDR_WHITE_MID_BOOST", + /* inputSrgbColor= */ Color.WHITE, + /* gainmapAlpha= */ 128, + /* expectedHlgRgb= */ Color.valueOf(0.8776f, 0.8776f, 0.8776f), + /* expectedBt2020LinearRgb= */ Color.valueOf(2.2256f, 2.2256f, 2.2256f)), + // Peak-boosted SDR diffuse white (gainmap alpha = 255 -> 4.9224x gain = 1,000 nits): + // Maps to 4.9224 in BT2020_LINEAR and 1.0 in BT2020_HLG. + new SolidColorTestCase( + /* name= */ "ULTRA_HDR_WHITE_PEAK_BOOST", + /* inputSrgbColor= */ Color.WHITE, + /* gainmapAlpha= */ 255, + /* expectedHlgRgb= */ Color.valueOf(1.0000f, 1.0000f, 1.0000f), + /* expectedBt2020LinearRgb= */ Color.valueOf(4.9224f, 4.9224f, 4.9224f)), + // Peak-boosted BT.709 primary red (gainmap alpha = 255 -> 4.9224x gain = 1,000-nit BT.709 + // red): + new SolidColorTestCase( + /* name= */ "ULTRA_HDR_RED_PEAK_BOOST", + /* inputSrgbColor= */ Color.RED, + /* gainmapAlpha= */ 255, + /* expectedHlgRgb= */ Color.valueOf(0.9618f, 0.5174f, 0.2523f), + /* expectedBt2020LinearRgb= */ Color.valueOf(3.0883f, 0.3401f, 0.0807f)), + // Peak-boosted BT.709 primary green (gainmap alpha = 255 -> 4.9224x gain = 1,000-nit + // BT.709 green): + new SolidColorTestCase( + /* name= */ "ULTRA_HDR_GREEN_PEAK_BOOST", + /* inputSrgbColor= */ Color.GREEN, + /* gainmapAlpha= */ 255, + /* expectedHlgRgb= */ Color.valueOf(0.8030f, 0.9949f, 0.5271f), + /* expectedBt2020LinearRgb= */ Color.valueOf(1.6209f, 4.5264f, 0.4332f)), + // Peak-boosted BT.709 primary blue (gainmap alpha = 255 -> 4.9224x gain = 1,000-nit + // BT.709 blue): + new SolidColorTestCase( + /* name= */ "ULTRA_HDR_BLUE_PEAK_BOOST", + /* inputSrgbColor= */ Color.BLUE, + /* gainmapAlpha= */ 255, + /* expectedHlgRgb= */ Color.valueOf(0.3809f, 0.1951f, 1.0000f), + /* expectedBt2020LinearRgb= */ Color.valueOf(0.2132f, 0.0559f, 4.4085f))); private final Context context = getApplicationContext(); @@ -122,30 +413,12 @@ private void tearDownInternal() { @SdkSuppress(minSdkVersion = 31) @Test - public void convert_withARGB8888HardwareBuffer_outputsCorrectGlTexture() throws Exception { + public void convert_withArgb8888HardwareBuffer_outputsCorrectGlTexture() throws Exception { Bitmap expectedBitmap = BitmapPixelTestUtil.readBitmap("media/png/first_frame_1920x1080.png"); Bitmap hardwareBitmap = expectedBitmap.copy(Bitmap.Config.HARDWARE, /* isMutable= */ false); HardwareBuffer hardwareBuffer = hardwareBitmap.getHardwareBuffer(); - AtomicReference completedFrame = new AtomicReference<>(); - FrameProcessor.Listener listener = - new FrameProcessor.Listener() { - @Override - public void onWakeup() {} - - @Override - public void onError(VideoFrameProcessingException exception) {} - - @Override - public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper releaseFence) { - if (releaseFence != null) { - assertThat(releaseFence.awaitMs(FENCE_TIMEOUT_MS)).isTrue(); - releaseFence.close(); - } - hardwareBuffer.close(); - completedFrame.set(frame); - } - }; + TestFrameProcessorListener listener = new TestFrameProcessorListener(hardwareBuffer); int width = expectedBitmap.getWidth(); int height = expectedBitmap.getHeight(); @@ -165,7 +438,7 @@ public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper releaseFenc getBitmapAveragePixelAbsoluteDifferenceArgb8888( expectedBitmap, actualBitmap, testName.getMethodName())) .isLessThan(MAX_PIXEL_DIFFERENCE); - assertThat(completedFrame.get()).isSameInstanceAs(hardwareBufferFrame); + assertThat(listener.completedFrame.get()).isSameInstanceAs(hardwareBufferFrame); } @SdkSuppress(minSdkVersion = 29) @@ -174,17 +447,85 @@ public void convert_withYuv420HardwareBuffer_outputsCorrectGlTexture() throws Ex int width = TEST_VIDEO_ASSET.videoFormat.width; int height = TEST_VIDEO_ASSET.videoFormat.height; - ImageReader inputImageReader = + try (ImageReader inputImageReader = ImageReader.newInstance( width, height, ImageFormat.YUV_420_888, /* maxImages= */ 1, + HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE)) { + AtomicReference inputMediaFormat = new AtomicReference<>(); + DecodeOneFrameUtil.decodeOneMediaItemFrame( + MediaItem.fromUri(AssetInfo.MP4_ASSET_WITH_INCREASING_TIMESTAMPS.uri), + new DecodeOneFrameUtil.Listener() { + @Override + public void onContainerExtracted(MediaFormat mediaFormat) {} + + @Override + public void onFrameDecoded(MediaFormat mediaFormat) { + inputMediaFormat.set(mediaFormat); + } + }, + inputImageReader.getSurface()); + + Image inputImage = checkNotNull(inputImageReader.acquireLatestImage()); + HardwareBuffer inputHardwareBuffer = checkNotNull(inputImage.getHardwareBuffer()); + + // Override the input format to force it to be 1920x1080 rather than 1920x1088. + Format inputFormat = + MediaFormatUtil.createFormatFromMediaFormat(inputMediaFormat.get()) + .buildUpon() + .setWidth(width) + .setHeight(height) + .build(); + + ColorInfo outputColorInfo = + inputFormat.colorInfo != null ? inputFormat.colorInfo : ColorInfo.SDR_BT709_LIMITED; + converter = + new HardwareBufferToGlTextureConverter( + context, HardwareBufferJni.INSTANCE, outputColorInfo, e -> {}); + + TestFrameProcessorListener listener = + new TestFrameProcessorListener(inputHardwareBuffer, inputImage); + + HardwareBufferFrame inputHardwareBufferFrame = + new DefaultHardwareBufferFrame.Builder(inputHardwareBuffer) + .setFormat(inputFormat) + .build(); + Bitmap expectedBitmap = BitmapPixelTestUtil.readBitmap("media/png/first_frame_1920x1080.png"); + + Bitmap actualBitmap = convertAndCaptureBitmap(inputHardwareBufferFrame, listener); + + assertThat( + getBitmapAveragePixelAbsoluteDifferenceArgb8888( + expectedBitmap, actualBitmap, testName.getMethodName())) + .isLessThan(MAX_PIXEL_DIFFERENCE); + assertThat(listener.completedFrame.get()).isSameInstanceAs(inputHardwareBufferFrame); + } + } + + @SdkSuppress(minSdkVersion = 31) + @Test + public void convert_withHlgHardwareBufferAndToneMapping_outputsCorrectGlTexture() + throws Exception { + assumeDeviceSupportsOpenGlToneMapping( + testName.getMethodName(), MP4_ASSET_COLOR_TEST_720P_STRIP_HLG.videoFormat); + + int width = MP4_ASSET_COLOR_TEST_720P_STRIP_HLG.videoFormat.width; + int height = MP4_ASSET_COLOR_TEST_720P_STRIP_HLG.videoFormat.height; + + ImageReader inputImageReader = + ImageReader.newInstance( + width, + height, + // Make sure ImageReader reads 10 bit YUV data. + ImageFormat.YCBCR_P010, + /* maxImages= */ 1, HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE); AtomicReference inputMediaFormat = new AtomicReference<>(); DecodeOneFrameUtil.decodeOneMediaItemFrame( - MediaItem.fromUri(AssetInfo.MP4_ASSET_WITH_INCREASING_TIMESTAMPS.uri), + MediaItem.fromUri(MP4_ASSET_COLOR_TEST_720P_STRIP_HLG.uri), new DecodeOneFrameUtil.Listener() { @Override public void onContainerExtracted(MediaFormat mediaFormat) {} @@ -199,7 +540,6 @@ public void onFrameDecoded(MediaFormat mediaFormat) { Image inputImage = checkNotNull(inputImageReader.acquireLatestImage()); HardwareBuffer inputHardwareBuffer = checkNotNull(inputImage.getHardwareBuffer()); - // Override the input format to force it to be 1920x1080 rather than 1920x1088. Format inputFormat = MediaFormatUtil.createFormatFromMediaFormat(inputMediaFormat.get()) .buildUpon() @@ -207,36 +547,25 @@ public void onFrameDecoded(MediaFormat mediaFormat) { .setHeight(height) .build(); - ColorInfo outputColorInfo = - inputFormat.colorInfo != null ? inputFormat.colorInfo : ColorInfo.SDR_BT709_LIMITED; + assumeTrue(GlUtil.isYuvTargetExtensionSupported()); + converter = new HardwareBufferToGlTextureConverter( - context, HardwareBufferJni.INSTANCE, outputColorInfo, e -> {}); - - AtomicReference completedFrame = new AtomicReference<>(); - FrameProcessor.Listener listener = - new FrameProcessor.Listener() { - @Override - public void onWakeup() {} - - @Override - public void onError(VideoFrameProcessingException exception) {} + context, + HardwareBufferJni.INSTANCE, + /* outputColorInfo= */ BT709_SRGB, + e -> { + throw new AssertionError(e); + }); - @Override - public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper releaseFence) { - if (releaseFence != null) { - assertThat(releaseFence.awaitMs(FENCE_TIMEOUT_MS)).isTrue(); - releaseFence.close(); - } - inputHardwareBuffer.close(); - inputImage.close(); - completedFrame.set(frame); - } - }; + TestFrameProcessorListener listener = + new TestFrameProcessorListener(inputHardwareBuffer, inputImage); HardwareBufferFrame inputHardwareBufferFrame = new DefaultHardwareBufferFrame.Builder(inputHardwareBuffer).setFormat(inputFormat).build(); - Bitmap expectedBitmap = BitmapPixelTestUtil.readBitmap("media/png/first_frame_1920x1080.png"); + Bitmap expectedBitmap = + BitmapPixelTestUtil.readBitmap( + "test-generated-goldens/sample_mp4_first_frame/electrical_colors/hlg_1000nit_strip_tonemap_with_gamut_compression.png"); Bitmap actualBitmap = convertAndCaptureBitmap(inputHardwareBufferFrame, listener); @@ -244,7 +573,7 @@ public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper releaseFenc getBitmapAveragePixelAbsoluteDifferenceArgb8888( expectedBitmap, actualBitmap, testName.getMethodName())) .isLessThan(MAX_PIXEL_DIFFERENCE); - assertThat(completedFrame.get()).isSameInstanceAs(inputHardwareBufferFrame); + assertThat(listener.completedFrame.get()).isSameInstanceAs(inputHardwareBufferFrame); inputImageReader.close(); } @@ -268,25 +597,7 @@ public void convert_withRgba8888HardwareBufferAndColorConversion_outputsCorrectG new HardwareBufferToGlTextureConverter( context, HardwareBufferJni.INSTANCE, differentSdrColorInfo, e -> {}); - AtomicReference completedFrame = new AtomicReference<>(); - FrameProcessor.Listener listener = - new FrameProcessor.Listener() { - @Override - public void onWakeup() {} - - @Override - public void onError(VideoFrameProcessingException exception) {} - - @Override - public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper releaseFence) { - if (releaseFence != null) { - assertThat(releaseFence.awaitMs(FENCE_TIMEOUT_MS)).isTrue(); - releaseFence.close(); - } - hardwareBuffer.close(); - completedFrame.set(frame); - } - }; + TestFrameProcessorListener listener = new TestFrameProcessorListener(hardwareBuffer); Format inputFormat = new Format.Builder() @@ -304,55 +615,455 @@ public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper releaseFenc getBitmapAveragePixelAbsoluteDifferenceArgb8888( expectedBitmap, actualBitmap, testName.getMethodName())) .isLessThan(MAX_PIXEL_DIFFERENCE); - assertThat(completedFrame.get()).isSameInstanceAs(inputHardwareBufferFrame); + assertThat(listener.completedFrame.get()).isSameInstanceAs(inputHardwareBufferFrame); + } + + @SdkSuppress(minSdkVersion = 29) + @Test + public void convert_withYuv420Bt601HardwareBuffer_convertsToBt709GlTexture() throws Exception { + int width = AssetInfo.BT601_MP4_ASSET.videoFormat.width; + int height = AssetInfo.BT601_MP4_ASSET.videoFormat.height; + + ImageReader inputImageReader = + ImageReader.newInstance( + width, + height, + ImageFormat.YUV_420_888, + /* maxImages= */ 1, + HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE); + + AtomicReference inputMediaFormat = new AtomicReference<>(); + DecodeOneFrameUtil.decodeOneMediaItemFrame( + MediaItem.fromUri(AssetInfo.BT601_MP4_ASSET.uri), + new DecodeOneFrameUtil.Listener() { + @Override + public void onContainerExtracted(MediaFormat mediaFormat) {} + + @Override + public void onFrameDecoded(MediaFormat mediaFormat) { + inputMediaFormat.set(mediaFormat); + } + }, + inputImageReader.getSurface()); + + Image inputImage = checkNotNull(inputImageReader.acquireLatestImage()); + HardwareBuffer inputHardwareBuffer = checkNotNull(inputImage.getHardwareBuffer()); + + Format inputFormat = + MediaFormatUtil.createFormatFromMediaFormat(inputMediaFormat.get()) + .buildUpon() + .setWidth(width) + .setHeight(height) + .build(); + + converter = + new HardwareBufferToGlTextureConverter( + context, + HardwareBufferJni.INSTANCE, + /* outputColorInfo= */ BT709_SRGB, + /* errorConsumer= */ e -> { + throw new AssertionError(e); + }); + + TestFrameProcessorListener listener = + new TestFrameProcessorListener(inputHardwareBuffer, inputImage); + + HardwareBufferFrame inputHardwareBufferFrame = + new DefaultHardwareBufferFrame.Builder(inputHardwareBuffer).setFormat(inputFormat).build(); + + Bitmap expectedBitmap = + readBitmap( + Util.formatInvariant( + "test-generated-goldens/HardwareBufferToGlTextureConverterTest/%s.png", + testName.getMethodName())); + + Bitmap actualBitmap = convertAndCaptureBitmap(inputHardwareBufferFrame, listener); + + assertThat( + getBitmapAveragePixelAbsoluteDifferenceArgb8888( + expectedBitmap, actualBitmap, testName.getMethodName())) + .isLessThan(MAX_PIXEL_DIFFERENCE); + assertThat(listener.completedFrame.get()).isSameInstanceAs(inputHardwareBufferFrame); } @SdkSuppress(minSdkVersion = 31) @Test - public void convert_withRgba8888HardwareBufferAndRotation_outputsCorrectGlTexture() + public void convert_withRgba8888HardwareBufferAndSdrToHdrConversion_outputsCorrectGlTexture() throws Exception { Bitmap inputBitmap = BitmapPixelTestUtil.readBitmap("media/png/first_frame_1920x1080.png"); Bitmap hardwareBitmap = inputBitmap.copy(Bitmap.Config.HARDWARE, /* isMutable= */ false); - HardwareBuffer hardwareBuffer = hardwareBitmap.getHardwareBuffer(); + HardwareBuffer hardwareBuffer = checkNotNull(hardwareBitmap.getHardwareBuffer()); assertThat(hardwareBuffer.getFormat()).isEqualTo(HardwareBuffer.RGBA_8888); - int rotationDegrees = 90; - int formatWidth = inputBitmap.getHeight(); - int formatHeight = inputBitmap.getWidth(); + Format inputFormat = + new Format.Builder() + .setWidth(inputBitmap.getWidth()) + .setHeight(inputBitmap.getHeight()) + .setColorInfo(ColorInfo.SRGB_BT709_FULL) + .build(); + HardwareBufferFrame inputHardwareBufferFrame = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer).setFormat(inputFormat).build(); - AtomicReference completedFrame = new AtomicReference<>(); - FrameProcessor.Listener listener = - new FrameProcessor.Listener() { - @Override - public void onWakeup() {} + try { + ColorInfo hlgColorInfo = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorTransfer(C.COLOR_TRANSFER_HLG) + .build(); + + converter = + new HardwareBufferToGlTextureConverter( + context, + HardwareBufferJni.INSTANCE, + // Forces SDR - HDR upsampling + /* outputColorInfo= */ hlgColorInfo, + e -> { + throw new AssertionError(e); + }); + + TestFrameProcessorListener listener = new TestFrameProcessorListener(); + + Bitmap expectedBitmap = + BitmapPixelTestUtil.readBitmap( + Util.formatInvariant( + "test-generated-goldens/HardwareBufferToGlTextureConverterTest/%s.png", + testName.getMethodName())); + + Bitmap actualBitmap = convertAndCaptureBitmap(inputHardwareBufferFrame, listener); + + assertFp16BitmapsAreSimilar(expectedBitmap, actualBitmap, TestUtil.PSNR_THRESHOLD); + assertThat(listener.completedFrame.get()).isSameInstanceAs(inputHardwareBufferFrame); + } finally { + hardwareBuffer.close(); + } + } - @Override - public void onError(VideoFrameProcessingException exception) {} + @SdkSuppress(minSdkVersion = 31) + @Test + public void convert_withSolidColorSdrBuffersAndHlgOutput_outputsCorrectValues( + @TestParameter(valuesProvider = SolidColorTestCasesProvider.class) + SolidColorTestCase testCase) + throws Exception { + assertSolidColorSdrUpsampling( + /* outputColorInfo= */ BT2020_HLG, + testCase.inputSrgbColor, + checkNotNull(testCase.expectedHlgRgb)); + } - @Override - public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper releaseFence) { - if (releaseFence != null) { - assertThat(releaseFence.awaitMs(FENCE_TIMEOUT_MS)).isTrue(); - releaseFence.close(); - } - hardwareBuffer.close(); - completedFrame.set(frame); - } - }; + /** + * Tests SDR to HDR (Linear) conversion. + * + *

Expected Value Calculation: + * + *

    + *
  • Input SDR electrical values in {@code [0.0, 1.0]} are linearized using the sRGB EOTF + * (ITU-R BT.709 display light). + *
  • Display light is converted to BT.2020 display light using {@code BT709_TO_BT2020} ({@code + * XYZ_TO_BT2020 * BT709_TO_XYZ}), anchored at {@code 1.0} = 203-nit ITU-R BT.2408 diffuse + * white. + *
  • Output is kept in optical linear BT.2020 display light without applying an OETF. + *
+ */ + @SdkSuppress(minSdkVersion = 31) + @Test + public void convert_withSolidColorSdrBuffersAndHdrLinearOutput_outputsCorrectValues( + @TestParameter(valuesProvider = SolidColorTestCasesProvider.class) + SolidColorTestCase testCase) + throws Exception { + assertSolidColorSdrUpsampling( + /* outputColorInfo= */ BT2020_LINEAR, + testCase.inputSrgbColor, + testCase.expectedBt2020LinearRgb); + } + + @SdkSuppress(minSdkVersion = 31) + @Test + public void convert_withSolidColorPqBuffersAndHdrLinearOutput_outputsCorrectValues( + @TestParameter(valuesProvider = PqSolidColorTestCasesProvider.class) + SolidColorTestCase testCase) + throws Exception { + assertSolidColorPqConversion( + /* outputColorInfo= */ BT2020_LINEAR, + /* inputPqColor= */ checkNotNull(testCase.inputPqColor), + testCase.expectedBt2020LinearRgb); + } + @SdkSuppress(minSdkVersion = 31) + @Test + public void convert_withSolidColorPqBuffersAndHlgOutput_outputsCorrectValues( + @TestParameter(valuesProvider = PqSolidColorTestCasesProvider.class) + SolidColorTestCase testCase) + throws Exception { + assertSolidColorPqConversion( + /* outputColorInfo= */ BT2020_HLG, + /* inputPqColor= */ checkNotNull(testCase.inputPqColor), + checkNotNull(testCase.expectedHlgRgb)); + } + + @SdkSuppress(minSdkVersion = 31) + @Test + public void convert_withSolidColorPqBuffersAndSdrLinearOutput_outputsCorrectValues( + @TestParameter(valuesProvider = PqSolidColorTestCasesProvider.class) + SolidColorTestCase testCase) + throws Exception { + assertSolidColorPqConversion( + /* outputColorInfo= */ BT709_LINEAR, + /* inputPqColor= */ checkNotNull(testCase.inputPqColor), + checkNotNull(testCase.expectedSdrLinearRgb)); + } + + @SdkSuppress(minSdkVersion = 31) + @Test + public void convert_withSolidColorHlgBuffersAndHdrLinearOutput_outputsCorrectValues( + @TestParameter(valuesProvider = HlgSolidColorTestCasesProvider.class) + SolidColorTestCase testCase) + throws Exception { + assertSolidColorHlgConversion( + /* outputColorInfo= */ BT2020_LINEAR, + /* inputHlgColor= */ checkNotNull(testCase.inputHlgColor), + testCase.expectedBt2020LinearRgb); + } + + @SdkSuppress(minSdkVersion = 31) + @Test + public void convert_withSolidColorHlgBuffersAndSdrLinearOutput_outputsCorrectValues( + @TestParameter(valuesProvider = HlgSolidColorTestCasesProvider.class) + SolidColorTestCase testCase) + throws Exception { + assertSolidColorHlgConversion( + /* outputColorInfo= */ BT709_LINEAR, + /* inputHlgColor= */ checkNotNull(testCase.inputHlgColor), + checkNotNull(testCase.expectedSdrLinearRgb)); + } + + /** + * Tests Ultra HDR (JPEG_R) to HDR (Linear) conversion. + * + *

Expected Value Calculation: + * + *

    + *
  • Input SDR base electrical values in {@code [0.0, 1.0]} are linearized using the sRGB EOTF + * to optical BT.709 display light ({@code 1.0} = 203-nit ITU-R BT.2408 diffuse white). + *
  • The 8-bit {@code ALPHA_8} gainmap value {@code g = gainmapAlpha / 255.0} interpolates the + * logarithmic boost between {@code ln(ratioMin)} and {@code ln(ratioMax)}, weighted by the + * target HDR headroom ratio ({@code 1000.0 / 203.1521 = ~4.9224}), and scales the linear + * BT.709 color. + *
  • The boosted linear BT.709 display light is converted to BT.2020 display light using + * {@code BT709_TO_BT2020} ({@code XYZ_TO_BT2020 * BT709_TO_XYZ}). + *
+ */ + @SdkSuppress(minSdkVersion = 34) + @Test + public void convert_withSolidColorUltraHdrBuffersAndHdrLinearOutput_outputsCorrectValues( + @TestParameter(valuesProvider = UltraHdrSolidColorTestCasesProvider.class) + SolidColorTestCase testCase) + throws Exception { + assertSolidColorUltraHdrConversion( + /* outputColorInfo= */ BT2020_LINEAR, + testCase.inputSrgbColor, + testCase.gainmapAlpha, + testCase.expectedBt2020LinearRgb); + } + + /** + * Tests Ultra HDR (JPEG_R) to HDR (HLG) conversion. + * + *

Expected Value Calculation: + * + *

    + *
  • Input SDR base and gainmap are reconstructed to optical linear BT.2020 display light + * anchored at {@code 1.0} = 203-nit diffuse white (up to {@code ~4.9224} = 1,000 nits). + *
  • Display light is scaled down by {@code HDR_DIFFUSE_WHITE_SCALE_DOWN} ({@code 0.2031521}) + * to normalize 1,000 nits to {@code 1.0}. + *
  • Normalized display light is converted to scene light via the inverse HLG OOTF (gamma + * {@code 1.2}, chromaticity-preserving clamp to {@code [0.0, 1.0]}) and encoded with the + * ITU-R BT.2100 HLG OETF. + *
+ */ + @SdkSuppress(minSdkVersion = 34) + @Test + public void convert_withSolidColorUltraHdrBuffersAndHlgOutput_outputsCorrectValues( + @TestParameter(valuesProvider = UltraHdrSolidColorTestCasesProvider.class) + SolidColorTestCase testCase) + throws Exception { + assertSolidColorUltraHdrConversion( + /* outputColorInfo= */ BT2020_HLG, + testCase.inputSrgbColor, + testCase.gainmapAlpha, + checkNotNull(testCase.expectedHlgRgb)); + } + + @RequiresApi(31) + private void assertSolidColorConversion( + Bitmap inputBitmap, + ColorInfo inputColorInfo, + ColorInfo outputColorInfo, + Color expectedOutputColorRgb) + throws Exception { + assertSolidColorConversion( + inputBitmap, + /* sampleMimeType= */ null, + /* internalImage= */ null, + inputColorInfo, + outputColorInfo, + expectedOutputColorRgb); + } + + @RequiresApi(31) + private void assertSolidColorConversion( + Bitmap inputBitmap, + @Nullable String sampleMimeType, + @Nullable Object internalImage, + ColorInfo inputColorInfo, + ColorInfo outputColorInfo, + Color expectedOutputColorRgb) + throws Exception { + int width = inputBitmap.getWidth(); + int height = inputBitmap.getHeight(); + Bitmap hardwareBitmap = inputBitmap.copy(Bitmap.Config.HARDWARE, /* isMutable= */ false); + HardwareBuffer hardwareBuffer = checkNotNull(hardwareBitmap.getHardwareBuffer()); + HardwareBufferFrame inputHardwareBufferFrame = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer) + .setFormat( + new Format.Builder() + .setSampleMimeType(sampleMimeType) + .setWidth(width) + .setHeight(height) + .setColorInfo(inputColorInfo) + .build()) + .setInternalImage(internalImage) + .build(); + + try { + converter = + new HardwareBufferToGlTextureConverter( + context, + HardwareBufferJni.INSTANCE, + /* outputColorInfo= */ outputColorInfo, + e -> { + throw new AssertionError(e); + }); + + TestFrameProcessorListener listener = new TestFrameProcessorListener(); + + Bitmap expectedBitmap = + createFp16BitmapWithSolidColor( + width, + height, + expectedOutputColorRgb.red(), + expectedOutputColorRgb.green(), + expectedOutputColorRgb.blue()); + + Bitmap actualBitmap = convertAndCaptureBitmap(inputHardwareBufferFrame, listener); + + assertFp16BitmapsAreSimilar(expectedBitmap, actualBitmap, TestUtil.PSNR_THRESHOLD); + assertThat(listener.completedFrame.get()).isSameInstanceAs(inputHardwareBufferFrame); + } finally { + hardwareBuffer.close(); + } + } + + @RequiresApi(31) + private void assertSolidColorSdrUpsampling( + ColorInfo outputColorInfo, int inputSrgbColor, Color expectedOutputColorRgb) + throws Exception { + assertSolidColorConversion( + createArgb8888BitmapWithSolidColor( + /* width= */ SOLID_COLOR_BITMAP_SIZE, + /* height= */ SOLID_COLOR_BITMAP_SIZE, + inputSrgbColor), + ColorInfo.SRGB_BT709_FULL, + outputColorInfo, + expectedOutputColorRgb); + } + + @RequiresApi(31) + private void assertSolidColorPqConversion( + ColorInfo outputColorInfo, Color inputPqColor, Color expectedOutputColorRgb) + throws Exception { + assertSolidColorConversion( + createFp16BitmapWithSolidColor( + /* width= */ SOLID_COLOR_BITMAP_SIZE, + /* height= */ SOLID_COLOR_BITMAP_SIZE, + inputPqColor.red(), + inputPqColor.green(), + inputPqColor.blue()), + /* inputColorInfo= */ PQ_INPUT_COLOR, + outputColorInfo, + expectedOutputColorRgb); + } + + @RequiresApi(31) + private void assertSolidColorHlgConversion( + ColorInfo outputColorInfo, Color inputHlgColor, Color expectedOutputColorRgb) + throws Exception { + assertSolidColorConversion( + createFp16BitmapWithSolidColor( + /* width= */ SOLID_COLOR_BITMAP_SIZE, + /* height= */ SOLID_COLOR_BITMAP_SIZE, + inputHlgColor.red(), + inputHlgColor.green(), + inputHlgColor.blue()), + /* inputColorInfo= */ HLG_INPUT_COLOR, + outputColorInfo, + expectedOutputColorRgb); + } + + @RequiresApi(34) + private void assertSolidColorUltraHdrConversion( + ColorInfo outputColorInfo, int baseSrgbColor, int gainmapAlpha, Color expectedOutputColorRgb) + throws Exception { + Bitmap baseBitmap = + createArgb8888BitmapWithSolidColor( + /* width= */ SOLID_COLOR_BITMAP_SIZE, + /* height= */ SOLID_COLOR_BITMAP_SIZE, + baseSrgbColor); + // Gainmap are sized 1/4 (width and height) of the original image. + Bitmap gainmapBitmap = + Bitmap.createBitmap( + SOLID_COLOR_BITMAP_SIZE / 4, SOLID_COLOR_BITMAP_SIZE / 4, Bitmap.Config.ALPHA_8); + gainmapBitmap.eraseColor(Color.argb(gainmapAlpha, /* red= */ 0, /* green= */ 0, /* blue= */ 0)); + + // The BT.2408 scale up (1000 / 203). + float hdrSdrDisplayRatio = 4.9224f; + Gainmap gainmap = new Gainmap(gainmapBitmap); + gainmap.setDisplayRatioForFullHdr(hdrSdrDisplayRatio); + gainmap.setMinDisplayRatioForHdrTransition(1.0f); + gainmap.setRatioMin(1.0f, 1.0f, 1.0f); + // The BT.2408 Scale up (1000 / 203) + gainmap.setRatioMax(hdrSdrDisplayRatio, hdrSdrDisplayRatio, hdrSdrDisplayRatio); + gainmap.setGamma(1.0f, 1.0f, 1.0f); + gainmap.setEpsilonSdr(0.0f, 0.0f, 0.0f); + gainmap.setEpsilonHdr(0.0f, 0.0f, 0.0f); + baseBitmap.setGainmap(gainmap); + + assertSolidColorConversion( + baseBitmap, + /* sampleMimeType= */ MimeTypes.IMAGE_JPEG_R, + /* internalImage= */ baseBitmap, + /* inputColorInfo= */ ColorInfo.SRGB_BT709_FULL, + outputColorInfo, + expectedOutputColorRgb); + } + + @SdkSuppress(minSdkVersion = 31) + @Test + public void convert_withRgba8888HardwareBufferAndRotation_outputsCorrectGlTexture() + throws Exception { + int rotationDegrees = 90; + Bitmap inputBitmap = BitmapPixelTestUtil.readBitmap("media/png/first_frame_1920x1080.png"); + Bitmap hardwareBitmap = inputBitmap.copy(Bitmap.Config.HARDWARE, /* isMutable= */ false); + HardwareBuffer hardwareBuffer = hardwareBitmap.getHardwareBuffer(); Format inputFormat = new Format.Builder() - .setWidth(formatWidth) - .setHeight(formatHeight) + .setWidth(inputBitmap.getWidth()) + .setHeight(inputBitmap.getHeight()) .setRotationDegrees(rotationDegrees) .setColorInfo(ColorInfo.SDR_BT709_LIMITED) .build(); - HardwareBufferFrame inputHardwareBufferFrame = new DefaultHardwareBufferFrame.Builder(hardwareBuffer).setFormat(inputFormat).build(); - Bitmap actualBitmap = convertAndCaptureBitmap(inputHardwareBufferFrame, listener); - Matrix rotationMatrix = new Matrix(); rotationMatrix.postRotate(rotationDegrees); Bitmap expectedBitmap = @@ -365,11 +1076,14 @@ public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper releaseFenc rotationMatrix, /* filter= */ true); + TestFrameProcessorListener listener = new TestFrameProcessorListener(hardwareBuffer); + + Bitmap actualBitmap = convertAndCaptureBitmap(inputHardwareBufferFrame, listener); + assertThat( getBitmapAveragePixelAbsoluteDifferenceArgb8888( expectedBitmap, actualBitmap, testName.getMethodName())) .isLessThan(MAX_PIXEL_DIFFERENCE); - assertThat(completedFrame.get()).isSameInstanceAs(inputHardwareBufferFrame); } @SdkSuppress(minSdkVersion = 31) @@ -383,25 +1097,7 @@ public void convert_withRgba8888HardwareBufferAndCrop_outputsCorrectGlTexture() int formatWidth = inputBitmap.getWidth() / 2; int formatHeight = inputBitmap.getHeight() / 2; - AtomicReference completedFrame = new AtomicReference<>(); - FrameProcessor.Listener listener = - new FrameProcessor.Listener() { - @Override - public void onWakeup() {} - - @Override - public void onError(VideoFrameProcessingException exception) {} - - @Override - public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper releaseFence) { - if (releaseFence != null) { - assertThat(releaseFence.awaitMs(FENCE_TIMEOUT_MS)).isTrue(); - releaseFence.close(); - } - hardwareBuffer.close(); - completedFrame.set(frame); - } - }; + TestFrameProcessorListener listener = new TestFrameProcessorListener(hardwareBuffer); Format inputFormat = new Format.Builder() @@ -422,82 +1118,106 @@ public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper releaseFenc getBitmapAveragePixelAbsoluteDifferenceArgb8888( expectedBitmap, actualBitmap, testName.getMethodName())) .isLessThan(MAX_PIXEL_DIFFERENCE); - assertThat(completedFrame.get()).isSameInstanceAs(inputHardwareBufferFrame); + assertThat(listener.completedFrame.get()).isSameInstanceAs(inputHardwareBufferFrame); } - private Bitmap convertAndCaptureBitmap( - HardwareBufferFrame hardwareBufferFrame, FrameProcessor.Listener listener) throws Exception { - AtomicReference actualBitmap = new AtomicReference<>(); - CountDownLatch bitmapCaptured = new CountDownLatch(1); + @SdkSuppress(minSdkVersion = 31) + @Test + public void convert_releaseAfterReleasingGlResources_doesNotThrow() throws Exception { + Bitmap expectedBitmap = BitmapPixelTestUtil.readBitmap("media/png/first_frame_1920x1080.png"); + Bitmap hardwareBitmap = expectedBitmap.copy(Bitmap.Config.HARDWARE, /* isMutable= */ false); + HardwareBuffer hardwareBuffer = hardwareBitmap.getHardwareBuffer(); + + HardwareBufferFrame hardwareBufferFrame = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer) + .setFormat(new Format.Builder().setWidth(100).setHeight(100).build()) + .build(); + glExecutorService .submit( () -> { try { - setupOpenGl(checkNotNull(glObjectsProvider)); + int unused = setupOpenGl(checkNotNull(glObjectsProvider)); GlTextureFrame glTextureFrame = converter.convert( - /* hardwareBufferFrame= */ hardwareBufferFrame, + hardwareBufferFrame, glExecutorService, - /* listenerExecutor= */ directExecutor(), - listener); + directExecutor(), + new TestFrameProcessorListener()); - int textureWidth = glTextureFrame.glTextureInfo.width; - int textureHeight = glTextureFrame.glTextureInfo.height; - int fboId = GlUtil.createFboForTexture(glTextureFrame.glTextureInfo.texId); - GlUtil.focusFramebufferUsingCurrentContext(fboId, textureWidth, textureHeight); - actualBitmap.set( - createArgb8888BitmapFromFocusedGlFramebuffer(textureWidth, textureHeight)); + converter.releaseGlResources(hardwareBufferFrame); + // This should be no-op as GL resources is already released. glTextureFrame.release(/* releaseFence= */ null); - GlUtil.deleteFbo(fboId); - // createArgb8888BitmapFromFocusedGlFramebuffer calls glReadPixels, which syncs - // OpenGL - bitmapCaptured.countDown(); } catch (GlException | VideoFrameProcessingException e) { throw new IllegalStateException(e); } }) .get(); - assertThat(bitmapCaptured.await(TEST_TIMEOUT_MS, MILLISECONDS)).isTrue(); - return actualBitmap.get(); + hardwareBuffer.close(); } - @SdkSuppress(minSdkVersion = 31) + @SdkSuppress(minSdkVersion = 34) @Test - public void convert_releaseAfterReleasingGlResources_doesNotThrow() throws Exception { - Bitmap expectedBitmap = BitmapPixelTestUtil.readBitmap("media/png/first_frame_1920x1080.png"); - Bitmap hardwareBitmap = expectedBitmap.copy(Bitmap.Config.HARDWARE, /* isMutable= */ false); + public void convert_withUltraHdrBitmapAndHdrOutput_outputsCorrectGlTexture() throws Exception { + assertUltraHdrConversion(BT2020_HLG); + } + + @SdkSuppress(minSdkVersion = 34) + @Test + public void convert_withUltraHdrBitmapAndSdrOutput_outputsCorrectToneMappedGlTexture() + throws Exception { + assertUltraHdrConversion(BT709_SRGB); + } + + @SdkSuppress(minSdkVersion = 34) + @Test + public void convert_withUltraHdr_releaseAfterReleasingGlResources_doesNotThrow() + throws Exception { + Bitmap ultraHdrBitmap = BitmapPixelTestUtil.readBitmap("media/jpeg/ultraHdr_360x480.jpg"); + assertThat(ultraHdrBitmap.hasGainmap()).isTrue(); + Bitmap hardwareBitmap = ultraHdrBitmap.copy(Bitmap.Config.HARDWARE, /* isMutable= */ false); HardwareBuffer hardwareBuffer = hardwareBitmap.getHardwareBuffer(); + converter = + new HardwareBufferToGlTextureConverter( + context, + HardwareBufferJni.INSTANCE, + BT2020_HLG, + e -> { + throw new IllegalStateException(e); + }); + + int width = ultraHdrBitmap.getWidth(); + int height = ultraHdrBitmap.getHeight(); + Format inputFormat = + new Format.Builder() + .setSampleMimeType(MimeTypes.IMAGE_JPEG_R) + .setWidth(width) + .setHeight(height) + .setColorInfo(ColorInfo.SRGB_BT709_FULL) + .build(); + HardwareBufferFrame hardwareBufferFrame = new DefaultHardwareBufferFrame.Builder(hardwareBuffer) - .setFormat(new Format.Builder().setWidth(100).setHeight(100).build()) + .setFormat(inputFormat) + .setInternalImage(ultraHdrBitmap) .build(); glExecutorService .submit( () -> { try { - setupOpenGl(checkNotNull(glObjectsProvider)); + int unused = setupOpenGl(checkNotNull(glObjectsProvider)); GlTextureFrame glTextureFrame = converter.convert( hardwareBufferFrame, glExecutorService, directExecutor(), - new FrameProcessor.Listener() { - @Override - public void onWakeup() {} - - @Override - public void onError(VideoFrameProcessingException exception) {} - - @Override - public void onFrameProcessed( - Frame frame, @Nullable SyncFenceWrapper releaseFence) {} - }); + new TestFrameProcessorListener()); converter.releaseGlResources(hardwareBufferFrame); - // This should be no-op as GL resources is already released. + // This should be no-op as GL resources are already released. glTextureFrame.release(/* releaseFence= */ null); } catch (GlException | VideoFrameProcessingException e) { throw new IllegalStateException(e); @@ -507,4 +1227,265 @@ public void onFrameProcessed( hardwareBuffer.close(); } + + private Bitmap convertAndCaptureBitmap( + HardwareBufferFrame hardwareBufferFrame, FrameProcessor.Listener listener) throws Exception { + AtomicReference actualBitmap = new AtomicReference<>(); + glExecutorService + .submit( + () -> { + try { + int unused = setupOpenGl(checkNotNull(glObjectsProvider)); + GlTextureFrame glTextureFrame = + converter.convert( + hardwareBufferFrame, + glExecutorService, + /* listenerExecutor= */ directExecutor(), + listener); + + int textureWidth = glTextureFrame.glTextureInfo.width; + int textureHeight = glTextureFrame.glTextureInfo.height; + int fboId = GlUtil.createFboForTexture(glTextureFrame.glTextureInfo.texId); + GlUtil.focusFramebufferUsingCurrentContext(fboId, textureWidth, textureHeight); + int[] params = new int[1]; + GLES30.glGetFramebufferAttachmentParameteriv( + GLES30.GL_FRAMEBUFFER, + GLES30.GL_COLOR_ATTACHMENT0, + GLES30.GL_FRAMEBUFFER_ATTACHMENT_RED_SIZE, + params, + /* offset= */ 0); + if (params[0] > 8) { + actualBitmap.set( + createFp16BitmapFromFocusedGlFramebuffer(textureWidth, textureHeight)); + } else { + actualBitmap.set( + createArgb8888BitmapFromFocusedGlFramebuffer(textureWidth, textureHeight)); + } + glTextureFrame.release(/* releaseFence= */ null); + assertThat(glTextureFrame.format.rotationDegrees).isEqualTo(0); + GlUtil.deleteFbo(fboId); + } catch (GlException | VideoFrameProcessingException e) { + throw new IllegalStateException(e); + } + }) + .get(TEST_TIMEOUT_MS, MILLISECONDS); + // glTextureFrame.release() posts releaseTextureCallback (which invokes + // listener.onFrameProcessed directly) onto glExecutorService. Wait for that queued + // task to finish. + glExecutorService.submit(() -> {}).get(TEST_TIMEOUT_MS, MILLISECONDS); + + return checkNotNull(actualBitmap.get()); + } + + @RequiresApi(34) + private void assertUltraHdrConversion(ColorInfo outputColorInfo) throws Exception { + Bitmap ultraHdrBitmap = BitmapPixelTestUtil.readBitmap("media/jpeg/ultraHdr_360x480.jpg"); + assertThat(ultraHdrBitmap.hasGainmap()).isTrue(); + Bitmap hardwareBitmap = ultraHdrBitmap.copy(Bitmap.Config.HARDWARE, /* isMutable= */ false); + HardwareBuffer hardwareBuffer = hardwareBitmap.getHardwareBuffer(); + + AtomicReference errorReference = new AtomicReference<>(); + converter = + new HardwareBufferToGlTextureConverter( + context, HardwareBufferJni.INSTANCE, outputColorInfo, errorReference::set); + + TestFrameProcessorListener listener = new TestFrameProcessorListener(hardwareBuffer); + + int width = ultraHdrBitmap.getWidth(); + int height = ultraHdrBitmap.getHeight(); + Format inputFormat = + new Format.Builder() + .setSampleMimeType(MimeTypes.IMAGE_JPEG_R) + .setWidth(width) + .setHeight(height) + .setColorInfo(ColorInfo.SRGB_BT709_FULL) + .build(); + + HardwareBufferFrame hardwareBufferFrame = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer) + .setFormat(inputFormat) + .setInternalImage(ultraHdrBitmap) + .build(); + + Bitmap actualBitmap = convertAndCaptureBitmap(hardwareBufferFrame, listener); + + assertThat(actualBitmap.getWidth()).isEqualTo(width); + assertThat(actualBitmap.getHeight()).isEqualTo(height); + if (ColorInfo.isWideColorGamut(outputColorInfo)) { + assertThat(actualBitmap.getConfig()).isEqualTo(Bitmap.Config.RGBA_F16); + } + assertThat(errorReference.get()).isNull(); + assertThat(listener.completedFrame.get()).isSameInstanceAs(hardwareBufferFrame); + } + + private static Bitmap createFp16BitmapWithSolidColor( + int width, int height, float r, float g, float b) { + // Half.toHalf returns a short. Mask to prevent sign-extension when upcasting. + long rHalf = (long) Half.toHalf(r) & 0xFFFFL; + long gHalf = (long) Half.toHalf(g) & 0xFFFFL; + long bHalf = (long) Half.toHalf(b) & 0xFFFFL; + long aHalf = (long) Half.toHalf(1.0f) & 0xFFFFL; + + long pixelColor; + if (ByteOrder.nativeOrder() == ByteOrder.LITTLE_ENDIAN) { + pixelColor = rHalf | (gHalf << 16) | (bHalf << 32) | (aHalf << 48); + } else { + pixelColor = (rHalf << 48) | (gHalf << 32) | (bHalf << 16) | aHalf; + } + + int numPixels = width * height; + long[] pixels = new long[numPixels]; + Arrays.fill(pixels, pixelColor); + + ByteBuffer buffer = ByteBuffer.allocateDirect(numPixels * 8).order(ByteOrder.nativeOrder()); + buffer.asLongBuffer().put(pixels); + buffer.rewind(); + + Bitmap bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.RGBA_F16); + bitmap.copyPixelsFromBuffer(buffer); + return bitmap; + } + + private static final class SolidColorTestCase { + final String name; + final int inputSrgbColor; + final int gainmapAlpha; + @Nullable final Color inputPqColor; + @Nullable final Color inputHlgColor; + @Nullable final Color expectedHlgRgb; + final Color expectedBt2020LinearRgb; + @Nullable final Color expectedSdrLinearRgb; + + SolidColorTestCase( + String name, int inputSrgbColor, Color expectedHlgRgb, Color expectedBt2020LinearRgb) { + this( + name, + inputSrgbColor, + /* gainmapAlpha= */ C.INDEX_UNSET, + expectedHlgRgb, + expectedBt2020LinearRgb); + } + + SolidColorTestCase( + String name, + int inputSrgbColor, + int gainmapAlpha, + Color expectedHlgRgb, + Color expectedBt2020LinearRgb) { + this.name = name; + this.inputSrgbColor = inputSrgbColor; + this.gainmapAlpha = gainmapAlpha; + this.inputPqColor = null; + this.inputHlgColor = null; + this.expectedHlgRgb = expectedHlgRgb; + this.expectedBt2020LinearRgb = expectedBt2020LinearRgb; + this.expectedSdrLinearRgb = null; + } + + SolidColorTestCase( + String name, + Color inputHlgColor, + Color expectedBt2020LinearRgb, + Color expectedSdrLinearRgb) { + this.name = name; + this.inputSrgbColor = Color.BLACK; + this.gainmapAlpha = C.INDEX_UNSET; + this.inputPqColor = null; + this.inputHlgColor = inputHlgColor; + this.expectedHlgRgb = null; + this.expectedBt2020LinearRgb = expectedBt2020LinearRgb; + this.expectedSdrLinearRgb = expectedSdrLinearRgb; + } + + SolidColorTestCase( + String name, + Color inputPqColor, + Color expectedHlgRgb, + Color expectedBt2020LinearRgb, + Color expectedSdrLinearRgb) { + this.name = name; + this.inputSrgbColor = Color.BLACK; + this.gainmapAlpha = C.INDEX_UNSET; + this.inputPqColor = inputPqColor; + this.inputHlgColor = null; + this.expectedHlgRgb = expectedHlgRgb; + this.expectedBt2020LinearRgb = expectedBt2020LinearRgb; + this.expectedSdrLinearRgb = expectedSdrLinearRgb; + } + + @Override + public String toString() { + return name; + } + } + + private static final class SolidColorTestCasesProvider extends TestParameterValuesProvider { + @Override + protected ImmutableList provideValues( + com.google.testing.junit.testparameterinjector.TestParameterValuesProvider.Context + context) { + return SOLID_COLOR_TEST_CASES; + } + } + + private static final class PqSolidColorTestCasesProvider extends TestParameterValuesProvider { + @Override + protected ImmutableList provideValues( + com.google.testing.junit.testparameterinjector.TestParameterValuesProvider.Context + context) { + return PQ_SOLID_COLOR_TEST_CASES; + } + } + + private static final class HlgSolidColorTestCasesProvider extends TestParameterValuesProvider { + @Override + protected ImmutableList provideValues( + com.google.testing.junit.testparameterinjector.TestParameterValuesProvider.Context + context) { + return HLG_SOLID_COLOR_TEST_CASES; + } + } + + private static final class UltraHdrSolidColorTestCasesProvider + extends TestParameterValuesProvider { + @Override + protected ImmutableList provideValues( + com.google.testing.junit.testparameterinjector.TestParameterValuesProvider.Context + context) { + return ULTRA_HDR_SOLID_COLOR_TEST_CASES; + } + } + + private static final class TestFrameProcessorListener implements FrameProcessor.Listener { + final AtomicReference completedFrame = new AtomicReference<>(); + private final AutoCloseable[] closeables; + + TestFrameProcessorListener(AutoCloseable... closeables) { + this.closeables = closeables; + } + + @Override + public void onWakeup() {} + + @Override + public void onError(VideoFrameProcessingException exception) { + throw new AssertionError(exception); + } + + @Override + public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper releaseFence) { + if (releaseFence != null) { + assertThat(releaseFence.awaitMs(FENCE_TIMEOUT_MS)).isTrue(); + releaseFence.close(); + } + for (AutoCloseable closeable : closeables) { + try { + closeable.close(); + } catch (Exception e) { + throw new AssertionError(e); + } + } + completedFrame.set(frame); + } + } } diff --git a/libraries/effect/src/androidTest/java/androidx/media3/effect/ImagePlanesToGlTextureConverterTest.java b/libraries/effect/src/androidTest/java/androidx/media3/effect/ImagePlanesToGlTextureConverterTest.java new file mode 100644 index 00000000000..c9bae8f3480 --- /dev/null +++ b/libraries/effect/src/androidTest/java/androidx/media3/effect/ImagePlanesToGlTextureConverterTest.java @@ -0,0 +1,472 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect; + +import static androidx.media3.common.util.Util.isRunningOnEmulator; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT709_LINEAR; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT709_SRGB; +import static androidx.media3.effect.FrameProcessorUtils.releaseOpenGl; +import static androidx.media3.effect.FrameProcessorUtils.setupOpenGl; +import static androidx.media3.effect.FrameProcessorUtils.shutdownGlExecutorService; +import static androidx.media3.test.utils.BitmapPixelTestUtil.MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE; +import static androidx.media3.test.utils.BitmapPixelTestUtil.MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE_DIFFERENT_DEVICE_FP16; +import static androidx.media3.test.utils.BitmapPixelTestUtil.createArgb8888BitmapFromGlTexture; +import static androidx.media3.test.utils.BitmapPixelTestUtil.createArgb8888BitmapWithSolidColor; +import static androidx.media3.test.utils.BitmapPixelTestUtil.createFp16BitmapFromGlTexture; +import static androidx.media3.test.utils.BitmapPixelTestUtil.createFp16BitmapWithSolidColor; +import static androidx.media3.test.utils.BitmapPixelTestUtil.getBitmapAveragePixelAbsoluteDifferenceArgb8888; +import static androidx.media3.test.utils.BitmapPixelTestUtil.getBitmapAveragePixelAbsoluteDifferenceFp16; +import static androidx.media3.test.utils.BitmapPixelTestUtil.readBitmap; +import static androidx.test.core.app.ApplicationProvider.getApplicationContext; +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.util.concurrent.MoreExecutors.directExecutor; +import static com.google.common.util.concurrent.MoreExecutors.listeningDecorator; +import static java.util.concurrent.TimeUnit.MILLISECONDS; + +import android.content.Context; +import android.graphics.Bitmap; +import android.graphics.Color; +import android.graphics.Matrix; +import android.graphics.PixelFormat; +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import androidx.media3.common.C; +import androidx.media3.common.ColorInfo; +import androidx.media3.common.Format; +import androidx.media3.common.GlObjectsProvider; +import androidx.media3.common.VideoFrameProcessingException; +import androidx.media3.common.util.GlUtil.GlException; +import androidx.media3.common.video.DefaultImagePlanesFrame; +import androidx.media3.common.video.DefaultImagePlanesFrame.DefaultPlane; +import androidx.media3.common.video.Frame; +import androidx.media3.common.video.FrameProcessor; +import androidx.media3.common.video.ImagePlanesFrame; +import androidx.media3.common.video.ImagePlanesFrame.Plane; +import androidx.media3.common.video.SyncFenceWrapper; +import androidx.media3.test.utils.FakeFrameProcessor; +import androidx.test.filters.SdkSuppress; +import com.google.common.collect.ImmutableList; +import com.google.common.util.concurrent.ListeningExecutorService; +import com.google.testing.junit.testparameterinjector.TestParameter; +import com.google.testing.junit.testparameterinjector.TestParameterInjector; +import java.nio.ByteBuffer; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.Executors; +import java.util.concurrent.atomic.AtomicReference; +import org.checkerframework.checker.nullness.qual.MonotonicNonNull; +import org.junit.After; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TestName; +import org.junit.runner.RunWith; + +/** Instrumentation tests for {@link ImagePlanesToGlTextureConverter}. */ +@RunWith(TestParameterInjector.class) +public final class ImagePlanesToGlTextureConverterTest { + + private static final long TEST_TIMEOUT_MS = isRunningOnEmulator() ? 20_000L : 10_000L; + private static final int DEFAULT_FRAME_WIDTH = 64; + private static final int DEFAULT_FRAME_HEIGHT = 32; + private static final int DEFAULT_COLOR = Color.GREEN; + private static final Bitmap EXPECTED_DEFAULT_COLOR_BITMAP = + createArgb8888BitmapWithSolidColor(DEFAULT_FRAME_WIDTH, DEFAULT_FRAME_HEIGHT, DEFAULT_COLOR); + + /** + * Number of padding pixels appended to each row, used to simulate producers that align rows to a + * stride wider than the visible image. + */ + private static final int RGBA_ROW_PADDING_PIXELS = 7; + + /** Padding fill color, chosen to contrast strongly with {@link #DEFAULT_COLOR}. */ + private static final int RGBA_PADDING_COLOR = Color.RED; + + private static final int RGBA_BYTES_PER_PIXEL = 4; + + @Rule public final TestName testName = new TestName(); + + private final Context context = getApplicationContext(); + + private @MonotonicNonNull ListeningExecutorService glExecutorService; + private @MonotonicNonNull GlObjectsProvider glObjectsProvider; + private @MonotonicNonNull ImagePlanesToGlTextureConverter converter; + + @Before + public void setUp() throws Exception { + glExecutorService = listeningDecorator(Executors.newSingleThreadExecutor()); + glObjectsProvider = new DefaultGlObjectsProvider(); + glExecutorService + .submit(() -> setupOpenGl(glObjectsProvider)) + .get(TEST_TIMEOUT_MS, MILLISECONDS); + converter = + new ImagePlanesToGlTextureConverter(context, BT709_SRGB, /* errorConsumer= */ e -> {}); + } + + @After + public void tearDown() throws Exception { + if (glExecutorService != null) { + glExecutorService.submit(this::tearDownInternal).get(TEST_TIMEOUT_MS, MILLISECONDS); + shutdownGlExecutorService(glExecutorService); + } + } + + private void tearDownInternal() { + try { + if (converter != null) { + // Releases all active and cached output textures created during the test. + converter.close(); + } + if (glObjectsProvider != null) { + releaseOpenGl(glObjectsProvider); + } + } catch (GlException | VideoFrameProcessingException e) { + throw new IllegalStateException(e); + } + } + + @Test + public void convert_validFrame_setsOutputFormat() throws Exception { + ImagePlanesFrame frame = createRgbaFrame(DEFAULT_COLOR); + + GlTextureFrame glTextureFrame = convert(frame); + + assertThat(glTextureFrame.glTextureInfo.texId).isNotEqualTo(C.INDEX_UNSET); + assertThat(glTextureFrame.format.width).isEqualTo(DEFAULT_FRAME_WIDTH); + assertThat(glTextureFrame.format.height).isEqualTo(DEFAULT_FRAME_HEIGHT); + assertThat(glTextureFrame.format.rotationDegrees).isEqualTo(0); + assertThat(glTextureFrame.format.colorInfo).isEqualTo(BT709_SRGB); + } + + @Test + public void convert_rotatedFrame_rotatesPixelsAndTransposesOutputDimensions() throws Exception { + int rotationDegrees = 90; + Bitmap inputBitmap = readBitmap("media/png/first_frame_1920x1080.png"); + ByteBuffer buffer = ByteBuffer.allocateDirect(inputBitmap.getByteCount()); + inputBitmap.copyPixelsToBuffer(buffer); + buffer.rewind(); + Plane plane = + new DefaultPlane( + buffer, + /* rowStride= */ inputBitmap.getRowBytes(), + /* pixelStride= */ RGBA_BYTES_PER_PIXEL); + Format inputFormat = + new Format.Builder() + .setWidth(inputBitmap.getWidth()) + .setHeight(inputBitmap.getHeight()) + .setRotationDegrees(rotationDegrees) + .setPixelFormat(PixelFormat.RGBA_8888) + .setColorInfo(ColorInfo.SDR_BT709_LIMITED) + .build(); + ImagePlanesFrame rotatedFrame = + new DefaultImagePlanesFrame.Builder(ImmutableList.of(plane)).setFormat(inputFormat).build(); + Matrix rotationMatrix = new Matrix(); + rotationMatrix.postRotate(rotationDegrees); + Bitmap expectedBitmap = + Bitmap.createBitmap( + inputBitmap, + /* x= */ 0, + /* y= */ 0, + inputBitmap.getWidth(), + inputBitmap.getHeight(), + rotationMatrix, + /* filter= */ true); + + GlTextureFrame glTextureFrame = convert(rotatedFrame); + + Bitmap actualBitmap = createArgb8888BitmapFromGlTextureFrame(glTextureFrame); + assertThat(glTextureFrame.glTextureInfo.width).isEqualTo(inputBitmap.getHeight()); + assertThat(glTextureFrame.glTextureInfo.height).isEqualTo(inputBitmap.getWidth()); + assertThat(glTextureFrame.format.width).isEqualTo(inputBitmap.getHeight()); + assertThat(glTextureFrame.format.height).isEqualTo(inputBitmap.getWidth()); + assertThat(glTextureFrame.format.rotationDegrees).isEqualTo(0); + assertThat( + getBitmapAveragePixelAbsoluteDifferenceArgb8888( + expectedBitmap, actualBitmap, testName.getMethodName())) + .isAtMost(MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE); + } + + @Test + public void convert_rgbaPlanesFrame_outputsCorrectGlTexture( + @TestParameter boolean usePaddedRowStride) throws Exception { + ImagePlanesFrame rgbaFrame = + createRgbaFrame(DEFAULT_COLOR, usePaddedRowStride ? RGBA_ROW_PADDING_PIXELS : 0); + + GlTextureFrame glTextureFrame = convert(rgbaFrame); + + Bitmap actualBitmap = createArgb8888BitmapFromGlTextureFrame(glTextureFrame); + assertThat( + getBitmapAveragePixelAbsoluteDifferenceArgb8888( + EXPECTED_DEFAULT_COLOR_BITMAP, actualBitmap, testName.getMethodName())) + .isAtMost(MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE); + } + + @Test + public void convert_rgbaPlanesFrameWithTruncatedFinalRow_outputsCorrectGlTexture() + throws Exception { + ImagePlanesFrame rgbaFrame = + createRgbaFrame( + DEFAULT_COLOR, + RGBA_ROW_PADDING_PIXELS, + DEFAULT_FRAME_WIDTH, + DEFAULT_FRAME_HEIGHT, + /* padFinalRow= */ false); + + GlTextureFrame glTextureFrame = convert(rgbaFrame); + + Bitmap actualBitmap = createArgb8888BitmapFromGlTextureFrame(glTextureFrame); + assertThat( + getBitmapAveragePixelAbsoluteDifferenceArgb8888( + EXPECTED_DEFAULT_COLOR_BITMAP, actualBitmap, testName.getMethodName())) + .isAtMost(MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE); + } + + @Test + public void convert_multipleFramesWithVaryingDimensionsAndStrides_outputsCorrectGlTextures() + throws Exception { + ImagePlanesFrame firstFrame = + createRgbaFrame( + Color.GREEN, + /* rowPaddingPixels= */ RGBA_ROW_PADDING_PIXELS, + /* width= */ 64, + /* height= */ 32); + ImagePlanesFrame secondFrameSameDimensions = + createRgbaFrame( + Color.BLUE, + /* rowPaddingPixels= */ RGBA_ROW_PADDING_PIXELS, + /* width= */ 64, + /* height= */ 32); + ImagePlanesFrame thirdFrameDifferentDimensions = + createRgbaFrame(Color.YELLOW, /* rowPaddingPixels= */ 0, /* width= */ 32, /* height= */ 16); + + GlTextureFrame firstGlTextureFrame = convert(firstFrame); + int firstTexId = firstGlTextureFrame.glTextureInfo.texId; + Bitmap firstBitmap = createArgb8888BitmapFromGlTextureFrame(firstGlTextureFrame); + firstGlTextureFrame.release(/* releaseFence= */ null); + GlTextureFrame secondGlTextureFrame = convert(secondFrameSameDimensions); + int secondTexId = secondGlTextureFrame.glTextureInfo.texId; + Bitmap secondBitmap = createArgb8888BitmapFromGlTextureFrame(secondGlTextureFrame); + secondGlTextureFrame.release(/* releaseFence= */ null); + GlTextureFrame thirdGlTextureFrame = convert(thirdFrameDifferentDimensions); + Bitmap thirdBitmap = createArgb8888BitmapFromGlTextureFrame(thirdGlTextureFrame); + thirdGlTextureFrame.release(/* releaseFence= */ null); + + assertThat(firstGlTextureFrame.glTextureInfo.width).isEqualTo(64); + assertThat(firstGlTextureFrame.glTextureInfo.height).isEqualTo(32); + assertThat( + getBitmapAveragePixelAbsoluteDifferenceArgb8888( + createArgb8888BitmapWithSolidColor(/* width= */ 64, /* height= */ 32, Color.GREEN), + firstBitmap, + testName.getMethodName())) + .isAtMost(MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE); + assertThat(secondTexId).isEqualTo(firstTexId); + assertThat(secondGlTextureFrame.glTextureInfo.width).isEqualTo(64); + assertThat(secondGlTextureFrame.glTextureInfo.height).isEqualTo(32); + assertThat( + getBitmapAveragePixelAbsoluteDifferenceArgb8888( + createArgb8888BitmapWithSolidColor(/* width= */ 64, /* height= */ 32, Color.BLUE), + secondBitmap, + testName.getMethodName())) + .isAtMost(MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE); + assertThat(thirdGlTextureFrame.glTextureInfo.width).isEqualTo(32); + assertThat(thirdGlTextureFrame.glTextureInfo.height).isEqualTo(16); + assertThat( + getBitmapAveragePixelAbsoluteDifferenceArgb8888( + createArgb8888BitmapWithSolidColor(/* width= */ 32, /* height= */ 16, Color.YELLOW), + thirdBitmap, + testName.getMethodName())) + .isAtMost(MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE); + } + + @SdkSuppress(minSdkVersion = 29) + @Test + public void convert_rgbaPlanesFrameWithLinearOutputTransfer_appliesSrgbEotf() throws Exception { + int srgbMidTone = Color.rgb(128, 128, 128); + // srgbEotf(128 / 255.0) = ((128 / 255.0 + 0.055) / 1.055)^2.4 = 0.2158605f + float expectedLinearChannel = 0.2158605f; + Bitmap expectedLinearBitmap = + createFp16BitmapWithSolidColor( + DEFAULT_FRAME_WIDTH, + DEFAULT_FRAME_HEIGHT, + expectedLinearChannel, + expectedLinearChannel, + expectedLinearChannel); + ImagePlanesFrame rgbaFrame = createRgbaFrame(srgbMidTone); + converter = + new ImagePlanesToGlTextureConverter(context, BT709_LINEAR, /* errorConsumer= */ e -> {}); + + GlTextureFrame glTextureFrame = convert(rgbaFrame); + + Bitmap actualBitmap = createFp16BitmapFromGlTextureFrame(glTextureFrame); + assertThat(glTextureFrame.format.colorInfo).isEqualTo(BT709_LINEAR); + assertThat(getBitmapAveragePixelAbsoluteDifferenceFp16(expectedLinearBitmap, actualBitmap)) + .isAtMost(MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE_DIFFERENT_DEVICE_FP16); + } + + @Test + public void release_validFrame_notifiesListener() throws Exception { + ImagePlanesFrame frame = createRgbaFrame(DEFAULT_COLOR); + CapturingFrameProcessorListener listener = new CapturingFrameProcessorListener(); + + GlTextureFrame glTextureFrame = convert(frame, listener); + glTextureFrame.release(/* releaseFence= */ null); + + assertThat(listener.frameProcessed.await(TEST_TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(listener.processedFrame.get()).isSameInstanceAs(frame); + assertThat(listener.onCompleteFence.get()).isNull(); + } + + @Test + public void release_afterReleasingGlResources_doesNotNotifyListener() throws Exception { + ImagePlanesFrame frame = createRgbaFrame(DEFAULT_COLOR); + CapturingFrameProcessorListener listener = new CapturingFrameProcessorListener(); + + glExecutorService + .submit( + () -> { + GlTextureFrame glTextureFrame = + converter.convert( + frame, + /* glExecutor= */ glExecutorService, + /* listenerExecutor= */ directExecutor(), + listener); + converter.releaseGlResources(frame); + glTextureFrame.release(/* releaseFence= */ null); + return null; + }) + .get(TEST_TIMEOUT_MS, MILLISECONDS); + + // The release callback runs on the GL executor and notifies the listener synchronously via + // directExecutor(), so once the GL executor is drained the listener would already have been + // notified if the converter were going to notify it at all. + glExecutorService.submit(() -> {}).get(TEST_TIMEOUT_MS, MILLISECONDS); + assertThat(listener.processedFrame.get()).isNull(); + } + + private GlTextureFrame convert(ImagePlanesFrame frame) throws Exception { + return convert(frame, new FakeFrameProcessor.Listener()); + } + + private GlTextureFrame convert(ImagePlanesFrame frame, FrameProcessor.Listener listener) + throws Exception { + return glExecutorService + .submit( + () -> + converter.convert( + frame, + /* glExecutor= */ glExecutorService, + /* listenerExecutor= */ directExecutor(), + listener)) + .get(TEST_TIMEOUT_MS, MILLISECONDS); + } + + private Bitmap createArgb8888BitmapFromGlTextureFrame(GlTextureFrame glTextureFrame) + throws Exception { + return glExecutorService + .submit(() -> createArgb8888BitmapFromGlTexture(glTextureFrame.glTextureInfo)) + .get(TEST_TIMEOUT_MS, MILLISECONDS); + } + + @RequiresApi(26) + private Bitmap createFp16BitmapFromGlTextureFrame(GlTextureFrame glTextureFrame) + throws Exception { + return glExecutorService + .submit(() -> createFp16BitmapFromGlTexture(glTextureFrame.glTextureInfo)) + .get(TEST_TIMEOUT_MS, MILLISECONDS); + } + + private static DefaultImagePlanesFrame createRgbaFrame(int color) { + return createRgbaFrame(color, /* rowPaddingPixels= */ 0); + } + + private static DefaultImagePlanesFrame createRgbaFrame(int color, int rowPaddingPixels) { + return createRgbaFrame(color, rowPaddingPixels, DEFAULT_FRAME_WIDTH, DEFAULT_FRAME_HEIGHT); + } + + private static DefaultImagePlanesFrame createRgbaFrame( + int color, int rowPaddingPixels, int width, int height) { + return createRgbaFrame(color, rowPaddingPixels, width, height, /* padFinalRow= */ true); + } + + /** + * Returns a solid color RGBA_8888 frame, with each row padded by {@code rowPaddingPixels} pixels + * of {@link #RGBA_PADDING_COLOR}, or with the final row ending at the last visible pixel when + * {@code padFinalRow} is {@code false} (matching {@code ImageReader} plane buffers). + */ + private static DefaultImagePlanesFrame createRgbaFrame( + int color, int rowPaddingPixels, int width, int height, boolean padFinalRow) { + int paddedWidth = width + rowPaddingPixels; + int rowStride = paddedWidth * RGBA_BYTES_PER_PIXEL; + int bufferPixels = paddedWidth * (height - 1) + (padFinalRow ? paddedWidth : width); + int bufferSize = bufferPixels * RGBA_BYTES_PER_PIXEL; + ByteBuffer buffer = ByteBuffer.allocateDirect(bufferSize); + // The first `height - 1` rows are always padded. + for (int y = 0; y < height - 1; y++) { + for (int x = 0; x < width; x++) { + addRgbaToBuffer(color, buffer); + } + for (int x = width; x < paddedWidth; x++) { + addRgbaToBuffer(RGBA_PADDING_COLOR, buffer); + } + } + // The final row only includes padding when `padFinalRow` is true. + for (int x = 0; x < width; x++) { + addRgbaToBuffer(color, buffer); + } + if (padFinalRow) { + for (int x = width; x < paddedWidth; x++) { + addRgbaToBuffer(RGBA_PADDING_COLOR, buffer); + } + } + buffer.flip(); + Plane plane = new DefaultPlane(buffer, rowStride, /* pixelStride= */ RGBA_BYTES_PER_PIXEL); + Format format = + new Format.Builder() + .setWidth(width) + .setHeight(height) + .setPixelFormat(PixelFormat.RGBA_8888) + .setColorInfo(ColorInfo.SDR_BT709_LIMITED) + .build(); + return new DefaultImagePlanesFrame.Builder(ImmutableList.of(plane)).setFormat(format).build(); + } + + private static void addRgbaToBuffer(int pixelColor, ByteBuffer buffer) { + buffer.put((byte) Color.red(pixelColor)); + buffer.put((byte) Color.green(pixelColor)); + buffer.put((byte) Color.blue(pixelColor)); + buffer.put((byte) Color.alpha(pixelColor)); + } + + private static final class CapturingFrameProcessorListener implements FrameProcessor.Listener { + final CountDownLatch frameProcessed = new CountDownLatch(1); + final AtomicReference processedFrame = new AtomicReference<>(); + final AtomicReference onCompleteFence = new AtomicReference<>(); + + @Override + public void onWakeup() {} + + @Override + public void onError(VideoFrameProcessingException exception) { + throw new AssertionError(exception); + } + + @Override + public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper onCompleteFence) { + this.processedFrame.set(frame); + this.onCompleteFence.set(onCompleteFence); + this.frameProcessed.countDown(); + } + } +} diff --git a/libraries/effect/src/main/java/androidx/media3/effect/AlphaScale.java b/libraries/effect/src/main/java/androidx/media3/effect/AlphaScale.java index f74534c0597..478f33e8ff2 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/AlphaScale.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/AlphaScale.java @@ -15,16 +15,19 @@ */ package androidx.media3.effect; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; import static com.google.common.base.Preconditions.checkArgument; import android.content.Context; import androidx.annotation.FloatRange; +import androidx.annotation.RestrictTo; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.UnstableApi; /** Scales the alpha value (i.e. the translucency) of a frame. */ @UnstableApi -public final class AlphaScale implements GlEffect { +public final class AlphaScale implements GlEffect, MetricsProvider { private final float alphaScale; /** @@ -49,4 +52,10 @@ public GlShaderProgram toGlShaderProgram(Context context, boolean useHdr) public boolean isNoOp(int inputWidth, int inputHeight) { return alphaScale == 1f; } + + @Override + @RestrictTo(LIBRARY_GROUP) + public void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_EFFECT_COLOR); + } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/AlphaScaleShaderProgram.java b/libraries/effect/src/main/java/androidx/media3/effect/AlphaScaleShaderProgram.java index 17ae24beb07..91386f787eb 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/AlphaScaleShaderProgram.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/AlphaScaleShaderProgram.java @@ -84,4 +84,14 @@ public void drawFrame(int inputTexId, long presentationTimeUs) throw new VideoFrameProcessingException(e, presentationTimeUs); } } + + @Override + public void release() throws VideoFrameProcessingException { + super.release(); + try { + glProgram.delete(); + } catch (GlUtil.GlException e) { + throw new VideoFrameProcessingException(e); + } + } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/BaseGlShaderProgram.java b/libraries/effect/src/main/java/androidx/media3/effect/BaseGlShaderProgram.java index 72d2f8f7270..6c9e236d682 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/BaseGlShaderProgram.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/BaseGlShaderProgram.java @@ -133,7 +133,6 @@ public void setErrorListener(Executor errorListenerExecutor, ErrorListener error * {@link GlUtil#clearFocusedBuffers()}. */ public boolean shouldClearTextureBuffer() { - // TODO - b/309428083: Clear the texture before first use. return true; } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/ByteBufferConcurrentEffect.java b/libraries/effect/src/main/java/androidx/media3/effect/ByteBufferConcurrentEffect.java index cf1d3df2342..3ad00d0e4e3 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/ByteBufferConcurrentEffect.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/ByteBufferConcurrentEffect.java @@ -15,12 +15,10 @@ */ package androidx.media3.effect; -import static android.os.Build.VERSION.SDK_INT; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.base.Preconditions.checkState; import static com.google.common.util.concurrent.Futures.immediateFailedFuture; -import android.opengl.GLES20; import androidx.media3.common.C; import androidx.media3.common.GlObjectsProvider; import androidx.media3.common.GlTextureInfo; @@ -208,35 +206,14 @@ public void schedulePixelBufferRead(PixelBufferObjectProvider pixelBufferObjectP throws GlUtil.GlException { int pixelBufferSize = texturePixelBufferSize(textureInfo); pixelBufferObjectInfo = pixelBufferObjectProvider.getPixelBufferObject(pixelBufferSize); - if (SDK_INT >= 24) { - GlUtil.schedulePixelBufferRead( - textureInfo.fboId, textureInfo.width, textureInfo.height, pixelBufferObjectInfo.id); - } + GlUtil.schedulePixelBufferRead( + textureInfo.fboId, textureInfo.width, textureInfo.height, pixelBufferObjectInfo.id); } public void map() throws GlUtil.GlException { checkNotNull(pixelBufferObjectInfo); - ByteBuffer byteBuffer; - if (SDK_INT >= 24) { - byteBuffer = - GlUtil.mapPixelBufferObject(pixelBufferObjectInfo.id, pixelBufferObjectInfo.size); - } else { - // Asynchronous OpenGL reading isn't supported. Fall back to blocking glReadPixels. - int pixelBufferSize = texturePixelBufferSize(textureInfo); - byteBuffer = ByteBuffer.allocateDirect(pixelBufferSize); - GlUtil.focusFramebufferUsingCurrentContext( - textureInfo.fboId, textureInfo.width, textureInfo.height); - GlUtil.checkGlError(); - GLES20.glReadPixels( - /* x= */ 0, - /* y= */ 0, - textureInfo.width, - textureInfo.height, - GLES20.GL_RGBA, - GLES20.GL_UNSIGNED_BYTE, - byteBuffer); - GlUtil.checkGlError(); - } + ByteBuffer byteBuffer = + GlUtil.mapPixelBufferObject(pixelBufferObjectInfo.id, pixelBufferObjectInfo.size); imageSettableFuture.set( new ByteBufferGlEffect.Image(textureInfo.width, textureInfo.height, byteBuffer)); mapped = true; @@ -245,7 +222,7 @@ public void map() throws GlUtil.GlException { public void unmapAndRecycle(PixelBufferObjectProvider pixelBufferObjectProvider) throws GlUtil.GlException { checkNotNull(pixelBufferObjectInfo); - if (mapped && SDK_INT >= 24) { + if (mapped) { GlUtil.unmapPixelBufferObject(pixelBufferObjectInfo.id); } pixelBufferObjectProvider.recycle(pixelBufferObjectInfo); diff --git a/libraries/effect/src/main/java/androidx/media3/effect/ByteBufferGlEffect.java b/libraries/effect/src/main/java/androidx/media3/effect/ByteBufferGlEffect.java index 28444c7ac87..eaf7a607a61 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/ByteBufferGlEffect.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/ByteBufferGlEffect.java @@ -15,13 +15,16 @@ */ package androidx.media3.effect; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; import static com.google.common.base.Preconditions.checkArgument; import android.content.Context; import android.graphics.Bitmap; import android.graphics.Matrix; import android.opengl.GLES20; +import androidx.annotation.RestrictTo; import androidx.media3.common.GlTextureInfo; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.GlRect; import androidx.media3.common.util.Size; @@ -39,7 +42,7 @@ * available such as another GPU context, FPGAs, or NPUs. */ @UnstableApi -public class ByteBufferGlEffect implements GlEffect { +public class ByteBufferGlEffect implements GlEffect, MetricsProvider { private static final int DEFAULT_QUEUE_SIZE = 6; private static final int DEFAULT_PENDING_PIXEL_BUFFER_QUEUE_SIZE = 1; @@ -201,4 +204,10 @@ public GlShaderProgram toGlShaderProgram(Context context, boolean useHdr) new ByteBufferConcurrentEffect<>( /* pendingPixelBufferQueueSize= */ DEFAULT_PENDING_PIXEL_BUFFER_QUEUE_SIZE, processor)); } + + @Override + @RestrictTo(LIBRARY_GROUP) + public void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_EFFECT_BUFFER); + } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/ColorConversion.java b/libraries/effect/src/main/java/androidx/media3/effect/ColorConversion.java new file mode 100644 index 00000000000..3017e2d1baf --- /dev/null +++ b/libraries/effect/src/main/java/androidx/media3/effect/ColorConversion.java @@ -0,0 +1,117 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect; + +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT2020_HLG; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT709_SRGB; +import static com.google.common.base.Preconditions.checkArgument; +import static com.google.common.base.Preconditions.checkNotNull; + +import android.content.Context; +import androidx.annotation.RestrictTo; +import androidx.media3.common.C; +import androidx.media3.common.ColorInfo; +import androidx.media3.common.MetricsProvider; +import androidx.media3.common.VideoFrameProcessingException; +import androidx.media3.common.util.ExperimentalApi; + +/** + * Converts frames from an input color space to an electrical output color space. + * + *

Applying this effect last in an effect chain sets the output color space of the video frame + * processing pipeline. + */ +@ExperimentalApi // TODO: b/505721737 Remove once FrameProcessor is production ready. +public final class ColorConversion implements GlEffect, MetricsProvider { + + private final ColorInfo outputColorInfo; + + /** + * Creates an instance that converts frames to {@code outputColorInfo}. + * + *

Supported outputs are BT.2020 with {@link C#COLOR_TRANSFER_HLG}, and BT.709 with {@link + * C#COLOR_TRANSFER_SRGB}. Other properties of {@code outputColorInfo}, such as the {@linkplain + * ColorInfo#colorRange color range}, do not affect the conversion and are propagated to the + * output frames unchanged. + * + * @param outputColorInfo The {@link ColorInfo} to convert frames to. + * @return A new instance. + * @throws IllegalArgumentException If {@code outputColorInfo} is not supported. + */ + public static ColorConversion create(ColorInfo outputColorInfo) { + checkNotNull(outputColorInfo); + checkArgument( + hasSameColorSpaceAndTransfer(outputColorInfo, BT2020_HLG) + || hasSameColorSpaceAndTransfer(outputColorInfo, BT709_SRGB), + "Unsupported output color space %s with color transfer %s.", + outputColorInfo.colorSpace, + outputColorInfo.colorTransfer); + return new ColorConversion(outputColorInfo); + } + + private ColorConversion(ColorInfo outputColorInfo) { + this.outputColorInfo = outputColorInfo; + } + + // GlEffect implementation. + + /** + * Not supported, because the input color space is only known once the pipeline receives frames. + * + * @throws UnsupportedOperationException Always. This effect is only supported by {@link + * DefaultGlFrameProcessor}. + */ + @Override + public GlShaderProgram toGlShaderProgram(Context context, boolean useHdr) { + throw new UnsupportedOperationException( + "ColorConversion is only supported by DefaultGlFrameProcessor."); + } + + /** + * Returns a {@link GlShaderProgram} that converts frames from {@code inputColorInfo} to this + * effect's output color space. + * + * @param context The {@link Context}. + * @param inputColorInfo The {@link ColorInfo} of the frames this effect receives, which is the + * working color space. + * @return The {@link GlShaderProgram}. + * @throws VideoFrameProcessingException If an error occurs while creating the {@link + * GlShaderProgram}. + */ + /* package */ GlShaderProgram toGlShaderProgram(Context context, ColorInfo inputColorInfo) + throws VideoFrameProcessingException { + return new ColorConversionShaderProgram(context, inputColorInfo, outputColorInfo); + } + + /** Returns the {@link ColorInfo} this effect converts frames to. */ + /* package */ ColorInfo getOutputColorInfo() { + return outputColorInfo; + } + + // MetricsProvider implementation. + + @Override + @RestrictTo(LIBRARY_GROUP) + public void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_EFFECT_COLOR); + } + + private static boolean hasSameColorSpaceAndTransfer(ColorInfo colorInfo, ColorInfo other) { + return colorInfo.colorSpace == other.colorSpace + && colorInfo.colorTransfer == other.colorTransfer; + } +} diff --git a/libraries/effect/src/main/java/androidx/media3/effect/ColorConversionShaderProgram.java b/libraries/effect/src/main/java/androidx/media3/effect/ColorConversionShaderProgram.java new file mode 100644 index 00000000000..a1ea7097d0e --- /dev/null +++ b/libraries/effect/src/main/java/androidx/media3/effect/ColorConversionShaderProgram.java @@ -0,0 +1,112 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect; + +import static androidx.media3.effect.FrameProcessorUtils.useHighPrecisionColorComponents; + +import android.content.Context; +import android.opengl.GLES20; +import androidx.media3.common.ColorInfo; +import androidx.media3.common.VideoFrameProcessingException; +import androidx.media3.common.util.GlProgram; +import androidx.media3.common.util.GlUtil; +import androidx.media3.common.util.Size; +import java.io.IOException; + +/** + * Converts frames from an input color space to an electrical output color space. + * + *

Accepts input in any of the supported working color spaces, whether linear optical or + * electrical, and outputs electrical colors expected by an encoder or a {@link + * android.view.Surface}. + */ +/* package */ final class ColorConversionShaderProgram extends BaseGlShaderProgram { + + private final GlProgram glProgram; + + /** + * Creates a new instance. + * + * @param context The {@link Context}. + * @param inputColorInfo The {@link ColorInfo} of the input frames, which is the frame processor's + * working color space. + * @param outputColorInfo The {@link ColorInfo} to convert frames to. + * @throws VideoFrameProcessingException If a problem occurs while reading shader files. + */ + public ColorConversionShaderProgram( + Context context, ColorInfo inputColorInfo, ColorInfo outputColorInfo) + throws VideoFrameProcessingException { + // The output texture holds the electrical output colors, so its precision is determined by the + // output color space rather than by the working color space the frames arrive in. + super( + /* useHighPrecisionColorComponents= */ useHighPrecisionColorComponents(outputColorInfo), + /* texturePoolCapacity= */ 1); + + try { + glProgram = + new GlProgram( + context, + /* vertexShaderResId= */ R.raw.vertex_shader_transformation_es3, + /* firstFragmentShaderResId= */ R.raw.color_conversions_es3, + R.raw.fragment_shader_color_conversion_es3); + } catch (IOException | GlUtil.GlException e) { + throw new VideoFrameProcessingException(e); + } + + glProgram.setBufferAttribute( + "aFramePosition", + GlUtil.getNormalizedCoordinateBounds(), + GlUtil.HOMOGENEOUS_COORDINATE_VECTOR_SIZE); + + float[] identityMatrix = GlUtil.create4x4IdentityMatrix(); + glProgram.setFloatsUniform("uTransformationMatrix", identityMatrix); + glProgram.setFloatsUniform("uTexTransformationMatrix", identityMatrix); + + glProgram.setIntUniform("uInputColorGamut", inputColorInfo.colorSpace); + glProgram.setIntUniform("uInputColorTransfer", inputColorInfo.colorTransfer); + glProgram.setIntUniform("uOutputColorTransfer", outputColorInfo.colorTransfer); + } + + @Override + public Size configure(int inputWidth, int inputHeight) { + return new Size(inputWidth, inputHeight); + } + + @Override + public void drawFrame(int inputTexId, long presentationTimeUs) + throws VideoFrameProcessingException { + try { + glProgram.use(); + glProgram.setSamplerTexIdUniform("uTexSampler", inputTexId, /* texUnitIndex= */ 0); + glProgram.bindAttributesAndUniforms(); + + GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, /* first= */ 0, /* count= */ 4); + GlUtil.checkGlError(); + } catch (GlUtil.GlException e) { + throw new VideoFrameProcessingException(e, presentationTimeUs); + } + } + + @Override + public void release() throws VideoFrameProcessingException { + super.release(); + try { + glProgram.delete(); + } catch (GlUtil.GlException e) { + throw new VideoFrameProcessingException(e); + } + } +} diff --git a/libraries/effect/src/main/java/androidx/media3/effect/ColorLut.java b/libraries/effect/src/main/java/androidx/media3/effect/ColorLut.java index 96f133bf295..eff3da6d6a6 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/ColorLut.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/ColorLut.java @@ -16,7 +16,11 @@ package androidx.media3.effect; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; + import android.content.Context; +import androidx.annotation.RestrictTo; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.GlUtil; import androidx.media3.common.util.UnstableApi; @@ -26,7 +30,7 @@ * shader. */ @UnstableApi -public interface ColorLut extends GlEffect { +public interface ColorLut extends GlEffect, MetricsProvider { /** * Returns the OpenGL texture ID of the LUT to apply to the pixels of the frame with the given @@ -45,4 +49,10 @@ default GlShaderProgram toGlShaderProgram(Context context, boolean useHdr) throws VideoFrameProcessingException { return new ColorLutShaderProgram(context, /* colorLut= */ this, useHdr); } + + @Override + @RestrictTo(LIBRARY_GROUP) + default void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_EFFECT_COLOR); + } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/DebugViewShaderProgram.java b/libraries/effect/src/main/java/androidx/media3/effect/DebugViewShaderProgram.java index 7ddaf9934bc..fb3c4525bfc 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/DebugViewShaderProgram.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/DebugViewShaderProgram.java @@ -195,7 +195,7 @@ private void ensureConfigured(int inputWidth, int inputHeight) * and makes rendering a no-op if not. */ private static final class SurfaceViewWrapper implements SurfaceHolder.Callback { - public final @C.ColorTransfer int outputColorTransfer; + final @C.ColorTransfer int outputColorTransfer; private final EGLDisplay eglDisplay; private final EGLContext eglContext; @@ -210,7 +210,7 @@ private static final class SurfaceViewWrapper implements SurfaceHolder.Callback private int width; private int height; - public SurfaceViewWrapper( + SurfaceViewWrapper( EGLDisplay eglDisplay, EGLContext eglContext, SurfaceView surfaceView, @@ -260,10 +260,10 @@ public synchronized void surfaceDestroyed(SurfaceHolder holder) { * *

Must be called on the GL thread. */ - public synchronized void maybeRenderToSurfaceView( + synchronized void maybeRenderToSurfaceView( VideoFrameProcessingTaskExecutor.Task renderingTask, GlObjectsProvider glObjectsProvider) throws GlUtil.GlException, VideoFrameProcessingException { - if (surface == null) { + if (surface == null || !surface.isValid()) { return; } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/DefaultCompositorGlProgram.java b/libraries/effect/src/main/java/androidx/media3/effect/DefaultCompositorGlProgram.java index 356da737063..18fc13ad7ee 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/DefaultCompositorGlProgram.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/DefaultCompositorGlProgram.java @@ -25,6 +25,7 @@ import androidx.media3.common.util.GlUtil; import androidx.media3.common.util.Log; import androidx.media3.common.util.Size; +import androidx.media3.effect.GlTextureFrameCompositor.CompositorGlProgram; import java.io.IOException; import java.util.List; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; @@ -35,8 +36,29 @@ * *

All methods must be called on a GL thread, unless otherwise stated. */ -/* package */ final class DefaultCompositorGlProgram - implements DefaultGlTextureFrameCompositingProcessor.CompositorGlProgram { +/* package */ final class DefaultCompositorGlProgram implements CompositorGlProgram { + + /** + * A {@link CompositorGlProgram.Factory} that creates {@link DefaultCompositorGlProgram} + * instances. + */ + public static final class Factory implements CompositorGlProgram.Factory { + private final Context context; + + /** + * Creates an instance. + * + * @param context The {@link Context}. + */ + public Factory(Context context) { + this.context = context.getApplicationContext(); + } + + @Override + public CompositorGlProgram create() { + return new DefaultCompositorGlProgram(context); + } + } private static final String TAG = "CompositorGlProgram"; diff --git a/libraries/effect/src/main/java/androidx/media3/effect/DefaultGlFrameProcessor.java b/libraries/effect/src/main/java/androidx/media3/effect/DefaultGlFrameProcessor.java index 6b7c146a559..ffcf329e749 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/DefaultGlFrameProcessor.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/DefaultGlFrameProcessor.java @@ -15,181 +15,270 @@ */ package androidx.media3.effect; +import static androidx.media3.common.ColorInfo.SDR_BT709_LIMITED; +import static androidx.media3.common.ColorInfo.isWideColorGamut; +import static androidx.media3.effect.FrameProcessorUtils.OPEN_GL_VERSION_3; +import static androidx.media3.effect.FrameProcessorUtils.releaseOpenGl; import static androidx.media3.effect.FrameProcessorUtils.runAllAndAccumulateExceptions; +import static androidx.media3.effect.FrameProcessorUtils.setupOpenGl; +import static androidx.media3.effect.FrameProcessorUtils.shutdownGlExecutorService; import static androidx.media3.effect.FrameProcessorUtils.waitAndCloseFence; import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkNotNull; +import static com.google.common.base.Preconditions.checkState; import static com.google.common.util.concurrent.MoreExecutors.listeningDecorator; +import static java.util.concurrent.TimeUnit.MILLISECONDS; import android.content.Context; -import android.hardware.HardwareBuffer; import android.opengl.GLES20; import android.util.SparseArray; import androidx.annotation.GuardedBy; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; import androidx.annotation.VisibleForTesting; +import androidx.media3.common.C; import androidx.media3.common.ColorInfo; import androidx.media3.common.Effect; +import androidx.media3.common.Format; import androidx.media3.common.GlObjectsProvider; +import androidx.media3.common.MimeTypes; import androidx.media3.common.VideoCompositorSettings; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.Consumer; import androidx.media3.common.util.ExperimentalApi; +import androidx.media3.common.util.GlUtil.GlException; import androidx.media3.common.util.Log; +import androidx.media3.common.util.ThrowingRunnable; import androidx.media3.common.video.AsyncFrame; import androidx.media3.common.video.Frame; import androidx.media3.common.video.FrameProcessor; import androidx.media3.common.video.FrameWriter; -import androidx.media3.common.video.HardwareBufferFrame; -import androidx.media3.effect.DefaultGlTextureFrameCompositingProcessor.CompositorGlProgram; -import androidx.media3.effect.FrameProcessorUtils.ThrowingRunnable; import com.google.common.collect.ImmutableList; import com.google.common.util.concurrent.ListeningExecutorService; +import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.util.Collections; import java.util.HashSet; import java.util.IdentityHashMap; import java.util.List; +import java.util.Objects; import java.util.Set; import java.util.concurrent.Callable; +import java.util.concurrent.ExecutionException; import java.util.concurrent.Executor; import java.util.concurrent.ExecutorService; +import java.util.concurrent.LinkedBlockingQueue; +import java.util.concurrent.ThreadPoolExecutor; +import java.util.concurrent.TimeoutException; +import org.checkerframework.checker.nullness.qual.MonotonicNonNull; /** * A {@link FrameProcessor} implementation that executes a chain of {@link GlTextureFrameConsumer}s. */ @ExperimentalApi // TODO: b/505721737 Remove once FrameProcessor is production ready. -@RequiresApi(26) public final class DefaultGlFrameProcessor implements FrameProcessor { - private static final String TAG = "GlFrameProcessor"; + /** A {@link FrameProcessor.Factory} that creates {@link DefaultGlFrameProcessor} instances. */ + // TODO: b/531653682 - Remove @RequiresApi(26) once the pipeline supports API 24+ end to end. + @RequiresApi(26) + public static final class Factory implements FrameProcessor.Factory { - /** Converts from {@link HardwareBuffer} to {@link GlTextureFrame}. */ - interface HardwareBufferConverter extends AutoCloseable { + /** A builder for {@link Factory} instances. */ + public static final class Builder { + private final Context context; + private final HardwareBufferJniWrapper hardwareBufferJniWrapper; + @Nullable private GlObjectsProvider glObjectsProvider; + @Nullable private ExecutorService executorService; + @Nullable private FrameToGlTextureConverter.Factory frameToGlTextureConverterFactory; + @Nullable private GlTextureFrameConsumer frameWriterGlTextureFrameConsumer; + private GlTextureFrameCompositor.Factory glTextureFrameCompositorFactory; + private boolean assumeSurfacelessContextExtensionSupported; + + /** + * Creates an instance. + * + * @param context The {@link Context}. + * @param hardwareBufferJniWrapper The {@link HardwareBufferJniWrapper}, for example {@code + * HardwareBufferJni.INSTANCE}. + */ + public Builder(Context context, HardwareBufferJniWrapper hardwareBufferJniWrapper) { + this.context = context.getApplicationContext(); + this.hardwareBufferJniWrapper = hardwareBufferJniWrapper; + glTextureFrameCompositorFactory = + new DefaultGlTextureFrameCompositor.Factory( + new DefaultCompositorGlProgram.Factory(this.context)); + } - // TODO: b/517424999 - Unify the listeners to follow the same pattern as FrameProcessor. - /** - * Converts from {@link HardwareBufferFrame} to {@link GlTextureFrame}. - * - *

The returned {@link GlTextureFrame}'s texture is in standard OpenGL coordinate space - * (upright, Y-up, with origin at bottom-left of the image). - */ - GlTextureFrame convert( - HardwareBufferFrame hardwareBufferFrame, - Executor glExecutor, - Executor listenerExecutor, - Listener listener) - throws VideoFrameProcessingException; + /** + * Sets the {@link GlObjectsProvider} and the {@link ExecutorService} to execute OpenGL + * commands on. + * + *

If not set, each created {@link DefaultGlFrameProcessor} uses a new {@link + * DefaultGlObjectsProvider} and a new single-thread executor, and releases both when it's + * {@linkplain FrameProcessor#close closed}. + * + *

If set, the caller owns both. After all the {@link DefaultGlFrameProcessor} instances + * created by the built {@link Factory} have been closed, the caller must release the {@link + * GlObjectsProvider} on the {@link ExecutorService}'s thread, and then {@linkplain + * ExecutorService#shutdown shut down} the {@link ExecutorService}. + * + * @param glObjectsProvider The {@link GlObjectsProvider}. + * @param executorService The {@link ExecutorService}. + * @return This builder. + */ + @CanIgnoreReturnValue + public Builder setGlObjectsProviderAndExecutorService( + GlObjectsProvider glObjectsProvider, ExecutorService executorService) { + this.glObjectsProvider = glObjectsProvider; + this.executorService = executorService; + return this; + } - /** - * Releases the resources for a converted {@link HardwareBufferFrame}. - * - *

Don't call this method if the {@linkplain #convert converted} {@link GlTextureFrame} is - * accepted by a downstream {@link GlTextureFrameConsumer}. - * - *

This releases associated resources without notifying the {@link Listener#onFrameProcessed} - * that was passed in via {@link #convert}. - * - *

This is used when a converted frame is rejected by downstream pipeline consumers, - * preserving the underlying {@code HardwareBuffer} for subsequent queue retries. - */ - void releaseGlResources(HardwareBufferFrame hardwareBufferFrame) - throws VideoFrameProcessingException; + /** + * Sets the {@link FrameToGlTextureConverter.Factory}. + * + * @param frameToGlTextureConverterFactory The {@link FrameToGlTextureConverter.Factory}. + * @return This builder. + */ + @VisibleForTesting + @CanIgnoreReturnValue + /* package */ Builder setFrameToGlTextureConverterFactory( + FrameToGlTextureConverter.Factory frameToGlTextureConverterFactory) { + this.frameToGlTextureConverterFactory = frameToGlTextureConverterFactory; + return this; + } - @Override - void close() throws VideoFrameProcessingException; - } + /** + * Sets the {@link GlTextureFrameConsumer} for writing frames. + * + * @param frameWriterGlTextureFrameConsumer The {@link GlTextureFrameConsumer}. + * @return This builder. + */ + @VisibleForTesting + @CanIgnoreReturnValue + /* package */ Builder setFrameWriterGlTextureFrameConsumer( + GlTextureFrameConsumer frameWriterGlTextureFrameConsumer) { + this.frameWriterGlTextureFrameConsumer = frameWriterGlTextureFrameConsumer; + return this; + } - /** A {@link FrameProcessor.Factory} that creates {@link DefaultGlFrameProcessor} instances. */ - public static final class Factory implements FrameProcessor.Factory { - private final Context context; - private final GlObjectsProvider glObjectsProvider; - private final ExecutorService glExecutorService; - @Nullable private final HardwareBufferConverter hardwareBufferConverter; - @Nullable private final HardwareBufferJniWrapper hardwareBufferJniWrapper; - @Nullable private final GlTextureFrameConsumer frameWriterGlTextureFrameConsumer; - private final CompositorGlProgram compositorGlProgram; - private final TexturePool compositorTexturePool; + /** + * Sets the {@link GlTextureFrameCompositor.Factory}. + * + * @param glTextureFrameCompositorFactory The {@link GlTextureFrameCompositor.Factory}. + * @return This builder. + */ + @VisibleForTesting + @CanIgnoreReturnValue + /* package */ Builder setGlTextureFrameCompositorFactory( + GlTextureFrameCompositor.Factory glTextureFrameCompositorFactory) { + this.glTextureFrameCompositorFactory = glTextureFrameCompositorFactory; + return this; + } - /** - * Creates an instance. - * - *

The caller is responsible for setting up OpenGL resources and releasing them after - * {@linkplain FrameProcessor#close closing} the built {@link DefaultGlFrameProcessor}. The - * caller should also shut down the {@link ExecutorService glExecutorService}. - */ - public Factory( - Context context, - GlObjectsProvider glObjectsProvider, - HardwareBufferJniWrapper hardwareBufferJniWrapper, - ListeningExecutorService glExecutorService) { - this.context = context.getApplicationContext(); - this.glObjectsProvider = glObjectsProvider; - this.hardwareBufferJniWrapper = hardwareBufferJniWrapper; - this.glExecutorService = glExecutorService; - hardwareBufferConverter = null; - frameWriterGlTextureFrameConsumer = null; - compositorGlProgram = new DefaultCompositorGlProgram(context); - compositorTexturePool = - new TexturePool( - /* useHighPrecisionColorComponents= */ false, DEFAULT_COMPOSITOR_CAPACITY); + /** + * Sets whether to assume the surfaceless context extension is supported, instead of detecting + * support at runtime. + * + *

The default value is {@code false}, meaning support is detected at runtime. + * + * @param assumeSurfacelessContextExtensionSupported Whether to assume the extension is + * supported. + * @return This builder. + */ + @VisibleForTesting + @CanIgnoreReturnValue + /* package */ Builder setAssumeSurfacelessContextExtensionSupported( + boolean assumeSurfacelessContextExtensionSupported) { + this.assumeSurfacelessContextExtensionSupported = + assumeSurfacelessContextExtensionSupported; + return this; + } + + /** Builds a {@link Factory} instance. */ + public Factory build() { + if (frameToGlTextureConverterFactory == null) { + frameToGlTextureConverterFactory = + (outputColorInfo, errorConsumer) -> + new HardwareBufferToGlTextureConverter( + context, hardwareBufferJniWrapper, outputColorInfo, errorConsumer); + } + return new Factory(this); + } } - /** - * Creates an instance for testing only. - * - *

The caller is responsible for shutting down the {@link ListeningExecutorService - * glExecutorService} after {@linkplain FrameProcessor#close closing} the built {@link - * DefaultGlFrameProcessor}. - */ - @VisibleForTesting - /* package */ Factory( - Context context, - GlObjectsProvider glObjectsProvider, - ListeningExecutorService glExecutorService, - HardwareBufferConverter hardwareBufferConverter, - GlTextureFrameConsumer frameWriterGlTextureFrameConsumer, - CompositorGlProgram compositorGlProgram, - TexturePool compositorTexturePool) { - this.context = context; - this.glObjectsProvider = glObjectsProvider; - this.glExecutorService = glExecutorService; - this.hardwareBufferConverter = hardwareBufferConverter; - this.frameWriterGlTextureFrameConsumer = frameWriterGlTextureFrameConsumer; - this.compositorGlProgram = compositorGlProgram; - this.compositorTexturePool = compositorTexturePool; - hardwareBufferJniWrapper = null; + private static final String THREAD_NAME = "Effect:DefaultGlFrameProcessor:GlThread"; + + private final Context context; + @Nullable private final GlObjectsProvider glObjectsProvider; + @Nullable private final ExecutorService glExecutorService; + private final FrameToGlTextureConverter.Factory frameToGlTextureConverterFactory; + private final HardwareBufferJniWrapper hardwareBufferJniWrapper; + @Nullable private final GlTextureFrameConsumer frameWriterGlTextureFrameConsumer; + private final GlTextureFrameCompositor.Factory glTextureFrameCompositorFactory; + // Whether to assume the surfaceless context extension is supported instead of detecting support + // at runtime. Only set to true in tests. + private final boolean assumeSurfacelessContextExtensionSupported; + // TODO(b/545584738): Allow setting a working color space. + @Nullable private ColorInfo workingColorSpace; + + private Factory(Builder builder) { + context = builder.context; + glObjectsProvider = builder.glObjectsProvider; + glExecutorService = builder.executorService; + hardwareBufferJniWrapper = builder.hardwareBufferJniWrapper; + frameToGlTextureConverterFactory = checkNotNull(builder.frameToGlTextureConverterFactory); + frameWriterGlTextureFrameConsumer = builder.frameWriterGlTextureFrameConsumer; + glTextureFrameCompositorFactory = builder.glTextureFrameCompositorFactory; + assumeSurfacelessContextExtensionSupported = + builder.assumeSurfacelessContextExtensionSupported; + workingColorSpace = null; } @Override public DefaultGlFrameProcessor create( FrameWriter output, Executor listenerExecutor, Listener listener) { GlTextureFrameConsumer frameWriterGlTextureFrameConsumer = - this.frameWriterGlTextureFrameConsumer; - if (frameWriterGlTextureFrameConsumer == null && hardwareBufferJniWrapper != null) { - frameWriterGlTextureFrameConsumer = - new FrameWriterGlTextureFrameConsumer(context, output, hardwareBufferJniWrapper); - } - HardwareBufferConverter hardwareBufferConverter = this.hardwareBufferConverter; - if (hardwareBufferConverter == null && hardwareBufferJniWrapper != null) { - hardwareBufferConverter = - new HardwareBufferToGlTextureConverter( - context, - hardwareBufferJniWrapper, - // TODO: b/517525358 - Use correct output color info once HDR is supported. - ColorInfo.SDR_BT709_LIMITED, - e -> listenerExecutor.execute(() -> listener.onError(e))); - } + this.frameWriterGlTextureFrameConsumer != null + ? this.frameWriterGlTextureFrameConsumer + : new FrameWriterGlTextureFrameConsumer(context, output, hardwareBufferJniWrapper); + boolean shouldReleaseGlResources = this.glExecutorService == null; + GlObjectsProvider glObjectsProvider = + this.glObjectsProvider != null ? this.glObjectsProvider : new DefaultGlObjectsProvider(); + ExecutorService glExecutorService = + this.glExecutorService != null + ? this.glExecutorService + : createDefaultGlExecutorService(); return new DefaultGlFrameProcessor( context, listeningDecorator(glExecutorService), glObjectsProvider, - checkNotNull(hardwareBufferConverter), - checkNotNull(frameWriterGlTextureFrameConsumer), - compositorGlProgram, - compositorTexturePool, + frameToGlTextureConverterFactory, + frameWriterGlTextureFrameConsumer, + glTextureFrameCompositorFactory, listenerExecutor, - listener); + listener, + workingColorSpace, + shouldReleaseGlResources, + assumeSurfacelessContextExtensionSupported); + } + + /** + * Returns a single-thread {@link ExecutorService} that drops, rather than rejects, tasks + * submitted after it's shut down. + * + *

Components such as the {@link FrameWriter} can post tasks, for example wakeups, to the GL + * executor after the processor that owns it has been {@linkplain FrameProcessor#close closed}. + */ + private static ExecutorService createDefaultGlExecutorService() { + return new ThreadPoolExecutor( + /* corePoolSize= */ 1, + /* maximumPoolSize= */ 1, + /* keepAliveTime= */ 0, + /* unit= */ MILLISECONDS, + /* workQueue= */ new LinkedBlockingQueue<>(), + /* threadFactory= */ runnable -> new Thread(runnable, THREAD_NAME), + /* handler= */ (runnable, executor) -> + Log.w(TAG, "Dropping GL task submitted after close")); } } @@ -242,26 +331,87 @@ public DefaultGlFrameProcessor create( */ public static final String KEY_FRAME_DISCONTINUITY_NUMBER = "KEY_FRAME_DISCONTINUITY_NUMBER"; - private static final int DEFAULT_COMPOSITOR_CAPACITY = 2; + private static final String TAG = "GlFrameProcessor"; + private static final long RELEASE_TIMEOUT_MS = 1_000; + + /** An SDR color space with BT.709 / sRGB color primaries, and linear transfer function. */ + /* package */ static final ColorInfo BT709_LINEAR = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT709) + .setColorTransfer(C.COLOR_TRANSFER_LINEAR) + .build(); + + /** An SDR color space with BT.709 / sRGB color primaries, and sRGB transfer function. */ + /* package */ static final ColorInfo BT709_SRGB = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT709) + .setColorTransfer(C.COLOR_TRANSFER_SRGB) + .build(); + + /** + * An HDR color space with BT.2020 color primaries, and linear transfer function. + * + *

Values are display-referred linear optical light anchored on the ITU-R BT.2408 203-nit + * diffuse white reference, so {@code 1.0} represents diffuse white, the 1,000-nit HLG reference + * peak evaluates to {@code ~4.9224}, and the 10,000-nit PQ peak evaluates to {@code ~49.2242}. + * + *

When input is HLG, HLG scene-referred electrical values are converted to display light using + * the ITU-R BT.2100 HLG EOTF ({@code OOTF_1.2(OETF^-1(E))}) and scaled onto the diffuse white + * anchor. When input is PQ, PQ electrical values are linearized using the SMPTE ST 2084 EOTF and + * scaled onto the diffuse white anchor. When input is Ultra HDR (JPEG_R), the sRGB base image is + * linearized to optical BT.709 display light ({@code 1.0} = 203-nit diffuse white), boosted by + * the gainmap up to the 1,000-nit reference peak ({@code ~4.9224}), and converted to BT.2020. + */ + /* package */ static final ColorInfo BT2020_LINEAR = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorTransfer(C.COLOR_TRANSFER_LINEAR) + .build(); + + /** + * An HDR color space with BT.2020 color primaries, and HLG transfer function. + * + *

When input is PQ or Ultra HDR (JPEG_R), values are first converted to optical linear BT.2020 + * display light and normalized against the 1,000-nit HLG reference display peak. The resulting + * display light is converted to scene light via inverse HLG OOTF (system gamma 1.2) and encoded + * into an HLG electrical signal using the BT.2100 HLG OETF. + * + *

HLG signal cannot represent scene light above 1.0 (either from display light above the + * 1,000-nit reference peak or saturated primaries where inverse OOTF boosts channel values above + * 1.0), so scene light is scaled down to fit in {@code [0.0, 1.0]} after the inverse HLG OOTF and + * before the OETF; all channels are scaled equally to preserve chromaticity. + * + *

The values are always scene-referred. + */ + /* package */ static final ColorInfo BT2020_HLG = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorTransfer(C.COLOR_TRANSFER_HLG) + .build(); private final Context context; private final GlObjectsProvider glObjectsProvider; private final ListeningExecutorService glExecutorService; - private final HardwareBufferConverter hardwareBufferConverter; + private final FrameToGlTextureConverter.Factory frameToGlTextureConverterFactory; + private final GlTextureFrameCompositor.Factory glTextureFrameCompositorFactory; private final GlTextureFrameConsumer frameWriterGlTextureFrameConsumer; private final Executor listenerExecutor; private final Listener listener; private final Consumer errorConsumer; // Accessed on the OpenGL thread. private final SparseArray preProcessingChains; - private final GlTextureFrameAggregator frameAggregator; - private final DefaultGlTextureFrameCompositingProcessor compositingProcessor; - private final GlTextureFrameProcessorChain postProcessingChain; private final Set glTextureFramesQueuedDownstream; private final SparseArray convertedGlTextureFrames; private final Object lock; // Accessed only on GL thread. This field is to avoid creating a new set on queueing every time. private final Set activeSequenceIndices; + private final boolean shouldReleaseGlResources; + private final boolean assumeSurfacelessContextExtensionSupported; + + private @MonotonicNonNull GlTextureFrameAggregator frameAggregator; + private @MonotonicNonNull GlTextureFrameCompositor compositingProcessor; + private @MonotonicNonNull GlTextureFrameProcessorChain postProcessingChain; + private @MonotonicNonNull FrameToGlTextureConverter frameToGlTextureConverter; @GuardedBy("lock") @Nullable @@ -276,50 +426,44 @@ public DefaultGlFrameProcessor create( @GuardedBy("lock") private boolean shouldSignalEos; + // Accessed on the GL thread. + // A null color space means it's unset; and it's inferred from the input color. + @Nullable private ColorInfo workingColorSpace; + private boolean isPipelineInitialized; + private boolean isGlSetup; + private boolean isHdrSupported; + private DefaultGlFrameProcessor( Context context, ListeningExecutorService glExecutorService, GlObjectsProvider glObjectsProvider, - HardwareBufferConverter hardwareBufferConverter, + FrameToGlTextureConverter.Factory frameToGlTextureConverterFactory, GlTextureFrameConsumer frameWriterGlTextureFrameConsumer, - CompositorGlProgram compositorGlProgram, - TexturePool compositorTexturePool, + GlTextureFrameCompositor.Factory glTextureFrameCompositorFactory, Executor listenerExecutor, - Listener listener) { + Listener listener, + @Nullable ColorInfo workingColorSpace, + boolean shouldReleaseGlResources, + boolean assumeSurfacelessContextExtensionSupported) { this.context = context; this.glObjectsProvider = glObjectsProvider; this.glExecutorService = glExecutorService; - this.hardwareBufferConverter = hardwareBufferConverter; + this.frameToGlTextureConverterFactory = frameToGlTextureConverterFactory; this.frameWriterGlTextureFrameConsumer = frameWriterGlTextureFrameConsumer; + this.glTextureFrameCompositorFactory = glTextureFrameCompositorFactory; this.listenerExecutor = listenerExecutor; this.listener = listener; + this.workingColorSpace = workingColorSpace; + this.shouldReleaseGlResources = shouldReleaseGlResources; + this.assumeSurfacelessContextExtensionSupported = assumeSurfacelessContextExtensionSupported; this.errorConsumer = e -> listenerExecutor.execute(() -> listener.onError(e)); this.glTextureFramesQueuedDownstream = Collections.newSetFromMap(new IdentityHashMap<>()); this.convertedGlTextureFrames = new SparseArray<>(); - postProcessingChain = - new GlTextureFrameProcessorChain( - context, - glObjectsProvider, - glExecutorService, - errorConsumer, - frameWriterGlTextureFrameConsumer, - KEY_COMPOSITION_EFFECTS); - - compositingProcessor = - new DefaultGlTextureFrameCompositingProcessor( - glObjectsProvider, - compositorTexturePool, - errorConsumer, - compositorGlProgram, - glExecutorService, - postProcessingChain); - - frameAggregator = - new GlTextureFrameAggregator(compositingProcessor, glExecutorService, errorConsumer); preProcessingChains = new SparseArray<>(); lock = new Object(); activeSequenceIndices = new HashSet<>(); + isPipelineInitialized = false; } @Override @@ -344,12 +488,43 @@ public boolean queue(List frames) { return null; } } + if (!isGlSetup) { + int openGlVersion; + try { + openGlVersion = + assumeSurfacelessContextExtensionSupported + ? setupOpenGl( + glObjectsProvider, /* isSurfacelessContextExtensionSupported= */ true) + : setupOpenGl(glObjectsProvider); + } catch (GlException e) { + isGlSetup = true; + throw VideoFrameProcessingException.from(e); + } + isHdrSupported = openGlVersion == OPEN_GL_VERSION_3; + isGlSetup = true; + } + if (!isPipelineInitialized) { + boolean isAnyInputHdr = false; + for (int i = 0; i < frames.size(); i++) { + Format inputFormat = frames.get(i).frame.getFormat(); + if (workingColorSpace == null) { + workingColorSpace = resolveWorkingColorspace(inputFormat); + } + isAnyInputHdr |= isWideColorGamut(inputFormat.colorInfo); + } + if (isAnyInputHdr || isWideColorGamut(workingColorSpace)) { + checkState( + isHdrSupported, + "OpenGL ES3 and 10 bit context support required for HDR inputs"); + } + initializePipeline(); + } activeSequenceIndices.clear(); for (int i = 0; i < frames.size(); i++) { activeSequenceIndices.add(extractSequenceIndex(frames.get(i).frame)); } // TODO: b/528240409 - Make config signal in-band. - frameAggregator.configureSequenceIndices(activeSequenceIndices); + checkNotNull(frameAggregator).configureSequenceIndices(activeSequenceIndices); convertToGlTextureFrames(frames); for (int i = 0; i < frames.size(); i++) { @@ -385,26 +560,76 @@ public void signalEndOfStream() { @Override public void close() { synchronized (lock) { + if (closed) { + return; + } closed = true; pendingFrames = null; } - submitToGlExecutor( + Callable releaseTask = () -> { - ImmutableList.Builder closeActions = ImmutableList.builder(); + ImmutableList.Builder> closeActions = ImmutableList.builder(); closeActions.addAll(getReleaseUnqueuedFramesActions()); - closeActions.add(hardwareBufferConverter::close); + if (frameToGlTextureConverter != null) { + closeActions.add(frameToGlTextureConverter::close); + } for (int i = 0; i < preProcessingChains.size(); i++) { GlTextureFrameProcessorChain processorChain = preProcessingChains.valueAt(i); closeActions.add(processorChain::close); } - closeActions - .add(frameAggregator::close) - .add(compositingProcessor::close) - .add(postProcessingChain::close) - .add(frameWriterGlTextureFrameConsumer::close); - runAllAndAccumulateExceptions(closeActions.build().toArray(new ThrowingRunnable[0])); + if (frameAggregator != null) { + closeActions.add(frameAggregator::close); + } + if (compositingProcessor != null) { + closeActions.add(compositingProcessor::close); + } + if (postProcessingChain != null) { + closeActions.add(postProcessingChain::close); + } + closeActions.add(frameWriterGlTextureFrameConsumer::close); + if (isGlSetup && shouldReleaseGlResources) { + closeActions.add(() -> releaseOpenGl(glObjectsProvider)); + } + runAllAndAccumulateExceptions(closeActions.build().toArray(new ThrowingRunnable[0])); return null; - }); + }; + try { + Object unused = glExecutorService.submit(releaseTask).get(RELEASE_TIMEOUT_MS, MILLISECONDS); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + errorConsumer.accept(VideoFrameProcessingException.from(e)); + } catch (ExecutionException | TimeoutException e) { + errorConsumer.accept(VideoFrameProcessingException.from(e)); + } finally { + if (shouldReleaseGlResources) { + shutdownGlExecutorService(glExecutorService); + } + } + } + + private void initializePipeline() { + ColorInfo workingColorSpace = checkNotNull(this.workingColorSpace); + frameToGlTextureConverter = + frameToGlTextureConverterFactory.create(workingColorSpace, errorConsumer); + postProcessingChain = + new GlTextureFrameProcessorChain( + context, + glObjectsProvider, + glExecutorService, + errorConsumer, + frameWriterGlTextureFrameConsumer, + KEY_COMPOSITION_EFFECTS, + workingColorSpace); + compositingProcessor = + glTextureFrameCompositorFactory.create( + glObjectsProvider, + workingColorSpace, + errorConsumer, + glExecutorService, + postProcessingChain); + frameAggregator = + new GlTextureFrameAggregator(compositingProcessor, glExecutorService, errorConsumer); + isPipelineInitialized = true; } private void submitToGlExecutor(Callable operation) { @@ -433,8 +658,9 @@ private void queueOrRetry(int sequenceIndex) { glObjectsProvider, glExecutorService, errorConsumer, - frameAggregator.getInputConsumer(sequenceIndex), - KEY_ITEM_EFFECTS); + checkNotNull(frameAggregator).getInputConsumer(sequenceIndex), + KEY_ITEM_EFFECTS, + checkNotNull(this.workingColorSpace)); preProcessingChains.put(sequenceIndex, processingChain); } @@ -464,21 +690,29 @@ private void convertToGlTextureFrames(List frames) if (!waitAndCloseFence(asyncFrame)) { GLES20.glFinish(); } - convertedGlTextureFrames.put( - sequenceIndex, - checkNotNull( - hardwareBufferConverter.convert( - (HardwareBufferFrame) frame, glExecutorService, listenerExecutor, listener))); + @Nullable + GlTextureFrame convertedFrame = + checkNotNull(frameToGlTextureConverter) + .convert(frame, glExecutorService, listenerExecutor, listener); + if (convertedFrame == null) { + // The converter was closed while this batch was being converted. + Log.w(TAG, "Converter closed, dropping frame for sequence = " + sequenceIndex); + continue; + } + convertedGlTextureFrames.put(sequenceIndex, convertedFrame); } } private void handleError(Exception exception) { synchronized (lock) { + if (closed) { + return; + } pendingFrames = null; } runAllAndAccumulateExceptions( /* errorConsumer= */ exception::addSuppressed, - getReleaseUnqueuedFramesActions().toArray(new ThrowingRunnable[0])); + getReleaseUnqueuedFramesActions().toArray(new ThrowingRunnable[0])); convertedGlTextureFrames.clear(); glTextureFramesQueuedDownstream.clear(); listenerExecutor.execute(() -> listener.onError(VideoFrameProcessingException.from(exception))); @@ -486,7 +720,9 @@ private void handleError(Exception exception) { private void onFramesQueued() { synchronized (lock) { - if (checkNotNull(pendingFrames).size() != glTextureFramesQueuedDownstream.size()) { + if (closed + || pendingFrames == null + || pendingFrames.size() != glTextureFramesQueuedDownstream.size()) { return; } } @@ -524,8 +760,8 @@ private static int extractSequenceIndex(Frame frame) { return (Integer) checkNotNull(frame.getMetadata().get(KEY_COMPOSITION_SEQUENCE_INDEX)); } - private ImmutableList getReleaseUnqueuedFramesActions() { - ImmutableList.Builder actions = ImmutableList.builder(); + private ImmutableList> getReleaseUnqueuedFramesActions() { + ImmutableList.Builder> actions = ImmutableList.builder(); for (int i = 0; i < convertedGlTextureFrames.size(); i++) { GlTextureFrame glTextureFrame = convertedGlTextureFrames.valueAt(i); if (!glTextureFramesQueuedDownstream.contains(glTextureFrame)) { @@ -539,4 +775,22 @@ private ImmutableList getReleaseUnqueuedFramesActions() { }); return actions.build(); } + + private static ColorInfo resolveWorkingColorspace(Format format) { + ColorInfo inputColorInfo = format.colorInfo == null ? SDR_BT709_LIMITED : format.colorInfo; + if (Objects.equals(format.sampleMimeType, MimeTypes.IMAGE_JPEG_R) + && inputColorInfo.colorTransfer == C.COLOR_TRANSFER_SRGB) { + // UltraHDR image input. + return BT2020_LINEAR; + } + if (isWideColorGamut(inputColorInfo)) { + // Process HDR in linear light. FrameWriterGlTextureFrameConsumer converts frames to an + // electrical color space before writing them, unless an effect already did. + return BT2020_LINEAR; + } else { + // All SDR input are treated as sRGB. + // TODO(b/545591224): Allow converting outputting to other gamut, for example BT.601. + return BT709_SRGB; + } + } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/DefaultGlTextureFrameCompositingProcessor.java b/libraries/effect/src/main/java/androidx/media3/effect/DefaultGlTextureFrameCompositor.java similarity index 80% rename from libraries/effect/src/main/java/androidx/media3/effect/DefaultGlTextureFrameCompositingProcessor.java rename to libraries/effect/src/main/java/androidx/media3/effect/DefaultGlTextureFrameCompositor.java index b6d24a2d41a..3a2c2cfa12e 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/DefaultGlTextureFrameCompositingProcessor.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/DefaultGlTextureFrameCompositor.java @@ -18,12 +18,14 @@ import static androidx.media3.effect.DefaultGlFrameProcessor.KEY_COMPOSITION_SEQUENCE_INDEX; import static androidx.media3.effect.DefaultGlFrameProcessor.KEY_COMPOSITOR_SETTINGS; import static androidx.media3.effect.FrameProcessorUtils.runAllAndAccumulateExceptions; +import static androidx.media3.effect.FrameProcessorUtils.useHighPrecisionColorComponents; import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkNotNull; import androidx.annotation.GuardedBy; import androidx.annotation.Nullable; -import androidx.annotation.RequiresApi; +import androidx.annotation.VisibleForTesting; +import androidx.media3.common.ColorInfo; import androidx.media3.common.GlObjectsProvider; import androidx.media3.common.GlTextureInfo; import androidx.media3.common.VideoCompositorSettings; @@ -31,6 +33,7 @@ import androidx.media3.common.util.Consumer; import androidx.media3.common.util.GlUtil.GlException; import androidx.media3.common.util.Size; +import androidx.media3.effect.GlTextureFrameCompositor.CompositorGlProgram; import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableMap; import java.util.HashMap; @@ -44,23 +47,61 @@ * *

Methods in this class must be called on a GL thread. */ -@RequiresApi(26) -/* package */ final class DefaultGlTextureFrameCompositingProcessor implements AutoCloseable { +/* package */ final class DefaultGlTextureFrameCompositor implements GlTextureFrameCompositor { + + private static final int DEFAULT_COMPOSITOR_CAPACITY = 2; + + /** + * A {@link GlTextureFrameCompositor.Factory} that creates {@link DefaultGlTextureFrameCompositor} + * instances. + */ + public static final class Factory implements GlTextureFrameCompositor.Factory { + private final CompositorGlProgram.Factory compositorGlProgramFactory; + private final TexturePool.Factory texturePoolFactory; + + /** + * Creates an instance. + * + * @param compositorGlProgramFactory The {@link CompositorGlProgram.Factory}. + */ + public Factory(CompositorGlProgram.Factory compositorGlProgramFactory) { + this( + compositorGlProgramFactory, + workingColorSpace -> + new TexturePool( + useHighPrecisionColorComponents(workingColorSpace), + /* capacity= */ DEFAULT_COMPOSITOR_CAPACITY)); + } - /** Draws multiple input frames onto one output texture using a GL program. */ - public interface CompositorGlProgram { /** - * Draws the input frames onto the output texture. + * Creates an instance for testing only. * - * @param framesToComposite The {@linkplain GlCompositionFrame input frames} to composite. - * @param outputTexture The {@link GlTextureInfo} to draw onto. - * @throws VideoFrameProcessingException If an error occurs during frame processing. + * @param compositorGlProgramFactory The {@link CompositorGlProgram.Factory}. + * @param texturePoolFactory The {@link TexturePool.Factory}. */ - void drawFrame(List framesToComposite, GlTextureInfo outputTexture) - throws GlException, VideoFrameProcessingException; + @VisibleForTesting + /* package */ Factory( + CompositorGlProgram.Factory compositorGlProgramFactory, + TexturePool.Factory texturePoolFactory) { + this.compositorGlProgramFactory = compositorGlProgramFactory; + this.texturePoolFactory = texturePoolFactory; + } - /** Releases all associated resources. */ - void release() throws VideoFrameProcessingException; + @Override + public GlTextureFrameCompositor create( + GlObjectsProvider glObjectsProvider, + ColorInfo workingColorSpace, + Consumer errorConsumer, + Executor glExecutor, + GlTextureFrameConsumer downstreamConsumer) { + return new DefaultGlTextureFrameCompositor( + glObjectsProvider, + texturePoolFactory.create(workingColorSpace), + errorConsumer, + compositorGlProgramFactory.create(), + glExecutor, + downstreamConsumer); + } } private final GlObjectsProvider glObjectsProvider; @@ -81,6 +122,7 @@ void drawFrame(List framesToComposite, GlTextureInfo outputT @Nullable private Executor pendingWakeupExecutor; + // This is reset when a new batch of frames is queued. private boolean inputStreamEnded; /** @@ -96,7 +138,7 @@ void drawFrame(List framesToComposite, GlTextureInfo outputT * this thread. * @param downstreamConsumer The {@link GlTextureFrameConsumer} that consumes the output frames. */ - public DefaultGlTextureFrameCompositingProcessor( + public DefaultGlTextureFrameCompositor( GlObjectsProvider glObjectsProvider, TexturePool outputTexturePool, Consumer errorConsumer, @@ -112,20 +154,7 @@ public DefaultGlTextureFrameCompositingProcessor( lock = new Object(); } - /** - * Attempts to queue frames for compositing. - * - *

If the compositor is at capacity, it returns {@code false} and the {@code wakeupListener} - * will be invoked on the {@code listenerExecutor} when capacity becomes available. - * - *

If {@code frames} contains only a single frame, the compositor is bypassed and the frame is - * passed directly downstream. - * - * @param frames The input frames to composite. - * @param listenerExecutor The executor to run the {@code wakeupListener} on. - * @param wakeupListener The callback to run when capacity is freed. - * @return {@code true} if queued successfully, {@code false} otherwise. - */ + @Override public boolean queue( List frames, Executor listenerExecutor, Runnable wakeupListener) throws VideoFrameProcessingException { @@ -142,6 +171,7 @@ public boolean queue( return false; } + inputStreamEnded = false; if (frames.size() == 1) { return downstreamConsumer.queue(frames.get(0), listenerExecutor, wakeupListener); } @@ -220,7 +250,7 @@ public boolean queue( return true; } - /** Notifies the current stream has ended. */ + @Override public void signalEndOfStream() { inputStreamEnded = true; if (pendingOutputFrame == null) { diff --git a/libraries/effect/src/main/java/androidx/media3/effect/DefaultHardwareBufferEffectsPipeline.kt b/libraries/effect/src/main/java/androidx/media3/effect/DefaultHardwareBufferEffectsPipeline.kt deleted file mode 100644 index 91b9f6b751c..00000000000 --- a/libraries/effect/src/main/java/androidx/media3/effect/DefaultHardwareBufferEffectsPipeline.kt +++ /dev/null @@ -1,539 +0,0 @@ -/* - * Copyright 2026 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package androidx.media3.effect - -import android.content.Context -import android.hardware.HardwareBuffer -import android.media.MediaFormat.COLOR_TRANSFER_SDR_VIDEO -import android.opengl.EGL14 -import android.opengl.EGL15 -import android.opengl.EGLContext -import android.opengl.EGLDisplay -import android.opengl.EGLExt -import android.opengl.EGLSurface -import android.opengl.GLES11Ext -import android.opengl.GLES20 -import android.opengl.Matrix -import android.os.Build.VERSION.SDK_INT -import androidx.annotation.RequiresApi -import androidx.media3.common.C -import androidx.media3.common.ColorInfo -import androidx.media3.common.ColorInfo.SDR_BT709_LIMITED -import androidx.media3.common.Format -import androidx.media3.common.GlObjectsProvider -import androidx.media3.common.GlTextureInfo -import androidx.media3.common.OverlaySettings -import androidx.media3.common.VideoFrameProcessingException -import androidx.media3.common.util.Consumer -import androidx.media3.common.util.ExperimentalApi -import androidx.media3.common.util.GlProgram -import androidx.media3.common.util.GlUtil -import androidx.media3.common.util.Size -import androidx.media3.common.video.SyncFenceWrapper -import androidx.media3.effect.DefaultVideoFrameProcessor.WORKING_COLOR_SPACE_ORIGINAL -import androidx.media3.effect.PacketConsumer.Packet -import com.google.common.collect.ImmutableList -import java.time.Duration -import java.util.concurrent.Executors -import java.util.concurrent.atomic.AtomicBoolean -import kotlinx.coroutines.CompletableDeferred -import kotlinx.coroutines.asCoroutineDispatcher -import kotlinx.coroutines.withContext -import kotlinx.coroutines.withTimeout - -/** - * A [PacketProcessor] that composites the [HardwareBufferFrame]s from the input packet onto the - * output [HardwareBufferFrame] using OpenGL. - */ -@RequiresApi(26) -@ExperimentalApi // TODO: b/449956776 - Remove once FrameConsumer API is finalized. -class DefaultHardwareBufferEffectsPipeline -private constructor( - private val context: Context, - private val hardwareBufferJniWrapper: HardwareBufferJniWrapper, - private val overlaySettingsProvider: (HardwareBufferFrame) -> OverlaySettings, -) : RenderingPacketConsumer, HardwareBufferFrameQueue> { - - /** Executor used for all blocking [SyncFenceWrapper.await] calls and GL operations. */ - private val internalExecutor = Executors.newSingleThreadExecutor() - private val internalDispatcher = internalExecutor.asCoroutineDispatcher() - private val isReleased = AtomicBoolean(false) - private val glObjectsProvider: GlObjectsProvider = DefaultGlObjectsProvider() - - private var eglDisplay: EGLDisplay? = null - private var eglContext: EGLContext? = null - private var eglSurface: EGLSurface? = null - private var compositorGlProgram: DefaultCompositorGlProgram? = null - private var externalCopyGlProgram: GlProgram? = null - - // TODO: b/479134794 - This being nullable and mutable adds complexity, simplify this. - private var outputBufferQueue: HardwareBufferFrameQueue? = null - private val overlayMatrixProvider = OverlayMatrixProvider() - private var targetFormat: Format? = null - private var presentationEffect: Presentation? = null - - override fun setRenderOutput(output: HardwareBufferFrameQueue?) { - this.outputBufferQueue = output - } - - override fun setErrorConsumer(errorConsumer: Consumer) {} - - override suspend fun queuePacket(packet: Packet>) { - check(!isReleased.get()) - when (packet) { - is Packet.EndOfStream -> outputBufferQueue!!.signalEndOfStream() - is Packet.Payload -> { - if (packet.payload.isNotEmpty()) { - if (targetFormat == null) { - val firstFrame = packet.payload[0] - targetFormat = firstFrame.format - presentationEffect = - Presentation.createForWidthAndHeight( - targetFormat!!.width, - targetFormat!!.height, - Presentation.LAYOUT_SCALE_TO_FIT, - ) - } - // Step 1: Dequeue an output frame, use the target format. - val outputFrame = getOutputFrame(targetFormat!!) - var inputReleaseFences: List = emptyList() - try { - // Step 2: Composite the input frames onto the output frame on the internal thread. - val result = processFrames(packet.payload, outputFrame) - val outputFrameWithMetadata = result.outputFrame - inputReleaseFences = result.inputReleaseFences - // Step 3: Forward output frame down stream. - outputBufferQueue!!.queue(outputFrameWithMetadata) - } catch (e: Exception) { - // Ensure the dequeued output frame is released on failures. - outputFrame.release(/* releaseFence= */ null) - throw e - } finally { - // Always release the input frames, ensuring the release waits until after drawing - // completes. - for (i in packet.payload.indices) { - val inputFrame = packet.payload[i] - val releaseFence = if (i < inputReleaseFences.size) inputReleaseFences[i] else null - inputFrame.release(releaseFence) - } - } - } - } - } - } - - override suspend fun release() { - if (!isReleased.getAndSet(true)) { - withContext(internalDispatcher) { - compositorGlProgram?.release() - compositorGlProgram = null - externalCopyGlProgram?.delete() - externalCopyGlProgram = null - eglSurface?.let { GlUtil.destroyEglSurface(checkNotNull(eglDisplay), it) } - eglSurface = null - eglDisplay?.let { glObjectsProvider.release(it) } - eglDisplay = null - eglContext = null - } - internalExecutor.shutdown() - } - } - - private fun maybeSetupGl() { - if (eglContext != null) { - return - } - val eglDisplay = GlUtil.getDefaultEglDisplay() - this.eglDisplay = eglDisplay - val eglContext = - glObjectsProvider.createEglContext( - eglDisplay, - /* openGlVersion= */ 3, - /* configAttributes= */ GlUtil.EGL_CONFIG_ATTRIBUTES_RGBA_8888, - ) - this.eglContext = eglContext - eglSurface = glObjectsProvider.createFocusedPlaceholderEglSurface(eglContext, eglDisplay) - compositorGlProgram = DefaultCompositorGlProgram(context) - val externalCopyGlProgram = - GlProgram( - context, - /* vertexShaderResId= */ R.raw.vertex_shader_transformation_es2, - /* fragmentShaderResId= */ R.raw.fragment_shader_transformation_sdr_external_es2, - ) - externalCopyGlProgram.setBufferAttribute( - "aFramePosition", - GlUtil.getNormalizedCoordinateBounds(), - GlUtil.HOMOGENEOUS_COORDINATE_VECTOR_SIZE, - ) - // TODO: b/475511306 - Ensure the frame is correctly rotated. - externalCopyGlProgram.setFloatsUniform( - "uTexTransformationMatrix", - GlUtil.create4x4IdentityMatrix(), - ) - externalCopyGlProgram.setFloatsUniform( - "uTransformationMatrix", - GlUtil.create4x4IdentityMatrix(), - ) - externalCopyGlProgram.setFloatsUniform("uRgbMatrix", GlUtil.create4x4IdentityMatrix()) - externalCopyGlProgram.setIntUniform("uSdrWorkingColorSpace", WORKING_COLOR_SPACE_ORIGINAL) - externalCopyGlProgram.setIntUniform("uOutputColorTransfer", COLOR_TRANSFER_SDR_VIDEO) - this.externalCopyGlProgram = externalCopyGlProgram - } - - private suspend fun processFrames( - inputFrames: ImmutableList, - outputFrame: HardwareBufferFrame, - ): ProcessedFrameResult = - withContext(internalDispatcher) { - val inputFrameInfos = mutableListOf() - val inputEglImages = mutableListOf() - val inputTextures = mutableListOf() - val intermediateFbos = mutableListOf() - var outputEglImage = 0L - var outputTexId = 0 - var outputFboId = 0 - - try { - // Ensure the OpenGL context is configured. - maybeSetupGl() - - val outputHardwareBuffer = checkNotNull(outputFrame.hardwareBuffer) - val outputWidth = outputHardwareBuffer.width - val outputHeight = outputHardwareBuffer.height - if (outputWidth <= 0 || outputHeight <= 0) { - throw VideoFrameProcessingException( - "Invalid output dimensions: ${outputWidth}x${outputHeight}. " + - "Format: ${outputFrame.format.width}x${outputFrame.format.height}. " + - "HardwareBuffer: ${outputHardwareBuffer.width}x${outputHardwareBuffer.height}." - ) - } - - val targetWidth = targetFormat!!.width - val targetHeight = targetFormat!!.height - overlayMatrixProvider.configure(Size(targetWidth, targetHeight)) - - for (i in inputFrames.indices) { - val inputFrame = inputFrames[i] - val hardwareBuffer = checkNotNull(inputFrame.hardwareBuffer) - - // TODO: b/479415385 - Replace this with a GPU wait. - // Wait on the input fences, to ensure the input frames are ready to be read from. - waitAndClose(inputFrame.acquireFence) - - // Create an EGL image and texture for each input frame. - val eglImage = - hardwareBufferJniWrapper.nativeCreateEglImageFromHardwareBuffer( - checkNotNull(eglDisplay).nativeHandle, - hardwareBuffer, - ) - if (eglImage == 0L) throw GlUtil.GlException("Failed to create input EGLImage") - inputEglImages.add(eglImage) - - val isRgba8888 = hardwareBuffer.format == HardwareBuffer.RGBA_8888 - val target = if (isRgba8888) GLES20.GL_TEXTURE_2D else GLES11Ext.GL_TEXTURE_EXTERNAL_OES - - // TODO: b/459374133 - Reuse textures. - val texId = GlUtil.generateTexture() - GLES20.glBindTexture(target, texId) - check(hardwareBufferJniWrapper.nativeBindEGLImage(target, eglImage)) - inputTextures.add(texId) - - var finalTexId = texId - val isExternalInput = !isRgba8888 - - // Draw into a GL_TEXTURE_2D if the input is an external texture, as the compositor can - // only handle GL_TEXTURE_2D. - // TODO: b/286211012 - Handle external textures in the compositor. - if (isExternalInput) { - // TODO: b/459374133 - Reuse textures. - // TODO: b/449957627 - This will not correctly handle mixing inputs with different - // colors. Map Everything to a working color space before compositing. - val internalTexId = - GlUtil.createTexture( - hardwareBuffer.width, - hardwareBuffer.height, - /* useHighPrecisionColorComponents= */ false, - ) - inputTextures.add(internalTexId) - val intermediateFboId = GlUtil.createFboForTexture(internalTexId) - intermediateFbos.add(intermediateFboId) - GlUtil.focusFramebufferUsingCurrentContext( - intermediateFboId, - hardwareBuffer.width, - hardwareBuffer.height, - ) - val copyProgram = checkNotNull(externalCopyGlProgram) - copyProgram.use() - copyProgram.setSamplerTexIdUniform("uTexSampler", texId, /* texUnitIndex= */ 0) - copyProgram.bindAttributesAndUniforms() - GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, /* first= */ 0, /* count= */ 4) - GlUtil.checkGlError() - - finalTexId = internalTexId - } - - // Calculate the placement of each input texture on the output texture. - val textureInfo = - GlTextureInfo( - finalTexId, - /* fboId= */ C.INDEX_UNSET, - /* rboId= */ C.INDEX_UNSET, - hardwareBuffer.width, - hardwareBuffer.height, - ) - val overlaySettings = overlaySettingsProvider(inputFrame) - - // Configure presentation effect with current input size - presentationEffect!!.configure(hardwareBuffer.width, hardwareBuffer.height) - val presentationMatrix = - presentationEffect!!.getGlMatrixArray(inputFrame.presentationTimeUs) - - // Calculate overlay matrix assuming the input is logically target size - val overlayMatrix = - overlayMatrixProvider.getTransformationMatrix( - Size(targetWidth, targetHeight), - overlaySettings, - ) - - // Combine them: combined = overlayMatrix * presentationMatrix - val combinedMatrix = FloatArray(16) - Matrix.multiplyMM( - combinedMatrix, - /* resultOffset= */ 0, - overlayMatrix, - /* lhsOffset= */ 0, - presentationMatrix, - /* rhsOffset= */ 0, - ) - - inputFrameInfos.add(GlCompositionFrame(textureInfo, overlaySettings, combinedMatrix)) - } - - // Wait on the output fence, to ensure it is ready to be written to. - waitAndClose(outputFrame.acquireFence) - - // Create the output EGL image and texture. - outputEglImage = - hardwareBufferJniWrapper.nativeCreateEglImageFromHardwareBuffer( - checkNotNull(eglDisplay).nativeHandle, - outputHardwareBuffer, - ) - if (outputEglImage == 0L) { - throw GlUtil.GlException("Failed to create output EGLImage") - } - - // TODO: b/459374133 - Reuse textures. - outputTexId = GlUtil.generateTexture() - GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, outputTexId) - check(hardwareBufferJniWrapper.nativeBindEGLImage(GLES20.GL_TEXTURE_2D, outputEglImage)) - - outputFboId = GlUtil.createFboForTexture(outputTexId) - val outputTextureInfo = - GlTextureInfo( - outputTexId, - outputFboId, - /* rboId= */ C.INDEX_UNSET, - outputWidth, - outputHeight, - ) - - // Draw the input textures into the output texture. - checkNotNull(compositorGlProgram).drawFrame(inputFrameInfos, outputTextureInfo) - - var drawingCompletionFence: SyncFenceWrapper? = null - val inputReleaseFences = - ArrayList(/* initialCapacity= */ inputFrames.size) - val eglDisplay = checkNotNull(eglDisplay) - val extensions = EGL14.eglQueryString(eglDisplay, EGL14.EGL_EXTENSIONS) ?: "" - if (SDK_INT >= 33 && extensions.contains("EGL_ANDROID_native_fence_sync")) { - val sync = - EGL15.eglCreateSync( - eglDisplay, - /* type= */ EGLExt.EGL_SYNC_NATIVE_FENCE_ANDROID, - /* attrib_list= */ longArrayOf(EGL14.EGL_NONE.toLong()), - /* offset= */ 0, - ) - GlUtil.checkEglException("eglCreateSync failed") - if (sync != EGL15.EGL_NO_SYNC) { - try { - var nativeFence = EGLExt.eglDupNativeFenceFDANDROID(eglDisplay, sync) - GlUtil.checkEglException("eglDupNativeFenceFDANDROID failed") - if (!nativeFence.isValid) { - // Calling eglDupNativeFenceAndroid may produce an invalid fence the first time it - // is called. See b/18052459. - GLES20.glFlush() - nativeFence = EGLExt.eglDupNativeFenceFDANDROID(eglDisplay, sync) - GlUtil.checkEglException("eglDupNativeFenceFDANDROID failed after glFlush") - } - if (nativeFence.isValid) { - drawingCompletionFence = SyncFenceWrapper.of(/* syncFence= */ nativeFence) - for (i in inputFrames.indices) { - val inputNativeFence = EGLExt.eglDupNativeFenceFDANDROID(eglDisplay, sync) - GlUtil.checkEglException("eglDupNativeFenceFDANDROID failed for input frame") - check(inputNativeFence.isValid) - inputReleaseFences.add(SyncFenceWrapper.of(/* syncFence= */ inputNativeFence)) - } - } - } finally { - EGL15.eglDestroySync(eglDisplay, sync) - GlUtil.checkEglException("eglDestroySync failed") - } - } - } - - if (drawingCompletionFence == null) { - GLES20.glFinish() - } - - val outputFrameWithMetadata = - outputFrame - .buildUpon() - .setPresentationTimeUs(inputFrames[0].presentationTimeUs) - .setSequencePresentationTimeUs(inputFrames[0].sequencePresentationTimeUs) - .setReleaseTimeNs(inputFrames[0].releaseTimeNs) - .setFormat(inputFrames[0].format) - .setMetadata(inputFrames[0].metadata) - .setAcquireFence(drawingCompletionFence) - .build() - ProcessedFrameResult(outputFrameWithMetadata, inputReleaseFences) - } finally { - // Clean up GL resources. - for (fboId in intermediateFbos) { - GlUtil.deleteFbo(fboId) - } - if (outputFboId != 0) { - GlUtil.deleteFbo(outputFboId) - } - if (outputTexId != 0) { - GlUtil.deleteTexture(outputTexId) - } - if (outputEglImage != 0L) { - check( - hardwareBufferJniWrapper.nativeDestroyEGLImage( - checkNotNull(eglDisplay).nativeHandle, - outputEglImage, - ) - ) - } - for (texId in inputTextures) { - GlUtil.deleteTexture(texId) - } - for (eglImage in inputEglImages) { - check( - hardwareBufferJniWrapper.nativeDestroyEGLImage( - checkNotNull(eglDisplay).nativeHandle, - eglImage, - ) - ) - } - } - } - - private suspend fun getOutputFrame(format: Format): HardwareBufferFrame { - val width = format.width - val height = format.height - - val bufferFormat = - HardwareBufferFrameQueue.FrameFormat.Builder() - .setWidth(width) - .setHeight(height) - .setRotationDegrees(format.rotationDegrees) - .setPixelFormat( - if (ColorInfo.isTransferHdr(format.colorInfo)) HardwareBuffer.RGBA_1010102 - else HardwareBuffer.RGBA_8888 - ) - .setUsageFlags( - HardwareBuffer.USAGE_GPU_COLOR_OUTPUT or HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE - ) - .setColorInfo(format.colorInfo ?: SDR_BT709_LIMITED) - .build() - - // Try and get an output buffer from the queue. If not immediately available, suspend until - // notified and retry. - val capacityAvailable = CompletableDeferred() - var outputFrame = - outputBufferQueue!!.dequeue(bufferFormat) - /* wakeupListener= */ { - capacityAvailable.complete(Unit) - } - if (outputFrame == null) { - withTimeout(TIMEOUT_MS) { capacityAvailable.await() } - outputFrame = outputBufferQueue!!.dequeue(bufferFormat) /* wakeupListener= */ {} - // Throw the second time there is no buffer available. - check(outputFrame != null) - } - return outputFrame - } - - // TODO: b/479415385 - Replace this with a GPU wait. - private fun waitAndClose(fence: SyncFenceWrapper?) { - fence?.let { fence -> - check(fence.await(FENCE_TIMEOUT)) - fence.close() - } - } - - /** - * Result of [processFrames]. - * - * @property outputFrame The processed output [HardwareBufferFrame]. - * @property inputReleaseFences The [SyncFenceWrapper] instances that signal when the input frames - * have been read by the GPU. - */ - private data class ProcessedFrameResult( - val outputFrame: HardwareBufferFrame, - val inputReleaseFences: List, - ) - - companion object { - // It can take multiple seconds for the encoder to be configured and the first frame to be - // encoded. - private const val TIMEOUT_MS = 10_000L - private val FENCE_TIMEOUT = Duration.ofMillis(500) - - /** - * Creates a new [DefaultHardwareBufferEffectsPipeline], that uses a default - * [StaticOverlaySettings] to composite each sequence.. - */ - @JvmStatic - fun create( - context: Context, - hardwareBufferJniWrapper: HardwareBufferJniWrapper, - ): DefaultHardwareBufferEffectsPipeline { - val overlaySettingsProvider: (HardwareBufferFrame) -> OverlaySettings = { _ -> - StaticOverlaySettings.Builder().build() - } - return create(context, hardwareBufferJniWrapper, overlaySettingsProvider) - } - - /** - * Creates a new [DefaultHardwareBufferEffectsPipeline] that calls the given - * [overlaySettingsProvider] for each frame when compositing. - */ - @JvmStatic - fun create( - context: Context, - hardwareBufferJniWrapper: HardwareBufferJniWrapper, - overlaySettingsProvider: (HardwareBufferFrame) -> OverlaySettings, - ): DefaultHardwareBufferEffectsPipeline { - return DefaultHardwareBufferEffectsPipeline( - context, - hardwareBufferJniWrapper, - overlaySettingsProvider, - ) - } - } -} diff --git a/libraries/effect/src/main/java/androidx/media3/effect/DefaultShaderProgram.java b/libraries/effect/src/main/java/androidx/media3/effect/DefaultShaderProgram.java index a8702493e48..092fbf0759d 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/DefaultShaderProgram.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/DefaultShaderProgram.java @@ -211,8 +211,6 @@ public static DefaultShaderProgram createWithInternalSampler( @WorkingColorSpace int sdrWorkingColorSpace, @InputType int inputType) throws VideoFrameProcessingException { - checkState( - inputColorInfo.colorTransfer != C.COLOR_TRANSFER_SRGB || inputType == INPUT_TYPE_BITMAP); boolean isInputTransferHdr = ColorInfo.isTransferHdr(inputColorInfo); boolean isUsingUltraHdr = inputType == INPUT_TYPE_BITMAP && outputColorInfo.colorSpace == C.COLOR_SPACE_BT2020; @@ -284,6 +282,13 @@ public static DefaultShaderProgram createWithExternalSampler( boolean sampleWithNearest) throws VideoFrameProcessingException { boolean isInputTransferHdr = ColorInfo.isTransferHdr(inputColorInfo); + if (isInputTransferHdr) { + // In HDR editing mode the decoder output is sampled in YUV. + if (!GlUtil.isYuvTargetExtensionSupported()) { + throw new VideoFrameProcessingException( + "The EXT_YUV_target extension is required for HDR editing input."); + } + } int vertexShaderResId = isInputTransferHdr ? R.raw.vertex_shader_transformation_es3 @@ -294,11 +299,6 @@ public static DefaultShaderProgram createWithExternalSampler( : R.raw.fragment_shader_transformation_sdr_external_es2; GlProgram glProgram = createGlProgram(context, vertexShaderResId, fragmentShaderResId); if (isInputTransferHdr) { - // In HDR editing mode the decoder output is sampled in YUV. - if (!GlUtil.isYuvTargetExtensionSupported()) { - throw new VideoFrameProcessingException( - "The EXT_YUV_target extension is required for HDR editing input."); - } glProgram.setFloatsUniform( "uYuvToRgbColorTransform", inputColorInfo.colorRange == C.COLOR_RANGE_FULL @@ -420,8 +420,10 @@ private static DefaultShaderProgram createWithSampler( glProgram.setIntUniform("uSdrWorkingColorSpace", sdrWorkingColorSpace); checkArgument( outputColorTransfer == C.COLOR_TRANSFER_SDR + || outputColorTransfer == C.COLOR_TRANSFER_SRGB || outputColorTransfer == C.COLOR_TRANSFER_LINEAR); - // The SDR shader automatically applies a COLOR_TRANSFER_SDR EOTF. + // The SDR shader applies the COLOR_TRANSFER_SDR (SMPTE 170M) EOTF if the input is linear, + // passes through otherwise. glProgram.setIntUniform("uOutputColorTransfer", outputColorTransfer); } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/FrameCache.java b/libraries/effect/src/main/java/androidx/media3/effect/FrameCache.java index db825228ff8..ade0c653b64 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/FrameCache.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/FrameCache.java @@ -15,10 +15,13 @@ */ package androidx.media3.effect; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; import static com.google.common.base.Preconditions.checkArgument; import android.content.Context; import androidx.annotation.IntRange; +import androidx.annotation.RestrictTo; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.UnstableApi; @@ -29,7 +32,7 @@ * the possible fluctuation in video frame processing time between frames. */ @UnstableApi -public final class FrameCache implements GlEffect { +public final class FrameCache implements GlEffect, MetricsProvider { /** The capacity of the frame cache. */ public final int capacity; @@ -54,4 +57,10 @@ public GlShaderProgram toGlShaderProgram(Context context, boolean useHdr) throws VideoFrameProcessingException { return new FrameCacheGlShaderProgram(context, capacity, useHdr); } + + @Override + @RestrictTo(LIBRARY_GROUP) + public void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_EFFECT_BUFFER); + } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/FrameDropEffect.java b/libraries/effect/src/main/java/androidx/media3/effect/FrameDropEffect.java index 2805b98b6b6..06418293909 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/FrameDropEffect.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/FrameDropEffect.java @@ -16,14 +16,18 @@ package androidx.media3.effect; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; + import android.content.Context; +import androidx.annotation.RestrictTo; import androidx.media3.common.C; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.UnstableApi; /** Drops frames to lower average frame rate to around {@code targetFrameRate}. */ @UnstableApi -public final class FrameDropEffect implements GlEffect { +public final class FrameDropEffect implements GlEffect, MetricsProvider { private final float inputFrameRate; private final float targetFrameRate; @@ -63,6 +67,11 @@ public static FrameDropEffect createSimpleFrameDropEffect( return new FrameDropEffect(expectedFrameRate, targetFrameRate); } + private FrameDropEffect(float inputFrameRate, float targetFrameRate) { + this.inputFrameRate = inputFrameRate; + this.targetFrameRate = targetFrameRate; + } + @Override public GlShaderProgram toGlShaderProgram(Context context, boolean useHdr) throws VideoFrameProcessingException { @@ -73,8 +82,9 @@ public GlShaderProgram toGlShaderProgram(Context context, boolean useHdr) } } - private FrameDropEffect(float inputFrameRate, float targetFrameRate) { - this.inputFrameRate = inputFrameRate; - this.targetFrameRate = targetFrameRate; + @Override + @RestrictTo(LIBRARY_GROUP) + public void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_EFFECT_TEMPORAL); } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/FrameProcessorUtils.java b/libraries/effect/src/main/java/androidx/media3/effect/FrameProcessorUtils.java index a00ed0d0ef7..4eb3d7576a6 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/FrameProcessorUtils.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/FrameProcessorUtils.java @@ -15,22 +15,18 @@ */ package androidx.media3.effect; -import static android.os.Build.VERSION.SDK_INT; -import static com.google.common.base.Preconditions.checkState; import static com.google.common.util.concurrent.MoreExecutors.directExecutor; import static java.util.concurrent.TimeUnit.MILLISECONDS; import android.hardware.HardwareBuffer; -import android.hardware.SyncFence; -import android.opengl.EGL14; -import android.opengl.EGL15; +import android.opengl.EGLContext; import android.opengl.EGLDisplay; -import android.opengl.EGLExt; -import android.opengl.EGLSync; import android.opengl.GLES20; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; +import androidx.annotation.VisibleForTesting; import androidx.media3.common.C; +import androidx.media3.common.ColorInfo; import androidx.media3.common.GlObjectsProvider; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.Consumer; @@ -38,23 +34,21 @@ import androidx.media3.common.util.GlUtil; import androidx.media3.common.util.GlUtil.GlException; import androidx.media3.common.util.Log; +import androidx.media3.common.util.ThrowingRunnable; import androidx.media3.common.video.AsyncFrame; import androidx.media3.common.video.SyncFenceWrapper; -import com.google.common.collect.ImmutableList; import com.google.common.util.concurrent.FutureCallback; import com.google.common.util.concurrent.Futures; import com.google.common.util.concurrent.ListeningExecutorService; -import java.time.Duration; import java.util.concurrent.Callable; import java.util.concurrent.ExecutorService; +/** Utility methods for FrameProcessor implementations. */ @ExperimentalApi // TODO: b/505721737 Remove once FrameProcessor is production ready. -@RequiresApi(26) public final class FrameProcessorUtils { private static final String TAG = "FrameProcessorUtils"; private static final long TIMEOUT_MS = 500; - private static final Duration FENCE_TIMEOUT_DURATION = Duration.ofMillis(TIMEOUT_MS); /** Data class holding handles to {@code EglImage}, texture ID and frame buffer object ID. */ public static final class EglImageTextureWrapper { @@ -85,17 +79,68 @@ public EglImageTextureWrapper withOutputTexId(int outputTexId) { } } + /** OpenGL ES 2.0 context created. */ + public static final int OPEN_GL_VERSION_2 = 2; + + /** 10-bit HDR-capable OpenGL ES 3.0 context created. */ + public static final int OPEN_GL_VERSION_3 = 3; + /** - * Sets up the OpenGL resources. + * Sets up the OpenGL context on the current thread. * - *

This method must run on the thread that owns the OpenGL context. + *

Attempts to create an OpenGL ES 3.0 context with 10-bit HDR configuration ({@link + * GlUtil#EGL_CONFIG_ATTRIBUTES_RGBA_1010102}) if surfaceless contexts are supported. This allows + * the pipeline to handle SDR and HDR contents without recreating the context. + * + *

Falls back to OpenGL ES 2.0 with {@link GlUtil#EGL_CONFIG_ATTRIBUTES_RGBA_8888} if + * surfaceless contexts are unsupported (e.g., on emulators) or if ES 3.0 context creation fails. + * + * @param glObjectsProvider The {@link GlObjectsProvider}. + * @return The created OpenGL version ({@link #OPEN_GL_VERSION_3} or {@link #OPEN_GL_VERSION_2}). */ - public static void setupOpenGl(GlObjectsProvider glObjectsProvider) throws GlException { + public static int setupOpenGl(GlObjectsProvider glObjectsProvider) throws GlException { + return setupOpenGl(glObjectsProvider, GlUtil.isSurfacelessContextExtensionSupported()); + } + + @VisibleForTesting(otherwise = VisibleForTesting.PACKAGE_PRIVATE) + /* package */ static int setupOpenGl( + GlObjectsProvider glObjectsProvider, boolean isSurfacelessContextExtensionSupported) + throws GlException { + EGLDisplay eglDisplay = GlUtil.getDefaultEglDisplay(); + // 10-bit contexts require EGL_KHR_surfaceless_context support because placeholder surfaces + // default to 8-bit RGBA_8888, which causes EGL_BAD_MATCH on non-surfaceless platforms (like + // emulators). + if (isSurfacelessContextExtensionSupported) { + EGLContext eglContext = null; + try { + eglContext = + glObjectsProvider.createEglContext( + eglDisplay, /* openGlVersion= */ 3, GlUtil.EGL_CONFIG_ATTRIBUTES_RGBA_1010102); + glObjectsProvider.createFocusedPlaceholderEglSurface(eglContext, eglDisplay); + return OPEN_GL_VERSION_3; + } catch (GlException e) { + Log.w(TAG, "Failed to create OpenGL ES 3.0 context or surface", e); + if (eglContext != null) { + GlUtil.destroyEglContext(eglDisplay, eglContext); + } + } + } + + return setupOpenGl2(glObjectsProvider); + } + + private static int setupOpenGl2(GlObjectsProvider glObjectsProvider) throws GlException { EGLDisplay eglDisplay = GlUtil.getDefaultEglDisplay(); - glObjectsProvider.createFocusedPlaceholderEglSurface( + EGLContext eglContext = glObjectsProvider.createEglContext( - eglDisplay, /* openGlVersion= */ 2, GlUtil.EGL_CONFIG_ATTRIBUTES_RGBA_8888), - eglDisplay); + eglDisplay, /* openGlVersion= */ 2, GlUtil.EGL_CONFIG_ATTRIBUTES_RGBA_8888); + try { + glObjectsProvider.createFocusedPlaceholderEglSurface(eglContext, eglDisplay); + return OPEN_GL_VERSION_2; + } catch (GlException e) { + GlUtil.destroyEglContext(eglDisplay, eglContext); + throw e; + } } /** @@ -124,6 +169,7 @@ public static void shutdownGlExecutorService(ExecutorService glExecutorService) * *

This method should only be called on the thread that owns the OpenGL context. */ + @RequiresApi(26) public static EglImageTextureWrapper createAndBindEglImage( EGLDisplay eglDisplay, HardwareBuffer buffer, @@ -156,6 +202,7 @@ public static EglImageTextureWrapper createAndBindEglImage( } /** Releases the EGL image, focused frame buffer object, and texture. */ + @RequiresApi(26) public static void releaseEglImageTexture( EglImageTextureWrapper eglImageTextureWrapper, HardwareBufferJniWrapper jniWrapper) throws GlException { @@ -170,54 +217,6 @@ public static void releaseEglImageTexture( boolean unused = jniWrapper.nativeDestroyEGLImage(eglDisplay, eglImageTextureWrapper.eglImage); } - /** Generates the requested number of native sync fences. Must be called on the GL thread. */ - public static ImmutableList generateSyncFences(int count) throws GlException { - // TODO: b/505721737 - Move to utility class. - EGLDisplay eglDisplay = GlUtil.getDefaultEglDisplay(); - String extensions = EGL14.eglQueryString(eglDisplay, EGL14.EGL_EXTENSIONS); - if (SDK_INT < 33 || !extensions.contains("EGL_ANDROID_native_fence_sync")) { - return ImmutableList.of(); - } - EGLSync eglSync = EGL15.EGL_NO_SYNC; - ImmutableList.Builder fences = new ImmutableList.Builder<>(); - try { - eglSync = - EGL15.eglCreateSync( - eglDisplay, - EGLExt.EGL_SYNC_NATIVE_FENCE_ANDROID, - /* attrib_list= */ new long[] {EGL14.EGL_NONE}, - /* offset= */ 0); - GlUtil.checkEglException("eglCreateSync failed"); - if (eglSync == EGL15.EGL_NO_SYNC) { - return ImmutableList.of(); - } - SyncFence syncFence = EGLExt.eglDupNativeFenceFDANDROID(eglDisplay, eglSync); - GlUtil.checkEglException("eglDupNativeFenceFDANDROID failed"); - if (!syncFence.isValid()) { - // Calling eglDupNativeFenceAndroid may produce an invalid fence the first time it - // is called. See b/18052459. - GLES20.glFlush(); - syncFence = EGLExt.eglDupNativeFenceFDANDROID(eglDisplay, eglSync); - GlUtil.checkEglException("eglDupNativeFenceFDANDROID failed after glFlush"); - } - if (!syncFence.isValid()) { - return ImmutableList.of(); - } - fences.add(SyncFenceWrapper.of(syncFence)); - for (int i = 0; i < count - 1; i++) { - SyncFence duplicatedFence = EGLExt.eglDupNativeFenceFDANDROID(eglDisplay, eglSync); - GlUtil.checkEglException("eglDupNativeFenceFDANDROID failed for input frame"); - checkState(duplicatedFence.isValid()); - fences.add(SyncFenceWrapper.of(duplicatedFence)); - } - - } finally { - EGL15.eglDestroySync(eglDisplay, eglSync); - GlUtil.checkEglException("eglDestroySync failed"); - } - return fences.build(); - } - /** * Awaits the acquire fence on the {@link AsyncFrame}. Returns {@code true} if the fence was * signaled before the timeout, {@code false} otherwise. @@ -228,7 +227,7 @@ public static boolean waitAndCloseFence(AsyncFrame asyncFrame) { return true; } - boolean fenceSignaled = acquireFence.await(FENCE_TIMEOUT_DURATION); + boolean fenceSignaled = acquireFence.awaitMs(TIMEOUT_MS); acquireFence.close(); return fenceSignaled; } @@ -256,12 +255,6 @@ public void onFailure(Throwable t) { directExecutor()); } - /** A runnable that throws an exception. */ - public interface ThrowingRunnable { - /** Runs the operation. */ - void run() throws Exception; - } - /** * Executes multiple throwing actions sequentially, ensuring all are run even if some fail. * @@ -271,7 +264,7 @@ public interface ThrowingRunnable { * @param actions The throwing actions to execute. * @throws VideoFrameProcessingException If any of the actions fail. */ - public static void runAllAndAccumulateExceptions(ThrowingRunnable... actions) + public static void runAllAndAccumulateExceptions(ThrowingRunnable... actions) throws VideoFrameProcessingException { @Nullable VideoFrameProcessingException exception = runAllAndAccumulateExceptionInternal(actions); @@ -291,7 +284,7 @@ public static void runAllAndAccumulateExceptions(ThrowingRunnable... actions) * @param actions The throwing actions to execute. */ public static void runAllAndAccumulateExceptions( - Consumer errorConsumer, ThrowingRunnable... actions) { + Consumer errorConsumer, ThrowingRunnable... actions) { @Nullable VideoFrameProcessingException exception = runAllAndAccumulateExceptionInternal(actions); if (exception != null) { @@ -301,9 +294,9 @@ public static void runAllAndAccumulateExceptions( @Nullable private static VideoFrameProcessingException runAllAndAccumulateExceptionInternal( - ThrowingRunnable... actions) { + ThrowingRunnable... actions) { @Nullable VideoFrameProcessingException firstException = null; - for (ThrowingRunnable action : actions) { + for (ThrowingRunnable action : actions) { if (action == null) { continue; } @@ -321,5 +314,19 @@ private static VideoFrameProcessingException runAllAndAccumulateExceptionInterna return firstException; } + /** + * Returns whether the color space requires high precision (16-bit floating-point) color + * components. + */ + public static boolean useHighPrecisionColorComponents(@Nullable ColorInfo colorInfo) { + if (colorInfo == null) { + return false; + } + return colorInfo.colorTransfer == C.COLOR_TRANSFER_LINEAR + || ColorInfo.isTransferHdr(colorInfo) + || ColorInfo.isWideColorGamut(colorInfo) + || colorInfo.lumaBitdepth > 8; + } + private FrameProcessorUtils() {} } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/FrameToGlTextureConverter.java b/libraries/effect/src/main/java/androidx/media3/effect/FrameToGlTextureConverter.java new file mode 100644 index 00000000000..8794ebb1002 --- /dev/null +++ b/libraries/effect/src/main/java/androidx/media3/effect/FrameToGlTextureConverter.java @@ -0,0 +1,66 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect; + +import androidx.annotation.Nullable; +import androidx.media3.common.ColorInfo; +import androidx.media3.common.VideoFrameProcessingException; +import androidx.media3.common.util.Consumer; +import androidx.media3.common.util.ExperimentalApi; +import androidx.media3.common.video.Frame; +import androidx.media3.common.video.FrameProcessor.Listener; +import java.util.concurrent.Executor; + +/** Converts from {@link Frame} to {@link GlTextureFrame}. */ +@ExperimentalApi // TODO: b/505721737 Remove once FrameProcessor is production ready. +/* package */ interface FrameToGlTextureConverter extends AutoCloseable { + + /** A factory for {@link FrameToGlTextureConverter} instances. */ + interface Factory { + FrameToGlTextureConverter create( + ColorInfo outputColorInfo, Consumer errorConsumer); + } + + // TODO: b/517424999 - Unify the listeners to follow the same pattern as FrameProcessor. + /** + * Converts from {@link Frame} to {@link GlTextureFrame}, or returns {@code null} if this + * converter is closed. + * + *

The returned {@link GlTextureFrame}'s texture is in standard OpenGL coordinate space + * (upright, Y-up, with origin at bottom-left of the image). + */ + @Nullable + GlTextureFrame convert( + Frame frame, Executor glExecutor, Executor listenerExecutor, Listener listener) + throws VideoFrameProcessingException; + + /** + * Releases the resources for a converted {@link Frame}. + * + *

Don't call this method if the {@linkplain #convert converted} {@link GlTextureFrame} is + * accepted by a downstream {@link GlTextureFrameConsumer}. + * + *

This releases associated resources without notifying the {@link Listener#onFrameProcessed} + * that was passed in via {@link #convert}. + * + *

This is used when a converted frame is rejected by downstream pipeline consumers, preserving + * the underlying frame for subsequent queue retries. + */ + void releaseGlResources(Frame frame) throws VideoFrameProcessingException; + + @Override + void close() throws VideoFrameProcessingException; +} diff --git a/libraries/effect/src/main/java/androidx/media3/effect/FrameWriterGlTextureFrameConsumer.java b/libraries/effect/src/main/java/androidx/media3/effect/FrameWriterGlTextureFrameConsumer.java index 0e6d2045714..af406c9b4d9 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/FrameWriterGlTextureFrameConsumer.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/FrameWriterGlTextureFrameConsumer.java @@ -15,12 +15,17 @@ */ package androidx.media3.effect; +import static androidx.media3.common.ColorInfo.isWideColorGamut; import static androidx.media3.common.video.Frame.USAGE_GPU_COLOR_OUTPUT; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT2020_HLG; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT709_SRGB; import static androidx.media3.effect.FrameProcessorUtils.createAndBindEglImage; -import static androidx.media3.effect.FrameProcessorUtils.generateSyncFences; import static androidx.media3.effect.FrameProcessorUtils.releaseEglImageTexture; +import static androidx.media3.effect.FrameProcessorUtils.runAllAndAccumulateExceptions; +import static androidx.media3.effect.FrameProcessorUtils.useHighPrecisionColorComponents; import static androidx.media3.effect.FrameProcessorUtils.waitAndCloseFence; import static com.google.common.base.Preconditions.checkArgument; +import static com.google.common.base.Preconditions.checkNotNull; import android.content.Context; import android.hardware.HardwareBuffer; @@ -28,33 +33,45 @@ import android.opengl.GLES20; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; +import androidx.media3.common.C; +import androidx.media3.common.ColorInfo; +import androidx.media3.common.Format; +import androidx.media3.common.GlTextureInfo; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.ExperimentalApi; import androidx.media3.common.util.GlUtil; import androidx.media3.common.util.GlUtil.GlException; import androidx.media3.common.util.Log; +import androidx.media3.common.util.Size; import androidx.media3.common.video.AsyncFrame; import androidx.media3.common.video.FrameWriter; import androidx.media3.common.video.HardwareBufferFrame; import androidx.media3.common.video.SyncFenceWrapper; import androidx.media3.effect.FrameProcessorUtils.EglImageTextureWrapper; import com.google.common.collect.ImmutableList; +import java.util.Objects; import java.util.concurrent.Executor; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; -/** A {@link GlTextureFrameConsumer} that writes the input to a {@link FrameWriter}. */ +/** Writes input texture frames to a {@link FrameWriter}. */ @ExperimentalApi // TODO: b/505721737 Remove once FrameProcessor is production ready. @RequiresApi(26) /* package */ final class FrameWriterGlTextureFrameConsumer implements GlTextureFrameConsumer { private static final String TAG = "FrameWriterGlTexCons"; + private static final long OUTPUT_USAGE = USAGE_GPU_COLOR_OUTPUT; private final Context context; private final FrameWriter frameWriter; private final HardwareBufferJniWrapper hardwareBufferJniWrapper; @Nullable private DefaultShaderProgram defaultShaderProgram; + @Nullable private ColorConversionShaderProgram colorConversionShaderProgram; + @Nullable private GlTextureInfo colorConversionOutputTexture; private @MonotonicNonNull EGLDisplay eglDisplay; + private @MonotonicNonNull Size inputSize; + private @MonotonicNonNull ColorInfo inputColorInfo; + private @MonotonicNonNull Format outputFormat; private boolean isFrameWriterConfigured; FrameWriterGlTextureFrameConsumer( @@ -70,10 +87,13 @@ public boolean queue( GlTextureFrame inputFrame, Executor listenerExecutor, Runnable wakeupListener) throws VideoFrameProcessingException { if (!isFrameWriterConfigured) { - frameWriter.configure(inputFrame.format, USAGE_GPU_COLOR_OUTPUT); + outputFormat = establishOutputFormat(inputFrame.format); + frameWriter.configure(outputFormat, OUTPUT_USAGE); isFrameWriterConfigured = true; } + maybeReconfigureShader(inputFrame); + AsyncFrame asyncFrame = frameWriter.dequeueInputFrame(listenerExecutor, wakeupListener); if (asyncFrame == null) { return false; @@ -93,6 +113,17 @@ public boolean queue( if (eglDisplay == null) { eglDisplay = GlUtil.getDefaultEglDisplay(); } + int currentTexId = inputFrame.glTextureInfo.texId; + if (colorConversionShaderProgram != null) { + // Convert into the intermediate texture first. createAndBindEglImage below focuses the + // output buffer, which the output pass then draws into. + GlTextureInfo conversionOutput = checkNotNull(colorConversionOutputTexture); + GlUtil.focusFramebufferUsingCurrentContext( + conversionOutput.fboId, conversionOutput.width, conversionOutput.height); + checkNotNull(colorConversionShaderProgram) + .drawFrame(currentTexId, inputFrame.presentationTimeUs); + currentTexId = conversionOutput.texId; + } eglImageTextureWrapper = createAndBindEglImage( eglDisplay, @@ -101,22 +132,15 @@ public boolean queue( /* target= */ GLES20.GL_TEXTURE_2D, /* writesToBoundImage= */ true); - if (defaultShaderProgram == null) { - defaultShaderProgram = - DefaultShaderProgram.create( - context, - /* matrixTransformations= */ ImmutableList.of(), - /* rgbMatrices= */ ImmutableList.of(), - /* useHdr= */ false); - } - defaultShaderProgram.drawFrame(inputFrame.glTextureInfo.texId, inputFrame.presentationTimeUs); + GlUtil.clearFocusedBuffersOpaque(); + checkNotNull(defaultShaderProgram).drawFrame(currentTexId, inputFrame.presentationTimeUs); GlUtil.checkGlError(); - ImmutableList fences = generateSyncFences(/* count= */ 2); + ImmutableList fences = GlUtil.createSyncFences(/* count= */ 2); if (fences.size() == 2) { glReadCompleteFence = fences.get(0); glWriteCompleteFence = fences.get(1); } - } catch (GlException e) { + } catch (GlException | VideoFrameProcessingException e) { if (eglImageTextureWrapper != null) { try { releaseEglImageTexture(eglImageTextureWrapper, hardwareBufferJniWrapper); @@ -151,8 +175,219 @@ public void signalEndOfStream() { @Override public void close() throws VideoFrameProcessingException { - if (defaultShaderProgram != null) { - defaultShaderProgram.release(); + releaseGlResources(); + } + + /** Establishes the output format based on the first frame. */ + private Format establishOutputFormat(Format inputFormat) { + ColorInfo outputColorInfo = resolveOutputColorInfo(checkNotNull(inputFormat.colorInfo)); + // This only sets the tags of the output format, the pixel values don't change. SDR pixels are + // sRGB encoded, which is preferred for RGB content throughout Android, even where it's only + // labeled "RGB". Some encoders don't accept an sRGB tag, so SDR output is tagged with the SDR + // (BT.709) transfer instead. Encoders output limited range for both SDR and HDR. + // TODO(b/545572413): Allow converting outputting to other gamut, for example BT.601. + ColorInfo frameWriterColorInfo = + isWideColorGamut(outputColorInfo) + ? outputColorInfo.buildUpon().setColorRange(C.COLOR_RANGE_LIMITED).build() + : ColorInfo.SDR_BT709_LIMITED; + int rotationDegrees = + calculateOutputRotationDegrees( + inputFormat.buildUpon().setColorInfo(frameWriterColorInfo).build()); + int outputWidth = rotationDegrees == 90 ? inputFormat.height : inputFormat.width; + int outputHeight = rotationDegrees == 90 ? inputFormat.width : inputFormat.height; + return updateFormat( + inputFormat, + outputWidth, + outputHeight, + // Sets the degrees that the player needs to rotate. If we rotated 90 degrees, the player + // needs to rotate -90 degrees, which is equivalent to rotating it 270 degrees. + /* rotationDegrees= */ (360 - rotationDegrees) % 360, + frameWriterColorInfo); + } + + /** Reconfigures the shader programs if the input size or color space changed. */ + private void maybeReconfigureShader(GlTextureFrame inputFrame) + throws VideoFrameProcessingException { + Format inputFormat = inputFrame.format; + // The frames in the GL pipeline should always be in the intended orientation. + checkArgument(inputFormat.rotationDegrees == 0); + ColorInfo inputColorInfo = checkNotNull(inputFormat.colorInfo); + if (inputSize != null + && inputFormat.width == inputSize.getWidth() + && inputFormat.height == inputSize.getHeight() + && Objects.equals(inputColorInfo, this.inputColorInfo)) { + return; + } + + releaseGlResources(); + + int inputTextureWidth = inputFrame.glTextureInfo.width; + int inputTextureHeight = inputFrame.glTextureInfo.height; + ColorInfo outputColorInfo = resolveOutputColorInfo(inputColorInfo); + if (!outputColorInfo.equals(inputColorInfo)) { + // TODO: b/565711655 - Do the color conversion and the output transformations in one pass, + // to avoid rendering to an intermediate texture. + colorConversionShaderProgram = + new ColorConversionShaderProgram(context, inputColorInfo, outputColorInfo); + Size conversionOutputSize = + colorConversionShaderProgram.configure(inputTextureWidth, inputTextureHeight); + colorConversionOutputTexture = + createTextureWithFbo( + conversionOutputSize.getWidth(), + conversionOutputSize.getHeight(), + useHighPrecisionColorComponents(outputColorInfo)); + } + + // Force physical rotation to match the logical rotation of the established output format. + // The output format stores the rotation that player applies. We convert it back to the degrees + // the pipeline needs to rotate. + int rotationDegrees = (360 - checkNotNull(outputFormat).rotationDegrees) % 360; + + // Flip OpenGL coordinate system back and rotate. + GlMatrixTransformation flipAndRotate = + new ScaleAndRotateTransformation.Builder() + .setScale(/* scaleX= */ 1f, /* scaleY= */ -1f) + .setRotationDegrees(rotationDegrees) + .build(); + + ImmutableList.Builder transformationsBuilder = ImmutableList.builder(); + transformationsBuilder + .add(flipAndRotate) + .add( + Presentation.createForWidthAndHeight( + outputFormat.width, outputFormat.height, Presentation.LAYOUT_SCALE_TO_FIT)); + + defaultShaderProgram = + DefaultShaderProgram.create( + context, + /* matrixTransformations= */ transformationsBuilder.build(), + /* rgbMatrices= */ ImmutableList.of(), + /* useHdr= */ ColorInfo.isTransferHdr(outputColorInfo)); + + inputSize = new Size(inputFormat.width, inputFormat.height); + this.inputColorInfo = inputColorInfo; + Size unusedSize = defaultShaderProgram.configure(inputTextureWidth, inputTextureHeight); + } + + private void releaseGlResources() throws VideoFrameProcessingException { + @Nullable DefaultShaderProgram defaultShaderProgram = this.defaultShaderProgram; + @Nullable + ColorConversionShaderProgram colorConversionShaderProgram = this.colorConversionShaderProgram; + @Nullable GlTextureInfo colorConversionOutputTexture = this.colorConversionOutputTexture; + this.defaultShaderProgram = null; + this.colorConversionShaderProgram = null; + this.colorConversionOutputTexture = null; + runAllAndAccumulateExceptions( + () -> { + if (defaultShaderProgram != null) { + defaultShaderProgram.release(); + } + }, + () -> { + if (colorConversionShaderProgram != null) { + colorConversionShaderProgram.release(); + } + }, + () -> { + if (colorConversionOutputTexture != null) { + colorConversionOutputTexture.release(); + } + }); + } + + /** Creates a texture and an FBO for it, deleting the texture if the FBO can't be created. */ + private static GlTextureInfo createTextureWithFbo( + int width, int height, boolean useHighPrecisionColorComponents) + throws VideoFrameProcessingException { + int texId = C.INDEX_UNSET; + try { + texId = GlUtil.createTexture(width, height, useHighPrecisionColorComponents); + return new GlTextureInfo( + texId, GlUtil.createFboForTexture(texId), /* rboId= */ C.INDEX_UNSET, width, height); + } catch (GlException e) { + if (texId != C.INDEX_UNSET) { + try { + GlUtil.deleteTexture(texId); + } catch (GlException deleteException) { + e.addSuppressed(deleteException); + } + } + throw VideoFrameProcessingException.from(e); + } + } + + /** + * Returns the electrical {@link ColorInfo} that frames arriving in {@code inputColorInfo} are + * written in. + * + *

Frames in a linear color space are converted to HLG for BT.2020, and to sRGB for BT.709. + * Frames that are already electrical, for example because a {@link ColorConversion} was applied, + * are written unchanged. + */ + private static ColorInfo resolveOutputColorInfo(ColorInfo inputColorInfo) { + if (inputColorInfo.colorTransfer != C.COLOR_TRANSFER_LINEAR) { + return inputColorInfo; + } + return isWideColorGamut(inputColorInfo) ? BT2020_HLG : BT709_SRGB; + } + + private int calculateOutputRotationDegrees(Format format) { + if (format.width >= format.height) { + // Input is landscape, no rotation needed. + return 0; + } + + if (frameWriter.getInfo().isSupported(format, OUTPUT_USAGE)) { + // Portrait is supported, no rotation needed. + return 0; } + + // If frameWriter doesn't support portrait, try rotating the input by 90 degrees to swap to + // landscape dimensions supported by the encoder. + int rotatedWidth = format.height; + int rotatedHeight = format.width; + Format formatRotate90 = + // Setting rotation degrees to zero because the encoder doesn't rotate the frame and thus + // the value is irrelevant. Here we use the swapped dimension to check encoder capability. + updateFormat( + format, + rotatedWidth, + rotatedHeight, + /* rotationDegrees= */ 0, + checkNotNull(format.colorInfo)); + if (frameWriter.getInfo().isSupported(formatRotate90, OUTPUT_USAGE)) { + return 90; + } + + // Fallback if nothing is supported. + return 0; + } + + private static Format updateFormat( + Format format, int width, int height, int rotationDegrees, ColorInfo colorInfo) { + return format + .buildUpon() + .setWidth(width) + .setHeight(height) + .setRotationDegrees(rotationDegrees) + .setColorInfo(colorInfo) + .build(); + } + + @Override + public boolean isOutputFormatSupported(Format format) { + // TODO(b/545572413): Check for color space support and fallback to SDR if necessary. + if (!frameWriter.getInfo().isSupported(format, OUTPUT_USAGE)) { + return false; + } + try { + if (format.colorInfo != null + && !GlUtil.isColorTransferSupported(format.colorInfo.colorTransfer)) { + return false; + } + } catch (GlException e) { + return false; + } + return true; } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/GaussianBlur.java b/libraries/effect/src/main/java/androidx/media3/effect/GaussianBlur.java index 1b4d24bf935..74d5abde846 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/GaussianBlur.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/GaussianBlur.java @@ -15,7 +15,10 @@ */ package androidx.media3.effect; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; + import androidx.annotation.FloatRange; +import androidx.annotation.RestrictTo; import androidx.media3.common.util.UnstableApi; /** @@ -55,4 +58,12 @@ public GaussianBlur(float sigma) { public ConvolutionFunction1D getConvolution(long presentationTimeUs) { return new GaussianFunction(sigma, numStandardDeviations); } + + @Override + @RestrictTo(LIBRARY_GROUP) + public void populateMetrics(MetricConsumer consumer) { + super.populateMetrics(consumer); + int sampleCount = Math.round(2 * sigma * numStandardDeviations) + 1; + consumer.addMetric(MetricConsumer.METRIC_FILTER_SAMPLE_COUNT, sampleCount); + } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/GlShaderProgramAdapter.java b/libraries/effect/src/main/java/androidx/media3/effect/GlShaderProgramAdapter.java index bd44d45f9fc..53f81b78388 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/GlShaderProgramAdapter.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/GlShaderProgramAdapter.java @@ -22,8 +22,8 @@ import static com.google.common.util.concurrent.MoreExecutors.directExecutor; import androidx.annotation.Nullable; -import androidx.annotation.RequiresApi; import androidx.media3.common.C; +import androidx.media3.common.ColorInfo; import androidx.media3.common.Format; import androidx.media3.common.GlObjectsProvider; import androidx.media3.common.GlTextureInfo; @@ -50,7 +50,6 @@ * *

The underlying {@link GlShaderProgram} must not alter frame presentation times. */ -@RequiresApi(26) @ExperimentalApi // TODO: b/505721737 Remove once FrameProcessor is production ready. /* package */ final class GlShaderProgramAdapter implements GlTextureFrameProcessor, InputListener, OutputListener { @@ -62,6 +61,7 @@ private final Queue pendingOutputFrames; private final Executor glExecutor; private final Consumer errorConsumer; + private final ColorInfo outputColorInfo; private @MonotonicNonNull GlTextureFrameConsumer downstreamConsumer; @Nullable private ImmutableMap lastQueuedMetadata; @@ -71,16 +71,28 @@ private int currentStreamDiscontinuityNumber; @Nullable private Format currentFormat; private boolean inputStreamEnded; - private boolean receivedEosSignal; + private boolean hasSignaledEosToWrappedShader; + private boolean receivedEosSignalFromWrappedShader; + /** + * Creates an instance. + * + * @param glShaderProgram The {@link GlShaderProgram} to wrap. + * @param glObjectsProvider The {@link GlObjectsProvider}. + * @param glExecutor The {@link Executor} to run GL operations on. + * @param outputColorInfo The {@link ColorInfo} to propagate to output frames. + * @param errorConsumer The {@link Consumer} to report errors to. + */ public GlShaderProgramAdapter( GlShaderProgram glShaderProgram, GlObjectsProvider glObjectsProvider, Executor glExecutor, + ColorInfo outputColorInfo, Consumer errorConsumer) { this.glShaderProgram = glShaderProgram; this.glObjectsProvider = glObjectsProvider; this.glExecutor = glExecutor; + this.outputColorInfo = outputColorInfo; this.errorConsumer = errorConsumer; @SuppressWarnings("nullness:assignment") @Initialized @@ -126,14 +138,19 @@ public boolean queue(GlTextureFrame frame, Executor listenerExecutor, Runnable w } if (currentFormat == null) { + // currentFormat is set to null after the wrapped shader has processed the EOS currentFormat = frame.format; - inputStreamEnded = false; + // Resets pending EOS signal from format transition to prevent prematurely signaling EOS to + // downstreamConsumer if the upstream signals EOS. + receivedEosSignalFromWrappedShader = false; + hasSignaledEosToWrappedShader = false; } if (!frame.format.equals(currentFormat)) { pendingWakeupListener = wakeupListener; pendingWakeupExecutor = listenerExecutor; - if (!inputStreamEnded) { - inputStreamEnded = true; + // TODO: b/524240959 - Don't use format change to signal EOS. + if (!hasSignaledEosToWrappedShader) { + hasSignaledEosToWrappedShader = true; glShaderProgram.signalEndOfCurrentInputStream(); } return false; @@ -181,7 +198,7 @@ public void onOutputFrameAvailable(GlTextureInfo outputTexture, long presentatio .setFormat( inputFormat .buildUpon() - // TODO: b/505721737 - set color info for tone mapping. + .setColorInfo(outputColorInfo) .setWidth(outputTexture.width) .setHeight(outputTexture.height) .build()) @@ -194,19 +211,21 @@ public void onOutputFrameAvailable(GlTextureInfo outputTexture, long presentatio @Override public void onCurrentOutputStreamEnded() { - receivedEosSignal = true; + receivedEosSignalFromWrappedShader = true; currentFormat = null; - if (pendingOutputFrames.isEmpty() && downstreamConsumer != null) { - downstreamConsumer.signalEndOfStream(); - } + // Attempts to drain the output queue and sends EOS downstream if there's no frame pending. + tryQueueFramesDownstream(); maybeInvokePendingWakeupListener(); } @Override public void signalEndOfStream() { if (!inputStreamEnded) { - glShaderProgram.signalEndOfCurrentInputStream(); inputStreamEnded = true; + if (!hasSignaledEosToWrappedShader) { + hasSignaledEosToWrappedShader = true; + glShaderProgram.signalEndOfCurrentInputStream(); + } } } @@ -216,7 +235,7 @@ public void close() throws VideoFrameProcessingException { glShaderProgram.release(); } finally { releasePendingFrames(); - receivedEosSignal = false; + receivedEosSignalFromWrappedShader = false; } } @@ -243,7 +262,8 @@ private void flushInternal() { shaderInputCapacity = 0; releasePendingFrames(); glShaderProgram.flush(); - receivedEosSignal = false; + hasSignaledEosToWrappedShader = false; + receivedEosSignalFromWrappedShader = false; } private void releasePendingFrames() { @@ -282,8 +302,11 @@ private void tryQueueFramesDownstream() { return; } } - if (pendingOutputFrames.isEmpty() && receivedEosSignal && downstreamConsumer != null) { - downstreamConsumer.signalEndOfStream(); + if (pendingOutputFrames.isEmpty() && receivedEosSignalFromWrappedShader) { + receivedEosSignalFromWrappedShader = false; + if (inputStreamEnded) { + downstreamConsumer.signalEndOfStream(); + } } } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/GlTextureCopier.java b/libraries/effect/src/main/java/androidx/media3/effect/GlTextureCopier.java index d097bc33b44..234e737bce2 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/GlTextureCopier.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/GlTextureCopier.java @@ -15,12 +15,14 @@ */ package androidx.media3.effect; -import static com.google.common.base.Preconditions.checkArgument; +import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.base.Preconditions.checkState; import android.content.Context; +import android.graphics.Gainmap; import android.opengl.GLES20; import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; import androidx.media3.common.C; import androidx.media3.common.ColorInfo; import androidx.media3.common.VideoFrameProcessingException; @@ -28,7 +30,6 @@ import androidx.media3.common.util.GlUtil; import androidx.media3.common.util.GlUtil.GlException; import androidx.media3.common.util.Log; -import androidx.media3.common.util.Util; import java.io.IOException; /** @@ -48,6 +49,9 @@ private final Context context; @Nullable private GlProgram sdrExternalCopyGlProgram; @Nullable private GlProgram sdrInternalCopyGlProgram; + @Nullable private GlProgram hdrExternalCopyGlProgram; + @Nullable private GlProgram hdrInternalCopyGlProgram; + @Nullable private GlProgram ultraHdrCopyGlProgram; private int fboId = C.INDEX_UNSET; public GlTextureCopier(Context context) { @@ -57,6 +61,10 @@ public GlTextureCopier(Context context) { /** * Copies the content of an input texture to an output texture. * + *

The {@code requestedOutputColorInfo.colorTransfer} is configured during shader + * initialization and must remain invariant across calls for this {@link GlTextureCopier} + * instance. + * * @param inputTexId The ID of the input texture. * @param outputTexId The ID of the output texture. * @param outputWidth The width of the output texture. @@ -78,67 +86,126 @@ public void copyTexture( float[] textureTransformMatrix, boolean isExternalTexture) throws VideoFrameProcessingException { - checkArgument( - !isExternalTexture || inputColorInfo.equals(requestedOutputColorInfo), - Util.formatInvariant( - "Color conversion for external textures is not supported yet. Input: %s, Output: %s", - inputColorInfo, requestedOutputColorInfo)); + // Assuming BT.2020 color space for HDR. + boolean isInputHdr = ColorInfo.isWideColorGamut(inputColorInfo); + boolean isOutputHdr = ColorInfo.isWideColorGamut(requestedOutputColorInfo); + try { GlProgram copyGlProgram; - if (isExternalTexture) { - if (sdrExternalCopyGlProgram == null) { - sdrExternalCopyGlProgram = - new GlProgram( - context, - R.raw.vertex_shader_transformation_es2, - R.raw.fragment_shader_transformation_sdr_external_es2); - setupCommonAttributesAndUniforms(sdrExternalCopyGlProgram); - sdrExternalCopyGlProgram.setIntUniform("uOutputColorTransfer", C.COLOR_TRANSFER_SDR); + if (isInputHdr || isOutputHdr) { + if (isExternalTexture) { + setupExternalHdrGlProgram(requestedOutputColorInfo); + copyGlProgram = checkNotNull(hdrExternalCopyGlProgram); + copyGlProgram.setIntUniform( + "uIsInputColorRangeFull", inputColorInfo.colorRange == C.COLOR_RANGE_FULL ? 1 : 0); + } else { + // Internal HDR texture + setupInternalHdrGlProgram(requestedOutputColorInfo); + copyGlProgram = checkNotNull(hdrInternalCopyGlProgram); } - copyGlProgram = sdrExternalCopyGlProgram; + + // Not setting output related uniforms, they were set during creating the GlProgram. + copyGlProgram.setIntUniform("uInputColorTransfer", inputColorInfo.colorTransfer); } else { - if (sdrInternalCopyGlProgram == null) { - sdrInternalCopyGlProgram = - new GlProgram( - context, - R.raw.vertex_shader_transformation_es2, - R.raw.fragment_shader_transformation_sdr_internal_es2); - setupCommonAttributesAndUniforms(sdrInternalCopyGlProgram); - sdrInternalCopyGlProgram.setIntUniform("uSdrWorkingColorSpace", 0); + // SDR input + if (isExternalTexture) { + setupExternalSdrGlProgram(requestedOutputColorInfo); + copyGlProgram = checkNotNull(sdrExternalCopyGlProgram); + } else { + // Internal SDR texture + setupInternalSdrGlProgram(requestedOutputColorInfo); + copyGlProgram = checkNotNull(sdrInternalCopyGlProgram); } - copyGlProgram = sdrInternalCopyGlProgram; - copyGlProgram.setIntUniform("uInputColorTransfer", inputColorInfo.colorTransfer); - copyGlProgram.setIntUniform("uOutputColorTransfer", requestedOutputColorInfo.colorTransfer); + copyGlProgram.setIntUniform("uInputColorGamut", inputColorInfo.colorSpace); } - copyGlProgram.setFloatsUniform("uTexTransformationMatrix", textureTransformMatrix); + copyGlProgram.setSamplerTexIdUniform("uTexSampler", inputTexId, /* texUnitIndex= */ 0); - if (fboId == C.INDEX_UNSET) { - int[] fboIds = new int[1]; - GLES20.glGenFramebuffers(/* n= */ 1, fboIds, /* offset= */ 0); - GlUtil.checkGlError(); - fboId = fboIds[0]; - } - - GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fboId); - GlUtil.checkGlError(); - GLES20.glFramebufferTexture2D( - GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0, GLES20.GL_TEXTURE_2D, outputTexId, 0); - GlUtil.checkGlError(); + drawGlProgramToOutputTexture(copyGlProgram, outputTexId, outputWidth, outputHeight); + } catch (GlException | IOException e) { + throw VideoFrameProcessingException.from(e); + } + } - GlUtil.focusFramebufferUsingCurrentContext(fboId, outputWidth, outputHeight); + /** + * Copies the content of an input base texture and an Ultra HDR gainmap texture to an output + * texture, applying the gainmap and converting to the target color space. + * + *

The {@code requestedOutputColorInfo.colorTransfer} is configured during shader + * initialization and must remain invariant across calls for this {@link GlTextureCopier} + * instance. + * + * @param inputTexId The ID of the input base texture. + * @param gainmapTexId The ID of the gainmap texture. + * @param gainmap The {@link Gainmap} containing mathematical parameters. + * @param outputTexId The ID of the output texture. + * @param outputWidth The width of the output texture. + * @param outputHeight The height of the output texture. + * @param requestedOutputColorInfo The {@link ColorInfo} of the requested output texture. + * @param textureTransformMatrix The 4x4 matrix to apply to texture coordinates. + * @throws VideoFrameProcessingException If an OpenGL error occurs during the copy. + */ + @RequiresApi(34) + public void copyTextureUltraHdr( + int inputTexId, + int gainmapTexId, + Gainmap gainmap, + int outputTexId, + int outputWidth, + int outputHeight, + ColorInfo requestedOutputColorInfo, + float[] textureTransformMatrix) + throws VideoFrameProcessingException { + try { + GlProgram ultraHdrCopyGlProgram = this.ultraHdrCopyGlProgram; + if (ultraHdrCopyGlProgram == null) { + ultraHdrCopyGlProgram = + new GlProgram( + context, + R.raw.vertex_shader_transformation_es3, + R.raw.color_conversions_es3, + R.raw.fragment_shader_transformation_ultra_hdr_color_conversion_es3); + setupCommonAttributesAndUniforms(ultraHdrCopyGlProgram); + ultraHdrCopyGlProgram.setIntUniform( + "uOutputColorTransfer", requestedOutputColorInfo.colorTransfer); + this.ultraHdrCopyGlProgram = ultraHdrCopyGlProgram; + } + ultraHdrCopyGlProgram.setSamplerTexIdUniform( + "uTexSampler", inputTexId, /* texUnitIndex= */ 0); + ultraHdrCopyGlProgram.setSamplerTexIdUniform( + "uGainmapTexSampler", gainmapTexId, /* texUnitIndex= */ 1); + ultraHdrCopyGlProgram.setFloatsUniform("uTexTransformationMatrix", textureTransformMatrix); + GainmapUtil.setGainmapUniforms(ultraHdrCopyGlProgram, gainmap, C.INDEX_UNSET); - checkState(copyGlProgram != null); - copyGlProgram.use(); - copyGlProgram.setSamplerTexIdUniform("uTexSampler", inputTexId, /* texUnitIndex= */ 0); - copyGlProgram.bindAttributesAndUniforms(); - GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, /* first= */ 0, /* count= */ 4); - GlUtil.checkGlError(); + drawGlProgramToOutputTexture(ultraHdrCopyGlProgram, outputTexId, outputWidth, outputHeight); } catch (GlException | IOException e) { throw VideoFrameProcessingException.from(e); } } + private void drawGlProgramToOutputTexture( + GlProgram glProgram, int outputTexId, int outputWidth, int outputHeight) throws GlException { + if (fboId == C.INDEX_UNSET) { + int[] fboIds = new int[1]; + GLES20.glGenFramebuffers(/* n= */ 1, fboIds, /* offset= */ 0); + GlUtil.checkGlError(); + fboId = fboIds[0]; + } + + GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fboId); + GlUtil.checkGlError(); + GLES20.glFramebufferTexture2D( + GLES20.GL_FRAMEBUFFER, GLES20.GL_COLOR_ATTACHMENT0, GLES20.GL_TEXTURE_2D, outputTexId, 0); + GlUtil.checkGlError(); + + GlUtil.focusFramebufferUsingCurrentContext(fboId, outputWidth, outputHeight); + + glProgram.use(); + glProgram.bindAttributesAndUniforms(); + GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, /* first= */ 0, /* count= */ 4); + GlUtil.checkGlError(); + } + /** Releases resources used by the copier. */ public void release() throws VideoFrameProcessingException { @Nullable Exception firstException = null; @@ -163,6 +230,45 @@ public void release() throws VideoFrameProcessingException { } sdrInternalCopyGlProgram = null; } + if (hdrExternalCopyGlProgram != null) { + try { + hdrExternalCopyGlProgram.delete(); + } catch (GlException e) { + if (firstException == null) { + firstException = e; + } else { + Log.w(TAG, "Failed to delete hdrExternalCopyGlProgram", e); + firstException.addSuppressed(e); + } + } + hdrExternalCopyGlProgram = null; + } + if (hdrInternalCopyGlProgram != null) { + try { + hdrInternalCopyGlProgram.delete(); + } catch (GlException e) { + if (firstException == null) { + firstException = e; + } else { + Log.w(TAG, "Failed to delete hdrInternalCopyGlProgram", e); + firstException.addSuppressed(e); + } + } + hdrInternalCopyGlProgram = null; + } + if (ultraHdrCopyGlProgram != null) { + try { + ultraHdrCopyGlProgram.delete(); + } catch (GlException e) { + if (firstException == null) { + firstException = e; + } else { + Log.w(TAG, "Failed to delete ultraHdrCopyGlProgram", e); + firstException.addSuppressed(e); + } + } + ultraHdrCopyGlProgram = null; + } if (fboId != C.INDEX_UNSET) { try { GlUtil.deleteFbo(fboId); @@ -181,12 +287,79 @@ public void release() throws VideoFrameProcessingException { } } + private void setupExternalHdrGlProgram(ColorInfo outputColorInfo) + throws GlException, IOException { + checkState(isOpenGl3Supported() && GlUtil.isYuvTargetExtensionSupported()); + if (hdrExternalCopyGlProgram == null) { + hdrExternalCopyGlProgram = + new GlProgram( + context, + R.raw.vertex_shader_transformation_es3, + R.raw.color_conversions_es3, + R.raw.fragment_shader_transformation_external_yuv_color_conversion_es3); + setupCommonAttributesAndUniforms(hdrExternalCopyGlProgram); + + hdrExternalCopyGlProgram.setIntUniform("uOutputColorTransfer", outputColorInfo.colorTransfer); + hdrExternalCopyGlProgram.setIntUniform("uOutputColorGamut", outputColorInfo.colorSpace); + } + } + + private void setupInternalHdrGlProgram(ColorInfo outputColorInfo) + throws GlException, IOException { + checkState(isOpenGl3Supported()); + if (hdrInternalCopyGlProgram == null) { + hdrInternalCopyGlProgram = + new GlProgram( + context, + R.raw.vertex_shader_transformation_es3, + R.raw.color_conversions_es3, + R.raw.fragment_shader_transformation_hdr_internal_color_conversion_es3); + setupCommonAttributesAndUniforms(hdrInternalCopyGlProgram); + hdrInternalCopyGlProgram.setIntUniform("uOutputColorTransfer", outputColorInfo.colorTransfer); + hdrInternalCopyGlProgram.setIntUniform("uOutputColorGamut", outputColorInfo.colorSpace); + } + } + + private void setupInternalSdrGlProgram(ColorInfo outputColorInfo) + throws GlException, IOException { + if (sdrInternalCopyGlProgram == null) { + sdrInternalCopyGlProgram = + new GlProgram( + context, + R.raw.vertex_shader_transformation_es2, + R.raw.color_conversions_es2, + R.raw.fragment_shader_transformation_sdr_internal_color_conversion_es2); + setupCommonAttributesAndUniforms(sdrInternalCopyGlProgram); + sdrInternalCopyGlProgram.setIntUniform("uOutputColorTransfer", outputColorInfo.colorTransfer); + sdrInternalCopyGlProgram.setIntUniform("uOutputColorGamut", outputColorInfo.colorSpace); + } + } + + private void setupExternalSdrGlProgram(ColorInfo outputColorInfo) + throws GlException, IOException { + if (sdrExternalCopyGlProgram == null) { + sdrExternalCopyGlProgram = + new GlProgram( + context, + R.raw.vertex_shader_transformation_es2, + R.raw.color_conversions_es2, + R.raw.fragment_shader_transformation_sdr_external_color_conversion_es2); + setupCommonAttributesAndUniforms(sdrExternalCopyGlProgram); + sdrExternalCopyGlProgram.setIntUniform("uOutputColorTransfer", outputColorInfo.colorTransfer); + sdrExternalCopyGlProgram.setIntUniform("uOutputColorGamut", outputColorInfo.colorSpace); + } + } + private static void setupCommonAttributesAndUniforms(GlProgram glProgram) { glProgram.setBufferAttribute( "aFramePosition", GlUtil.getNormalizedCoordinateBounds(), GlUtil.HOMOGENEOUS_COORDINATE_VECTOR_SIZE); glProgram.setFloatsUniform("uTransformationMatrix", GlUtil.create4x4IdentityMatrix()); - glProgram.setFloatsUniform("uRgbMatrix", GlUtil.create4x4IdentityMatrix()); + glProgram.setFloatsUniformIfPresent("uRgbMatrix", GlUtil.create4x4IdentityMatrix()); + } + + private static boolean isOpenGl3Supported() throws GlException { + return GlUtil.getContextMajorVersion() >= 3; } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/GlTextureFrameAggregator.java b/libraries/effect/src/main/java/androidx/media3/effect/GlTextureFrameAggregator.java index e9808e26145..04059280ab1 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/GlTextureFrameAggregator.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/GlTextureFrameAggregator.java @@ -23,10 +23,9 @@ import android.util.SparseArray; import androidx.annotation.Nullable; -import androidx.annotation.RequiresApi; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.Consumer; -import androidx.media3.effect.FrameProcessorUtils.ThrowingRunnable; +import androidx.media3.common.util.ThrowingRunnable; import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableSet; import com.google.common.util.concurrent.ListeningExecutorService; @@ -38,24 +37,24 @@ * Encapsulates N-to-1 frame aggregation logic. * *

It receives input via {@link GlTextureFrameConsumer} acquired via {@link #getInputConsumer}, - * and outputs aggregated frames to a {@link DefaultGlTextureFrameCompositingProcessor}. + * and outputs aggregated frames to a {@link GlTextureFrameCompositor}. * *

All methods must be called on a GL thread. */ // TODO: b/524240959 - Consider sharing logic with FrameAggregator. -@RequiresApi(26) /* package */ final class GlTextureFrameAggregator implements AutoCloseable { - private final DefaultGlTextureFrameCompositingProcessor compositingProcessor; + private final GlTextureFrameCompositor compositingProcessor; private final ListeningExecutorService glExecutorService; private final Consumer errorConsumer; private final SparseArray sequenceConsumers; @Nullable private ImmutableSet pendingSequenceIndices; private boolean isClosed; + // This is reset when a new batch of frames is queued. private boolean allInputStreamsEnded; public GlTextureFrameAggregator( - DefaultGlTextureFrameCompositingProcessor compositingProcessor, + GlTextureFrameCompositor compositingProcessor, ListeningExecutorService glExecutorService, Consumer errorConsumer) { this.compositingProcessor = compositingProcessor; @@ -288,12 +287,12 @@ private ImmutableList retrieveAndClearPendingFrames() { } private void releaseFrames(List frames) throws VideoFrameProcessingException { - ImmutableList.Builder releaseActions = new ImmutableList.Builder<>(); + ImmutableList.Builder> releaseActions = new ImmutableList.Builder<>(); for (int i = 0; i < frames.size(); i++) { GlTextureFrame frame = frames.get(i); releaseActions.add(() -> frame.release(/* releaseFence= */ null)); } - runAllAndAccumulateExceptions(releaseActions.build().toArray(new ThrowingRunnable[0])); + runAllAndAccumulateExceptions(releaseActions.build().toArray(new ThrowingRunnable[0])); } private final class InputFrameConsumer implements GlTextureFrameConsumer { @@ -301,6 +300,7 @@ private final class InputFrameConsumer implements GlTextureFrameConsumer { @Nullable private Runnable pendingWakeupListener; @Nullable private Executor pendingWakeupExecutor; @Nullable private GlTextureFrame pendingFrame; + // This is reset when a new batch of frames is queued. private boolean isEnded; private boolean isActive; @@ -324,6 +324,8 @@ public boolean queue(GlTextureFrame frame, Executor listenerExecutor, Runnable w return false; } + isEnded = false; + allInputStreamsEnded = false; pendingFrame = frame; tryQueueToCompositingProcessor(); // Returning true to signal the input frame is queued. If queueing to the compositing diff --git a/libraries/effect/src/main/java/androidx/media3/effect/GlTextureFrameCompositor.java b/libraries/effect/src/main/java/androidx/media3/effect/GlTextureFrameCompositor.java new file mode 100644 index 00000000000..f81324147a5 --- /dev/null +++ b/libraries/effect/src/main/java/androidx/media3/effect/GlTextureFrameCompositor.java @@ -0,0 +1,102 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect; + +import androidx.media3.common.ColorInfo; +import androidx.media3.common.GlObjectsProvider; +import androidx.media3.common.GlTextureInfo; +import androidx.media3.common.VideoFrameProcessingException; +import androidx.media3.common.util.Consumer; +import androidx.media3.common.util.GlUtil.GlException; +import java.util.List; +import java.util.concurrent.Executor; + +/** + * Composites a list of {@link GlTextureFrame GlTextureFrames}, and outputs to a downstream {@link + * GlTextureFrameConsumer}. + * + *

Methods in this class must be called on a GL thread. + */ +/* package */ interface GlTextureFrameCompositor extends AutoCloseable { + + /** A factory for {@link GlTextureFrameCompositor} instances. */ + interface Factory { + /** + * Creates a {@link GlTextureFrameCompositor}. + * + * @param glObjectsProvider The {@link GlObjectsProvider} for OpenGL operations. + * @param workingColorSpace The {@link ColorInfo} representing the working color space that all + * inputs and the output of the compositor should be in. + * @param errorConsumer A consumer for {@link VideoFrameProcessingException}. + * @param glExecutor The {@link Executor} running OpenGL tasks on the GL thread. + * @param downstreamConsumer The downstream {@link GlTextureFrameConsumer}. + * @return A new {@link GlTextureFrameCompositor}. + */ + GlTextureFrameCompositor create( + GlObjectsProvider glObjectsProvider, + ColorInfo workingColorSpace, + Consumer errorConsumer, + Executor glExecutor, + GlTextureFrameConsumer downstreamConsumer); + } + + /** Draws multiple input frames onto one output texture using a GL program. */ + interface CompositorGlProgram { + + /** A factory for {@link CompositorGlProgram} instances. */ + interface Factory { + /** Creates a new {@link CompositorGlProgram}. */ + CompositorGlProgram create(); + } + + /** + * Draws the input frames onto the output texture. + * + * @param framesToComposite The {@linkplain GlCompositionFrame input frames} to composite. + * @param outputTexture The {@link GlTextureInfo} to draw onto. + * @throws VideoFrameProcessingException If an error occurs during frame processing. + */ + void drawFrame(List framesToComposite, GlTextureInfo outputTexture) + throws GlException, VideoFrameProcessingException; + + /** Releases all associated resources. */ + void release() throws VideoFrameProcessingException; + } + + /** + * Attempts to queue frames for compositing. + * + *

If the compositor is at capacity, it returns {@code false} and the {@code wakeupListener} + * will be invoked on the {@code listenerExecutor} when capacity becomes available. + * + *

If {@code frames} contains only a single frame, the compositor is bypassed and the frame is + * passed directly downstream. + * + * @param frames The input frames to composite. + * @param listenerExecutor The executor to run the {@code wakeupListener} on. + * @param wakeupListener The callback to run when capacity is freed. + * @return {@code true} if queued successfully, {@code false} otherwise. + */ + // TODO: b/517424999 - Unify the listeners to follow the same pattern as FrameProcessor. + boolean queue(List frames, Executor listenerExecutor, Runnable wakeupListener) + throws VideoFrameProcessingException; + + /** Notifies the current stream has ended. */ + void signalEndOfStream(); + + @Override + void close() throws VideoFrameProcessingException; +} diff --git a/libraries/effect/src/main/java/androidx/media3/effect/GlTextureFrameConsumer.java b/libraries/effect/src/main/java/androidx/media3/effect/GlTextureFrameConsumer.java index 5b49d314677..3804370c89f 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/GlTextureFrameConsumer.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/GlTextureFrameConsumer.java @@ -15,6 +15,7 @@ */ package androidx.media3.effect; +import androidx.media3.common.Format; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.ExperimentalApi; import androidx.media3.common.video.FrameProcessor; @@ -55,6 +56,16 @@ boolean queue(GlTextureFrame frame, Executor listenerExecutor, Runnable wakeupLi /** Notifies the current stream has ended. */ void signalEndOfStream(); + /** + * Returns whether this consumer supports the provided {@link Format}. + * + *

Returns {@code true} by default. + */ + // TODO: b/535069683 - Remove default implementation once all consumers are updated. + default boolean isOutputFormatSupported(Format format) { + return true; + } + @Override void close() throws VideoFrameProcessingException; } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/GlTextureFrameProcessorChain.java b/libraries/effect/src/main/java/androidx/media3/effect/GlTextureFrameProcessorChain.java index ef49887f156..3f3565a3f11 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/GlTextureFrameProcessorChain.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/GlTextureFrameProcessorChain.java @@ -15,6 +15,7 @@ */ package androidx.media3.effect; +import static androidx.media3.common.ColorInfo.isWideColorGamut; import static androidx.media3.effect.FrameProcessorUtils.runAllAndAccumulateExceptions; import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkNotNull; @@ -22,13 +23,13 @@ import static com.google.common.collect.Iterables.getLast; import android.content.Context; -import androidx.annotation.RequiresApi; +import androidx.media3.common.ColorInfo; import androidx.media3.common.Effect; import androidx.media3.common.GlObjectsProvider; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.Consumer; +import androidx.media3.common.util.ThrowingRunnable; import androidx.media3.common.util.Util; -import androidx.media3.effect.FrameProcessorUtils.ThrowingRunnable; import com.google.common.collect.ImmutableList; import com.google.common.util.concurrent.ListeningExecutorService; import java.util.ArrayList; @@ -39,7 +40,6 @@ * A {@link GlTextureFrameConsumer} that creates and executes a chain of {@link * GlShaderProgramAdapter}s. */ -@RequiresApi(26) /* package */ final class GlTextureFrameProcessorChain implements GlTextureFrameConsumer { private final Context context; private final GlObjectsProvider glObjectsProvider; @@ -49,22 +49,37 @@ private final List currentEffects; private final GlTextureFrameConsumer downstreamFrameConsumer; private final String effectKey; + private final ColorInfo workingColorSpace; private GlTextureFrameConsumer firstGlTextureFrameConsumer; + /** + * Creates an instance. + * + * @param context The {@link Context}. + * @param glObjectsProvider The {@link GlObjectsProvider}. + * @param glExecutorService The {@link ListeningExecutorService} to execute OpenGL commands on. + * @param errorConsumer The {@link Consumer} to report errors to. + * @param downstreamFrameConsumer The {@link GlTextureFrameConsumer} to output frames to. + * @param effectKey The {@linkplain GlTextureFrame#getMetadata() frame metadata} key holding the + * {@linkplain Effect effects} to apply. + * @param workingColorSpace The {@link ColorInfo} that frames are processed in. + */ public GlTextureFrameProcessorChain( Context context, GlObjectsProvider glObjectsProvider, ListeningExecutorService glExecutorService, Consumer errorConsumer, GlTextureFrameConsumer downstreamFrameConsumer, - String effectKey) { + String effectKey, + ColorInfo workingColorSpace) { this.context = context; this.glObjectsProvider = glObjectsProvider; this.glExecutorService = glExecutorService; this.errorConsumer = errorConsumer; this.downstreamFrameConsumer = downstreamFrameConsumer; this.effectKey = effectKey; + this.workingColorSpace = workingColorSpace; this.firstGlTextureFrameConsumer = downstreamFrameConsumer; effectProcessorChain = new ArrayList<>(); currentEffects = new ArrayList<>(); @@ -108,29 +123,84 @@ private void configure(List effects) throws VideoFrameProcessingExceptio closeAllProcessors(); currentEffects.clear(); List newProcessorChain = new ArrayList<>(); - for (int i = 0; i < effects.size(); i++) { - Effect effect = effects.get(i); - checkArgument( - effect instanceof GlEffect, - Util.formatInvariant("%s supports only GlEffects", getClass().getSimpleName())); - // TODO: b/505721737 - Support HDR. - try { + ImmutableList.Builder matrixTransformationBuilder = + new ImmutableList.Builder<>(); + ImmutableList.Builder colorTransformationBuilder = new ImmutableList.Builder<>(); + // TODO: b/545584738 - GlEffect currently overloads useHdr to mean both color gamut (BT.2020 vs + // BT.709) and texture precision (GL_RGBA16F vs GL_RGBA8). For BT709_LINEAR, the + // gamut is BT.709 (useHdr = false), causing BaseGlShaderProgram to allocate 8-bit textures + // (GL_RGBA8) and band in linear light. Pass ColorInfo to effects instead of a boolean, or + // decouple gamut from texture precision. + ColorInfo currentColorSpace = workingColorSpace; + try { + for (int i = 0; i < effects.size(); i++) { + Effect effect = effects.get(i); + checkArgument( + effect instanceof GlEffect, + Util.formatInvariant("%s supports only GlEffects", getClass().getSimpleName())); + GlEffect glEffect = (GlEffect) effect; + // Merge consecutive matrix and color effects. + if (glEffect instanceof GlMatrixTransformation) { + matrixTransformationBuilder.add((GlMatrixTransformation) glEffect); + continue; + } + if (glEffect instanceof RgbMatrix) { + colorTransformationBuilder.add((RgbMatrix) glEffect); + continue; + } + + ImmutableList matrixTransformations = + matrixTransformationBuilder.build(); + ImmutableList colorTransformations = colorTransformationBuilder.build(); + + if (!matrixTransformations.isEmpty() || !colorTransformations.isEmpty()) { + newProcessorChain.add( + createMergedProcessor( + matrixTransformations, colorTransformations, currentColorSpace)); + matrixTransformationBuilder = new ImmutableList.Builder<>(); + colorTransformationBuilder = new ImmutableList.Builder<>(); + } + + // ColorConversion converts between color spaces, so it needs the full input color space + // rather than useHdr, which cannot express an electrical transfer. + GlShaderProgram shaderProgram; + ColorInfo outputColorSpace; + if (glEffect instanceof ColorConversion) { + ColorConversion colorConversion = (ColorConversion) glEffect; + shaderProgram = colorConversion.toGlShaderProgram(context, currentColorSpace); + outputColorSpace = colorConversion.getOutputColorInfo(); + } else { + shaderProgram = glEffect.toGlShaderProgram(context, isWideColorGamut(currentColorSpace)); + outputColorSpace = currentColorSpace; + } newProcessorChain.add( new GlShaderProgramAdapter( - ((GlEffect) effect).toGlShaderProgram(context, /* useHdr= */ false), + shaderProgram, glObjectsProvider, glExecutorService, + outputColorSpace, errorConsumer)); - } catch (VideoFrameProcessingException | RuntimeException e) { - for (int j = 0; j < newProcessorChain.size(); j++) { - try { - newProcessorChain.get(j).close(); - } catch (VideoFrameProcessingException | RuntimeException closeException) { - // Ignore exceptions when closing during an error recovery. - } + currentColorSpace = outputColorSpace; + } + + ImmutableList remainingMatrixTransformations = + matrixTransformationBuilder.build(); + ImmutableList remainingColorTransformations = colorTransformationBuilder.build(); + + if (!remainingMatrixTransformations.isEmpty() || !remainingColorTransformations.isEmpty()) { + newProcessorChain.add( + createMergedProcessor( + remainingMatrixTransformations, remainingColorTransformations, currentColorSpace)); + } + } catch (VideoFrameProcessingException | RuntimeException e) { + for (int j = 0; j < newProcessorChain.size(); j++) { + try { + newProcessorChain.get(j).close(); + } catch (VideoFrameProcessingException | RuntimeException closeException) { + // Ignore exceptions when closing during an error recovery. } - throw e; } + throw e; } for (int i = 0; i < newProcessorChain.size() - 1; i++) { newProcessorChain.get(i).setOutput(newProcessorChain.get(i + 1)); @@ -155,8 +225,22 @@ private ImmutableList extractEffects(GlTextureFrame frame) { return (ImmutableList) checkNotNull(frame.getMetadata().get(effectKey)); } + private GlTextureFrameProcessor createMergedProcessor( + List matrixTransformations, + List colorTransformations, + ColorInfo colorInfo) + throws VideoFrameProcessingException { + return new GlShaderProgramAdapter( + DefaultShaderProgram.create( + context, matrixTransformations, colorTransformations, isWideColorGamut(colorInfo)), + glObjectsProvider, + glExecutorService, + colorInfo, + errorConsumer); + } + private void closeAllProcessors() throws VideoFrameProcessingException { - ThrowingRunnable[] closeActions = new ThrowingRunnable[effectProcessorChain.size()]; + ThrowingRunnable[] closeActions = new ThrowingRunnable[effectProcessorChain.size()]; for (int i = 0; i < effectProcessorChain.size(); i++) { closeActions[i] = effectProcessorChain.get(i)::close; } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/HardwareBufferFrame.java b/libraries/effect/src/main/java/androidx/media3/effect/HardwareBufferFrame.java deleted file mode 100644 index 2d1c9be52e7..00000000000 --- a/libraries/effect/src/main/java/androidx/media3/effect/HardwareBufferFrame.java +++ /dev/null @@ -1,391 +0,0 @@ -/* - * Copyright 2025 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package androidx.media3.effect; - -import static com.google.common.base.Preconditions.checkArgument; -import static com.google.common.util.concurrent.MoreExecutors.directExecutor; - -import android.hardware.HardwareBuffer; -import androidx.annotation.GuardedBy; -import androidx.annotation.Nullable; -import androidx.media3.common.C; -import androidx.media3.common.Format; -import androidx.media3.common.util.Consumer; -import androidx.media3.common.util.ExperimentalApi; -import androidx.media3.common.video.SyncFenceWrapper; -import com.google.errorprone.annotations.CanIgnoreReturnValue; -import java.util.ArrayList; -import java.util.List; -import java.util.concurrent.Executor; - -/** - * A frame backed by a {@link HardwareBuffer}. - * - *

Frames of this type may be mappable to memory accessible by various hardware systems, such as - * GPU, media codecs, NPU, or other auxiliary processing units. - * - *

On API levels before 26, where {@link HardwareBuffer} is not defined, {@link #internalFrame} - * will be set for use within this package. {@link #internalFrame} is not intended for use by third - * party apps. - */ -@ExperimentalApi // TODO: b/449956776 - Remove once FrameConsumer API is finalized. -public final class HardwareBufferFrame { - private static final String TAG = "HardwareBufferFrame"; - - /** A callback to be invoked when the {@link HardwareBufferFrame} is released. */ - public interface ReleaseCallback { - /** - * Releases the underlying resources of the {@link HardwareBufferFrame}. - * - * @param releaseFence A {@link SyncFenceWrapper} that must signal before the underlying - * resources can be fully released, or {@code null} if the resources can be released - * immediately. - */ - void release(@Nullable SyncFenceWrapper releaseFence); - } - - /** - * A marker interface for storing arbitrary metadata associated with a {@link - * HardwareBufferFrame}. - */ - public interface Metadata {} - - public static final HardwareBufferFrame END_OF_STREAM_FRAME = - new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, directExecutor(), /* releaseCallback= */ (fence) -> {}) - .setInternalFrame(new Object()) - .build(); - - /** - * Returns the {@link HardwareBuffer} which holds the frame data, or {@code null} if hardware - * buffers are not supported on the current API level. - */ - @Nullable public final HardwareBuffer hardwareBuffer; - - /** - * The presentation time of the frame relative to the start of the media item it belongs to, in - * microseconds. - */ - public final long presentationTimeUs; - - /** - * The presentation time of the frame relative to the start of the entire sequence, in - * microseconds. This time is monotonically increasing across all media items in the sequence. - */ - public final long sequencePresentationTimeUs; - - /** The release time of the frame, in nanoseconds. */ - public final long releaseTimeNs; - - /** The format of the frame. */ - public final Format format; - - private final Metadata metadata; - - /** - * An acquire {@link SyncFenceWrapper} for the {@linkplain #hardwareBuffer HardwareBuffer}. - * - *

Callers should ensure that the acquire fence has signaled before accessing {@linkplain - * #hardwareBuffer HardwareBuffer}. - * - *

If the acquire fence is {@code null}, it's safe to access {@linkplain #hardwareBuffer - * HardwareBuffer}. - */ - @Nullable public final SyncFenceWrapper acquireFence; - - /** An optional internal frame type that is used when {@link #hardwareBuffer} is not supported. */ - @Nullable public final Object internalFrame; - - /** The shared state object managing reference counts and release orchestration. */ - private final SharedState sharedState; - - @GuardedBy("this") - private boolean isReleased; - - /** A builder for {@link HardwareBufferFrame} instances. */ - public static final class Builder { - @Nullable private final HardwareBuffer hardwareBuffer; - private final Executor releaseExecutor; - private final ReleaseCallback releaseCallback; - - private long presentationTimeUs; - private long sequencePresentationTimeUs; - private Format format; - private long releaseTimeNs; - private Metadata metadata; - @Nullable private SyncFenceWrapper acquireFence; - @Nullable private Object internalFrame; - @Nullable private SharedState sharedState; - - /** - * Creates a new {@link Builder}. - * - * @param hardwareBuffer The {@link HardwareBuffer} supplier, or {@code null} if a hardware - * buffer cannot be constructed on the current API level. - * @param releaseExecutor The {@link Executor} on which the {@code releaseCallback} is called. - * @param releaseCallback The {@link Consumer} to call to release the texture. - */ - public Builder( - @Nullable HardwareBuffer hardwareBuffer, - Executor releaseExecutor, - ReleaseCallback releaseCallback) { - this.hardwareBuffer = hardwareBuffer; - this.releaseExecutor = releaseExecutor; - this.releaseCallback = releaseCallback; - this.metadata = new Metadata() {}; - presentationTimeUs = C.TIME_UNSET; - sequencePresentationTimeUs = C.TIME_UNSET; - format = new Format.Builder().build(); - releaseTimeNs = C.TIME_UNSET; - } - - private Builder(HardwareBufferFrame frame) { - this(frame.hardwareBuffer, frame.sharedState.executor, frame.sharedState.callback); - this.metadata = frame.metadata; - this.presentationTimeUs = frame.presentationTimeUs; - this.sequencePresentationTimeUs = frame.sequencePresentationTimeUs; - this.format = frame.format; - this.releaseTimeNs = frame.releaseTimeNs; - this.acquireFence = frame.acquireFence; - this.internalFrame = frame.internalFrame; - this.sharedState = frame.sharedState; - } - - /** Sets the {@link HardwareBufferFrame#presentationTimeUs}. */ - @CanIgnoreReturnValue - public Builder setPresentationTimeUs(long presentationTimeUs) { - this.presentationTimeUs = presentationTimeUs; - return this; - } - - /** Sets the {@link HardwareBufferFrame#sequencePresentationTimeUs}. */ - @CanIgnoreReturnValue - public Builder setSequencePresentationTimeUs(long sequencePresentationTimeUs) { - this.sequencePresentationTimeUs = sequencePresentationTimeUs; - return this; - } - - /** Sets the {@link HardwareBufferFrame#format}. */ - @CanIgnoreReturnValue - public Builder setFormat(Format format) { - this.format = format; - return this; - } - - /** Sets the {@link HardwareBufferFrame#releaseTimeNs}. */ - @CanIgnoreReturnValue - public Builder setReleaseTimeNs(long releaseTimeNs) { - this.releaseTimeNs = releaseTimeNs; - return this; - } - - /** Sets the {@link HardwareBufferFrame#metadata}. */ - @CanIgnoreReturnValue - public Builder setMetadata(Metadata metadata) { - this.metadata = metadata; - return this; - } - - /** - * Sets the {@link HardwareBufferFrame#acquireFence}. - * - *

The default value is {@code null}. - */ - @CanIgnoreReturnValue - public Builder setAcquireFence(@Nullable SyncFenceWrapper acquireFence) { - this.acquireFence = acquireFence; - return this; - } - - /** - * Sets the {@link HardwareBufferFrame#internalFrame}. - * - *

The default value is {@code null}. - * - * @param internalFrame The internal frame. - */ - @CanIgnoreReturnValue - public Builder setInternalFrame(@Nullable Object internalFrame) { - this.internalFrame = internalFrame; - return this; - } - - /** Builds the {@link HardwareBufferFrame} instance. */ - public HardwareBufferFrame build() { - return new HardwareBufferFrame(this); - } - } - - private HardwareBufferFrame(Builder builder) { - checkArgument(builder.hardwareBuffer != null || builder.internalFrame != null); - this.hardwareBuffer = builder.hardwareBuffer; - this.presentationTimeUs = builder.presentationTimeUs; - this.sequencePresentationTimeUs = builder.sequencePresentationTimeUs; - this.releaseTimeNs = builder.releaseTimeNs; - this.format = builder.format; - this.metadata = builder.metadata; - this.acquireFence = builder.acquireFence; - this.internalFrame = builder.internalFrame; - if (builder.sharedState != null) { - this.sharedState = builder.sharedState; - } else { - this.sharedState = - new SharedState(builder.releaseCallback, builder.releaseExecutor, builder.acquireFence); - } - isReleased = false; - } - - /** Private constructor for creating retained handles sharing the same state. */ - private HardwareBufferFrame(HardwareBufferFrame original, SharedState sharedState) { - this.hardwareBuffer = original.hardwareBuffer; - this.presentationTimeUs = original.presentationTimeUs; - this.sequencePresentationTimeUs = original.sequencePresentationTimeUs; - this.releaseTimeNs = original.releaseTimeNs; - this.format = original.format; - this.metadata = original.metadata; - this.acquireFence = original.acquireFence; - this.internalFrame = original.internalFrame; - this.sharedState = sharedState; - isReleased = false; - } - - /** - * Returns a {@link Builder} initialized with the values of this instance. This method does not - * increase the reference count. - * - *

The returned builder creates handles that share the same underlying reference count as this - * instance. To create a new reference that increments the reference count, call {@link #retain()} - * instead. - */ - public Builder buildUpon() { - return new Builder(this); - } - - public Metadata getMetadata() { - return metadata; - } - - /** - * Releases the frame and its underlying resources. - * - *

This implementation is idempotent if called after the frame has already been released. It - * will strictly release the underlying resources only when the count transitions from 1 to 0. - * - * @param releaseFence A {@link SyncFenceWrapper} that must signal before the underlying resources - * can be fully released, or {@code null} if the resources can be released immediately. - */ - public void release(@Nullable SyncFenceWrapper releaseFence) { - synchronized (this) { - if (isReleased) { - closeFenceSilently(releaseFence); - return; - } - isReleased = true; - } - - sharedState.release(releaseFence); - } - - /** - * Creates a new reference to this frame. - * - *

For every call to this method there must be an extra {@link #release} call. The underlying - * buffer will only be released when the reference count drops to zero. - * - * @throws IllegalStateException if called after the frame has been released. - */ - public synchronized HardwareBufferFrame retain() { - if (isReleased) { - throw new IllegalStateException("Cannot retain a frame that has already been released."); - } - sharedState.retain(); - return new HardwareBufferFrame(this, sharedState); - } - - private static void closeFenceSilently(@Nullable SyncFenceWrapper fence) { - if (fence == null) { - return; - } - fence.close(); - } - - /** - * Internal Shared State. - * - *

This class manages the synchronization and reference counting between multiple handles to + - * the same underlying frame. - */ - private static final class SharedState { - - private final ReleaseCallback callback; - private final Executor executor; - @Nullable private final SyncFenceWrapper acquireFence; - - @GuardedBy("this") - private int refCount = 1; - - @GuardedBy("this") - private final List releaseFences = new ArrayList<>(); - - private SharedState( - ReleaseCallback callback, Executor executor, @Nullable SyncFenceWrapper acquireFence) { - this.callback = callback; - this.executor = executor; - this.acquireFence = acquireFence; - } - - private synchronized void retain() { - if (refCount <= 0) { - throw new IllegalStateException("retain() called on a released HardwareBufferFrame."); - } - refCount++; - } - - private void release(@Nullable SyncFenceWrapper releaseFence) { - List fencesToWaitOn; - - synchronized (this) { - if (releaseFence != null) { - releaseFences.add(releaseFence); - } - refCount--; - if (refCount != 0) { - // The buffer is still retained by other references, do nothing further. - return; - } - - // Collect all fences for the final release signal. - // The acquire fence should have signaled before the release fences, but we combine it - // here anyway to be safe. - fencesToWaitOn = new ArrayList<>(releaseFences); - if (acquireFence != null) { - fencesToWaitOn.add(acquireFence); - } - releaseFences.clear(); - } - - // TODO: b/479415385 - Consider forwarding a List to callback.release(). - executor.execute( - () -> { - for (SyncFenceWrapper fence : fencesToWaitOn) { - fence.awaitForever(); - fence.close(); - } - callback.release(/* releaseFence= */ null); - }); - } - } -} diff --git a/libraries/effect/src/main/java/androidx/media3/effect/HardwareBufferFrameQueue.java b/libraries/effect/src/main/java/androidx/media3/effect/HardwareBufferFrameQueue.java deleted file mode 100644 index 2f6c655c3e5..00000000000 --- a/libraries/effect/src/main/java/androidx/media3/effect/HardwareBufferFrameQueue.java +++ /dev/null @@ -1,251 +0,0 @@ -/* - * Copyright 2026 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package androidx.media3.effect; - -import android.hardware.HardwareBuffer; -import androidx.annotation.Nullable; -import androidx.annotation.RequiresApi; -import androidx.media3.common.ColorInfo; -import androidx.media3.common.util.ExperimentalApi; -import com.google.errorprone.annotations.CanIgnoreReturnValue; -import java.util.Objects; - -/** - * A provider for {@link HardwareBufferFrame} instances that manages allocating the underlying - * {@link HardwareBuffer} instances, and forwards frames downstream for processing. - */ -@RequiresApi(26) -@ExperimentalApi // TODO: b/449956776 - Remove once FrameConsumer API is finalized. -public interface HardwareBufferFrameQueue { - - /** Defines the configuration parameters required for a {@link HardwareBuffer}. */ - final class FrameFormat { - - /** A builder for {@link FrameFormat} instances. */ - public static final class Builder { - private int width; - private int height; - private int rotationDegrees; - private int pixelFormat; - private long usageFlags; - private ColorInfo colorInfo; - - /** - * Creates a new builder with default values. - * - *

Width and height default to 0, {@code rotationDegrees} defaults to 0, {@code - * pixelFormat} defaults to {@link HardwareBuffer#RGBA_8888}, {@code usageFlags} defaults to - * {@link HardwareBuffer#USAGE_GPU_SAMPLED_IMAGE}, and {@link ColorInfo} defaults to {@link - * ColorInfo#SDR_BT709_LIMITED}. - */ - public Builder() { - pixelFormat = HardwareBuffer.RGBA_8888; - usageFlags = HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE; - colorInfo = ColorInfo.SDR_BT709_LIMITED; - } - - /** - * Sets the width of the buffer in pixels. - * - * @param width The width. - * @return This builder. - */ - @CanIgnoreReturnValue - public Builder setWidth(int width) { - this.width = width; - return this; - } - - /** - * Sets the height of the buffer in pixels. - * - * @param height The height. - * @return This builder. - */ - @CanIgnoreReturnValue - public Builder setHeight(int height) { - this.height = height; - return this; - } - - /** - * Sets the rotation degrees of the buffer. - * - * @param rotationDegrees The rotation degrees. - * @return This builder. - */ - @CanIgnoreReturnValue - public Builder setRotationDegrees(int rotationDegrees) { - this.rotationDegrees = rotationDegrees; - return this; - } - - /** - * Sets the {@linkplain HardwareBuffer#getFormat() format} of the {@link HardwareBuffer}. - * - * @param pixelFormat The pixel format. - * @return This builder. - */ - @CanIgnoreReturnValue - public Builder setPixelFormat(int pixelFormat) { - this.pixelFormat = pixelFormat; - return this; - } - - /** - * Sets the {@linkplain HardwareBuffer#getUsage() usage flags} of the {@link HardwareBuffer}. - * - * @param usageFlags The usage flags. - * @return This builder. - */ - @CanIgnoreReturnValue - public Builder setUsageFlags(long usageFlags) { - this.usageFlags = usageFlags; - return this; - } - - /** - * Sets the {@link ColorInfo}. - * - * @param colorInfo The {@link ColorInfo}. - * @return This builder. - */ - @CanIgnoreReturnValue - public Builder setColorInfo(ColorInfo colorInfo) { - this.colorInfo = colorInfo; - return this; - } - - /** Builds the {@link FrameFormat} instance. */ - public FrameFormat build() { - return new FrameFormat(width, height, rotationDegrees, pixelFormat, usageFlags, colorInfo); - } - } - - public final int width; - public final int height; - public final int rotationDegrees; - public final int pixelFormat; - public final long usageFlags; - public final ColorInfo colorInfo; - - private FrameFormat( - int width, - int height, - int rotationDegrees, - int pixelFormat, - long usageFlags, - ColorInfo colorInfo) { - this.width = width; - this.height = height; - this.rotationDegrees = rotationDegrees; - this.pixelFormat = pixelFormat; - this.usageFlags = usageFlags; - this.colorInfo = colorInfo; - } - - /** Returns a {@link Builder} initialized with the values of this instance. */ - public Builder buildUpon() { - return new Builder() - .setWidth(width) - .setHeight(height) - .setRotationDegrees(rotationDegrees) - .setColorInfo(colorInfo) - .setPixelFormat(pixelFormat) - .setUsageFlags(usageFlags); - } - - @Override - public boolean equals(@Nullable Object o) { - if (this == o) { - return true; - } - if (o == null || getClass() != o.getClass()) { - return false; - } - FrameFormat that = (FrameFormat) o; - return width == that.width - && height == that.height - && rotationDegrees == that.rotationDegrees - && pixelFormat == that.pixelFormat - && usageFlags == that.usageFlags - && Objects.equals(colorInfo, that.colorInfo); - } - - @Override - public int hashCode() { - return Objects.hash(width, height, rotationDegrees, pixelFormat, usageFlags, colorInfo); - } - - @Override - public String toString() { - return "FrameFormat{" - + "width=" - + width - + ", height=" - + height - + ", rotationDegrees=" - + rotationDegrees - + ", pixelFormat=" - + pixelFormat - + ", usageFlags=" - + usageFlags - + ", colorInfo=" - + colorInfo - + '}'; - } - } - - /** - * Attempts to dequeue a {@link HardwareBufferFrame} matching the specified {@code format}. - * - *

If the queue has reached its capacity and no buffers are available for reuse, this method - * returns {@code null}. In this case, the {@code wakeupListener} will be invoked when a buffer is - * eventually returned to the pool. - * - *

If this method is called multiple times without returning a frame, only the most recent - * {@code wakeupListener} is guaranteed to be invoked. - * - *

If a dequeued {@link HardwareBufferFrame} cannot be filled and {@linkplain #queue queued} - * back for any reason, it must be {@linkplain HardwareBufferFrame#release released}. - * - * @param format The required format for the dequeued buffer. - * @param wakeupListener A callback to notify the caller when a buffer becomes available. - * @return A {@link HardwareBufferFrame}, or {@code null} if the queue is currently full. - */ - @Nullable - HardwareBufferFrame dequeue(FrameFormat format, Runnable wakeupListener); - - /** - * Queues a {@link HardwareBufferFrame} for consumption by the downstream component. - * - *

The implementation is responsible for ensuring the buffer is correctly reused or released - * once the downstream component has finished processing it. - * - * @param frame The frame to be processed. - */ - void queue(HardwareBufferFrame frame); - - /** - * Signals that no more frames will be queued. - * - *

This propagates an end-of-stream signal to the downstream consumer. - */ - void signalEndOfStream(); - - /** Releases all resources associated with this instance. */ - void release(); -} diff --git a/libraries/effect/src/main/java/androidx/media3/effect/HardwareBufferJni.java b/libraries/effect/src/main/java/androidx/media3/effect/HardwareBufferJni.java new file mode 100644 index 00000000000..d6336c82390 --- /dev/null +++ b/libraries/effect/src/main/java/androidx/media3/effect/HardwareBufferJni.java @@ -0,0 +1,88 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect; + +import static com.google.common.base.Preconditions.checkState; + +import android.graphics.Bitmap; +import android.hardware.HardwareBuffer; +import androidx.annotation.RequiresApi; +import androidx.media3.common.util.ExperimentalApi; +import androidx.media3.common.util.LibraryLoader; + +/** JNI methods for HardwareBuffer interaction. */ +@RequiresApi(26) +@ExperimentalApi // TODO: b/449956776 - Remove once FrameConsumer API is finalized. +public final class HardwareBufferJni implements HardwareBufferJniWrapper { + + private static final LibraryLoader LOADER = + new LibraryLoader("media3EffectJNI") { + @Override + protected void loadLibrary(String name) { + System.loadLibrary(name); + } + }; + + public static final HardwareBufferJni INSTANCE = new HardwareBufferJni(); + + private HardwareBufferJni() { + checkState(LOADER.isAvailable()); + } + + /** Creates an EGLImage from a {@link HardwareBuffer}. */ + @Override + public native long nativeCreateEglImageFromHardwareBuffer( + long displayHandle, HardwareBuffer hardwareBuffer); + + /** + * Binds an EGLImage to the specified texture target. Returns whether the binding is successful. + */ + @Override + public native boolean nativeBindEGLImage(int target, long eglImageHandle); + + /** Destroys an EGLImage. Returns whether the deletion is successful. */ + @Override + public native boolean nativeDestroyEGLImage(long displayHandle, long imageHandle); + + /** + * Copies a {@link Bitmap} to a {@link HardwareBuffer}. Returns whether the copy is successful. + * + *

The destination {@link HardwareBuffer} must have {@link + * HardwareBuffer#USAGE_CPU_WRITE_OFTEN} usage. + * + *

The source {@link Bitmap.Config} must match the destination {@link + * HardwareBuffer#getFormat}, and be either {@link Bitmap.Config#ARGB_8888} and {@link + * HardwareBuffer#RGBA_8888} or {@link Bitmap.Config#RGBA_1010102} and {@link + * HardwareBuffer#RGBA_1010102}. + */ + @Override + public native boolean nativeCopyBitmapToHardwareBuffer(Bitmap bitmap, HardwareBuffer hb); + + /** + * Copies the contents of a source {@link HardwareBuffer} to a destination {@link HardwareBuffer}. + * Returns whether the copy is successful. + * + *

The source {@link HardwareBuffer} must have {@link HardwareBuffer#USAGE_CPU_READ_OFTEN} + * usage, and the destination {@link HardwareBuffer} must have {@link + * HardwareBuffer#USAGE_CPU_WRITE_OFTEN} usage. + * + *

The formats of the source and destination buffers must match, and be either {@link + * HardwareBuffer#RGBA_8888} or {@link HardwareBuffer#RGBA_1010102}. + */ + @Override + public native boolean nativeCopyHardwareBufferToHardwareBuffer( + HardwareBuffer srcHb, HardwareBuffer dstHb); +} diff --git a/libraries/effect/src/main/java/androidx/media3/effect/HardwareBufferToGlTextureConverter.java b/libraries/effect/src/main/java/androidx/media3/effect/HardwareBufferToGlTextureConverter.java index 62da11b6324..ce80ad1d2e1 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/HardwareBufferToGlTextureConverter.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/HardwareBufferToGlTextureConverter.java @@ -15,13 +15,18 @@ */ package androidx.media3.effect; +import static android.os.Build.VERSION.SDK_INT; import static androidx.media3.effect.FrameProcessorUtils.createAndBindEglImage; -import static androidx.media3.effect.FrameProcessorUtils.generateSyncFences; import static androidx.media3.effect.FrameProcessorUtils.releaseEglImageTexture; +import static androidx.media3.effect.FrameProcessorUtils.runAllAndAccumulateExceptions; +import static androidx.media3.effect.FrameProcessorUtils.useHighPrecisionColorComponents; +import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.collect.Iterables.getFirst; import android.content.Context; +import android.graphics.Bitmap; +import android.graphics.Gainmap; import android.hardware.HardwareBuffer; import android.opengl.EGLDisplay; import android.opengl.GLES11Ext; @@ -32,25 +37,32 @@ import androidx.media3.common.ColorInfo; import androidx.media3.common.Format; import androidx.media3.common.GlTextureInfo; +import androidx.media3.common.MimeTypes; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.Consumer; import androidx.media3.common.util.ExperimentalApi; import androidx.media3.common.util.GlUtil; import androidx.media3.common.util.GlUtil.GlException; import androidx.media3.common.util.Log; +import androidx.media3.common.util.ThrowingRunnable; +import androidx.media3.common.video.DefaultHardwareBufferFrame; +import androidx.media3.common.video.Frame; import androidx.media3.common.video.FrameProcessor; import androidx.media3.common.video.HardwareBufferFrame; import androidx.media3.common.video.SyncFenceWrapper; import androidx.media3.effect.FrameProcessorUtils.EglImageTextureWrapper; +import java.util.ArrayList; import java.util.HashMap; +import java.util.List; import java.util.Map; +import java.util.Objects; import java.util.concurrent.Executor; +import java.util.concurrent.atomic.AtomicReference; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; @ExperimentalApi // TODO: b/505721737 Remove once FrameProcessor is production ready. @RequiresApi(26) -/* package */ final class HardwareBufferToGlTextureConverter - implements DefaultGlFrameProcessor.HardwareBufferConverter { +/* package */ final class HardwareBufferToGlTextureConverter implements FrameToGlTextureConverter { private static final String TAG = "HB2GLConverter"; @@ -59,6 +71,7 @@ private final ColorInfo outputColorInfo; private final Consumer errorConsumer; private final Map activeEglImageTextureWrappers; + private final Map activeGainmapTextures; @Nullable private GlTextureCopier glTextureCopier; private @MonotonicNonNull EGLDisplay eglDisplay; @@ -74,6 +87,7 @@ this.outputColorInfo = outputColorInfo; this.errorConsumer = errorConsumer; this.activeEglImageTextureWrappers = new HashMap<>(); + this.activeGainmapTextures = new HashMap<>(); this.textureTransformMatrix = new float[16]; } @@ -85,20 +99,21 @@ */ @Override public GlTextureFrame convert( - HardwareBufferFrame hardwareBufferFrame, - Executor glExecutor, - Executor listenerExecutor, - FrameProcessor.Listener listener) + Frame frame, Executor glExecutor, Executor listenerExecutor, FrameProcessor.Listener listener) throws VideoFrameProcessingException { + checkArgument( + frame instanceof HardwareBufferFrame, "Expected HardwareBufferFrame, got: %s", frame); + HardwareBufferFrame hardwareBufferFrame = (HardwareBufferFrame) frame; HardwareBuffer hardwareBuffer = checkNotNull(hardwareBufferFrame.getHardwareBuffer()); - boolean isExternalTexture = hardwareBuffer.getFormat() != HardwareBuffer.RGBA_8888; + boolean isExternalTexture = !isRgbHardwareBuffer(hardwareBuffer.getFormat()); int outputTexId; @Nullable EglImageTextureWrapper eglImageTextureWrapper = null; int internalTexId = C.INDEX_UNSET; Format inputFormat = hardwareBufferFrame.getFormat(); - int outputWidth = inputFormat.width; - int outputHeight = inputFormat.height; + boolean isRotated = inputFormat.rotationDegrees == 90 || inputFormat.rotationDegrees == 270; + int outputWidth = isRotated ? inputFormat.height : inputFormat.width; + int outputHeight = isRotated ? inputFormat.width : inputFormat.height; try { if (eglDisplay == null) { eglDisplay = GlUtil.getDefaultEglDisplay(); @@ -123,9 +138,7 @@ public GlTextureFrame convert( } internalTexId = GlUtil.createTexture( - outputWidth, - outputHeight, - /* useHighPrecisionColorComponents= */ ColorInfo.isTransferHdr(outputColorInfo)); + outputWidth, outputHeight, useHighPrecisionColorComponents(outputColorInfo)); MatrixUtils.populateTransformationMatrix( textureTransformMatrix, @@ -135,20 +148,54 @@ public GlTextureFrame convert( /* formatHeight= */ outputHeight, inputFormat.rotationDegrees); - copier.copyTexture( - texId, - /* outputTexId= */ internalTexId, - outputWidth, - outputHeight, - inputColorInfo, - outputColorInfo, - textureTransformMatrix, - isExternalTexture); + // On API 34+, if the input is an Ultra HDR image with a gainmap and the pipeline output is + // a wide color gamut, apply the gainmap to reconstruct the HDR image. On API <= 33 or when + // the pipeline output is SDR, falls back to copying the base SDR image without gainmap + // application. + boolean isUltraHdr = + SDK_INT >= 34 + && Objects.equals(inputFormat.sampleMimeType, MimeTypes.IMAGE_JPEG_R) + && ColorInfo.isWideColorGamut(outputColorInfo); + + if (isUltraHdr) { + Bitmap bitmap = (Bitmap) checkNotNull(getInternalFrame(hardwareBufferFrame)); + Gainmap gainmap = checkNotNull(bitmap.getGainmap()); + int gainmapTexId = GlUtil.createTexture(checkNotNull(gainmap.getGainmapContents())); + activeGainmapTextures.put(hardwareBufferFrame, gainmapTexId); + copier.copyTextureUltraHdr( + texId, + gainmapTexId, + gainmap, + /* outputTexId= */ internalTexId, + outputWidth, + outputHeight, + outputColorInfo, + textureTransformMatrix); + } else { + copier.copyTexture( + texId, + /* outputTexId= */ internalTexId, + outputWidth, + outputHeight, + inputColorInfo, + outputColorInfo, + textureTransformMatrix, + isExternalTexture); + } outputTexId = internalTexId; internalTexId = C.INDEX_UNSET; eglImageTextureWrapper = eglImageTextureWrapper.withOutputTexId(outputTexId); } catch (GlException | VideoFrameProcessingException e) { + Integer activeGainmapTexId = activeGainmapTextures.remove(hardwareBufferFrame); + if (activeGainmapTexId != null) { + try { + GlUtil.deleteTexture(activeGainmapTexId); + } catch (GlException exception) { + Log.w(TAG, "Failed to delete gainmap texture during error recovery", exception); + e.addSuppressed(exception); + } + } if (eglImageTextureWrapper != null) { try { releaseEglImageTexture(eglImageTextureWrapper, hardwareBufferJniWrapper); @@ -178,70 +225,107 @@ public GlTextureFrame convert( outputHeight), /* releaseTextureExecutor= */ glExecutor, /* releaseTextureCallback= */ info -> { - try { - EglImageTextureWrapper activeWrapper = - activeEglImageTextureWrappers.remove(hardwareBufferFrame); - if (activeWrapper != null) { - releaseEglImageTexture(activeWrapper, hardwareBufferJniWrapper); - } - // The GlTextureFrame is released by the first effect in the chain after it has - // finished reading from it. Generate a sync fence here to signal when the input - // frame is no longer needed. - @Nullable SyncFenceWrapper glReadFence = getFirst(generateSyncFences(1), null); - if (glReadFence == null) { - GLES20.glFinish(); - } - listenerExecutor.execute( - () -> listener.onFrameProcessed(hardwareBufferFrame, glReadFence)); - } catch (GlException e) { - errorConsumer.accept(VideoFrameProcessingException.from(e)); + Integer activeGainmapTexId = activeGainmapTextures.remove(hardwareBufferFrame); + EglImageTextureWrapper activeWrapper = + activeEglImageTextureWrappers.remove(hardwareBufferFrame); + if (activeGainmapTexId == null && activeWrapper == null) { + return; } + AtomicReference glReadFence = new AtomicReference<>(); + + runAllAndAccumulateExceptions( + errorConsumer, + () -> { + if (activeGainmapTexId != null) { + GlUtil.deleteTexture(activeGainmapTexId); + } + }, + () -> { + if (activeWrapper != null) { + releaseEglImageTexture(activeWrapper, hardwareBufferJniWrapper); + } + }, + () -> { + @Nullable + SyncFenceWrapper syncFence = getFirst(GlUtil.createSyncFences(1), null); + if (syncFence != null) { + glReadFence.set(syncFence); + } else { + GLES20.glFinish(); + } + }); + + listenerExecutor.execute( + () -> listener.onFrameProcessed(hardwareBufferFrame, glReadFence.get())); }) .setPresentationTimeUs(hardwareBufferFrame.getContentTimeUs()) - .setFormat(hardwareBufferFrame.getFormat()) + .setFormat( + hardwareBufferFrame + .getFormat() + .buildUpon() + .setWidth(outputWidth) + .setHeight(outputHeight) + .setColorInfo(outputColorInfo) + // Reset rotation to 0 because we rotated the frame physically with OpenGL. The + // pipeline should always receive frames in their intended orientation. + .setRotationDegrees(0) + .setColorInfo(outputColorInfo) + .build()) .setMetadata(hardwareBufferFrame.getMetadata()) .build(); } @Override - public void releaseGlResources(HardwareBufferFrame hardwareBufferFrame) - throws VideoFrameProcessingException { + public void releaseGlResources(Frame frame) throws VideoFrameProcessingException { + checkArgument( + frame instanceof HardwareBufferFrame, "Expected HardwareBufferFrame, got: %s", frame); + HardwareBufferFrame hardwareBufferFrame = (HardwareBufferFrame) frame; + Integer activeGainmapTexId = activeGainmapTextures.remove(hardwareBufferFrame); EglImageTextureWrapper wrapper = activeEglImageTextureWrappers.remove(hardwareBufferFrame); - if (wrapper == null) { - return; - } - try { - releaseEglImageTexture(wrapper, hardwareBufferJniWrapper); - } catch (GlException e) { - throw VideoFrameProcessingException.from(e); - } + + runAllAndAccumulateExceptions( + () -> { + if (activeGainmapTexId != null) { + GlUtil.deleteTexture(activeGainmapTexId); + } + }, + () -> { + if (wrapper != null) { + releaseEglImageTexture(wrapper, hardwareBufferJniWrapper); + } + }); } @Override public void close() throws VideoFrameProcessingException { - @Nullable Exception firstException = null; + List> actions = new ArrayList<>(); if (glTextureCopier != null) { - try { - glTextureCopier.release(); - } catch (VideoFrameProcessingException e) { - firstException = e; - } + GlTextureCopier copier = glTextureCopier; glTextureCopier = null; + actions.add(copier::release); + } + for (int gainmapTexId : activeGainmapTextures.values()) { + actions.add(() -> GlUtil.deleteTexture(gainmapTexId)); } + activeGainmapTextures.clear(); for (EglImageTextureWrapper wrapper : activeEglImageTextureWrappers.values()) { - try { - releaseEglImageTexture(wrapper, hardwareBufferJniWrapper); - } catch (GlException e) { - if (firstException == null) { - firstException = e; - } else { - Log.w(TAG, "Failed to release EGLImage during close", e); - } - } + actions.add(() -> releaseEglImageTexture(wrapper, hardwareBufferJniWrapper)); } activeEglImageTextureWrappers.clear(); - if (firstException != null) { - throw VideoFrameProcessingException.from(firstException); + runAllAndAccumulateExceptions(actions.toArray(new ThrowingRunnable[0])); + } + + @Nullable + private static Object getInternalFrame(HardwareBufferFrame hardwareBufferFrame) { + if (hardwareBufferFrame instanceof DefaultHardwareBufferFrame) { + return ((DefaultHardwareBufferFrame) hardwareBufferFrame).getInternalImage(); } + return null; + } + + private static boolean isRgbHardwareBuffer(int format) { + return format == HardwareBuffer.RGBA_8888 + || format == HardwareBuffer.RGBA_FP16 + || format == HardwareBuffer.RGBA_1010102; } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/HslAdjustment.java b/libraries/effect/src/main/java/androidx/media3/effect/HslAdjustment.java index da824ec60d6..112987b60a1 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/HslAdjustment.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/HslAdjustment.java @@ -16,17 +16,20 @@ package androidx.media3.effect; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; import static com.google.common.base.Preconditions.checkArgument; import android.content.Context; import androidx.annotation.FloatRange; +import androidx.annotation.RestrictTo; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.UnstableApi; import com.google.errorprone.annotations.CanIgnoreReturnValue; /** Adjusts the HSL (Hue, Saturation, and Lightness) of a frame. */ @UnstableApi -public final class HslAdjustment implements GlEffect { +public final class HslAdjustment implements GlEffect, MetricsProvider { /** A builder for {@code HslAdjustment} instances. */ public static final class Builder { @@ -125,4 +128,10 @@ public BaseGlShaderProgram toGlShaderProgram(Context context, boolean useHdr) public boolean isNoOp(int inputWidth, int inputHeight) { return hueAdjustmentDegrees == 0f && saturationAdjustment == 0f && lightnessAdjustment == 0f; } + + @Override + @RestrictTo(LIBRARY_GROUP) + public void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_EFFECT_COLOR); + } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/ImagePlanesToGlTextureConverter.java b/libraries/effect/src/main/java/androidx/media3/effect/ImagePlanesToGlTextureConverter.java new file mode 100644 index 00000000000..27bb5007168 --- /dev/null +++ b/libraries/effect/src/main/java/androidx/media3/effect/ImagePlanesToGlTextureConverter.java @@ -0,0 +1,494 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect; + +import static androidx.media3.common.util.GlUtil.HOMOGENEOUS_COORDINATE_VECTOR_SIZE; +import static androidx.media3.common.util.GlUtil.checkGlError; +import static androidx.media3.effect.FrameProcessorUtils.runAllAndAccumulateExceptions; +import static androidx.media3.effect.FrameProcessorUtils.useHighPrecisionColorComponents; +import static com.google.common.base.Preconditions.checkArgument; +import static com.google.common.base.Preconditions.checkNotNull; + +import android.content.Context; +import android.graphics.PixelFormat; +import android.opengl.GLES20; +import androidx.annotation.Nullable; +import androidx.media3.common.C; +import androidx.media3.common.ColorInfo; +import androidx.media3.common.Format; +import androidx.media3.common.GlTextureInfo; +import androidx.media3.common.VideoFrameProcessingException; +import androidx.media3.common.util.Consumer; +import androidx.media3.common.util.ExperimentalApi; +import androidx.media3.common.util.GlProgram; +import androidx.media3.common.util.GlUtil; +import androidx.media3.common.util.GlUtil.GlException; +import androidx.media3.common.util.ThrowingRunnable; +import androidx.media3.common.video.Frame; +import androidx.media3.common.video.FrameProcessor; +import androidx.media3.common.video.ImagePlanesFrame; +import androidx.media3.common.video.ImagePlanesFrame.Plane; +import com.google.common.collect.ImmutableList; +import java.io.IOException; +import java.nio.ByteBuffer; +import java.util.ArrayDeque; +import java.util.ArrayList; +import java.util.Deque; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.concurrent.Executor; + +/** Converts an {@link ImagePlanesFrame} to a {@link GlTextureFrame}. */ +@ExperimentalApi // TODO: b/505721737 Remove once FrameProcessor is production ready. +/* package */ final class ImagePlanesToGlTextureConverter implements FrameToGlTextureConverter { + + private static final int RGBA_BYTES_PER_PIXEL = 4; + + private final Context context; + private final ColorInfo outputColorInfo; + private final Consumer errorConsumer; + private final Map activeOutputTextures; + private final Deque availableOutputTextures; + private final float[] textureTransformMatrix; + + @Nullable private GlProgram glProgram; + @Nullable private PlaneTexture planeTexture; + private int fboId; + private boolean isClosed; + + /* package */ ImagePlanesToGlTextureConverter( + Context context, + ColorInfo outputColorInfo, + Consumer errorConsumer) { + this.context = context.getApplicationContext(); + this.outputColorInfo = outputColorInfo; + this.errorConsumer = errorConsumer; + this.activeOutputTextures = new HashMap<>(); + this.availableOutputTextures = new ArrayDeque<>(); + this.textureTransformMatrix = new float[16]; + this.fboId = C.INDEX_UNSET; + } + + @Override + @Nullable + public GlTextureFrame convert( + Frame frame, Executor glExecutor, Executor listenerExecutor, FrameProcessor.Listener listener) + throws VideoFrameProcessingException { + checkArgument(frame instanceof ImagePlanesFrame, "Expected ImagePlanesFrame, got: %s", frame); + ImagePlanesFrame imagePlanesFrame = (ImagePlanesFrame) frame; + if (isClosed) { + return null; + } + + validatePlanes(imagePlanesFrame); + + Format inputFormat = imagePlanesFrame.getFormat(); + boolean isRotated = inputFormat.rotationDegrees == 90 || inputFormat.rotationDegrees == 270; + int outputWidth = isRotated ? inputFormat.height : inputFormat.width; + int outputHeight = isRotated ? inputFormat.width : inputFormat.height; + + @Nullable GlTextureInfo outputTexture = null; + try { + GlProgram program = getOrCreateGlProgram(); + uploadTexture( + imagePlanesFrame.getPlanes().get(0), + inputFormat.width, + inputFormat.height, + GLES20.GL_RGBA, + RGBA_BYTES_PER_PIXEL); + outputTexture = setupOutputTextureAndFbo(outputWidth, outputHeight); + activeOutputTextures.put(imagePlanesFrame, outputTexture); + bindProgramAndUniforms(program, inputFormat, outputWidth, outputHeight); + + GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, /* first= */ 0, /* count= */ 4); + checkGlError(); + } catch (GlException | IOException e) { + activeOutputTextures.remove(imagePlanesFrame); + if (outputTexture != null) { + try { + outputTexture.release(); + } catch (GlException ex) { + e.addSuppressed(ex); + } + } + throw VideoFrameProcessingException.from(e); + } + + return buildGlTextureFrame( + imagePlanesFrame, outputTexture, glExecutor, listenerExecutor, listener); + } + + @Override + public void releaseGlResources(Frame frame) throws VideoFrameProcessingException { + checkArgument(frame instanceof ImagePlanesFrame, "Expected ImagePlanesFrame, got: %s", frame); + @Nullable GlTextureInfo outputTexture = activeOutputTextures.remove(frame); + if (outputTexture != null) { + if (isClosed) { + try { + outputTexture.release(); + } catch (GlException e) { + throw VideoFrameProcessingException.from(e); + } + } else { + availableOutputTextures.addLast(outputTexture); + } + } + } + + @Override + public void close() throws VideoFrameProcessingException { + if (isClosed) { + return; + } + List> cleanupActions = new ArrayList<>(); + if (glProgram != null) { + GlProgram programToDelete = glProgram; + glProgram = null; + cleanupActions.add(programToDelete::delete); + } + if (planeTexture != null) { + int texId = planeTexture.texId; + planeTexture = null; + cleanupActions.add(() -> GlUtil.deleteTexture(texId)); + } + for (GlTextureInfo outputTexture : activeOutputTextures.values()) { + cleanupActions.add(outputTexture::release); + } + activeOutputTextures.clear(); + for (GlTextureInfo outputTexture : availableOutputTextures) { + cleanupActions.add(outputTexture::release); + } + availableOutputTextures.clear(); + if (fboId != C.INDEX_UNSET) { + int fboToDelete = fboId; + fboId = C.INDEX_UNSET; + cleanupActions.add(() -> GlUtil.deleteFbo(fboToDelete)); + } + try { + runAllAndAccumulateExceptions(cleanupActions.toArray(new ThrowingRunnable[0])); + } finally { + isClosed = true; + } + } + + // TODO: b/531653682 - Add support for planar (I420) and semi-planar (NV12 and NV21) formats. + private static void validatePlanes(ImagePlanesFrame frame) { + Format inputFormat = frame.getFormat(); + ImmutableList planes = frame.getPlanes(); + checkArgument( + inputFormat.width > 0 && inputFormat.height > 0, + "Unsupported frame dimensions: %sx%s", + inputFormat.width, + inputFormat.height); + if (inputFormat.colorInfo != null) { + int colorSpace = inputFormat.colorInfo.colorSpace; + checkArgument( + colorSpace == Format.NO_VALUE + || colorSpace == C.COLOR_SPACE_BT601 + || colorSpace == C.COLOR_SPACE_BT709, + "Unsupported color space: %s", + colorSpace); + } + switch (inputFormat.pixelFormat) { + case PixelFormat.RGBA_8888: + checkArgument( + planes.size() == 1 && planes.get(0).getPixelStride() == RGBA_BYTES_PER_PIXEL, + "Expected 1 plane with pixelStride of %s for RGBA_8888", + RGBA_BYTES_PER_PIXEL); + break; + default: + throw new UnsupportedOperationException( + "Unsupported pixel format: " + inputFormat.pixelFormat); + } + validatePlaneDimensions( + planes.get(0), inputFormat.width, inputFormat.height, RGBA_BYTES_PER_PIXEL); + } + + private static void validatePlaneDimensions( + Plane plane, int width, int height, int bytesPerPixel) { + ByteBuffer buffer = plane.getBuffer(); + int rowStride = plane.getRowStride(); + int rowBytes = width * bytesPerPixel; + checkArgument( + rowStride >= rowBytes, + "rowStride (%s) must be at least rowBytes (%s)", + rowStride, + rowBytes); + checkArgument( + rowStride % bytesPerPixel == 0, + "rowStride (%s) must be a multiple of bytesPerPixel (%s)", + rowStride, + bytesPerPixel); + // Producers such as ImageReader end a plane's buffer at its last visible pixel, so the final + // row does not need to be padded to a full rowStride. + int minimumBufferSize = rowStride * (height - 1) + rowBytes; + checkArgument( + buffer.remaining() >= minimumBufferSize, + "Plane buffer holds %s bytes, but rowStride (%s) and height (%s) require at least %s", + buffer.remaining(), + rowStride, + height, + minimumBufferSize); + } + + private GlProgram getOrCreateGlProgram() throws GlException, IOException { + if (glProgram == null) { + glProgram = + new GlProgram( + context, + R.raw.vertex_shader_transformation_es2, + R.raw.color_conversions_es2, + R.raw.fragment_shader_transformation_sdr_internal_color_conversion_es2); + glProgram.setBufferAttribute( + "aFramePosition", + GlUtil.getNormalizedCoordinateBounds(), + HOMOGENEOUS_COORDINATE_VECTOR_SIZE); + glProgram.setFloatsUniform("uTransformationMatrix", GlUtil.create4x4IdentityMatrix()); + glProgram.setFloatsUniform("uRgbMatrix", GlUtil.create4x4IdentityMatrix()); + int outputColorSpace = + outputColorInfo.colorSpace != Format.NO_VALUE + ? outputColorInfo.colorSpace + : C.COLOR_SPACE_BT709; + glProgram.setIntUniform("uOutputColorGamut", outputColorSpace); + glProgram.setIntUniform("uOutputColorTransfer", outputColorInfo.colorTransfer); + } + return glProgram; + } + + private void uploadTexture(Plane plane, int width, int height, int glFormat, int bytesPerPixel) + throws GlException { + // OpenGL ES 2.0 lacks GL_UNPACK_ROW_LENGTH, so upload the full rowStride width and crop the + // row padding via uTexTransformationMatrix. + int rowStride = plane.getRowStride(); + int textureWidth = rowStride / bytesPerPixel; + // Bind before both glTexImage2D and glTexSubImage2D, which operate on the currently bound + // GL_TEXTURE_2D texture (GlUtil.generateTexture() returns an unbound texture ID). + int texId = planeTexture != null ? planeTexture.texId : GlUtil.generateTexture(); + GlUtil.bindTexture(GLES20.GL_TEXTURE_2D, texId, GLES20.GL_LINEAR); + + boolean needsTextureAllocation = + planeTexture == null + || planeTexture.width != textureWidth + || planeTexture.height != height + || planeTexture.glFormat != glFormat; + if (needsTextureAllocation) { + // Allocate GPU texture storage on the first frame, or reallocate if plane dimensions or + // format change (glTexImage2D automatically releases and replaces any previous texture + // memory for texId). The pixel contents are always updated in-place below, to avoid + // per-frame texture memory reallocation overhead. + GLES20.glTexImage2D( + GLES20.GL_TEXTURE_2D, + /* level= */ 0, + glFormat, + textureWidth, + height, + /* border= */ 0, + glFormat, + GLES20.GL_UNSIGNED_BYTE, + /* pixels= */ null); + checkGlError(); + planeTexture = new PlaneTexture(texId, textureWidth, height, glFormat); + } + + // The final row can be shorter than rowStride (see validatePlaneDimensions), so upload all + // other rows at their full stride, and then only the visible part of the final row. + ByteBuffer buffer = plane.getBuffer(); + int finalRowIndex = height - 1; + if (finalRowIndex > 0) { + GLES20.glTexSubImage2D( + GLES20.GL_TEXTURE_2D, + /* level= */ 0, + /* xoffset= */ 0, + /* yoffset= */ 0, + textureWidth, + /* height= */ finalRowIndex, + glFormat, + GLES20.GL_UNSIGNED_BYTE, + buffer); + checkGlError(); + } + + // glTexSubImage2D reads starting from buffer.position() without advancing it, so advance the + // position to the start of the final row and restore it afterwards. + int initialPosition = buffer.position(); + try { + buffer.position(initialPosition + rowStride * finalRowIndex); + GLES20.glTexSubImage2D( + GLES20.GL_TEXTURE_2D, + /* level= */ 0, + /* xoffset= */ 0, + /* yoffset= */ finalRowIndex, + width, + /* height= */ 1, + glFormat, + GLES20.GL_UNSIGNED_BYTE, + buffer); + checkGlError(); + } finally { + buffer.position(initialPosition); + } + } + + private GlTextureInfo setupOutputTextureAndFbo(int outputWidth, int outputHeight) + throws GlException { + @Nullable GlTextureInfo outputTexture = null; + while (!availableOutputTextures.isEmpty()) { + GlTextureInfo candidateOutputTexture = availableOutputTextures.removeFirst(); + if (candidateOutputTexture.width == outputWidth + && candidateOutputTexture.height == outputHeight) { + outputTexture = candidateOutputTexture; + break; + } + // Release cached textures whose dimensions no longer match the current frame. + candidateOutputTexture.release(); + } + if (outputTexture == null) { + int outputTexId = + GlUtil.createTexture( + outputWidth, outputHeight, useHighPrecisionColorComponents(outputColorInfo)); + outputTexture = + new GlTextureInfo( + outputTexId, + /* fboId= */ C.INDEX_UNSET, + /* rboId= */ C.INDEX_UNSET, + outputWidth, + outputHeight); + } + + try { + if (fboId == C.INDEX_UNSET) { + int[] fboIds = new int[1]; + GLES20.glGenFramebuffers(/* n= */ 1, fboIds, /* offset= */ 0); + checkGlError(); + fboId = fboIds[0]; + } + + GLES20.glBindFramebuffer(GLES20.GL_FRAMEBUFFER, fboId); + checkGlError(); + GLES20.glFramebufferTexture2D( + GLES20.GL_FRAMEBUFFER, + GLES20.GL_COLOR_ATTACHMENT0, + GLES20.GL_TEXTURE_2D, + outputTexture.texId, + /* level= */ 0); + checkGlError(); + + GlUtil.focusFramebufferUsingCurrentContext(fboId, outputWidth, outputHeight); + } catch (GlException e) { + try { + outputTexture.release(); + } catch (GlException ex) { + e.addSuppressed(ex); + } + throw e; + } + return outputTexture; + } + + private void bindProgramAndUniforms( + GlProgram program, Format inputFormat, int outputWidth, int outputHeight) throws GlException { + PlaneTexture planeTexture = checkNotNull(this.planeTexture); + program.use(); + + MatrixUtils.populateTransformationMatrix( + textureTransformMatrix, + /* bufferWidth= */ planeTexture.width, + /* bufferHeight= */ inputFormat.height, + /* formatWidth= */ outputWidth, + /* formatHeight= */ outputHeight, + inputFormat.rotationDegrees); + program.setFloatsUniform("uTexTransformationMatrix", textureTransformMatrix); + + int inputColorSpace = + inputFormat.colorInfo != null && inputFormat.colorInfo.colorSpace != Format.NO_VALUE + ? inputFormat.colorInfo.colorSpace + : C.COLOR_SPACE_BT709; + program.setIntUniform("uInputColorGamut", inputColorSpace); + program.setSamplerTexIdUniform("uTexSampler", planeTexture.texId, /* texUnitIndex= */ 0); + program.bindAttributesAndUniforms(); + } + + private GlTextureFrame buildGlTextureFrame( + ImagePlanesFrame inputFrame, + GlTextureInfo outputTexture, + Executor glExecutor, + Executor listenerExecutor, + FrameProcessor.Listener listener) { + return new GlTextureFrame.Builder( + outputTexture, + /* releaseTextureExecutor= */ glExecutor, + /* releaseTextureCallback= */ info -> { + @Nullable GlTextureInfo activeTexture = activeOutputTextures.remove(inputFrame); + if (activeTexture == null) { + // The frame's GL resources were already reclaimed, by releaseGlResources() after + // a downstream rejection or by close(). Notifying the listener here would + // falsely signal that processing succeeded. + return; + } + if (isClosed) { + try { + activeTexture.release(); + } catch (GlException e) { + errorConsumer.accept(VideoFrameProcessingException.from(e)); + } + } else { + availableOutputTextures.addLast(activeTexture); + } + listenerExecutor.execute( + () -> listener.onFrameProcessed(inputFrame, /* onCompleteFence= */ null)); + }) + .setPresentationTimeUs(inputFrame.getContentTimeUs()) + .setFormat( + inputFrame + .getFormat() + .buildUpon() + .setWidth(outputTexture.width) + .setHeight(outputTexture.height) + .setColorInfo(outputColorInfo) + // Reset rotation to 0 because we rotated the frame physically with OpenGL. The + // pipeline should always receive frames in their intended orientation. + .setRotationDegrees(0) + .build()) + .setMetadata(inputFrame.getMetadata()) + .build(); + } + + /** + * The GL texture holding one uploaded image plane, and the storage currently allocated for it. + */ + private static final class PlaneTexture { + + /** The texture ID. */ + private final int texId; + + /** The allocated width in pixels. */ + private final int width; + + /** The allocated height in pixels. */ + private final int height; + + /** The allocated GL format. */ + private final int glFormat; + + private PlaneTexture(int texId, int width, int height, int glFormat) { + this.texId = texId; + this.width = width; + this.height = height; + this.glFormat = glFormat; + } + } +} diff --git a/libraries/effect/src/main/java/androidx/media3/effect/LanczosResample.java b/libraries/effect/src/main/java/androidx/media3/effect/LanczosResample.java index 9124cb24690..d3f3e84821e 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/LanczosResample.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/LanczosResample.java @@ -15,6 +15,7 @@ */ package androidx.media3.effect; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; import static com.google.common.base.Preconditions.checkArgument; import static java.lang.Math.abs; import static java.lang.Math.min; @@ -23,6 +24,8 @@ import android.content.Context; import androidx.annotation.FloatRange; import androidx.annotation.IntRange; +import androidx.annotation.RestrictTo; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.Size; import androidx.media3.common.util.UnstableApi; @@ -34,7 +37,7 @@ *

The filter rescales images in both dimensions with the same scaling factor. */ @UnstableApi -public final class LanczosResample implements GlEffect { +public final class LanczosResample implements GlEffect, MetricsProvider { // Default value for the radius, or alpha parameter used by Lanczos filter. A value of 3 is // used by ffmpeg (https://ffmpeg.org/ffmpeg-scaler.html), libplacebo, or Apple's vImage library. private static final float DEFAULT_RADIUS = 3f; @@ -126,6 +129,12 @@ public boolean isNoOp(int inputWidth, int inputHeight) { < NO_OP_THRESHOLD; } + @Override + @RestrictTo(LIBRARY_GROUP) + public void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_EFFECT_SPATIAL); + } + /** * Returns the scaling factor required to fit an input image into a target rectangle. * @@ -146,6 +155,19 @@ private static float scalingFactorToFit( } } + private static Size getTargetSize( + int inputWidth, + int inputHeight, + int longSide, + int shortSide, + boolean assumeLandscapeOrientation) { + if (assumeLandscapeOrientation || inputWidth > inputHeight) { + return new Size(longSide, shortSide); + } else { + return new Size(shortSide, longSide); + } + } + private static class LanczosResampleScaledFunctionProvider implements ConvolutionFunction1D.Provider { // Note: We deliberately don't use Float.MIN_VALUE because it's positive & very close to zero. @@ -198,17 +220,4 @@ public Size configure(Size inputSize) { return new Size(round(inputSize.getWidth() * scale), round(inputSize.getHeight() * scale)); } } - - private static Size getTargetSize( - int inputWidth, - int inputHeight, - int longSide, - int shortSide, - boolean assumeLandscapeOrientation) { - if (assumeLandscapeOrientation || inputWidth > inputHeight) { - return new Size(longSide, shortSide); - } else { - return new Size(shortSide, longSide); - } - } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/MatrixTransformation.java b/libraries/effect/src/main/java/androidx/media3/effect/MatrixTransformation.java index 5473485caed..8274fa250fb 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/MatrixTransformation.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/MatrixTransformation.java @@ -15,7 +15,11 @@ */ package androidx.media3.effect; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; + import android.graphics.Matrix; +import androidx.annotation.RestrictTo; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.util.UnstableApi; /** @@ -29,7 +33,7 @@ * applicable. */ @UnstableApi -public interface MatrixTransformation extends GlMatrixTransformation { +public interface MatrixTransformation extends GlMatrixTransformation, MetricsProvider { /** * Returns the 3x3 transformation {@link Matrix} to apply to the frame with the given timestamp. */ @@ -39,4 +43,10 @@ public interface MatrixTransformation extends GlMatrixTransformation { default float[] getGlMatrixArray(long presentationTimeUs) { return MatrixUtils.getGlMatrixArray(getMatrix(presentationTimeUs)); } + + @Override + @RestrictTo(LIBRARY_GROUP) + default void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_EFFECT_SPATIAL); + } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/OverlayEffect.java b/libraries/effect/src/main/java/androidx/media3/effect/OverlayEffect.java index 49416f29e31..dca9151fe63 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/OverlayEffect.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/OverlayEffect.java @@ -15,7 +15,11 @@ */ package androidx.media3.effect; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; + import android.content.Context; +import androidx.annotation.RestrictTo; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.UnstableApi; import com.google.common.collect.ImmutableList; @@ -30,7 +34,7 @@ * DefaultVideoFrameProcessor#WORKING_COLOR_SPACE_LINEAR linear} working color space or HDR input. */ @UnstableApi -public final class OverlayEffect implements GlEffect { +public final class OverlayEffect implements GlEffect, MetricsProvider { private final ImmutableList overlays; @@ -50,4 +54,11 @@ public BaseGlShaderProgram toGlShaderProgram(Context context, boolean useHdr) throws VideoFrameProcessingException { return new OverlayShaderProgram(context, useHdr, overlays); } + + @Override + @RestrictTo(LIBRARY_GROUP) + public void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_EFFECT_OVERLAY); + consumer.addMetric(MetricConsumer.METRIC_OVERLAY_COUNT, overlays.size()); + } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/RgbMatrix.java b/libraries/effect/src/main/java/androidx/media3/effect/RgbMatrix.java index 05f0be162bb..f82bc8159ef 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/RgbMatrix.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/RgbMatrix.java @@ -16,7 +16,11 @@ package androidx.media3.effect; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; + import android.content.Context; +import androidx.annotation.RestrictTo; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.UnstableApi; import com.google.common.collect.ImmutableList; @@ -25,7 +29,7 @@ * Specifies a 4x4 RGB color transformation matrix to apply to each frame in the fragment shader. */ @UnstableApi -public interface RgbMatrix extends GlEffect { +public interface RgbMatrix extends GlEffect, MetricsProvider { /** * Returns the 4x4 RGB transformation {@linkplain android.opengl.Matrix matrix} to apply to the @@ -48,4 +52,10 @@ default BaseGlShaderProgram toGlShaderProgram(Context context, boolean useHdr) /* rgbMatrices= */ ImmutableList.of(this), useHdr); } + + @Override + @RestrictTo(LIBRARY_GROUP) + default void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_EFFECT_COLOR); + } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/ScaleAndRotateTransformation.java b/libraries/effect/src/main/java/androidx/media3/effect/ScaleAndRotateTransformation.java index 5b17b91f960..dcd511ce724 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/ScaleAndRotateTransformation.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/ScaleAndRotateTransformation.java @@ -15,12 +15,14 @@ */ package androidx.media3.effect; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkNotNull; import static java.lang.Math.max; import static java.lang.Math.min; import android.graphics.Matrix; +import androidx.annotation.RestrictTo; import androidx.media3.common.util.GlUtil; import androidx.media3.common.util.Size; import androidx.media3.common.util.UnstableApi; @@ -171,4 +173,13 @@ public boolean isNoOp(int inputWidth, int inputHeight) { && inputWidth == outputSize.getWidth() && inputHeight == outputSize.getHeight(); } + + @Override + @RestrictTo(LIBRARY_GROUP) + public void populateMetrics(MetricConsumer consumer) { + MatrixTransformation.super.populateMetrics(consumer); + if (rotationDegrees != 0f) { + consumer.addMetric(MetricConsumer.METRIC_ROTATION_DEGREES, Math.round(rotationDegrees)); + } + } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/SeparableConvolution.java b/libraries/effect/src/main/java/androidx/media3/effect/SeparableConvolution.java index b9c032333d5..695edb4e631 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/SeparableConvolution.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/SeparableConvolution.java @@ -15,7 +15,11 @@ */ package androidx.media3.effect; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; + import android.content.Context; +import androidx.annotation.RestrictTo; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.UnstableApi; @@ -26,7 +30,7 @@ * second pass. */ @UnstableApi -public abstract class SeparableConvolution implements GlEffect { +public abstract class SeparableConvolution implements GlEffect, MetricsProvider { private final float scaleWidth; private final float scaleHeight; @@ -61,4 +65,10 @@ public GlShaderProgram toGlShaderProgram(Context context, boolean useHdr) return new SeparableConvolutionShaderProgram( context, useHdr, /* convolution= */ this, scaleWidth, scaleHeight); } + + @Override + @RestrictTo(LIBRARY_GROUP) + public void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_EFFECT_CONVOLUTION); + } } diff --git a/libraries/effect/src/main/java/androidx/media3/effect/SimpleGlFrameProcessor.kt b/libraries/effect/src/main/java/androidx/media3/effect/SimpleGlFrameProcessor.kt deleted file mode 100644 index df46d63608d..00000000000 --- a/libraries/effect/src/main/java/androidx/media3/effect/SimpleGlFrameProcessor.kt +++ /dev/null @@ -1,612 +0,0 @@ -/* - * Copyright 2026 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package androidx.media3.effect - -import android.content.Context -import android.hardware.HardwareBuffer -import android.media.MediaFormat.COLOR_TRANSFER_SDR_VIDEO -import android.opengl.EGL14 -import android.opengl.EGL15 -import android.opengl.EGLContext -import android.opengl.EGLDisplay -import android.opengl.EGLExt -import android.opengl.EGLSurface -import android.opengl.GLES11Ext -import android.opengl.GLES20 -import android.opengl.Matrix -import android.os.Build.VERSION.SDK_INT -import androidx.annotation.GuardedBy -import androidx.annotation.RequiresApi -import androidx.media3.common.C -import androidx.media3.common.Format -import androidx.media3.common.GlObjectsProvider -import androidx.media3.common.GlTextureInfo -import androidx.media3.common.OverlaySettings -import androidx.media3.common.VideoFrameProcessingException -import androidx.media3.common.util.ExperimentalApi -import androidx.media3.common.util.GlProgram -import androidx.media3.common.util.GlUtil -import androidx.media3.common.util.Size -import androidx.media3.common.util.Util -import androidx.media3.common.video.AsyncFrame -import androidx.media3.common.video.Frame -import androidx.media3.common.video.FrameProcessor -import androidx.media3.common.video.FrameWriter -import androidx.media3.common.video.HardwareBufferFrame -import androidx.media3.common.video.SyncFenceWrapper -import androidx.media3.effect.DefaultVideoFrameProcessor.WORKING_COLOR_SPACE_ORIGINAL -import com.google.common.util.concurrent.FutureCallback -import com.google.common.util.concurrent.Futures -import com.google.common.util.concurrent.MoreExecutors -import java.time.Duration -import java.util.concurrent.Executor -import java.util.concurrent.RejectedExecutionException -import java.util.concurrent.atomic.AtomicBoolean - -/** - * Composites the [HardwareBufferFrame]s from the input frames onto the output [HardwareBufferFrame] - * using OpenGL. - */ -@RequiresApi(26) -@ExperimentalApi // TODO: b/449956776 - Remove once FrameConsumer API is finalized. -class SimpleGlFrameProcessor -private constructor( - private val context: Context, - private val hardwareBufferJniWrapper: HardwareBufferJniWrapper, - private val overlaySettingsProvider: (HardwareBufferFrame) -> OverlaySettings, - private val output: FrameWriter, - private val listenerExecutor: Executor, - private val listener: FrameProcessor.Listener, -) : FrameProcessor { - - class Factory - @JvmOverloads - constructor( - private val context: Context, - private val hardwareBufferJniWrapper: HardwareBufferJniWrapper, - private val overlaySettingsProvider: (HardwareBufferFrame) -> OverlaySettings = { _ -> - StaticOverlaySettings.Builder().build() - }, - ) : FrameProcessor.Factory { - override fun create( - output: FrameWriter, - listenerExecutor: Executor, - listener: FrameProcessor.Listener, - ): FrameProcessor { - return SimpleGlFrameProcessor( - context, - hardwareBufferJniWrapper, - overlaySettingsProvider, - output, - listenerExecutor, - listener, - ) - } - } - - /** Executor used for all blocking [SyncFenceWrapper.await] calls and GL operations. */ - private val glExecutorService = - MoreExecutors.listeningDecorator( - Util.newSingleThreadExecutor("SimpleGlFrameProcessor::InternalThread") - ) - private val internalExecutor = InternalExecutor() - private val isReleased = AtomicBoolean(false) - private val glObjectsProvider: GlObjectsProvider = DefaultGlObjectsProvider() - - private var eglDisplay: EGLDisplay? = null - private var eglContext: EGLContext? = null - private var eglSurface: EGLSurface? = null - private var compositorGlProgram: DefaultCompositorGlProgram? = null - private var externalCopyGlProgram: GlProgram? = null - - private val overlayMatrixProvider = OverlayMatrixProvider() - private var targetFormat: Format? = null - private var presentationEffect: Presentation? = null - - private val lock = Any() - @GuardedBy("lock") private var needsWakeup = false - @GuardedBy("lock") private var hasPendingTask = false - private var endOfStreamSignaled = false - - override fun queue(frames: List): Boolean { - check(!isReleased.get()) - - synchronized(lock) { - if (hasPendingTask) { - needsWakeup = true - return false - } - hasPendingTask = true - } - - internalExecutor.execute { queueInternal(frames) } - return true - } - - override fun signalEndOfStream() { - if (isReleased.get()) { - return - } - internalExecutor.execute { - synchronized(lock) { - // If hasPendingTask is true, queueInternal is waiting for capacity. - // We must defer the EOS until that frame completes. - if (!hasPendingTask) { - output.signalEndOfStream() - } else { - endOfStreamSignaled = true - } - } - } - } - - override fun close() { - if (!isReleased.getAndSet(true)) { - internalExecutor.execute { - compositorGlProgram?.release() - compositorGlProgram = null - externalCopyGlProgram?.delete() - externalCopyGlProgram = null - eglSurface?.let { GlUtil.destroyEglSurface(checkNotNull(eglDisplay), it) } - eglSurface = null - eglDisplay?.let { glObjectsProvider.release(it) } - eglDisplay = null - eglContext = null - } - glExecutorService.shutdown() - } - } - - /** - * Processes the input frames and queues them downstream. - * - * Called on the internal thread. - */ - private fun queueInternal(inputFrames: List) { - val firstInputFrame = inputFrames[0].frame - // Step 1: Check format support and configure the output. - if (targetFormat == null) { - // Mark the frames as complete and notify the listener of an exception when the format is - // unsupported. - if (!output.info.isSupported(firstInputFrame.format, USAGE)) { - listenerExecutor.execute { - for (i in inputFrames.indices) { - listener.onFrameProcessed(inputFrames[i].frame, /* onCompleteFence= */ null) - } - } - listenerExecutor.execute { - listener.onError( - VideoFrameProcessingException("Format ${firstInputFrame.format} is unsupported.") - ) - } - return - } - targetFormat = firstInputFrame.format - presentationEffect = - Presentation.createForWidthAndHeight( - targetFormat!!.width, - targetFormat!!.height, - Presentation.LAYOUT_SCALE_TO_FIT, - ) - output.configure( - targetFormat!!, - Frame.USAGE_GPU_COLOR_OUTPUT or Frame.USAGE_GPU_SAMPLED_IMAGE, - ) - } - - // Step 2: Attempt to dequeue output frame. - val outputFrame = - output.dequeueInputFrame( - /* wakeupExecutor= */ internalExecutor, - /* wakeupListener= */ { queueInternal(inputFrames) }, - ) - if (outputFrame == null) { - return - } - - var inputReleaseFences: List = emptyList() - try { - - // Step 3: Composite the input frames onto the output frame. - val result = processFrames(inputFrames, outputFrame) - val drawingCompletionFence = result.drawCompleteFence - inputReleaseFences = result.readCompleteFences - - // Step 4: Update the output frame with the new content time and metadata. - val updatedOutputFrame = - (outputFrame.frame as HardwareBufferFrame) - .buildUpon() - .setContentTimeUs(firstInputFrame.contentTimeUs) - .setMetadata(firstInputFrame.metadata) - .build() - - // Step 5: Send the output frame downstream. - output.queueInputFrame(updatedOutputFrame, drawingCompletionFence) - - // Step 6: Send any pending EOS signal downstream if needed. - if (endOfStreamSignaled) { - output.signalEndOfStream() - endOfStreamSignaled = false - } - } finally { - // Step 7: Always mark the input frames as complete. - listenerExecutor.execute { - for (i in inputFrames.indices) { - val asyncFrame = inputFrames[i] - val releaseFence = if (i < inputReleaseFences.size) inputReleaseFences[i] else null - listener.onFrameProcessed(asyncFrame.frame, releaseFence) - } - } - - // Step 8: Allow the next frame to be queued. - synchronized(lock) { hasPendingTask = false } - - // Step 9: Notify the wakeup listener if it is set. - maybeTriggerWakeup() - } - } - - /** Called on the internal thread. */ - private fun maybeSetupGl() { - if (eglContext != null) { - return - } - val eglDisplay = GlUtil.getDefaultEglDisplay() - this.eglDisplay = eglDisplay - val eglContext = - glObjectsProvider.createEglContext( - eglDisplay, - /* openGlVersion= */ 3, - /* configAttributes= */ GlUtil.EGL_CONFIG_ATTRIBUTES_RGBA_8888, - ) - this.eglContext = eglContext - eglSurface = glObjectsProvider.createFocusedPlaceholderEglSurface(eglContext, eglDisplay) - compositorGlProgram = DefaultCompositorGlProgram(context) - val externalCopyGlProgram = - GlProgram( - context, - /* vertexShaderResId= */ R.raw.vertex_shader_transformation_es2, - /* fragmentShaderResId= */ R.raw.fragment_shader_transformation_sdr_external_es2, - ) - externalCopyGlProgram.setBufferAttribute( - "aFramePosition", - GlUtil.getNormalizedCoordinateBounds(), - GlUtil.HOMOGENEOUS_COORDINATE_VECTOR_SIZE, - ) - // TODO: b/475511306 - Ensure the frame is correctly rotated. - externalCopyGlProgram.setFloatsUniform( - "uTexTransformationMatrix", - GlUtil.create4x4IdentityMatrix(), - ) - externalCopyGlProgram.setFloatsUniform( - "uTransformationMatrix", - GlUtil.create4x4IdentityMatrix(), - ) - externalCopyGlProgram.setFloatsUniform("uRgbMatrix", GlUtil.create4x4IdentityMatrix()) - externalCopyGlProgram.setIntUniform("uSdrWorkingColorSpace", WORKING_COLOR_SPACE_ORIGINAL) - externalCopyGlProgram.setIntUniform("uOutputColorTransfer", COLOR_TRANSFER_SDR_VIDEO) - this.externalCopyGlProgram = externalCopyGlProgram - } - - /** - * Composites the input frames onto the output frame. - * - * Called on the internal thread. - */ - private fun processFrames( - inputAsyncFrames: List, - outputAsyncFrame: AsyncFrame, - ): ProcessedFrameResult { - val inputFrameInfos = mutableListOf() - val inputEglImages = mutableListOf() - val inputTextures = mutableListOf() - val intermediateFbos = mutableListOf() - var outputEglImage = 0L - var outputTexId = 0 - var outputFboId = 0 - - var drawCompleteFence: SyncFenceWrapper? = null - val readCompleteFences = mutableListOf() - - try { - // Ensure the OpenGL context is configured. - maybeSetupGl() - - val outputFrame = outputAsyncFrame.frame as HardwareBufferFrame - val outputHardwareBuffer = outputFrame.hardwareBuffer - val outputWidth = outputHardwareBuffer.width - val outputHeight = outputHardwareBuffer.height - if (outputWidth <= 0 || outputHeight <= 0) { - throw VideoFrameProcessingException( - "Invalid output dimensions: ${outputWidth}x${outputHeight}. " + - "Format: ${outputFrame.format.width}x${outputFrame.format.height}. " + - "HardwareBuffer: ${outputHardwareBuffer.width}x${outputHardwareBuffer.height}." - ) - } - - val targetWidth = targetFormat!!.width - val targetHeight = targetFormat!!.height - overlayMatrixProvider.configure(Size(targetWidth, targetHeight)) - - for (i in inputAsyncFrames.indices) { - val inputAsyncFrame = inputAsyncFrames[i] - val inputFrame = inputAsyncFrame.frame as HardwareBufferFrame - val hardwareBuffer = inputFrame.hardwareBuffer - - // TODO: b/479415385 - Replace this with a GPU wait. - // Wait on the input fences, to ensure the input frames are ready to be read from. - waitAndClose(inputAsyncFrame.acquireFence) - - // Create an EGL image and texture for each input frame. - val eglImage = - hardwareBufferJniWrapper.nativeCreateEglImageFromHardwareBuffer( - checkNotNull(eglDisplay).nativeHandle, - hardwareBuffer, - ) - if (eglImage == 0L) throw GlUtil.GlException("Failed to create input EGLImage") - inputEglImages.add(eglImage) - - val isRgba8888 = hardwareBuffer.format == HardwareBuffer.RGBA_8888 - val target = if (isRgba8888) GLES20.GL_TEXTURE_2D else GLES11Ext.GL_TEXTURE_EXTERNAL_OES - - // TODO: b/459374133 - Reuse textures. - val texId = GlUtil.generateTexture() - GLES20.glBindTexture(target, texId) - check(hardwareBufferJniWrapper.nativeBindEGLImage(target, eglImage)) - inputTextures.add(texId) - - var finalTexId = texId - val isExternalInput = !isRgba8888 - - // Draw into a GL_TEXTURE_2D if the input is an external texture, as the compositor can - // only handle GL_TEXTURE_2D. - // TODO: b/286211012 - Handle external textures in the compositor. - if (isExternalInput) { - // TODO: b/459374133 - Reuse textures. - // TODO: b/449957627 - This will not correctly handle mixing inputs with different - // colors. Map Everything to a working color space before compositing. - val internalTexId = - GlUtil.createTexture( - hardwareBuffer.width, - hardwareBuffer.height, - /* useHighPrecisionColorComponents= */ false, - ) - inputTextures.add(internalTexId) - val intermediateFboId = GlUtil.createFboForTexture(internalTexId) - intermediateFbos.add(intermediateFboId) - GlUtil.focusFramebufferUsingCurrentContext( - intermediateFboId, - hardwareBuffer.width, - hardwareBuffer.height, - ) - val copyProgram = checkNotNull(externalCopyGlProgram) - copyProgram.use() - copyProgram.setSamplerTexIdUniform("uTexSampler", texId, /* texUnitIndex= */ 0) - copyProgram.bindAttributesAndUniforms() - GLES20.glDrawArrays(GLES20.GL_TRIANGLE_STRIP, /* first= */ 0, /* count= */ 4) - GlUtil.checkGlError() - - finalTexId = internalTexId - } - - // Calculate the placement of each input texture on the output texture. - val textureInfo = - GlTextureInfo( - finalTexId, - /* fboId= */ C.INDEX_UNSET, - /* rboId= */ C.INDEX_UNSET, - hardwareBuffer.width, - hardwareBuffer.height, - ) - val overlaySettings = overlaySettingsProvider(inputFrame) - - // Configure presentation effect with current input size - presentationEffect!!.configure(hardwareBuffer.width, hardwareBuffer.height) - val presentationMatrix = presentationEffect!!.getGlMatrixArray(inputFrame.contentTimeUs) - - // Calculate overlay matrix assuming the input is logically target size - val overlayMatrix = - overlayMatrixProvider.getTransformationMatrix( - Size(targetWidth, targetHeight), - overlaySettings, - ) - - // Combine them: combined = overlayMatrix * presentationMatrix - val combinedMatrix = FloatArray(16) - Matrix.multiplyMM( - combinedMatrix, - /* resultOffset= */ 0, - overlayMatrix, - /* lhsOffset= */ 0, - presentationMatrix, - /* rhsOffset= */ 0, - ) - - inputFrameInfos.add(GlCompositionFrame(textureInfo, overlaySettings, combinedMatrix)) - } - - // Wait on the output fence, to ensure it is ready to be written to. - waitAndClose(outputAsyncFrame.acquireFence) - - // Create the output EGL image and texture. - outputEglImage = - hardwareBufferJniWrapper.nativeCreateEglImageFromHardwareBuffer( - checkNotNull(eglDisplay).nativeHandle, - outputHardwareBuffer, - ) - if (outputEglImage == 0L) { - throw GlUtil.GlException("Failed to create output EGLImage") - } - - // TODO: b/459374133 - Reuse textures. - outputTexId = GlUtil.generateTexture() - GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, outputTexId) - check(hardwareBufferJniWrapper.nativeBindEGLImage(GLES20.GL_TEXTURE_2D, outputEglImage)) - - outputFboId = GlUtil.createFboForTexture(outputTexId) - val outputTextureInfo = - GlTextureInfo( - outputTexId, - outputFboId, - /* rboId= */ C.INDEX_UNSET, - outputWidth, - outputHeight, - ) - - // Draw the input textures into the output texture. - checkNotNull(compositorGlProgram).drawFrame(inputFrameInfos, outputTextureInfo) - - val eglDisplay = checkNotNull(eglDisplay) - val extensions = EGL14.eglQueryString(eglDisplay, EGL14.EGL_EXTENSIONS) ?: "" - if (SDK_INT >= 33 && extensions.contains("EGL_ANDROID_native_fence_sync")) { - val sync = - EGL15.eglCreateSync( - eglDisplay, - /* type= */ EGLExt.EGL_SYNC_NATIVE_FENCE_ANDROID, - /* attrib_list= */ longArrayOf(EGL14.EGL_NONE.toLong()), - /* offset= */ 0, - ) - GlUtil.checkEglException("eglCreateSync failed") - if (sync != EGL15.EGL_NO_SYNC) { - try { - var nativeFence = EGLExt.eglDupNativeFenceFDANDROID(eglDisplay, sync) - GlUtil.checkEglException("eglDupNativeFenceFDANDROID failed") - if (!nativeFence.isValid) { - // Calling eglDupNativeFenceAndroid may produce an invalid fence the first time it - // is called. See b/18052459. - GLES20.glFlush() - nativeFence = EGLExt.eglDupNativeFenceFDANDROID(eglDisplay, sync) - GlUtil.checkEglException("eglDupNativeFenceFDANDROID failed after glFlush") - } - if (nativeFence.isValid) { - drawCompleteFence = SyncFenceWrapper.of(/* syncFence= */ nativeFence) - for (i in inputAsyncFrames.indices) { - val inputNativeFence = EGLExt.eglDupNativeFenceFDANDROID(eglDisplay, sync) - GlUtil.checkEglException("eglDupNativeFenceFDANDROID failed for input frame") - check(inputNativeFence.isValid) - readCompleteFences.add(SyncFenceWrapper.of(/* syncFence= */ inputNativeFence)) - } - } - } finally { - EGL15.eglDestroySync(eglDisplay, sync) - GlUtil.checkEglException("eglDestroySync failed") - } - } - } - - if (drawCompleteFence == null) { - GLES20.glFinish() - } - - return ProcessedFrameResult(drawCompleteFence, readCompleteFences) - } catch (e: Exception) { - drawCompleteFence?.close() - readCompleteFences.forEach { it?.close() } - throw e - } finally { - // Clean up GL resources. - for (fboId in intermediateFbos) { - GlUtil.deleteFbo(fboId) - } - if (outputFboId != 0) { - GlUtil.deleteFbo(outputFboId) - } - if (outputTexId != 0) { - GlUtil.deleteTexture(outputTexId) - } - if (outputEglImage != 0L) { - check( - hardwareBufferJniWrapper.nativeDestroyEGLImage( - checkNotNull(eglDisplay).nativeHandle, - outputEglImage, - ) - ) - } - for (texId in inputTextures) { - GlUtil.deleteTexture(texId) - } - for (eglImage in inputEglImages) { - check( - hardwareBufferJniWrapper.nativeDestroyEGLImage( - checkNotNull(eglDisplay).nativeHandle, - eglImage, - ) - ) - } - } - } - - // TODO: b/479415385 - Replace this with a GPU wait. - private fun waitAndClose(fence: SyncFenceWrapper?) { - if (fence == null) { - return - } - try { - check(fence.await(FENCE_TIMEOUT)) { "Fence timeout" } - } finally { - fence.close() - } - } - - private fun maybeTriggerWakeup() { - var needsWakeup = false - synchronized(lock) { - needsWakeup = this.needsWakeup - this.needsWakeup = false - } - if (needsWakeup) { - listenerExecutor.execute { listener.onWakeup() } - } - } - - /** - * Result of [processFrames]. - * - * @property drawCompleteFence The [SyncFenceWrapper] that signals when the output frame is ready. - * @property readCompleteFences The [SyncFenceWrapper] instances that signal when the input frames - * have been read by the GPU. - */ - private data class ProcessedFrameResult( - val drawCompleteFence: SyncFenceWrapper?, - val readCompleteFences: List, - ) - - /** Helper class to submit work to [glExecutorService] and handle exceptions. */ - private inner class InternalExecutor : Executor { - override fun execute(runnable: Runnable) { - try { - Futures.addCallback( - glExecutorService.submit(runnable), - object : FutureCallback { - override fun onSuccess(result: Any?) {} - - override fun onFailure(t: Throwable) { - listener.onError(VideoFrameProcessingException(t)) - } - }, - listenerExecutor, - ) - } catch (e: RejectedExecutionException) { - listenerExecutor.execute { listener.onError(VideoFrameProcessingException(e)) } - } - } - } - - companion object { - private val FENCE_TIMEOUT = Duration.ofMillis(500) - private val USAGE = Frame.USAGE_GPU_COLOR_OUTPUT or Frame.USAGE_GPU_SAMPLED_IMAGE - } -} diff --git a/libraries/effect/src/main/java/androidx/media3/effect/TexturePool.java b/libraries/effect/src/main/java/androidx/media3/effect/TexturePool.java index 093b400855f..a7c35cf8420 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/TexturePool.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/TexturePool.java @@ -19,6 +19,7 @@ import androidx.annotation.Nullable; import androidx.annotation.VisibleForTesting; +import androidx.media3.common.ColorInfo; import androidx.media3.common.GlObjectsProvider; import androidx.media3.common.GlTextureInfo; import androidx.media3.common.util.GlUtil; @@ -34,6 +35,10 @@ int createTexture(int width, int height, boolean useHighPrecisionColorComponents throws GlUtil.GlException; } + /* package */ interface Factory { + TexturePool create(ColorInfo workingColorSpace); + } + private final TextureAllocator textureAllocator; private final Deque freeTextures; private final Deque inUseTextures; diff --git a/libraries/effect/src/main/java/androidx/media3/effect/TimestampAdjustment.java b/libraries/effect/src/main/java/androidx/media3/effect/TimestampAdjustment.java index 63af603471f..7260fb57b3f 100644 --- a/libraries/effect/src/main/java/androidx/media3/effect/TimestampAdjustment.java +++ b/libraries/effect/src/main/java/androidx/media3/effect/TimestampAdjustment.java @@ -66,6 +66,7 @@ public TimestampAdjustment(TimestampMap timestampMap, SpeedProvider speedProvide } @Override + @SuppressWarnings("deprecation") // Uses deprecated TimestampAdjustmentShaderProgram. public GlShaderProgram toGlShaderProgram(Context context, boolean useHdr) { return new TimestampAdjustmentShaderProgram(timestampMap); } diff --git a/libraries/effect_ndk/src/main/jni/CMakeLists.txt b/libraries/effect/src/main/jni/CMakeLists.txt similarity index 85% rename from libraries/effect_ndk/src/main/jni/CMakeLists.txt rename to libraries/effect/src/main/jni/CMakeLists.txt index 635554366e6..f3dcbf041ee 100644 --- a/libraries/effect_ndk/src/main/jni/CMakeLists.txt +++ b/libraries/effect/src/main/jni/CMakeLists.txt @@ -19,7 +19,7 @@ cmake_minimum_required(VERSION 3.21.0 FATAL_ERROR) # Enable C++11 features. set(CMAKE_CXX_STANDARD 11) -project(hardwareBufferJNI C CXX) +project(media3EffectJNI C CXX) # Add the NDK libraries find_library(log-lib log) @@ -30,7 +30,9 @@ find_library(jnigraphics-lib jnigraphics) add_library(${PROJECT_NAME} SHARED hardware_buffer_jni.cc) - +set_target_properties(${PROJECT_NAME} PROPERTIES + CXX_VISIBILITY_PRESET hidden + VISIBILITY_INLINES_HIDDEN ON) target_link_libraries(${PROJECT_NAME} ${log-lib} diff --git a/libraries/effect_ndk/src/main/jni/hardware_buffer_jni.cc b/libraries/effect/src/main/jni/hardware_buffer_jni.cc similarity index 74% rename from libraries/effect_ndk/src/main/jni/hardware_buffer_jni.cc rename to libraries/effect/src/main/jni/hardware_buffer_jni.cc index 103962e8117..605e9a69d2c 100644 --- a/libraries/effect_ndk/src/main/jni/hardware_buffer_jni.cc +++ b/libraries/effect/src/main/jni/hardware_buffer_jni.cc @@ -59,39 +59,9 @@ static void initializeEGLFunctions() { } } -jint JNI_OnLoad(JavaVM* vm, void* reserved) { - JNIEnv* env; - if (vm->GetEnv(reinterpret_cast(&env), JNI_VERSION_1_6) != JNI_OK) { - return -1; - } - - if (API_AT_LEAST(26)) { - s_AHardwareBuffer_fromHardwareBuffer = AHardwareBuffer_fromHardwareBuffer; - s_AHardwareBuffer_describe = AHardwareBuffer_describe; - s_AHardwareBuffer_lock = AHardwareBuffer_lock; - s_AHardwareBuffer_unlock = AHardwareBuffer_unlock; - } else { - LOGE( - "AHardwareBuffer APIs are not available on this Android version (API < " - "26)"); - return -1; - } - - initializeEGLFunctions(); - if (eglCreateImageKHR == nullptr || - eglGetNativeClientBufferANDROID == nullptr || - glEGLImageTargetTexture2DOES == nullptr || - eglDestroyImageKHR == nullptr) { - LOGE("Failed to get addresses of GL/EGL functions."); - return -1; - } - return JNI_VERSION_1_6; -} - -extern "C" JNIEXPORT jlong JNICALL -Java_androidx_media3_effect_ndk_HardwareBufferJni_nativeCreateEglImageFromHardwareBuffer( - JNIEnv* env, jobject /* this */, jlong displayHandle, - jobject hardwareBufferJava) { +jlong nativeCreateEglImageFromHardwareBuffer(JNIEnv* env, jobject /* this */, + jlong displayHandle, + jobject hardwareBufferJava) { EGLDisplay display = reinterpret_cast(displayHandle); if (display == EGL_NO_DISPLAY) { LOGE("Invalid EGL display"); @@ -119,9 +89,8 @@ Java_androidx_media3_effect_ndk_HardwareBufferJni_nativeCreateEglImageFromHardwa return (jlong)eglImage; } -extern "C" JNIEXPORT jboolean JNICALL -Java_androidx_media3_effect_ndk_HardwareBufferJni_nativeBindEGLImage( - JNIEnv* env, jobject clazz, jint target, jlong eglImageHandle) { +jboolean nativeBindEGLImage(JNIEnv* /* env */, jobject /* this */, jint target, + jlong eglImageHandle) { if (eglImageHandle == 0) { LOGE("Invalid eglImageHandle (0)"); return JNI_FALSE; @@ -140,10 +109,9 @@ Java_androidx_media3_effect_ndk_HardwareBufferJni_nativeBindEGLImage( return JNI_TRUE; } -extern "C" JNIEXPORT jboolean JNICALL -Java_androidx_media3_effect_ndk_HardwareBufferJni_nativeDestroyEGLImage( - JNIEnv* env, jobject clazz, jlong displayHandle, jlong imageHandle) { - if (imageHandle == 0) { +jboolean nativeDestroyEGLImage(JNIEnv* /* env */, jobject /* this */, + jlong displayHandle, jlong eglImageHandle) { + if (eglImageHandle == 0) { LOGE("Invalid eglImageHandle (0)"); return JNI_FALSE; } @@ -153,7 +121,7 @@ Java_androidx_media3_effect_ndk_HardwareBufferJni_nativeDestroyEGLImage( return JNI_FALSE; } - EGLImageKHR image = reinterpret_cast(imageHandle); + EGLImageKHR image = reinterpret_cast(eglImageHandle); EGLBoolean result = eglDestroyImageKHR(display, image); if (result == EGL_TRUE) { @@ -164,12 +132,12 @@ Java_androidx_media3_effect_ndk_HardwareBufferJni_nativeDestroyEGLImage( } } -extern "C" JNIEXPORT jboolean JNICALL -Java_androidx_media3_effect_ndk_HardwareBufferJni_nativeCopyBitmapToHardwareBuffer( - JNIEnv* env, jobject clazz, jobject bitmap, jobject hardwareBuffer) { +jboolean nativeCopyBitmapToHardwareBuffer(JNIEnv* env, jobject /* this */, + jobject bitmap, + jobject hardwareBufferJava) { AHardwareBuffer* hb; if (API_AT_LEAST(26)) { - hb = s_AHardwareBuffer_fromHardwareBuffer(env, hardwareBuffer); + hb = s_AHardwareBuffer_fromHardwareBuffer(env, hardwareBufferJava); } else { LOGE( "AHardwareBuffer APIs are not available on this Android version (API < " @@ -188,10 +156,11 @@ Java_androidx_media3_effect_ndk_HardwareBufferJni_nativeCopyBitmapToHardwareBuff } if (bitmapInfo.format != ANDROID_BITMAP_FORMAT_RGBA_8888 && - bitmapInfo.format != ANDROID_BITMAP_FORMAT_RGBA_1010102) { + bitmapInfo.format != ANDROID_BITMAP_FORMAT_RGBA_1010102 && + bitmapInfo.format != ANDROID_BITMAP_FORMAT_RGBA_F16) { LOGE( - "Unsupported bitmap format: %d. Only RGBA_8888 and RGBA_1010102 are " - "supported.", + "Unsupported bitmap format: %d. Only RGBA_8888, RGBA_1010102, and " + "RGBA_F16 are supported.", bitmapInfo.format); return JNI_FALSE; } @@ -203,10 +172,18 @@ Java_androidx_media3_effect_ndk_HardwareBufferJni_nativeCopyBitmapToHardwareBuff return JNI_FALSE; } - uint32_t expectedHbFormat = - (bitmapInfo.format == ANDROID_BITMAP_FORMAT_RGBA_1010102) - ? AHARDWAREBUFFER_FORMAT_R10G10B10A2_UNORM - : AHARDWAREBUFFER_FORMAT_R8G8B8A8_UNORM; + uint32_t expectedHbFormat; + int bpp; + if (bitmapInfo.format == ANDROID_BITMAP_FORMAT_RGBA_1010102) { + expectedHbFormat = AHARDWAREBUFFER_FORMAT_R10G10B10A2_UNORM; + bpp = 4; + } else if (bitmapInfo.format == ANDROID_BITMAP_FORMAT_RGBA_F16) { + expectedHbFormat = AHARDWAREBUFFER_FORMAT_R16G16B16A16_FLOAT; + bpp = 8; + } else { + expectedHbFormat = AHARDWAREBUFFER_FORMAT_R8G8B8A8_UNORM; + bpp = 4; + } if (hbDesc.format != expectedHbFormat) { LOGE( @@ -235,8 +212,6 @@ Java_androidx_media3_effect_ndk_HardwareBufferJni_nativeCopyBitmapToHardwareBuff uint8_t* src = static_cast(bitmapPixels); uint8_t* dst = static_cast(hbPixels); - // Both RGBA_8888 and RGBA_1010102 formats are 32-bit (4 bytes per pixel). - const int bpp = 4; const size_t rowSize = bitmapInfo.width * bpp; for (uint32_t y = 0; y < bitmapInfo.height; ++y) { @@ -254,9 +229,10 @@ Java_androidx_media3_effect_ndk_HardwareBufferJni_nativeCopyBitmapToHardwareBuff return success; } -extern "C" JNIEXPORT jboolean JNICALL -Java_androidx_media3_effect_ndk_HardwareBufferJni_nativeCopyHardwareBufferToHardwareBuffer( - JNIEnv* env, jobject clazz, jobject srcHbJava, jobject dstHbJava) { +jboolean nativeCopyHardwareBufferToHardwareBuffer(JNIEnv* env, + jobject /* this */, + jobject srcHbJava, + jobject dstHbJava) { if (env->IsSameObject(srcHbJava, dstHbJava)) { return JNI_TRUE; } @@ -299,10 +275,12 @@ Java_androidx_media3_effect_ndk_HardwareBufferJni_nativeCopyHardwareBufferToHard if (srcDesc.format == AHARDWAREBUFFER_FORMAT_R8G8B8A8_UNORM || srcDesc.format == AHARDWAREBUFFER_FORMAT_R10G10B10A2_UNORM) { bpp = 4; + } else if (srcDesc.format == AHARDWAREBUFFER_FORMAT_R16G16B16A16_FLOAT) { + bpp = 8; } else { LOGE( - "Unsupported hardware buffer format: %u. Only RGBA_8888 and " - "RGBA_1010102 are supported.", + "Unsupported hardware buffer format: %u. Only RGBA_8888, " + "RGBA_1010102, and RGBA_FP16 are supported.", srcDesc.format); return JNI_FALSE; } @@ -338,3 +316,59 @@ Java_androidx_media3_effect_ndk_HardwareBufferJni_nativeCopyHardwareBufferToHard return JNI_TRUE; } + +extern "C" JNIEXPORT jint JNICALL JNI_OnLoad(JavaVM* vm, void* reserved) { + JNIEnv* env; + if (vm->GetEnv(reinterpret_cast(&env), JNI_VERSION_1_6) != JNI_OK) { + return -1; + } + + if (API_AT_LEAST(26)) { + s_AHardwareBuffer_fromHardwareBuffer = AHardwareBuffer_fromHardwareBuffer; + s_AHardwareBuffer_describe = AHardwareBuffer_describe; + s_AHardwareBuffer_lock = AHardwareBuffer_lock; + s_AHardwareBuffer_unlock = AHardwareBuffer_unlock; + } else { + LOGE( + "AHardwareBuffer APIs are not available on this Android version (API < " + "26)"); + return -1; + } + + initializeEGLFunctions(); + if (eglCreateImageKHR == nullptr || + eglGetNativeClientBufferANDROID == nullptr || + glEGLImageTargetTexture2DOES == nullptr || + eglDestroyImageKHR == nullptr) { + LOGE("Failed to get addresses of GL/EGL functions."); + return -1; + } + + jclass clazz = env->FindClass("androidx/media3/effect/HardwareBufferJni"); + if (!clazz) { + LOGE("JNI_OnLoad: FindClass failed for HardwareBufferJni"); + return -1; + } + static const JNINativeMethod kMethods[] = { + {"nativeCreateEglImageFromHardwareBuffer", + "(JLandroid/hardware/HardwareBuffer;)J", + reinterpret_cast(nativeCreateEglImageFromHardwareBuffer)}, + {"nativeBindEGLImage", "(IJ)Z", + reinterpret_cast(nativeBindEGLImage)}, + {"nativeDestroyEGLImage", "(JJ)Z", + reinterpret_cast(nativeDestroyEGLImage)}, + {"nativeCopyBitmapToHardwareBuffer", + "(Landroid/graphics/Bitmap;Landroid/hardware/HardwareBuffer;)Z", + reinterpret_cast(nativeCopyBitmapToHardwareBuffer)}, + {"nativeCopyHardwareBufferToHardwareBuffer", + "(Landroid/hardware/HardwareBuffer;Landroid/hardware/HardwareBuffer;)Z", + reinterpret_cast(nativeCopyHardwareBufferToHardwareBuffer)}, + }; + if (env->RegisterNatives(clazz, kMethods, + sizeof(kMethods) / sizeof(kMethods[0])) < 0) { + LOGE("JNI_OnLoad: RegisterNatives failed for HardwareBufferJni"); + return -1; + } + + return JNI_VERSION_1_6; +} diff --git a/libraries/effect/src/main/res/raw/color_conversions_es2.glsl b/libraries/effect/src/main/res/raw/color_conversions_es2.glsl new file mode 100644 index 00000000000..d3ce4f25e8f --- /dev/null +++ b/libraries/effect/src/main/res/raw/color_conversions_es2.glsl @@ -0,0 +1,94 @@ +#version 100 +// Copyright 2026 The Android Open Source Project +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// Shared SDR color science conversion functions for OpenGL ES 2.0. +// This file is included as the base fragment shader resource and provides +// #version, #extension, and precision declarations. + +#extension GL_OES_EGL_image_external : enable +precision mediump float; + +// LINT.IfChange(color_space) +const int COLOR_SPACE_BT709 = 1; +const int COLOR_SPACE_BT601 = 2; +// LINT.ThenChange(../../../../../common/src/main/java/androidx/media3/common/C.java:color_space) + +// LINT.IfChange(color_transfer) +const int COLOR_TRANSFER_LINEAR = 1; +const int COLOR_TRANSFER_SRGB = 2; +// LINT.ThenChange(../../../../../common/src/main/java/androidx/media3/common/C.java:color_transfer) + +// Matrix values based on computeXYZMatrix(BT601_525Primaries, D65WhitePoint) and +// computeXYZMatrix(BT709Primaries, D65WhitePoint) inverted. +// Direct BT.601 (SMPTE 170M 525) to BT.709 linear optical conversion matrix. +// Column-major representation: +const mat3 BT601_TO_BT709 = + mat3( 0.93954206, 0.01777222, -0.00162160, + 0.05018136, 0.96579286, -0.00436975, + 0.01027660, 0.01643490, 1.00599135); + +// Transforms electrical SDR to linear optical SDR using the vectorized sRGB EOTF. +// References: +// - IEC 61966-2-1: Multimedia systems and equipment - Colour measurement and management - Part 2-1: Colour management - Default RGB colour space - sRGB +// - Khronos Data Format Specification 1.3 (TRANSFER_SRGB): +// https://registry.khronos.org/DataFormat/specs/1.3/dataformat.1.3.inline.html#TRANSFER_SRGB +vec3 srgbEotf(vec3 electricalColor) { + vec3 linearLow = electricalColor / 12.92; + vec3 linearHigh = pow((max(electricalColor, vec3(0.0)) + vec3(0.055)) / 1.055, vec3(2.4)); + return mix(linearLow, linearHigh, step(vec3(0.04045), electricalColor)); +} + +// Transforms linear optical light to electrical SDR using the vectorized sRGB OETF. +// References: +// - IEC 61966-2-1: Multimedia systems and equipment - Colour measurement and management - Part 2-1: Colour management - Default RGB colour space - sRGB +// - Khronos Data Format Specification 1.3 (TRANSFER_SRGB): +// https://registry.khronos.org/DataFormat/specs/1.3/dataformat.1.3.inline.html#TRANSFER_SRGB +vec3 srgbOetf(vec3 opticalColor) { + vec3 clampedOpticalColor = max(opticalColor, vec3(0.0)); + vec3 electricalLow = 12.92 * clampedOpticalColor; + vec3 electricalHigh = 1.055 * pow(clampedOpticalColor, vec3(1.0 / 2.4)) - vec3(0.055); + return mix(electricalLow, electricalHigh, step(vec3(0.0031308), clampedOpticalColor)); +} + +// Converts linear optical light from BT.601 gamut to BT.709 gamut. +vec3 convertBt601ToBt709(vec3 linearBt601) { + return clamp(BT601_TO_BT709 * linearBt601, 0.0, 1.0); +} + +// Processes SDR electrical color, assuming sRGB input transfer, applying optional BT.601 to +// BT.709 gamut transformation, and converting to the requested output transfer (linear or sRGB electrical). +vec3 processColor( + vec3 inputRgbElectricalColor, + int inputColorGamut, + int outputColorGamut, + int outputColorTransfer) { + bool needsGamutConversion = + (inputColorGamut == COLOR_SPACE_BT601 && outputColorGamut == COLOR_SPACE_BT709); + + // Pass through when neither gamut conversion nor linear output is requested. + if (outputColorTransfer != COLOR_TRANSFER_LINEAR && !needsGamutConversion) { + return inputRgbElectricalColor; + } + + vec3 linearOpticalColor = srgbEotf(inputRgbElectricalColor); + if (needsGamutConversion) { + linearOpticalColor = convertBt601ToBt709(linearOpticalColor); + } + + if (outputColorTransfer == COLOR_TRANSFER_LINEAR) { + return linearOpticalColor; + } + return srgbOetf(linearOpticalColor); +} diff --git a/libraries/effect/src/main/res/raw/color_conversions_es3.glsl b/libraries/effect/src/main/res/raw/color_conversions_es3.glsl new file mode 100644 index 00000000000..1f2e2acad5e --- /dev/null +++ b/libraries/effect/src/main/res/raw/color_conversions_es3.glsl @@ -0,0 +1,469 @@ +#version 300 es +// Copyright 2026 The Android Open Source Project +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// Shared color science conversion functions. This file is intended to be included as the first +// fragment shader resource and provides the #version, #extension, and precision declarations. + +#extension GL_OES_EGL_image_external : enable +#extension GL_EXT_YUV_target : enable +precision highp float; + +// LINT.IfChange(color_space) +const int COLOR_SPACE_BT709 = 1; +const int COLOR_SPACE_BT2020 = 6; +// LINT.ThenChange(../../../../../common/src/main/java/androidx/media3/common/C.java:color_space) + +// LINT.IfChange(color_transfer) +const int COLOR_TRANSFER_LINEAR = 1; +const int COLOR_TRANSFER_SRGB = 2; +const int COLOR_TRANSFER_ST2084 = 6; +const int COLOR_TRANSFER_HLG = 7; +// LINT.ThenChange(../../../../../common/src/main/java/androidx/media3/common/C.java:color_transfer) + +// Returns whether the color transfer is an HDR transfer (HLG or PQ). +bool isTransferHdr(int colorTransfer) { + return colorTransfer == COLOR_TRANSFER_HLG || colorTransfer == COLOR_TRANSFER_ST2084; +} + +// Returns whether the color gamut is an HDR wide gamut (BT.2020). +bool isGamutHdr(int colorGamut) { + return colorGamut == COLOR_SPACE_BT2020; +} + +// Matrix values based on computeXYZMatrix(BT2020Primaries, D65WhitePoint) +// References: +// - ITU-R BT.2020-2: Parameter values for ultra-high definition television systems. +// - Android HWUI HostColorSpace: +// https://cs.android.com/android/platform/superproject/+/master:frameworks/base/libs/hwui/utils/HostColorSpace.cpp;l=200-232;drc=86bd214059cd6150304888a285941bf74af5b687 +// Column-major representation: +const mat3 BT2020_TO_XYZ = + mat3(0.63695805, 0.26270021, 0.00000000, + 0.14461690, 0.67799807, 0.02807269, + 0.16888098, 0.05930172, 1.06098506); + +// Matrix values based on computeXYZMatrix(BT2020Primaries, D65WhitePoint) inverted +// References: +// - ITU-R BT.2020-2: Parameter values for ultra-high definition television systems. +// Column-major representation: +const mat3 XYZ_TO_BT2020 = + mat3( 1.71665118, -0.66668434, 0.01763986, + -0.35567077, 1.61648124, -0.04277061, + -0.25336629, 0.01576854, 0.94210312); + +// Matrix values based on computeXYZMatrix(BT709Primaries, D65WhitePoint) +// References: +// - ITU-R BT.709-6: Parameter values for the HDTV standard for production and international +// programme exchange. +// Column-major representation: +const mat3 BT709_TO_XYZ = + mat3(0.41239080, 0.21263901, 0.01933082, + 0.35758434, 0.71516868, 0.11919478, + 0.18048079, 0.07219231, 0.95053216); + +// Matrix values based on computeXYZMatrix(BT709Primaries, D65WhitePoint) inverted +// References: +// - ITU-R BT.709-6: Parameter values for the HDTV standard for production and international +// programme exchange. +// Column-major representation: +const mat3 XYZ_TO_BT709 = + mat3(3.24096994, -0.96924364, 0.05563008, + -1.53738318, 1.87596750, -0.20397696, + -0.49861076, 0.04155506, 1.05697151); + +// YUV to RGB transform matrices (column-major format for OpenGL) +// ITU-R BT.709 +const highp mat3 BT709_FULL_RANGE_YUV_TO_RGB = + mat3(1.0000, 1.0000, 1.0000, + 0.0000, -0.1873, 1.8556, + 1.5748, -0.4681, 0.0000); +const highp mat3 BT709_LIMITED_RANGE_YUV_TO_RGB = + mat3(1.1644, 1.1644, 1.1644, + 0.0000, -0.2132, 2.1124, + 1.7927, -0.5329, 0.0000); +// ITU-R BT.2020 +const highp mat3 BT2020_FULL_RANGE_YUV_TO_RGB = + mat3(1.0000, 1.0000, 1.0000, + 0.0000, -0.1646, 1.8814, + 1.4746, -0.5714, 0.0000); +const highp mat3 BT2020_LIMITED_RANGE_YUV_TO_RGB = + mat3(1.1689, 1.1689, 1.1689, + 0.0000, -0.1881, 2.1502, + 1.6853, -0.6530, 0.0000); + +// Direct linear gamut conversion matrices (column-major): +const highp mat3 BT709_TO_BT2020 = XYZ_TO_BT2020 * BT709_TO_XYZ; +const highp mat3 BT2020_TO_BT709 = XYZ_TO_BT709 * BT2020_TO_XYZ; + +// Relative luminance weights for BT.2020 primaries (second row of BT2020_TO_XYZ). +const highp vec3 BT2020_LUMINANCE_WEIGHTS = vec3(0.26270021, 0.67799807, 0.05930172); + +// ITU-R BT.2408 diffuse white reference, expressed in display-referred light. +// +// BT.2408 defines diffuse white as 203 nits of display light, which for HLG is the 75% signal +// level: hlgEotf(0.75) = pow(hlgInverseOetf(0.75), 1.2) = pow(0.26496256, 1.2) = 0.2031521 +// (203.1521 nits on a nominal 1000-nit reference display). +// +// Scaling display light by HDR_DIFFUSE_WHITE_SCALE_UP makes 1.0 represent 203-nit diffuse white: +// - 1.0 = 203.15 nits (SDR reference white / HLG 75% / PQ 0.5807) +// - ~4.9224 = 1,000 nits (nominal HLG / HDR10 peak) +// - ~49.2242 = 10,000 nits (PQ absolute peak) +// +// The two factors are exact inverses, so scaling up on ingress and down on egress round-trips +// without loss. +// References: +// - ITU-R Report BT.2408: Guidance for operational practices in HDR television production. +const highp float HDR_DIFFUSE_WHITE_SCALE_DOWN = 0.2031521; // hlgEotf(0.75) = 203.1521 / 1000.0 +const highp float HDR_DIFFUSE_WHITE_SCALE_UP = 1.0 / HDR_DIFFUSE_WHITE_SCALE_DOWN; // ~4.9224197 + +// BT.2100 / BT.2020 HLG Inverse OETF for 3-channel RGB. +// Converts non-linear electrical HLG values [0.0, 1.0] to normalized linear optical scene light +// [0.0, 1.0] in BT.2020 color space. +// +// References: +// - ITU-R Recommendation BT.2100-2 (Table 5: "Hybrid Log-Gamma reference OETF / EOTF"): +// https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.2100-3-202502-I!!PDF-E.pdf +// - Khronos Data Format Specification 1.3 (TRANSFER_HLG): +// https://www.khronos.org/registry/DataFormat/specs/1.3/dataformat.1.3.inline.html#TRANSFER_HLG +// - Android RenderEngine ProgramCache: +// https://cs.android.com/android/platform/superproject/+/master:frameworks/native/libs/renderengine/gl/ProgramCache.cpp;l=265-279;drc=de09f10aa504fd8066370591a00c9ff1cafbb7fa +// +// The constants a, b, c are specified by ITU-R BT.2100 to ensure: +// 1. C0 value continuity: Both branches equal 1/12 at the transition point (channel = 0.5). +// 2. C1 derivative smoothness: First derivatives match at channel = 0.5 (no slope kink). +// 3. Peak normalization: Peak electrical signal 1.0 evaluates to 1.0. +// a = 0.17883277 +// b = 1.0 - 4.0 * a = 0.28466892 +// c = 0.5 - a * ln(4.0 * a) = 0.55991073 +highp vec3 hlgInverseOetf(highp vec3 hlgElectrical) { + const highp float a = 0.17883277; + const highp float b = 0.28466892; + const highp float c = 0.55991073; + highp vec3 lowBranch = (hlgElectrical * hlgElectrical) / 3.0; + highp vec3 highBranch = (vec3(b) + exp((hlgElectrical - vec3(c)) / a)) / 12.0; + return mix(lowBranch, highBranch, step(0.5, hlgElectrical)); +} + +// BT.2100 / BT.2020 HLG EOTF for 3-channel RGB. +// Converts non-linear electrical HLG values [0.0, 1.0] to normalized linear optical display light +// [0.0, 1.0] (where 1.0 = 1,000 nits) by applying OOTF(OETF^-1(E)) with nominal system gamma = 1.2. +highp vec3 hlgEotf(highp vec3 hlgElectrical) { + highp vec3 sceneLinear = hlgInverseOetf(hlgElectrical); + highp float sceneLuminanceY = max(dot(sceneLinear, BT2020_LUMINANCE_WEIGHTS), 1e-6); + return sceneLinear * pow(sceneLuminanceY, 0.2); +} + +// BT.2100 / BT.2020 HLG Inverse OOTF for 3-channel RGB. +// Converts normalized linear optical display light [0.0, 1.0] (1.0 = 1,000 nits) in BT.2020 color +// space to normalized linear optical scene light [0.0, 1.0] using the inverse HLG OOTF (gamma = 1.2, +// exponent = 1/1.2 - 1 = -1/6). +// +// Scene light values exceeding 1.0 (either from display light above 1,000 nits or saturated +// primaries near 1,000 nits where Y_d < 1.0) are scaled down to [0.0, 1.0] while preserving +// chromaticity. +// TODO(b/564898615): Roll off with the BT.2408 Annex 5 EETF instead of a single ratio, once the +// mastering peak is known. +highp vec3 hlgInverseOotf(highp vec3 displayLinear) { + displayLinear = max(displayLinear, vec3(0.0)); + highp float displayLuminanceY = max(dot(displayLinear, BT2020_LUMINANCE_WEIGHTS), 1e-6); + highp vec3 sceneLinear = displayLinear * pow(displayLuminanceY, -0.16666667); + return sceneLinear / max(1.0, max(sceneLinear.r, max(sceneLinear.g, sceneLinear.b))); +} + +// BT.2100 / BT.2020 HLG OETF for 3-channel RGB. +// Converts linear optical scene light [0.0, 1.0] in BT.2020 color space to electrical HLG values +// [0.0, 1.0]. +// +// References: +// - ITU-R Recommendation BT.2100-2 (Table 5: "Hybrid Log-Gamma reference OETF / EOTF"): +// https://www.itu.int/dms_pubrec/itu-r/rec/bt/R-REC-BT.2100-3-202502-I!!PDF-E.pdf +// - Khronos Data Format Specification 1.3 (TRANSFER_HLG): +// https://www.khronos.org/registry/DataFormat/specs/1.3/dataformat.1.3.inline.html#TRANSFER_HLG +// - Android RenderEngine ProgramCache: +// https://cs.android.com/android/platform/superproject/+/master:frameworks/native/libs/renderengine/gl/ProgramCache.cpp;l=529-543;drc=de09f10aa504fd8066370591a00c9ff1cafbb7fa +highp vec3 hlgOetf(highp vec3 linearColor) { + const highp float a = 0.17883277; + const highp float b = 0.28466892; + const highp float c = 0.55991073; + highp vec3 lowBranch = sqrt(3.0 * max(linearColor, vec3(0.0))); + highp vec3 highBranch = a * log(max(12.0 * linearColor - vec3(b), vec3(1e-6))) + vec3(c); + return mix(lowBranch, highBranch, step(1.0 / 12.0, linearColor)); +} + +// BT.2100 / BT.2020 PQ (SMPTE ST 2084) EOTF for 3-channel RGB. +// Converts electrical PQ values [0.0, 1.0] to normalized linear display values [0.0, 1.0] (1.0 = 10,000 nits). +// References: +// - SMPTE ST 2084 / ITU-R BT.2100-2 +// - Khronos Data Format Specification 1.3 (TRANSFER_PQ): +// https://registry.khronos.org/DataFormat/specs/1.3/dataformat.1.3.inline.html#TRANSFER_PQ +// - Android RenderEngine ProgramCache: +// https://cs.android.com/android/platform/superproject/+/master:frameworks/native/libs/renderengine/gl/ProgramCache.cpp;l=250-263;drc=de09f10aa504fd8066370591a00c9ff1cafbb7fa +highp vec3 pqEotf(highp vec3 pqElectricalColor) { + // SMPTE ST 2084 constants and reciprocal exponents (1.0 / m1, 1.0 / m2): + // m1 = 2610.0 / 16384.0 => invM1 = 1.0 / m1 = 16384.0 / 2610.0 + // m2 = 2523.0 / 32.0 => invM2 = 1.0 / m2 = 32.0 / 2523.0 + const highp float invM1 = 6.27739464; + const highp float invM2 = 0.0126833135; + const highp float c1 = 0.8359375; // 3424.0 / 4096.0 + const highp float c2 = 18.8515625; // 2413.0 / 128.0 + const highp float c3 = 18.6875; // 2392.0 / 128.0 + + highp vec3 temp = pow(clamp(pqElectricalColor, 0.0, 1.0), vec3(invM2)); + temp = max(temp - vec3(c1), vec3(0.0)) / (vec3(c2) - c3 * temp); + return pow(temp, vec3(invM1)); +} + +// Transforms electrical SDR to linear optical SDR using the sRGB EOTF. +// References: +// - IEC 61966-2-1: Multimedia systems and equipment - Colour measurement and management - +// Part 2-1: Colour management - Default RGB colour space - sRGB +// - Khronos Data Format Specification 1.3 (TRANSFER_SRGB): +// https://registry.khronos.org/DataFormat/specs/1.3/dataformat.1.3.inline.html#TRANSFER_SRGB +// - Android RenderEngine ProgramCache: +// https://cs.android.com/android/platform/superproject/+/master:frameworks/native/libs/renderengine/gl/ProgramCache.cpp;l=265-279;drc=de09f10aa504fd8066370591a00c9ff1cafbb7fa +highp vec3 srgbEotf(highp vec3 electricalColor) { + highp vec3 lowBranch = electricalColor / 12.92; + highp vec3 highBranch = pow((electricalColor + vec3(0.055)) / 1.055, vec3(2.4)); + return mix(lowBranch, highBranch, step(0.04045, electricalColor)); +} + +// Transforms linear optical light to electrical SDR using the sRGB OETF. +// References: +// - IEC 61966-2-1: Multimedia systems and equipment - Colour measurement and management - +// Part 2-1: Colour management - Default RGB colour space - sRGB +// - Khronos Data Format Specification 1.3 (TRANSFER_SRGB): +// https://registry.khronos.org/DataFormat/specs/1.3/dataformat.1.3.inline.html#TRANSFER_SRGB +// - Android RenderEngine ProgramCache: +// https://cs.android.com/android/platform/superproject/+/master:frameworks/native/libs/renderengine/gl/ProgramCache.cpp;l=281-285;drc=de09f10aa504fd8066370591a00c9ff1cafbb7fa +highp vec3 srgbOetf(highp vec3 opticalColor) { + highp vec3 lowBranch = 12.92 * opticalColor; + highp vec3 highBranch = 1.055 * pow(max(opticalColor, vec3(0.0)), vec3(1.0 / 2.4)) - vec3(0.055); + return mix(lowBranch, highBranch, step(0.0031308, opticalColor)); +} + +// Transforms YUV electrical values to RGB electrical values in [0.0, 1.0]. +highp vec3 yuvToRgb(highp vec3 yuv, int inputColorTransfer, int isInputColorRangeFull) { + highp vec3 yuvOffset = + (isInputColorRangeFull == 1) ? vec3(0.0, 0.5, 0.5) : vec3(0.0625, 0.5, 0.5); + highp mat3 yuvToRgbMatrix; + if (isTransferHdr(inputColorTransfer)) { + yuvToRgbMatrix = (isInputColorRangeFull == 1) + ? BT2020_FULL_RANGE_YUV_TO_RGB + : BT2020_LIMITED_RANGE_YUV_TO_RGB; + } else { + // Default to BT.709 for SDR + yuvToRgbMatrix = (isInputColorRangeFull == 1) + ? BT709_FULL_RANGE_YUV_TO_RGB + : BT709_LIMITED_RANGE_YUV_TO_RGB; + } + return clamp(yuvToRgbMatrix * (yuv - yuvOffset), 0.0, 1.0); +} + +// Converts SDR electrical BT.709 to linear optical BT.2020 display light, in the linear HDR +// optical working space where 1.0 represents both SDR reference white and BT.2408 203-nit diffuse +// white. +highp vec3 sdrElectricalToHdrDisplayLinear(highp vec3 sdrElectricalColor) { + return clamp(BT709_TO_BT2020 * srgbEotf(sdrElectricalColor), 0.0, 1.0); +} + + +// Converts HDR HLG electrical BT.2020 to linear optical BT.2020 display light, in the linear HDR +// optical working space where 1.0 represents the BT.2408 203-nit diffuse white reference. +highp vec3 hlgElectricalToHdrDisplayLinear(highp vec3 hlgElectricalColor) { + return hlgEotf(hlgElectricalColor) * HDR_DIFFUSE_WHITE_SCALE_UP; +} + +// Converts PQ electrical BT.2020 to linear optical BT.2020 display light, in the linear HDR +// optical working space where 1.0 represents the BT.2408 203-nit diffuse white reference. +highp vec3 pqElectricalToHdrDisplayLinear(highp vec3 pqElectricalColor) { + // PQ reference peak is 10,000 nits, normalize against 1,000 nits + highp vec3 displayLinear = pqEotf(pqElectricalColor) * 10.0; + return displayLinear * HDR_DIFFUSE_WHITE_SCALE_UP; +} + +// Converts linear optical BT.2020 display light (anchored at 1.0 = 203-nit diffuse white) to HLG +// electrical BT.2020. +// Egress from the linear HDR optical working space scales down by HDR_DIFFUSE_WHITE_SCALE_DOWN +// (normalizing 1,000 nits to 1.0), applies the inverse HLG OOTF to obtain scene light, and encodes +// with the HLG OETF. +highp vec3 hdrDisplayLinearToHlgElectrical(highp vec3 hdrDisplayLinear) { + // Display light is referenced at 1000 nits. + highp vec3 displayLinear = hdrDisplayLinear * HDR_DIFFUSE_WHITE_SCALE_DOWN; + return hlgOetf(hlgInverseOotf(displayLinear)); +} + +// Compresses each channel's distance from the achromatic axis max(R, G, B) into the BT.709 +// gamut along lines of constant hue (ACES 1.3 Reference Gamut Compression architecture, Academy +// TB-2021-001). +// +// Scale-invariant (f(k * rgb) == k * f(rgb)) and leaves max(R, G, B) unchanged. Operates directly +// on absolute nits before tone mapping. +highp vec3 softCompressGamutBt2020ToBt709(highp vec3 rgbBt709) { + // Knee threshold, colors within 95% of the BT.709 boundary pass through untouched. + const highp float t0 = 0.95; + // Headroom between the knee and the BT.709 boundary (1.0 - t0). + const highp float h = 0.05; + // Maximum excess distance (d_max - t0) across all BT.2020 colors, where d_max occurs on the + // BT.2020 cyan edge (G = B in BT.709). + const highp float eMax = 0.6438584; + // Curvature parameter (1.0 - h / eMax) ensuring unit slope at the knee (smooth transition with no + // kink) and mapping eMax exactly to headroom h. + const highp float a = 0.9223432; + + // Max achromatic light + highp float ach = max(rgbBt709.r, max(rgbBt709.g, rgbBt709.b)); + if (ach <= 0.0) { + return vec3(0.0); + } + + // Use a C1 smooth, simple tone curve to avoid executing the heavy ACES / BT.2407 math. + highp vec3 d = (vec3(ach) - rgbBt709) / ach; + highp vec3 e = clamp(d - vec3(t0), vec3(0.0), vec3(eMax)); + highp vec3 dCompressed = vec3(t0) + (h * e) / (a * e + vec3(h)); + highp vec3 dOut = mix(d, dCompressed, step(vec3(t0), d)); + highp vec3 compressed = vec3(ach) - dOut * ach; + return max(compressed, vec3(0.0)); +} + +// Converts BT.2020 linear display light (in nits) to linear optical BT.709 display light. +// +// Compresses BT.2020 display light down to 0-203.15 nits of SDR display light (anchored at the +// BT.2408 diffuse white reference). BT.2020 display light is first converted to BT.709 with soft +// out-of-gamut color compression (ACES 1.3 Reference Gamut Compression architecture), and then +// tone mapped using the parametric curve from Report ITU-R BT.2446-1 (Section 6.1.4, Method C). +// +// Input display light is expected up to 1,000 nits. Highlights exceeding 1,000 nits must be +// tone-mapped upstream (e.g. via BT.2408 Annex 5 EETF). +// +// Gamut compression runs first on absolute nits so the tone curve sees the peak channel of the +// target BT.709 gamut, preventing BT.2020 -> BT.709 from inflating max(R, G, B) by up to 1.66x for +// saturated colors. Tone-mapping after gamut conversion rolls off the overshot color without clipping. +// +// The tone curve is driven by max(R, G, B), and all three channels are scaled by the same ratio so +// that chromaticity is preserved in linear space. +// +// Report ITU-R BT.2446-1, Method C defines a two-segment piecewise curve: +// Y_SDR = k1 * Y_HDR for Y_HDR < Y_HDR_ip +// Y_SDR = k2 * ln(Y_HDR / Y_HDR_ip - k3) + k4 for Y_HDR >= Y_HDR_ip +// +// Its published constants target a 100/120-nit SDR display and are re-derived here for +// maxInputNits = 1000.0 and maxOutputNits = 203.1521 (BT.2408 diffuse white reference): +// 1. Linear pass-through (k1 = 1.0): Leaves shadows and midtones below the inflection point at +// 1:1 nits with 203.15-nit SDR reference white (zero shadow/midtone dimming). +// 2. Inflection point (Y_HDR_ip = 118.9 nits): Placed at 80% of the SDR electrical range +// (0.80^2.4 = 58.5% optical luminance, 118.9 nits), reserving the top 20% of the electrical +// range for compressing the upper diffuse range and 203-1,000 nit HDR highlights. +// 3. C1 derivative smoothness at Y_HDR_ip: k2 = k1 * (1.0 - k3) * Y_HDR_ip = 22.9291635. +// 4. C0 value continuity at Y_HDR_ip: k4 = k1 * Y_HDR_ip - k2 * ln(1.0 - k3) = 156.6384954. +// 5. Peak normalization Y_SDR(1000.0) = 203.1521: Solves k3 = 0.8071559. +highp vec3 bt2020DisplayLinearNitsToBt709DisplayLinear(highp vec3 nitsInBt2020) { + highp vec3 nitsIn = softCompressGamutBt2020ToBt709(BT2020_TO_BT709 * nitsInBt2020); + highp float maxColorIn = max(nitsIn.r, max(nitsIn.g, nitsIn.b)); + if (maxColorIn <= 0.0) { + return vec3(0.0); + } + const highp float maxInputNits = 1000.0; + const highp float maxOutputNits = 203.1521; + highp float nits = min(maxColorIn, maxInputNits); + + const highp float k1 = 1.0; + const highp float k2 = 22.9291635; + const highp float k3 = 0.8071559; + const highp float k4 = 156.6384954; + const highp float inflectionPointNits = 118.9; + + highp float lowBranch = k1 * nits; + highp float highBranch = k2 * log(max(nits / inflectionPointNits - k3, 1e-6)) + k4; + highp float maxColorOut = mix(lowBranch, highBranch, step(inflectionPointNits, nits)); + + // Chromaticity preservation: + // Scaling linear RGB by the tone mapping ratio (maxColorOut / maxColorIn) preserves original + // chromaticity in linear space, and dividing by maxOutputNits normalizes to [0.0, 1.0]. + // Clamping to [0.0, 1.0] guards against floating-point rounding errors on unclipped targets (e.g. FP16). + return clamp(nitsIn * (maxColorOut / (maxColorIn * maxOutputNits)), 0.0, 1.0); +} + +// Converts PQ electrical BT.2020 to linear optical BT.709 display light. +highp vec3 pqElectricalToBt709DisplayLinear(highp vec3 pqElectricalColor) { + highp vec3 nitsInBt2020 = pqEotf(pqElectricalColor) * 10000.0; + // TODO(b/314971953): Tone-map highlights above 1,000 nits using the BT.2408 Annex 5 EETF. + return bt2020DisplayLinearNitsToBt709DisplayLinear(nitsInBt2020); +} + +// Converts HDR HLG electrical BT.2020 to linear optical BT.709 display light. +highp vec3 hlgElectricalToBt709DisplayLinear(highp vec3 hlgElectricalColor) { + highp vec3 nitsInBt2020 = hlgEotf(hlgElectricalColor) * 1000.0; + return bt2020DisplayLinearNitsToBt709DisplayLinear(nitsInBt2020); +} + +// Processes input electrical color (SDR BT.709 or HDR BT.2020), applying EOTF/OETF +// transformations and tone mapping based on the requested output color gamut and transfer. +// Linear outputs (COLOR_TRANSFER_LINEAR) are always in display-referred light. +// TODO(b/549154420): Measure performance cost of branches in fragment shaders versus simpler +// specialized shader variants generated in Java. +highp vec3 processColor( + highp vec3 inputRgbElectricalColor, + int outputColorGamut, + int inputColorTransfer, + int outputColorTransfer) { + bool isInputHdr = isTransferHdr(inputColorTransfer); + bool isOutputHdr = isGamutHdr(outputColorGamut); + + // 1. SDR -> SDR (Same gamut, pass-through or linearize) + if (!isInputHdr && !isOutputHdr) { + return (outputColorTransfer == COLOR_TRANSFER_LINEAR) + ? srgbEotf(inputRgbElectricalColor) + : inputRgbElectricalColor; + } + + // 2. SDR -> HDR (Gamut expansion to display-referred linear HDR; inverse OOTF + OETF if HLG output) + // sdrElectricalToHdrDisplayLinear anchors SDR reference white (1.0) directly on the BT.2408 + // 203-nit diffuse white reference (1.0). Egress to HLG scales down to [0.0, 1.0], applies the + // inverse HLG OOTF to obtain scene light, and encodes with the HLG OETF (mapping 1.0 to 0.75). + // TODO(b/545590806): Support PQ (ST 2084) HDR output processing. + if (!isInputHdr && isOutputHdr) { + highp vec3 hdrDisplayLinear = sdrElectricalToHdrDisplayLinear(inputRgbElectricalColor); + return (outputColorTransfer == COLOR_TRANSFER_LINEAR) + ? hdrDisplayLinear + : hdrDisplayLinearToHlgElectrical(hdrDisplayLinear); + } + + // 3. HDR -> HDR (Same gamut, pass-through or linearize to display light) + if (isInputHdr && isOutputHdr) { + if (inputColorTransfer == COLOR_TRANSFER_ST2084) { + highp vec3 hdrDisplayLinear = pqElectricalToHdrDisplayLinear(inputRgbElectricalColor); + if (outputColorTransfer == COLOR_TRANSFER_LINEAR) { + return hdrDisplayLinear; + } else if (outputColorTransfer == COLOR_TRANSFER_HLG) { + return hdrDisplayLinearToHlgElectrical(hdrDisplayLinear); + } + // Green for visible error. + return vec3(0.0f, 1.0f, 0.0f); + } + // HLG input + return (outputColorTransfer == COLOR_TRANSFER_LINEAR) + ? hlgElectricalToHdrDisplayLinear(inputRgbElectricalColor) + : inputRgbElectricalColor; + } + + // 4. HDR -> SDR (Tone-mapping + Gamut reduction) + // This path never enters the linear working space, so the BT.2408 ingress scale-up and egress + // scale-down cancel out and no scaling is applied. + highp vec3 sdrDisplayLinear = (inputColorTransfer == COLOR_TRANSFER_ST2084) + ? pqElectricalToBt709DisplayLinear(inputRgbElectricalColor) + : hlgElectricalToBt709DisplayLinear(inputRgbElectricalColor); + return (outputColorTransfer == COLOR_TRANSFER_LINEAR) + ? sdrDisplayLinear + : srgbOetf(sdrDisplayLinear); +} diff --git a/libraries/effect/src/main/res/raw/fragment_shader_alpha_scale_es2.glsl b/libraries/effect/src/main/res/raw/fragment_shader_alpha_scale_es2.glsl index 154a3e5ff97..6eca1f93c29 100644 --- a/libraries/effect/src/main/res/raw/fragment_shader_alpha_scale_es2.glsl +++ b/libraries/effect/src/main/res/raw/fragment_shader_alpha_scale_es2.glsl @@ -13,10 +13,10 @@ // See the License for the specific language governing permissions and // limitations under the License. -// ES 2 fragment shader that samples from a (non-external) texture with +// Fragment shader that samples from a (non-external) texture with // uTexSampler, and multiplies its alpha value by uAlphaScale. -precision mediump float; +precision highp float; uniform sampler2D uTexSampler; uniform float uAlphaScale; varying vec2 vTexSamplingCoord; diff --git a/libraries/effect/src/main/res/raw/fragment_shader_color_conversion_es3.glsl b/libraries/effect/src/main/res/raw/fragment_shader_color_conversion_es3.glsl new file mode 100644 index 00000000000..62d6334a3f8 --- /dev/null +++ b/libraries/effect/src/main/res/raw/fragment_shader_color_conversion_es3.glsl @@ -0,0 +1,75 @@ +// Copyright 2026 The Android Open Source Project +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// ES 3 fragment shader that converts frames from the working color space, which can be linear +// optical or electrical, to an electrical output color space. +// +// The output gamut is implied by the output transfer: BT.2020 for HLG, BT.709 for sRGB. Converting +// between SDR and HDR therefore also converts the gamut, and tone maps HDR to SDR with soft gamut +// compression and ITU-R BT.2446 Method C, or maps SDR to HDR via the BT.2408 diffuse white anchor. +// +// This is not a well-formed GLSL shader on its own and must be compiled together with +// color_conversions_es3.glsl (which provides #version, precision, and color conversion functions). + +uniform sampler2D uTexSampler; +// C.java#ColorSpace value of the input working color space. Only used to decode the input: the +// input gamut is not preserved, because the output gamut is implied by uOutputColorTransfer. +uniform int uInputColorGamut; +// C.java#ColorTransfer value of the input working color space. COLOR_TRANSFER_LINEAR denotes +// display-referred optical light: BT.2020 anchored on the BT.2408 diffuse white reference, or +// BT.709 anchored on SDR reference white. +uniform int uInputColorTransfer; +// C.java#ColorTransfer value of the electrical output. Only COLOR_TRANSFER_HLG and +// COLOR_TRANSFER_SRGB are supported; any other transfer outputs magenta. The output gamut is +// implied by the transfer: BT.2020 for HLG, BT.709 for sRGB. +uniform int uOutputColorTransfer; +in vec2 vTexSamplingCoord; +out vec4 outColor; + +// Decodes the input working color space to linear optical display light, in the input gamut. +highp vec3 toOpticalColor(highp vec3 inputColor) { + if (uInputColorTransfer == COLOR_TRANSFER_LINEAR) { + return inputColor; + } + if (isTransferHdr(uInputColorTransfer)) { + return hlgElectricalToHdrDisplayLinear(inputColor); + } + return srgbEotf(inputColor); +} + +void main() { + highp vec4 inputColor = texture(uTexSampler, vTexSamplingCoord); + highp vec3 opticalColor = toOpticalColor(inputColor.rgb); + highp vec3 electricalColor; + if (uOutputColorTransfer == COLOR_TRANSFER_HLG) { + // BT.2020 HLG output. SDR reference white maps onto the BT.2408 diffuse white anchor. + highp vec3 hdrDisplayLinear = isGamutHdr(uInputColorGamut) + ? opticalColor + : clamp(BT709_TO_BT2020 * opticalColor, 0.0, 1.0); + electricalColor = hdrDisplayLinearToHlgElectrical(hdrDisplayLinear); + } else if (uOutputColorTransfer == COLOR_TRANSFER_SRGB) { + // BT.709 sRGB output. HDR display light is scaled from its BT.2408 diffuse white anchor + // (1.0 = 203.1521 nits) to nits, which bt2020DisplayLinearNitsToBt709DisplayLinear expects. + highp vec3 sdrDisplayLinear = isGamutHdr(uInputColorGamut) + ? bt2020DisplayLinearNitsToBt709DisplayLinear( + opticalColor * HDR_DIFFUSE_WHITE_SCALE_DOWN * 1000.0) + : opticalColor; + electricalColor = srgbOetf(sdrDisplayLinear); + } else { + // Unsupported output transfer. Output magenta to make the error visible. + outColor = vec4(1.0, 0.0, 1.0, 1.0); + return; + } + outColor = vec4(electricalColor, inputColor.a); +} diff --git a/libraries/effect/src/main/res/raw/fragment_shader_transformation_external_yuv_color_conversion_es3.glsl b/libraries/effect/src/main/res/raw/fragment_shader_transformation_external_yuv_color_conversion_es3.glsl new file mode 100644 index 00000000000..652adda0b29 --- /dev/null +++ b/libraries/effect/src/main/res/raw/fragment_shader_transformation_external_yuv_color_conversion_es3.glsl @@ -0,0 +1,38 @@ +// Copyright 2026 The Android Open Source Project +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// ES 3 fragment shader that samples from an external YUV HDR texture and applies +// color transformations and tone mapping. +// +// This is not a well-formed GLSL shader on its own and must be compiled together +// with color_conversions_es3.glsl (which provides #version, precision, and color conversion functions). + +uniform __samplerExternal2DY2YEXT uTexSampler; +uniform int uIsInputColorRangeFull; +uniform int uOutputColorGamut; +uniform int uInputColorTransfer; +uniform int uOutputColorTransfer; +in vec2 vTexSamplingCoord; +out vec4 outColor; + +void main() { + highp vec3 srcYuv = texture(uTexSampler, vTexSamplingCoord).xyz; + highp vec3 inputRgb = yuvToRgb(srcYuv, uInputColorTransfer, uIsInputColorRangeFull); + highp vec3 processedRgb = processColor( + inputRgb, + uOutputColorGamut, + uInputColorTransfer, + uOutputColorTransfer); + outColor = vec4(processedRgb, 1.0); +} diff --git a/libraries/effect/src/main/res/raw/fragment_shader_transformation_external_yuv_es3.glsl b/libraries/effect/src/main/res/raw/fragment_shader_transformation_external_yuv_es3.glsl index d28264aeb9b..5a8dcba4e10 100644 --- a/libraries/effect/src/main/res/raw/fragment_shader_transformation_external_yuv_es3.glsl +++ b/libraries/effect/src/main/res/raw/fragment_shader_transformation_external_yuv_es3.glsl @@ -305,5 +305,8 @@ void main() { ? vec4(applyBt2020ToBt709Ootf(opticalColorBt2020), 1.0) : vec4(scaleHdrLuminance(opticalColorBt2020), 1.0); vec4 transformedColors = uRgbMatrix * opticalColor; - outColor = vec4(applyOetf(transformedColors.rgb), 1.0); + vec3 clampedColors = (uApplyHdrToSdrToneMapping == 1) + ? clamp(transformedColors.rgb, 0.0, 1.0) + : max(transformedColors.rgb, 0.0); + outColor = vec4(applyOetf(clampedColors), 1.0); } diff --git a/libraries/effect/src/main/res/raw/fragment_shader_transformation_hdr_internal_color_conversion_es3.glsl b/libraries/effect/src/main/res/raw/fragment_shader_transformation_hdr_internal_color_conversion_es3.glsl new file mode 100644 index 00000000000..07c3fdee1d6 --- /dev/null +++ b/libraries/effect/src/main/res/raw/fragment_shader_transformation_hdr_internal_color_conversion_es3.glsl @@ -0,0 +1,36 @@ +// Copyright 2026 The Android Open Source Project +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// ES 3 fragment shader that samples from an internal 2D HDR texture and applies +// color transformations and tone mapping. +// +// This is not a well-formed GLSL shader on its own and must be compiled together +// with color_conversions_es3.glsl (which provides #version, precision, and color conversion functions). + +uniform sampler2D uTexSampler; +uniform int uOutputColorGamut; +uniform int uInputColorTransfer; +uniform int uOutputColorTransfer; +in vec2 vTexSamplingCoord; +out vec4 outColor; + +void main() { + highp vec4 inputColor = texture(uTexSampler, vTexSamplingCoord); + highp vec3 processedRgb = processColor( + inputColor.rgb, + uOutputColorGamut, + uInputColorTransfer, + uOutputColorTransfer); + outColor = vec4(processedRgb, inputColor.a); +} diff --git a/libraries/effect/src/main/res/raw/fragment_shader_transformation_hdr_internal_es3.glsl b/libraries/effect/src/main/res/raw/fragment_shader_transformation_hdr_internal_es3.glsl index b50114187d1..f62291d4bd5 100644 --- a/libraries/effect/src/main/res/raw/fragment_shader_transformation_hdr_internal_es3.glsl +++ b/libraries/effect/src/main/res/raw/fragment_shader_transformation_hdr_internal_es3.glsl @@ -292,5 +292,8 @@ void main() { ? vec4(applyBt2020ToBt709Ootf(opticalColorBt2020), 1.0) : vec4(scaleHdrLuminance(opticalColorBt2020), 1.0); vec4 transformedColors = uRgbMatrix * opticalColor; - outColor = vec4(applyOetf(transformedColors.rgb), 1.0); + vec3 clampedColors = (uApplyHdrToSdrToneMapping == 1) + ? clamp(transformedColors.rgb, 0.0, 1.0) + : max(transformedColors.rgb, 0.0); + outColor = vec4(applyOetf(clampedColors), 1.0); } diff --git a/libraries/effect/src/main/res/raw/fragment_shader_transformation_sdr_external_color_conversion_es2.glsl b/libraries/effect/src/main/res/raw/fragment_shader_transformation_sdr_external_color_conversion_es2.glsl new file mode 100644 index 00000000000..a276c4f551f --- /dev/null +++ b/libraries/effect/src/main/res/raw/fragment_shader_transformation_sdr_external_color_conversion_es2.glsl @@ -0,0 +1,37 @@ +// Copyright 2026 The Android Open Source Project +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// ES 2 fragment shader that samples from an external SDR texture and applies +// color conversions and RGB matrix transformations. +// +// This is not a well-formed GLSL shader on its own and must be compiled together +// with color_conversions_es2.glsl (which provides #version, precision, and color conversion functions). + +uniform samplerExternalOES uTexSampler; +uniform mat4 uRgbMatrix; +uniform int uInputColorGamut; +uniform int uOutputColorGamut; +uniform int uOutputColorTransfer; +varying vec2 vTexSamplingCoord; + +void main() { + vec4 inputColor = texture2D(uTexSampler, vTexSamplingCoord); + vec3 processedRgb = processColor( + inputColor.rgb, + uInputColorGamut, + uOutputColorGamut, + uOutputColorTransfer); + vec4 transformedColors = uRgbMatrix * vec4(processedRgb, 1.0); + gl_FragColor = vec4(transformedColors.rgb, inputColor.a); +} diff --git a/libraries/effect/src/main/res/raw/fragment_shader_transformation_sdr_external_es2.glsl b/libraries/effect/src/main/res/raw/fragment_shader_transformation_sdr_external_es2.glsl index 01a6a422579..9bc8703806a 100644 --- a/libraries/effect/src/main/res/raw/fragment_shader_transformation_sdr_external_es2.glsl +++ b/libraries/effect/src/main/res/raw/fragment_shader_transformation_sdr_external_es2.glsl @@ -28,7 +28,8 @@ uniform samplerExternalOES uTexSampler; uniform mat4 uRgbMatrix; varying vec2 vTexSamplingCoord; // C.java#ColorTransfer value. -// Only COLOR_TRANSFER_LINEAR and COLOR_TRANSFER_SDR_VIDEO are allowed. +// Only COLOR_TRANSFER_LINEAR, COLOR_TRANSFER_SRGB, and COLOR_TRANSFER_SDR_VIDEO +// are allowed. uniform int uOutputColorTransfer; uniform int uSdrWorkingColorSpace; @@ -98,12 +99,14 @@ vec3 convertToWorkingColors(vec3 inputColor) { highp vec3 convertToOutputColors(highp vec3 workingColors) { // LINT.IfChange(color_transfer) const int COLOR_TRANSFER_LINEAR = 1; + const int COLOR_TRANSFER_SRGB = 2; const int COLOR_TRANSFER_SDR_VIDEO = 3; // LINT.ThenChange(../../../../../common/src/main/java/androidx/media3/common/C.java:color_transfer) if (uSdrWorkingColorSpace == WORKING_COLOR_SPACE_DEFAULT) { if (uOutputColorTransfer == COLOR_TRANSFER_LINEAR) { return smpte170mEotf(workingColors); - } else if (uOutputColorTransfer == COLOR_TRANSFER_SDR_VIDEO) { + } else if (uOutputColorTransfer == COLOR_TRANSFER_SDR_VIDEO || + uOutputColorTransfer == COLOR_TRANSFER_SRGB) { return workingColors; } else { return ERROR_COLOR_RED; diff --git a/libraries/effect/src/main/res/raw/fragment_shader_transformation_sdr_internal_color_conversion_es2.glsl b/libraries/effect/src/main/res/raw/fragment_shader_transformation_sdr_internal_color_conversion_es2.glsl new file mode 100644 index 00000000000..8d38aa7749f --- /dev/null +++ b/libraries/effect/src/main/res/raw/fragment_shader_transformation_sdr_internal_color_conversion_es2.glsl @@ -0,0 +1,37 @@ +// Copyright 2026 The Android Open Source Project +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// ES 2 fragment shader that samples from an internal 2D SDR texture and applies +// color conversions and RGB matrix transformations. +// +// This is not a well-formed GLSL shader on its own and must be compiled together +// with color_conversions_es2.glsl (which provides #version, precision, and color conversion functions). + +uniform sampler2D uTexSampler; +uniform mat4 uRgbMatrix; +uniform int uInputColorGamut; +uniform int uOutputColorGamut; +uniform int uOutputColorTransfer; +varying vec2 vTexSamplingCoord; + +void main() { + vec4 inputColor = texture2D(uTexSampler, vTexSamplingCoord); + vec3 processedRgb = processColor( + inputColor.rgb, + uInputColorGamut, + uOutputColorGamut, + uOutputColorTransfer); + vec4 transformedColors = uRgbMatrix * vec4(processedRgb, 1.0); + gl_FragColor = vec4(transformedColors.rgb, inputColor.a); +} diff --git a/libraries/effect/src/main/res/raw/fragment_shader_transformation_ultra_hdr_color_conversion_es3.glsl b/libraries/effect/src/main/res/raw/fragment_shader_transformation_ultra_hdr_color_conversion_es3.glsl new file mode 100644 index 00000000000..04952913577 --- /dev/null +++ b/libraries/effect/src/main/res/raw/fragment_shader_transformation_ultra_hdr_color_conversion_es3.glsl @@ -0,0 +1,107 @@ +// Copyright 2026 The Android Open Source Project +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +// ES 3 fragment shader that samples from an internal 2D SDR base texture and a +// 2D Ultra HDR gainmap texture, reconstructs the HDR image, and converts to the +// requested HDR output color transfer (BT.2020 linear or HLG). +// +// This is not a well-formed GLSL shader on its own and must be compiled together +// with color_conversions_es3.glsl (which provides #version, precision, and color +// conversion functions). + +uniform sampler2D uTexSampler; +uniform sampler2D uGainmapTexSampler; +uniform int uOutputColorTransfer; + +// Gainmap uniforms set by GainmapUtil.setGainmapUniforms (see android.graphics.Gainmap). +// 1 (GL_TRUE) if the gainmap texture is single-channel ALPHA_8 (stored in .a), 0 (GL_FALSE) if RGB. +uniform int uGainmapIsAlpha; +// 1 (GL_TRUE) if uGainmapGamma is (1.0, 1.0, 1.0) so gamma decoding can be skipped, 0 otherwise. +uniform int uNoGamma; +// 1 (GL_TRUE) if gamma, ratioMin, and ratioMax are identical across R, G, and B channels. +uniform int uSingleChannel; +// Natural logarithm of the minimum gainmap boost ratio per channel: ln(Gainmap.getRatioMin()). +uniform highp vec3 uLogRatioMin; +// Natural logarithm of the maximum gainmap boost ratio per channel: ln(Gainmap.getRatioMax()). +uniform highp vec3 uLogRatioMax; +// Offset added to linear SDR values before applying gain to avoid division by zero near black. +uniform highp vec3 uEpsilonSdr; +// Offset subtracted from reconstructed linear HDR values after applying gain. +uniform highp vec3 uEpsilonHdr; +// Gamma exponent applied to decode the normalized gainmap texture values: Gainmap.getGamma(). +uniform highp vec3 uGainmapGamma; +// Display HDR/SDR ratio at which the full gainmap boost is applied: +// Gainmap.getDisplayRatioForFullHdr(). +uniform highp float uDisplayRatioHdr; +// Minimum display HDR/SDR ratio at which gainmap boost begins: +// Gainmap.getMinDisplayRatioForHdrTransition(). +uniform highp float uDisplayRatioSdr; + +in vec2 vTexSamplingCoord; +out vec4 outColor; + +// Applies the UltraHDR gainmap to the SDR electrical color to reconstruct the linear HDR image. +// The resulting linear optical BT.709 display light has 1.0 anchored at 203.1521-nit diffuse white +// and highlights boosted up to HDR_DIFFUSE_WHITE_SCALE_UP (~4.9224 = 1,000 nits). +// References: +// - Android Gainmap specification: +// https://developer.android.com/reference/android/graphics/Gainmap#applying-a-gainmap-manually +// - ISO 21496-1 / libultrahdr gainmapmath. +highp vec3 applyGainmap(highp vec3 sdrElectricalColor, highp vec4 gainmapPixel) { + // 1. Linearize the electrical sRGB base image to optical BT.709 display light (1.0 = 203 nits). + highp vec3 sdrDisplayLinear = srgbEotf(sdrElectricalColor); + + // 2. Unpack the normalized [0, 1] gainmap value and decode its gamma curve if needed. + highp vec3 gainmapValue = (uGainmapIsAlpha == 1) ? vec3(gainmapPixel.a) : gainmapPixel.rgb; + if (uSingleChannel == 1) { + gainmapValue = vec3(gainmapValue.r); + } + if (uNoGamma != 1) { + gainmapValue = pow(max(gainmapValue, vec3(0.0)), uGainmapGamma); + } + + // 3. Interpolate the logarithmic gain factor between ln(ratioMin) and ln(ratioMax). + highp vec3 logBoost = mix(uLogRatioMin, uLogRatioMax, gainmapValue); + + // 4. Compute the display headroom weight in [0, 1] by mapping our target HDR/SDR ratio + // (1000 nits / 203 nits = HDR_DIFFUSE_WHITE_SCALE_UP) into [uDisplayRatioSdr, uDisplayRatioHdr]. + highp float headroomWeight = clamp( + (log(HDR_DIFFUSE_WHITE_SCALE_UP) - log(uDisplayRatioSdr)) / + max(log(uDisplayRatioHdr) - log(uDisplayRatioSdr), 1e-6), + 0.0, 1.0); + + // 5. Convert the weighted log boost back to a linear multiplier and scale the SDR base color. + highp vec3 linearBoost = exp(logBoost * headroomWeight); + highp vec3 boostedBt709DisplayLinear = + (sdrDisplayLinear + uEpsilonSdr) * linearBoost - uEpsilonHdr; + return max(boostedBt709DisplayLinear, vec3(0.0)); +} + +void main() { + highp vec4 inputColor = texture(uTexSampler, vTexSamplingCoord); + highp vec4 gainmapPixel = texture(uGainmapTexSampler, vTexSamplingCoord); + + highp vec3 boostedBt709DisplayLinear = applyGainmap(inputColor.rgb, gainmapPixel); + highp vec3 bt2020DisplayLinear = max(BT709_TO_BT2020 * boostedBt709DisplayLinear, vec3(0.0)); + + // This code path only outputs HDR (BT.2020 linear or HLG). When the target output is SDR, + // HardwareBufferToGlTextureConverter copies the base SDR image directly without gainmap + // application. + highp vec3 processedRgb = + (uOutputColorTransfer == COLOR_TRANSFER_LINEAR) + ? bt2020DisplayLinear + : hdrDisplayLinearToHlgElectrical(bt2020DisplayLinear); + + outColor = vec4(processedRgb, inputColor.a); +} diff --git a/libraries/effect/src/test/java/androidx/media3/effect/ColorConversionTest.java b/libraries/effect/src/test/java/androidx/media3/effect/ColorConversionTest.java new file mode 100644 index 00000000000..37fb1dbb041 --- /dev/null +++ b/libraries/effect/src/test/java/androidx/media3/effect/ColorConversionTest.java @@ -0,0 +1,164 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect; + +import static androidx.media3.effect.DefaultGlFrameProcessor.BT2020_HLG; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT2020_LINEAR; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT709_LINEAR; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT709_SRGB; +import static androidx.test.core.app.ApplicationProvider.getApplicationContext; +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; + +import android.content.Context; +import androidx.media3.common.C; +import androidx.media3.common.ColorInfo; +import androidx.media3.common.MetricsProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.util.concurrent.atomic.AtomicInteger; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link ColorConversion}. */ +@RunWith(AndroidJUnit4.class) +public final class ColorConversionTest { + + private final Context context = getApplicationContext(); + + @Test + public void create_withHdrHlg_setsOutputColorInfo() { + ColorConversion colorConversion = ColorConversion.create(BT2020_HLG); + + assertThat(colorConversion.getOutputColorInfo()).isEqualTo(BT2020_HLG); + } + + @Test + public void create_withSdrBt709Limited_throwsIllegalArgumentException() { + // COLOR_TRANSFER_SDR is not accepted, so that there is one way to request a BT.709 output. + assertThrows( + IllegalArgumentException.class, () -> ColorConversion.create(ColorInfo.SDR_BT709_LIMITED)); + } + + @Test + public void create_withSdrSrgb_setsOutputColorInfo() { + ColorConversion colorConversion = ColorConversion.create(BT709_SRGB); + + assertThat(colorConversion.getOutputColorInfo()).isEqualTo(BT709_SRGB); + } + + @Test + public void create_withSrgbBt709Full_setsOutputColorInfo() { + // The color range does not affect the conversion, so a full range variant of a supported color + // space is accepted and propagated unchanged. + ColorConversion colorConversion = ColorConversion.create(ColorInfo.SRGB_BT709_FULL); + + assertThat(colorConversion.getOutputColorInfo()).isEqualTo(ColorInfo.SRGB_BT709_FULL); + } + + @Test + public void create_withHdrHlgLimitedRange_setsOutputColorInfo() { + ColorInfo hlgLimitedRange = BT2020_HLG.buildUpon().setColorRange(C.COLOR_RANGE_LIMITED).build(); + + ColorConversion colorConversion = ColorConversion.create(hlgLimitedRange); + + assertThat(colorConversion.getOutputColorInfo()).isEqualTo(hlgLimitedRange); + } + + @Test + public void create_withHdrPq_throwsIllegalArgumentException() { + ColorInfo hdrPq = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorTransfer(C.COLOR_TRANSFER_ST2084) + .build(); + + assertThrows(IllegalArgumentException.class, () -> ColorConversion.create(hdrPq)); + } + + @Test + public void create_withHdrLinear_throwsIllegalArgumentException() { + assertThrows(IllegalArgumentException.class, () -> ColorConversion.create(BT2020_LINEAR)); + } + + @Test + public void create_withSdrLinear_throwsIllegalArgumentException() { + assertThrows(IllegalArgumentException.class, () -> ColorConversion.create(BT709_LINEAR)); + } + + @Test + public void create_withHlgTransferAndBt709Gamut_throwsIllegalArgumentException() { + // The shader derives the output gamut from the output transfer, so a BT.709 HLG output would be + // mislabelled rather than converted. + ColorInfo bt709Hlg = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT709) + .setColorTransfer(C.COLOR_TRANSFER_HLG) + .build(); + + assertThrows(IllegalArgumentException.class, () -> ColorConversion.create(bt709Hlg)); + } + + @Test + public void create_withSdrTransferAndBt2020Gamut_throwsIllegalArgumentException() { + ColorInfo bt2020Sdr = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorTransfer(C.COLOR_TRANSFER_SDR) + .build(); + + assertThrows(IllegalArgumentException.class, () -> ColorConversion.create(bt2020Sdr)); + } + + @Test + public void create_withUnsetColorTransfer_throwsIllegalArgumentException() { + ColorInfo unsetTransfer = new ColorInfo.Builder().setColorSpace(C.COLOR_SPACE_BT709).build(); + + assertThrows(IllegalArgumentException.class, () -> ColorConversion.create(unsetTransfer)); + } + + @Test + public void create_withNullOutputColorInfo_throwsNullPointerException() { + assertThrows(NullPointerException.class, () -> ColorConversion.create(null)); + } + + @Test + public void toGlShaderProgram_withUseHdr_throwsUnsupportedOperationException() { + ColorConversion colorConversion = ColorConversion.create(BT2020_HLG); + + assertThrows( + UnsupportedOperationException.class, + () -> colorConversion.toGlShaderProgram(context, /* useHdr= */ true)); + } + + @Test + public void populateMetrics_withConsumer_setsColorEffectCategory() { + ColorConversion colorConversion = ColorConversion.create(BT2020_HLG); + AtomicInteger categoryRef = new AtomicInteger(); + + colorConversion.populateMetrics( + new MetricsProvider.MetricConsumer() { + @Override + public void setCategory(int category) { + categoryRef.set(category); + } + + @Override + public void addMetric(int metricKey, long metricValue) {} + }); + + assertThat(categoryRef.get()).isEqualTo(MetricsProvider.MetricConsumer.CATEGORY_EFFECT_COLOR); + } +} diff --git a/libraries/effect/src/test/java/androidx/media3/effect/DefaultGlFrameProcessorTest.java b/libraries/effect/src/test/java/androidx/media3/effect/DefaultGlFrameProcessorTest.java index ec1b6e4a2fe..0a0432f1fe1 100644 --- a/libraries/effect/src/test/java/androidx/media3/effect/DefaultGlFrameProcessorTest.java +++ b/libraries/effect/src/test/java/androidx/media3/effect/DefaultGlFrameProcessorTest.java @@ -19,34 +19,52 @@ import static androidx.media3.effect.DefaultGlFrameProcessor.KEY_COMPOSITION_SEQUENCE_INDEX; import static androidx.media3.effect.DefaultGlFrameProcessor.KEY_COMPOSITOR_SETTINGS; import static androidx.media3.effect.DefaultGlFrameProcessor.KEY_ITEM_EFFECTS; +import static androidx.media3.effect.FrameProcessorUtils.shutdownGlExecutorService; import static androidx.test.core.app.ApplicationProvider.getApplicationContext; +import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.truth.Truth.assertThat; import static com.google.common.util.concurrent.MoreExecutors.directExecutor; import static com.google.common.util.concurrent.MoreExecutors.listeningDecorator; import static java.util.concurrent.TimeUnit.MILLISECONDS; import static org.junit.Assert.assertThrows; +import static org.mockito.Mockito.mock; import android.content.Context; +import android.opengl.EGL14; +import android.opengl.EGLContext; +import android.opengl.EGLDisplay; +import android.opengl.EGLSurface; import androidx.annotation.Nullable; +import androidx.media3.common.C; +import androidx.media3.common.ColorInfo; +import androidx.media3.common.Format; +import androidx.media3.common.GlObjectsProvider; +import androidx.media3.common.GlTextureInfo; +import androidx.media3.common.MimeTypes; import androidx.media3.common.VideoCompositorSettings; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.ConditionVariable; +import androidx.media3.common.util.Consumer; +import androidx.media3.common.util.GlUtil.GlException; import androidx.media3.common.util.Util; import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.DefaultImagePlanesFrame; +import androidx.media3.common.video.DefaultImagePlanesFrame.DefaultPlane; import androidx.media3.common.video.Frame; import androidx.media3.common.video.FrameProcessor; import androidx.media3.common.video.SyncFenceWrapper; import androidx.media3.effect.GlFrameProcessorTestUtil.FakeCompositorGlProgram; +import androidx.media3.effect.GlFrameProcessorTestUtil.FakeFrameToGlTextureConverter; +import androidx.media3.effect.GlFrameProcessorTestUtil.FakeGlObjectsProvider; import androidx.media3.effect.GlFrameProcessorTestUtil.FakeGlShaderProgram; import androidx.media3.effect.GlFrameProcessorTestUtil.FakeGlTextureFrameConsumer; -import androidx.media3.effect.GlFrameProcessorTestUtil.FakeHardwareBufferConverter; import androidx.media3.effect.GlFrameProcessorTestUtil.FakeHardwareBufferFrame; import androidx.media3.effect.GlFrameProcessorTestUtil.NoOpFrameWriter; import androidx.test.ext.junit.runners.AndroidJUnit4; -import androidx.test.filters.SdkSuppress; import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableMap; import com.google.common.util.concurrent.ListeningExecutorService; +import java.nio.ByteBuffer; import java.util.ArrayList; import java.util.List; import java.util.Queue; @@ -56,16 +74,18 @@ import java.util.concurrent.ExecutionException; import java.util.concurrent.Executor; import java.util.concurrent.TimeoutException; +import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicInteger; +import java.util.concurrent.atomic.AtomicReference; import org.junit.After; import org.junit.Before; -import org.junit.Ignore; import org.junit.Test; import org.junit.runner.RunWith; +import org.robolectric.annotation.Config; /** Unit tests for {@link DefaultGlFrameProcessor} using pluggable non-OpenGL fakes. */ @RunWith(AndroidJUnit4.class) -@SdkSuppress(minSdkVersion = 29) +@Config(minSdk = 29) public final class DefaultGlFrameProcessorTest { private static final int COMPOSITOR_CAPACITY = 1; @@ -75,7 +95,7 @@ public final class DefaultGlFrameProcessorTest { private Queue queuedFrameProcessedTasks; private Executor testExecutor; - private FakeHardwareBufferConverter fakeHardwareBufferConverter; + private FakeFrameToGlTextureConverter fakeFrameToGlTextureConverter; private AtomicInteger glTextureFramesReleased; private FakeGlShaderProgram fakeGlShaderProgram; private FakeGlTextureFrameConsumer fakeFrameWriterGlTextureFrameConsumer; @@ -88,8 +108,8 @@ public final class DefaultGlFrameProcessorTest { public void setUp() { frameWriter = new NoOpFrameWriter(); glTextureFramesReleased = new AtomicInteger(); - fakeHardwareBufferConverter = - new FakeHardwareBufferConverter(glTextureFramesReleased::incrementAndGet); + fakeFrameToGlTextureConverter = + new FakeFrameToGlTextureConverter(glTextureFramesReleased::incrementAndGet); fakeGlShaderProgram = new FakeGlShaderProgram(); fakeFrameWriterGlTextureFrameConsumer = new FakeGlTextureFrameConsumer(frameWriter); fakeEffect = (context, useHdr) -> fakeGlShaderProgram; @@ -123,7 +143,7 @@ public void tearDown() { processor.close(); } if (glExecutorService != null) { - FrameProcessorUtils.shutdownGlExecutorService(glExecutorService); + shutdownGlExecutorService(glExecutorService); } } @@ -135,7 +155,7 @@ public void queue_singleSequenceWithNonDefaultSequenceIndex_propagatesToAllPipel assertThat(processor.queue(ImmutableList.of(new AsyncFrame(frame, /* acquireFence= */ null)))) .isTrue(); waitUntilGlThreadFinishes(); - assertThat(fakeHardwareBufferConverter.framesReceived).isEqualTo(1); + assertThat(fakeFrameToGlTextureConverter.framesReceived).isEqualTo(1); assertThat(fakeGlShaderProgram.framesReceived).isEqualTo(1); assertThat(fakeFrameWriterGlTextureFrameConsumer.framesReceived).isEqualTo(1); assertThat(frameWriter.queuedFrames).isEqualTo(1); @@ -148,6 +168,209 @@ public void queue_singleSequenceWithNonDefaultSequenceIndex_propagatesToAllPipel assertThat(frameWriter.signalEndOfStreamCalled).isTrue(); } + @Test + public void constructor_defersComponentInitializationToQueue() throws Exception { + AtomicBoolean componentCreated = new AtomicBoolean(false); + DefaultGlFrameProcessor.Factory customFactory = + createCustomFactory(outputColorInfo -> componentCreated.set(true)); + + try (DefaultGlFrameProcessor customProcessor = + customFactory.create( + frameWriter, + glExecutorService, + new FrameProcessor.Listener() { + @Override + public void onWakeup() {} + + @Override + public void onError(VideoFrameProcessingException exception) { + throw new AssertionError(exception); + } + + @Override + public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper fence) {} + })) { + + // Assert that the closure has not been executed yet. + assertThat(componentCreated.get()).isFalse(); + + // Call queue. + Frame frame = createFakeHardwareBufferFrame(/* sequenceIndex= */ 0, fakeEffect); + assertThat( + customProcessor.queue( + ImmutableList.of(new AsyncFrame(frame, /* acquireFence= */ null)))) + .isTrue(); + waitUntilGlThreadFinishes(); + + // Now the components should be lazily instantiated! + assertThat(componentCreated.get()).isTrue(); + } + } + + @Test + public void queue_withHdrHlgFrameWorkingColorspaceUnset_resolvesWorkingColorspaceToHdrLinear() + throws Exception { + ColorInfo hdrColorInfo = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorTransfer(C.COLOR_TRANSFER_HLG) + .setColorRange(C.COLOR_RANGE_LIMITED) + .build(); + Format format = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H265) + .setColorInfo(hdrColorInfo) + .build(); + ColorInfo actualColorInfo = + queueFrameAndGetColorInfo(format, /* forceUnsupportedFormat= */ false); + + assertThat(actualColorInfo).isEqualTo(DefaultGlFrameProcessor.BT2020_LINEAR); + } + + @Test + public void queue_withHdrPqFrameWorkingColorspaceUnset_resolvesWorkingColorspaceToHdrLinear() + throws Exception { + ColorInfo hdrColorInfo = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorTransfer(C.COLOR_TRANSFER_ST2084) + .setColorRange(C.COLOR_RANGE_LIMITED) + .build(); + Format format = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H265) + .setColorInfo(hdrColorInfo) + .build(); + ColorInfo actualColorInfo = + queueFrameAndGetColorInfo(format, /* forceUnsupportedFormat= */ false); + + assertThat(actualColorInfo).isEqualTo(DefaultGlFrameProcessor.BT2020_LINEAR); + } + + @Test + public void queue_withHdrVideoFrameAndEs3ContextCreationFails_throwsIllegalStateException() + throws Exception { + ColorInfo hdrColorInfo = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorTransfer(C.COLOR_TRANSFER_HLG) + .setColorRange(C.COLOR_RANGE_LIMITED) + .build(); + Format format = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H265) + .setColorInfo(hdrColorInfo) + .build(); + TestGlObjectsProvider glObjectsProvider = new TestGlObjectsProvider(/* failVersion3= */ true); + + AtomicReference expectedException = new AtomicReference<>(); + ColorInfo unused = + queueFrameAndGetColorInfo( + format, /* forceUnsupportedFormat= */ false, glObjectsProvider, expectedException); + waitUntilGlThreadFinishes(); + assertThat(expectedException.get()).hasCauseThat().isInstanceOf(IllegalStateException.class); + } + + @Test + public void queue_whenSetupOpenGlFails_notifiesError() throws Exception { + ColorInfo hdrColorInfo = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorTransfer(C.COLOR_TRANSFER_HLG) + .setColorRange(C.COLOR_RANGE_LIMITED) + .build(); + Format format = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H265) + .setColorInfo(hdrColorInfo) + .build(); + ImmutableMap metadata = + ImmutableMap.of( + KEY_COMPOSITION_SEQUENCE_INDEX, + 0, + KEY_COMPOSITOR_SETTINGS, + VideoCompositorSettings.DEFAULT, + DefaultGlFrameProcessor.KEY_COMPOSITION_EFFECTS, + ImmutableList.of()); + + Frame frame = new FakeHardwareBufferFrame(format, metadata); + TestGlObjectsProvider glObjectsProvider = + new TestGlObjectsProvider(/* failVersion3= */ false, /* failSurfaceCreation= */ true); + + AtomicReference thrownException = new AtomicReference<>(); + DefaultGlFrameProcessor processor = + createDefaultGlFrameProcessorFactoryBuilderWithTestGlResources(glObjectsProvider) + .build() + .create( + frameWriter, + glExecutorService, + new FrameProcessor.Listener() { + @Override + public void onWakeup() {} + + @Override + public void onError(VideoFrameProcessingException exception) { + thrownException.set(exception); + } + + @Override + public void onFrameProcessed( + Frame frame, @Nullable SyncFenceWrapper onCompleteFence) {} + }); + + assertThat(processor.queue(ImmutableList.of(new AsyncFrame(frame, /* acquireFence= */ null)))) + .isTrue(); + waitUntilGlThreadFinishes(); + processor.close(); + waitUntilGlThreadFinishes(); + + assertThat(thrownException.get()).isInstanceOf(VideoFrameProcessingException.class); + // Unwrap the VideoFrameProcessingException and validate the diagnostic message + // originating directly from the GlException stubbed in TestGlObjectsProvider. + assertThat(thrownException.get()) + .hasCauseThat() + .hasMessageThat() + .contains("Test Surface Creation failed"); + } + + @Test + public void queue_withSdrFrameWorkingColorspaceUnset_resolvesWorkingColorspaceToSdrSrgb() + throws Exception { + ColorInfo sdrColorInfo = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT709) + .setColorTransfer(C.COLOR_TRANSFER_SDR) + .setColorRange(C.COLOR_RANGE_LIMITED) + .build(); + Format format = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H264) + .setColorInfo(sdrColorInfo) + .build(); + ColorInfo actualColorInfo = + queueFrameAndGetColorInfo(format, /* forceUnsupportedFormat= */ false); + + assertThat(actualColorInfo) + .isEqualTo( + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT709) + .setColorTransfer(C.COLOR_TRANSFER_SRGB) + .build()); + } + + @Test + public void queue_withJpegRAndSrgbTransfer_infersHdrLinear() throws Exception { + Format format = + new Format.Builder() + .setSampleMimeType(MimeTypes.IMAGE_JPEG_R) + .setColorInfo(ColorInfo.SRGB_BT709_FULL) // SRGB metadata typical of JPEG input + .build(); + ColorInfo actualColorInfo = + queueFrameAndGetColorInfo(format, /* forceUnsupportedFormat= */ false); + + assertThat(actualColorInfo).isEqualTo(DefaultGlFrameProcessor.BT2020_LINEAR); + } + @Test public void queue_hardwareBufferFrame_propagatesToAllPipelineComponents() throws Exception { Frame frame = createFakeHardwareBufferFrame(/* sequenceIndex= */ 0, fakeEffect); @@ -155,7 +378,7 @@ public void queue_hardwareBufferFrame_propagatesToAllPipelineComponents() throws assertThat(processor.queue(ImmutableList.of(new AsyncFrame(frame, /* acquireFence= */ null)))) .isTrue(); waitUntilGlThreadFinishes(); - assertThat(fakeHardwareBufferConverter.framesReceived).isEqualTo(1); + assertThat(fakeFrameToGlTextureConverter.framesReceived).isEqualTo(1); assertThat(fakeGlShaderProgram.framesReceived).isEqualTo(1); assertThat(fakeFrameWriterGlTextureFrameConsumer.framesReceived).isEqualTo(1); assertThat(frameWriter.queuedFrames).isEqualTo(1); @@ -168,6 +391,21 @@ public void queue_hardwareBufferFrame_propagatesToAllPipelineComponents() throws assertThat(frameWriter.signalEndOfStreamCalled).isTrue(); } + @Test + public void queue_imagePlanesFrame_propagatesToAllPipelineComponents() throws Exception { + Frame frame = createFakeImagePlanesFrame(fakeEffect); + + boolean queued = + processor.queue(ImmutableList.of(new AsyncFrame(frame, /* acquireFence= */ null))); + waitUntilGlThreadFinishes(); + + assertThat(queued).isTrue(); + assertThat(fakeFrameToGlTextureConverter.framesReceived).isEqualTo(1); + assertThat(fakeGlShaderProgram.framesReceived).isEqualTo(1); + assertThat(fakeFrameWriterGlTextureFrameConsumer.framesReceived).isEqualTo(1); + assertThat(frameWriter.queuedFrames).isEqualTo(1); + } + @Test public void queue_hardwareBufferFrameWithAdditionalMetadata_forwardsMetadata() throws Exception { FakeGlShaderProgram fakeCompositionEffectShaderProgram = new FakeGlShaderProgram(); @@ -257,7 +495,7 @@ public void onFrameProcessed( waitUntilGlThreadFinishes(); executeAllQueuedTasks(queuedFrameProcessedTasks); assertThat(completedFrames).containsExactly(frame0, frame1, frame2); - assertThat(fakeHardwareBufferConverter.framesReceived).isEqualTo(3); + assertThat(fakeFrameToGlTextureConverter.framesReceived).isEqualTo(3); assertThat(fakeGlShaderProgram.framesReceived).isEqualTo(1); assertThat(fakeFrameWriterGlTextureFrameConsumer.framesReceived).isEqualTo(1); assertThat(frameWriter.queuedFrames).isEqualTo(1); @@ -309,7 +547,7 @@ public void onFrameProcessed( waitUntilGlThreadFinishes(); executeAllQueuedTasks(queuedFrameProcessedTasks); assertThat(completedFrames).containsExactly(frame0, frame1, frame2); - assertThat(fakeHardwareBufferConverter.framesReceived).isEqualTo(3); + assertThat(fakeFrameToGlTextureConverter.framesReceived).isEqualTo(3); // fakeGlShaderProgram is wired to frame 0. assertThat(fakeGlShaderProgram.framesReceived).isEqualTo(1); assertThat(fakeFrameWriterGlTextureFrameConsumer.framesReceived).isEqualTo(1); @@ -449,7 +687,7 @@ public void queue_singleSequenceFailedToQueueConvertedGlTextureFrame_retainsFram assertThat(processor.queue(ImmutableList.of(new AsyncFrame(frame3, /* acquireFence= */ null)))) .isFalse(); waitUntilGlThreadFinishes(); - assertThat(fakeHardwareBufferConverter.framesReceived).isEqualTo(2); + assertThat(fakeFrameToGlTextureConverter.framesReceived).isEqualTo(2); assertThat(glTextureFramesReleased.get()).isEqualTo(0); assertThat(fakeFrameWriterGlTextureFrameConsumer.framesReceived).isEqualTo(0); @@ -473,7 +711,7 @@ public void queue_multiSequenceFailedToQueueConvertedGlTextureFrame_retainsFrame .isTrue(); waitUntilGlThreadFinishes(); - assertThat(fakeHardwareBufferConverter.framesReceived).isEqualTo(2); + assertThat(fakeFrameToGlTextureConverter.framesReceived).isEqualTo(2); assertThat(glTextureFramesReleased.get()).isEqualTo(1); processor.close(); @@ -515,7 +753,7 @@ public void queue_singleSequencePostProcessorChainRejectsInput_retainsFramesForR .isFalse(); // Rejected by frameAggregator waitUntilGlThreadFinishes(); - assertThat(fakeHardwareBufferConverter.framesReceived).isEqualTo(3); + assertThat(fakeFrameToGlTextureConverter.framesReceived).isEqualTo(3); assertThat(glTextureFramesReleased.get()).isEqualTo(1); processor.close(); @@ -523,7 +761,6 @@ public void queue_singleSequencePostProcessorChainRejectsInput_retainsFramesForR assertThat(glTextureFramesReleased.get()).isEqualTo(3); } - @Ignore // TODO: b/529360695 - Re-enable when no longer flaky. @Test public void queue_withFrameWriterRejectingInput_buffersAndRetriesCorrectly() throws Exception { // batch A @@ -601,10 +838,12 @@ public void onFrameProcessed( waitUntilGlThreadFinishes(); // Now unblock downstream and trigger wakeup retry loop + fakeFrameWriterGlTextureFrameConsumer.setExpectedFrameCount(3); fakeFrameWriterGlTextureFrameConsumer.shouldAcceptIncomingFrames = true; fakeFrameWriterGlTextureFrameConsumer.triggerWakeup(); - waitUntilGlThreadFinishes(); assertThat(completedFramesLatch.await(SYNC_TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(fakeFrameWriterGlTextureFrameConsumer.awaitFramesReceived(SYNC_TIMEOUT_MS)) + .isTrue(); assertThat(fakeFrameWriterGlTextureFrameConsumer.framesReceived).isEqualTo(3); // frame1C and frame2C bypass the preprocessor chain (no effects) and were initially rejected @@ -908,12 +1147,15 @@ public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper onCompleteF .isFalse(); // Downstream is ready to accept frames now. + fakeFrameWriterGlTextureFrameConsumer.setExpectedFrameCount(2); fakeFrameWriterGlTextureFrameConsumer.shouldAcceptIncomingFrames = true; fakeFrameWriterGlTextureFrameConsumer.triggerWakeup(); // Verify wakeup is notified. assertThat(latch.await(SYNC_TIMEOUT_MS, MILLISECONDS)).isTrue(); assertThat(wakeupNotified).containsExactly(true); + assertThat(fakeFrameWriterGlTextureFrameConsumer.awaitFramesReceived(SYNC_TIMEOUT_MS)) + .isTrue(); // Verify that the first frame (frame0) and second frame (frame1) were accepted. assertThat(fakeFrameWriterGlTextureFrameConsumer.framesReceived).isEqualTo(2); @@ -1141,47 +1383,329 @@ public void queue_whenEosSignaledWhileFramePending_defersEos() throws Exception public void close_withPendingFramesInShader_releasesFrames() throws Exception { fakeGlShaderProgram.delayProcessing = true; Frame frame = createFakeHardwareBufferFrame(/* sequenceIndex= */ 0, fakeEffect); - assertThat(processor.queue(ImmutableList.of(new AsyncFrame(frame, /* acquireFence= */ null)))) .isTrue(); waitUntilGlThreadFinishes(); - - // Frame is in shader, not released yet. assertThat(glTextureFramesReleased.get()).isEqualTo(0); processor.close(); - waitUntilGlThreadFinishes(); - // Frame should be released on close. + // Frame should be released synchronously by close(). assertThat(glTextureFramesReleased.get()).isEqualTo(1); } + @Test + public void close_calledTwice_doesNotThrow() throws Exception { + DefaultGlFrameProcessor customProcessor = + createDefaultGlFrameProcessorFactoryBuilderWithDefaultGlResources() + .build() + .create(frameWriter, directExecutor(), createFailOnErrorListener()); + Frame frame = createFakeHardwareBufferFrame(/* sequenceIndex= */ 0, fakeEffect); + fakeFrameWriterGlTextureFrameConsumer.setExpectedFrameCount(1); + + assertThat( + customProcessor.queue( + ImmutableList.of(new AsyncFrame(frame, /* acquireFence= */ null)))) + .isTrue(); + assertThat(fakeFrameWriterGlTextureFrameConsumer.awaitFramesReceived(SYNC_TIMEOUT_MS)).isTrue(); + + customProcessor.close(); + // Call close() again on purpose, to check it's a no-op after the GL executor is shut down. + customProcessor.close(); + } + + @Test + public void close_withGlObjectsProviderAndExecutorServiceSet_doesNotReleaseThem() + throws Exception { + TestGlObjectsProvider glObjectsProvider = new TestGlObjectsProvider(/* failVersion3= */ false); + DefaultGlFrameProcessor customProcessor = + createDefaultGlFrameProcessorFactoryBuilderWithTestGlResources(glObjectsProvider) + .build() + .create(frameWriter, directExecutor(), createFailOnErrorListener()); + Frame frame = createFakeHardwareBufferFrame(/* sequenceIndex= */ 0, fakeEffect); + fakeFrameWriterGlTextureFrameConsumer.setExpectedFrameCount(1); + + assertThat( + customProcessor.queue( + ImmutableList.of(new AsyncFrame(frame, /* acquireFence= */ null)))) + .isTrue(); + + assertThat(fakeFrameWriterGlTextureFrameConsumer.awaitFramesReceived(SYNC_TIMEOUT_MS)).isTrue(); + + customProcessor.close(); + + assertThat(glObjectsProvider.releaseCalled.get()).isFalse(); + assertThat(glExecutorService.isShutdown()).isFalse(); + } + + @Test + public void close_withDefaultGlResources_terminatesGlThread() throws Exception { + DefaultGlFrameProcessor customProcessor = + createDefaultGlFrameProcessorFactoryBuilderWithDefaultGlResources() + .build() + .create(frameWriter, directExecutor(), createFailOnErrorListener()); + + customProcessor.close(); + Thread glThread = checkNotNull(fakeFrameWriterGlTextureFrameConsumer.closeThread); + glThread.join(SYNC_TIMEOUT_MS); + + assertThat(glThread.isAlive()).isFalse(); + } + + @Test + public void close_withDefaultGlResources_wakeupFromDownstreamAfterCloseDoesNotThrow() + throws Exception { + fakeFrameWriterGlTextureFrameConsumer.shouldAcceptIncomingFrames = false; + DefaultGlFrameProcessor customProcessor = + createDefaultGlFrameProcessorFactoryBuilderWithDefaultGlResources() + .build() + .create(frameWriter, directExecutor(), createFailOnErrorListener()); + Frame frame = createFakeHardwareBufferFrame(/* sequenceIndex= */ 0, fakeEffect); + + assertThat( + customProcessor.queue( + ImmutableList.of(new AsyncFrame(frame, /* acquireFence= */ null)))) + .isTrue(); + + assertThat( + fakeFrameWriterGlTextureFrameConsumer.wakeupListenerRegistered.block(SYNC_TIMEOUT_MS)) + .isTrue(); + customProcessor.close(); + + // Simulates the downstream consumer posting a wakeup to the GL executor, which is shut down. + fakeFrameWriterGlTextureFrameConsumer.triggerWakeup(); + } + + @Test + public void create_multipleProcessorsSequentiallyWithDefaultGlResources_eachProcessesFrame() + throws Exception { + DefaultGlFrameProcessor.Factory factory = + createDefaultGlFrameProcessorFactoryBuilderWithDefaultGlResources().build(); + fakeFrameWriterGlTextureFrameConsumer.setExpectedFrameCount(1); + DefaultGlFrameProcessor processor1 = + factory.create(frameWriter, directExecutor(), createFailOnErrorListener()); + Frame frame1 = createFakeHardwareBufferFrame(/* sequenceIndex= */ 0, fakeEffect); + + assertThat(processor1.queue(ImmutableList.of(new AsyncFrame(frame1, /* acquireFence= */ null)))) + .isTrue(); + assertThat(fakeFrameWriterGlTextureFrameConsumer.awaitFramesReceived(SYNC_TIMEOUT_MS)).isTrue(); + + processor1.close(); + + fakeFrameWriterGlTextureFrameConsumer.setExpectedFrameCount(1); + DefaultGlFrameProcessor processor2 = + factory.create(frameWriter, directExecutor(), createFailOnErrorListener()); + Frame frame2 = createFakeHardwareBufferFrame(/* sequenceIndex= */ 0, fakeEffect); + + assertThat(processor2.queue(ImmutableList.of(new AsyncFrame(frame2, /* acquireFence= */ null)))) + .isTrue(); + assertThat(fakeFrameWriterGlTextureFrameConsumer.awaitFramesReceived(SYNC_TIMEOUT_MS)).isTrue(); + + processor2.close(); + + assertThat(frameWriter.queuedFrames).isEqualTo(2); + } + + private DefaultGlFrameProcessor.Factory.Builder + createDefaultGlFrameProcessorFactoryBuilderWithTestGlResources( + GlObjectsProvider glObjectsProvider) { + return createDefaultGlFrameProcessorFactoryBuilderWithDefaultGlResources() + .setGlObjectsProviderAndExecutorService(glObjectsProvider, glExecutorService); + } + + private DefaultGlFrameProcessor.Factory.Builder + createDefaultGlFrameProcessorFactoryBuilderWithDefaultGlResources() { + return new DefaultGlFrameProcessor.Factory.Builder( + context, /* hardwareBufferJniWrapper= */ mock(HardwareBufferJniWrapper.class)) + .setFrameToGlTextureConverterFactory( + (outputColorInfo, errorConsumer) -> fakeFrameToGlTextureConverter) + .setFrameWriterGlTextureFrameConsumer(fakeFrameWriterGlTextureFrameConsumer) + .setGlTextureFrameCompositorFactory( + new DefaultGlTextureFrameCompositor.Factory( + /* compositorGlProgramFactory= */ FakeCompositorGlProgram::new, + /* texturePoolFactory= */ workingColorSpace -> + new TexturePool( + /* textureAllocator= */ (width, height, useHighPrecisionColorComponents) -> + 100, + /* useHighPrecisionColorComponents= */ false, + /* capacity= */ COMPOSITOR_CAPACITY))) + .setAssumeSurfacelessContextExtensionSupported(true); + } + + private static FrameProcessor.Listener createFailOnErrorListener() { + return new FrameProcessor.Listener() { + @Override + public void onWakeup() {} + + @Override + public void onError(VideoFrameProcessingException exception) { + throw new AssertionError(exception); + } + + @Override + public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper fence) {} + }; + } + private DefaultGlFrameProcessor.Factory createDefaultGlFrameProcessorFactory() { - return new DefaultGlFrameProcessor.Factory( - context, - new GlFrameProcessorTestUtil.FakeGlObjectsProvider(), - glExecutorService, - fakeHardwareBufferConverter, - fakeFrameWriterGlTextureFrameConsumer, - new FakeCompositorGlProgram(), - new TexturePool( - /* textureAllocator= */ (width, height, useHighPrecisionColorComponents) -> 100, - /* useHighPrecisionColorComponents= */ false, - /* capacity= */ COMPOSITOR_CAPACITY)); + return createDefaultGlFrameProcessorFactoryBuilderWithTestGlResources( + new FakeGlObjectsProvider()) + .build(); } - private static Frame createFakeHardwareBufferFrame( + private DefaultGlFrameProcessor.Factory createCustomFactory( + Consumer colorInfoConsumer) { + return createDefaultGlFrameProcessorFactoryBuilderWithTestGlResources( + new FakeGlObjectsProvider()) + .setFrameToGlTextureConverterFactory( + (outputColorInfo, errorConsumer) -> { + colorInfoConsumer.accept(outputColorInfo); + return fakeFrameToGlTextureConverter; + }) + .build(); + } + + private static ImmutableMap createFrameMetadata( int sequenceIndex, @Nullable GlEffect itemEffect) { - ImmutableMap.Builder metadataBuilder = new ImmutableMap.Builder<>(); - metadataBuilder + return ImmutableMap.builder() .put(KEY_COMPOSITION_SEQUENCE_INDEX, sequenceIndex) .put(KEY_COMPOSITOR_SETTINGS, VideoCompositorSettings.DEFAULT) - .put(DefaultGlFrameProcessor.KEY_COMPOSITION_EFFECTS, ImmutableList.of()); - if (itemEffect != null) { - metadataBuilder.put(KEY_ITEM_EFFECTS, ImmutableList.of(itemEffect)); - } else { - metadataBuilder.put(KEY_ITEM_EFFECTS, ImmutableList.of()); + .put(KEY_COMPOSITION_EFFECTS, ImmutableList.of()) + .put( + KEY_ITEM_EFFECTS, + itemEffect == null ? ImmutableList.of() : ImmutableList.of(itemEffect)) + .buildOrThrow(); + } + + private static Frame createFakeHardwareBufferFrame( + int sequenceIndex, @Nullable GlEffect itemEffect) { + return new FakeHardwareBufferFrame(createFrameMetadata(sequenceIndex, itemEffect)); + } + + private static Frame createFakeImagePlanesFrame(GlEffect itemEffect) { + return new DefaultImagePlanesFrame.Builder( + ImmutableList.of( + new DefaultPlane( + ByteBuffer.allocateDirect(16), /* rowStride= */ 4, /* pixelStride= */ 1)), + /* releaseExecutor= */ Runnable::run, + /* releaseCallback= */ unused -> {}) + .setFormat(new Format.Builder().setWidth(4).setHeight(4).build()) + .setMetadata(createFrameMetadata(/* sequenceIndex= */ 0, itemEffect)) + .build(); + } + + private ColorInfo queueFrameAndGetColorInfo(Format format, boolean forceUnsupportedFormat) + throws Exception { + return queueFrameAndGetColorInfo( + format, forceUnsupportedFormat, new GlFrameProcessorTestUtil.FakeGlObjectsProvider()); + } + + private ColorInfo queueFrameAndGetColorInfo( + Format format, boolean forceUnsupportedFormat, GlObjectsProvider glObjectsProvider) + throws Exception { + return queueFrameAndGetColorInfo( + format, forceUnsupportedFormat, glObjectsProvider, /* thrownException= */ null); + } + + private ColorInfo queueFrameAndGetColorInfo( + Format format, + boolean forceUnsupportedFormat, + GlObjectsProvider glObjectsProvider, + @Nullable AtomicReference thrownException) + throws Exception { + ImmutableMap metadata = + ImmutableMap.of( + KEY_COMPOSITION_SEQUENCE_INDEX, + 0, + KEY_COMPOSITOR_SETTINGS, + VideoCompositorSettings.DEFAULT, + DefaultGlFrameProcessor.KEY_COMPOSITION_EFFECTS, + ImmutableList.of()); + + Frame frame = new FakeHardwareBufferFrame(format, metadata); + fakeFrameWriterGlTextureFrameConsumer.forceUnsupportedFormat = forceUnsupportedFormat; + + AtomicReference actualColorInfo = new AtomicReference<>(); + DefaultGlFrameProcessor.Factory customFactory = + createDefaultGlFrameProcessorFactoryBuilderWithTestGlResources(glObjectsProvider) + .setFrameToGlTextureConverterFactory( + (outputColorInfo, errorConsumer) -> { + actualColorInfo.set(outputColorInfo); + return fakeFrameToGlTextureConverter; + }) + .build(); + + try (DefaultGlFrameProcessor customProcessor = + customFactory.create( + frameWriter, + glExecutorService, + new FrameProcessor.Listener() { + @Override + public void onWakeup() {} + + @Override + public void onError(VideoFrameProcessingException exception) { + if (thrownException == null) { + throw new AssertionError(exception); + } + thrownException.set(exception); + } + + @Override + public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper fence) {} + })) { + assertThat(customProcessor.queue(ImmutableList.of(new AsyncFrame(frame, null)))).isTrue(); + waitUntilGlThreadFinishes(); + return actualColorInfo.get(); + } + } + + private static class TestGlObjectsProvider implements GlObjectsProvider { + private final boolean failVersion3; + private final boolean failSurfaceCreation; + final AtomicBoolean releaseCalled = new AtomicBoolean(); + + TestGlObjectsProvider(boolean failVersion3) { + this(failVersion3, /* failSurfaceCreation= */ false); + } + + TestGlObjectsProvider(boolean failVersion3, boolean failSurfaceCreation) { + this.failVersion3 = failVersion3; + this.failSurfaceCreation = failSurfaceCreation; + } + + @Override + public EGLContext createEglContext( + EGLDisplay eglDisplay, int openGlVersion, int[] configAttributes) throws GlException { + if (openGlVersion == 3 && failVersion3) { + throw new GlException("Test Version 3 unsupported"); + } + return EGL14.EGL_NO_CONTEXT; + } + + @Override + public EGLSurface createEglSurface( + EGLDisplay eglDisplay, Object surface, int colorTransfer, boolean isEncoderInputSurface) { + return EGL14.EGL_NO_SURFACE; + } + + @Override + public EGLSurface createFocusedPlaceholderEglSurface( + EGLContext eglContext, EGLDisplay eglDisplay) throws GlException { + if (failSurfaceCreation) { + throw new GlException("Test Surface Creation failed"); + } + return EGL14.EGL_NO_SURFACE; + } + + @Override + public GlTextureInfo createBuffersForTexture(int texId, int width, int height) { + return new GlTextureInfo(texId, -1, -1, width, height); + } + + @Override + public void release(EGLDisplay eglDisplay) { + releaseCalled.set(true); } - return new FakeHardwareBufferFrame(metadataBuilder.buildOrThrow()); } } diff --git a/libraries/effect/src/test/java/androidx/media3/effect/DefaultGlTextureFrameCompositingProcessorTest.java b/libraries/effect/src/test/java/androidx/media3/effect/DefaultGlTextureFrameCompositorTest.java similarity index 69% rename from libraries/effect/src/test/java/androidx/media3/effect/DefaultGlTextureFrameCompositingProcessorTest.java rename to libraries/effect/src/test/java/androidx/media3/effect/DefaultGlTextureFrameCompositorTest.java index 08af529b044..204c5851c83 100644 --- a/libraries/effect/src/test/java/androidx/media3/effect/DefaultGlTextureFrameCompositingProcessorTest.java +++ b/libraries/effect/src/test/java/androidx/media3/effect/DefaultGlTextureFrameCompositorTest.java @@ -21,6 +21,7 @@ import static java.util.concurrent.TimeUnit.MILLISECONDS; import androidx.media3.common.C; +import androidx.media3.common.ColorInfo; import androidx.media3.common.GlTextureInfo; import androidx.media3.common.VideoCompositorSettings; import androidx.media3.common.VideoFrameProcessingException; @@ -29,7 +30,6 @@ import androidx.media3.effect.GlFrameProcessorTestUtil.FakeGlObjectsProvider; import androidx.media3.effect.GlFrameProcessorTestUtil.FakeGlTextureFrameConsumer; import androidx.test.ext.junit.runners.AndroidJUnit4; -import androidx.test.filters.SdkSuppress; import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableMap; import java.util.concurrent.ExecutorService; @@ -39,17 +39,16 @@ import org.junit.Test; import org.junit.runner.RunWith; -/** Unit tests for {@link DefaultGlTextureFrameCompositingProcessor}. */ -@SdkSuppress(minSdkVersion = 26) +/** Unit tests for {@link DefaultGlTextureFrameCompositor}. */ @RunWith(AndroidJUnit4.class) -public final class DefaultGlTextureFrameCompositingProcessorTest { +public final class DefaultGlTextureFrameCompositorTest { private static final int WIDTH = 100; private static final int HEIGHT = 200; private static final int FAKE_GL_TEXTURE_ID = 999; private static final long TIMEOUT_MS = 1_000; - private DefaultGlTextureFrameCompositingProcessor compositingProcessor; + private DefaultGlTextureFrameCompositor compositingProcessor; private FakeGlTextureFrameConsumer downstreamConsumer; private AtomicReference errorReference; private FakeCompositorGlProgram compositorGlProgram; @@ -61,7 +60,7 @@ public void setUp() { compositorGlProgram = new FakeCompositorGlProgram(); compositingProcessor = - new DefaultGlTextureFrameCompositingProcessor( + new DefaultGlTextureFrameCompositor( new FakeGlObjectsProvider(), new TexturePool( /* textureAllocator= */ (width, height, useHighPrecisionColorComponents) -> @@ -108,8 +107,8 @@ public void queue_multipleFramesWhenDownstreamRejectsInput_retriesOnGlThread() t GlTextureFrame frame1 = createGlTextureFrame(/* texId= */ 0, /* sequenceIndex= */ 0); GlTextureFrame frame2 = createGlTextureFrame(/* texId= */ 1, /* sequenceIndex= */ 1); - try (DefaultGlTextureFrameCompositingProcessor processor = - new DefaultGlTextureFrameCompositingProcessor( + try (DefaultGlTextureFrameCompositor processor = + new DefaultGlTextureFrameCompositor( new FakeGlObjectsProvider(), new TexturePool( /* textureAllocator= */ (width, height, useHighPrecisionColorComponents) -> @@ -250,6 +249,106 @@ public void queue_multipleFrames_preservesPrimaryFrameMetadata() throws Exceptio .isEqualTo("CUSTOM_VALUE"); } + @Test + public void queue_afterSignalEndOfStream_doesNotPrematurelySignalEos() throws Exception { + GlTextureFrame frame1 = createGlTextureFrame(/* texId= */ 0, /* sequenceIndex= */ 0); + GlTextureFrame frame2 = createGlTextureFrame(/* texId= */ 1, /* sequenceIndex= */ 1); + assertThat( + compositingProcessor.queue( + ImmutableList.of(frame1, frame2), directExecutor(), () -> {})) + .isTrue(); + assertThat(downstreamConsumer.framesReceived).isEqualTo(1); + + compositingProcessor.signalEndOfStream(); + assertThat(downstreamConsumer.signalEndOfStreamCalled).isTrue(); + + downstreamConsumer.signalEndOfStreamCalled = false; + + GlTextureFrame frame3 = createGlTextureFrame(/* texId= */ 2, /* sequenceIndex= */ 0); + GlTextureFrame frame4 = createGlTextureFrame(/* texId= */ 3, /* sequenceIndex= */ 1); + assertThat( + compositingProcessor.queue( + ImmutableList.of(frame3, frame4), directExecutor(), () -> {})) + .isTrue(); + + assertThat(downstreamConsumer.framesReceived).isEqualTo(2); + assertThat(downstreamConsumer.signalEndOfStreamCalled).isFalse(); + + compositingProcessor.signalEndOfStream(); + assertThat(downstreamConsumer.signalEndOfStreamCalled).isTrue(); + } + + @Test + public void queue_singleFrameAfterSignalEndOfStream_doesNotPrematurelySignalEos() + throws Exception { + GlTextureFrame frame1 = createGlTextureFrame(/* texId= */ 0, /* sequenceIndex= */ 0); + assertThat(compositingProcessor.queue(ImmutableList.of(frame1), directExecutor(), () -> {})) + .isTrue(); + assertThat(downstreamConsumer.framesReceived).isEqualTo(1); + + compositingProcessor.signalEndOfStream(); + assertThat(downstreamConsumer.signalEndOfStreamCalled).isTrue(); + + downstreamConsumer.signalEndOfStreamCalled = false; + + GlTextureFrame frame2 = createGlTextureFrame(/* texId= */ 1, /* sequenceIndex= */ 0); + assertThat(compositingProcessor.queue(ImmutableList.of(frame2), directExecutor(), () -> {})) + .isTrue(); + + assertThat(downstreamConsumer.framesReceived).isEqualTo(2); + assertThat(downstreamConsumer.signalEndOfStreamCalled).isFalse(); + + compositingProcessor.signalEndOfStream(); + assertThat(downstreamConsumer.signalEndOfStreamCalled).isTrue(); + } + + @Test + public void create_withDefaultTexturePool_createsInstance() throws Exception { + DefaultGlTextureFrameCompositor.Factory factory = + new DefaultGlTextureFrameCompositor.Factory( + /* compositorGlProgramFactory= */ () -> compositorGlProgram); + + try (GlTextureFrameCompositor compositor = + factory.create( + new FakeGlObjectsProvider(), + /* workingColorSpace= */ ColorInfo.SDR_BT709_LIMITED, + /* errorConsumer= */ errorReference::set, + /* glExecutor= */ directExecutor(), + downstreamConsumer)) { + GlTextureFrame frame = createGlTextureFrame(/* texId= */ 0, /* sequenceIndex= */ 0); + assertThat(compositor.queue(ImmutableList.of(frame), directExecutor(), () -> {})).isTrue(); + assertThat(downstreamConsumer.framesReceived).isEqualTo(1); + } + } + + @Test + public void create_withCustomTexturePool_createsInstance() throws Exception { + DefaultGlTextureFrameCompositor.Factory factory = + new DefaultGlTextureFrameCompositor.Factory( + /* compositorGlProgramFactory= */ () -> compositorGlProgram, + /* texturePoolFactory= */ workingColorSpace -> + new TexturePool( + /* textureAllocator= */ (width, height, useHighPrecisionColorComponents) -> + FAKE_GL_TEXTURE_ID, + /* useHighPrecisionColorComponents= */ false, + /* capacity= */ 1)); + + try (GlTextureFrameCompositor compositor = + factory.create( + new FakeGlObjectsProvider(), + /* workingColorSpace= */ ColorInfo.SDR_BT709_LIMITED, + /* errorConsumer= */ errorReference::set, + /* glExecutor= */ directExecutor(), + downstreamConsumer)) { + GlTextureFrame frame1 = createGlTextureFrame(/* texId= */ 0, /* sequenceIndex= */ 0); + GlTextureFrame frame2 = createGlTextureFrame(/* texId= */ 1, /* sequenceIndex= */ 1); + assertThat(compositor.queue(ImmutableList.of(frame1, frame2), directExecutor(), () -> {})) + .isTrue(); + assertThat(downstreamConsumer.lastReceivedFrame.glTextureInfo.texId) + .isEqualTo(FAKE_GL_TEXTURE_ID); + } + } + private static GlTextureFrame createGlTextureFrame(int texId, int sequenceIndex) { return new GlTextureFrame.Builder( new GlTextureInfo( diff --git a/libraries/effect/src/test/java/androidx/media3/effect/DefaultShaderProgramTest.java b/libraries/effect/src/test/java/androidx/media3/effect/DefaultShaderProgramTest.java new file mode 100644 index 00000000000..b33c5954dea --- /dev/null +++ b/libraries/effect/src/test/java/androidx/media3/effect/DefaultShaderProgramTest.java @@ -0,0 +1,62 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect; + +import static androidx.media3.effect.DefaultVideoFrameProcessor.WORKING_COLOR_SPACE_DEFAULT; +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; +import static org.junit.Assume.assumeFalse; + +import androidx.media3.common.C; +import androidx.media3.common.ColorInfo; +import androidx.media3.common.VideoFrameProcessingException; +import androidx.media3.common.util.GlUtil; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link DefaultShaderProgram}. */ +@RunWith(AndroidJUnit4.class) +public final class DefaultShaderProgramTest { + + private static final ColorInfo HDR_HLG_COLOR_INFO = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorRange(C.COLOR_RANGE_LIMITED) + .setColorTransfer(C.COLOR_TRANSFER_HLG) + .build(); + + @Test + public void createWithExternalSampler_hdrInputWithoutYuvTargetExtension_throwsException() { + assumeFalse(GlUtil.isYuvTargetExtensionSupported()); + + VideoFrameProcessingException exception = + assertThrows( + VideoFrameProcessingException.class, + () -> + DefaultShaderProgram.createWithExternalSampler( + ApplicationProvider.getApplicationContext(), + /* inputColorInfo= */ HDR_HLG_COLOR_INFO, + /* outputColorInfo= */ ColorInfo.SDR_BT709_LIMITED, + WORKING_COLOR_SPACE_DEFAULT, + /* sampleWithNearest= */ false)); + + assertThat(exception) + .hasMessageThat() + .contains("The EXT_YUV_target extension is required for HDR editing input."); + } +} diff --git a/libraries/effect/src/test/java/androidx/media3/effect/FrameProcessorUtilsTest.java b/libraries/effect/src/test/java/androidx/media3/effect/FrameProcessorUtilsTest.java index 69d35982143..b57f67e597a 100644 --- a/libraries/effect/src/test/java/androidx/media3/effect/FrameProcessorUtilsTest.java +++ b/libraries/effect/src/test/java/androidx/media3/effect/FrameProcessorUtilsTest.java @@ -16,11 +16,23 @@ package androidx.media3.effect; import static androidx.media3.effect.FrameProcessorUtils.runAllAndAccumulateExceptions; +import static androidx.media3.effect.FrameProcessorUtils.useHighPrecisionColorComponents; import static com.google.common.truth.Truth.assertThat; import static org.junit.Assert.assertThrows; +import android.opengl.EGL14; +import android.opengl.EGLContext; +import android.opengl.EGLDisplay; +import android.opengl.EGLSurface; +import androidx.media3.common.C; +import androidx.media3.common.ColorInfo; +import androidx.media3.common.GlObjectsProvider; +import androidx.media3.common.GlTextureInfo; import androidx.media3.common.VideoFrameProcessingException; +import androidx.media3.common.util.GlUtil.GlException; import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.util.ArrayList; +import java.util.List; import java.util.concurrent.atomic.AtomicBoolean; import org.junit.Test; import org.junit.runner.RunWith; @@ -99,4 +111,182 @@ public void runAllAndAccumulateExceptions_exceptionInMiddle_runsSubsequentAction assertThat(secondActionExecuted.get()).isTrue(); assertThat(thirdActionExecuted.get()).isTrue(); } + + @Test + public void setupOpenGl_surfacelessSupported_successfulVersion3_returnsVersion3() + throws Exception { + TestGlObjectsProvider glObjectsProvider = + new TestGlObjectsProvider( + /* failVersion3= */ false, /* failVersion2= */ false, /* failSurfaceCreation= */ false); + + int result = + FrameProcessorUtils.setupOpenGl( + glObjectsProvider, /* isSurfacelessContextExtensionSupported= */ true); + + assertThat(result).isEqualTo(FrameProcessorUtils.OPEN_GL_VERSION_3); + assertThat(glObjectsProvider.requestedVersions).containsExactly(3); + } + + @Test + public void + setupOpenGl_surfacelessSupported_version3Throws_fallsBackToVersion2AndReturnsVersion2() + throws Exception { + TestGlObjectsProvider glObjectsProvider = + new TestGlObjectsProvider( + /* failVersion3= */ true, /* failVersion2= */ false, /* failSurfaceCreation= */ false); + + int result = + FrameProcessorUtils.setupOpenGl( + glObjectsProvider, /* isSurfacelessContextExtensionSupported= */ true); + + assertThat(result).isEqualTo(FrameProcessorUtils.OPEN_GL_VERSION_2); + assertThat(glObjectsProvider.requestedVersions).containsExactly(3, 2).inOrder(); + } + + @Test + public void setupOpenGl_surfacelessSupported_surfaceCreationThrows_throwsGlException() { + TestGlObjectsProvider glObjectsProvider = + new TestGlObjectsProvider( + /* failVersion3= */ false, /* failVersion2= */ false, /* failSurfaceCreation= */ true); + + assertThrows( + GlException.class, + () -> + FrameProcessorUtils.setupOpenGl( + glObjectsProvider, /* isSurfacelessContextExtensionSupported= */ true)); + assertThat(glObjectsProvider.requestedVersions).containsExactly(3, 2).inOrder(); + } + + @Test + public void setupOpenGl_surfacelessUnsupported_createsVersion2AndReturnsVersion2() + throws Exception { + TestGlObjectsProvider glObjectsProvider = + new TestGlObjectsProvider( + /* failVersion3= */ false, /* failVersion2= */ false, /* failSurfaceCreation= */ false); + + int result = + FrameProcessorUtils.setupOpenGl( + glObjectsProvider, /* isSurfacelessContextExtensionSupported= */ false); + + assertThat(result).isEqualTo(FrameProcessorUtils.OPEN_GL_VERSION_2); + assertThat(glObjectsProvider.requestedVersions).containsExactly(2); + } + + @Test + public void setupOpenGl_surfacelessUnsupported_version2Throws_throwsGlException() { + TestGlObjectsProvider glObjectsProvider = + new TestGlObjectsProvider( + /* failVersion3= */ false, /* failVersion2= */ true, /* failSurfaceCreation= */ false); + + assertThrows( + GlException.class, + () -> + FrameProcessorUtils.setupOpenGl( + glObjectsProvider, /* isSurfacelessContextExtensionSupported= */ false)); + assertThat(glObjectsProvider.requestedVersions).containsExactly(2); + } + + @Test + public void useHighPrecisionColorComponents_withHdrHlg_returnsTrue() { + ColorInfo hdrHlg = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorTransfer(C.COLOR_TRANSFER_HLG) + .build(); + + assertThat(useHighPrecisionColorComponents(hdrHlg)).isTrue(); + } + + @Test + public void useHighPrecisionColorComponents_withSdrLinear_returnsTrue() { + ColorInfo sdrLinear = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT709) + .setColorTransfer(C.COLOR_TRANSFER_LINEAR) + .build(); + + assertThat(useHighPrecisionColorComponents(sdrLinear)).isTrue(); + } + + @Test + public void useHighPrecisionColorComponents_withSdrSrgb_returnsFalse() { + assertThat(useHighPrecisionColorComponents(ColorInfo.SDR_BT709_LIMITED)).isFalse(); + } + + @Test + public void useHighPrecisionColorComponents_withHighBitdepth_returnsTrue() { + ColorInfo tenBitColorInfo = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT709) + .setColorTransfer(C.COLOR_TRANSFER_SDR) + .setLumaBitdepth(10) + .build(); + + assertThat(useHighPrecisionColorComponents(tenBitColorInfo)).isTrue(); + } + + @Test + public void useHighPrecisionColorComponents_withWideColorGamut_returnsTrue() { + ColorInfo wideColorGamut = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorTransfer(C.COLOR_TRANSFER_SDR) + .build(); + + assertThat(useHighPrecisionColorComponents(wideColorGamut)).isTrue(); + } + + @Test + public void useHighPrecisionColorComponents_withNullColorInfo_returnsFalse() { + assertThat(useHighPrecisionColorComponents(null)).isFalse(); + } + + private static class TestGlObjectsProvider implements GlObjectsProvider { + private final boolean failVersion3; + private final boolean failVersion2; + private final boolean failSurfaceCreation; + final List requestedVersions = new ArrayList<>(); + + TestGlObjectsProvider(boolean failVersion3, boolean failVersion2, boolean failSurfaceCreation) { + this.failVersion3 = failVersion3; + this.failVersion2 = failVersion2; + this.failSurfaceCreation = failSurfaceCreation; + } + + @Override + public EGLContext createEglContext( + EGLDisplay eglDisplay, int openGlVersion, int[] configAttributes) throws GlException { + requestedVersions.add(openGlVersion); + if (openGlVersion == 3 && failVersion3) { + throw new GlException("Test Version 3 unsupported"); + } + if (openGlVersion == 2 && failVersion2) { + throw new GlException("Test Version 2 unsupported"); + } + return EGL14.EGL_NO_CONTEXT; + } + + @Override + public EGLSurface createEglSurface( + EGLDisplay eglDisplay, Object surface, int colorTransfer, boolean isEncoderInputSurface) { + return EGL14.EGL_NO_SURFACE; + } + + @Override + public EGLSurface createFocusedPlaceholderEglSurface( + EGLContext eglContext, EGLDisplay eglDisplay) throws GlException { + if (failSurfaceCreation) { + throw new GlException("Test Surface Creation failed"); + } + return EGL14.EGL_NO_SURFACE; + } + + @Override + public GlTextureInfo createBuffersForTexture(int texId, int width, int height) { + return new GlTextureInfo(texId, -1, -1, width, height); + } + + @Override + public void release(EGLDisplay eglDisplay) {} + } } diff --git a/libraries/effect/src/test/java/androidx/media3/effect/GlFrameProcessorTestUtil.java b/libraries/effect/src/test/java/androidx/media3/effect/GlFrameProcessorTestUtil.java index ef3850c5d1f..847769326c0 100644 --- a/libraries/effect/src/test/java/androidx/media3/effect/GlFrameProcessorTestUtil.java +++ b/libraries/effect/src/test/java/androidx/media3/effect/GlFrameProcessorTestUtil.java @@ -15,7 +15,9 @@ */ package androidx.media3.effect; +import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.util.concurrent.MoreExecutors.directExecutor; +import static java.util.concurrent.TimeUnit.MILLISECONDS; import android.hardware.HardwareBuffer; import android.opengl.EGL14; @@ -24,10 +26,12 @@ import android.opengl.EGLSurface; import androidx.annotation.Nullable; import androidx.media3.common.C; +import androidx.media3.common.ColorInfo; import androidx.media3.common.Format; import androidx.media3.common.GlObjectsProvider; import androidx.media3.common.GlTextureInfo; import androidx.media3.common.VideoFrameProcessingException; +import androidx.media3.common.util.ConditionVariable; import androidx.media3.common.util.GlUtil; import androidx.media3.common.video.AsyncFrame; import androidx.media3.common.video.Frame; @@ -35,10 +39,12 @@ import androidx.media3.common.video.FrameWriter; import androidx.media3.common.video.HardwareBufferFrame; import androidx.media3.common.video.SyncFenceWrapper; +import androidx.media3.effect.GlTextureFrameCompositor.CompositorGlProgram; import com.google.common.collect.ImmutableMap; import java.util.ArrayList; import java.util.List; import java.util.Map; +import java.util.concurrent.CountDownLatch; import java.util.concurrent.Executor; /** Utility class containing fakes for GlFrameProcessor testing. */ @@ -83,8 +89,9 @@ public void close() {} public static final class FakeGlTextureFrameConsumer implements GlTextureFrameConsumer { public volatile boolean shouldAcceptIncomingFrames; - public int framesReceived; - public boolean signalEndOfStreamCalled; + public volatile int framesReceived; + public volatile boolean signalEndOfStreamCalled; + public boolean forceUnsupportedFormat; @Nullable public Runnable wakeupListener; @Nullable public Executor listenerExecutor; @Nullable public GlTextureFrame lastReceivedFrame; @@ -92,9 +99,44 @@ public static final class FakeGlTextureFrameConsumer implements GlTextureFrameCo @Nullable public VideoFrameProcessingException exceptionToThrowOnQueueing; @Nullable public RuntimeException runtimeExceptionToThrowOnQueueing; + /** Opened when a frame is rejected and a wakeup listener is registered. */ + public final ConditionVariable wakeupListenerRegistered; + + /** The thread that called {@link #close()}, or {@code null} if not closed. */ + @Nullable public volatile Thread closeThread; + + @Nullable private volatile CountDownLatch framesReceivedLatch; + public FakeGlTextureFrameConsumer(@Nullable FrameWriter frameWriter) { this.frameWriter = frameWriter; shouldAcceptIncomingFrames = true; + wakeupListenerRegistered = new ConditionVariable(); + } + + @Override + public boolean isOutputFormatSupported(Format format) { + return !forceUnsupportedFormat; + } + + /** + * Sets the number of expected frames to be received. + * + *

This method must be called before the expected frames are queued. + */ + public void setExpectedFrameCount(int count) { + this.framesReceivedLatch = new CountDownLatch(count); + } + + public boolean awaitFramesReceived(long timeoutMs) { + if (framesReceivedLatch == null) { + return true; + } + try { + return checkNotNull(framesReceivedLatch).await(timeoutMs, MILLISECONDS); + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + return false; + } } @Override @@ -110,10 +152,14 @@ public synchronized boolean queue( if (!shouldAcceptIncomingFrames) { this.listenerExecutor = listenerExecutor; this.wakeupListener = wakeupListener; + wakeupListenerRegistered.open(); return false; } lastReceivedFrame = frame; framesReceived++; + if (framesReceivedLatch != null) { + framesReceivedLatch.countDown(); + } if (frameWriter != null) { frameWriter.queueInputFrame(null, null); } @@ -137,11 +183,13 @@ public void signalEndOfStream() { } @Override - public void close() {} + public void close() { + closeThread = Thread.currentThread(); + } } /** Fake {@link GlShaderProgram} for testing. */ - public static final class FakeGlShaderProgram implements GlShaderProgram { + public static class FakeGlShaderProgram implements GlShaderProgram { public int framesReceived; public boolean signalEndOfCurrentInputStreamCalled; public boolean delayReadyToAcceptInputFrame; @@ -228,9 +276,21 @@ public void release() { /** Fake {@link HardwareBufferFrame} for testing. */ public static final class FakeHardwareBufferFrame implements HardwareBufferFrame { private final ImmutableMap metadata; + private final Format format; public FakeHardwareBufferFrame(Map metadata) { - this.metadata = ImmutableMap.copyOf(metadata); + this( + new Format.Builder() + .setWidth(100) + .setHeight(100) + .setColorInfo(ColorInfo.SDR_BT709_LIMITED) + .build(), + metadata); + } + + public FakeHardwareBufferFrame(Format format, Map metadata) { + this.format = format; + this.metadata = ImmutableMap.builder().putAll(metadata).buildOrThrow(); } @Override @@ -245,7 +305,7 @@ public long getContentTimeUs() { @Override public Format getFormat() { - return new Format.Builder().setWidth(100).setHeight(100).build(); + return format; } @Override @@ -259,24 +319,23 @@ public HardwareBufferFrame.Builder buildUpon() { } } - /** Fake {@link DefaultGlFrameProcessor.HardwareBufferConverter} for testing. */ - public static final class FakeHardwareBufferConverter - implements DefaultGlFrameProcessor.HardwareBufferConverter { + /** Fake {@link FrameToGlTextureConverter} for testing. */ + public static final class FakeFrameToGlTextureConverter implements FrameToGlTextureConverter { public int framesReceived; public int framesWithGlResourceReleased; private final Runnable releaseListener; - public FakeHardwareBufferConverter(Runnable releaseListener) { + public FakeFrameToGlTextureConverter(Runnable releaseListener) { this.releaseListener = releaseListener; } - public FakeHardwareBufferConverter() { + public FakeFrameToGlTextureConverter() { this.releaseListener = () -> {}; } @Override public GlTextureFrame convert( - HardwareBufferFrame hardwareBufferFrame, + Frame frame, Executor glExecutor, Executor listenerExecutor, FrameProcessor.Listener listener) { @@ -293,19 +352,17 @@ public GlTextureFrame convert( releaseListener.run(); if (listener != null) { listenerExecutor.execute( - () -> - listener.onFrameProcessed( - hardwareBufferFrame, /* onCompleteFence= */ null)); + () -> listener.onFrameProcessed(frame, /* onCompleteFence= */ null)); } }) - .setPresentationTimeUs(hardwareBufferFrame.getContentTimeUs()) - .setFormat(hardwareBufferFrame.getFormat()) - .setMetadata(hardwareBufferFrame.getMetadata()) + .setPresentationTimeUs(frame.getContentTimeUs()) + .setFormat(frame.getFormat()) + .setMetadata(frame.getMetadata()) .build(); } @Override - public void releaseGlResources(HardwareBufferFrame hardwareBufferFrame) { + public void releaseGlResources(Frame frame) { framesWithGlResourceReleased++; } @@ -313,8 +370,7 @@ public void releaseGlResources(HardwareBufferFrame hardwareBufferFrame) { public void close() {} } - public static final class FakeCompositorGlProgram - implements DefaultGlTextureFrameCompositingProcessor.CompositorGlProgram { + public static final class FakeCompositorGlProgram implements CompositorGlProgram { @Nullable public VideoFrameProcessingException exceptionToThrow; @Nullable public GlUtil.GlException glExceptionToThrow; diff --git a/libraries/effect/src/test/java/androidx/media3/effect/GlShaderProgramAdapterTest.java b/libraries/effect/src/test/java/androidx/media3/effect/GlShaderProgramAdapterTest.java index ed55b0621ab..586145844aa 100644 --- a/libraries/effect/src/test/java/androidx/media3/effect/GlShaderProgramAdapterTest.java +++ b/libraries/effect/src/test/java/androidx/media3/effect/GlShaderProgramAdapterTest.java @@ -15,9 +15,11 @@ */ package androidx.media3.effect; +import static androidx.media3.common.C.INDEX_UNSET; import static androidx.media3.effect.DefaultGlFrameProcessor.KEY_FRAME_DISCONTINUITY_NUMBER; import static com.google.common.truth.Truth.assertThat; import static com.google.common.util.concurrent.MoreExecutors.directExecutor; +import static java.util.concurrent.TimeUnit.MILLISECONDS; import static org.junit.Assert.assertThrows; import android.opengl.EGLContext; @@ -35,6 +37,7 @@ import java.util.ArrayDeque; import java.util.ArrayList; import java.util.List; +import java.util.concurrent.CountDownLatch; import java.util.concurrent.Executor; import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicReference; @@ -49,6 +52,7 @@ public class GlShaderProgramAdapterTest { private static final FakeGlObjectsProvider FAKE_GL_OBJECTS_PROVIDER = new FakeGlObjectsProvider(); private static final int TEXTURE_SIZE = 16; + private static final long TIMEOUT_MS = 5_000; private GlShaderProgramAdapter glShaderProgramAdapter; private FakeGlShaderProgram fakeGlShaderProgram; @@ -63,7 +67,11 @@ public void setUp() { glShaderProgramAdapter = new GlShaderProgramAdapter( - fakeGlShaderProgram, FAKE_GL_OBJECTS_PROVIDER, directExecutor(), errorReference::set); + fakeGlShaderProgram, + FAKE_GL_OBJECTS_PROVIDER, + directExecutor(), + /* outputColorInfo= */ ColorInfo.SDR_BT709_LIMITED, + errorReference::set); glShaderProgramAdapter.setOutput(downstreamConsumer); } @@ -151,6 +159,48 @@ public void onOutputFrameAvailable_whenDownstreamHasCapacity_forwardsCorrectForm assertThat(errorReference.get()).isNull(); } + @Test + public void onOutputFrameAvailable_withConfiguredOutputColorInfo_updatesFormatColorInfo() { + GlShaderProgramAdapter adapter = + new GlShaderProgramAdapter( + fakeGlShaderProgram, + FAKE_GL_OBJECTS_PROVIDER, + directExecutor(), + /* outputColorInfo= */ DefaultGlFrameProcessor.BT2020_HLG, + errorReference::set); + adapter.setOutput(downstreamConsumer); + Format inputFormat = + new Format.Builder() + .setWidth(TEXTURE_SIZE) + .setHeight(TEXTURE_SIZE) + .setColorInfo(DefaultGlFrameProcessor.BT709_SRGB) + .build(); + GlTextureFrame inputFrame = + createTestFrameWithFormat( + /* texId= */ 1, + /* timestampUs= */ 1000L, + /* format= */ inputFormat, + /* released= */ new AtomicBoolean()); + + assertThat( + adapter.queue( + inputFrame, /* listenerExecutor= */ task -> {}, /* wakeupListener= */ () -> {})) + .isTrue(); + adapter.onOutputFrameAvailable( + new GlTextureInfo( + /* texId= */ 101, + /* fboId= */ INDEX_UNSET, + /* rboId= */ INDEX_UNSET, + /* width= */ TEXTURE_SIZE, + /* height= */ TEXTURE_SIZE), + /* presentationTimeUs= */ 1000L); + + assertThat(downstreamConsumer.queuedFrames).hasSize(1); + assertThat(downstreamConsumer.queuedFrames.get(0).format.colorInfo) + .isEqualTo(DefaultGlFrameProcessor.BT2020_HLG); + assertThat(errorReference.get()).isNull(); + } + @Test public void onOutputFrameAvailable_whenDownstreamNoCapacity_queuesPendingAndForwardsOnWakeup() { AtomicBoolean frameReleased = new AtomicBoolean(); @@ -182,17 +232,48 @@ public void onOutputFrameAvailable_whenDownstreamNoCapacity_queuesPendingAndForw assertThat(errorReference.get()).isNull(); } + @Test + public void onCurrentOutputStreamEnded_withPendingFrame_onlySignalsEosOnce() { + AtomicBoolean frameReleased = new AtomicBoolean(); + GlTextureFrame inputFrame = + createTestFrame(/* texId= */ 1, /* timestampUs= */ 1000L, frameReleased); + assertThat(glShaderProgramAdapter.queue(inputFrame, task -> {}, () -> {})).isTrue(); + + // Force the downstream capacity to block queueing + downstreamConsumer.setCapacity(0); + GlTextureInfo outputTexture = + new GlTextureInfo( + /* texId= */ 101, + /* fboId= */ -1, + /* rboId= */ -1, + /* width= */ 100, + /* height= */ 100); + glShaderProgramAdapter.onOutputFrameAvailable(outputTexture, 1000L); + + // Call onCurrentOutputStreamEnded while there is a pending frame, EOS shouldn't be signaled yet + glShaderProgramAdapter.signalEndOfStream(); + + assertThat(downstreamConsumer.eosSignalsReceived).isEqualTo(0); + + // Make capacity available, the queued frame and EOS should now be processed. + downstreamConsumer.setCapacity(1); + + assertThat(downstreamConsumer.queuedFrames).hasSize(1); + assertThat(downstreamConsumer.eosSignalsReceived).isEqualTo(1); + assertThat(errorReference.get()).isNull(); + } + @Test public void onCurrentOutputStreamEnded_signalsEndOfStreamToDownstream() { - glShaderProgramAdapter.onCurrentOutputStreamEnded(); - assertThat(downstreamConsumer.endOfStreamSignaled).isTrue(); + glShaderProgramAdapter.signalEndOfStream(); + assertThat(downstreamConsumer.eosSignalsReceived).isEqualTo(1); assertThat(errorReference.get()).isNull(); } @Test public void signalEndOfStream_signalsEndOfInputStreamToShaderProgram() { glShaderProgramAdapter.signalEndOfStream(); - assertThat(fakeGlShaderProgram.endOfInputStreamSignaled).isTrue(); + assertThat(fakeGlShaderProgram.eosSignalsReceived).isEqualTo(1); assertThat(errorReference.get()).isNull(); } @@ -572,7 +653,7 @@ public void queue_withDifferentFormat_signalsEosRejectsFrameAndWakesUp() { glShaderProgramAdapter.queue(inputFrame2, testExecutor, () -> wakeupCalled.set(true))) .isFalse(); - assertThat(fakeGlShaderProgram.endOfInputStreamSignaled).isTrue(); + assertThat(fakeGlShaderProgram.eosSignalsReceived).isEqualTo(1); assertThat(queuedWakeupTasks).hasSize(1); queuedWakeupTasks.remove().run(); @@ -629,9 +710,9 @@ public void onCurrentOutputStreamEnded_withDroppedFrames_completesWithoutError() glShaderProgramAdapter.onOutputFrameAvailable(outputTexture, 1000L); glShaderProgramAdapter.onInputFrameProcessed(inputFrame2.glTextureInfo); - glShaderProgramAdapter.onCurrentOutputStreamEnded(); + glShaderProgramAdapter.signalEndOfStream(); - assertThat(downstreamConsumer.endOfStreamSignaled).isTrue(); + assertThat(downstreamConsumer.eosSignalsReceived).isEqualTo(1); assertThat(downstreamConsumer.queuedFrames).hasSize(1); assertThat(downstreamConsumer.queuedFrames.get(0).presentationTimeUs).isEqualTo(1000L); assertThat(frame1Released.get()).isTrue(); @@ -683,7 +764,7 @@ public void queue_withDifferentFormat_wakeupListenerRequeuesFrameSuccessfully() assertThat( glShaderProgramAdapter.queue(inputFrame1, testExecutor, /* wakeupListener= */ () -> {})) .isTrue(); - assertThat(downstreamConsumer.endOfStreamSignaled).isFalse(); + assertThat(downstreamConsumer.eosSignalsReceived).isEqualTo(0); // Simulate the shader processing the frame to restore capacity. glShaderProgramAdapter.onInputFrameProcessed(inputFrame1.glTextureInfo); @@ -702,13 +783,16 @@ public void queue_withDifferentFormat_wakeupListenerRequeuesFrameSuccessfully() .isFalse(); assertThat(queuedWakeupTasks).hasSize(1); - assertThat(downstreamConsumer.endOfStreamSignaled).isTrue(); + // Suppressed EOS on format change, so EOS is not forwarded downstream yet. + assertThat(downstreamConsumer.eosSignalsReceived).isEqualTo(0); // Execute the wakeup listener. It should requeue the frame successfully since currentFormat // is now null and capacity has been restored. queuedWakeupTasks.remove().run(); assertThat(frame2Queued.get()).isTrue(); + glShaderProgramAdapter.signalEndOfStream(); + assertThat(downstreamConsumer.eosSignalsReceived).isEqualTo(1); assertThat(errorReference.get()).isNull(); } @@ -730,7 +814,7 @@ public void signalEndOfStream_afterFormatChange_doesNotDoubleSignal() { // Calling signalEndOfStream() should not double-signal the underlying glShaderProgram. glShaderProgramAdapter.signalEndOfStream(); - assertThat(fakeGlShaderProgram.endOfInputStreamSignaled).isTrue(); + assertThat(fakeGlShaderProgram.eosSignalsReceived).isEqualTo(1); assertThat(errorReference.get()).isNull(); } @@ -782,6 +866,81 @@ public void queue_withDifferentMetadata_propagatesMetadataCorrectly() { assertThat(downstreamConsumer.queuedFrames.get(1).getMetadata()).isEqualTo(metadata2); } + @Test + public void signalEndOfStream_followedByNewFormatWithDiscontinuity_isReusable() throws Exception { + Executor testExecutor = task -> {}; + Format format1 = new Format.Builder().setWidth(100).setHeight(100).build(); + Format format2 = new Format.Builder().setWidth(200).setHeight(200).build(); + + GlTextureFrame inputFrame1 = + createTestFrameWithFormat( + /* texId= */ 1, /* timestampUs= */ 1000L, format1, new AtomicBoolean()) + .buildUpon() + .setMetadata(ImmutableMap.of(KEY_FRAME_DISCONTINUITY_NUMBER, 0)) + .build(); + + GlTextureFrame inputFrame2 = + createTestFrameWithFormat( + /* texId= */ 2, /* timestampUs= */ 2000L, format2, new AtomicBoolean()) + .buildUpon() + // Format change is not accompanied by a discontinuity + .setMetadata(ImmutableMap.of(KEY_FRAME_DISCONTINUITY_NUMBER, 0)) + .build(); + + assertThat( + glShaderProgramAdapter.queue( + inputFrame1, /* listenerExecutor= */ testExecutor, /* wakeupListener= */ () -> {})) + .isTrue(); + glShaderProgramAdapter.onInputFrameProcessed(inputFrame1.glTextureInfo); + glShaderProgramAdapter.onReadyToAcceptInputFrame(); + glShaderProgramAdapter.onOutputFrameAvailable( + new GlTextureInfo( + /* texId= */ 100, + /* fboId= */ -1, + /* rboId= */ -1, + /* width= */ 100, + /* height= */ 100), + /* presentationTimeUs= */ 1000L); + + downstreamConsumer.setCapacity(1); + // Queue format change should return false. + CountDownLatch eosPropagated = new CountDownLatch(1); + assertThat( + glShaderProgramAdapter.queue( + inputFrame2, /* listenerExecutor= */ testExecutor, /* wakeupListener= */ () -> {})) + .isFalse(); + // Wait until EOS is propagated. + fakeGlShaderProgram.setEosReceivedLatch(eosPropagated); + eosPropagated.await(TIMEOUT_MS, MILLISECONDS); + + // Queueing the second time succeeds, as the states are reset internally. + assertThat( + glShaderProgramAdapter.queue( + inputFrame2, /* listenerExecutor= */ testExecutor, /* wakeupListener= */ () -> {})) + .isTrue(); + + // Format change signals shader.onCurrentInputStreamEnded, but doesn't signal EOS to downstream + // consumer + assertThat(fakeGlShaderProgram.eosSignalsReceived).isEqualTo(1); + assertThat(downstreamConsumer.eosSignalsReceived).isEqualTo(0); + + glShaderProgramAdapter.onInputFrameProcessed(inputFrame2.glTextureInfo); + glShaderProgramAdapter.onReadyToAcceptInputFrame(); + glShaderProgramAdapter.onOutputFrameAvailable( + new GlTextureInfo( + /* texId= */ 200, + /* fboId= */ -1, + /* rboId= */ -1, + /* width= */ 200, + /* height= */ 200), + /* presentationTimeUs= */ 2000L); + + glShaderProgramAdapter.signalEndOfStream(); + assertThat(fakeGlShaderProgram.eosSignalsReceived).isEqualTo(2); + assertThat(downstreamConsumer.eosSignalsReceived).isEqualTo(1); + assertThat(errorReference.get()).isNull(); + } + private static GlTextureFrame createTestFrameWithDiscontinuityNumber( int texId, long timestampUs, int discontinuityNumber, @Nullable AtomicBoolean released) { return new GlTextureFrame.Builder( @@ -867,7 +1026,7 @@ private static final class FakeGlTextureFrameConsumer implements GlTextureFrameC @Nullable private Runnable pendingWakeupListener; private int capacity = 1; - boolean endOfStreamSignaled; + int eosSignalsReceived; @Nullable private VideoFrameProcessingException exceptionToThrowOnQueue; FakeGlTextureFrameConsumer() { @@ -902,7 +1061,7 @@ public boolean queue(GlTextureFrame frame, Executor listenerExecutor, Runnable w @Override public void signalEndOfStream() { - endOfStreamSignaled = true; + eosSignalsReceived++; } @Override @@ -925,13 +1084,13 @@ static class QueuedFrameInfo { private InputListener inputListener; private OutputListener outputListener; @Nullable private ErrorListener errorListener; - boolean endOfInputStreamSignaled; + int eosSignalsReceived; boolean flushed; + @Nullable private CountDownLatch eosReceivedLatch; FakeGlShaderProgram() { queuedFrames = new ArrayList<>(); releasedFrames = new ArrayList<>(); - endOfInputStreamSignaled = false; flushed = false; inputListener = new InputListener() {}; } @@ -972,10 +1131,13 @@ public void releaseOutputFrame(GlTextureInfo outputTexture) { @Override public void signalEndOfCurrentInputStream() { - endOfInputStreamSignaled = true; + eosSignalsReceived++; if (outputListener != null) { outputListener.onCurrentOutputStreamEnded(); } + if (eosReceivedLatch != null) { + eosReceivedLatch.countDown(); + } } @Override @@ -987,5 +1149,9 @@ public void flush() { @Override public void release() {} + + void setEosReceivedLatch(CountDownLatch latch) { + this.eosReceivedLatch = latch; + } } } diff --git a/libraries/effect/src/test/java/androidx/media3/effect/GlTextureFrameAggregatorTest.java b/libraries/effect/src/test/java/androidx/media3/effect/GlTextureFrameAggregatorTest.java index a0ada35128a..ba7312bd90b 100644 --- a/libraries/effect/src/test/java/androidx/media3/effect/GlTextureFrameAggregatorTest.java +++ b/libraries/effect/src/test/java/androidx/media3/effect/GlTextureFrameAggregatorTest.java @@ -45,7 +45,7 @@ public final class GlTextureFrameAggregatorTest { private static final int FAKE_GL_TEXTURE_ID = 999; - private DefaultGlTextureFrameCompositingProcessor compositingProcessor; + private DefaultGlTextureFrameCompositor compositingProcessor; private FakeGlTextureFrameConsumer downstreamConsumer; private FakeCompositorGlProgram compositorGlProgram; private GlTextureFrameAggregator frameAggregator; @@ -58,7 +58,7 @@ public void setUp() { compositorGlProgram = new FakeCompositorGlProgram(); ListeningExecutorService glExecutorService = newDirectExecutorService(); compositingProcessor = - new DefaultGlTextureFrameCompositingProcessor( + new DefaultGlTextureFrameCompositor( new FakeGlObjectsProvider(), new TexturePool( /* textureAllocator= */ (width, height, useHighPrecisionColorComponents) -> @@ -214,6 +214,63 @@ public void signalEndOfStream_whenCompositorRejectsInput_defersEosUntilWakeup() assertThat(downstreamConsumer.signalEndOfStreamCalled).isTrue(); } + @Test + public void signalEndOfStream_afterEndOfStreamAndQueue_signalsDownstreamAgain() throws Exception { + frameAggregator.configureSequenceIndices(ImmutableSet.of(0)); + GlTextureFrame frame1 = createGlTextureFrame(/* sequenceIndex= */ 0); + + assertThat(frameAggregator.getInputConsumer(0).queue(frame1, directExecutor(), () -> {})) + .isTrue(); + frameAggregator.getInputConsumer(0).signalEndOfStream(); + assertThat(downstreamConsumer.signalEndOfStreamCalled).isTrue(); + + downstreamConsumer.signalEndOfStreamCalled = false; + + GlTextureFrame frame2 = createGlTextureFrame(/* sequenceIndex= */ 0); + assertThat(frameAggregator.getInputConsumer(0).queue(frame2, directExecutor(), () -> {})) + .isTrue(); + assertThat(downstreamConsumer.signalEndOfStreamCalled).isFalse(); + + frameAggregator.getInputConsumer(0).signalEndOfStream(); + assertThat(downstreamConsumer.signalEndOfStreamCalled).isTrue(); + } + + @Test + public void + signalEndOfStream_afterEndOfStreamAndQueueWithMultipleSequences_signalsDownstreamAgain() + throws Exception { + frameAggregator.configureSequenceIndices(/* requestedSequenceIndices= */ ImmutableSet.of(0, 1)); + // Batch A, sequences 0 and 1 + GlTextureFrame frame0A = createGlTextureFrame(/* sequenceIndex= */ 0); + GlTextureFrame frame1A = createGlTextureFrame(/* sequenceIndex= */ 1); + + assertThat(frameAggregator.getInputConsumer(0).queue(frame0A, directExecutor(), () -> {})) + .isTrue(); + assertThat(frameAggregator.getInputConsumer(1).queue(frame1A, directExecutor(), () -> {})) + .isTrue(); + frameAggregator.getInputConsumer(0).signalEndOfStream(); + frameAggregator.getInputConsumer(1).signalEndOfStream(); + assertThat(downstreamConsumer.signalEndOfStreamCalled).isTrue(); + + downstreamConsumer.signalEndOfStreamCalled = false; + // Batch B, sequences 0 and 1 + GlTextureFrame frame0B = createGlTextureFrame(/* sequenceIndex= */ 0); + GlTextureFrame frame1B = createGlTextureFrame(/* sequenceIndex= */ 1); + assertThat(frameAggregator.getInputConsumer(0).queue(frame0B, directExecutor(), () -> {})) + .isTrue(); + assertThat(frameAggregator.getInputConsumer(1).queue(frame1B, directExecutor(), () -> {})) + .isTrue(); + assertThat(downstreamConsumer.signalEndOfStreamCalled).isFalse(); + + frameAggregator.getInputConsumer(0).signalEndOfStream(); + // Downstream EOS is not yet signaled because sequence 1 hasn't signaled EOS since frame1B. + assertThat(downstreamConsumer.signalEndOfStreamCalled).isFalse(); + + // Verify downstream EOS is signaled only after sequence 1 also signals EOS again. + frameAggregator.getInputConsumer(1).signalEndOfStream(); + assertThat(downstreamConsumer.signalEndOfStreamCalled).isTrue(); + } + @Test public void queue_whenDownstreamThrowsVideoFrameProcessingException_throwsIllegalStateException() throws Exception { diff --git a/libraries/effect/src/test/java/androidx/media3/effect/GlTextureFrameProcessorChainTest.java b/libraries/effect/src/test/java/androidx/media3/effect/GlTextureFrameProcessorChainTest.java index 1ccea460ada..132a2be98f1 100644 --- a/libraries/effect/src/test/java/androidx/media3/effect/GlTextureFrameProcessorChainTest.java +++ b/libraries/effect/src/test/java/androidx/media3/effect/GlTextureFrameProcessorChainTest.java @@ -15,15 +15,22 @@ */ package androidx.media3.effect; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT2020_HLG; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT2020_LINEAR; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT709_LINEAR; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT709_SRGB; import static androidx.media3.effect.DefaultGlFrameProcessor.KEY_ITEM_EFFECTS; import static androidx.test.core.app.ApplicationProvider.getApplicationContext; +import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.truth.Truth.assertThat; import static com.google.common.util.concurrent.MoreExecutors.directExecutor; import static com.google.common.util.concurrent.MoreExecutors.listeningDecorator; import static org.junit.Assert.assertThrows; import android.content.Context; +import androidx.media3.common.ColorInfo; import androidx.media3.common.Effect; +import androidx.media3.common.GlObjectsProvider; import androidx.media3.common.GlTextureInfo; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.Util; @@ -35,6 +42,8 @@ import com.google.common.util.concurrent.ListeningExecutorService; import java.util.List; import java.util.concurrent.ExecutionException; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicReference; import org.junit.After; import org.junit.Before; import org.junit.Test; @@ -55,14 +64,7 @@ public void setUp() { downstreamFrameConsumer = new FakeGlTextureFrameConsumer(/* frameWriter= */ null); glExecutorService = listeningDecorator(Util.newSingleThreadExecutor("Effect:GlThread")); - glTextureFrameProcessorChain = - new GlTextureFrameProcessorChain( - context, - new DefaultGlObjectsProvider(), - glExecutorService, - /* errorConsumer= */ e -> {}, - downstreamFrameConsumer, - KEY_ITEM_EFFECTS); + glTextureFrameProcessorChain = createProcessorChain(BT709_SRGB); } @After @@ -177,6 +179,120 @@ public void signalEndOfStream_delegatesToFirstConsumer() throws Exception { assertThat(downstreamFrameConsumer.signalEndOfStreamCalled).isTrue(); } + @Test + public void queue_withMatrixAndRgbEffects_mergesIntoSingleProcessor() throws Exception { + AtomicBoolean matrixToGlShaderProgramCalled = new AtomicBoolean(); + GlMatrixTransformation fakeMatrixEffect = + new GlMatrixTransformation() { + @Override + public float[] getGlMatrixArray(long presentationTimeUs) { + return new float[16]; + } + + @Override + public BaseGlShaderProgram toGlShaderProgram(Context context, boolean useHdr) + throws VideoFrameProcessingException { + matrixToGlShaderProgramCalled.set(true); + return GlMatrixTransformation.super.toGlShaderProgram(context, useHdr); + } + }; + + AtomicBoolean rgbToGlShaderProgramCalled = new AtomicBoolean(); + RgbMatrix fakeColorEffect = + new RgbMatrix() { + @Override + public float[] getMatrix(long presentationTimeUs, boolean useHdr) { + return new float[16]; + } + + @Override + public BaseGlShaderProgram toGlShaderProgram(Context context, boolean useHdr) + throws VideoFrameProcessingException { + rgbToGlShaderProgramCalled.set(true); + return RgbMatrix.super.toGlShaderProgram(context, useHdr); + } + }; + + // Queue frame with Matrix and RGB effects. + GlTextureFrame frame = + createGlTextureFrameWithEffects(ImmutableList.of(fakeMatrixEffect, fakeColorEffect)); + + try { + boolean unused = + glTextureFrameProcessorChain.queue( + frame, glExecutorService, /* wakeupListener= */ () -> {}); + } catch (Exception e) { + // We can't create actual GL shaders in unit tests, ignore the exceptions thrown. + } + + // If these effects are merged properly, they are added directly to the DefaultShaderProgram's + // constructor arguments, and their individual toGlShaderProgram(...) method is skipped. + assertThat(matrixToGlShaderProgramCalled.get()).isFalse(); + assertThat(rgbToGlShaderProgramCalled.get()).isFalse(); + } + + @Test + public void queue_withHdrHlgWorkingColorSpace_createsShaderProgramWithHdrGamut() + throws Exception { + assertThat(queueFrameAndGetEffectUseHdr(BT2020_HLG)).isTrue(); + } + + @Test + public void queue_withLinearHdrWorkingColorSpace_createsShaderProgramWithHdrGamut() + throws Exception { + assertThat(queueFrameAndGetEffectUseHdr(BT2020_LINEAR)).isTrue(); + } + + @Test + public void queue_withSdrSrgbWorkingColorSpace_createsShaderProgramWithSdrGamut() + throws Exception { + assertThat(queueFrameAndGetEffectUseHdr(BT709_SRGB)).isFalse(); + } + + @Test + public void queue_withLinearSdrWorkingColorSpace_createsShaderProgramWithSdrGamut() + throws Exception { + // This pins the gamut decision only, as linear SDR colors are BT.709 rather than BT.2020. It is + // not an endorsement of the 8-bit output textures that useHdr being false also selects, see the + // TODOs in GlTextureFrameProcessorChain.configure(). + assertThat(queueFrameAndGetEffectUseHdr(BT709_LINEAR)).isFalse(); + } + + /** + * Queues a frame with a single {@link GlEffect} through a chain configured with the given {@code + * workingColorSpace}, and returns the {@code useHdr} value the chain used to create the effect's + * {@link GlShaderProgram}. + */ + private boolean queueFrameAndGetEffectUseHdr(ColorInfo workingColorSpace) throws Exception { + AtomicReference effectUseHdr = new AtomicReference<>(); + GlEffect fakeEffect = + (context, useHdr) -> { + effectUseHdr.set(useHdr); + return new FakeGlShaderProgram(); + }; + glTextureFrameProcessorChain = createProcessorChain(workingColorSpace); + GlTextureFrame frame = createGlTextureFrameWithEffects(ImmutableList.of(fakeEffect)); + + assertThat( + glTextureFrameProcessorChain.queue( + frame, glExecutorService, /* wakeupListener= */ () -> {})) + .isTrue(); + waitUntilGlThreadFinishes(); + + return checkNotNull(effectUseHdr.get()); + } + + private GlTextureFrameProcessorChain createProcessorChain(ColorInfo workingColorSpace) { + return new GlTextureFrameProcessorChain( + context, + new DefaultGlObjectsProvider(), + glExecutorService, + /* errorConsumer= */ e -> {}, + downstreamFrameConsumer, + KEY_ITEM_EFFECTS, + workingColorSpace); + } + private void waitUntilGlThreadFinishes() throws ExecutionException, InterruptedException { glExecutorService.submit(() -> {}).get(); } @@ -195,4 +311,49 @@ private static GlTextureFrame createGlTextureFrameWithEffects(List effec .setMetadata(ImmutableMap.of(KEY_ITEM_EFFECTS, ImmutableList.copyOf(effects))) .build(); } + + @Test + public void queue_withSizeChangingEffect_propagatesNewSizeDownstream() throws Exception { + int newWidth = 200; + int newHeight = 300; + SizeChangingGlShaderProgram sizeChangingShaderProgram = + new SizeChangingGlShaderProgram(newWidth, newHeight); + GlEffect sizeChangingEffect = (context, useHdr) -> sizeChangingShaderProgram; + + GlTextureFrame frame = createGlTextureFrameWithEffects(ImmutableList.of(sizeChangingEffect)); + + // Process the frame. + assertThat( + glTextureFrameProcessorChain.queue( + frame, glExecutorService, /* wakeupListener= */ () -> {})) + .isTrue(); + waitUntilGlThreadFinishes(); + + assertThat(downstreamFrameConsumer.framesReceived).isEqualTo(1); + assertThat(downstreamFrameConsumer.lastReceivedFrame.format.width).isEqualTo(newWidth); + assertThat(downstreamFrameConsumer.lastReceivedFrame.format.height).isEqualTo(newHeight); + } + + private static final class SizeChangingGlShaderProgram extends FakeGlShaderProgram { + private final int outputWidth; + private final int outputHeight; + + SizeChangingGlShaderProgram(int outputWidth, int outputHeight) { + this.outputWidth = outputWidth; + this.outputHeight = outputHeight; + } + + @Override + public void queueInputFrame( + GlObjectsProvider glObjectsProvider, GlTextureInfo inputTexture, long presentationTimeUs) { + GlTextureInfo sizeChangedInputTexture = + new GlTextureInfo( + /* texId= */ inputTexture.texId, + /* fboId= */ inputTexture.fboId, + /* rboId= */ inputTexture.rboId, + /* width= */ outputWidth, + /* height= */ outputHeight); + super.queueInputFrame(glObjectsProvider, sizeChangedInputTexture, presentationTimeUs); + } + } } diff --git a/libraries/effect/src/test/java/androidx/media3/effect/GlslShaderResourcesTest.java b/libraries/effect/src/test/java/androidx/media3/effect/GlslShaderResourcesTest.java new file mode 100644 index 00000000000..533502a8d4f --- /dev/null +++ b/libraries/effect/src/test/java/androidx/media3/effect/GlslShaderResourcesTest.java @@ -0,0 +1,69 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect; + +import static com.google.common.truth.Truth.assertThat; + +import android.content.Context; +import android.content.res.Resources; +import android.util.TypedValue; +import androidx.media3.common.util.Util; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableMap; +import com.google.common.collect.Maps; +import java.io.IOException; +import java.lang.reflect.Field; +import java.lang.reflect.Modifier; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Tests for the GLSL shaders that are packaged as raw resources. */ +@RunWith(AndroidJUnit4.class) +public final class GlslShaderResourcesTest { + + @Test + public void loadRawResource_glslShaders_haveNoComments() throws Exception { + Context context = ApplicationProvider.getApplicationContext(); + + ImmutableMap shadersByName = loadGlslRawResources(context); + + assertThat(shadersByName).isNotEmpty(); + assertThat(Maps.filterValues(shadersByName, GlslShaderResourcesTest::containsComment).keySet()) + .isEmpty(); + } + + private static ImmutableMap loadGlslRawResources(Context context) + throws IllegalAccessException, IOException { + Resources resources = context.getResources(); + TypedValue value = new TypedValue(); + ImmutableMap.Builder shadersByName = ImmutableMap.builder(); + for (Field field : R.raw.class.getFields()) { + if (Modifier.isStatic(field.getModifiers()) && field.getType() == int.class) { + int resourceId = field.getInt(/* obj= */ null); + resources.getValue(resourceId, value, /* resolveRefs= */ true); + if (String.valueOf(value.string).endsWith(".glsl")) { + shadersByName.put(field.getName(), Util.loadRawResource(context, resourceId)); + } + } + } + return shadersByName.buildOrThrow(); + } + + private static boolean containsComment(String shader) { + return shader.contains("//") || shader.contains("/*"); + } +} diff --git a/libraries/effect/src/test/java/androidx/media3/effect/HardwareBufferFrameTest.java b/libraries/effect/src/test/java/androidx/media3/effect/HardwareBufferFrameTest.java deleted file mode 100644 index 2a1646f1bbe..00000000000 --- a/libraries/effect/src/test/java/androidx/media3/effect/HardwareBufferFrameTest.java +++ /dev/null @@ -1,229 +0,0 @@ -/* - * Copyright 2026 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package androidx.media3.effect; - -import static com.google.common.truth.Truth.assertThat; -import static com.google.common.util.concurrent.MoreExecutors.directExecutor; -import static org.junit.Assert.assertThrows; - -import androidx.test.ext.junit.runners.AndroidJUnit4; -import java.util.concurrent.atomic.AtomicBoolean; -import java.util.concurrent.atomic.AtomicInteger; -import org.junit.Test; -import org.junit.runner.RunWith; - -/** Unit tests for {@link HardwareBufferFrame}. */ -@RunWith(AndroidJUnit4.class) -public class HardwareBufferFrameTest { - - @Test - public void releaseWithoutRetain_releasesFrame() { - AtomicBoolean isReleased = new AtomicBoolean(false); - HardwareBufferFrame frame = - new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, - directExecutor(), - (releaseFence) -> isReleased.set(true)) - .setInternalFrame(new Object()) - .build(); - - frame.release(/* releaseFence= */ null); - - assertThat(isReleased.get()).isTrue(); - } - - @Test - public void releaseAfterRetain_doesNotReleaseFrame() { - AtomicBoolean isReleased = new AtomicBoolean(false); - HardwareBufferFrame frame = - new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, - directExecutor(), - (releaseFence) -> isReleased.set(true)) - .setInternalFrame(new Object()) - .build(); - - HardwareBufferFrame unused = frame.retain(); - frame.release(/* releaseFence= */ null); - - assertThat(isReleased.get()).isFalse(); - } - - @Test - public void matchingReleaseAndRetainCalls_releasesFrame() { - AtomicBoolean isReleased = new AtomicBoolean(false); - HardwareBufferFrame frame = - new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, - directExecutor(), - (releaseFence) -> isReleased.set(true)) - .setInternalFrame(new Object()) - .build(); - - HardwareBufferFrame retainedFrame = frame.retain(); - - frame.release(/* releaseFence= */ null); - assertThat(isReleased.get()).isFalse(); - - retainedFrame.release(/* releaseFence= */ null); - assertThat(isReleased.get()).isTrue(); - } - - @Test - public void release_idempotentOnSameHandle() { - AtomicInteger releaseCount = new AtomicInteger(0); - HardwareBufferFrame frame = - new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, - directExecutor(), - (releaseFence) -> releaseCount.incrementAndGet()) - .setInternalFrame(new Object()) - .build(); - - frame.release(/* releaseFence= */ null); - assertThat(releaseCount.get()).isEqualTo(1); - - // Second release on the same handle should have no effect. - frame.release(/* releaseFence= */ null); - assertThat(releaseCount.get()).isEqualTo(1); - } - - @Test - public void retainAfterRelease_throwsIllegalStateException() { - HardwareBufferFrame frame = - new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, directExecutor(), (releaseFence) -> {}) - .setInternalFrame(new Object()) - .build(); - frame.release(/* releaseFence= */ null); - - assertThrows(IllegalStateException.class, frame::retain); - } - - @Test - public void release_multipleTimesOnSingleHandle_doesNotReleaseUnderlyingFrameIfRetained() { - AtomicInteger releaseCount = new AtomicInteger(0); - HardwareBufferFrame frame = - new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, - directExecutor(), - (releaseFence) -> releaseCount.incrementAndGet()) - .setInternalFrame(new Object()) - .build(); - - HardwareBufferFrame retainedFrame = frame.retain(); - - // Releasing the original handle multiple times. - frame.release(/* releaseFence= */ null); - frame.release(/* releaseFence= */ null); - - // Should not be released because retainedFrame is still active. - assertThat(releaseCount.get()).isEqualTo(0); - - // Releasing the retained handle should trigger the final release. - retainedFrame.release(/* releaseFence= */ null); - assertThat(releaseCount.get()).isEqualTo(1); - } - - @Test - public void retain_multipleLevels_releasesOnlyAfterAllReleased() { - AtomicInteger releaseCount = new AtomicInteger(0); - HardwareBufferFrame frame = - new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, - directExecutor(), - (releaseFence) -> releaseCount.incrementAndGet()) - .setInternalFrame(new Object()) - .build(); - - // Create a chain of retained handles. - HardwareBufferFrame retained1 = frame.retain(); - HardwareBufferFrame retained2 = retained1.retain(); - - frame.release(/* releaseFence= */ null); - retained1.release(/* releaseFence= */ null); - assertThat(releaseCount.get()).isEqualTo(0); - - retained2.release(/* releaseFence= */ null); - assertThat(releaseCount.get()).isEqualTo(1); - } - - @Test - public void buildUpon_builtFrameSharesLifetimeWithOriginalFrame() { - AtomicInteger releaseCount = new AtomicInteger(0); - HardwareBufferFrame frame1 = - new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, - directExecutor(), - (releaseFence) -> releaseCount.incrementAndGet()) - .setInternalFrame(new Object()) - .build(); - - HardwareBufferFrame frame2 = frame1.buildUpon().build(); - - frame1.release(/* releaseFence= */ null); - assertThat(releaseCount.get()).isEqualTo(1); - - // Releasing frame2 should be a no-op as frame is already released. - frame2.release(/* releaseFence= */ null); - assertThat(releaseCount.get()).isEqualTo(1); - } - - @Test - public void buildUpon_originalFrameSharesLifetimeWithBuiltFrame() { - AtomicInteger releaseCount = new AtomicInteger(0); - HardwareBufferFrame frame1 = - new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, - directExecutor(), - (releaseFence) -> releaseCount.incrementAndGet()) - .setInternalFrame(new Object()) - .build(); - - HardwareBufferFrame frame2 = frame1.buildUpon().build(); - - frame2.release(/* releaseFence= */ null); - assertThat(releaseCount.get()).isEqualTo(1); - - // Releasing frame1 should be a no-op as frame is already released. - frame1.release(/* releaseFence= */ null); - assertThat(releaseCount.get()).isEqualTo(1); - } - - @Test - public void buildUpon_multipleBuilds_allFramesShareLifetime() { - AtomicInteger releaseCount = new AtomicInteger(0); - HardwareBufferFrame frame1 = - new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, - directExecutor(), - (releaseFence) -> releaseCount.incrementAndGet()) - .setInternalFrame(new Object()) - .build(); - - HardwareBufferFrame.Builder builder = frame1.buildUpon(); - HardwareBufferFrame frame2 = builder.build(); - HardwareBufferFrame frame3 = builder.build(); - - frame2.release(/* releaseFence= */ null); - assertThat(releaseCount.get()).isEqualTo(1); - - frame1.release(/* releaseFence= */ null); - assertThat(releaseCount.get()).isEqualTo(1); - frame3.release(/* releaseFence= */ null); - assertThat(releaseCount.get()).isEqualTo(1); - } -} diff --git a/libraries/effect/src/test/java/androidx/media3/effect/ImagePlanesToGlTextureConverterTest.java b/libraries/effect/src/test/java/androidx/media3/effect/ImagePlanesToGlTextureConverterTest.java new file mode 100644 index 00000000000..9a4c1c3eda1 --- /dev/null +++ b/libraries/effect/src/test/java/androidx/media3/effect/ImagePlanesToGlTextureConverterTest.java @@ -0,0 +1,330 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect; + +import static androidx.media3.effect.DefaultGlFrameProcessor.BT2020_LINEAR; +import static androidx.media3.effect.DefaultGlFrameProcessor.BT709_SRGB; +import static androidx.test.core.app.ApplicationProvider.getApplicationContext; +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.util.concurrent.MoreExecutors.directExecutor; +import static org.junit.Assert.assertThrows; + +import android.graphics.PixelFormat; +import androidx.media3.common.Format; +import androidx.media3.common.video.DefaultImagePlanesFrame; +import androidx.media3.common.video.DefaultImagePlanesFrame.DefaultPlane; +import androidx.media3.common.video.FrameProcessor; +import androidx.media3.common.video.ImagePlanesFrame; +import androidx.media3.common.video.ImagePlanesFrame.Plane; +import androidx.media3.test.utils.FakeFrameProcessor; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import java.nio.ByteBuffer; +import java.util.concurrent.Executor; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** + * Unit tests for {@link ImagePlanesToGlTextureConverter}. + * + *

Converting valid frames requires an OpenGL ES context, so happy paths are tested in the {@code + * androidTest} instrumentation tests. + */ +@RunWith(AndroidJUnit4.class) +public final class ImagePlanesToGlTextureConverterTest { + + private static final Executor DIRECT_EXECUTOR = directExecutor(); + private static final FrameProcessor.Listener LISTENER = new FakeFrameProcessor.Listener(); + private static final int DEFAULT_WIDTH = 4; + private static final int DEFAULT_HEIGHT = 4; + private static final int RGBA_BYTES_PER_PIXEL = 4; + private static final int DEFAULT_RGBA_ROW_STRIDE = DEFAULT_WIDTH * RGBA_BYTES_PER_PIXEL; + private static final Plane DEFAULT_RGBA_PLANE = + new DefaultPlane( + ByteBuffer.allocateDirect(DEFAULT_RGBA_ROW_STRIDE * DEFAULT_HEIGHT), + DEFAULT_RGBA_ROW_STRIDE, + /* pixelStride= */ RGBA_BYTES_PER_PIXEL); + + private ImagePlanesToGlTextureConverter converter; + + @Before + public void setUp() { + converter = + new ImagePlanesToGlTextureConverter( + getApplicationContext(), BT709_SRGB, /* errorConsumer= */ error -> {}); + } + + @Test + public void convert_whenClosed_returnsNull() throws Exception { + converter.close(); + ImagePlanesFrame frame = + new DefaultImagePlanesFrame.Builder(ImmutableList.of(DEFAULT_RGBA_PLANE)) + .setFormat( + new Format.Builder().setWidth(DEFAULT_WIDTH).setHeight(DEFAULT_HEIGHT).build()) + .build(); + + assertThat( + converter.convert( + frame, + /* glExecutor= */ DIRECT_EXECUTOR, + /* listenerExecutor= */ DIRECT_EXECUTOR, + LISTENER)) + .isNull(); + } + + @Test + public void convert_unsetFrameDimensions_throwsIllegalArgumentException() { + ImagePlanesFrame frame = + new DefaultImagePlanesFrame.Builder(ImmutableList.of(DEFAULT_RGBA_PLANE)) + .setFormat(new Format.Builder().setPixelFormat(PixelFormat.RGBA_8888).build()) + .build(); + + assertThrows( + IllegalArgumentException.class, + () -> + converter.convert( + frame, + /* glExecutor= */ DIRECT_EXECUTOR, + /* listenerExecutor= */ DIRECT_EXECUTOR, + LISTENER)); + } + + @Test + public void convert_unsetPixelFormat_throwsUnsupportedOperationException() { + Format format = new Format.Builder().setWidth(DEFAULT_WIDTH).setHeight(DEFAULT_HEIGHT).build(); + ImagePlanesFrame frame = + new DefaultImagePlanesFrame.Builder(ImmutableList.of(DEFAULT_RGBA_PLANE)) + .setFormat(format) + .build(); + + assertThrows( + UnsupportedOperationException.class, + () -> + converter.convert( + frame, + /* glExecutor= */ DIRECT_EXECUTOR, + /* listenerExecutor= */ DIRECT_EXECUTOR, + LISTENER)); + } + + @Test + public void convert_unsupportedPixelFormat_throwsUnsupportedOperationException() { + Format format = + new Format.Builder() + .setWidth(DEFAULT_WIDTH) + .setHeight(DEFAULT_HEIGHT) + .setPixelFormat(PixelFormat.RGB_565) + .build(); + ImagePlanesFrame frame = + new DefaultImagePlanesFrame.Builder(ImmutableList.of(DEFAULT_RGBA_PLANE)) + .setFormat(format) + .build(); + + assertThrows( + UnsupportedOperationException.class, + () -> + converter.convert( + frame, + /* glExecutor= */ DIRECT_EXECUTOR, + /* listenerExecutor= */ DIRECT_EXECUTOR, + LISTENER)); + } + + @Test + public void convert_bt2020ColorSpace_throwsIllegalArgumentException() { + Format format = + new Format.Builder() + .setWidth(DEFAULT_WIDTH) + .setHeight(DEFAULT_HEIGHT) + .setPixelFormat(PixelFormat.RGBA_8888) + .setColorInfo(BT2020_LINEAR) + .build(); + ImagePlanesFrame frame = + new DefaultImagePlanesFrame.Builder(ImmutableList.of(DEFAULT_RGBA_PLANE)) + .setFormat(format) + .build(); + + assertThrows( + IllegalArgumentException.class, + () -> + converter.convert( + frame, + /* glExecutor= */ DIRECT_EXECUTOR, + /* listenerExecutor= */ DIRECT_EXECUTOR, + LISTENER)); + } + + @Test + public void convert_multiplePlanesForRgba_throwsIllegalArgumentException() { + Format format = + new Format.Builder() + .setWidth(DEFAULT_WIDTH) + .setHeight(DEFAULT_HEIGHT) + .setPixelFormat(PixelFormat.RGBA_8888) + .build(); + ImagePlanesFrame frame = + new DefaultImagePlanesFrame.Builder( + ImmutableList.of(DEFAULT_RGBA_PLANE, DEFAULT_RGBA_PLANE)) + .setFormat(format) + .build(); + + assertThrows( + IllegalArgumentException.class, + () -> + converter.convert( + frame, + /* glExecutor= */ DIRECT_EXECUTOR, + /* listenerExecutor= */ DIRECT_EXECUTOR, + LISTENER)); + } + + @Test + public void convert_invalidPixelStride_throwsIllegalArgumentException() { + Format format = + new Format.Builder() + .setWidth(DEFAULT_WIDTH) + .setHeight(DEFAULT_HEIGHT) + .setPixelFormat(PixelFormat.RGBA_8888) + .build(); + // RGBA_8888 requires tightly-packed 4-byte pixels (pixelStride == RGBA_BYTES_PER_PIXEL). + int invalidPixelStride = RGBA_BYTES_PER_PIXEL * 2; + int rowStride = DEFAULT_WIDTH * invalidPixelStride; + Plane plane = + new DefaultPlane( + ByteBuffer.allocateDirect(rowStride * DEFAULT_HEIGHT), rowStride, invalidPixelStride); + ImagePlanesFrame frame = + new DefaultImagePlanesFrame.Builder(ImmutableList.of(plane)).setFormat(format).build(); + + assertThrows( + IllegalArgumentException.class, + () -> + converter.convert( + frame, + /* glExecutor= */ DIRECT_EXECUTOR, + /* listenerExecutor= */ DIRECT_EXECUTOR, + LISTENER)); + } + + @Test + public void convert_rowStrideSmallerThanExpectedRowBytes_throwsIllegalArgumentException() { + Format format = + new Format.Builder() + .setWidth(DEFAULT_WIDTH) + .setHeight(DEFAULT_HEIGHT) + .setPixelFormat(PixelFormat.RGBA_8888) + .build(); + // Subtract a full pixel so rowStride remains a multiple of RGBA_BYTES_PER_PIXEL. + Plane plane = + new DefaultPlane( + ByteBuffer.allocateDirect(DEFAULT_RGBA_ROW_STRIDE * DEFAULT_HEIGHT), + /* rowStride= */ DEFAULT_RGBA_ROW_STRIDE - RGBA_BYTES_PER_PIXEL, + /* pixelStride= */ RGBA_BYTES_PER_PIXEL); + ImagePlanesFrame frame = + new DefaultImagePlanesFrame.Builder(ImmutableList.of(plane)).setFormat(format).build(); + + assertThrows( + IllegalArgumentException.class, + () -> + converter.convert( + frame, + /* glExecutor= */ DIRECT_EXECUTOR, + /* listenerExecutor= */ DIRECT_EXECUTOR, + LISTENER)); + } + + @Test + public void convert_rowStrideNotMultipleOfBytesPerPixel_throwsIllegalArgumentException() { + Format format = + new Format.Builder() + .setWidth(DEFAULT_WIDTH) + .setHeight(DEFAULT_HEIGHT) + .setPixelFormat(PixelFormat.RGBA_8888) + .build(); + int unalignedRowStride = DEFAULT_RGBA_ROW_STRIDE + 1; + Plane plane = + new DefaultPlane( + ByteBuffer.allocateDirect(unalignedRowStride * DEFAULT_HEIGHT), + unalignedRowStride, + /* pixelStride= */ RGBA_BYTES_PER_PIXEL); + ImagePlanesFrame frame = + new DefaultImagePlanesFrame.Builder(ImmutableList.of(plane)).setFormat(format).build(); + + assertThrows( + IllegalArgumentException.class, + () -> + converter.convert( + frame, + /* glExecutor= */ DIRECT_EXECUTOR, + /* listenerExecutor= */ DIRECT_EXECUTOR, + LISTENER)); + } + + @Test + public void convert_undersizedPlaneBuffer_throwsIllegalArgumentException() { + Format format = + new Format.Builder() + .setWidth(DEFAULT_WIDTH) + .setHeight(DEFAULT_HEIGHT) + .setPixelFormat(PixelFormat.RGBA_8888) + .build(); + Plane plane = + new DefaultPlane( + ByteBuffer.allocateDirect(DEFAULT_RGBA_ROW_STRIDE * DEFAULT_HEIGHT - 1), + DEFAULT_RGBA_ROW_STRIDE, + /* pixelStride= */ RGBA_BYTES_PER_PIXEL); + ImagePlanesFrame frame = + new DefaultImagePlanesFrame.Builder(ImmutableList.of(plane)).setFormat(format).build(); + + assertThrows( + IllegalArgumentException.class, + () -> + converter.convert( + frame, + /* glExecutor= */ DIRECT_EXECUTOR, + /* listenerExecutor= */ DIRECT_EXECUTOR, + LISTENER)); + } + + @Test + public void convert_planeBufferEndingBeforeLastVisiblePixel_throwsIllegalArgumentException() { + Format format = + new Format.Builder() + .setWidth(DEFAULT_WIDTH) + .setHeight(DEFAULT_HEIGHT) + .setPixelFormat(PixelFormat.RGBA_8888) + .build(); + int rowBytes = DEFAULT_WIDTH * RGBA_BYTES_PER_PIXEL; + int paddedRowStride = rowBytes + RGBA_BYTES_PER_PIXEL; + // The final row only needs to extend to the last visible pixel, so this is one byte short. + Plane plane = + new DefaultPlane( + ByteBuffer.allocateDirect(paddedRowStride * (DEFAULT_HEIGHT - 1) + rowBytes - 1), + paddedRowStride, + /* pixelStride= */ RGBA_BYTES_PER_PIXEL); + ImagePlanesFrame frame = + new DefaultImagePlanesFrame.Builder(ImmutableList.of(plane)).setFormat(format).build(); + + assertThrows( + IllegalArgumentException.class, + () -> + converter.convert( + frame, + /* glExecutor= */ DIRECT_EXECUTOR, + /* listenerExecutor= */ DIRECT_EXECUTOR, + LISTENER)); + } +} diff --git a/libraries/effect_ndk/README.md b/libraries/effect_ndk/README.md index 58389be592c..2b1a55c4996 100644 --- a/libraries/effect_ndk/README.md +++ b/libraries/effect_ndk/README.md @@ -1,28 +1,9 @@ -# Effect NDK module +# Effect NDK module (deprecated) Provides functionality for `lib-effect`, that depends on native code. -## Getting the module - -The easiest way to get the module is to add it as a gradle dependency: - -```kotlin -implementation("androidx.media3:media3-effect-ndk:1.X.X") -``` - -Or in Gradle Groovy DSL `build.gradle`: - -```groovy -implementation 'androidx.media3:media3-effect-ndk:1.X.X' -``` - -where `1.X.X` is the version, which must match the version of the other media -modules being used. - -Alternatively, you can clone this GitHub project and depend on the module -locally. Instructions for doing this can be found in the [top level README][]. - -[top level README]: ../../README.md +Deprecated: This module is deprecated. All functionality has been moved to +`lib-effect`. Application code should depend on `lib-effect` directly. ## Links diff --git a/libraries/effect_ndk/build.gradle.kts b/libraries/effect_ndk/build.gradle.kts index 49b086e923a..5f57d86d8f3 100644 --- a/libraries/effect_ndk/build.gradle.kts +++ b/libraries/effect_ndk/build.gradle.kts @@ -14,33 +14,11 @@ plugins { id("media3.android-library") } -android { - namespace = "androidx.media3.effect.ndk" - - defaultConfig { - externalNativeBuild { - cmake { - arguments.add("-DANDROID_WEAK_API_DEFS=ON") - arguments.add("-Werror=unguarded-availability") - targets.add("hardwareBufferJNI") - } - } - } - externalNativeBuild { cmake { path = file("src/main/jni/CMakeLists.txt") } } - - sourceSets { - getByName("androidTest").assets.directories.add("../test_data/src/test/assets/") - getByName("test").assets.directories.add("../test_data/src/test/assets/") - } -} +android { namespace = "androidx.media3.effect.ndk" } dependencies { implementation(libs.androidx.annotation) api(project(":lib-common")) api(project(":lib-effect")) compileOnly(libs.errorprone.annotations) - androidTestImplementation(libs.junit) - androidTestImplementation(libs.androidx.test.runner) - androidTestImplementation(libs.truth) - androidTestImplementation(project(":test-utils")) } diff --git a/libraries/effect_ndk/src/main/java/androidx/media3/effect/ndk/HardwareBufferJni.java b/libraries/effect_ndk/src/main/java/androidx/media3/effect/ndk/HardwareBufferJni.java index e10a7e670d9..a0f28b34eee 100644 --- a/libraries/effect_ndk/src/main/java/androidx/media3/effect/ndk/HardwareBufferJni.java +++ b/libraries/effect_ndk/src/main/java/androidx/media3/effect/ndk/HardwareBufferJni.java @@ -15,48 +15,82 @@ */ package androidx.media3.effect.ndk; -import static com.google.common.base.Preconditions.checkState; - import android.graphics.Bitmap; import android.hardware.HardwareBuffer; import androidx.annotation.RequiresApi; import androidx.media3.common.util.ExperimentalApi; -import androidx.media3.common.util.LibraryLoader; import androidx.media3.effect.HardwareBufferJniWrapper; +import com.google.errorprone.annotations.InlineMe; -/** JNI methods for HardwareBuffer interaction. */ +/** + * JNI methods for HardwareBuffer interaction. + * + * @deprecated Use {@link androidx.media3.effect.HardwareBufferJni} instead. + */ @RequiresApi(26) +@Deprecated @ExperimentalApi // TODO: b/449956776 - Remove once FrameConsumer API is finalized. public final class HardwareBufferJni implements HardwareBufferJniWrapper { - private static final LibraryLoader LOADER = - new LibraryLoader("hardwareBufferJNI") { - @Override - protected void loadLibrary(String name) { - System.loadLibrary(name); - } - }; + /** + * @deprecated Use {@link androidx.media3.effect.HardwareBufferJni#INSTANCE} instead. + */ + @Deprecated public static final HardwareBufferJni INSTANCE = new HardwareBufferJni(); - public static final HardwareBufferJni INSTANCE = new HardwareBufferJni(); + private HardwareBufferJni() {} - private HardwareBufferJni() { - checkState(LOADER.isAvailable()); - } - - /** Creates an EGLImage from a {@link HardwareBuffer}. */ + /** + * Creates an EGLImage from a {@link HardwareBuffer}. + * + * @deprecated Use {@link + * androidx.media3.effect.HardwareBufferJni#nativeCreateEglImageFromHardwareBuffer(long, + * HardwareBuffer)} instead. + */ + @InlineMe( + replacement = + "androidx.media3.effect.HardwareBufferJni.INSTANCE.nativeCreateEglImageFromHardwareBuffer(displayHandle," + + " hardwareBuffer)") + @Deprecated @Override - public native long nativeCreateEglImageFromHardwareBuffer( - long displayHandle, HardwareBuffer hardwareBuffer); + public long nativeCreateEglImageFromHardwareBuffer( + long displayHandle, HardwareBuffer hardwareBuffer) { + return androidx.media3.effect.HardwareBufferJni.INSTANCE.nativeCreateEglImageFromHardwareBuffer( + displayHandle, hardwareBuffer); + } /** * Binds an EGLImage to the specified texture target. Returns whether the binding is successful. + * + * @deprecated Use {@link androidx.media3.effect.HardwareBufferJni#nativeBindEGLImage(int, long)} + * instead. */ + @InlineMe( + replacement = + "androidx.media3.effect.HardwareBufferJni.INSTANCE.nativeBindEGLImage(target," + + " eglImageHandle)") + @Deprecated @Override - public native boolean nativeBindEGLImage(int target, long eglImageHandle); + public boolean nativeBindEGLImage(int target, long eglImageHandle) { + return androidx.media3.effect.HardwareBufferJni.INSTANCE.nativeBindEGLImage( + target, eglImageHandle); + } - /** Destroys an EGLImage. Returns whether the deletion is successful. */ + /** + * Destroys an EGLImage. Returns whether the deletion is successful. + * + * @deprecated Use {@link androidx.media3.effect.HardwareBufferJni#nativeDestroyEGLImage(long, + * long)} instead. + */ + @InlineMe( + replacement = + "androidx.media3.effect.HardwareBufferJni.INSTANCE.nativeDestroyEGLImage(displayHandle," + + " imageHandle)") + @Deprecated @Override - public native boolean nativeDestroyEGLImage(long displayHandle, long imageHandle); + public boolean nativeDestroyEGLImage(long displayHandle, long imageHandle) { + return androidx.media3.effect.HardwareBufferJni.INSTANCE.nativeDestroyEGLImage( + displayHandle, imageHandle); + } /** * Copies a {@link Bitmap} to a {@link HardwareBuffer}. Returns whether the copy is successful. @@ -68,9 +102,21 @@ public native long nativeCreateEglImageFromHardwareBuffer( * HardwareBuffer#getFormat}, and be either {@link Bitmap.Config#ARGB_8888} and {@link * HardwareBuffer#RGBA_8888} or {@link Bitmap.Config#RGBA_1010102} and {@link * HardwareBuffer#RGBA_1010102}. + * + * @deprecated Use {@link + * androidx.media3.effect.HardwareBufferJni#nativeCopyBitmapToHardwareBuffer(Bitmap, + * HardwareBuffer)} instead. */ + @InlineMe( + replacement = + "androidx.media3.effect.HardwareBufferJni.INSTANCE.nativeCopyBitmapToHardwareBuffer(bitmap," + + " hb)") + @Deprecated @Override - public native boolean nativeCopyBitmapToHardwareBuffer(Bitmap bitmap, HardwareBuffer hb); + public boolean nativeCopyBitmapToHardwareBuffer(Bitmap bitmap, HardwareBuffer hb) { + return androidx.media3.effect.HardwareBufferJni.INSTANCE.nativeCopyBitmapToHardwareBuffer( + bitmap, hb); + } /** * Copies the contents of a source {@link HardwareBuffer} to a destination {@link HardwareBuffer}. @@ -82,8 +128,20 @@ public native long nativeCreateEglImageFromHardwareBuffer( * *

The formats of the source and destination buffers must match, and be either {@link * HardwareBuffer#RGBA_8888} or {@link HardwareBuffer#RGBA_1010102}. + * + * @deprecated Use {@link + * androidx.media3.effect.HardwareBufferJni#nativeCopyHardwareBufferToHardwareBuffer(HardwareBuffer, + * HardwareBuffer)} instead. */ + @InlineMe( + replacement = + "androidx.media3.effect.HardwareBufferJni.INSTANCE.nativeCopyHardwareBufferToHardwareBuffer(srcHb," + + " dstHb)") + @Deprecated @Override - public native boolean nativeCopyHardwareBufferToHardwareBuffer( - HardwareBuffer srcHb, HardwareBuffer dstHb); + public boolean nativeCopyHardwareBufferToHardwareBuffer( + HardwareBuffer srcHb, HardwareBuffer dstHb) { + return androidx.media3.effect.HardwareBufferJni.INSTANCE + .nativeCopyHardwareBufferToHardwareBuffer(srcHb, dstHb); + } } diff --git a/libraries/effect_play_services/README.md b/libraries/effect_play_services/README.md new file mode 100644 index 00000000000..4b8542c96f7 --- /dev/null +++ b/libraries/effect_play_services/README.md @@ -0,0 +1,35 @@ +# Effect Play services module + +Provides functionality for applying Google Play services video enhancement +effects to video frames using +[`EnhancementSession`](https://developers.google.com/android/reference/com/google/android/gms/media/effect/enhancement/EnhancementSession). + +## Getting the module + + + +The easiest way to get the module is to add it as a gradle dependency: + +```kotlin +implementation("androidx.media3:media3-effect-play-services:1.X.X") +``` + +Or in Gradle Groovy DSL `build.gradle`: + +```groovy +implementation 'androidx.media3:media3-effect-play-services:1.X.X' +``` + +where `1.X.X` is the version, which must match the version of the other media +modules being used. + +Alternatively, you can clone this GitHub project and depend on the module +locally. Instructions for doing this can be found in the [top level README][]. + +[top level README]: ../../README.md + +## Links + +* [Javadoc][] + +[Javadoc]: https://developer.android.com/reference/androidx/media3/effect/playservices/package-summary diff --git a/libraries/effect_play_services/build.gradle.kts b/libraries/effect_play_services/build.gradle.kts new file mode 100644 index 00000000000..6549b07b3b0 --- /dev/null +++ b/libraries/effect_play_services/build.gradle.kts @@ -0,0 +1,43 @@ +// Copyright 2026 The Android Open Source Project +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. + +plugins { id("media3.android-library") } + +android { + namespace = "androidx.media3.effect.playservices" + + sourceSets { getByName("androidTest").assets.directories.add("../test_data/src/test/assets/") } +} + +dependencies { + implementation(libs.androidx.annotation) + implementation(libs.guava) + api(project(":lib-common")) + implementation(project(":lib-effect")) + implementation(libs.play.services.media.effect.enhancement) + + testImplementation(project(":test-utils")) + testImplementation(libs.androidx.test.ext.junit) + testImplementation(libs.robolectric) + testImplementation(libs.truth) + + androidTestImplementation(project(":lib-effect")) + androidTestImplementation(project(":lib-extractor")) + androidTestImplementation(project(":lib-transformer")) + androidTestImplementation(project(":test-utils")) + androidTestImplementation(project(":lib-inspector-frame")) + androidTestImplementation(libs.androidx.test.runner) + androidTestImplementation(libs.androidx.test.ext.junit) + androidTestImplementation(libs.truth) +} diff --git a/libraries/effect_ndk/src/androidTest/AndroidManifest.xml b/libraries/effect_play_services/src/androidTest/AndroidManifest.xml similarity index 75% rename from libraries/effect_ndk/src/androidTest/AndroidManifest.xml rename to libraries/effect_play_services/src/androidTest/AndroidManifest.xml index 2febab19ea9..62889bb05ce 100644 --- a/libraries/effect_ndk/src/androidTest/AndroidManifest.xml +++ b/libraries/effect_play_services/src/androidTest/AndroidManifest.xml @@ -13,19 +13,15 @@ See the License for the specific language governing permissions and limitations under the License. --> - - + - + + diff --git a/libraries/effect_play_services/src/androidTest/java/androidx/media3/effect/playservices/FrameToSurfaceFrameWriterTest.java b/libraries/effect_play_services/src/androidTest/java/androidx/media3/effect/playservices/FrameToSurfaceFrameWriterTest.java new file mode 100644 index 00000000000..0d0da71e9ff --- /dev/null +++ b/libraries/effect_play_services/src/androidTest/java/androidx/media3/effect/playservices/FrameToSurfaceFrameWriterTest.java @@ -0,0 +1,279 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect.playservices; + +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; + +import android.graphics.PixelFormat; +import android.media.Image; +import android.media.ImageReader; +import androidx.media3.common.C; +import androidx.media3.common.Format; +import androidx.media3.common.VideoFrameProcessingException; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.DefaultHardwareBufferFrame; +import androidx.media3.common.video.Frame; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import androidx.test.filters.SdkSuppress; +import com.google.common.collect.ImmutableMap; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicLong; +import java.util.concurrent.atomic.AtomicReference; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Hermetic unit tests for {@link FrameToSurfaceFrameWriter}. */ +@RunWith(AndroidJUnit4.class) +@SdkSuppress(minSdkVersion = 33) +public final class FrameToSurfaceFrameWriterTest { + + private static final int WIDTH = 16; + private static final int HEIGHT = 16; + + private ImageReader imageReader; + private TestListener testListener; + private FrameToSurfaceFrameWriter frameWriter; + + @Before + public void setUp() { + imageReader = ImageReader.newInstance(WIDTH, HEIGHT, PixelFormat.RGBA_8888, 4); + testListener = new TestListener(); + frameWriter = new FrameToSurfaceFrameWriter(Runnable::run, testListener); + frameWriter.setOutputSurface(imageReader.getSurface(), WIDTH, HEIGHT); + } + + @After + public void tearDown() { + frameWriter.close(); + imageReader.close(); + } + + @Test + public void configure_forwardsFormatToListener() { + Format format = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + frameWriter.configure(format, Frame.USAGE_GPU_COLOR_OUTPUT); + + assertThat(testListener.configuredFormat.get()).isEqualTo(format); + } + + @Test + public void configure_calledMultipleTimes_throwsIllegalStateException() { + Format format = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + frameWriter.configure(format, Frame.USAGE_GPU_COLOR_OUTPUT); + + assertThrows( + IllegalStateException.class, + () -> frameWriter.configure(format, Frame.USAGE_GPU_COLOR_OUTPUT)); + } + + @Test + public void dequeueInputFrame_whenAtCapacity_returnsNullUntilReleased() { + Format format = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + frameWriter.configure(format, Frame.USAGE_GPU_COLOR_OUTPUT); + + AsyncFrame firstFrame = frameWriter.dequeueInputFrame(Runnable::run, () -> {}); + assertThat(firstFrame).isNotNull(); + frameWriter.queueInputFrame(firstFrame.frame, /* writeCompleteFence= */ null); + + // At capacity (1 in-flight frame), next dequeue returns null. + assertThat(frameWriter.dequeueInputFrame(Runnable::run, () -> {})).isNull(); + + try (Image outputImage = imageReader.acquireLatestImage()) { + assertThat(outputImage).isNotNull(); + } + frameWriter.onFrameReleased(); + + AsyncFrame secondFrame = frameWriter.dequeueInputFrame(Runnable::run, () -> {}); + assertThat(secondFrame).isNotNull(); + } + + @Test + public void dequeueInputFrame_whenSurfaceNotConfigured_returnsNull() { + try (FrameToSurfaceFrameWriter unsetFrameWriter = + new FrameToSurfaceFrameWriter(Runnable::run, testListener)) { + AsyncFrame frame = unsetFrameWriter.dequeueInputFrame(Runnable::run, () -> {}); + + assertThat(frame).isNull(); + } + } + + @Test + public void dequeueInputFrame_whenAvailable_returnsAsyncFrameWrappingHardwareBuffer() { + Format format = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + frameWriter.configure(format, Frame.USAGE_GPU_COLOR_OUTPUT); + + AsyncFrame asyncFrame = frameWriter.dequeueInputFrame(Runnable::run, () -> {}); + + assertThat(asyncFrame).isNotNull(); + assertThat(asyncFrame.frame).isInstanceOf(DefaultHardwareBufferFrame.class); + assertThat(asyncFrame.frame.getFormat()).isEqualTo(format); + } + + @Test + public void onFrameReleased_triggersRegisteredWakeupListener() { + Format format = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + frameWriter.configure(format, Frame.USAGE_GPU_COLOR_OUTPUT); + + AsyncFrame firstFrame = frameWriter.dequeueInputFrame(Runnable::run, () -> {}); + assertThat(firstFrame).isNotNull(); + frameWriter.queueInputFrame(firstFrame.frame, /* writeCompleteFence= */ null); + + AtomicBoolean wakeupTriggered = new AtomicBoolean(false); + assertThat(frameWriter.dequeueInputFrame(Runnable::run, () -> wakeupTriggered.set(true))) + .isNull(); + + try (Image outputImage = imageReader.acquireLatestImage()) { + assertThat(outputImage).isNotNull(); + } + frameWriter.onFrameReleased(); + + assertThat(wakeupTriggered.get()).isTrue(); + } + + @Test + public void queueInputFrame_assignsMonotonicTimestampAndNotifiesListenerWithOriginalPts() { + Format format = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + frameWriter.configure(format, Frame.USAGE_GPU_COLOR_OUTPUT); + + AsyncFrame asyncFrame = frameWriter.dequeueInputFrame(Runnable::run, () -> {}); + assertThat(asyncFrame).isNotNull(); + DefaultHardwareBufferFrame dequeuedFrame = (DefaultHardwareBufferFrame) asyncFrame.frame; + DefaultHardwareBufferFrame frameWithTimestamp = + dequeuedFrame.buildUpon().setContentTimeUs(123_456L).build(); + frameWriter.queueInputFrame(frameWithTimestamp, /* writeCompleteFence= */ null); + + try (Image outputImage = imageReader.acquireLatestImage()) { + assertThat(outputImage).isNotNull(); + assertThat(outputImage.getTimestamp()).isEqualTo(0L); + assertThat(testListener.lastQueuedPtsUs.get()).isEqualTo(123_456L); + } + } + + @Test + public void + queueInputFrame_whenPresentationTimestampsDecreaseOnSeek_assignsIncreasingImageTimestamps() { + Format format = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + frameWriter.configure(format, Frame.USAGE_GPU_COLOR_OUTPUT); + + AsyncFrame firstAsyncFrame = frameWriter.dequeueInputFrame(Runnable::run, () -> {}); + assertThat(firstAsyncFrame).isNotNull(); + DefaultHardwareBufferFrame firstFrame = + ((DefaultHardwareBufferFrame) firstAsyncFrame.frame) + .buildUpon() + .setContentTimeUs(500_000L) + .build(); + frameWriter.queueInputFrame(firstFrame, /* writeCompleteFence= */ null); + + try (Image firstImage = imageReader.acquireLatestImage()) { + assertThat(firstImage).isNotNull(); + assertThat(firstImage.getTimestamp()).isEqualTo(0L); + assertThat(testListener.lastQueuedPtsUs.get()).isEqualTo(500_000L); + } + frameWriter.onFrameReleased(); + + // Simulate a seek backwards (500ms -> 100ms). + AsyncFrame secondAsyncFrame = frameWriter.dequeueInputFrame(Runnable::run, () -> {}); + assertThat(secondAsyncFrame).isNotNull(); + DefaultHardwareBufferFrame secondFrame = + ((DefaultHardwareBufferFrame) secondAsyncFrame.frame) + .buildUpon() + .setContentTimeUs(100_000L) + .build(); + frameWriter.queueInputFrame(secondFrame, /* writeCompleteFence= */ null); + + try (Image secondImage = imageReader.acquireLatestImage()) { + assertThat(secondImage).isNotNull(); + assertThat(secondImage.getTimestamp()).isEqualTo(1_000L); + assertThat(testListener.lastQueuedPtsUs.get()).isEqualTo(100_000L); + } + } + + @Test + public void queueInputFrame_whenClosed_reportsErrorToListener() { + Format format = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + frameWriter.configure(format, Frame.USAGE_GPU_COLOR_OUTPUT); + + AsyncFrame asyncFrame = frameWriter.dequeueInputFrame(Runnable::run, () -> {}); + assertThat(asyncFrame).isNotNull(); + frameWriter.close(); + + frameWriter.queueInputFrame(asyncFrame.frame, /* writeCompleteFence= */ null); + + assertThat(testListener.lastError.get()).isInstanceOf(VideoFrameProcessingException.class); + } + + @Test + public void queueInputFrame_withInvalidFrameType_throwsIllegalArgumentException() { + Frame invalidFrame = + new Frame() { + @Override + public Format getFormat() { + return new Format.Builder().build(); + } + + @Override + public ImmutableMap getMetadata() { + return ImmutableMap.of(); + } + + @Override + public long getContentTimeUs() { + return 0; + } + }; + + assertThrows( + IllegalArgumentException.class, + () -> frameWriter.queueInputFrame(invalidFrame, /* writeCompleteFence= */ null)); + } + + @Test + public void signalEndOfStream_notifiesListener() { + frameWriter.signalEndOfStream(); + + assertThat(testListener.endOfStreamSignaled.get()).isTrue(); + } + + private static class TestListener implements FrameToSurfaceFrameWriter.Listener { + final AtomicReference configuredFormat = new AtomicReference<>(); + final AtomicLong lastQueuedPtsUs = new AtomicLong(C.TIME_UNSET); + final AtomicBoolean endOfStreamSignaled = new AtomicBoolean(false); + final AtomicReference lastError = new AtomicReference<>(); + + @Override + public void onFormatConfigured(Format format) { + configuredFormat.set(format); + } + + @Override + public void onFrameQueued(long presentationTimeUs) { + lastQueuedPtsUs.set(presentationTimeUs); + } + + @Override + public void onError(VideoFrameProcessingException exception) { + lastError.set(exception); + } + + @Override + public void onEndOfStream() { + endOfStreamSignaled.set(true); + } + } +} diff --git a/libraries/effect_play_services/src/androidTest/java/androidx/media3/effect/playservices/SurfaceToFrameWriterAdapterTest.java b/libraries/effect_play_services/src/androidTest/java/androidx/media3/effect/playservices/SurfaceToFrameWriterAdapterTest.java new file mode 100644 index 00000000000..0c2874c1769 --- /dev/null +++ b/libraries/effect_play_services/src/androidTest/java/androidx/media3/effect/playservices/SurfaceToFrameWriterAdapterTest.java @@ -0,0 +1,415 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect.playservices; + +import static com.google.common.truth.Truth.assertThat; +import static java.util.concurrent.TimeUnit.MILLISECONDS; + +import android.graphics.PixelFormat; +import android.hardware.HardwareBuffer; +import android.media.Image; +import android.media.ImageWriter; +import android.os.Handler; +import android.os.HandlerThread; +import android.view.Surface; +import androidx.annotation.Nullable; +import androidx.media3.common.Format; +import androidx.media3.common.VideoFrameProcessingException; +import androidx.media3.common.video.DefaultHardwareBufferFrame; +import androidx.media3.test.utils.FakeFrameWriter; +import androidx.media3.test.utils.FakeHardwareBufferNativeHelpers; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import androidx.test.filters.SdkSuppress; +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.Callable; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.concurrent.atomic.AtomicReference; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Tests for {@link SurfaceToFrameWriterAdapter}. */ +@RunWith(AndroidJUnit4.class) +@SdkSuppress(minSdkVersion = 33) +public final class SurfaceToFrameWriterAdapterTest { + + private static final int WIDTH = 128; + private static final int HEIGHT = 128; + private static final int TIMEOUT_MS = 1_000; + + private List buffersToRelease; + private FakeFrameWriter fakeDownstreamOutput; + private FakeHardwareBufferNativeHelpers fakeNativeHelpers; + private HandlerThread handlerThread; + private Handler handler; + private TestListener testListener; + private SurfaceToFrameWriterAdapter writer; + @Nullable private ImageWriter testProducerWriter; + + @Before + public void setUp() { + buffersToRelease = new ArrayList<>(); + handlerThread = new HandlerThread("SurfaceToFrameWriterAdapterTestThread"); + handlerThread.start(); + handler = new Handler(handlerThread.getLooper()); + + fakeDownstreamOutput = new FakeFrameWriter(); + fakeNativeHelpers = new FakeHardwareBufferNativeHelpers(); + testListener = new TestListener(); + + writer = + new SurfaceToFrameWriterAdapter( + fakeDownstreamOutput, fakeNativeHelpers, handler, testListener); + } + + @After + public void tearDown() throws Exception { + if (writer != null) { + closeWriter(); + } + if (testProducerWriter != null) { + testProducerWriter.close(); + } + for (HardwareBuffer buffer : buffersToRelease) { + buffer.close(); + } + handlerThread.quitSafely(); + } + + @Test + public void configure_standardOptions_configuresDownstreamFormatWithSameDimensions() + throws Exception { + Format inputFormat = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + Surface surface = configureWriter(WIDTH, HEIGHT, inputFormat); + + assertThat(fakeDownstreamOutput.configuredFormat).isNotNull(); + assertThat(fakeDownstreamOutput.configuredFormat.width).isEqualTo(WIDTH); + assertThat(fakeDownstreamOutput.configuredFormat.height).isEqualTo(HEIGHT); + assertThat(surface).isNotNull(); + } + + @Test + public void draining_queuesFrameDownstreamWithCorrectPresentationTimeUs() throws Exception { + Format inputFormat = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + Surface surface = configureWriter(WIDTH, HEIGHT, inputFormat); + + long expectedPtsUs = 42_000L; + onInputFrameQueued(expectedPtsUs); + + fakeDownstreamOutput.prepareInputFrame(createHardwareBufferFrame(WIDTH, HEIGHT)); + produceInputFrame(surface, expectedPtsUs); + + assertThat(fakeDownstreamOutput.frameLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(fakeDownstreamOutput.queuedFrames).hasSize(1); + assertThat(fakeDownstreamOutput.queuedFrames.get(0).getContentTimeUs()) + .isEqualTo(expectedPtsUs); + assertThat(testListener.imageReleasedCount.get()).isAtLeast(1); + } + + @Test + public void draining_multipleFrames_maintainsFifoOrderOfTimestamps() throws Exception { + Format inputFormat = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + Surface surface = configureWriter(WIDTH, HEIGHT, inputFormat); + + onInputFrameQueued(/* presentationTimeUs= */ 10_000L); + onInputFrameQueued(/* presentationTimeUs= */ 20_000L); + + fakeDownstreamOutput.frameLatch = new CountDownLatch(2); + fakeDownstreamOutput.prepareInputFrame(createHardwareBufferFrame(WIDTH, HEIGHT)); + fakeDownstreamOutput.prepareInputFrame(createHardwareBufferFrame(WIDTH, HEIGHT)); + + produceInputFrame(surface, /* presentationTimeUs= */ 10_000L); + produceInputFrame(surface, /* presentationTimeUs= */ 20_000L); + + assertThat(fakeDownstreamOutput.frameLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(fakeDownstreamOutput.queuedFrames).hasSize(2); + assertThat(fakeDownstreamOutput.queuedFrames.get(0).getContentTimeUs()).isEqualTo(10_000L); + assertThat(fakeDownstreamOutput.queuedFrames.get(1).getContentTimeUs()).isEqualTo(20_000L); + } + + @Test + public void draining_whenInputTimestampsDecreaseOnSeek_restoresOriginalPresentationTimestamps() + throws Exception { + Format inputFormat = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + Surface surface = configureWriter(WIDTH, HEIGHT, inputFormat); + + // Simulate a seek backwards where input presentation timestamps decrease (100ms -> 20ms). + onInputFrameQueued(/* presentationTimeUs= */ 100_000L); + onInputFrameQueued(/* presentationTimeUs= */ 20_000L); + + fakeDownstreamOutput.frameLatch = new CountDownLatch(2); + fakeDownstreamOutput.prepareInputFrame(createHardwareBufferFrame(WIDTH, HEIGHT)); + fakeDownstreamOutput.prepareInputFrame(createHardwareBufferFrame(WIDTH, HEIGHT)); + + // Surface receives strictly increasing monotonic timestamps from Google Play services. + produceInputFrame(surface, /* presentationTimeUs= */ 1_000_000L); + produceInputFrame(surface, /* presentationTimeUs= */ 1_001_000L); + + assertThat(fakeDownstreamOutput.frameLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(fakeDownstreamOutput.queuedFrames).hasSize(2); + assertThat(fakeDownstreamOutput.queuedFrames.get(0).getContentTimeUs()).isEqualTo(100_000L); + assertThat(fakeDownstreamOutput.queuedFrames.get(1).getContentTimeUs()).isEqualTo(20_000L); + } + + @Test + public void draining_extraFramesBeyondInputCount_triggersOnError() throws Exception { + Format inputFormat = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + Surface surface = configureWriter(WIDTH, HEIGHT, inputFormat); + + // Only 1 frame registered on input. + onInputFrameQueued(/* presentationTimeUs= */ 0L); + + fakeDownstreamOutput.prepareInputFrame(createHardwareBufferFrame(WIDTH, HEIGHT)); + fakeDownstreamOutput.prepareInputFrame(createHardwareBufferFrame(WIDTH, HEIGHT)); + + // Produce 2 frames to the surface when only 1 input frame was registered. + produceInputFrame(surface); + + assertThat(fakeDownstreamOutput.frameLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(fakeDownstreamOutput.queuedFrames).hasSize(1); + + produceInputFrame(surface); + + assertThat(testListener.errorLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(testListener.error.get()).isNotNull(); + assertThat(testListener.error.get()) + .hasMessageThat() + .contains("Unexpected output frame acquired beyond input frame count"); + assertThat(fakeDownstreamOutput.queuedFrames).hasSize(1); + } + + @Test + public void + eventDrivenEos_whenInputEosArrivesBeforeFramesDrained_signalsDownstreamOnlyAfterLastFrame() + throws Exception { + Format inputFormat = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + Surface surface = configureWriter(WIDTH, HEIGHT, inputFormat); + + onInputFrameQueued(/* presentationTimeUs= */ 0L); + onInputFrameQueued(/* presentationTimeUs= */ 1_000L); + + // Input EOS arrives before outputs have been drained. + onEndOfStream(); + + // Wait for the handler thread to process any pending tasks before checking EOS state. + CountDownLatch syncLatch = new CountDownLatch(1); + handler.post(syncLatch::countDown); + syncLatch.await(TIMEOUT_MS, MILLISECONDS); + + assertThat(fakeDownstreamOutput.eosSignaled.get()).isFalse(); + + // Drain first frame. + fakeDownstreamOutput.frameLatch = new CountDownLatch(1); + fakeDownstreamOutput.prepareInputFrame(createHardwareBufferFrame(WIDTH, HEIGHT)); + produceInputFrame(surface); + + assertThat(fakeDownstreamOutput.frameLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(fakeDownstreamOutput.eosSignaled.get()).isFalse(); + + // Drain second frame. + fakeDownstreamOutput.frameLatch = new CountDownLatch(1); + fakeDownstreamOutput.prepareInputFrame(createHardwareBufferFrame(WIDTH, HEIGHT)); + produceInputFrame(surface); + + assertThat(fakeDownstreamOutput.frameLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + + // Both frames drained: downstream EOS must be signaled! + assertThat(fakeDownstreamOutput.eosLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(fakeDownstreamOutput.eosSignaled.get()).isTrue(); + } + + @Test + public void eventDrivenEos_whenInputEosArrivesAfterAllFramesDrained_signalsDownstreamImmediately() + throws Exception { + Format inputFormat = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + Surface surface = configureWriter(WIDTH, HEIGHT, inputFormat); + + onInputFrameQueued(/* presentationTimeUs= */ 0L); + fakeDownstreamOutput.prepareInputFrame(createHardwareBufferFrame(WIDTH, HEIGHT)); + + produceInputFrame(surface); + + assertThat(fakeDownstreamOutput.frameLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(fakeDownstreamOutput.eosSignaled.get()).isFalse(); + + // Input EOS arrives after frame is already processed. + onEndOfStream(); + + assertThat(fakeDownstreamOutput.eosLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(fakeDownstreamOutput.eosSignaled.get()).isTrue(); + } + + @Test + public void draining_whenNativeCopyFails_triggersOnError() throws Exception { + Format inputFormat = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + Surface surface = configureWriter(WIDTH, HEIGHT, inputFormat); + + onInputFrameQueued(/* presentationTimeUs= */ 0L); + fakeNativeHelpers.shouldSucceed = false; + + fakeDownstreamOutput.prepareInputFrame(createHardwareBufferFrame(WIDTH, HEIGHT)); + produceInputFrame(surface); + + assertThat(testListener.errorLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(testListener.error.get()).isNotNull(); + assertThat(testListener.error.get()) + .hasMessageThat() + .contains("nativeCopyHardwareBufferToHardwareBuffer failed"); + } + + @Test + public void close_releasesSurfacesAndCleansUp() throws Exception { + Format inputFormat = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + Surface unused = configureWriter(WIDTH, HEIGHT, inputFormat); + closeWriter(); + + // Further outputs or close calls do not throw. + closeWriter(); + } + + @Test + public void close_whenFrameHeldInPendingImage_closesCleanlyWithoutError() throws Exception { + Format inputFormat = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + Surface surface = configureWriter(WIDTH, HEIGHT, inputFormat); + onInputFrameQueued(/* presentationTimeUs= */ 0L); + produceInputFrame(surface); + + assertThat(fakeDownstreamOutput.dequeueAttemptedLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(fakeDownstreamOutput.queuedFrames).isEmpty(); + + closeWriter(); + + assertThat(testListener.error.get()).isNull(); + assertThat(fakeDownstreamOutput.queuedFrames).isEmpty(); + } + + private Surface configureWriter(int width, int height, Format inputFormat) throws Exception { + return runOnHandler(() -> writer.configure(width, height, inputFormat)); + } + + private void onInputFrameQueued(long presentationTimeUs) throws Exception { + runOnHandler(() -> writer.onInputFrameQueued(presentationTimeUs)); + } + + private void onEndOfStream() throws Exception { + runOnHandler(writer::onEndOfStream); + } + + private void closeWriter() throws Exception { + runOnHandler(writer::close); + } + + private T runOnHandler(Callable callable) throws Exception { + AtomicReference result = new AtomicReference<>(); + AtomicReference exception = new AtomicReference<>(); + CountDownLatch latch = new CountDownLatch(1); + handler.post( + () -> { + try { + result.set(callable.call()); + } catch (Throwable t) { + exception.set(t); + } finally { + latch.countDown(); + } + }); + assertThat(latch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + if (exception.get() != null) { + if (exception.get() instanceof Exception) { + throw (Exception) exception.get(); + } + throw new RuntimeException(exception.get()); + } + return result.get(); + } + + private void runOnHandler(Runnable runnable) throws Exception { + AtomicReference exception = new AtomicReference<>(); + CountDownLatch latch = new CountDownLatch(1); + handler.post( + () -> { + try { + runnable.run(); + } catch (Throwable t) { + exception.set(t); + } finally { + latch.countDown(); + } + }); + assertThat(latch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + if (exception.get() != null) { + if (exception.get() instanceof Exception) { + throw (Exception) exception.get(); + } + throw new RuntimeException(exception.get()); + } + } + + private void produceInputFrame(Surface surface) { + produceInputFrame(surface, /* presentationTimeUs= */ 0L); + } + + private void produceInputFrame(Surface surface, long presentationTimeUs) { + if (testProducerWriter == null) { + testProducerWriter = ImageWriter.newInstance(surface, 4, PixelFormat.RGBA_8888); + } + Image image = testProducerWriter.dequeueInputImage(); + image.setTimestamp(presentationTimeUs * 1000L); + testProducerWriter.queueInputImage(image); + } + + private DefaultHardwareBufferFrame createHardwareBufferFrame(int width, int height) { + HardwareBuffer buffer = + HardwareBuffer.create( + width, + height, + HardwareBuffer.RGBA_8888, + /* layers= */ 1, + HardwareBuffer.USAGE_GPU_COLOR_OUTPUT + | HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE + | HardwareBuffer.USAGE_CPU_READ_OFTEN); + buffersToRelease.add(buffer); + return new DefaultHardwareBufferFrame.Builder(buffer) + .setFormat(new Format.Builder().setWidth(width).setHeight(height).build()) + .build(); + } + + private static class TestListener implements SurfaceToFrameWriterAdapter.Listener { + final AtomicInteger imageReleasedCount; + final AtomicReference error; + final CountDownLatch errorLatch; + + TestListener() { + imageReleasedCount = new AtomicInteger(0); + error = new AtomicReference<>(); + errorLatch = new CountDownLatch(1); + } + + @Override + public void onFrameReleased() { + imageReleasedCount.incrementAndGet(); + } + + @Override + public void onError(VideoFrameProcessingException exception) { + error.set(exception); + errorLatch.countDown(); + } + } +} diff --git a/libraries/effect_play_services/src/androidTest/java/androidx/media3/effect/playservices/mh/EnhancementSessionDecoratorAndroidTest.java b/libraries/effect_play_services/src/androidTest/java/androidx/media3/effect/playservices/mh/EnhancementSessionDecoratorAndroidTest.java new file mode 100644 index 00000000000..9c9025be68c --- /dev/null +++ b/libraries/effect_play_services/src/androidTest/java/androidx/media3/effect/playservices/mh/EnhancementSessionDecoratorAndroidTest.java @@ -0,0 +1,334 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect.playservices.mh; + +import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_15S; +import static androidx.media3.test.utils.BitmapPixelTestUtil.maybeSaveTestBitmap; +import static androidx.media3.test.utils.BitmapPixelTestUtil.readBitmap; +import static androidx.media3.test.utils.FormatSupportAssumptions.assumeFormatsSupported; +import static androidx.media3.test.utils.TestUtil.PSNR_THRESHOLD; +import static androidx.media3.test.utils.TestUtil.assertBitmapsAreSimilar; +import static androidx.media3.test.utils.TestUtil.extractAllSamplesFromFilePath; +import static androidx.media3.test.utils.TestUtil.retrieveTrackFormat; +import static androidx.test.core.app.ApplicationProvider.getApplicationContext; +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.util.concurrent.MoreExecutors.directExecutor; +import static java.util.concurrent.TimeUnit.SECONDS; +import static org.junit.Assume.assumeTrue; + +import android.content.Context; +import android.graphics.Bitmap; +import android.hardware.HardwareBuffer; +import androidx.annotation.Nullable; +import androidx.media3.common.C; +import androidx.media3.common.ColorInfo; +import androidx.media3.common.Format; +import androidx.media3.common.MediaItem; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.DefaultHardwareBufferFrame; +import androidx.media3.common.video.FrameProcessor; +import androidx.media3.effect.DefaultGlFrameProcessor; +import androidx.media3.effect.HardwareBufferJni; +import androidx.media3.effect.playservices.EnhancementSessionDecorator; +import androidx.media3.extractor.mp4.Mp4Extractor; +import androidx.media3.extractor.text.DefaultSubtitleParserFactory; +import androidx.media3.inspector.frame.FrameExtractor; +import androidx.media3.test.utils.FakeExtractorOutput; +import androidx.media3.test.utils.FakeFrameProcessor; +import androidx.media3.test.utils.FakeFrameWriter; +import androidx.media3.transformer.Composition; +import androidx.media3.transformer.DefaultEncoderFactory; +import androidx.media3.transformer.EditedMediaItem; +import androidx.media3.transformer.EditedMediaItemSequence; +import androidx.media3.transformer.ExportException; +import androidx.media3.transformer.ExportResult; +import androidx.media3.transformer.Transformer; +import androidx.media3.transformer.VideoEncoderSettings; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import androidx.test.filters.SdkSuppress; +import androidx.test.platform.app.InstrumentationRegistry; +import com.google.android.gms.media.effect.enhancement.Enhancement; +import com.google.android.gms.tasks.Tasks; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import com.google.common.collect.Iterables; +import com.google.common.util.concurrent.SettableFuture; +import java.io.File; +import java.util.ArrayList; +import java.util.List; +import org.junit.After; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import org.junit.rules.TestName; +import org.junit.runner.RunWith; + +/** Instrumentation tests for {@link EnhancementSessionDecorator} using Google Play services. */ +@RunWith(AndroidJUnit4.class) +@SdkSuppress(minSdkVersion = 33) +public final class EnhancementSessionDecoratorAndroidTest { + + private static final int WIDTH = 128; + private static final int HEIGHT = 128; + private static final long SESSION_TIMEOUT_SECONDS = 30; + private static final String GOLDEN_IMAGE_PATH = + "media/mp4/sample_with_increasing_timestamps_320w_240h_5s_play_services_upscaled_000.png"; + private static final int EXPECTED_FRAME_COUNT = 84; + + @Rule public final TestName testName = new TestName(); + @Rule public final TemporaryFolder temporaryFolder = new TemporaryFolder(); + + private List buffersToClose; + private Context context; + private File outputVideoFile; + private FrameProcessor.Factory baseGlFactory; + private FakeFrameWriter fakeDownstreamOutput; + private FakeFrameProcessor.Listener testListener; + @Nullable private FrameProcessor decorator; + @Nullable private Transformer transformer; + + @Before + public void setUp() throws Exception { + buffersToClose = new ArrayList<>(); + context = getApplicationContext(); + outputVideoFile = temporaryFolder.newFile(); + baseGlFactory = + new DefaultGlFrameProcessor.Factory.Builder(context, HardwareBufferJni.INSTANCE).build(); + fakeDownstreamOutput = new FakeFrameWriter(); + testListener = new FakeFrameProcessor.Listener(); + } + + @After + public void tearDown() throws Exception { + if (transformer != null) { + InstrumentationRegistry.getInstrumentation().runOnMainSync(() -> transformer.cancel()); + } + if (decorator != null) { + decorator.close(); + } + for (HardwareBuffer buffer : buffersToClose) { + buffer.close(); + } + buffersToClose.clear(); + } + + @Test + public void queue_withPlayServicesSession_producesOutput() throws Exception { + assumeDeviceSupported(); + setUpPlayServicesDecorator(/* isUpscaleVideoEnabled= */ false); + AsyncFrame inputFrame = new AsyncFrame(createHardwareBufferFrame(), /* acquireFence= */ null); + + assertThat(decorator.queue(ImmutableList.of(inputFrame))).isTrue(); + + assertThat(fakeDownstreamOutput.frameLatch.await(SESSION_TIMEOUT_SECONDS, SECONDS)).isTrue(); + assertThat(fakeDownstreamOutput.queuedFrames).hasSize(1); + assertThat(testListener.error.get()).isNull(); + } + + @Test + public void queue_withUpscaleEnabled_configuresDoubleOutputSize() throws Exception { + assumeDeviceSupported(); + setUpPlayServicesDecorator(/* isUpscaleVideoEnabled= */ true); + AsyncFrame inputFrame = new AsyncFrame(createHardwareBufferFrame(), /* acquireFence= */ null); + + assertThat(decorator.queue(ImmutableList.of(inputFrame))).isTrue(); + + assertThat(fakeDownstreamOutput.frameLatch.await(SESSION_TIMEOUT_SECONDS, SECONDS)).isTrue(); + assertThat(fakeDownstreamOutput.configuredFormat).isNotNull(); + assertThat(fakeDownstreamOutput.configuredFormat.width).isEqualTo(WIDTH * 2); + assertThat(fakeDownstreamOutput.configuredFormat.height).isEqualTo(HEIGHT * 2); + } + + @Test + public void + exportWithTransformer_decoratingDefaultGlFrameProcessor_producesExpectedFrameCountAndGolden() + throws Exception { + assumeDeviceSupported(); + + ExportResult exportResult = + runTransformer( + /* isTonemappingEnabled= */ false, + /* isDeblurAndDenoiseEnabled= */ false, + /* isUpscaleVideoEnabled= */ false); + + assertThat(exportResult.videoFrameCount).isEqualTo(EXPECTED_FRAME_COUNT); + assertOutputFileSampleCount(EXPECTED_FRAME_COUNT); + Bitmap outputFrame = extractFirstFrame(outputVideoFile.getAbsolutePath()); + maybeSaveTestBitmap( + testName.getMethodName(), /* bitmapLabel= */ "actual", outputFrame, /* path= */ null); + Bitmap expectedFrame = + extractFirstFrame(MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_15S.uri); + assertBitmapsAreSimilar(expectedFrame, outputFrame, PSNR_THRESHOLD); + } + + @Test + public void exportWithTransformer_upscaleEnabled_producesDoubledResolution() throws Exception { + assumeDeviceSupported(); + + ExportResult exportResult = + runTransformer( + /* isTonemappingEnabled= */ false, + /* isDeblurAndDenoiseEnabled= */ false, + /* isUpscaleVideoEnabled= */ true); + + assertThat(exportResult.videoFrameCount).isEqualTo(EXPECTED_FRAME_COUNT); + assertOutputFileSampleCount(EXPECTED_FRAME_COUNT); + Format trackFormat = + retrieveTrackFormat(context, outputVideoFile.getAbsolutePath(), C.TRACK_TYPE_VIDEO); + assertThat(trackFormat.width).isEqualTo(640); + assertThat(trackFormat.height).isEqualTo(480); + Bitmap outputFrame = extractFirstFrame(outputVideoFile.getAbsolutePath()); + maybeSaveTestBitmap( + testName.getMethodName(), /* bitmapLabel= */ "actual", outputFrame, /* path= */ null); + Bitmap goldenBitmap = readBitmap(GOLDEN_IMAGE_PATH); + assertBitmapsAreSimilar(goldenBitmap, outputFrame, PSNR_THRESHOLD); + } + + @Test + public void exportWithTransformer_withTonemappingAndDeblurAndDenoise_producesExpectedFrameCount() + throws Exception { + assumeDeviceSupported(); + + ExportResult exportResult = + runTransformer( + /* isTonemappingEnabled= */ true, + /* isDeblurAndDenoiseEnabled= */ true, + /* isUpscaleVideoEnabled= */ false); + + assertThat(exportResult.videoFrameCount).isEqualTo(EXPECTED_FRAME_COUNT); + assertOutputFileSampleCount(EXPECTED_FRAME_COUNT); + } + + private void assumeDeviceSupported() throws Exception { + assumeTrue( + "Device not supported by Google Play services video enhancement", + Tasks.await(Enhancement.getClient(context).isDeviceSupported())); + } + + private void setUpPlayServicesDecorator(boolean isUpscaleVideoEnabled) { + int outputWidth = isUpscaleVideoEnabled ? WIDTH * 2 : WIDTH; + int outputHeight = isUpscaleVideoEnabled ? HEIGHT * 2 : HEIGHT; + fakeDownstreamOutput.prepareInputFrame(createHardwareBufferFrame(outputWidth, outputHeight)); + decorator = + new EnhancementSessionDecorator.Builder(context, baseGlFactory) + .setTonemappingEnabled(false) + .setDeblurAndDenoiseVideoEnabled(false) + .setUpscaleVideoEnabled(isUpscaleVideoEnabled) + .build() + .create(fakeDownstreamOutput, directExecutor(), testListener); + } + + private void assertOutputFileSampleCount(int expectedSampleCount) throws Exception { + FakeExtractorOutput fakeExtractorOutput = + extractAllSamplesFromFilePath( + new Mp4Extractor(new DefaultSubtitleParserFactory()), + outputVideoFile.getAbsolutePath()); + Iterables.getOnlyElement(fakeExtractorOutput.getTrackOutputsForType(C.TRACK_TYPE_VIDEO)) + .assertSampleCount(expectedSampleCount); + } + + private ExportResult runTransformer( + boolean isTonemappingEnabled, + boolean isDeblurAndDenoiseEnabled, + boolean isUpscaleVideoEnabled) + throws Exception { + assumeFormatsSupported( + context, + testName.getMethodName(), + /* inputFormat= */ MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_15S.videoFormat, + /* outputFormat= */ null); + FrameProcessor.Factory factory = + new EnhancementSessionDecorator.Builder(context, baseGlFactory) + .setTonemappingEnabled(isTonemappingEnabled) + .setDeblurAndDenoiseVideoEnabled(isDeblurAndDenoiseEnabled) + .setUpscaleVideoEnabled(isUpscaleVideoEnabled) + .build(); + SettableFuture exportResultFuture = SettableFuture.create(); + transformer = + new Transformer.Builder(context) + .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) + .setFrameProcessorFactory(factory) + .setEncoderFactory( + new DefaultEncoderFactory.Builder(context) + .setRequestedVideoEncoderSettings( + new VideoEncoderSettings.Builder().setBitrate(5_000_000).build()) + .build()) + .addListener( + new Transformer.Listener() { + @Override + public void onCompleted(Composition composition, ExportResult exportResult) { + exportResultFuture.set(exportResult); + } + + @Override + public void onError( + Composition composition, + ExportResult exportResult, + ExportException exportException) { + exportResultFuture.setException(exportException); + } + }) + .build(); + MediaItem mediaItem = + new MediaItem.Builder() + .setUri(MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_15S.uri) + .setClippingConfiguration( + new MediaItem.ClippingConfiguration.Builder().setEndPositionMs(1_400).build()) + .build(); + EditedMediaItem editedMediaItem = new EditedMediaItem.Builder(mediaItem).build(); + Composition composition = + new Composition.Builder( + EditedMediaItemSequence.withAudioAndVideoFrom(ImmutableList.of(editedMediaItem))) + .build(); + InstrumentationRegistry.getInstrumentation() + .runOnMainSync(() -> transformer.start(composition, outputVideoFile.getAbsolutePath())); + return exportResultFuture.get(SESSION_TIMEOUT_SECONDS, SECONDS); + } + + private Bitmap extractFirstFrame(String filePath) throws Exception { + try (FrameExtractor frameExtractor = + new FrameExtractor.Builder(context, MediaItem.fromUri(filePath)).build()) { + return frameExtractor.getFrame(0).get().bitmap; + } + } + + private DefaultHardwareBufferFrame createHardwareBufferFrame() { + return createHardwareBufferFrame(WIDTH, HEIGHT); + } + + private DefaultHardwareBufferFrame createHardwareBufferFrame(int width, int height) { + HardwareBuffer buffer = + HardwareBuffer.create( + width, + height, + HardwareBuffer.RGBA_8888, + /* layers= */ 1, + HardwareBuffer.USAGE_GPU_COLOR_OUTPUT + | HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE + | HardwareBuffer.USAGE_CPU_WRITE_OFTEN); + buffersToClose.add(buffer); + return new DefaultHardwareBufferFrame.Builder(buffer) + .setFormat( + new Format.Builder() + .setWidth(width) + .setHeight(height) + .setColorInfo(ColorInfo.SDR_BT709_LIMITED) + .build()) + .setMetadata(ImmutableMap.of(DefaultGlFrameProcessor.KEY_COMPOSITION_SEQUENCE_INDEX, 0)) + .build(); + } +} diff --git a/libraries/session/src/main/res/values/colors.xml b/libraries/effect_play_services/src/main/AndroidManifest.xml similarity index 68% rename from libraries/session/src/main/res/values/colors.xml rename to libraries/effect_play_services/src/main/AndroidManifest.xml index e0dd8c965be..59b7ba05951 100644 --- a/libraries/session/src/main/res/values/colors.xml +++ b/libraries/effect_play_services/src/main/AndroidManifest.xml @@ -1,5 +1,5 @@ - - - - #ff424242 - + + + diff --git a/libraries/effect_play_services/src/main/java/androidx/media3/effect/playservices/EnhancementSessionDecorator.java b/libraries/effect_play_services/src/main/java/androidx/media3/effect/playservices/EnhancementSessionDecorator.java new file mode 100644 index 00000000000..37f19d56958 --- /dev/null +++ b/libraries/effect_play_services/src/main/java/androidx/media3/effect/playservices/EnhancementSessionDecorator.java @@ -0,0 +1,520 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect.playservices; + +import static com.google.common.base.Preconditions.checkNotNull; +import static com.google.common.base.Preconditions.checkState; +import static java.lang.annotation.ElementType.TYPE_USE; +import static java.lang.annotation.RetentionPolicy.SOURCE; +import static java.util.concurrent.TimeUnit.MILLISECONDS; + +import android.content.Context; +import android.graphics.Bitmap; +import android.os.Handler; +import android.os.HandlerThread; +import android.os.Looper; +import android.view.Surface; +import androidx.annotation.IntDef; +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import androidx.annotation.VisibleForTesting; +import androidx.media3.common.Format; +import androidx.media3.common.VideoFrameProcessingException; +import androidx.media3.common.util.ExperimentalApi; +import androidx.media3.common.util.HandlerExecutor; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.Frame; +import androidx.media3.common.video.FrameProcessor; +import androidx.media3.common.video.FrameWriter; +import androidx.media3.common.video.HardwareBufferNativeHelpers; +import androidx.media3.common.video.SyncFenceWrapper; +import androidx.media3.effect.HardwareBufferJni; +import com.google.android.gms.media.effect.enhancement.EnhancementCallback; +import com.google.android.gms.media.effect.enhancement.EnhancementOptions; +import com.google.android.gms.media.effect.enhancement.EnhancementSession; +import com.google.errorprone.annotations.CanIgnoreReturnValue; +import java.lang.annotation.Documented; +import java.lang.annotation.Retention; +import java.lang.annotation.Target; +import java.util.List; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.Executor; +import org.checkerframework.checker.nullness.qual.MonotonicNonNull; + +/** + * A {@link FrameProcessor} decorator that routes video frames through Google Play services {@link + * EnhancementSession}. + * + *

Timestamp handling across the surface bridge: + * + *

    + *
  1. {@link FrameToSurfaceFrameWriter} receives an upstream {@link Frame} with {@code + * presentationTimeUs}, stamps the {@link android.media.Image} with a synthetic strictly + * increasing nanosecond timestamp from {@link MonotonicTimestampGenerator} (required because + * the underlying MediaPipe graph rejects non-increasing timestamps on seeks), and notifies + * {@link FrameToSurfaceFrameWriter.Listener#onFrameQueued(long)} with the original {@code + * presentationTimeUs}. + *
  2. {@link EnhancementSessionDecorator} forwards {@code presentationTimeUs} from {@code + * onFrameQueued} to {@link SurfaceToFrameWriterAdapter#onInputFrameQueued(long)}. + *
  3. {@link SurfaceToFrameWriterAdapter} stores {@code presentationTimeUs} in a FIFO queue and + * restores it onto the output {@link Frame} when the processed image arrives on the output + * {@link Surface}. A 1:1 FIFO queue is guaranteed to match because {@link + * FrameToSurfaceFrameWriter} enforces at most one frame in flight ({@code + * MAX_IN_FLIGHT_FRAMES = 1}), waiting for {@link + * SurfaceToFrameWriterAdapter.Listener#onFrameReleased()} before queuing the next frame. + *
+ */ +@ExperimentalApi +@RequiresApi(33) +public final class EnhancementSessionDecorator implements FrameProcessor { + + private static final int UPSCALE_FACTOR = 2; + private static final long CLOSE_TIMEOUT_MS = 500L; + + @Documented + @Retention(SOURCE) + @Target(TYPE_USE) + @IntDef({ + STATE_WAITING_FOR_FORMAT, + STATE_INITIALIZING, + STATE_READY, + STATE_ERROR, + STATE_CLOSED, + }) + private @interface State {} + + private static final int STATE_WAITING_FOR_FORMAT = 0; + private static final int STATE_INITIALIZING = 1; + private static final int STATE_READY = 2; + private static final int STATE_ERROR = 3; + private static final int STATE_CLOSED = 4; + + /** Builder for {@link EnhancementSessionDecorator} factories. */ + public static final class Builder { + private final Context context; + private final FrameProcessor.Factory baseFrameProcessorFactory; + private boolean isTonemappingEnabled; + private boolean isDeblurAndDenoiseVideoEnabled; + private boolean isUpscaleVideoEnabled; + @Nullable private Looper looper; + @Nullable private EnhancementSessionManager.Client sessionManagerClient; + @Nullable private HardwareBufferNativeHelpers hardwareBufferNativeHelpers; + + /** Creates a builder with the specified context and base {@link FrameProcessor.Factory}. */ + public Builder(Context context, FrameProcessor.Factory baseFrameProcessorFactory) { + this.context = context.getApplicationContext(); + this.baseFrameProcessorFactory = baseFrameProcessorFactory; + } + + /** Sets whether tone mapping is enabled. */ + @CanIgnoreReturnValue + public Builder setTonemappingEnabled(boolean isTonemappingEnabled) { + this.isTonemappingEnabled = isTonemappingEnabled; + return this; + } + + /** Sets whether deblur and denoise is enabled. */ + @CanIgnoreReturnValue + public Builder setDeblurAndDenoiseVideoEnabled(boolean isDeblurAndDenoiseVideoEnabled) { + this.isDeblurAndDenoiseVideoEnabled = isDeblurAndDenoiseVideoEnabled; + return this; + } + + /** Sets whether upscale is enabled. */ + @CanIgnoreReturnValue + public Builder setUpscaleVideoEnabled(boolean isUpscaleVideoEnabled) { + this.isUpscaleVideoEnabled = isUpscaleVideoEnabled; + return this; + } + + /** + * Sets the {@link Looper} that must be used for all calls to the enhancement session and + * internal callbacks. + * + *

If not set, an internal {@link HandlerThread} will be created and managed by each {@link + * EnhancementSessionDecorator} instance. + */ + @CanIgnoreReturnValue + public Builder setLooper(Looper looper) { + this.looper = looper; + return this; + } + + @VisibleForTesting + @CanIgnoreReturnValue + /* package */ Builder setSessionManagerClient( + EnhancementSessionManager.Client sessionManagerClient) { + this.sessionManagerClient = sessionManagerClient; + return this; + } + + @VisibleForTesting + @CanIgnoreReturnValue + /* package */ Builder setHardwareBufferNativeHelpers( + HardwareBufferNativeHelpers hardwareBufferNativeHelpers) { + this.hardwareBufferNativeHelpers = hardwareBufferNativeHelpers; + return this; + } + + /** Builds the {@link FrameProcessor.Factory}. */ + public Factory build() { + return new Factory( + context, + baseFrameProcessorFactory, + isTonemappingEnabled, + isDeblurAndDenoiseVideoEnabled, + isUpscaleVideoEnabled, + looper, + sessionManagerClient, + hardwareBufferNativeHelpers); + } + } + + /** Factory for {@link EnhancementSessionDecorator}. */ + public static final class Factory implements FrameProcessor.Factory { + private final FrameProcessor.Factory baseFrameProcessorFactory; + private final boolean isTonemappingEnabled; + private final boolean isDeblurAndDenoiseVideoEnabled; + private final boolean isUpscaleVideoEnabled; + @Nullable private final Looper looper; + private final EnhancementSessionManager.Client sessionManagerClient; + private final HardwareBufferNativeHelpers hardwareBufferNativeHelpers; + + private Factory( + Context context, + FrameProcessor.Factory baseFrameProcessorFactory, + boolean isTonemappingEnabled, + boolean isDeblurAndDenoiseVideoEnabled, + boolean isUpscaleVideoEnabled, + @Nullable Looper looper, + @Nullable EnhancementSessionManager.Client sessionManagerClient, + @Nullable HardwareBufferNativeHelpers hardwareBufferNativeHelpers) { + this.baseFrameProcessorFactory = baseFrameProcessorFactory; + this.isTonemappingEnabled = isTonemappingEnabled; + this.isDeblurAndDenoiseVideoEnabled = isDeblurAndDenoiseVideoEnabled; + this.isUpscaleVideoEnabled = isUpscaleVideoEnabled; + this.looper = looper; + this.sessionManagerClient = + sessionManagerClient != null + ? sessionManagerClient + : new EnhancementSessionManager.PlayServicesClientAdapter( + context.getApplicationContext()); + this.hardwareBufferNativeHelpers = + hardwareBufferNativeHelpers != null + ? hardwareBufferNativeHelpers + : HardwareBufferJni.INSTANCE; + } + + @Override + public FrameProcessor create(FrameWriter output, Executor listenerExecutor, Listener listener) { + return new EnhancementSessionDecorator( + baseFrameProcessorFactory, + output, + isTonemappingEnabled, + isDeblurAndDenoiseVideoEnabled, + isUpscaleVideoEnabled, + listenerExecutor, + listener, + looper, + sessionManagerClient, + hardwareBufferNativeHelpers); + } + } + + private final boolean isTonemappingEnabled; + private final boolean isDeblurAndDenoiseVideoEnabled; + private final boolean isUpscaleVideoEnabled; + private final Executor listenerExecutor; + private final Listener listener; + + private final Looper looper; + @Nullable private final HandlerThread handlerThread; + private final Handler handler; + private final Executor handlerExecutor; + + private final FrameProcessor baseFrameProcessor; + private final FrameToSurfaceFrameWriter frameToSurfaceFrameWriter; + private final EnhancementSessionManager sessionManager; + private final SurfaceToFrameWriterAdapter surfaceToFrameWriterAdapter; + + private @State int state; + + // Accessed only on handlerExecutor. + private @MonotonicNonNull Format inputFormat; + + // Safe because handleError is only invoked asynchronously on handlerThread. + @SuppressWarnings("method.invocation") + private EnhancementSessionDecorator( + FrameProcessor.Factory baseFrameProcessorFactory, + FrameWriter downstreamOutput, + boolean isTonemappingEnabled, + boolean isDeblurAndDenoiseVideoEnabled, + boolean isUpscaleVideoEnabled, + Executor listenerExecutor, + Listener listener, + @Nullable Looper looper, + EnhancementSessionManager.Client sessionManagerClient, + HardwareBufferNativeHelpers hardwareBufferNativeHelpers) { + state = STATE_WAITING_FOR_FORMAT; + this.isTonemappingEnabled = isTonemappingEnabled; + this.isDeblurAndDenoiseVideoEnabled = isDeblurAndDenoiseVideoEnabled; + this.isUpscaleVideoEnabled = isUpscaleVideoEnabled; + this.listenerExecutor = listenerExecutor; + this.listener = listener; + + if (looper != null) { + handlerThread = null; + this.looper = looper; + } else { + handlerThread = new HandlerThread("EnhancementDecorator"); + handlerThread.start(); + this.looper = handlerThread.getLooper(); + } + handler = new Handler(this.looper); + handlerExecutor = + new HandlerExecutor(handler, e -> handleError(VideoFrameProcessingException.from(e))); + + sessionManager = + new EnhancementSessionManager( + handlerExecutor, new SessionManagerListener(), sessionManagerClient); + + surfaceToFrameWriterAdapter = + new SurfaceToFrameWriterAdapter( + downstreamOutput, + hardwareBufferNativeHelpers, + handler, + new SurfaceToFrameWriterAdapterListener()); + + frameToSurfaceFrameWriter = + new FrameToSurfaceFrameWriter(handlerExecutor, new FrameToSurfaceFrameWriterListener()); + + baseFrameProcessor = + baseFrameProcessorFactory.create( + frameToSurfaceFrameWriter, listenerExecutor, new BaseFrameProcessorListener()); + } + + @Override + public boolean queue(List frames) { + return baseFrameProcessor.queue(frames); + } + + @Override + public void signalEndOfStream() { + baseFrameProcessor.signalEndOfStream(); + } + + /** + * {@inheritDoc} + * + *

May be called from any thread. + */ + @Override + public void close() { + if (Looper.myLooper() != looper) { + CountDownLatch latch = new CountDownLatch(1); + boolean posted = + handler.post( + () -> { + try { + closeInternal(); + } finally { + latch.countDown(); + } + }); + if (posted) { + try { + if (!latch.await(CLOSE_TIMEOUT_MS, MILLISECONDS)) { + listener.onError( + new VideoFrameProcessingException( + "Timeout waiting for SurfaceToFrameWriterAdapter to close")); + } + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + listener.onError(new VideoFrameProcessingException(e)); + } + } + } else { + closeInternal(); + } + } + + private void closeInternal() { + checkState(Looper.myLooper() == looper); + if (state == STATE_CLOSED) { + return; + } + state = STATE_CLOSED; + baseFrameProcessor.close(); + frameToSurfaceFrameWriter.close(); + sessionManager.release(); + surfaceToFrameWriterAdapter.close(); + if (handlerThread != null) { + handlerThread.quitSafely(); + } + } + + private void onSessionReady(EnhancementSession session, EnhancementOptions options) { + checkState(Looper.myLooper() == looper); + if (state == STATE_CLOSED || state == STATE_ERROR) { + sessionManager.release(); + return; + } else { + checkState(state == STATE_INITIALIZING); + state = STATE_READY; + } + try { + Format currentInputFormat = checkNotNull(inputFormat); + int outputWidth = options.getWidth(); + int outputHeight = options.getHeight(); + if (options.isUpscaleVideoEnabled()) { + outputWidth *= UPSCALE_FACTOR; + outputHeight *= UPSCALE_FACTOR; + } + Surface outputSurface = + surfaceToFrameWriterAdapter.configure(outputWidth, outputHeight, currentInputFormat); + session.setOutputSurface( + outputSurface, + options, + // EnhancementCallback methods called on undefined thread. + new EnhancementCallback() { + @Override + public void onSurfaceProcessed(long timestamp) {} + + @Override + public void onBitmapProcessed(Bitmap bitmap) {} + + @Override + public void onError(int statusCode) { + handleError( + new VideoFrameProcessingException( + "Google Play services video enhancement error: " + statusCode)); + } + + @Override + public void onCancelled(int statusCode) { + handleError( + new VideoFrameProcessingException( + "Google Play services video enhancement cancelled: " + statusCode)); + } + }); + frameToSurfaceFrameWriter.setOutputSurface( + session.getInputSurface(), options.getWidth(), options.getHeight()); + } catch (RuntimeException e) { + sessionManager.release(); + handleError(e); + } + } + + /** Called on any thread. */ + private void handleError(Exception e) { + handlerExecutor.execute( + () -> { + if (state == STATE_ERROR || state == STATE_CLOSED) { + return; + } + state = STATE_ERROR; + closeInternal(); + }); + listenerExecutor.execute(() -> listener.onError(VideoFrameProcessingException.from(e))); + } + + /** Methods called on {@link #looper}. */ + private final class SessionManagerListener implements EnhancementSessionManager.Listener { + @Override + public void onSessionReady(EnhancementSession session, EnhancementOptions options) { + checkState(Looper.myLooper() == looper); + EnhancementSessionDecorator.this.onSessionReady(session, options); + } + + @Override + public void onError(VideoFrameProcessingException exception) { + checkState(Looper.myLooper() == looper); + handleError(exception); + } + } + + /** Methods called on {@link #looper}. */ + private final class SurfaceToFrameWriterAdapterListener + implements SurfaceToFrameWriterAdapter.Listener { + @Override + public void onFrameReleased() { + checkState(Looper.myLooper() == looper); + frameToSurfaceFrameWriter.onFrameReleased(); + listenerExecutor.execute(listener::onWakeup); + } + + @Override + public void onError(VideoFrameProcessingException exception) { + checkState(Looper.myLooper() == looper); + handleError(exception); + } + } + + /** Methods called on {@link #listenerExecutor}. */ + private final class BaseFrameProcessorListener implements Listener { + @Override + public void onWakeup() { + listener.onWakeup(); + } + + @Override + public void onError(VideoFrameProcessingException exception) { + handleError(exception); + } + + @Override + public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper onCompleteFence) { + listener.onFrameProcessed(frame, onCompleteFence); + } + } + + /** Methods called on {@link #looper}. */ + private final class FrameToSurfaceFrameWriterListener + implements FrameToSurfaceFrameWriter.Listener { + @Override + public void onFormatConfigured(Format format) { + checkState(Looper.myLooper() == looper); + if (state == STATE_WAITING_FOR_FORMAT) { + inputFormat = format; + sessionManager.initializeSession( + format.width, + format.height, + isTonemappingEnabled, + isDeblurAndDenoiseVideoEnabled, + isUpscaleVideoEnabled); + state = STATE_INITIALIZING; + } + } + + @Override + public void onFrameQueued(long presentationTimeUs) { + checkState(Looper.myLooper() == looper); + surfaceToFrameWriterAdapter.onInputFrameQueued(presentationTimeUs); + } + + @Override + public void onEndOfStream() { + checkState(Looper.myLooper() == looper); + surfaceToFrameWriterAdapter.onEndOfStream(); + } + + @Override + public void onError(VideoFrameProcessingException exception) { + checkState(Looper.myLooper() == looper); + handleError(exception); + } + } +} diff --git a/libraries/effect_play_services/src/main/java/androidx/media3/effect/playservices/EnhancementSessionManager.java b/libraries/effect_play_services/src/main/java/androidx/media3/effect/playservices/EnhancementSessionManager.java new file mode 100644 index 00000000000..02cc57ab053 --- /dev/null +++ b/libraries/effect_play_services/src/main/java/androidx/media3/effect/playservices/EnhancementSessionManager.java @@ -0,0 +1,288 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect.playservices; + +import static com.google.common.base.Preconditions.checkArgument; +import static com.google.common.base.Preconditions.checkState; + +import android.content.Context; +import androidx.annotation.GuardedBy; +import androidx.annotation.IntRange; +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import androidx.media3.common.VideoFrameProcessingException; +import com.google.android.gms.common.api.Status; +import com.google.android.gms.media.effect.enhancement.Enhancement; +import com.google.android.gms.media.effect.enhancement.EnhancementClient; +import com.google.android.gms.media.effect.enhancement.EnhancementMode; +import com.google.android.gms.media.effect.enhancement.EnhancementOptions; +import com.google.android.gms.media.effect.enhancement.EnhancementSession; +import com.google.android.gms.media.effect.enhancement.EnhancementSessionCallback; +import com.google.android.gms.tasks.Task; +import com.google.android.gms.tasks.Tasks; +import java.util.concurrent.Executor; + +/** + * Manages Google Play services {@link EnhancementClient} connection and {@link EnhancementSession} + * lifecycle. + * + *

Handles device capability verification, optional module installation via Google Play services, + * and session establishment using task chaining. + */ +@RequiresApi(30) +/* package */ final class EnhancementSessionManager { + + /** Listener for session lifecycle events. */ + interface Listener { + /** Called when the enhancement session has been successfully created and is ready. */ + void onSessionReady(EnhancementSession session, EnhancementOptions options); + + /** Called when an unrecoverable error occurs during session initialization. */ + void onError(VideoFrameProcessingException exception); + } + + /** Abstraction over {@link EnhancementClient} for testing. */ + interface Client { + Task isDeviceSupported(); + + Task isModuleInstalled(); + + Task installModule(@Nullable EnhancementClient.InstallStatusCallback callback); + + Task createSession( + EnhancementOptions options, EnhancementSessionCallback callback); + } + + private final Object lock = new Object(); + private final Executor handlerExecutor; + private final Listener listener; + private final Client client; + + @GuardedBy("lock") + @Nullable + private EnhancementSession session; + + @GuardedBy("lock") + private boolean isInitialized; + + @GuardedBy("lock") + private boolean isReleased; + + @GuardedBy("lock") + private boolean hasNotifiedError; + + /** + * Creates an instance. + * + * @param handlerExecutor The executor on which session callbacks and listener events are run. + * @param listener The listener notified of session readiness or errors. + * @param client The client used to query support, install the module, and create sessions. + */ + /* package */ EnhancementSessionManager( + Executor handlerExecutor, Listener listener, Client client) { + this.handlerExecutor = handlerExecutor; + this.listener = listener; + this.client = client; + } + + /** + * Initializes an enhancement session asynchronously. + * + *

Must be called at most once. + * + * @param width Input frame width in pixels. + * @param height Input frame height in pixels. + * @param isTonemappingEnabled Whether tone mapping is enabled. + * @param isDeblurAndDenoiseVideoEnabled Whether video deblurring and denoising is enabled. + * @param isUpscaleVideoEnabled Whether 2x video upscaling is enabled. + */ + public void initializeSession( + @IntRange(from = 1) int width, + @IntRange(from = 1) int height, + boolean isTonemappingEnabled, + boolean isDeblurAndDenoiseVideoEnabled, + boolean isUpscaleVideoEnabled) { + checkArgument(width >= 1); + checkArgument(height >= 1); + synchronized (lock) { + checkState(!isInitialized, "initializeSession must only be called once"); + isInitialized = true; + if (isReleased) { + return; + } + } + + EnhancementOptions options = + new EnhancementOptions( + width, + height, + EnhancementMode.SURFACE, + isTonemappingEnabled, + /* isDeblurAndDenoisePhotoEnabled= */ false, + isDeblurAndDenoiseVideoEnabled, + /* isUpscalePhotoEnabled= */ false, + isUpscaleVideoEnabled); + + client + .isDeviceSupported() + .onSuccessTask( + handlerExecutor, + supported -> { + if (supported == null || !supported) { + throw new VideoFrameProcessingException( + "Device not supported for Google Play services video enhancement"); + } + return client.isModuleInstalled(); + }) + .onSuccessTask( + handlerExecutor, + installed -> { + if (installed != null && installed) { + return Tasks.forResult(Boolean.TRUE); + } + synchronized (lock) { + if (isReleased) { + return Tasks.forCanceled(); + } + } + return client.installModule(/* callback= */ null); + }) + .onSuccessTask( + handlerExecutor, + installed -> { + if (installed == null || !installed) { + throw new VideoFrameProcessingException( + "Failed to install Google Play services video enhancement module"); + } + synchronized (lock) { + if (isReleased) { + return Tasks.forCanceled(); + } + } + return client.createSession(options, new SessionCallback(options)); + }) + .onSuccessTask( + handlerExecutor, + createdSession -> { + if (createdSession == null) { + throw new VideoFrameProcessingException("Session creation returned null"); + } + return Tasks.forResult(createdSession); + }) + .addOnFailureListener( + handlerExecutor, + e -> + notifyError( + e instanceof VideoFrameProcessingException + ? (VideoFrameProcessingException) e + : new VideoFrameProcessingException(e))); + } + + /** Releases the underlying enhancement session and resources. */ + public void release() { + @Nullable EnhancementSession sessionToRelease; + synchronized (lock) { + isReleased = true; + sessionToRelease = session; + session = null; + } + if (sessionToRelease != null) { + sessionToRelease.release(); + } + } + + private void notifyError(VideoFrameProcessingException exception) { + synchronized (lock) { + if (isReleased || hasNotifiedError) { + return; + } + hasNotifiedError = true; + } + listener.onError(exception); + } + + /* package */ static final class PlayServicesClientAdapter implements Client { + private final EnhancementClient enhancementClient; + + PlayServicesClientAdapter(Context context) { + enhancementClient = Enhancement.getClient(context.getApplicationContext()); + } + + @Override + public Task isDeviceSupported() { + return enhancementClient.isDeviceSupported(); + } + + @Override + public Task isModuleInstalled() { + return enhancementClient.isModuleInstalled(); + } + + @Override + public Task installModule(@Nullable EnhancementClient.InstallStatusCallback callback) { + return enhancementClient.installModule(callback); + } + + @Override + public Task createSession( + EnhancementOptions options, EnhancementSessionCallback callback) { + return enhancementClient.createSession(options, callback); + } + } + + private final class SessionCallback implements EnhancementSessionCallback { + private final EnhancementOptions options; + + SessionCallback(EnhancementOptions options) { + this.options = options; + } + + @Override + public void onSessionCreated(EnhancementSession session) { + handlerExecutor.execute( + () -> { + boolean shouldRelease; + synchronized (lock) { + shouldRelease = isReleased; + if (!shouldRelease) { + EnhancementSessionManager.this.session = session; + } + } + if (shouldRelease) { + session.release(); + return; + } + listener.onSessionReady(session, options); + }); + } + + @Override + public void onSessionCreationFailed(Status status) { + handlerExecutor.execute( + () -> + notifyError(new VideoFrameProcessingException("Session creation failed: " + status))); + } + + @Override + public void onSessionDisconnected(Status status) { + handlerExecutor.execute( + () -> notifyError(new VideoFrameProcessingException("Session disconnected: " + status))); + } + + @Override + public void onSessionDestroyed() {} + } +} diff --git a/libraries/effect_play_services/src/main/java/androidx/media3/effect/playservices/FrameToSurfaceFrameWriter.java b/libraries/effect_play_services/src/main/java/androidx/media3/effect/playservices/FrameToSurfaceFrameWriter.java new file mode 100644 index 00000000000..6ece0a5d646 --- /dev/null +++ b/libraries/effect_play_services/src/main/java/androidx/media3/effect/playservices/FrameToSurfaceFrameWriter.java @@ -0,0 +1,254 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect.playservices; + +import static com.google.common.base.Preconditions.checkArgument; +import static com.google.common.base.Preconditions.checkNotNull; +import static com.google.common.base.Preconditions.checkState; + +import android.hardware.HardwareBuffer; +import android.media.Image; +import android.media.ImageWriter; +import android.view.Surface; +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import androidx.media3.common.Format; +import androidx.media3.common.VideoFrameProcessingException; +import androidx.media3.common.util.NullableType; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.DefaultHardwareBufferFrame; +import androidx.media3.common.video.Frame; +import androidx.media3.common.video.FrameWriter; +import androidx.media3.common.video.SyncFenceWrapper; +import java.io.IOException; +import java.util.concurrent.Executor; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.concurrent.atomic.AtomicReference; +import org.checkerframework.checker.nullness.qual.MonotonicNonNull; + +/** + * A {@link FrameWriter} that receives input frames from upstream and feeds them into a downstream + * {@link Surface} via an {@link ImageWriter}. + * + *

Handles zero-copy {@link HardwareBuffer} wrapping, sync fence propagation, and backpressure + * coordination via atomic wakeup listeners. + */ +@RequiresApi(33) +/* package */ final class FrameToSurfaceFrameWriter implements FrameWriter { + + // Bounded to 1 because the downstream Surface drops frames if multiple are pending concurrently. + // Tracks frames in flight across ImageWriter and downstream processing until onFrameReleased(). + private static final int MAX_IN_FLIGHT_FRAMES = 1; + + /** + * Listener for events from {@link FrameToSurfaceFrameWriter}. + * + *

All listener methods are invoked on the {@link Executor} passed to the constructor. + */ + interface Listener { + /** Called when the input format has been configured. */ + void onFormatConfigured(Format format); + + /** + * Called when a frame with the given presentation timestamp (in microseconds) is about to be + * queued into the {@link Surface}. + * + *

While the queued {@link Image} is stamped with a synthetic monotonically increasing + * timestamp from {@link MonotonicTimestampGenerator}, this callback passes the original + * upstream {@code presentationTimeUs}. + * + *

Used in conjunction with {@link #onFrameReleased()} to ensure only one frame is on the + * {@link Surface} at any one point so that the {@link Surface} does not drop frames. + */ + void onFrameQueued(long presentationTimeUs); + + /** Called when a frame processing error occurs. */ + void onError(VideoFrameProcessingException exception); + + /** Called when end-of-stream has been received from upstream. */ + void onEndOfStream(); + } + + private static final class WakeupListenerHolder { + final Executor executor; + final Runnable listener; + + WakeupListenerHolder(Executor executor, Runnable listener) { + this.executor = executor; + this.listener = listener; + } + } + + private final Executor listenerExecutor; + private final Listener listener; + + // Coordinates capacity and backpressure wakeups between the upstream frame processor thread + // (which calls dequeueInputFrame, queueInputFrame, and close) and the downstream thread (which + // calls onFrameReleased when an output frame has been consumed or setOutputSurface when ready) + // without locking. + private final AtomicInteger inFlightCount; + private final AtomicReference<@NullableType WakeupListenerHolder> pendingWakeup; + private final MonotonicTimestampGenerator timestampGenerator; + + private volatile @MonotonicNonNull ImageWriter imageWriter; + + @Nullable private volatile Format configuredFormat; + + public FrameToSurfaceFrameWriter(Executor listenerExecutor, Listener listener) { + this.listenerExecutor = listenerExecutor; + this.listener = listener; + this.inFlightCount = new AtomicInteger(0); + this.pendingWakeup = new AtomicReference<>(); + timestampGenerator = new MonotonicTimestampGenerator(); + } + + /** Sets the target output {@link Surface} and wakes up any waiting upstream producer. */ + public void setOutputSurface(Surface surface, int width, int height) { + checkState(imageWriter == null, "Surface already configured"); + imageWriter = + new ImageWriter.Builder(surface) + .setWidthAndHeight(width, height) + .setMaxImages(MAX_IN_FLIGHT_FRAMES) + .setUsage( + HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE | HardwareBuffer.USAGE_GPU_COLOR_OUTPUT) + .build(); + wakeup(); + } + + @Override + public Info getInfo() { + // TODO(b/531659135): Implement getInfo() to validate supported formats and usages (e.g. + // RGBA_8888 input, possibly of specific resolution). + return (format, usage) -> true; + } + + @Override + public void configure(Format format, @Frame.Usage long usage) { + // TODO(b/531659135): Take usage flags into account when setting up the ImageWriter. + checkState(configuredFormat == null, "configure() must only be called once"); + configuredFormat = format; + listenerExecutor.execute(() -> listener.onFormatConfigured(format)); + } + + @Nullable + @Override + public AsyncFrame dequeueInputFrame(Executor wakeupExecutor, Runnable wakeupListener) { + // Atomically register the wakeup listener before checking capacity to avoid missed wakeups. + pendingWakeup.set(new WakeupListenerHolder(wakeupExecutor, wakeupListener)); + + if (inFlightCount.get() >= MAX_IN_FLIGHT_FRAMES) { + return null; + } + ImageWriter currentImageWriter = imageWriter; + if (currentImageWriter == null) { + return null; + } + Format format = checkNotNull(configuredFormat); + Image image; + try { + image = currentImageWriter.dequeueInputImage(); + } catch (IllegalStateException e) { + // ImageWriter#dequeueInputImage throws IllegalStateException when all buffer slots + // (maxImages) are currently dequeued. Map this to returning null per FrameWriter contract. + return null; + } + if (image == null) { + return null; + } + HardwareBuffer hardwareBuffer = image.getHardwareBuffer(); + if (hardwareBuffer == null) { + image.close(); + throw new IllegalStateException("Dequeued image does not have a HardwareBuffer"); + } + pendingWakeup.set(null); + DefaultHardwareBufferFrame frame = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer) + .setFormat(format) + .setInternalImage(image) + .build(); + return new AsyncFrame(frame, /* acquireFence= */ null); + } + + @Override + public void queueInputFrame(Frame frame, @Nullable SyncFenceWrapper writeCompleteFence) { + checkArgument( + frame instanceof DefaultHardwareBufferFrame, + "Expected DefaultHardwareBufferFrame, got %s", + frame.getClass().getName()); + DefaultHardwareBufferFrame hardwareBufferFrame = (DefaultHardwareBufferFrame) frame; + Image image = (Image) checkNotNull(hardwareBufferFrame.getInternalImage()); + try { + long presentationTimeUs = frame.getContentTimeUs(); + image.setTimestamp(timestampGenerator.getNextTimestampNs()); + if (writeCompleteFence != null) { + image.setFence(writeCompleteFence.asSyncFence()); + } + ImageWriter currentImageWriter = checkNotNull(imageWriter); + inFlightCount.incrementAndGet(); + listenerExecutor.execute(() -> listener.onFrameQueued(presentationTimeUs)); + try { + currentImageWriter.queueInputImage(image); + } catch (IllegalStateException e) { + inFlightCount.decrementAndGet(); + throw e; + } + } catch (IllegalStateException | IOException e) { + try { + image.close(); + } catch (RuntimeException ignored) { + // Ignore if image is already closed. + } + listenerExecutor.execute( + () -> + listener.onError( + new VideoFrameProcessingException("Failed to queue input image", e))); + } + } + + @Override + public void signalEndOfStream() { + listenerExecutor.execute(listener::onEndOfStream); + } + + @Override + public void close() { + pendingWakeup.set(null); + ImageWriter writer = imageWriter; + if (writer != null) { + writer.close(); + } + } + + /** + * Notifies the frame writer that a downstream output frame has been processed and released, + * decrementing the in-flight frame count and waking up any waiting upstream producer. + */ + public void onFrameReleased() { + inFlightCount.decrementAndGet(); + wakeup(); + } + + /** + * Notifies the frame writer that downstream capacity or session state has freed up, triggering + * any pending wakeup listener. + */ + private void wakeup() { + WakeupListenerHolder holder = pendingWakeup.getAndSet(null); + if (holder != null) { + holder.executor.execute(holder.listener); + } + } +} diff --git a/libraries/effect_play_services/src/main/java/androidx/media3/effect/playservices/MonotonicTimestampGenerator.java b/libraries/effect_play_services/src/main/java/androidx/media3/effect/playservices/MonotonicTimestampGenerator.java new file mode 100644 index 00000000000..f57f318c8bf --- /dev/null +++ b/libraries/effect_play_services/src/main/java/androidx/media3/effect/playservices/MonotonicTimestampGenerator.java @@ -0,0 +1,32 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect.playservices; + +/** Generator of strictly monotonically increasing synthetic timestamps in nanoseconds. */ +/* package */ final class MonotonicTimestampGenerator { + + // 1 microsecond in nanoseconds so timestamps remain strictly increasing after conversion to + // microseconds. + private static final long TIMESTAMP_STEP_NS = 1_000L; + + private long nextTimestampNs; + + public long getNextTimestampNs() { + long timestampNs = nextTimestampNs; + nextTimestampNs += TIMESTAMP_STEP_NS; + return timestampNs; + } +} diff --git a/libraries/effect_play_services/src/main/java/androidx/media3/effect/playservices/SurfaceToFrameWriterAdapter.java b/libraries/effect_play_services/src/main/java/androidx/media3/effect/playservices/SurfaceToFrameWriterAdapter.java new file mode 100644 index 00000000000..a571bbb200e --- /dev/null +++ b/libraries/effect_play_services/src/main/java/androidx/media3/effect/playservices/SurfaceToFrameWriterAdapter.java @@ -0,0 +1,304 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect.playservices; + +import static com.google.common.base.Preconditions.checkNotNull; +import static com.google.common.base.Preconditions.checkState; + +import android.graphics.PixelFormat; +import android.hardware.HardwareBuffer; +import android.media.Image; +import android.media.ImageReader; +import android.os.Handler; +import android.os.Looper; +import android.view.Surface; +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import androidx.media3.common.Format; +import androidx.media3.common.VideoFrameProcessingException; +import androidx.media3.common.util.HandlerExecutor; +import androidx.media3.common.util.LongArrayQueue; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.Frame; +import androidx.media3.common.video.FrameWriter; +import androidx.media3.common.video.HardwareBufferFrame; +import androidx.media3.common.video.HardwareBufferNativeHelpers; +import java.util.concurrent.Executor; + +/** + * Creates and provides an output {@link Surface} and forwards frames rendered to it into a + * downstream {@link FrameWriter} via a hardware buffer copy. + * + *

Handles {@link ImageReader} buffer acquisition, native hardware buffer copying, and + * end-of-stream signaling. + * + *

Thread safety: All methods must be called on the thread associated with the {@link Handler} + * passed to the constructor. + */ +@RequiresApi(33) +/* package */ final class SurfaceToFrameWriterAdapter implements AutoCloseable { + + private static final int MAX_IMAGES = 1; + + /** + * Listener for {@link SurfaceToFrameWriterAdapter} events. + * + *

All methods are invoked on the handler thread. + */ + interface Listener { + /** Called on the handler thread when an output frame has been released back to the surface. */ + void onFrameReleased(); + + /** Called when an error occurs during output frame draining or copying. */ + void onError(VideoFrameProcessingException exception); + } + + private final FrameWriter downstreamOutput; + private final HardwareBufferNativeHelpers hardwareBufferNativeHelpers; + private final Handler handler; + private final Executor handlerExecutor; + private final Listener listener; + private final LongArrayQueue inputPresentationTimestampsUs; + + @Nullable private ImageReader imageReader; + @Nullable private Surface surface; + @Nullable private Image pendingImage; + + // All state counters and flags below are accessed exclusively on the Handler thread. + private int totalInputFramesCount; + private int outputFramesCount; + private boolean isEndOfStreamReceived; + private boolean isDownstreamEosSignaled; + private volatile boolean isClosed; + + /** + * Creates a new instance. + * + * @param downstreamOutput The downstream {@link FrameWriter} to which copied output frames are + * forwarded. + * @param hardwareBufferNativeHelpers Native helper for copying {@link HardwareBuffer} contents. + * @param handler The {@link Handler} whose thread is used for all public method calls, {@link + * ImageReader} callbacks, downstream wakeup notifications, and {@link Listener} callbacks. + * @param listener The {@link Listener} notified of image releases and errors on the {@code + * handler} thread. + */ + public SurfaceToFrameWriterAdapter( + FrameWriter downstreamOutput, + HardwareBufferNativeHelpers hardwareBufferNativeHelpers, + Handler handler, + Listener listener) { + this.downstreamOutput = downstreamOutput; + this.hardwareBufferNativeHelpers = hardwareBufferNativeHelpers; + this.handler = handler; + this.handlerExecutor = + new HandlerExecutor(handler, e -> listener.onError(VideoFrameProcessingException.from(e))); + this.listener = listener; + inputPresentationTimestampsUs = new LongArrayQueue(); + } + + /** + * Configures the surface and the downstream {@link FrameWriter}. + * + *

Must be called on a handler thread. + * + * @param width The surface width in pixels. + * @param height The surface height in pixels. + * @param inputFormat The input {@link Format}. + * @return The {@link Surface} created by this instance. + */ + public Surface configure(int width, int height, Format inputFormat) { + checkState(Looper.myLooper() == handler.getLooper()); + checkState(imageReader == null); + // TODO(b/531659135): Set ColorInfo on outputFormat. + Format outputFormat = inputFormat.buildUpon().setWidth(width).setHeight(height).build(); + downstreamOutput.configure( + outputFormat, + Frame.USAGE_GPU_COLOR_OUTPUT | Frame.USAGE_GPU_SAMPLED_IMAGE | Frame.USAGE_CPU_WRITE_OFTEN); + + imageReader = + ImageReader.newInstance( + width, + height, + PixelFormat.RGBA_8888, + MAX_IMAGES, + HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE + | HardwareBuffer.USAGE_GPU_COLOR_OUTPUT + | HardwareBuffer.USAGE_CPU_READ_OFTEN); + + imageReader.setOnImageAvailableListener(this::onImageAvailable, handler); + Surface configuredSurface = imageReader.getSurface(); + surface = configuredSurface; + return configuredSurface; + } + + /** + * Notifies the adapter that an input frame is queued to the surface. + * + *

Records the original upstream {@code presentationTimeUs} (forwarded by {@link + * EnhancementSessionDecorator} from {@link FrameToSurfaceFrameWriter.Listener#onFrameQueued}). + * + *

Must be called on a handler thread. + */ + public void onInputFrameQueued(long presentationTimeUs) { + checkState(Looper.myLooper() == handler.getLooper()); + checkState(!isClosed && !isEndOfStreamReceived); + inputPresentationTimestampsUs.add(presentationTimeUs); + totalInputFramesCount++; + } + + /** + * Signals end-of-stream on input. Checks if downstream EOS can be signaled immediately. + * + *

Must be called on a handler thread. + */ + public void onEndOfStream() { + checkState(Looper.myLooper() == handler.getLooper()); + isEndOfStreamReceived = true; + maybeSignalDownstreamEos(); + } + + /** + * Closes the adapter and releases all surfaces and buffers. May be called from any thread. If + * called from a thread other than the handler thread, this method blocks until cleanup finishes + * on the handler thread or times out. + */ + @Override + public void close() { + checkState(Looper.myLooper() == handler.getLooper()); + if (isClosed) { + return; + } + isClosed = true; + inputPresentationTimestampsUs.clear(); + if (pendingImage != null) { + pendingImage.close(); + pendingImage = null; + } + if (imageReader != null) { + imageReader.close(); + imageReader = null; + } + if (surface != null) { + surface.release(); + surface = null; + } + } + + private void onImageAvailable(ImageReader reader) { + if (isClosed) { + return; + } + tryProcessPendingOutputs(); + } + + private void tryProcessPendingOutputs() { + if (isClosed) { + return; + } + + ImageReader reader = imageReader; + if (reader == null) { + return; + } + + Image image = pendingImage; + if (image == null) { + try { + image = reader.acquireNextImage(); + } catch (IllegalStateException e) { + image = null; + } + } + + if (image == null) { + return; + } + + if (outputFramesCount >= totalInputFramesCount) { + close(); + listener.onError( + new VideoFrameProcessingException( + "Unexpected output frame acquired beyond input frame count")); + return; + } + + AsyncFrame outputFrame = + downstreamOutput.dequeueInputFrame(handlerExecutor, this::tryProcessPendingOutputs); + if (outputFrame == null) { + pendingImage = image; + return; + } + + pendingImage = null; + copyOutputAndQueue(image, outputFrame); + } + + private void copyOutputAndQueue(Image sourceImage, AsyncFrame targetFrame) { + VideoFrameProcessingException copyException = null; + Frame newFrame = null; + try { + if (targetFrame.acquireFence != null) { + targetFrame.acquireFence.awaitForever(); + targetFrame.acquireFence.close(); + } + + try (HardwareBuffer sourceBuffer = checkNotNull(sourceImage.getHardwareBuffer())) { + HardwareBuffer targetBuffer = ((HardwareBufferFrame) targetFrame.frame).getHardwareBuffer(); + boolean copySuccess = + hardwareBufferNativeHelpers.nativeCopyHardwareBufferToHardwareBuffer( + sourceBuffer, targetBuffer); + if (copySuccess) { + newFrame = + ((HardwareBufferFrame) targetFrame.frame) + .buildUpon() + .setContentTimeUs(inputPresentationTimestampsUs.remove()) + .build(); + } else { + copyException = + new VideoFrameProcessingException( + "nativeCopyHardwareBufferToHardwareBuffer failed for output"); + } + } + } catch (RuntimeException e) { + copyException = new VideoFrameProcessingException("Failed to copy output frame", e); + } finally { + sourceImage.close(); + listener.onFrameReleased(); + } + + if (newFrame != null) { + outputFramesCount++; + downstreamOutput.queueInputFrame(newFrame, /* writeCompleteFence= */ null); + } + if (copyException != null) { + listener.onError(copyException); + } + maybeSignalDownstreamEos(); + } + + /** + * Signals end-of-stream to the downstream {@link FrameWriter} once end-of-stream has been + * received on input and all registered input frames have been processed and forwarded downstream. + */ + private void maybeSignalDownstreamEos() { + if (isEndOfStreamReceived + && outputFramesCount >= totalInputFramesCount + && !isDownstreamEosSignaled) { + isDownstreamEosSignaled = true; + downstreamOutput.signalEndOfStream(); + } + } +} diff --git a/libraries/effect_ndk/src/androidTest/java/androidx/media3/effect/ndk/package-info.java b/libraries/effect_play_services/src/main/java/androidx/media3/effect/playservices/package-info.java similarity index 93% rename from libraries/effect_ndk/src/androidTest/java/androidx/media3/effect/ndk/package-info.java rename to libraries/effect_play_services/src/main/java/androidx/media3/effect/playservices/package-info.java index cd498c0eaf2..5393401e9e9 100644 --- a/libraries/effect_ndk/src/androidTest/java/androidx/media3/effect/ndk/package-info.java +++ b/libraries/effect_play_services/src/main/java/androidx/media3/effect/playservices/package-info.java @@ -13,8 +13,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ - @NonNullApi -package androidx.media3.effect.ndk; +package androidx.media3.effect.playservices; import androidx.media3.common.util.NonNullApi; diff --git a/libraries/session/src/main/res/values/config.xml b/libraries/effect_play_services/src/test/AndroidManifest.xml similarity index 70% rename from libraries/session/src/main/res/values/config.xml rename to libraries/effect_play_services/src/test/AndroidManifest.xml index 1a048e9e51c..8c3f7c143f7 100644 --- a/libraries/session/src/main/res/values/config.xml +++ b/libraries/effect_play_services/src/test/AndroidManifest.xml @@ -1,5 +1,5 @@ - - - - 127 - + + + diff --git a/libraries/effect_play_services/src/test/java/androidx/media3/effect/playservices/EnhancementSessionDecoratorTest.java b/libraries/effect_play_services/src/test/java/androidx/media3/effect/playservices/EnhancementSessionDecoratorTest.java new file mode 100644 index 00000000000..297a64fb54e --- /dev/null +++ b/libraries/effect_play_services/src/test/java/androidx/media3/effect/playservices/EnhancementSessionDecoratorTest.java @@ -0,0 +1,150 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect.playservices; + +import static androidx.test.core.app.ApplicationProvider.getApplicationContext; +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.util.concurrent.MoreExecutors.directExecutor; +import static org.robolectric.Shadows.shadowOf; + +import android.os.Looper; +import androidx.media3.common.Format; +import androidx.media3.common.VideoFrameProcessingException; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.Frame; +import androidx.media3.common.video.FrameProcessor; +import androidx.media3.test.utils.FakeFrameProcessor; +import androidx.media3.test.utils.FakeFrameWriter; +import androidx.media3.test.utils.FakeHardwareBufferNativeHelpers; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.gms.tasks.Tasks; +import com.google.common.collect.ImmutableList; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Robolectric unit tests for {@link EnhancementSessionDecorator}. */ +@RunWith(AndroidJUnit4.class) +public final class EnhancementSessionDecoratorTest { + + private static final int WIDTH = 128; + private static final int HEIGHT = 128; + + private FakeFrameProcessor.Factory fakeBaseProcessorFactory; + private FakeFrameWriter fakeDownstreamOutput; + private FakeFrameProcessor.Listener testListener; + private EnhancementSessionTestUtil.FakeEnhancementSession fakeSession; + private EnhancementSessionTestUtil.FakeClient fakeClient; + private FrameProcessor decorator; + + @Before + public void setUp() { + fakeBaseProcessorFactory = + new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ true); + fakeDownstreamOutput = new FakeFrameWriter(); + testListener = new FakeFrameProcessor.Listener(); + fakeSession = new EnhancementSessionTestUtil.FakeEnhancementSession(); + fakeClient = new EnhancementSessionTestUtil.FakeClient(fakeSession); + fakeClient.autoTriggerSessionCallback = false; + + decorator = + new EnhancementSessionDecorator.Builder(getApplicationContext(), fakeBaseProcessorFactory) + .setTonemappingEnabled(true) + .setLooper(Looper.getMainLooper()) + .setSessionManagerClient(fakeClient) + .setHardwareBufferNativeHelpers(new FakeHardwareBufferNativeHelpers()) + .build() + .create(fakeDownstreamOutput, directExecutor(), testListener); + } + + @After + public void tearDown() { + if (decorator != null) { + decorator.close(); + } + } + + @Test + public void queue_emptyPacket_forwardsDirectlyToBaseProcessor() { + ImmutableList frames = ImmutableList.of(); + + boolean result = decorator.queue(frames); + + assertThat(result).isTrue(); + assertThat(fakeBaseProcessorFactory.createdProcessor.queuedFramesCount).isEqualTo(1); + } + + @Test + public void configure_validFormat_triggersSessionInitialization() { + Format format = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + + fakeBaseProcessorFactory.createdProcessor.output.configure( + format, Frame.USAGE_GPU_COLOR_OUTPUT); + shadowOf(Looper.getMainLooper()).idle(); + + assertThat(fakeClient.createSessionCalled.get()).isTrue(); + assertThat(fakeClient.lastOptions).isNotNull(); + assertThat(fakeClient.lastOptions.getWidth()).isEqualTo(WIDTH); + assertThat(fakeClient.lastOptions.getHeight()).isEqualTo(HEIGHT); + assertThat(fakeClient.lastOptions.isTonemappingEnabled()).isTrue(); + assertThat(fakeClient.lastOptions.isDeblurAndDenoiseVideoEnabled()).isFalse(); + assertThat(fakeClient.lastOptions.isUpscaleVideoEnabled()).isFalse(); + } + + @Test + public void close_beforeSessionCreated_releasesSessionWhenCreated() { + Format format = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + fakeBaseProcessorFactory.createdProcessor.output.configure( + format, Frame.USAGE_GPU_COLOR_OUTPUT); + shadowOf(Looper.getMainLooper()).idle(); + + decorator.close(); + fakeClient.savedSessionCallback.onSessionCreated(fakeSession); + shadowOf(Looper.getMainLooper()).idle(); + + assertThat(fakeSession.isReleased.get()).isTrue(); + assertThat(testListener.error.get()).isNull(); + } + + @Test + public void configure_whenSessionManagerReportsError_notifiesListener() { + fakeClient.deviceSupportedTask = + Tasks.forException(new VideoFrameProcessingException("Session initialization failed")); + Format format = new Format.Builder().setWidth(WIDTH).setHeight(HEIGHT).build(); + + fakeBaseProcessorFactory.createdProcessor.output.configure( + format, Frame.USAGE_GPU_COLOR_OUTPUT); + shadowOf(Looper.getMainLooper()).idle(); + + assertThat(testListener.error.get()).isNotNull(); + assertThat(testListener.error.get()).hasMessageThat().contains("Session initialization failed"); + } + + @Test + public void signalEndOfStream_whenCalled_forwardsToBaseProcessor() { + decorator.signalEndOfStream(); + + assertThat(fakeBaseProcessorFactory.createdProcessor.eosSignaled.get()).isTrue(); + } + + @Test + public void close_whenCalled_shutsDownBaseProcessor() { + decorator.close(); + + assertThat(fakeBaseProcessorFactory.createdProcessor.closed.get()).isTrue(); + } +} diff --git a/libraries/effect_play_services/src/test/java/androidx/media3/effect/playservices/EnhancementSessionManagerTest.java b/libraries/effect_play_services/src/test/java/androidx/media3/effect/playservices/EnhancementSessionManagerTest.java new file mode 100644 index 00000000000..df8c778aa94 --- /dev/null +++ b/libraries/effect_play_services/src/test/java/androidx/media3/effect/playservices/EnhancementSessionManagerTest.java @@ -0,0 +1,422 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect.playservices; + +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.util.concurrent.MoreExecutors.directExecutor; +import static org.junit.Assert.assertThrows; + +import androidx.media3.common.VideoFrameProcessingException; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.android.gms.common.api.CommonStatusCodes; +import com.google.android.gms.common.api.Status; +import com.google.android.gms.media.effect.enhancement.EnhancementMode; +import com.google.android.gms.media.effect.enhancement.EnhancementOptions; +import com.google.android.gms.media.effect.enhancement.EnhancementSession; +import com.google.android.gms.tasks.TaskCompletionSource; +import com.google.android.gms.tasks.Tasks; +import java.io.IOException; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.concurrent.atomic.AtomicReference; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.annotation.Config; + +/** Unit tests for {@link EnhancementSessionManager}. */ +@RunWith(AndroidJUnit4.class) +@Config(minSdk = 30) +public final class EnhancementSessionManagerTest { + + private static final int WIDTH = 640; + private static final int HEIGHT = 480; + + private EnhancementSessionTestUtil.FakeClient fakeClient; + private EnhancementSessionTestUtil.FakeEnhancementSession fakeSession; + private TestListener testListener; + private EnhancementSessionManager sessionManager; + + @Before + public void setUp() { + fakeSession = new EnhancementSessionTestUtil.FakeEnhancementSession(); + fakeClient = new EnhancementSessionTestUtil.FakeClient(fakeSession); + testListener = new TestListener(); + sessionManager = new EnhancementSessionManager(directExecutor(), testListener, fakeClient); + } + + @Test + public void initializeSession_whenDeviceNotSupported_triggersOnError() { + fakeClient.deviceSupportedTask = Tasks.forResult(false); + + sessionManager.initializeSession( + WIDTH, + HEIGHT, + /* isTonemappingEnabled= */ false, + /* isDeblurAndDenoiseVideoEnabled= */ false, + /* isUpscaleVideoEnabled= */ false); + + assertThat(testListener.error.get()).isNotNull(); + assertThat(testListener.error.get()).hasMessageThat().contains("Device not supported"); + assertThat(testListener.readySession.get()).isNull(); + } + + @Test + public void initializeSession_whenDeviceCheckFails_triggersOnError() { + fakeClient.deviceSupportedTask = Tasks.forException(new IOException("Service disconnected")); + + sessionManager.initializeSession( + WIDTH, + HEIGHT, + /* isTonemappingEnabled= */ false, + /* isDeblurAndDenoiseVideoEnabled= */ false, + /* isUpscaleVideoEnabled= */ false); + + assertThat(testListener.error.get()).isNotNull(); + assertThat(testListener.readySession.get()).isNull(); + } + + @Test + public void initializeSession_whenModuleAlreadyInstalled_skipsInstallAndCreatesSession() { + fakeClient.deviceSupportedTask = Tasks.forResult(true); + fakeClient.moduleInstalledTask = Tasks.forResult(true); + + sessionManager.initializeSession( + WIDTH, + HEIGHT, + /* isTonemappingEnabled= */ true, + /* isDeblurAndDenoiseVideoEnabled= */ false, + /* isUpscaleVideoEnabled= */ false); + + assertThat(fakeClient.installModuleCalled.get()).isFalse(); + assertThat(fakeClient.createSessionCalled.get()).isTrue(); + assertThat(testListener.readySession.get()).isEqualTo(fakeSession); + assertThat(testListener.readyOptions.get()).isNotNull(); + assertThat(testListener.readyOptions.get().isTonemappingEnabled()).isTrue(); + assertThat(testListener.error.get()).isNull(); + } + + @Test + public void initializeSession_whenModuleNotInstalled_installsModuleAndCreatesSession() { + fakeClient.deviceSupportedTask = Tasks.forResult(true); + fakeClient.moduleInstalledTask = Tasks.forResult(false); + fakeClient.installModuleTask = Tasks.forResult(true); + + sessionManager.initializeSession( + WIDTH, + HEIGHT, + /* isTonemappingEnabled= */ false, + /* isDeblurAndDenoiseVideoEnabled= */ true, + /* isUpscaleVideoEnabled= */ false); + + assertThat(fakeClient.installModuleCalled.get()).isTrue(); + assertThat(fakeClient.createSessionCalled.get()).isTrue(); + assertThat(testListener.readySession.get()).isEqualTo(fakeSession); + assertThat(testListener.readyOptions.get().isDeblurAndDenoiseVideoEnabled()).isTrue(); + assertThat(testListener.error.get()).isNull(); + } + + @Test + public void initializeSession_whenModuleInstallFails_triggersOnError() { + fakeClient.deviceSupportedTask = Tasks.forResult(true); + fakeClient.moduleInstalledTask = Tasks.forResult(false); + fakeClient.installModuleTask = Tasks.forResult(false); + + sessionManager.initializeSession( + WIDTH, + HEIGHT, + /* isTonemappingEnabled= */ false, + /* isDeblurAndDenoiseVideoEnabled= */ false, + /* isUpscaleVideoEnabled= */ false); + + assertThat(fakeClient.installModuleCalled.get()).isTrue(); + assertThat(fakeClient.createSessionCalled.get()).isFalse(); + assertThat(testListener.error.get()).isNotNull(); + assertThat(testListener.error.get()).hasMessageThat().contains("Failed to install"); + } + + @Test + public void initializeSession_whenSessionCreationFailed_triggersOnError() { + fakeClient.deviceSupportedTask = Tasks.forResult(true); + fakeClient.moduleInstalledTask = Tasks.forResult(true); + fakeClient.autoTriggerSessionCallback = false; + + sessionManager.initializeSession( + WIDTH, + HEIGHT, + /* isTonemappingEnabled= */ false, + /* isDeblurAndDenoiseVideoEnabled= */ false, + /* isUpscaleVideoEnabled= */ false); + + assertThat(fakeClient.savedSessionCallback).isNotNull(); + fakeClient.savedSessionCallback.onSessionCreationFailed( + new Status(CommonStatusCodes.INTERNAL_ERROR, "Initialization failed")); + + assertThat(testListener.error.get()).isNotNull(); + assertThat(testListener.error.get()).hasMessageThat().contains("Session creation failed"); + assertThat(testListener.errorCount.get()).isEqualTo(1); + assertThat(testListener.readySession.get()).isNull(); + } + + @Test + public void + initializeSession_whenSessionCreationFailedAndCreateSessionTaskFails_notifiesErrorOnce() { + fakeClient.deviceSupportedTask = Tasks.forResult(true); + fakeClient.moduleInstalledTask = Tasks.forResult(true); + fakeClient.autoTriggerSessionCallback = false; + fakeClient.createSessionTask = Tasks.forException(new IOException("Creation failed")); + + sessionManager.initializeSession( + WIDTH, + HEIGHT, + /* isTonemappingEnabled= */ false, + /* isDeblurAndDenoiseVideoEnabled= */ false, + /* isUpscaleVideoEnabled= */ false); + + assertThat(fakeClient.savedSessionCallback).isNotNull(); + fakeClient.savedSessionCallback.onSessionCreationFailed( + new Status(CommonStatusCodes.INTERNAL_ERROR, "Initialization failed")); + + assertThat(testListener.error.get()).isNotNull(); + assertThat(testListener.errorCount.get()).isEqualTo(1); + } + + @Test + public void initializeSession_whenSessionDisconnected_triggersOnError() { + fakeClient.deviceSupportedTask = Tasks.forResult(true); + fakeClient.moduleInstalledTask = Tasks.forResult(true); + fakeClient.autoTriggerSessionCallback = false; + + sessionManager.initializeSession( + WIDTH, + HEIGHT, + /* isTonemappingEnabled= */ false, + /* isDeblurAndDenoiseVideoEnabled= */ false, + /* isUpscaleVideoEnabled= */ false); + + assertThat(fakeClient.savedSessionCallback).isNotNull(); + fakeClient.savedSessionCallback.onSessionDisconnected( + new Status(CommonStatusCodes.NETWORK_ERROR, "Service died")); + + assertThat(testListener.error.get()).isNotNull(); + assertThat(testListener.error.get()).hasMessageThat().contains("Session disconnected"); + } + + @Test + public void initializeSession_afterRelease_doesNotCreateSession() { + sessionManager.release(); + + sessionManager.initializeSession( + WIDTH, + HEIGHT, + /* isTonemappingEnabled= */ false, + /* isDeblurAndDenoiseVideoEnabled= */ false, + /* isUpscaleVideoEnabled= */ false); + + assertThat(fakeClient.createSessionCalled.get()).isFalse(); + assertThat(testListener.readySession.get()).isNull(); + assertThat(testListener.error.get()).isNull(); + } + + @Test + public void release_whileCheckingDeviceSupport_doesNotInstallModuleOrCreateSession() { + TaskCompletionSource deviceSupportedCompletionSource = new TaskCompletionSource<>(); + fakeClient.deviceSupportedTask = deviceSupportedCompletionSource.getTask(); + fakeClient.moduleInstalledTask = Tasks.forResult(false); + + sessionManager.initializeSession( + WIDTH, + HEIGHT, + /* isTonemappingEnabled= */ false, + /* isDeblurAndDenoiseVideoEnabled= */ false, + /* isUpscaleVideoEnabled= */ false); + sessionManager.release(); + deviceSupportedCompletionSource.setResult(true); + + assertThat(fakeClient.installModuleCalled.get()).isFalse(); + assertThat(fakeClient.createSessionCalled.get()).isFalse(); + assertThat(testListener.readySession.get()).isNull(); + assertThat(testListener.error.get()).isNull(); + } + + @Test + public void release_beforeSessionCreated_releasesSessionWhenItArrivesAndDoesNotNotifyListener() { + fakeClient.deviceSupportedTask = Tasks.forResult(true); + fakeClient.moduleInstalledTask = Tasks.forResult(true); + fakeClient.autoTriggerSessionCallback = false; + + sessionManager.initializeSession( + WIDTH, + HEIGHT, + /* isTonemappingEnabled= */ false, + /* isDeblurAndDenoiseVideoEnabled= */ false, + /* isUpscaleVideoEnabled= */ false); + + sessionManager.release(); + + assertThat(fakeClient.savedSessionCallback).isNotNull(); + fakeClient.savedSessionCallback.onSessionCreated(fakeSession); + + assertThat(fakeSession.isReleased.get()).isTrue(); + assertThat(testListener.readySession.get()).isNull(); + assertThat(testListener.error.get()).isNull(); + } + + @Test + public void release_beforeSessionCreationFailed_doesNotNotifyListener() { + fakeClient.deviceSupportedTask = Tasks.forResult(true); + fakeClient.moduleInstalledTask = Tasks.forResult(true); + fakeClient.autoTriggerSessionCallback = false; + + sessionManager.initializeSession( + WIDTH, + HEIGHT, + /* isTonemappingEnabled= */ false, + /* isDeblurAndDenoiseVideoEnabled= */ false, + /* isUpscaleVideoEnabled= */ false); + + sessionManager.release(); + + assertThat(fakeClient.savedSessionCallback).isNotNull(); + fakeClient.savedSessionCallback.onSessionCreationFailed( + new Status(CommonStatusCodes.INTERNAL_ERROR, "Initialization failed")); + + assertThat(testListener.error.get()).isNull(); + } + + @Test + public void release_beforeSessionDisconnected_doesNotNotifyListener() { + fakeClient.deviceSupportedTask = Tasks.forResult(true); + fakeClient.moduleInstalledTask = Tasks.forResult(true); + fakeClient.autoTriggerSessionCallback = false; + + sessionManager.initializeSession( + WIDTH, + HEIGHT, + /* isTonemappingEnabled= */ false, + /* isDeblurAndDenoiseVideoEnabled= */ false, + /* isUpscaleVideoEnabled= */ false); + + sessionManager.release(); + + assertThat(fakeClient.savedSessionCallback).isNotNull(); + fakeClient.savedSessionCallback.onSessionDisconnected( + new Status(CommonStatusCodes.NETWORK_ERROR, "Service died")); + + assertThat(testListener.error.get()).isNull(); + } + + @Test + public void release_afterSessionCreated_releasesUnderlyingSession() { + fakeClient.deviceSupportedTask = Tasks.forResult(true); + fakeClient.moduleInstalledTask = Tasks.forResult(true); + + sessionManager.initializeSession( + WIDTH, + HEIGHT, + /* isTonemappingEnabled= */ false, + /* isDeblurAndDenoiseVideoEnabled= */ false, + /* isUpscaleVideoEnabled= */ false); + + assertThat(testListener.readySession.get()).isEqualTo(fakeSession); + assertThat(fakeSession.isReleased.get()).isFalse(); + + sessionManager.release(); + assertThat(fakeSession.isReleased.get()).isTrue(); + } + + @Test + public void initializeSession_configuresCorrectEnhancementOptions() { + fakeClient.deviceSupportedTask = Tasks.forResult(true); + fakeClient.moduleInstalledTask = Tasks.forResult(true); + + sessionManager.initializeSession( + WIDTH, + HEIGHT, + /* isTonemappingEnabled= */ true, + /* isDeblurAndDenoiseVideoEnabled= */ true, + /* isUpscaleVideoEnabled= */ true); + + EnhancementOptions options = testListener.readyOptions.get(); + assertThat(options).isNotNull(); + assertThat(options.getWidth()).isEqualTo(WIDTH); + assertThat(options.getHeight()).isEqualTo(HEIGHT); + assertThat(options.getEnhancementMode()).isEqualTo(EnhancementMode.SURFACE); + assertThat(options.isTonemappingEnabled()).isTrue(); + assertThat(options.isDeblurAndDenoiseVideoEnabled()).isTrue(); + assertThat(options.isUpscaleVideoEnabled()).isTrue(); + } + + @Test + public void initializeSession_whenCreateSessionReturnsNull_triggersOnError() { + fakeClient.deviceSupportedTask = Tasks.forResult(true); + fakeClient.moduleInstalledTask = Tasks.forResult(true); + fakeClient.autoTriggerSessionCallback = false; + fakeClient.createSessionTask = Tasks.forResult(null); + + sessionManager.initializeSession( + WIDTH, + HEIGHT, + /* isTonemappingEnabled= */ false, + /* isDeblurAndDenoiseVideoEnabled= */ false, + /* isUpscaleVideoEnabled= */ false); + + assertThat(testListener.error.get()).isNotNull(); + assertThat(testListener.error.get()) + .hasMessageThat() + .contains("Session creation returned null"); + assertThat(testListener.errorCount.get()).isEqualTo(1); + assertThat(testListener.readySession.get()).isNull(); + } + + @Test + public void initializeSession_whenCalledTwice_throwsIllegalStateException() { + sessionManager.initializeSession( + WIDTH, + HEIGHT, + /* isTonemappingEnabled= */ false, + /* isDeblurAndDenoiseVideoEnabled= */ false, + /* isUpscaleVideoEnabled= */ false); + + assertThrows( + IllegalStateException.class, + () -> + sessionManager.initializeSession( + WIDTH, + HEIGHT, + /* isTonemappingEnabled= */ false, + /* isDeblurAndDenoiseVideoEnabled= */ false, + /* isUpscaleVideoEnabled= */ false)); + } + + private static class TestListener implements EnhancementSessionManager.Listener { + final AtomicReference readySession = new AtomicReference<>(); + final AtomicReference readyOptions = new AtomicReference<>(); + final AtomicReference error = new AtomicReference<>(); + final AtomicInteger errorCount = new AtomicInteger(0); + + @Override + public void onSessionReady(EnhancementSession session, EnhancementOptions options) { + readySession.set(session); + readyOptions.set(options); + } + + @Override + public void onError(VideoFrameProcessingException exception) { + error.set(exception); + errorCount.incrementAndGet(); + } + } +} diff --git a/libraries/effect_play_services/src/test/java/androidx/media3/effect/playservices/EnhancementSessionTestUtil.java b/libraries/effect_play_services/src/test/java/androidx/media3/effect/playservices/EnhancementSessionTestUtil.java new file mode 100644 index 00000000000..30008d272c4 --- /dev/null +++ b/libraries/effect_play_services/src/test/java/androidx/media3/effect/playservices/EnhancementSessionTestUtil.java @@ -0,0 +1,120 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect.playservices; + +import android.graphics.Bitmap; +import android.view.Surface; +import androidx.annotation.Nullable; +import com.google.android.gms.media.effect.enhancement.EnhancementCallback; +import com.google.android.gms.media.effect.enhancement.EnhancementClient; +import com.google.android.gms.media.effect.enhancement.EnhancementOptions; +import com.google.android.gms.media.effect.enhancement.EnhancementSession; +import com.google.android.gms.media.effect.enhancement.EnhancementSessionCallback; +import com.google.android.gms.tasks.Task; +import com.google.android.gms.tasks.Tasks; +import java.util.concurrent.atomic.AtomicBoolean; + +/** Utilities and fakes for {@link EnhancementSessionManager} tests. */ +/* package */ final class EnhancementSessionTestUtil { + + private EnhancementSessionTestUtil() {} + + /** Fake {@link EnhancementSession} for tests. */ + /* package */ static final class FakeEnhancementSession implements EnhancementSession { + /* package */ final AtomicBoolean isReleased = new AtomicBoolean(false); + + @Override + public Surface getInputSurface() { + throw new UnsupportedOperationException(); + } + + @Override + public void setOutputSurface( + Surface surface, EnhancementOptions options, EnhancementCallback callback) {} + + @Override + public void process(Bitmap bitmap, EnhancementOptions options, EnhancementCallback callback) {} + + @Override + public void release() { + isReleased.set(true); + } + + @Override + public void cancel() {} + } + + /** Fake {@link EnhancementSessionManager.Client} for tests. */ + /* package */ static final class FakeClient implements EnhancementSessionManager.Client { + + /* package */ final AtomicBoolean installModuleCalled; + /* package */ final AtomicBoolean createSessionCalled; + + /* package */ Task deviceSupportedTask; + /* package */ Task moduleInstalledTask; + /* package */ Task installModuleTask; + @Nullable /* package */ Task createSessionTask; + /* package */ boolean autoTriggerSessionCallback; + @Nullable /* package */ EnhancementSessionCallback savedSessionCallback; + @Nullable /* package */ EnhancementOptions lastOptions; + + @Nullable private final EnhancementSession sessionToReturn; + + /* package */ FakeClient(@Nullable EnhancementSession sessionToReturn) { + this.installModuleCalled = new AtomicBoolean(false); + this.createSessionCalled = new AtomicBoolean(false); + this.deviceSupportedTask = Tasks.forResult(true); + this.moduleInstalledTask = Tasks.forResult(true); + this.installModuleTask = Tasks.forResult(true); + this.sessionToReturn = sessionToReturn; + this.autoTriggerSessionCallback = sessionToReturn != null; + } + + @Override + public Task isDeviceSupported() { + return deviceSupportedTask; + } + + @Override + public Task isModuleInstalled() { + return moduleInstalledTask; + } + + @Override + public Task installModule(@Nullable EnhancementClient.InstallStatusCallback callback) { + installModuleCalled.set(true); + if (callback != null) { + callback.onInstalled(); + } + return installModuleTask; + } + + @Override + public Task createSession( + EnhancementOptions options, EnhancementSessionCallback callback) { + createSessionCalled.set(true); + lastOptions = options; + savedSessionCallback = callback; + if (autoTriggerSessionCallback && sessionToReturn != null) { + callback.onSessionCreated(sessionToReturn); + } + if (createSessionTask != null) { + return createSessionTask; + } + return sessionToReturn != null ? Tasks.forResult(sessionToReturn) : Tasks.forCanceled(); + } + } +} diff --git a/libraries/effect_play_services/src/test/java/androidx/media3/effect/playservices/MonotonicTimestampGeneratorTest.java b/libraries/effect_play_services/src/test/java/androidx/media3/effect/playservices/MonotonicTimestampGeneratorTest.java new file mode 100644 index 00000000000..95663fd8db8 --- /dev/null +++ b/libraries/effect_play_services/src/test/java/androidx/media3/effect/playservices/MonotonicTimestampGeneratorTest.java @@ -0,0 +1,36 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.effect.playservices; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.test.ext.junit.runners.AndroidJUnit4; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link MonotonicTimestampGenerator}. */ +@RunWith(AndroidJUnit4.class) +public final class MonotonicTimestampGeneratorTest { + + @Test + public void getNextTimestampNs_generatesStrictlyMonotonicallyIncreasingSyntheticTimestamps() { + MonotonicTimestampGenerator generator = new MonotonicTimestampGenerator(); + + assertThat(generator.getNextTimestampNs()).isEqualTo(0L); + assertThat(generator.getNextTimestampNs()).isEqualTo(1_000L); + assertThat(generator.getNextTimestampNs()).isEqualTo(2_000L); + } +} diff --git a/libraries/effect_play_services/src/test/resources/robolectric.properties b/libraries/effect_play_services/src/test/resources/robolectric.properties new file mode 120000 index 00000000000..44fd5f3227b --- /dev/null +++ b/libraries/effect_play_services/src/test/resources/robolectric.properties @@ -0,0 +1 @@ +../../../../common/src/test/resources/robolectric.properties \ No newline at end of file diff --git a/libraries/exoplayer/src/androidTest/java/androidx/media3/exoplayer/AudioPositionAdvancingTest.java b/libraries/exoplayer/src/androidTest/java/androidx/media3/exoplayer/AudioPositionAdvancingTest.java index eb0a3b5f205..21884400770 100644 --- a/libraries/exoplayer/src/androidTest/java/androidx/media3/exoplayer/AudioPositionAdvancingTest.java +++ b/libraries/exoplayer/src/androidTest/java/androidx/media3/exoplayer/AudioPositionAdvancingTest.java @@ -35,7 +35,6 @@ import java.util.List; import java.util.concurrent.atomic.AtomicLong; import java.util.concurrent.atomic.AtomicReference; -import org.junit.Ignore; import org.junit.Test; import org.junit.runner.RunWith; @@ -185,13 +184,12 @@ public void onPlayerError(PlaybackException error) { getInstrumentation().runOnMainSync(() -> player.get().release()); } - @Ignore("Flaky: b/493133077") @Test public void playFromStart_reportedStartTimeEventuallyMatchesPlaybackPosition() throws Exception { - // Try up to two times as the audio track behavior is not fully reliable and can't be - // controlled, which results in a 1-2% flakiness rate. + // Try up to 5 times (retrying 4 times) as looper message dispatch delay between the internal + // playback thread and the main looper on slow emulators can occasionally exceed 50 ms. repeatFlakyTest( - /* maxRepetitions= */ 2, + /* maxRepetitions= */ 5, () -> { MediaItem mediaItem = MediaItem.fromUri("asset:///media/mp3/bear-id3.mp3"); AtomicLong reportedStartSystemTimeMs = new AtomicLong(); @@ -247,8 +245,13 @@ public void onPlayerError(PlaybackException error) { .that(playbackException.get()) .isNull(); assertThat(calculationMessageHandled.block(TIMEOUT_MS)).isTrue(); + // The message callback runs on the main looper, where System.currentTimeMillis() is + // evaluated after looper scheduling delay, while player.getCurrentPosition() returns the + // position sampled when the message was scheduled on the internal playback thread. This + // looper dispatch delay can easily exceed tens of milliseconds on slow emulators, so we + // use 50 ms. assertThat(reportedStartSystemTimeMs.get()) - .isWithin(10) + .isWithin(50) .of(calculatedStartSystemTimeMs.get()); getInstrumentation().runOnMainSync(() -> playerRef.get().release()); diff --git a/libraries/exoplayer/src/androidTest/java/androidx/media3/exoplayer/audio/DefaultAudioSinkTest.java b/libraries/exoplayer/src/androidTest/java/androidx/media3/exoplayer/audio/DefaultAudioSinkTest.java index c392e95eb62..7494a941ff8 100644 --- a/libraries/exoplayer/src/androidTest/java/androidx/media3/exoplayer/audio/DefaultAudioSinkTest.java +++ b/libraries/exoplayer/src/androidTest/java/androidx/media3/exoplayer/audio/DefaultAudioSinkTest.java @@ -17,20 +17,23 @@ import static androidx.media3.common.util.Util.sampleCountToDurationUs; import static androidx.media3.common.util.Util.usToMs; -import static androidx.test.platform.app.InstrumentationRegistry.getInstrumentation; import static com.google.common.collect.Iterables.getLast; import static com.google.common.truth.Truth.assertThat; import android.content.Context; +import android.os.Handler; +import android.os.HandlerThread; import androidx.media3.common.C; import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; +import androidx.media3.common.util.ThrowingRunnable; +import androidx.media3.common.util.Util; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; -import androidx.test.filters.SdkSuppress; import java.nio.ByteBuffer; import java.nio.ByteOrder; import java.util.ArrayList; +import java.util.concurrent.CountDownLatch; import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicInteger; import java.util.concurrent.atomic.AtomicReference; @@ -42,13 +45,16 @@ public class DefaultAudioSinkTest { @Test - @SdkSuppress(minSdkVersion = 24) // TODO: b/399130330 - Debug why this fails on API 23. - public void audioTrackExceedsSharedMemory_retriesUntilOngoingReleasesAreDone() throws Exception { + public void + audioTrackExceedsSharedMemory_playbackThreadStillAlive_retriesUntilOngoingReleasesAreDone() + throws Exception { Context context = ApplicationProvider.getApplicationContext(); + Handler mainHandler = Util.createHandlerForCurrentOrMainLooper(); // Create audio sinks in parallel until we exceed the device's shared audio memory. ArrayList audioSinks = new ArrayList<>(); while (true) { - runOnMainSync( + runOnHandlerSync( + mainHandler, () -> { AudioOutputProvider defaultProvider = new AudioTrackAudioOutputProvider.Builder(context).build(); @@ -71,7 +77,7 @@ public OutputConfig getOutputConfig(FormatConfig formatConfig) audioSinks.add(audioSink); }); try { - configureAudioSinkAndFeedData(getLast(audioSinks)); + configureAudioSinkAndFeedDataOnHandler(mainHandler, getLast(audioSinks)); } catch (Exception e) { // Expected to happen once we reached the shared audio memory limit of the device. break; @@ -79,19 +85,21 @@ public OutputConfig getOutputConfig(FormatConfig formatConfig) } // Trigger release of one sink and immediately try the failed sink again. This should // now succeed even if the sink is released asynchronously. - runOnMainSync( + runOnHandlerSync( + mainHandler, () -> { audioSinks.get(0).flush(); audioSinks.get(0).release(); }); try { - configureAudioSinkAndFeedData(getLast(audioSinks)); + configureAudioSinkAndFeedDataOnHandler(mainHandler, getLast(audioSinks)); } catch (Exception e) { throw new IllegalStateException(e); } // Clean-up - runOnMainSync( + runOnHandlerSync( + mainHandler, () -> { for (int i = 1; i < audioSinks.size(); i++) { audioSinks.get(i).flush(); @@ -101,7 +109,113 @@ public OutputConfig getOutputConfig(FormatConfig formatConfig) } @Test - @SdkSuppress(minSdkVersion = 24) // The test depends on AudioTrack#getUnderrunCount() (API 24+). + public void + audioTrackExceedsSharedMemory_playbackThreadNotAlive_retriesUntilOngoingReleasesAreDone() + throws Exception { + Context context = ApplicationProvider.getApplicationContext(); + Handler mainHandler = Util.createHandlerForCurrentOrMainLooper(); + // Create audio sinks in parallel until we exceed the device's shared audio memory. + ArrayList mainThreadSinks = new ArrayList<>(); + while (true) { + AtomicReference sinkRef = new AtomicReference<>(); + runOnHandlerSync( + mainHandler, + () -> { + AudioOutputProvider defaultProvider = + new AudioTrackAudioOutputProvider.Builder(context).build(); + DefaultAudioSink audioSink = + new DefaultAudioSink.Builder(context) + .setAudioOutputProvider( + new ForwardingAudioOutputProvider(defaultProvider) { + @Override + public OutputConfig getOutputConfig(FormatConfig formatConfig) + throws ConfigurationException { + return super.getOutputConfig(formatConfig) + .buildUpon() + .setBufferSize(2_000_000) + .build(); + } + }) + .build(); + sinkRef.set(audioSink); + mainThreadSinks.add(audioSink); + }); + try { + // Configure the audio sinks on the main thread. + configureAudioSinkAndFeedDataOnHandler(mainHandler, sinkRef.get()); + } catch (Exception e) { + // Expected to happen once we reached the shared audio memory limit. + break; + } + } + + // Free one audio sink on the main thread so we have room to create one sink on the background + // playback thread. + runOnHandlerSync( + mainHandler, + () -> { + mainThreadSinks.get(0).flush(); + mainThreadSinks.get(0).release(); + }); + mainThreadSinks.remove(0); + + HandlerThread playbackThread = new HandlerThread("PlaybackThread"); + playbackThread.start(); + Handler playbackHandler = new Handler(playbackThread.getLooper()); + // Create and configure one audio sink on the background playback thread. + AtomicReference bgSinkRef = new AtomicReference<>(); + runOnHandlerSync( + playbackHandler, + () -> { + AudioOutputProvider defaultProvider = + new AudioTrackAudioOutputProvider.Builder(context).build(); + DefaultAudioSink audioSink = + new DefaultAudioSink.Builder(context) + .setAudioOutputProvider( + new ForwardingAudioOutputProvider(defaultProvider) { + @Override + public OutputConfig getOutputConfig(FormatConfig formatConfig) + throws ConfigurationException { + return super.getOutputConfig(formatConfig) + .buildUpon() + .setBufferSize(2_000_000) + .build(); + } + }) + .build(); + bgSinkRef.set(audioSink); + }); + configureAudioSinkAndFeedDataOnHandler(playbackHandler, bgSinkRef.get()); + + // Trigger release of the background sink and quit the thread. This simulates the player being + // destroyed. + runOnHandlerSync( + playbackHandler, + () -> { + bgSinkRef.get().flush(); + bgSinkRef.get().release(); + }); + playbackThread.quit(); + + // Immediately configure an audio sink again on the main thread. + try { + configureAudioSinkAndFeedDataOnHandler(mainHandler, getLast(mainThreadSinks)); + } catch (Exception e) { + throw new IllegalStateException(e); + } + + // Clean-up all main thread sinks safely on the main thread. + runOnHandlerSync( + mainHandler, + () -> { + for (DefaultAudioSink sink : mainThreadSinks) { + sink.flush(); + sink.release(); + } + }); + } + + @Test public void audioTrackUnderruns_callsOnUnderrun() throws Exception { Context context = ApplicationProvider.getApplicationContext(); AtomicInteger underrunCount = new AtomicInteger(); @@ -152,7 +266,8 @@ public void onSkipSilenceEnabledChanged(boolean skipSilenceEnabled) {} sampleCountToDurationUs(/* sampleCount= */ 25, /* sampleRate= */ 44_100); ByteBuffer smallBuffer = ByteBuffer.allocateDirect(50).order(ByteOrder.nativeOrder()); - runOnMainSync( + runOnHandlerSync( + Util.createHandlerForCurrentOrMainLooper(), () -> { try { sink.configure(new AudioSink.AudioSinkConfig.Builder(format).build()); @@ -182,9 +297,11 @@ public void onSkipSilenceEnabledChanged(boolean skipSilenceEnabled) {} assertThat(underrunCount.get()).isGreaterThan(0); } - private void configureAudioSinkAndFeedData(DefaultAudioSink audioSink) throws Exception { + private void configureAudioSinkAndFeedDataOnHandler(Handler handler, DefaultAudioSink audioSink) + throws Exception { ByteBuffer buffer = ByteBuffer.allocateDirect(8000).order(ByteOrder.nativeOrder()); - runOnMainSync( + runOnHandlerSync( + handler, () -> { Format format = new Format.Builder() @@ -198,7 +315,8 @@ private void configureAudioSinkAndFeedData(DefaultAudioSink audioSink) throws Ex }); AtomicBoolean handledBuffer = new AtomicBoolean(); while (!handledBuffer.get()) { - runOnMainSync( + runOnHandlerSync( + handler, () -> handledBuffer.set( audioSink.handleBuffer( @@ -206,23 +324,23 @@ private void configureAudioSinkAndFeedData(DefaultAudioSink audioSink) throws Ex } } - private static void runOnMainSync(ThrowingRunnable runnable) throws Exception { - AtomicReference exceptionOnMain = new AtomicReference<>(); - getInstrumentation() - .runOnMainSync( - () -> { - try { - runnable.run(); - } catch (Exception e) { - exceptionOnMain.set(e); - } - }); - if (exceptionOnMain.get() != null) { - throw exceptionOnMain.get(); + private static void runOnHandlerSync(Handler handler, ThrowingRunnable runnable) + throws Exception { + AtomicReference exceptionOnHandler = new AtomicReference<>(); + CountDownLatch latch = new CountDownLatch(1); + handler.post( + () -> { + try { + runnable.run(); + } catch (Exception e) { + exceptionOnHandler.set(e); + } finally { + latch.countDown(); + } + }); + latch.await(); + if (exceptionOnHandler.get() != null) { + throw exceptionOnHandler.get(); } } - - private interface ThrowingRunnable { - void run() throws Exception; - } } diff --git a/libraries/exoplayer/src/androidTest/java/androidx/media3/exoplayer/drm/DrmPlaybackTest.kt b/libraries/exoplayer/src/androidTest/java/androidx/media3/exoplayer/drm/DrmPlaybackTest.kt index 076065eb744..24df8b32fbd 100644 --- a/libraries/exoplayer/src/androidTest/java/androidx/media3/exoplayer/drm/DrmPlaybackTest.kt +++ b/libraries/exoplayer/src/androidTest/java/androidx/media3/exoplayer/drm/DrmPlaybackTest.kt @@ -25,7 +25,6 @@ import androidx.media3.common.Player import androidx.media3.exoplayer.DefaultRenderersFactory import androidx.media3.exoplayer.ExoPlayer import androidx.media3.exoplayer.mediacodec.MediaCodecSelector -import androidx.media3.exoplayer.source.DefaultMediaSourceFactory import androidx.media3.test.utils.awaitPlaybackState import androidx.test.ext.junit.runners.AndroidJUnit4 import androidx.test.platform.app.InstrumentationRegistry @@ -34,6 +33,7 @@ import kotlinx.coroutines.Dispatchers import kotlinx.coroutines.runBlocking import okhttp3.mockwebserver.MockResponse import okhttp3.mockwebserver.MockWebServer +import org.junit.After import org.junit.Assume.assumeFalse import org.junit.Before import org.junit.Test @@ -55,6 +55,11 @@ class DrmPlaybackTest { mockWebServer.enqueue(MockResponse().setResponseCode(200).setBody(CLEARKEY_RESPONSE)) } + @After + fun tearDown() { + mockWebServer.shutdown() + } + @Test fun clearkeyPlayback() = runBlocking(Dispatchers.Main) { @@ -86,8 +91,6 @@ class DrmPlaybackTest { context, DefaultRenderersFactory(context) .experimentalSetLateThresholdToDropDecoderInputUs(-100000000L), - DefaultMediaSourceFactory(context) - .experimentalSetCodecsToParseWithinGopSampleDependencies(C.VIDEO_CODEC_FLAG_H264), ) .build() @@ -127,12 +130,7 @@ class DrmPlaybackTest { .build() ) .build() - val player = - ExoPlayer.Builder( - context, - DefaultRenderersFactory(context).experimentalSetParseAv1SampleDependencies(true), - ) - .build() + val player = ExoPlayer.Builder(context).build() player.setMediaItem(mediaItem) player.prepare() player.play() diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/DefaultRenderersFactory.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/DefaultRenderersFactory.java index 64bc61cd885..febb5280dbe 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/DefaultRenderersFactory.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/DefaultRenderersFactory.java @@ -15,7 +15,6 @@ */ package androidx.media3.exoplayer; -import static android.os.Build.VERSION.SDK_INT; import static androidx.media3.exoplayer.video.MediaCodecVideoRenderer.DEFAULT_EARLY_SCHEDULING_THRESHOLD_US; import static androidx.media3.exoplayer.video.MediaCodecVideoRenderer.DEFAULT_LATE_THRESHOLD_TO_DROP_DECODER_INPUT_US; import static com.google.common.base.Preconditions.checkNotNull; @@ -28,9 +27,7 @@ import android.os.Looper; import androidx.annotation.IntDef; import androidx.annotation.Nullable; -import androidx.annotation.RequiresApi; import androidx.media3.common.C; -import androidx.media3.common.MimeTypes; import androidx.media3.common.util.ExperimentalApi; import androidx.media3.common.util.Log; import androidx.media3.common.util.UnstableApi; @@ -124,11 +121,8 @@ public class DefaultRenderersFactory implements RenderersFactory { private boolean enableFloatOutput; private boolean enableAudioOutputPlaybackParameters; private boolean enableMediaCodecVideoRendererPrewarming; - private boolean parseAv1SampleDependencies; private long lateThresholdToDropDecoderInputUs; private long videoRendererEarlySchedulingThresholdUs; - private boolean enableMediaCodecBufferDecodeOnlyFlag; - private boolean enableMediaCodecVideoRendererDurationToProgressUs; /** * @param context A {@link Context}. @@ -139,7 +133,6 @@ public DefaultRenderersFactory(Context context) { extensionRendererMode = EXTENSION_RENDERER_MODE_OFF; allowedVideoJoiningTimeMs = DEFAULT_ALLOWED_VIDEO_JOINING_TIME_MS; mediaCodecSelector = MediaCodecSelector.DEFAULT; - parseAv1SampleDependencies = true; lateThresholdToDropDecoderInputUs = DEFAULT_LATE_THRESHOLD_TO_DROP_DECODER_INPUT_US; videoRendererEarlySchedulingThresholdUs = DEFAULT_EARLY_SCHEDULING_THRESHOLD_US; } @@ -312,74 +305,6 @@ public final DefaultRenderersFactory experimentalSetEnableMediaCodecVideoRendere return this; } - /** - * Sets whether {@link MimeTypes#VIDEO_AV1} bitstream parsing for sample dependency information is - * enabled. Knowing which input frames are not depended on can speed up seeking and reduce dropped - * frames. - * - *

Defaults to {@code true}. - * - *

This method is experimental and will be renamed or removed in a future release. - * - * @param parseAv1SampleDependencies Whether bitstream parsing is enabled. - */ - @CanIgnoreReturnValue - @ExperimentalApi // TODO: b/470365670 - Remove method once config is enabled by default. - public final DefaultRenderersFactory experimentalSetParseAv1SampleDependencies( - boolean parseAv1SampleDependencies) { - this.parseAv1SampleDependencies = parseAv1SampleDependencies; - return this; - } - - /** - * Sets whether the {@link MediaCodec#BUFFER_FLAG_DECODE_ONLY} flag will be included when queuing - * decode-only input buffers to the decoder. - * - *

If {@code false}, then only if the decoder is set up in tunneling mode will the decode-only - * input buffers be queued with the {@link MediaCodec#BUFFER_FLAG_DECODE_ONLY} flag. The default - * value is {@code false}. - * - *

Requires API 34. - * - *

This method is experimental and will be renamed or removed in a future release. - */ - @RequiresApi(34) - @CanIgnoreReturnValue - @ExperimentalApi // TODO: b/470367414 - Run experiments and enable by default. - public DefaultRenderersFactory experimentalSetEnableMediaCodecBufferDecodeOnlyFlag( - boolean enableMediaCodecBufferDecodeOnlyFlag) { - this.enableMediaCodecBufferDecodeOnlyFlag = enableMediaCodecBufferDecodeOnlyFlag; - return this; - } - - /** - * Sets whether {@link MediaCodecVideoRenderer} supports {@link Renderer#getDurationToProgressUs - * getDurationToProgressUs}. - * - *

When ExoPlayer's {@link ExoPlayer.Builder#experimentalSetDynamicSchedulingEnabled dynamic - * scheduling} is enabled, ExoPlayer uses {@link Renderer#getDurationToProgressUs} to better align - * when it wakes the CPU with when player progress can be made. - * - *

If {@code true}, then {@link MediaCodecVideoRenderer} will support {@link - * Renderer#getDurationToProgressUs getDurationToProgressUs} and only if its {@link MediaCodec} - * decoder is set up in asynchronous mode with a registered {@link MediaCodec.Callback} listener. - * With these conditions met {@link ExoPlayer} will adjust its task scheduling with when {@link - * MediaCodecVideoRenderer} can schedule its next output. This will increase CPU Idle time thereby - * reducing power consumption. The default value is {@code false}. - * - *

This method is experimental and will be renamed or removed in a future release. - * - * @see ExoPlayer.Builder#experimentalSetDynamicSchedulingEnabled - */ - @CanIgnoreReturnValue - @ExperimentalApi // TODO: b/369523131 - Remove once experiment is complete. - public DefaultRenderersFactory setEnableMediaCodecVideoRendererDurationToProgressUs( - boolean enableMediaCodecVideoRendererDurationToProgressUs) { - this.enableMediaCodecVideoRendererDurationToProgressUs = - enableMediaCodecVideoRendererDurationToProgressUs; - return this; - } - /** * Sets the maximum duration for which video renderers can attempt to seamlessly join an ongoing * playback. @@ -505,15 +430,8 @@ private MediaCodecVideoRenderer createMediaCodecVideoRenderer( .setEventHandler(eventHandler) .setEventListener(eventListener) .setMaxDroppedFramesToNotify(MAX_DROPPED_VIDEO_FRAME_COUNT_TO_NOTIFY) - .experimentalSetParseAv1SampleDependencies(parseAv1SampleDependencies) .experimentalSetLateThresholdToDropDecoderInputUs(lateThresholdToDropDecoderInputUs) - .setEnableDurationToProgressUs(enableMediaCodecVideoRendererDurationToProgressUs) .setEarlySchedulingThresholdUs(videoRendererEarlySchedulingThresholdUs); - if (SDK_INT >= 34) { - builder = - builder.experimentalSetEnableMediaCodecBufferDecodeOnlyFlag( - enableMediaCodecBufferDecodeOnlyFlag); - } return builder.build(); } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/ExoPlayer.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/ExoPlayer.java index caf35b34c13..9a0c90409e9 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/ExoPlayer.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/ExoPlayer.java @@ -40,6 +40,7 @@ import androidx.media3.common.AuxEffectInfo; import androidx.media3.common.C; import androidx.media3.common.Effect; +import androidx.media3.common.Flags; import androidx.media3.common.Format; import androidx.media3.common.MediaItem; import androidx.media3.common.Player; @@ -248,17 +249,6 @@ final class Builder { */ @UnstableApi public static final int DEFAULT_STUCK_SUPPRESSED_DETECTION_TIMEOUT_MS = 600_000; - /** - * Static override to allow stuck playing detection. If {@code false}, the provided timeouts - * default to {@link Integer#MAX_VALUE} instead of {@link - * #DEFAULT_STUCK_PLAYING_DETECTION_TIMEOUT_MS} and {@link - * #DEFAULT_STUCK_PLAYING_NOT_ENDING_TIMEOUT_MS}. - * - *

This value is experimental and will be removed in a future release. - */ - @ExperimentalApi // TODO: b/443074686 - Remove once global opt-out no longer needed. - public static boolean experimentalEnableStuckPlayingDetection = true; - /* package */ final Context context; /* package */ Clock clock; @@ -301,10 +291,8 @@ final class Builder { /* package */ boolean buildCalled; /* package */ boolean suppressPlaybackOnUnsuitableOutput; /* package */ String playerName; - /* package */ boolean dynamicSchedulingEnabled; /* package */ SuitableOutputChecker suitableOutputChecker; /* package */ boolean enforceAdPlaybackOnTimelineRefresh; - /* package */ boolean perStreamMediaProgressionEnabled; /** * Creates a builder. @@ -361,7 +349,6 @@ final class Builder { *

  • {@code usePlatformDiagnostics}: {@code true} *
  • {@link Clock}: {@link Clock#DEFAULT} *
  • {@code playbackLooper}: {@code null} (create new thread) - *
  • {@code dynamicSchedulingEnabled}: {@code true} * * * @param context A {@link Context}. @@ -523,11 +510,11 @@ private Builder( detachSurfaceTimeoutMs = DEFAULT_DETACH_SURFACE_TIMEOUT_MS; stuckBufferingDetectionTimeoutMs = DEFAULT_STUCK_BUFFERING_DETECTION_TIMEOUT_MS; stuckPlayingDetectionTimeoutMs = - experimentalEnableStuckPlayingDetection + Flags.isEnabled(Flags.FLAG_ENABLE_STUCK_PLAYING_DETECTION) ? DEFAULT_STUCK_PLAYING_DETECTION_TIMEOUT_MS : Integer.MAX_VALUE; stuckPlayingNotEndingTimeoutMs = - experimentalEnableStuckPlayingDetection + Flags.isEnabled(Flags.FLAG_ENABLE_STUCK_PLAYING_DETECTION) ? DEFAULT_STUCK_PLAYING_NOT_ENDING_TIMEOUT_MS : Integer.MAX_VALUE; stuckSuppressedDetectionTimeoutMs = DEFAULT_STUCK_SUPPRESSED_DETECTION_TIMEOUT_MS; @@ -535,7 +522,6 @@ private Builder( playerName = ""; priority = C.PRIORITY_PLAYBACK; suitableOutputChecker = new DefaultSuitableOutputChecker(); - dynamicSchedulingEnabled = true; enforceAdPlaybackOnTimelineRefresh = true; } @@ -557,51 +543,6 @@ public Builder experimentalSetForegroundModeTimeoutMs(long timeoutMs) { return this; } - /** - * Sets whether dynamic scheduling is enabled. - * - *

    If enabled, ExoPlayer's playback loop will run as rarely as possible by scheduling work - * for when {@link Renderer} progress can be made. - * - *

    If a custom {@link AudioSink} is used then it must correctly implement {@link - * AudioSink#getAudioTrackBufferSizeUs()} to enable dynamic scheduling for audio playback. - * - *

    Enabled by default (value is {@code true}). - * - *

    This method is experimental, and will be renamed or removed in a future release. - * - * @param dynamicSchedulingEnabled Whether to enable dynamic scheduling. - */ - @CanIgnoreReturnValue - @ExperimentalApi // TODO: b/500985770 - Remove this method. - public Builder experimentalSetDynamicSchedulingEnabled(boolean dynamicSchedulingEnabled) { - checkState(!buildCalled); - this.dynamicSchedulingEnabled = dynamicSchedulingEnabled; - return this; - } - - /** - * Sets whether ExoPlayer can advance its media processing on a per-stream basis. - * - *

    The default is {@code false}. - * - *

    If {@code false} then ExoPlayer will not start processing the next item in the playlist - * until it has finished with the current item. If {@code true} then ExoPlayer may enable - * renderers on subsequent playlist items as each finishes processing media. Enabling this - * feature can reduce startup latency between media items. - * - *

    This method is experimental, and will be renamed or removed in a future release. - * - * @param perStreamMediaProgressionEnabled Whether to enable media progression per stream. - */ - @CanIgnoreReturnValue - @ExperimentalApi // TODO: b/510217604 - Remove this method. - public Builder enablePerStreamMediaProgression(boolean perStreamMediaProgressionEnabled) { - checkState(!buildCalled); - this.perStreamMediaProgressionEnabled = perStreamMediaProgressionEnabled; - return this; - } - /** * Sets whether the player should suppress playback that is attempted on an unsuitable output. * An example of an unsuitable audio output is the built-in speaker on a Wear OS device (unless @@ -1841,24 +1782,49 @@ void setVideoChangeFrameRateStrategy( @C.VideoChangeFrameRateStrategy int getVideoChangeFrameRateStrategy(); + /** + * Adds a listener to receive video frame metadata events. + * + * @param listener The listener to add. + */ + @UnstableApi + void addVideoFrameMetadataListener(VideoFrameMetadataListener listener); + + /** + * Removes a listener to receive video frame metadata events. + * + * @param listener The listener to remove. + */ + @UnstableApi + void removeVideoFrameMetadataListener(VideoFrameMetadataListener listener); + /** * Sets a listener to receive video frame metadata events. * - *

    This method is intended to be called by the same component that sets the {@link Surface} - * onto which video will be rendered. If using ExoPlayer's standard UI components, this method - * should not be called directly from application code. + *

    Calling this method is equivalent to calling {@link + * #removeVideoFrameMetadataListener(VideoFrameMetadataListener)} for the listener previously set + * with this method (if any), and then calling {@link + * #addVideoFrameMetadataListener(VideoFrameMetadataListener)} with {@code listener} (if + * non-null). * * @param listener The listener. + * @deprecated Use {@link #addVideoFrameMetadataListener(VideoFrameMetadataListener)} and {@link + * #removeVideoFrameMetadataListener(VideoFrameMetadataListener)} instead. */ + @Deprecated @UnstableApi void setVideoFrameMetadataListener(VideoFrameMetadataListener listener); /** - * Clears the listener which receives video frame metadata events if it matches the one passed. - * Else does nothing. + * Clears the listener which receives video frame metadata events. + * + *

    Calling this method is equivalent to calling {@link + * #removeVideoFrameMetadataListener(VideoFrameMetadataListener)}. * * @param listener The listener to clear. + * @deprecated Use {@link #removeVideoFrameMetadataListener(VideoFrameMetadataListener)} instead. */ + @Deprecated @UnstableApi void clearVideoFrameMetadataListener(VideoFrameMetadataListener listener); diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/ExoPlayerImpl.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/ExoPlayerImpl.java index af45122644b..29fede9dfb4 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/ExoPlayerImpl.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/ExoPlayerImpl.java @@ -65,6 +65,7 @@ import androidx.media3.common.C.TrackType; import androidx.media3.common.DeviceInfo; import androidx.media3.common.Effect; +import androidx.media3.common.Flags; import androidx.media3.common.Format; import androidx.media3.common.IllegalSeekPositionException; import androidx.media3.common.MediaItem; @@ -230,6 +231,8 @@ private float unmuteVolume; private boolean skipSilenceEnabled; private CueGroup currentCueGroup; + // TODO(b/552972009): Remove when deprecated set/clearVideoFrameMetadataListener methods are + // removed. @Nullable private VideoFrameMetadataListener videoFrameMetadataListener; @Nullable private CameraMotionListener cameraMotionListener; private boolean throwsWhenUsingWrongThread; @@ -254,6 +257,7 @@ // Playback information when there is a pending seek/set source operation. private int maskingWindowIndex; private long maskingWindowPositionMs; + private long lastReturnedPositionUs; @SuppressLint("HandlerLeak") @SuppressWarnings("deprecation") // Control flow for old volume commands @@ -268,7 +272,8 @@ public ExoPlayerImpl(ExoPlayer.Builder builder, @Nullable Player wrappingPlayer) + MediaLibraryInfo.VERSION_SLASHY + "] [" + Util.DEVICE_DEBUG_INFO - + "]"); + + "] " + + Flags.getLogString()); applicationContext = builder.context.getApplicationContext(); analyticsCollector = builder.analyticsCollectorFunction.apply(builder.clock); priority = builder.priority; @@ -316,6 +321,7 @@ public ExoPlayerImpl(ExoPlayer.Builder builder, @Nullable Player wrappingPlayer) this.applicationLooper = builder.looper; this.clock = builder.clock; this.wrappingPlayer = wrappingPlayer == null ? this : wrappingPlayer; + this.lastReturnedPositionUs = C.TIME_UNSET; listeners = new ListenerSet<>( applicationLooper, @@ -392,8 +398,6 @@ public ExoPlayerImpl(ExoPlayer.Builder builder, @Nullable Player wrappingPlayer) builder.livePlaybackSpeedControl, builder.releaseTimeoutMs, pauseAtEndOfMediaItems, - builder.dynamicSchedulingEnabled, - builder.perStreamMediaProgressionEnabled, applicationLooper, clock, playbackInfoUpdateListener, @@ -1027,7 +1031,9 @@ public void setPlaybackParameters(PlaybackParameters playbackParameters) { if (playbackInfo.playbackParameters.equals(playbackParameters)) { return; } - PlaybackInfo newPlaybackInfo = playbackInfo.copyWithPlaybackParameters(playbackParameters); + PlaybackInfo newPlaybackInfo = + maybeMaskPositionWithEstimate(playbackInfo, clock.elapsedRealtime()); + newPlaybackInfo = newPlaybackInfo.copyWithPlaybackParameters(playbackParameters); pendingOperationAcks++; internalPlayer.setPlaybackParameters(playbackParameters); updatePlaybackInfo( @@ -1165,9 +1171,7 @@ public void release() { listeners.release(); playbackInfoUpdateHandler.removeCallbacksAndMessages(null); bandwidthMeter.removeEventListener(analyticsCollector); - if (playbackInfo.sleepingForOffload) { - playbackInfo = playbackInfo.copyWithEstimatedPosition(); - } + playbackInfo = maybeMaskPositionWithEstimate(playbackInfo, clock.elapsedRealtime()); playbackInfo = maskPlaybackState(playbackInfo, Player.STATE_IDLE); playbackInfo = playbackInfo.copyWithLoadingMediaPeriodId(playbackInfo.periodId); playbackInfo.bufferedPositionUs = playbackInfo.positionUs; @@ -1891,27 +1895,45 @@ public DecoderCounters getAudioDecoderCounters() { } @Override - public void setVideoFrameMetadataListener(VideoFrameMetadataListener listener) { + public void addVideoFrameMetadataListener(VideoFrameMetadataListener listener) { verifyApplicationThread(); - videoFrameMetadataListener = listener; createMessageInternal(frameMetadataListener) - .setType(FrameMetadataListener.MSG_SET_VIDEO_FRAME_METADATA_LISTENER) - .setPayload(listener) + .setType(FrameMetadataListener.MSG_ADD_VIDEO_FRAME_METADATA_LISTENER) + .setPayload(checkNotNull(listener)) .send(); } @Override - public void clearVideoFrameMetadataListener(VideoFrameMetadataListener listener) { + public void removeVideoFrameMetadataListener(VideoFrameMetadataListener listener) { verifyApplicationThread(); - if (videoFrameMetadataListener != listener) { - return; + if (Objects.equals(videoFrameMetadataListener, checkNotNull(listener))) { + videoFrameMetadataListener = null; } createMessageInternal(frameMetadataListener) - .setType(FrameMetadataListener.MSG_SET_VIDEO_FRAME_METADATA_LISTENER) - .setPayload(null) + .setType(FrameMetadataListener.MSG_REMOVE_VIDEO_FRAME_METADATA_LISTENER) + .setPayload(listener) .send(); } + @Override + @Deprecated + public void setVideoFrameMetadataListener(VideoFrameMetadataListener listener) { + verifyApplicationThread(); + if (videoFrameMetadataListener != null) { + removeVideoFrameMetadataListener(videoFrameMetadataListener); + } + videoFrameMetadataListener = listener; + if (listener != null) { + addVideoFrameMetadataListener(listener); + } + } + + @Override + @Deprecated + public void clearVideoFrameMetadataListener(VideoFrameMetadataListener listener) { + removeVideoFrameMetadataListener(listener); + } + @Override public void setCameraMotionListener(CameraMotionListener listener) { verifyApplicationThread(); @@ -2161,6 +2183,7 @@ public void removeVideoCodecParametersChangeListener(CodecParametersChangeListen private void stopInternal(@Nullable ExoPlaybackException error) { PlaybackInfo playbackInfo = this.playbackInfo.copyWithLoadingMediaPeriodId(this.playbackInfo.periodId); + playbackInfo = maybeMaskPositionWithEstimate(playbackInfo, clock.elapsedRealtime()); playbackInfo.bufferedPositionUs = playbackInfo.positionUs; playbackInfo.totalBufferedDurationUs = 0; playbackInfo = maskPlaybackState(playbackInfo, Player.STATE_IDLE); @@ -2206,10 +2229,18 @@ private long getCurrentPositionUsInternal(PlaybackInfo playbackInfo) { } long positionUs = - playbackInfo.sleepingForOffload - ? playbackInfo.getEstimatedPositionUs() + playbackInfo.useEstimatedPosition + ? playbackInfo.getEstimatedPositionUs(clock.elapsedRealtime()) : playbackInfo.positionUs; + if (playbackInfo == this.playbackInfo) { + if (lastReturnedPositionUs != C.TIME_UNSET && positionUs < lastReturnedPositionUs) { + positionUs = lastReturnedPositionUs; + } else { + lastReturnedPositionUs = positionUs; + } + } + if (playbackInfo.periodId.isAd()) { return positionUs; } @@ -2297,6 +2328,11 @@ private void updatePlaybackInfo( PlaybackInfo previousPlaybackInfo = this.playbackInfo; PlaybackInfo newPlaybackInfo = playbackInfo; this.playbackInfo = playbackInfo; + + if (positionDiscontinuity || hasPeriodChanged(newPlaybackInfo, previousPlaybackInfo)) { + lastReturnedPositionUs = C.TIME_UNSET; + } + // TODO (b/494325148): Remove assertion. if (!playbackInfo.timeline.isEmpty()) { checkState( @@ -2456,6 +2492,25 @@ private void updatePlaybackInfo( } } + private boolean hasPeriodChanged( + PlaybackInfo newPlaybackInfo, PlaybackInfo previousPlaybackInfo) { + if (!previousPlaybackInfo.periodId.equals(newPlaybackInfo.periodId)) { + return true; + } + // Detect if we are transitioning from a placeholder period to a real period. + if (!previousPlaybackInfo.timeline.isEmpty() && !newPlaybackInfo.timeline.isEmpty()) { + boolean wasPlaceholder = + previousPlaybackInfo.timeline.getPeriodByUid( + previousPlaybackInfo.periodId.periodUid, period) + .isPlaceholder; + boolean isPlaceholder = + newPlaybackInfo.timeline.getPeriodByUid(newPlaybackInfo.periodId.periodUid, period) + .isPlaceholder; + return wasPlaceholder && !isPlaceholder; + } + return false; + } + private PositionInfo getPreviousPositionInfo( @DiscontinuityReason int positionDiscontinuityReason, PlaybackInfo oldPlaybackInfo, @@ -2783,6 +2838,7 @@ private PlaybackInfo maskTimelineAndPosition( /* requestedContentPositionUs= */ positionUs, /* discontinuityStartPositionUs= */ positionUs, /* totalBufferedDurationUs= */ 0, + clock.elapsedRealtime(), TrackGroupArray.EMPTY, emptyTrackSelectorResult, /* staticMetadata= */ ImmutableList.of()); @@ -2820,6 +2876,7 @@ private PlaybackInfo maskTimelineAndPosition( /* requestedContentPositionUs= */ newContentPositionUs, /* discontinuityStartPositionUs= */ newContentPositionUs, /* totalBufferedDurationUs= */ 0, + clock.elapsedRealtime(), playingPeriodChanged ? TrackGroupArray.EMPTY : playbackInfo.trackGroups, playingPeriodChanged ? emptyTrackSelectorResult : playbackInfo.trackSelectorResult, playingPeriodChanged ? ImmutableList.of() : playbackInfo.staticMetadata); @@ -2846,6 +2903,7 @@ private PlaybackInfo maskTimelineAndPosition( /* requestedContentPositionUs= */ playbackInfo.positionUs, playbackInfo.discontinuityStartPositionUs, /* totalBufferedDurationUs= */ maskedBufferedPositionUs - playbackInfo.positionUs, + clock.elapsedRealtime(), playbackInfo.trackGroups, playbackInfo.trackSelectorResult, playbackInfo.staticMetadata); @@ -2870,6 +2928,7 @@ private PlaybackInfo maskTimelineAndPosition( /* requestedContentPositionUs= */ newContentPositionUs, /* discontinuityStartPositionUs= */ newContentPositionUs, maskedTotalBufferedDurationUs, + clock.elapsedRealtime(), playbackInfo.trackGroups, playbackInfo.trackSelectorResult, playbackInfo.staticMetadata); @@ -2878,6 +2937,14 @@ private PlaybackInfo maskTimelineAndPosition( return playbackInfo; } + private static PlaybackInfo maybeMaskPositionWithEstimate( + PlaybackInfo playbackInfo, long currentElapsedTimeMs) { + if (playbackInfo.useEstimatedPosition) { + playbackInfo = playbackInfo.copyWithEstimatedPosition(currentElapsedTimeMs); + } + return playbackInfo; + } + private static PlaybackInfo maskPlaybackState(PlaybackInfo playbackInfo, int playbackState) { playbackInfo = playbackInfo.copyWithPlaybackState(playbackState); if (playbackState == STATE_IDLE || playbackState == STATE_ENDED) { @@ -3077,11 +3144,8 @@ private void updatePlayWhenReady( return; } pendingOperationAcks++; - // Position estimation and copy must occur before changing/masking playback state. PlaybackInfo newPlaybackInfo = - this.playbackInfo.sleepingForOffload - ? this.playbackInfo.copyWithEstimatedPosition() - : this.playbackInfo; + maybeMaskPositionWithEstimate(this.playbackInfo, clock.elapsedRealtime()); newPlaybackInfo = newPlaybackInfo.copyWithPlayWhenReady( playWhenReady, playWhenReadyChangeReason, playbackSuppressionReason); @@ -3660,24 +3724,32 @@ public void onStuckPlayerDetected(StuckPlayerException exception) { private static final class FrameMetadataListener implements VideoFrameMetadataListener, CameraMotionListener, PlayerMessage.Target { - public static final @MessageType int MSG_SET_VIDEO_FRAME_METADATA_LISTENER = - Renderer.MSG_SET_VIDEO_FRAME_METADATA_LISTENER; - public static final @MessageType int MSG_SET_CAMERA_MOTION_LISTENER = Renderer.MSG_SET_CAMERA_MOTION_LISTENER; public static final @MessageType int MSG_SET_SPHERICAL_SURFACE_VIEW = Renderer.MSG_CUSTOM_BASE; + private static final @MessageType int MSG_ADD_VIDEO_FRAME_METADATA_LISTENER = + Renderer.MSG_CUSTOM_BASE + 1; + private static final @MessageType int MSG_REMOVE_VIDEO_FRAME_METADATA_LISTENER = + Renderer.MSG_CUSTOM_BASE + 2; - @Nullable private VideoFrameMetadataListener videoFrameMetadataListener; + private final CopyOnWriteArraySet videoFrameMetadataListeners; @Nullable private CameraMotionListener cameraMotionListener; @Nullable private VideoFrameMetadataListener internalVideoFrameMetadataListener; @Nullable private CameraMotionListener internalCameraMotionListener; + private FrameMetadataListener() { + videoFrameMetadataListeners = new CopyOnWriteArraySet<>(); + } + @Override public void handleMessage(@MessageType int messageType, @Nullable Object message) { switch (messageType) { - case MSG_SET_VIDEO_FRAME_METADATA_LISTENER: - videoFrameMetadataListener = (VideoFrameMetadataListener) message; + case MSG_ADD_VIDEO_FRAME_METADATA_LISTENER: + videoFrameMetadataListeners.add((VideoFrameMetadataListener) checkNotNull(message)); + break; + case MSG_REMOVE_VIDEO_FRAME_METADATA_LISTENER: + videoFrameMetadataListeners.remove(message); break; case MSG_SET_CAMERA_MOTION_LISTENER: cameraMotionListener = (CameraMotionListener) message; @@ -3718,8 +3790,8 @@ public void onVideoFrameAboutToBeRendered( internalVideoFrameMetadataListener.onVideoFrameAboutToBeRendered( presentationTimeUs, releaseTimeNs, format, mediaFormat); } - if (videoFrameMetadataListener != null) { - videoFrameMetadataListener.onVideoFrameAboutToBeRendered( + for (VideoFrameMetadataListener listener : videoFrameMetadataListeners) { + listener.onVideoFrameAboutToBeRendered( presentationTimeUs, releaseTimeNs, format, mediaFormat); } } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/ExoPlayerImplInternal.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/ExoPlayerImplInternal.java index 3f039101ca5..2039a322a83 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/ExoPlayerImplInternal.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/ExoPlayerImplInternal.java @@ -43,6 +43,7 @@ import androidx.media3.common.AdPlaybackState.AdGroup; import androidx.media3.common.AudioAttributes; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.Format; import androidx.media3.common.IllegalSeekPositionException; import androidx.media3.common.MediaItem; @@ -57,6 +58,7 @@ import androidx.media3.common.Player.PlaybackSuppressionReason; import androidx.media3.common.Player.RepeatMode; import androidx.media3.common.Timeline; +import androidx.media3.common.TrackSelectionParameters; import androidx.media3.common.audio.AudioFocusManager; import androidx.media3.common.util.Clock; import androidx.media3.common.util.ConditionVariable; @@ -192,7 +194,7 @@ public interface PlaybackInfoUpdateListener { private static final long BUFFERING_MAXIMUM_INTERVAL_MS = Util.usToMs(Renderer.DEFAULT_DURATION_TO_PROGRESS_US); - private static final long READY_MAXIMUM_INTERVAL_MS = 1000; + /* package */ static final long READY_MAXIMUM_INTERVAL_MS = 1000; /** * Duration for which the player needs to appear stuck before the playback is failed on the @@ -293,8 +295,6 @@ public ExoPlayerImplInternal( LivePlaybackSpeedControl livePlaybackSpeedControl, long releaseTimeoutMs, boolean pauseAtEndOfWindow, - boolean dynamicSchedulingEnabled, - boolean perStreamMediaProgressionEnabled, Looper applicationLooper, Clock clock, PlaybackInfoUpdateListener playbackInfoUpdateListener, @@ -314,8 +314,9 @@ public ExoPlayerImplInternal( this.releaseTimeoutMs = releaseTimeoutMs; this.setForegroundModeTimeoutMs = releaseTimeoutMs; this.pauseAtEndOfWindow = pauseAtEndOfWindow; - this.dynamicSchedulingEnabled = dynamicSchedulingEnabled; - this.perStreamMediaProgressionEnabled = perStreamMediaProgressionEnabled; + this.dynamicSchedulingEnabled = Flags.isEnabled(Flags.FLAG_DYNAMIC_SCHEDULING); + this.perStreamMediaProgressionEnabled = + Flags.isEnabled(Flags.FLAG_PER_STREAM_MEDIA_PROGRESSION); this.clock = clock; this.playerId = playerId; this.preloadConfiguration = preloadConfiguration; @@ -691,8 +692,8 @@ public void onContinueLoadingRequested(MediaPeriod source) { // TrackSelector.InvalidationListener implementation. @Override - public void onTrackSelectionsInvalidated() { - handler.sendEmptyMessage(MSG_TRACK_SELECTION_INVALIDATED); + public void onTrackSelectionsInvalidated(@Nullable TrackSelectionParameters parameters) { + handler.obtainMessage(MSG_TRACK_SELECTION_INVALIDATED, parameters).sendToTarget(); } @Override @@ -807,6 +808,8 @@ public boolean handleMessage(Message msg) { handleContinueLoadingRequested((MediaPeriod) msg.obj); break; case MSG_TRACK_SELECTION_INVALIDATED: + @Nullable TrackSelectionParameters parameters = (TrackSelectionParameters) msg.obj; + trackSelector.onParametersActivated(parameters); reselectTracksInternal(); break; case MSG_PLAYBACK_PARAMETERS_CHANGED_INTERNAL: @@ -916,6 +919,29 @@ && isRendererPrewarmingMediaPeriod(e.rendererIndex, e.mediaPeriodId)) { maybeContinueLoading(); handler.sendEmptyMessage(MSG_DO_SOME_WORK); } + } else if (e.type == ExoPlaybackException.TYPE_RENDERER + && e.mediaPeriodId != null + && isPeriodAfterEarliestReadingPeriod(e.mediaPeriodId)) { + // Disable pre-warming renderers and any renderers reading periods after the earliest + // reading period before removing those periods from the queue. Note that disabling an + // early-reading renderer flushes any remaining buffered tail it had for the current period. + disableAndResetPrewarmingRenderers(); + MediaPeriodHolder earliestReadingPeriod = checkNotNull(queue.getEarliestReadingPeriod()); + for (int i = 0; i < renderers.length; i++) { + if (i == e.rendererIndex || queue.getReadingPeriod(i) != earliestReadingPeriod) { + try { + disableRenderer(i); + } catch (ExoPlaybackException disableException) { + Log.e(TAG, "Failed to disable renderer.", disableException); + } + renderers[i].reset(); + } + } + queue.removeAfter(earliestReadingPeriod); + if (playbackInfo.playbackState != Player.STATE_ENDED) { + maybeContinueLoading(); + handler.sendEmptyMessage(MSG_DO_SOME_WORK); + } } else { if (pendingRecoverableRendererError != null) { pendingRecoverableRendererError.addSuppressed(e); @@ -1395,7 +1421,7 @@ private void updatePlaybackPositions() throws ExoPlaybackException { /* reportDiscontinuity= */ reportSilenceSkip, Player.DISCONTINUITY_REASON_SILENCE_SKIP); } else { - playbackInfo.updatePositionUs(periodPositionUs); + playbackInfo.updatePositionUs(periodPositionUs, clock.elapsedRealtime()); } } @@ -1491,6 +1517,7 @@ private void doSomeWork() throws ExoPlaybackException, IOException { maybeThrowRendererStreamError(/* rendererIndex= */ i); } } + maybeEndPendingScrubbingSeek(); } else { playingPeriodHolder.mediaPeriod.maybeThrowPrepareError(); } @@ -1572,6 +1599,9 @@ && shouldPlayWhenReady()) { requestForRendererSleep = false; // A sleep request is only valid for the current doSomeWork. if (sleepingForOffload || playbackInfo.playbackState == Player.STATE_ENDED) { + if (sleepingForOffload && !playbackInfo.useEstimatedPosition) { + playbackInfo = playbackInfo.copyWithUseEstimatedPosition(true); + } // No need to schedule next work. } else if ((isPlaying || playbackInfo.playbackState == Player.STATE_BUFFERING) || (playbackInfo.playbackState == Player.STATE_READY && enabledRendererCount != 0)) { @@ -1593,6 +1623,24 @@ private void maybeTriggerOnRendererReadyChanged(int rendererIndex, boolean allow } } + private void maybeEndPendingScrubbingSeek() { + if (!seekIsPendingWhileScrubbing || handler.hasMessages(MSG_SEEK_COMPLETED_IN_SCRUBBING_MODE)) { + return; + } + for (RendererHolder renderer : renderers) { + if (renderer.isRendererEnabled() + && (renderer.getTrackType() == C.TRACK_TYPE_VIDEO + || renderer.getTrackType() == C.TRACK_TYPE_IMAGE) + && !renderer.isEnded()) { + return; + } + } + // A pending seek in scrubbing mode normally completes when the next video frame or image is + // rendered. If all relevant renderers have ended, no frame will be rendered, so complete the + // seek immediately to avoid stalling subsequent seeks. + handler.obtainMessage(MSG_SEEK_COMPLETED_IN_SCRUBBING_MODE).sendToTarget(); + } + private long getCurrentLiveOffsetUs() { return getLiveOffsetUs( playbackInfo.timeline, playbackInfo.periodId.periodUid, playbackInfo.positionUs); @@ -1623,6 +1671,11 @@ private void scheduleNextWork(long thisOperationStartTimeMs) { isDynamicSchedulingEnabled() ? getDynamicSchedulingWakeUpIntervalMs() : getStaticSchedulingWakeUpIntervalMs(); + boolean useEstimatedPosition = + shouldPlayWhenReady() && wakeUpTimeIntervalMs > BUFFERING_MAXIMUM_INTERVAL_MS; + if (playbackInfo.useEstimatedPosition != useEstimatedPosition) { + playbackInfo = playbackInfo.copyWithUseEstimatedPosition(useEstimatedPosition); + } handler.sendEmptyMessageAtTime( MSG_DO_SOME_WORK, thisOperationStartTimeMs + wakeUpTimeIntervalMs); } @@ -2059,6 +2112,9 @@ private void resetInternal( boolean releaseMediaSourceList, boolean resetError) { handler.removeMessages(MSG_DO_SOME_WORK); + if (playbackInfo.useEstimatedPosition) { + playbackInfo = playbackInfo.copyWithEstimatedPosition(clock.elapsedRealtime()); + } seekIsPendingWhileScrubbing = false; if (queuedSeekWhileScrubbing != null) { playbackInfoUpdate.incrementPendingOperationAcks(/* operationAcks= */ 1); @@ -2147,7 +2203,8 @@ private void resetInternal( /* totalBufferedDurationUs= */ 0, /* positionUs= */ startPositionUs, /* positionUpdateTimeMs= */ 0, - /* sleepingForOffload= */ false); + /* sleepingForOffload= */ false, + /* useEstimatedPosition= */ false); if (releaseMediaSourceList) { queue.releasePreloadPool(); mediaSourceList.release(); @@ -2374,6 +2431,21 @@ private boolean isRendererPrewarmingMediaPeriod(int rendererIndex, MediaPeriodId return renderers[rendererIndex].isPrewarmingPeriod(queue.getPrewarmingPeriod(rendererIndex)); } + private boolean isPeriodAfterEarliestReadingPeriod(MediaPeriodId periodId) { + @Nullable MediaPeriodHolder earliestReadingPeriod = queue.getEarliestReadingPeriod(); + if (earliestReadingPeriod == null) { + return false; + } + MediaPeriodHolder period = earliestReadingPeriod.getNext(); + while (period != null) { + if (period.info.id.equals(periodId)) { + return true; + } + period = period.getNext(); + } + return false; + } + private void reselectTracksInternalAndSeek() throws ExoPlaybackException { reselectTracksInternal(); seekToCurrentPosition(/* sendDiscontinuity= */ true); @@ -3142,20 +3214,31 @@ private void maybeUpdateReadingRenderers() throws ExoPlaybackException { } private void maybeUpdateReadingRenderersPerStream() throws ExoPlaybackException { + boolean allUpdated = true; for (int i = 0; i < renderers.length; i++) { @Nullable MediaPeriodHolder readingPeriod = queue.getReadingPeriod(/* index= */ i); if (readingPeriod == null - || Objects.equals(queue.getPlayingPeriod(), readingPeriod) + || queue.getPlayingPeriod() == readingPeriod || readingPeriod.isRendererInCorrectState(/* index= */ i)) { // Not reading ahead or all renderers updated. continue; } if (updateRendererForTransition(i)) { - // Only if all streams for reading period has all renderers in correct state then its true - queue.getReadingPeriod(/* index= */ i).setRendererToCorrectState(/* index= */ i); + if (readingPeriod.prepared) { + readingPeriod.setRendererToCorrectState(/* index= */ i); + } + } else { + allUpdated = false; } } + @Nullable MediaPeriodHolder earliestReadingPeriod = queue.getEarliestReadingPeriod(); + if (earliestReadingPeriod != null + && queue.getPlayingPeriod() != earliestReadingPeriod + && !earliestReadingPeriod.prepared + && allUpdated) { + setAllRenderersToCorrectState(earliestReadingPeriod); + } } private boolean updateRenderersForTransition() throws ExoPlaybackException { @@ -3483,6 +3566,9 @@ private void handlePlaybackParameters( if (acknowledgeCommand) { playbackInfoUpdate.incrementPendingOperationAcks(1); } + if (playbackInfo.useEstimatedPosition) { + playbackInfo = playbackInfo.copyWithEstimatedPosition(clock.elapsedRealtime()); + } playbackInfo = playbackInfo.copyWithPlaybackParameters(playbackParameters); } updateTrackSelectionPlaybackSpeed(playbackParameters.speed); @@ -3613,6 +3699,7 @@ private PlaybackInfo handlePositionDiscontinuity( requestedContentPositionUs, discontinuityStartPositionUs, getTotalBufferedDurationUs(), + clock.elapsedRealtime(), trackGroupArray, trackSelectorResult, staticMetadata); @@ -4061,12 +4148,13 @@ private static PositionUpdateForPlaylistChange resolvePositionForPlaylistChange( // avoid any unintentional renderer reset. boolean isInStreamAdChange = isIgnorableServerSideAdInsertionPeriodChange( + timeline, isUsingPlaceholderPeriod, oldPeriodId, oldContentPositionUs, periodIdWithAds, - timeline.getPeriodByUid(newPeriodUid, period), - newContentPositionUs); + newContentPositionUs, + period); MediaPeriodId newPeriodId = onlyNextAdGroupIndexIncreased || isInStreamAdChange ? oldPeriodId : periodIdWithAds; @@ -4140,27 +4228,29 @@ private static PositionUpdateForPlaylistChange resolvePositionForPlaylistChange( } private static boolean isIgnorableServerSideAdInsertionPeriodChange( + Timeline timeline, boolean isUsingPlaceholderPeriod, MediaPeriodId oldPeriodId, long oldContentPositionUs, MediaPeriodId newPeriodId, - Timeline.Period newPeriod, - long newContentPositionUs) { + long newContentPositionUs, + Timeline.Period period) { if (isUsingPlaceholderPeriod || oldContentPositionUs != newContentPositionUs || !oldPeriodId.periodUid.equals(newPeriodId.periodUid)) { // The period position changed. return false; } - if (oldPeriodId.isAd() && newPeriod.isServerSideInsertedAdGroup(oldPeriodId.adGroupIndex)) { + timeline.getPeriodByUid(newPeriodId.periodUid, period); + if (oldPeriodId.isAd() && period.isServerSideInsertedAdGroup(oldPeriodId.adGroupIndex)) { // Whether the old period was a server side ad that doesn't need skipping to the content. - return newPeriod.getAdState(oldPeriodId.adGroupIndex, oldPeriodId.adIndexInAdGroup) + return period.getAdState(oldPeriodId.adGroupIndex, oldPeriodId.adIndexInAdGroup) != AdPlaybackState.AD_STATE_ERROR - && newPeriod.getAdState(oldPeriodId.adGroupIndex, oldPeriodId.adIndexInAdGroup) + && period.getAdState(oldPeriodId.adGroupIndex, oldPeriodId.adIndexInAdGroup) != AdPlaybackState.AD_STATE_SKIPPED; } // If the new period is a server side inserted ad, we can just continue playing. - return newPeriodId.isAd() && newPeriod.isServerSideInsertedAdGroup(newPeriodId.adGroupIndex); + return newPeriodId.isAd() && period.isServerSideInsertedAdGroup(newPeriodId.adGroupIndex); } private static boolean isUsingPlaceholderPeriod( diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/MediaPeriodHolder.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/MediaPeriodHolder.java index 0bd317d5871..9c5f40adc2f 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/MediaPeriodHolder.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/MediaPeriodHolder.java @@ -35,6 +35,7 @@ import androidx.media3.exoplayer.trackselection.TrackSelectorResult; import androidx.media3.exoplayer.upstream.Allocator; import java.io.IOException; +import java.util.Arrays; /** Holds a {@link MediaPeriod} with information required to play it as part of a timeline. */ /* package */ final class MediaPeriodHolder { @@ -211,6 +212,7 @@ public long getNextLoadPositionUs() { public void handlePrepared(float playbackSpeed, Timeline timeline, boolean playWhenReady) throws ExoPlaybackException { prepared = true; + Arrays.fill(renderersInCorrectState, false); trackGroups = mediaPeriod.getTrackGroups(); TrackSelectorResult selectorResult = selectTracks(playbackSpeed, timeline, playWhenReady); long requestedStartPositionUs = info.startPositionUs; diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/MediaPeriodQueue.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/MediaPeriodQueue.java index b694ff810f5..125eb6acb63 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/MediaPeriodQueue.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/MediaPeriodQueue.java @@ -561,8 +561,8 @@ public int removeAfter(MediaPeriodHolder mediaPeriodHolder) { mediaPeriodHolder = checkNotNull(mediaPeriodHolder.getNext()); for (int i = 0; i < readingPeriods.length; i++) { if (mediaPeriodHolder.equals(readingPeriods[i])) { - readingPeriods[i] = playing; - prewarmingPeriods[i] = playing; + readingPeriods[i] = loading; + prewarmingPeriods[i] = loading; removedResult |= UPDATE_PERIOD_QUEUE_ALTERED_READING_PERIOD; removedResult |= UPDATE_PERIOD_QUEUE_ALTERED_PREWARMING_PERIOD; } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/PlaybackInfo.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/PlaybackInfo.java index f27a4eb6337..16a7d4e9f85 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/PlaybackInfo.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/PlaybackInfo.java @@ -15,7 +15,9 @@ */ package androidx.media3.exoplayer; -import android.os.SystemClock; +import static androidx.media3.exoplayer.ExoPlayerImplInternal.READY_MAXIMUM_INTERVAL_MS; +import static java.lang.Math.min; + import androidx.annotation.CheckResult; import androidx.annotation.Nullable; import androidx.media3.common.C; @@ -24,6 +26,7 @@ import androidx.media3.common.Player; import androidx.media3.common.Player.PlaybackSuppressionReason; import androidx.media3.common.Timeline; +import androidx.media3.common.util.Clock; import androidx.media3.common.util.Util; import androidx.media3.exoplayer.source.MediaSource.MediaPeriodId; import androidx.media3.exoplayer.source.TrackGroupArray; @@ -41,6 +44,15 @@ private static final MediaPeriodId PLACEHOLDER_MEDIA_PERIOD_ID = new MediaPeriodId(/* periodUid= */ new Object()); + /** + * Elapsed time limit for position extrapolation when accounting for thread scheduling delays. + * + *

    The extrapolation must be capped to prevent unbounded ghost playback during thread + * deadlocks. The playback thread wakes up at most every READY_MAXIMUM_INTERVAL_MS (1000ms). A + * 1.5*READY_MAXIMUM_INTERVAL_MS cap provides a safe tolerance for normal thread scheduling. + */ + private static final long MAX_ELAPSED_TIME_MS = (long) (READY_MAXIMUM_INTERVAL_MS * 1.5); + /** The current {@link Timeline}. */ public final Timeline timeline; @@ -96,6 +108,11 @@ /** Whether the main player loop is sleeping, while using offload scheduling. */ public final boolean sleepingForOffload; + /** + * Whether the player should use {@link #getEstimatedPositionUs} for current playback position. + */ + public final boolean useEstimatedPosition; + /** * Position up to which media is buffered in {@link #loadingMediaPeriodId) relative to the start * of the associated period in the {@link #timeline}, in microseconds. @@ -115,11 +132,13 @@ public volatile long positionUs; /** - * The value of {@link SystemClock#elapsedRealtime()} when {@link #positionUs} was updated, in + * The value of {@link Clock#elapsedRealtime()} when {@link #positionUs} was updated, in * milliseconds. */ public volatile long positionUpdateTimeMs; + private volatile long positionUpdateSeq = 0; + /** * Creates an empty placeholder playback info which can be used for masking as long as no real * playback info is available. @@ -149,7 +168,8 @@ public static PlaybackInfo createDummy(TrackSelectorResult emptyTrackSelectorRes /* totalBufferedDurationUs= */ 0, /* positionUs= */ 0, /* positionUpdateTimeMs= */ 0, - /* sleepingForOffload= */ false); + /* sleepingForOffload= */ false, + /* useEstimatedPosition= */ false); } /** @@ -174,6 +194,7 @@ public static PlaybackInfo createDummy(TrackSelectorResult emptyTrackSelectorRes * @param positionUs See {@link #positionUs}. * @param positionUpdateTimeMs See {@link #positionUpdateTimeMs}. * @param sleepingForOffload See {@link #sleepingForOffload}. + * @param useEstimatedPosition See {@link #useEstimatedPosition}. */ public PlaybackInfo( Timeline timeline, @@ -195,7 +216,8 @@ public PlaybackInfo( long totalBufferedDurationUs, long positionUs, long positionUpdateTimeMs, - boolean sleepingForOffload) { + boolean sleepingForOffload, + boolean useEstimatedPosition) { this.timeline = timeline; this.periodId = periodId; this.requestedContentPositionUs = requestedContentPositionUs; @@ -216,6 +238,7 @@ public PlaybackInfo( this.positionUs = positionUs; this.positionUpdateTimeMs = positionUpdateTimeMs; this.sleepingForOffload = sleepingForOffload; + this.useEstimatedPosition = useEstimatedPosition; } /** Returns a placeholder period id for an empty timeline. */ @@ -231,6 +254,8 @@ public static MediaPeriodId getDummyPeriodForEmptyTimeline() { * @param requestedContentPositionUs New requested content position. See {@link * #requestedContentPositionUs}. * @param totalBufferedDurationUs New buffered duration. See {@link #totalBufferedDurationUs}. + * @param currentElapsedTimeMs The current elapsed real-time in milliseconds (e.g., from {@link + * Clock#elapsedRealtime()}). * @param trackGroups The track groups for the new position. See {@link #trackGroups}. * @param trackSelectorResult The track selector result for the new position. See {@link * #trackSelectorResult}. @@ -245,6 +270,7 @@ public PlaybackInfo copyWithNewPosition( long requestedContentPositionUs, long discontinuityStartPositionUs, long totalBufferedDurationUs, + long currentElapsedTimeMs, TrackGroupArray trackGroups, TrackSelectorResult trackSelectorResult, List staticMetadata) { @@ -267,8 +293,9 @@ public PlaybackInfo copyWithNewPosition( bufferedPositionUs, totalBufferedDurationUs, positionUs, - /* positionUpdateTimeMs= */ SystemClock.elapsedRealtime(), - sleepingForOffload); + currentElapsedTimeMs, + sleepingForOffload, + useEstimatedPosition); } /** @@ -299,7 +326,8 @@ public PlaybackInfo copyWithTimeline(Timeline timeline) { totalBufferedDurationUs, positionUs, positionUpdateTimeMs, - sleepingForOffload); + sleepingForOffload, + useEstimatedPosition); } /** @@ -330,7 +358,8 @@ public PlaybackInfo copyWithPlaybackState(int playbackState) { totalBufferedDurationUs, positionUs, positionUpdateTimeMs, - sleepingForOffload); + sleepingForOffload, + useEstimatedPosition); } /** @@ -361,7 +390,8 @@ public PlaybackInfo copyWithPlaybackError(@Nullable ExoPlaybackException playbac totalBufferedDurationUs, positionUs, positionUpdateTimeMs, - sleepingForOffload); + sleepingForOffload, + useEstimatedPosition); } /** @@ -392,7 +422,8 @@ public PlaybackInfo copyWithIsLoading(boolean isLoading) { totalBufferedDurationUs, positionUs, positionUpdateTimeMs, - sleepingForOffload); + sleepingForOffload, + useEstimatedPosition); } /** @@ -423,7 +454,8 @@ public PlaybackInfo copyWithLoadingMediaPeriodId(MediaPeriodId loadingMediaPerio totalBufferedDurationUs, positionUs, positionUpdateTimeMs, - sleepingForOffload); + sleepingForOffload, + useEstimatedPosition); } /** @@ -461,7 +493,8 @@ public PlaybackInfo copyWithPlayWhenReady( totalBufferedDurationUs, positionUs, positionUpdateTimeMs, - sleepingForOffload); + sleepingForOffload, + useEstimatedPosition); } /** @@ -492,7 +525,8 @@ public PlaybackInfo copyWithPlaybackParameters(PlaybackParameters playbackParame totalBufferedDurationUs, positionUs, positionUpdateTimeMs, - sleepingForOffload); + sleepingForOffload, + useEstimatedPosition); } /** @@ -523,7 +557,8 @@ public PlaybackInfo copyWithSleepingForOffload(boolean sleepingForOffload) { totalBufferedDurationUs, positionUs, positionUpdateTimeMs, - sleepingForOffload); + sleepingForOffload, + useEstimatedPosition); } /** @@ -532,10 +567,49 @@ public PlaybackInfo copyWithSleepingForOffload(boolean sleepingForOffload) { *

    Position is estimated with {@link #positionUs}, {@link #positionUpdateTimeMs}, and {@link * PlaybackParameters#speed}. * + * @param currentElapsedTimeMs The current elapsed real-time in milliseconds (e.g., from {@link + * Clock#elapsedRealtime()}). * @return Copied playback info with new, estimated playback position. */ @CheckResult - public PlaybackInfo copyWithEstimatedPosition() { + public PlaybackInfo copyWithEstimatedPosition(long currentElapsedTimeMs) { + return new PlaybackInfo( + timeline, + periodId, + requestedContentPositionUs, + discontinuityStartPositionUs, + playbackState, + playbackError, + isLoading, + trackGroups, + trackSelectorResult, + staticMetadata, + loadingMediaPeriodId, + playWhenReady, + playWhenReadyChangeReason, + playbackSuppressionReason, + playbackParameters, + bufferedPositionUs, + totalBufferedDurationUs, + getEstimatedPositionUs(currentElapsedTimeMs), + currentElapsedTimeMs, + sleepingForOffload, + useEstimatedPosition); + } + + /** + * Copies playback info with new value for whether the Player should use {@link + * #getEstimatedPositionUs} to retrieve the current playback position. + * + *

    If the next position update is scheduled infrequently, the position should be extrapolated + * using {@link #positionUs}, {@link #positionUpdateTimeMs}, and {@link PlaybackParameters#speed}. + * This can be retrieved through {@link #getEstimatedPositionUs}. + * + * @return The updated playback info with new value of whether position should be retrieved + * through {@link #getEstimatedPositionUs}. + */ + @CheckResult + public PlaybackInfo copyWithUseEstimatedPosition(boolean useEstimatedPosition) { return new PlaybackInfo( timeline, periodId, @@ -554,22 +628,30 @@ public PlaybackInfo copyWithEstimatedPosition() { playbackParameters, bufferedPositionUs, totalBufferedDurationUs, - getEstimatedPositionUs(), - SystemClock.elapsedRealtime(), - sleepingForOffload); + positionUs, + positionUpdateTimeMs, + sleepingForOffload, + useEstimatedPosition); } /** - * Sets new playing position with update time of {@link SystemClock#elapsedRealtime()}, time - * relative to the start of the associated period in the {@link #timeline} + * Sets new playing position with a provided update time and that is relative to the start of the + * associated period in the {@link #timeline}. + * + *

    Method updates volatile fields and so invocations must always occur on the same thread. * * @param positionUs The new playing position. + * @param currentElapsedTimeMs The current elapsed real-time in milliseconds (e.g., from {@link + * Clock#elapsedRealtime()}). */ - public void updatePositionUs(long positionUs) { - // Write order of positionUs then positionUpdateTimeMs in order to be reverse of - // retrieval in getExtrapolatedPositionUs(). + @SuppressWarnings("NonAtomicVolatileUpdate") // All write access occurs on same thread. + public void updatePositionUs(long positionUs, long currentElapsedTimeMs) { + // This is a "Sequence Lock" (Seqlock) implementation to ensure atomic reads of positionUs and + // positionUpdateTimeMs across threads without heavy synchronization. + positionUpdateSeq++; this.positionUs = positionUs; - this.positionUpdateTimeMs = SystemClock.elapsedRealtime(); + this.positionUpdateTimeMs = currentElapsedTimeMs; + positionUpdateSeq++; } /** @@ -578,23 +660,29 @@ public void updatePositionUs(long positionUs) { * *

    If not playing, then the estimated position is {@link #positionUs}. * + * @param currentElapsedTimeMs The current elapsed real-time in milliseconds (e.g., from {@link + * Clock#elapsedRealtime()}). * @return The estimated position. */ - public long getEstimatedPositionUs() { + public long getEstimatedPositionUs(long currentElapsedTimeMs) { if (!isPlaying()) { return this.positionUs; } // Snapshot of volatile position info + long positionUpdateSeq; long positionUs; long positionUpdateTimeMs; do { - // Read order of positionUpdateTimeMs then positionUs to be reverse of updatePositionUs write. - positionUpdateTimeMs = this.positionUpdateTimeMs; + positionUpdateSeq = this.positionUpdateSeq; positionUs = this.positionUs; - } while (positionUpdateTimeMs != this.positionUpdateTimeMs); + positionUpdateTimeMs = this.positionUpdateTimeMs; + } while (positionUpdateSeq % 2 != 0 || positionUpdateSeq != this.positionUpdateSeq); + + long elapsedTimeMs = currentElapsedTimeMs - positionUpdateTimeMs; + // Cap the extrapolation to prevent unbounded ghost playback during thread deadlocks. + elapsedTimeMs = min(elapsedTimeMs, MAX_ELAPSED_TIME_MS); - long elapsedTimeMs = SystemClock.elapsedRealtime() - positionUpdateTimeMs; long estimatedPositionMs = Util.usToMs(positionUs) + (long) (elapsedTimeMs * playbackParameters.speed); return Util.msToUs(estimatedPositionMs); diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/PlaylistTimeline.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/PlaylistTimeline.java index 63e450672b3..7689643d898 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/PlaylistTimeline.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/PlaylistTimeline.java @@ -152,7 +152,8 @@ public Period getPeriod(int periodIndex, Period period, boolean setIds) { period.durationUs, period.positionInWindowUs, AdPlaybackState.NONE, - /* isPlaceholder= */ true); + /* isPlaceholder= */ true, + period.originalDurationUs); } else { superPeriod.isPlaceholder = true; } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/RendererHolder.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/RendererHolder.java index bf693ca6e52..181a80bf682 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/RendererHolder.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/RendererHolder.java @@ -76,12 +76,15 @@ public boolean hasSecondary() { public void startPrewarming() { checkState(!isPrewarming()); - prewarmingState = - isRendererEnabled(primaryRenderer) - ? RENDERER_PREWARMING_STATE_TRANSITIONING_TO_SECONDARY - : secondaryRenderer != null && isRendererEnabled(secondaryRenderer) - ? RENDERER_PREWARMING_STATE_TRANSITIONING_TO_PRIMARY - : RENDERER_PREWARMING_STATE_PREWARMING_PRIMARY; + if (isRendererEnabled(primaryRenderer)) { + prewarmingState = RENDERER_PREWARMING_STATE_TRANSITIONING_TO_SECONDARY; + } else if (secondaryRenderer != null && isRendererEnabled(secondaryRenderer)) { + prewarmingState = RENDERER_PREWARMING_STATE_TRANSITIONING_TO_PRIMARY; + } else if (prewarmingState == RENDERER_PREWARMING_STATE_NOT_PREWARMING_USING_PRIMARY) { + prewarmingState = RENDERER_PREWARMING_STATE_PREWARMING_PRIMARY; + } else { + prewarmingState = RENDERER_PREWARMING_STATE_PREWARMING_SECONDARY; + } } public boolean isPrewarming() { @@ -94,7 +97,8 @@ private boolean isPrimaryRendererPrewarming() { } private boolean isSecondaryRendererPrewarming() { - return prewarmingState == RENDERER_PREWARMING_STATE_TRANSITIONING_TO_SECONDARY; + return prewarmingState == RENDERER_PREWARMING_STATE_PREWARMING_SECONDARY + || prewarmingState == RENDERER_PREWARMING_STATE_TRANSITIONING_TO_SECONDARY; } public int getEnabledRendererCount() { @@ -165,6 +169,7 @@ public void setCurrentStreamFinal(long streamEndPositionUs) { boolean isPrimaryRenderer = secondaryRenderer == null || prewarmingState == RENDERER_PREWARMING_STATE_TRANSITIONING_TO_SECONDARY + || prewarmingState == RENDERER_PREWARMING_STATE_PREWARMING_SECONDARY || prewarmingState == RENDERER_PREWARMING_STATE_NOT_PREWARMING_USING_PRIMARY; Renderer renderer = isPrimaryRenderer ? primaryRenderer : checkNotNull(secondaryRenderer); setCurrentStreamFinalInternal(renderer, streamEndPositionUs); @@ -190,6 +195,7 @@ public void maybeSetOldStreamToFinal( boolean isPrimaryOldRenderer = secondaryRenderer == null || prewarmingState == RENDERER_PREWARMING_STATE_TRANSITIONING_TO_SECONDARY + || prewarmingState == RENDERER_PREWARMING_STATE_PREWARMING_SECONDARY || (prewarmingState == RENDERER_PREWARMING_STATE_NOT_PREWARMING_USING_PRIMARY && isRendererEnabled(primaryRenderer)); Renderer oldRenderer = isPrimaryOldRenderer ? primaryRenderer : checkNotNull(secondaryRenderer); @@ -226,7 +232,8 @@ public void setAllNonPrewarmingRendererStreamsFinal(long streamEndPositionUs) { } if (secondaryRenderer != null && isRendererEnabled(secondaryRenderer) - && prewarmingState != RENDERER_PREWARMING_STATE_TRANSITIONING_TO_SECONDARY) { + && prewarmingState != RENDERER_PREWARMING_STATE_TRANSITIONING_TO_SECONDARY + && prewarmingState != RENDERER_PREWARMING_STATE_PREWARMING_SECONDARY) { setCurrentStreamFinalInternal(secondaryRenderer, streamEndPositionUs); } } @@ -594,8 +601,9 @@ public void disable(DefaultMediaClock mediaClock) throws ExoPlaybackException { disableRenderer(primaryRenderer, mediaClock); if (secondaryRenderer != null) { boolean shouldTransferResources = - isRendererEnabled(secondaryRenderer) - && prewarmingState != RENDERER_PREWARMING_STATE_TRANSITIONING_TO_SECONDARY; + prewarmingState == RENDERER_PREWARMING_STATE_TRANSITIONING_TO_PRIMARY + || prewarmingState == RENDERER_PREWARMING_STATE_NOT_PREWARMING_USING_SECONDARY + || prewarmingState == RENDERER_PREWARMING_STATE_PREWARMING_SECONDARY; disableRenderer(secondaryRenderer, mediaClock); maybeResetRenderer(/* resetPrimary= */ false); if (shouldTransferResources) { @@ -618,6 +626,8 @@ public void maybeHandlePrewarmingTransition() throws ExoPlaybackException { : RENDERER_PREWARMING_STATE_NOT_PREWARMING_USING_SECONDARY; } else if (prewarmingState == RENDERER_PREWARMING_STATE_PREWARMING_PRIMARY) { prewarmingState = RENDERER_PREWARMING_STATE_NOT_PREWARMING_USING_PRIMARY; + } else if (prewarmingState == RENDERER_PREWARMING_STATE_PREWARMING_SECONDARY) { + prewarmingState = RENDERER_PREWARMING_STATE_NOT_PREWARMING_USING_SECONDARY; } } @@ -637,7 +647,8 @@ public void disablePrewarming(DefaultMediaClock mediaClock) { prewarmingState == RENDERER_PREWARMING_STATE_TRANSITIONING_TO_PRIMARY || prewarmingState == RENDERER_PREWARMING_STATE_PREWARMING_PRIMARY; boolean isSecondaryActiveRenderer = - prewarmingState == RENDERER_PREWARMING_STATE_TRANSITIONING_TO_PRIMARY; + prewarmingState == RENDERER_PREWARMING_STATE_TRANSITIONING_TO_PRIMARY + || prewarmingState == RENDERER_PREWARMING_STATE_PREWARMING_SECONDARY; try { disableRenderer( isPrewarmingPrimary ? primaryRenderer : checkNotNull(secondaryRenderer), mediaClock); @@ -847,6 +858,7 @@ public void setVideoOutput(@Nullable Object videoOutput) throws ExoPlaybackExcep return; } if (prewarmingState == RENDERER_PREWARMING_STATE_TRANSITIONING_TO_PRIMARY + || prewarmingState == RENDERER_PREWARMING_STATE_PREWARMING_SECONDARY || prewarmingState == RENDERER_PREWARMING_STATE_NOT_PREWARMING_USING_SECONDARY) { checkNotNull(secondaryRenderer).handleMessage(Renderer.MSG_SET_VIDEO_OUTPUT, videoOutput); } else { @@ -949,6 +961,7 @@ private Renderer getRendererReadingFromPeriod(@Nullable MediaPeriodHolder period RENDERER_PREWARMING_STATE_NOT_PREWARMING_USING_PRIMARY, RENDERER_PREWARMING_STATE_NOT_PREWARMING_USING_SECONDARY, RENDERER_PREWARMING_STATE_PREWARMING_PRIMARY, + RENDERER_PREWARMING_STATE_PREWARMING_SECONDARY, RENDERER_PREWARMING_STATE_TRANSITIONING_TO_SECONDARY, RENDERER_PREWARMING_STATE_TRANSITIONING_TO_PRIMARY }) @@ -972,17 +985,23 @@ private Renderer getRendererReadingFromPeriod(@Nullable MediaPeriodHolder period */ /* package */ static final int RENDERER_PREWARMING_STATE_PREWARMING_PRIMARY = 2; + /** + * ExoPlayer is currently pre-warming the secondary renderer that is not being used for the + * current media item for a subsequent media item. + */ + /* package */ static final int RENDERER_PREWARMING_STATE_PREWARMING_SECONDARY = 3; + /** * Both a primary and secondary renderer are enabled and ExoPlayer is transitioning to a media * item using the secondary renderer. */ - /* package */ static final int RENDERER_PREWARMING_STATE_TRANSITIONING_TO_SECONDARY = 3; + /* package */ static final int RENDERER_PREWARMING_STATE_TRANSITIONING_TO_SECONDARY = 4; /** * Both a primary and secondary renderer are enabled and ExoPlayer is transitioning to a media * item using the primary renderer. */ - /* package */ static final int RENDERER_PREWARMING_STATE_TRANSITIONING_TO_PRIMARY = 4; + /* package */ static final int RENDERER_PREWARMING_STATE_TRANSITIONING_TO_PRIMARY = 5; /** Results for calls to {@link #replaceStreamsOrDisableRendererForTransition}. */ @Documented diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/ScrubbingModeParameters.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/ScrubbingModeParameters.java index 61c95e8c9e5..fe024858a6e 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/ScrubbingModeParameters.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/ScrubbingModeParameters.java @@ -23,8 +23,8 @@ import androidx.annotation.Nullable; import androidx.media3.common.C; import androidx.media3.common.C.TrackType; +import androidx.media3.common.Flags; import androidx.media3.common.util.UnstableApi; -import androidx.media3.exoplayer.video.MediaCodecVideoRenderer; import com.google.common.collect.ImmutableSet; import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.util.Objects; @@ -145,16 +145,14 @@ public Builder setShouldIncreaseCodecOperatingRate(boolean shouldIncreaseCodecOp } /** - * Sets whether ExoPlayer's {@linkplain - * ExoPlayer.Builder#experimentalSetDynamicSchedulingEnabled(boolean) dynamic scheduling} should + * Sets whether ExoPlayer's {@linkplain Flags#FLAG_DYNAMIC_SCHEDULING dynamic scheduling} should * be enabled in scrubbing mode. * *

    When used with {@link MediaCodec} in async mode, this can result in available output * buffers being handled more quickly when seeking. * - *

    If dynamic scheduling is enabled for all playback in {@link ExoPlayer.Builder} (which may - * become the default in a future release), this method is a no-op (i.e. you cannot disable - * dynamic scheduling when scrubbing using this method). + *

    If dynamic scheduling is enabled for all playback (which is the default), this method is a + * no-op (i.e. you cannot disable dynamic scheduling when scrubbing using this method). * *

    Defaults to {@code true} (this may change in a future release). * @@ -223,10 +221,9 @@ public Builder setAllowSkippingKeyFrameReset(boolean allowSkippingKeyFrameReset) * signalling that the decoded output of buffers between the previous keyframe and the target * frame is not needed by the player. * - *

    If the decode-only flag is {@linkplain - * MediaCodecVideoRenderer.Builder#experimentalSetEnableMediaCodecBufferDecodeOnlyFlag enabled} - * (which may become the default in a future release), this method is a no-op (i.e. you cannot - * disable usage of the decode-only flag when scrubbing using this method). + *

    If the decode-only flag is {@linkplain Flags#FLAG_ENABLE_MEDIACODEC_BUFFER_DECODE_ONLY + * enabled} (which may become the default in a future release), this method is a no-op (i.e. you + * cannot disable usage of the decode-only flag when scrubbing using this method). * *

    Defaults to {@code true} (this may change in a future release). */ @@ -289,8 +286,7 @@ public ScrubbingModeParameters build() { public final boolean allowSkippingMediaCodecFlush; /** - * Whether to enable ExoPlayer's {@linkplain - * ExoPlayer.Builder#experimentalSetDynamicSchedulingEnabled(boolean) dynamic scheduling} in + * Whether to enable ExoPlayer's {@linkplain Flags#FLAG_DYNAMIC_SCHEDULING dynamic scheduling} in * scrubbing mode. */ public final boolean shouldEnableDynamicScheduling; diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/SimpleExoPlayer.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/SimpleExoPlayer.java index 84cbf5443b7..77886d65bbf 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/SimpleExoPlayer.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/SimpleExoPlayer.java @@ -727,12 +727,26 @@ public DecoderCounters getAudioDecoderCounters() { } @Override + public void addVideoFrameMetadataListener(VideoFrameMetadataListener listener) { + blockUntilConstructorFinished(); + player.addVideoFrameMetadataListener(listener); + } + + @Override + public void removeVideoFrameMetadataListener(VideoFrameMetadataListener listener) { + blockUntilConstructorFinished(); + player.removeVideoFrameMetadataListener(listener); + } + + @Override + @Deprecated public void setVideoFrameMetadataListener(VideoFrameMetadataListener listener) { blockUntilConstructorFinished(); player.setVideoFrameMetadataListener(listener); } @Override + @Deprecated public void clearVideoFrameMetadataListener(VideoFrameMetadataListener listener) { blockUntilConstructorFinished(); player.clearVideoFrameMetadataListener(listener); diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/analytics/DefaultAnalyticsCollector.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/analytics/DefaultAnalyticsCollector.java index c0d2a9a55b6..b779b7d7922 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/analytics/DefaultAnalyticsCollector.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/analytics/DefaultAnalyticsCollector.java @@ -1163,7 +1163,7 @@ private void addTimelineForMediaPeriodId( if (mediaPeriodId == null) { return; } - if (preferredTimeline.getIndexOfPeriod(mediaPeriodId.periodUid) != C.INDEX_UNSET) { + if (isMediaPeriodIdInTimeline(mediaPeriodId, preferredTimeline, period)) { mediaPeriodTimelinesBuilder.put(mediaPeriodId, preferredTimeline); } else { @Nullable Timeline existingTimeline = mediaPeriodTimelines.get(mediaPeriodId); @@ -1173,6 +1173,22 @@ private void addTimelineForMediaPeriodId( } } + private static boolean isMediaPeriodIdInTimeline( + MediaPeriodId mediaPeriodId, Timeline timeline, Period period) { + int periodIndex = timeline.getIndexOfPeriod(mediaPeriodId.periodUid); + if (periodIndex == C.INDEX_UNSET) { + return false; + } + timeline.getPeriod(periodIndex, period); + if (mediaPeriodId.isAd()) { + return mediaPeriodId.adGroupIndex < period.getAdGroupCount() + && mediaPeriodId.adIndexInAdGroup + < period.getAdCountInAdGroup(mediaPeriodId.adGroupIndex); + } + return mediaPeriodId.nextAdGroupIndex == C.INDEX_UNSET + || mediaPeriodId.nextAdGroupIndex < period.getAdGroupCount(); + } + @Nullable private static MediaPeriodId findCurrentPlayerMediaPeriodInQueue( Player player, diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/analytics/MediaMetricsListener.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/analytics/MediaMetricsListener.java index 1f500e23c01..2c41d840408 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/analytics/MediaMetricsListener.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/analytics/MediaMetricsListener.java @@ -39,9 +39,11 @@ import android.util.Pair; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; +import androidx.annotation.VisibleForTesting; import androidx.media3.common.C; import androidx.media3.common.C.ContentType; import androidx.media3.common.DrmInitData; +import androidx.media3.common.Flags; import androidx.media3.common.Format; import androidx.media3.common.MediaItem; import androidx.media3.common.MediaLibraryInfo; @@ -140,6 +142,14 @@ public static MediaMetricsListener create(Context context) { private int audioUnderruns; private boolean reportedEventsForCurrentSession; + // Experiment ID encoding for reporting Media3 flag state via PlaybackMetrics.addExperimentId. + // Distinct, non-colliding base offsets are used to avoid collisions with other reporting systems. + // Only deviations from the default state are reported, keeping the footprint at zero for the vast + // majority of sessions running the default configuration. + private static final long CANARY_MODE_DISABLED_EXPERIMENT_ID = 7_000_000_000_000_000_000L; + private static final long FLAG_ENABLED_EXPERIMENT_ID_OFFSET = 7_001_000_000_000_000_000L; + private static final long FLAG_DISABLED_EXPERIMENT_ID_OFFSET = 7_002_000_000_000_000_000L; + /** * Creates the listener. * @@ -183,6 +193,7 @@ public void onSessionActive(EventTime eventTime, String sessionId) { new PlaybackMetrics.Builder() .setPlayerName(MediaLibraryInfo.TAG) .setPlayerVersion(MediaLibraryInfo.VERSION); + addFlagExperimentIds(metricsBuilder); maybeUpdateTimelineMetadata(eventTime.timeline, eventTime.mediaPeriodId); } @@ -637,6 +648,19 @@ private void finishCurrentSession() { reportedEventsForCurrentSession = false; } + @VisibleForTesting + /* package */ static void addFlagExperimentIds(PlaybackMetrics.Builder builder) { + if (!Flags.isCanaryModeEnabled()) { + builder = builder.addExperimentId(CANARY_MODE_DISABLED_EXPERIMENT_ID); + } + for (int flag : Flags.getEnabledOverrides()) { + builder = builder.addExperimentId(FLAG_ENABLED_EXPERIMENT_ID_OFFSET + flag); + } + for (int flag : Flags.getDisabledOverrides()) { + builder = builder.addExperimentId(FLAG_DISABLED_EXPERIMENT_ID_OFFSET + flag); + } + } + private static int getTrackChangeReason(@C.SelectionReason int trackSelectionReason) { switch (trackSelectionReason) { case C.SELECTION_REASON_INITIAL: diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioCapabilitiesReceiver.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioCapabilitiesReceiver.java index ea72450d504..6c4c8cf95ec 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioCapabilitiesReceiver.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioCapabilitiesReceiver.java @@ -29,6 +29,7 @@ import android.net.Uri; import android.os.Handler; import androidx.annotation.Nullable; +import androidx.annotation.VisibleForTesting; import androidx.media3.common.AudioAttributes; import androidx.media3.common.audio.AudioManagerCompat; import androidx.media3.common.util.UnstableApi; @@ -59,7 +60,7 @@ public interface Listener { private final Context context; private final Listener listener; private final Handler handler; - private final AudioDeviceCallback audioDeviceCallback; + @VisibleForTesting final android.media.AudioDeviceCallback audioDeviceCallback; private final BroadcastReceiver hdmiAudioPlugBroadcastReceiver; @Nullable private final ExternalSurroundSoundSettingObserver externalSurroundSoundSettingObserver; @Nullable private SpatializerWrapper spatializer; @@ -267,15 +268,25 @@ public void onChange(boolean selfChange) { private final class AudioDeviceCallback extends android.media.AudioDeviceCallback { @Override public void onAudioDevicesAdded(AudioDeviceInfo[] addedDevices) { - updateCurrentAudioCapabilities(); + // Some devices are known to invoke this callback on the wrong thread: + // https://github.com/androidx/media/issues/3386. + // This workaround can be removed when minSdk is bumped to API 38. + Util.postOrRun(handler, AudioCapabilitiesReceiver.this::updateCurrentAudioCapabilities); } @Override public void onAudioDevicesRemoved(AudioDeviceInfo[] removedDevices) { - if (Util.contains(removedDevices, routedDevice)) { - routedDevice = null; - } - updateCurrentAudioCapabilities(); + // Some devices are known to invoke this callback on the wrong thread: + // https://github.com/androidx/media/issues/3386. + // This workaround can be removed when minSdk is bumped to API 38. + Util.postOrRun( + handler, + () -> { + if (Util.contains(removedDevices, routedDevice)) { + routedDevice = null; + } + updateCurrentAudioCapabilities(); + }); } } } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioOutputProvider.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioOutputProvider.java index 85f4620973e..9f4aff290e2 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioOutputProvider.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioOutputProvider.java @@ -688,6 +688,16 @@ public InitializationException(Throwable cause) { @UnstableApi default void setClock(Clock clock) {} + /** + * Returns whether there are any pending asynchronous releases of audio outputs. + * + *

    Implementations that release resources synchronously should return false. + */ + @UnstableApi + default boolean hasPendingReleases() { + return false; + } + /** Releases resources held by the provider. */ void release(); } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioTrackAudioOutput.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioTrackAudioOutput.java index 3a6d02a3c85..4855ac05da4 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioTrackAudioOutput.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioTrackAudioOutput.java @@ -34,6 +34,7 @@ import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.MediaLibraryInfo; import androidx.media3.common.PlaybackParameters; import androidx.media3.common.util.BackgroundExecutor; @@ -44,6 +45,7 @@ import androidx.media3.common.util.Util; import androidx.media3.exoplayer.analytics.PlayerId; import androidx.media3.exoplayer.audio.AudioOutputProvider.OutputConfig; +import androidx.media3.extractor.ExtractorUtil; import java.nio.ByteBuffer; import java.nio.ByteOrder; import java.util.concurrent.Future; @@ -94,7 +96,7 @@ public interface CapabilityChangeListener { private final OutputConfig config; private final float maxPlaybackSpeed; @Nullable private final CapabilityChangeListener capabilityChangeListener; - @Nullable private OnRoutingChangedListenerApi24 onRoutingChangedListener; + @Nullable private OnRoutingChangedListener onRoutingChangedListener; private final AudioTrackPositionTracker audioTrackPositionTracker; private final boolean isOutputPcm; private final int pcmFrameSize; @@ -109,7 +111,6 @@ public interface CapabilityChangeListener { private int bytesUntilNextAvSync; private int framesPerEncodedSample; private int lastUnderrunCount; - private boolean hasData; /** * @deprecated Use {@link @@ -167,9 +168,8 @@ public AudioTrackAudioOutput( pcmFrameSize, config.bufferSize); - if (SDK_INT >= 24 && capabilityChangeListener != null) { - onRoutingChangedListener = - new OnRoutingChangedListenerApi24(audioTrack, capabilityChangeListener); + if (capabilityChangeListener != null) { + onRoutingChangedListener = new OnRoutingChangedListener(audioTrack, capabilityChangeListener); } offloadStreamEventCallbackV29 = isOffloadedPlayback() ? new StreamEventCallbackV29() : null; } @@ -241,7 +241,7 @@ public boolean write(ByteBuffer buffer, int encodedAccessUnitCount, long present throws WriteException { if (!isOutputPcm && framesPerEncodedSample == 0) { // If this is the first encoded sample, calculate the sample size in frames. - framesPerEncodedSample = DefaultAudioSink.getFramesPerEncodedSample(config.encoding, buffer); + framesPerEncodedSample = ExtractorUtil.getFramesPerEncodedSample(config.encoding, buffer); } maybeReportUnderrun(); int bytesRemaining = buffer.remaining(); @@ -313,7 +313,7 @@ public void release() { if (SDK_INT >= 29 && isOffloadedPlayback()) { checkNotNull(offloadStreamEventCallbackV29).unregister(); } - if (SDK_INT >= 24 && onRoutingChangedListener != null) { + if (onRoutingChangedListener != null) { onRoutingChangedListener.release(); onRoutingChangedListener = null; } @@ -397,6 +397,13 @@ public boolean isStalled() { return audioTrackPositionTracker.isStalled(getWrittenFrames()); } + /** Returns whether there are any pending asynchronous releases. */ + /* package */ static boolean hasPendingReleases() { + synchronized (releaseExecutorLock) { + return pendingReleaseCount > 0; + } + } + private long getWrittenFrames() { return isOutputPcm ? Util.ceilDivide(writtenPcmBytes, pcmFrameSize) : writtenEncodedFrames; } @@ -441,13 +448,13 @@ private int writeWithAvSync(AudioTrack audioTrack, ByteBuffer buffer, long prese } private void maybeReportUnderrun() { - if (listeners.isRunningOnCorrectThread() && hasPendingAudioTrackUnderruns(getWrittenFrames())) { + if (listeners.isRunningOnCorrectThread() && hasPendingAudioTrackUnderruns()) { listeners.sendEvent(Listener::onUnderrun); } } - private boolean hasPendingAudioTrackUnderruns(long writtenFrames) { - int underrunCount = getAudioOutputUnderrunCount(writtenFrames); + private boolean hasPendingAudioTrackUnderruns() { + int underrunCount = audioTrack.getUnderrunCount(); boolean result = underrunCount > lastUnderrunCount; // If the AudioTrack unexpectedly resets the underrun count, we should update it silently. @@ -456,25 +463,21 @@ private boolean hasPendingAudioTrackUnderruns(long writtenFrames) { return result; } - private int getAudioOutputUnderrunCount(long writtenFrames) { - if (SDK_INT >= 24) { - return audioTrack.getUnderrunCount(); - } - boolean hadData = hasData; - long currentPositionFrames = Util.durationUsToSampleCount(getPositionUs(), getSampleRate()); - hasData = writtenFrames > currentPositionFrames; - // For API 23- AudioTrack has no underrun API so we need to infer underruns heuristically. - boolean emitUnderrun = - hadData && !hasData && audioTrack.getPlayState() != AudioTrack.PLAYSTATE_STOPPED; - return emitUnderrun ? lastUnderrunCount + 1 : lastUnderrunCount; - } - private static void releaseAudioTrackAsync( AudioTrack audioTrack, ListenerSet listeners) { // AudioTrack.release can take some time, so we call it on a background thread. The background // thread is shared statically to avoid creating many threads when multiple players are released // at the same time. Handler audioTrackThreadHandler = Util.createHandlerForCurrentLooper(); + // Flush the audio track to avoid keeping state from previous playbacks (see b/22967293). + // Offload tracks do not need flush before release and flushing them may clear buffers of a + // subsequent track sharing the direct HAL stream (b/552739771). From API 29, AudioFlinger's + // DirectOutputThread also unconditionally flushes the HAL stream when adding a new track even + // if the previous track was already released (b/123082418). + boolean skipFlush = + SDK_INT >= 29 + && (Flags.isEnabled(Flags.FLAG_SKIP_AUDIO_TRACK_FLUSH_BEFORE_RELEASE) + || audioTrack.isOffloadedPlayback()); synchronized (releaseExecutorLock) { if (releaseExecutor == null) { releaseExecutor = @@ -485,9 +488,9 @@ private static void releaseAudioTrackAsync( releaseExecutor.schedule( () -> { try { - // We need to flush the audio track as some devices are known to keep state from - // previous playbacks if the track is not flushed at all (see b/22967293). - audioTrack.flush(); + if (!skipFlush) { + audioTrack.flush(); + } audioTrack.release(); } finally { if (audioTrackThreadHandler.getLooper().getThread().isAlive()) { @@ -515,8 +518,7 @@ private static void releaseAudioTrackAsync( } private static boolean isAudioTrackDeadObject(int status) { - return (SDK_INT >= 24 && status == AudioTrack.ERROR_DEAD_OBJECT) - || status == ERROR_NATIVE_DEAD_OBJECT; + return status == AudioTrack.ERROR_DEAD_OBJECT || status == ERROR_NATIVE_DEAD_OBJECT; } private final class PositionTrackerListener implements AudioTrackPositionTracker.Listener { @@ -599,8 +601,7 @@ private InvalidAudioTrackTimestampException(String message) { } } - @RequiresApi(24) - private static final class OnRoutingChangedListenerApi24 { + private static final class OnRoutingChangedListener { private final AudioTrack audioTrack; private final CapabilityChangeListener capabilityChangeListener; @@ -608,7 +609,7 @@ private static final class OnRoutingChangedListenerApi24 { @Nullable private AudioRouting.OnRoutingChangedListener listener; - private OnRoutingChangedListenerApi24( + private OnRoutingChangedListener( AudioTrack audioTrack, CapabilityChangeListener capabilityChangeListener) { this.audioTrack = audioTrack; this.capabilityChangeListener = capabilityChangeListener; diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioTrackAudioOutputProvider.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioTrackAudioOutputProvider.java index 77d9af183a5..6935b3d1456 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioTrackAudioOutputProvider.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioTrackAudioOutputProvider.java @@ -35,7 +35,6 @@ import android.os.Looper; import android.util.Pair; import androidx.annotation.Nullable; -import androidx.annotation.RequiresApi; import androidx.media3.common.AudioAttributes; import androidx.media3.common.C; import androidx.media3.common.Format; @@ -106,7 +105,6 @@ public Builder(@Nullable Context context) { * @return This builder. */ @CanIgnoreReturnValue - @RequiresApi(24) public Builder setAudioTrackBuilderModifier( BiConsumer audioTrackBuilderModifier) { this.audioTrackBuilderModifier = audioTrackBuilderModifier; @@ -381,7 +379,7 @@ public AudioTrackAudioOutput getAudioOutput(OutputConfig config) throws Initiali if (SDK_INT >= 34 && contextForAudioTrack != null) { audioTrackBuilder.setContext(contextForAudioTrack); } - if (builderModifier != null && SDK_INT >= 24) { + if (builderModifier != null) { builderModifier.accept(audioTrackBuilder, config); } audioTrack = audioTrackBuilder.build(); @@ -424,6 +422,12 @@ public void setClock(Clock clock) { this.clock = clock; } + @UnstableApi + @Override + public boolean hasPendingReleases() { + return AudioTrackAudioOutput.hasPendingReleases(); + } + @Override public void release() { if (listeners != null) { diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioTrackPositionTracker.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioTrackPositionTracker.java index 6f5c1c24f82..cd2f603024a 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioTrackPositionTracker.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/AudioTrackPositionTracker.java @@ -119,6 +119,13 @@ void onSystemTimeUsMismatch( private static final long FORCE_RESET_WORKAROUND_TIMEOUT_MS = 200; + /** + * The minimum distance between the previous and current raw playback head position to treat a + * decrease in position as a 32-bit unsigned integer wrap-around rather than an unexpected + * position decrease or reset. + */ + private static final long MIN_RAW_PLAYBACK_HEAD_POSITION_WRAP_DISTANCE = 1L << 31; + private static final int MAX_PLAYHEAD_OFFSET_COUNT = 10; private static final int MIN_PLAYHEAD_OFFSET_SAMPLE_INTERVAL_US = 30_000; private static final int MIN_LATENCY_SAMPLE_INTERVAL_US = 50_0000; @@ -551,9 +558,13 @@ private void updateRawPlaybackHeadPosition(long currentTimeMs) { if (expectRawPlaybackHeadReset) { sumRawPlaybackHeadPosition += this.rawPlaybackHeadPosition; expectRawPlaybackHeadReset = false; - } else { + } else if (this.rawPlaybackHeadPosition - rawPlaybackHeadPosition + >= MIN_RAW_PLAYBACK_HEAD_POSITION_WRAP_DISTANCE) { // The value must have wrapped around. rawPlaybackHeadWrapCount++; + } else { + resetSyncParams(); + audioTimestampPoller.reset(); } } this.rawPlaybackHeadPosition = rawPlaybackHeadPosition; diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/ChannelMappingAudioProcessor.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/ChannelMappingAudioProcessor.java index eb0316aee4a..9d5f9a8207c 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/ChannelMappingAudioProcessor.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/ChannelMappingAudioProcessor.java @@ -15,6 +15,7 @@ */ package androidx.media3.exoplayer.audio; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; import static androidx.media3.common.util.Util.getByteDepth; import static androidx.media3.common.util.Util.getInt24; import static androidx.media3.common.util.Util.isEncodingLinearPcm; @@ -22,7 +23,9 @@ import static com.google.common.base.Preconditions.checkNotNull; import androidx.annotation.Nullable; +import androidx.annotation.RestrictTo; import androidx.media3.common.C; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.audio.AudioProcessor; import androidx.media3.common.audio.BaseAudioProcessor; import androidx.media3.common.util.UnstableApi; @@ -34,7 +37,8 @@ * channels. This can be used to reorder, duplicate or discard channels. */ @UnstableApi -public final class ChannelMappingAudioProcessor extends BaseAudioProcessor { +public final class ChannelMappingAudioProcessor extends BaseAudioProcessor + implements MetricsProvider { @Nullable private ImmutableIntArray pendingOutputChannels; @Nullable private ImmutableIntArray outputChannels; @@ -144,4 +148,10 @@ protected void onReset() { outputChannels = null; pendingOutputChannels = null; } + + @Override + @RestrictTo(LIBRARY_GROUP) + public void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_AUDIO_CHANNEL_MANIPULATION); + } } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/DefaultAudioSink.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/DefaultAudioSink.java index 7cf699b3648..5800f9d5981 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/DefaultAudioSink.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/DefaultAudioSink.java @@ -54,18 +54,12 @@ import androidx.media3.common.util.Log; import androidx.media3.common.util.UnstableApi; import androidx.media3.common.util.Util; -import androidx.media3.container.OpusUtil; import androidx.media3.exoplayer.ExoPlayer.AudioOffloadListener; import androidx.media3.exoplayer.analytics.PlayerId; import androidx.media3.exoplayer.audio.AudioOutputProvider.FormatConfig; import androidx.media3.exoplayer.audio.AudioOutputProvider.FormatSupport; import androidx.media3.exoplayer.audio.AudioOutputProvider.OutputConfig; -import androidx.media3.extractor.AacUtil; -import androidx.media3.extractor.Ac3Util; -import androidx.media3.extractor.Ac4Util; -import androidx.media3.extractor.DtsUtil; import androidx.media3.extractor.ExtractorUtil; -import androidx.media3.extractor.MpegAudioUtil; import com.google.common.collect.ImmutableList; import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.lang.annotation.Documented; @@ -77,7 +71,6 @@ import java.nio.ByteOrder; import java.util.ArrayDeque; import java.util.Objects; -import java.util.concurrent.atomic.AtomicInteger; import java.util.function.BiConsumer; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; import org.checkerframework.checker.nullness.qual.RequiresNonNull; @@ -562,8 +555,6 @@ public DefaultAudioSink build() { private static final String TAG = "DefaultAudioSink"; - private static final AtomicInteger pendingReleaseCount = new AtomicInteger(); - @Nullable private final Context context; private final androidx.media3.common.audio.AudioProcessorChain audioProcessorChain; private final boolean enableFloatOutput; @@ -601,6 +592,8 @@ public DefaultAudioSink build() { private long writtenPcmBytes; private long writtenEncodedFrames; private int framesPerEncodedSample; + private int inputBufferOriginalRemaining; + private long currentBufferFramesWritten; private boolean startMediaTimeUsNeedsSync; private boolean startMediaTimeUsNeedsInit; private long startMediaTimeUs; @@ -1013,7 +1006,7 @@ public boolean handleBuffer( if (!configuration.isPcm() && framesPerEncodedSample == 0) { // If this is the first encoded sample, calculate the sample size in frames. framesPerEncodedSample = - getFramesPerEncodedSample(configuration.outputConfig.encoding, buffer); + ExtractorUtil.getFramesPerEncodedSample(configuration.outputConfig.encoding, buffer); if (framesPerEncodedSample == 0) { // We still don't know the number of frames per sample, so drop the buffer. // For TrueHD this can occur after some seek operations, as not every sample starts with @@ -1071,6 +1064,8 @@ public boolean handleBuffer( inputBuffer = buffer; inputBufferAccessUnitCount = encodedAccessUnitCount; + inputBufferOriginalRemaining = buffer.remaining(); + currentBufferFramesWritten = 0; } processBuffers(presentationTimeUs); @@ -1316,17 +1311,37 @@ private void drainOutputBuffer(long avSyncPresentationTimeUs) throws WriteExcept } } + int bytesWritten = bytesRemaining - outputBuffer.remaining(); if (configuration.isPcm()) { - writtenPcmBytes += bytesRemaining - outputBuffer.remaining(); + writtenPcmBytes += bytesWritten; } if (fullyHandled) { if (!configuration.isPcm()) { // When playing non-PCM, the inputBuffer is never processed, thus the last inputBuffer // must be the current input buffer. checkState(outputBuffer == inputBuffer); - writtenEncodedFrames += (long) framesPerEncodedSample * inputBufferAccessUnitCount; + // Add only the remaining unreconciled frames to ensure exact sample accuracy. + writtenEncodedFrames += + ((long) framesPerEncodedSample * inputBufferAccessUnitCount) + - currentBufferFramesWritten; + currentBufferFramesWritten = 0; } outputBuffer = null; + } else { + if (!configuration.isPcm() && bytesWritten > 0) { + checkState(inputBufferOriginalRemaining > 0); + int totalBytesWrittenForCurrentBuffer = + inputBufferOriginalRemaining - outputBuffer.remaining(); + long totalFramesWrittenForCurrentBuffer = + Util.scaleLargeValue( + totalBytesWrittenForCurrentBuffer, + (long) framesPerEncodedSample * inputBufferAccessUnitCount, + inputBufferOriginalRemaining, + RoundingMode.CEILING); + long newFramesWritten = totalFramesWrittenForCurrentBuffer - currentBufferFramesWritten; + writtenEncodedFrames += newFramesWritten; + currentBufferFramesWritten = totalFramesWrittenForCurrentBuffer; + } } } @@ -1575,7 +1590,6 @@ public void flush() { // We need to release the audio output on every flush because of known AudioTrack flush issues // on some devices. See b/7941810 or b/19193985. // TODO: b/143500232 - Experiment with not releasing AudioOutput on flush. - pendingReleaseCount.incrementAndGet(); audioOutput.release(); audioOutput = null; } @@ -1638,6 +1652,8 @@ private void resetSinkStateForFlush() { writtenEncodedFrames = 0; isWaitingForOffloadEndOfStreamHandled = false; framesPerEncodedSample = 0; + inputBufferOriginalRemaining = 0; + currentBufferFramesWritten = 0; mediaPositionParameters = new MediaPositionParameters( playbackParameters, /* mediaTimeUs= */ 0, /* audioOutputPositionUs= */ 0); @@ -1844,62 +1860,6 @@ private FormatConfig getFormatConfig(Format format, int preferredBufferSize) { .build(); } - /* package */ static int getFramesPerEncodedSample(@C.Encoding int encoding, ByteBuffer buffer) { - switch (encoding) { - case C.ENCODING_MP3: - int headerDataInBigEndian = Util.getBigEndianInt(buffer, buffer.position()); - int frameCount = MpegAudioUtil.parseMpegAudioFrameSampleCount(headerDataInBigEndian); - if (frameCount == C.LENGTH_UNSET) { - throw new IllegalArgumentException(); - } - return frameCount; - case C.ENCODING_AAC_LC: - return AacUtil.AAC_LC_AUDIO_SAMPLE_COUNT; - case C.ENCODING_AAC_HE_V1: - case C.ENCODING_AAC_HE_V2: - return AacUtil.AAC_HE_AUDIO_SAMPLE_COUNT; - case C.ENCODING_AAC_XHE: - return AacUtil.AAC_XHE_AUDIO_SAMPLE_COUNT; - case C.ENCODING_AAC_ELD: - return AacUtil.AAC_LD_AUDIO_SAMPLE_COUNT; - case C.ENCODING_DTS: - case C.ENCODING_DTS_HD: - case C.ENCODING_DTS_UHD_P2: - return DtsUtil.parseDtsAudioSampleCount(buffer); - case C.ENCODING_AC3: - case C.ENCODING_E_AC3: - case C.ENCODING_E_AC3_JOC: - return Ac3Util.parseAc3SyncframeAudioSampleCount(buffer); - case C.ENCODING_AC4: - return Ac4Util.parseAc4SyncframeAudioSampleCount(buffer); - case C.ENCODING_DOLBY_TRUEHD: - int syncframeOffset = Ac3Util.findTrueHdSyncframeOffset(buffer); - return syncframeOffset == C.INDEX_UNSET - ? 0 - : (Ac3Util.parseTrueHdSyncframeAudioSampleCount(buffer, syncframeOffset) - * Ac3Util.TRUEHD_RECHUNK_SAMPLE_COUNT); - case C.ENCODING_OPUS: - return OpusUtil.parseOggPacketAudioSampleCount(buffer); - case C.ENCODING_PCM_16BIT: - case C.ENCODING_PCM_16BIT_BIG_ENDIAN: - case C.ENCODING_PCM_24BIT: - case C.ENCODING_PCM_24BIT_BIG_ENDIAN: - case C.ENCODING_PCM_32BIT: - case C.ENCODING_PCM_32BIT_BIG_ENDIAN: - case C.ENCODING_PCM_8BIT: - case C.ENCODING_PCM_FLOAT: - case C.ENCODING_PCM_FLOAT_BIG_ENDIAN: - case C.ENCODING_PCM_DOUBLE: - case C.ENCODING_PCM_DOUBLE_BIG_ENDIAN: - case C.ENCODING_AAC_ER_BSAC: - case C.ENCODING_DSD: - case C.ENCODING_INVALID: - case Format.NO_VALUE: - default: - throw new IllegalStateException("Unexpected audio encoding: " + encoding); - } - } - private void playPendingData() { if (!stoppedAudioOutput) { stoppedAudioOutput = true; @@ -1938,8 +1898,8 @@ private boolean hasAudioOutputPendingData(long writtenFrames) { return writtenFrames > currentPositionFrames; } - private static boolean hasPendingAudioOutputReleases() { - return pendingReleaseCount.get() > 0; + private boolean hasPendingAudioOutputReleases() { + return audioOutputProvider.hasPendingReleases(); } @RequiresApi(34) @@ -2025,7 +1985,6 @@ public void onUnderrun() { public void onReleased() { // Don't check for stale events. It's expected that this event arrives after the class field // has been updated to null or a new listener. - pendingReleaseCount.getAndDecrement(); if (listener != null) { listener.onAudioTrackReleased( new AudioTrackConfig( @@ -2131,7 +2090,7 @@ private boolean isPcm() { } } - private static final class PendingExceptionHolder { + private final class PendingExceptionHolder { /** * The duration for which failed audio output operations may be retried before throwing an diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/ForwardingAudioOutputProvider.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/ForwardingAudioOutputProvider.java index 0b36623799a..85bf8384da3 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/ForwardingAudioOutputProvider.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/ForwardingAudioOutputProvider.java @@ -63,6 +63,12 @@ public void setClock(Clock clock) { audioOutputProvider.setClock(clock); } + @UnstableApi + @Override + public boolean hasPendingReleases() { + return audioOutputProvider.hasPendingReleases(); + } + @Override public void release() { audioOutputProvider.release(); diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/IamfUtil.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/IamfUtil.java index 277c69386cb..548c0631434 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/IamfUtil.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/IamfUtil.java @@ -378,9 +378,9 @@ public static int getChannelMaskForOutputLayout(@OutputLayout int outputLayout) * @param useIntegratedBinauralRenderer When it seems the user is using headphones (because {@link * AudioCapabilities#getSpatializerChannelMasks()} is non-empty), if {@code * useIntegratedBinauralRenderer} is {@code true}, this method will return {@link - * OUTPUT_LAYOUT_BINAURAL} to get binaural audio directly from the decoder, otherwise, it will - * return an output layout appropriate for the {@link android.media.Spatializer} based on the - * available Spatializer channel masks. + * #OUTPUT_LAYOUT_BINAURAL} to get binaural audio directly from the decoder, otherwise, it + * will return an output layout appropriate for the {@link android.media.Spatializer} based on + * the available Spatializer channel masks. */ public static @OutputLayout int getOutputLayoutForCurrentConfiguration( AudioCapabilities audioCapabilities, boolean useIntegratedBinauralRenderer) { diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/MediaCodecAudioRenderer.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/MediaCodecAudioRenderer.java index ec4d209aaad..a4c5b4845ea 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/MediaCodecAudioRenderer.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/MediaCodecAudioRenderer.java @@ -30,7 +30,6 @@ import android.media.MediaCodec; import android.media.MediaCrypto; import android.media.MediaFormat; -import android.os.Build; import android.os.Bundle; import android.os.Handler; import android.util.Pair; @@ -40,7 +39,6 @@ import androidx.media3.common.AuxEffectInfo; import androidx.media3.common.C; import androidx.media3.common.Format; -import androidx.media3.common.MediaLibraryInfo; import androidx.media3.common.MimeTypes; import androidx.media3.common.PlaybackException; import androidx.media3.common.PlaybackParameters; @@ -48,6 +46,7 @@ import androidx.media3.common.util.CodecSpecificDataUtil; import androidx.media3.common.util.Log; import androidx.media3.common.util.MediaFormatUtil; +import androidx.media3.common.util.ThrowingRunnable; import androidx.media3.common.util.UnstableApi; import androidx.media3.common.util.Util; import androidx.media3.decoder.DecoderInputBuffer; @@ -77,6 +76,7 @@ import java.nio.ByteOrder; import java.util.List; import java.util.Objects; +import java.util.concurrent.atomic.AtomicBoolean; /** * Decodes and renders audio using {@link MediaCodec} and an {@link AudioSink}. @@ -132,9 +132,9 @@ public class MediaCodecAudioRenderer extends MediaCodecRenderer implements Media private final EventDispatcher eventDispatcher; private final AudioSink audioSink; @Nullable private final LoudnessCodecController loudnessCodecController; + private final AtomicBoolean processOutputBufferResultHolder; private int codecMaxInputSize; - private boolean codecNeedsDiscardChannelsWorkaround; private boolean codecNeedsVorbisToAndroidChannelMappingWorkaround; @Nullable private Format inputFormat; @@ -330,6 +330,7 @@ public MediaCodecAudioRenderer( this.loudnessCodecController = loudnessCodecController; rendererPriority = C.PRIORITY_PLAYBACK; eventDispatcher = new EventDispatcher(eventHandler, eventListener); + processOutputBufferResultHolder = new AtomicBoolean(); nextBufferToWritePresentationTimeUs = C.TIME_UNSET; firstNotReadyTimeMs = C.TIME_UNSET; hasBeenReady = false; @@ -502,7 +503,6 @@ protected MediaCodecAdapter.Configuration getMediaCodecConfiguration( @Nullable MediaCrypto crypto, float codecOperatingRate) { codecMaxInputSize = getCodecMaxInputSize(codecInfo, format, getStreamFormats()); - codecNeedsDiscardChannelsWorkaround = codecNeedsDiscardChannelsWorkaround(codecInfo.name); codecNeedsVorbisToAndroidChannelMappingWorkaround = codecNeedsVorbisToAndroidChannelMappingWorkaround(codecInfo.name); MediaFormat mediaFormat = @@ -530,7 +530,7 @@ protected DecoderReuseEvaluation canReuseCodec( // to stereo. discardReasons |= DecoderReuseEvaluation.DISCARD_REASON_AUDIO_BYPASS_POSSIBLE; } - if (getCodecMaxInputSize(codecInfo, newFormat) > codecMaxInputSize) { + if (newFormat.maxInputSize > codecMaxInputSize) { discardReasons |= DISCARD_REASON_MAX_INPUT_SIZE_EXCEEDED; } @@ -650,7 +650,7 @@ protected void onOutputFormatChanged(Format format, @Nullable MediaFormat mediaF if (MimeTypes.AUDIO_RAW.equals(format.sampleMimeType)) { // For PCM streams, the encoder passes through int samples despite set to float mode. pcmEncoding = format.pcmEncoding; - } else if (SDK_INT >= 24 && mediaFormat.containsKey(MediaFormat.KEY_PCM_ENCODING)) { + } else if (mediaFormat.containsKey(MediaFormat.KEY_PCM_ENCODING)) { pcmEncoding = mediaFormat.getInteger(MediaFormat.KEY_PCM_ENCODING); } else if (mediaFormat.containsKey(VIVO_BITS_PER_SAMPLE_KEY)) { pcmEncoding = Util.getPcmEncoding(mediaFormat.getInteger(VIVO_BITS_PER_SAMPLE_KEY)); @@ -690,16 +690,7 @@ protected void onOutputFormatChanged(Format format, @Nullable MediaFormat mediaF .setChannelMask(channelMask) .setSampleRate(mediaFormat.getInteger(MediaFormat.KEY_SAMPLE_RATE)) .build(); - if (codecNeedsDiscardChannelsWorkaround - && audioSinkInputFormat.channelCount == 6 - && format.channelCount < 6) { - ImmutableIntArray.Builder channelMapBuilder = - ImmutableIntArray.builder(format.channelCount); - for (int i = 0; i < format.channelCount; i++) { - channelMapBuilder.add(i); - } - channelMap = channelMapBuilder.build(); - } else if (codecNeedsVorbisToAndroidChannelMappingWorkaround) { + if (codecNeedsVorbisToAndroidChannelMappingWorkaround) { channelMap = VorbisUtil.getVorbisToAndroidChannelLayoutMapping(audioSinkInputFormat.channelCount); } @@ -927,9 +918,16 @@ protected boolean processOutputBuffer( return true; } - boolean fullyConsumed; try { - fullyConsumed = audioSink.handleBuffer(buffer, bufferPresentationTimeUs, sampleCount); + ThrowingRunnable handleBufferOperation = + () -> + processOutputBufferResultHolder.set( + audioSink.handleBuffer(buffer, bufferPresentationTimeUs, sampleCount)); + if (codec != null) { + codec.useBuffer(handleBufferOperation); + } else { + handleBufferOperation.run(); + } } catch (InitializationException e) { throw createRendererException( e, @@ -948,8 +946,14 @@ && getConfiguration().offloadModePreferred != AudioSink.OFFLOAD_MODE_DISABLED && getConfiguration().offloadModePreferred != AudioSink.OFFLOAD_MODE_DISABLED ? PlaybackException.ERROR_CODE_AUDIO_TRACK_OFFLOAD_WRITE_FAILED : PlaybackException.ERROR_CODE_AUDIO_TRACK_WRITE_FAILED); + } catch (RuntimeException e) { + throw e; + } catch (Exception e) { + throw new IllegalStateException(e); } + boolean fullyConsumed = processOutputBufferResultHolder.get(); + if (fullyConsumed) { if (codec != null) { if (DEBUG_LOG_ENABLED) { @@ -1069,7 +1073,7 @@ && isBypassEnabled()) { */ protected int getCodecMaxInputSize( MediaCodecInfo codecInfo, Format format, Format[] streamFormats) { - int maxInputSize = getCodecMaxInputSize(codecInfo, format); + int maxInputSize = format.maxInputSize; if (streamFormats.length == 1) { // The single entry in streamFormats must correspond to the format for which the codec is // being configured. @@ -1077,33 +1081,12 @@ protected int getCodecMaxInputSize( } for (Format streamFormat : streamFormats) { if (codecInfo.canReuseCodec(format, streamFormat).result != REUSE_RESULT_NO) { - maxInputSize = max(maxInputSize, getCodecMaxInputSize(codecInfo, streamFormat)); + maxInputSize = max(maxInputSize, streamFormat.maxInputSize); } } return maxInputSize; } - /** - * Returns a maximum input buffer size for a given {@link Format}. - * - * @param codecInfo A {@link MediaCodecInfo} describing the decoder. - * @param format The {@link Format}. - * @return A maximum input buffer size in bytes, or {@link Format#NO_VALUE} if a maximum could not - * be determined. - */ - private int getCodecMaxInputSize(MediaCodecInfo codecInfo, Format format) { - if ("OMX.google.raw.decoder".equals(codecInfo.name)) { - // OMX.google.raw.decoder didn't resize its output buffers correctly prior to N, except on - // Android TV running M, so there's no point requesting a non-default input size. Doing so may - // cause a native crash, whereas not doing so will cause a more controlled failure when - // attempting to fill an input buffer. See: https://github.com/google/ExoPlayer/issues/4057. - if (SDK_INT == 23 && !Util.isTv(context)) { - return Format.NO_VALUE; - } - } - return format.maxInputSize; - } - /** * Returns the framework {@link MediaFormat} that can be used to configure a {@link MediaCodec} * for decoding the given {@link Format} for playback. @@ -1127,8 +1110,8 @@ protected MediaFormat getMediaFormat( // Set codec max values. MediaFormatUtil.maybeSetInteger(mediaFormat, MediaFormat.KEY_MAX_INPUT_SIZE, codecMaxInputSize); // Set codec configuration values. - mediaFormat.setInteger(MediaFormat.KEY_PRIORITY, 0 /* realtime priority */); - if (codecOperatingRate != CODEC_OPERATING_RATE_UNSET && !deviceDoesntSupportOperatingRate()) { + mediaFormat.setInteger(MediaFormat.KEY_PRIORITY, C.MEDIA_CODEC_PRIORITY_REALTIME); + if (codecOperatingRate != CODEC_OPERATING_RATE_UNSET) { mediaFormat.setFloat(MediaFormat.KEY_OPERATING_RATE, codecOperatingRate); } if (MimeTypes.AUDIO_AC4.equals(format.sampleMimeType)) { @@ -1143,10 +1126,9 @@ protected MediaFormat getMediaFormat( mediaFormat.setInteger("ac4-is-sync", 1); } } - if (SDK_INT >= 24 - && audioSink.getFormatSupport( - Util.getPcmFormat(C.ENCODING_PCM_FLOAT, format.channelCount, format.sampleRate)) - == AudioSink.SINK_FORMAT_SUPPORTED_DIRECTLY) { + if (audioSink.getFormatSupport( + Util.getPcmFormat(C.ENCODING_PCM_FLOAT, format.channelCount, format.sampleRate)) + == AudioSink.SINK_FORMAT_SUPPORTED_DIRECTLY) { mediaFormat.setInteger(MediaFormat.KEY_PCM_ENCODING, AudioFormat.ENCODING_PCM_FLOAT); } if (SDK_INT >= 32) { @@ -1210,37 +1192,6 @@ private void updateCurrentPosition() { } } - /** - * Returns whether the device's decoders are known to not support setting the codec operating - * rate. - * - *

    See GitHub issue #5821. - */ - private static boolean deviceDoesntSupportOperatingRate() { - return MediaLibraryInfo.enableWorkarounds() - && SDK_INT == 23 - && ("ZTE B2017G".equals(Build.MODEL) || "AXON 7 mini".equals(Build.MODEL)); - } - - /** - * Returns whether the decoder is known to output six audio channels when provided with input with - * fewer than six channels. - * - *

    See [Internal: b/35655036]. - */ - private static boolean codecNeedsDiscardChannelsWorkaround(String codecName) { - if (!MediaLibraryInfo.enableWorkarounds()) { - return false; - } - // The workaround applies to Samsung Galaxy S6 and Samsung Galaxy S7. - return SDK_INT < 24 - && "OMX.SEC.aac.dec".equals(codecName) - && "samsung".equals(Build.MANUFACTURER) - && (Build.DEVICE.startsWith("zeroflte") - || Build.DEVICE.startsWith("herolte") - || Build.DEVICE.startsWith("heroqlte")); - } - /** * Returns whether the decoder is known to output PCM samples in VORBIS order, which does not * match the channel layout required by AudioTrack. diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/PcmAudioUtil.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/PcmAudioUtil.java index 7e43f29f8ca..6fda8676ff9 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/PcmAudioUtil.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/PcmAudioUtil.java @@ -71,7 +71,7 @@ public static ByteBuffer rampUpVolume( public static int readAs32BitIntPcm(ByteBuffer buffer, @C.Encoding int pcmEncoding) { switch (pcmEncoding) { case C.ENCODING_PCM_8BIT: - return (buffer.get() & 0xFF) << 24; + return ((buffer.get() & 0xFF) - 128) << 24; case C.ENCODING_PCM_16BIT: return ((buffer.get() & 0xFF) << 16) | ((buffer.get() & 0xFF) << 24); case C.ENCODING_PCM_16BIT_BIG_ENDIAN: @@ -161,7 +161,7 @@ public static void write32BitIntPcm( buffer.put((byte) (pcm32bit >> 24)); return; case C.ENCODING_PCM_8BIT: - buffer.put((byte) (pcm32bit >> 24)); + buffer.put((byte) ((pcm32bit >> 24) + 128)); return; case C.ENCODING_PCM_32BIT_BIG_ENDIAN: buffer.put((byte) (pcm32bit >> 24)); diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/SilenceSkippingAudioProcessor.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/SilenceSkippingAudioProcessor.java index 3a9dc605fe4..c1ef64e2a1b 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/SilenceSkippingAudioProcessor.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/SilenceSkippingAudioProcessor.java @@ -15,14 +15,17 @@ */ package androidx.media3.exoplayer.audio; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkState; import static java.lang.Math.min; import static java.lang.annotation.ElementType.TYPE_USE; import androidx.annotation.IntDef; +import androidx.annotation.RestrictTo; import androidx.media3.common.C; import androidx.media3.common.Format; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.audio.AudioProcessor; import androidx.media3.common.audio.BaseAudioProcessor; import androidx.media3.common.util.UnstableApi; @@ -38,7 +41,8 @@ * PCM. */ @UnstableApi -public final class SilenceSkippingAudioProcessor extends BaseAudioProcessor { +public final class SilenceSkippingAudioProcessor extends BaseAudioProcessor + implements MetricsProvider { /** * Default fraction of the original silence to keep. Between [0, 1]. 1 means keep all silence. 0 @@ -309,7 +313,9 @@ public void onFlush(StreamMetadata streamMetadata) { bytesPerFrame = inputAudioFormat.channelCount * 2; // Divide by 2 to allow the buffer to be split into two bytesPerFrame aligned parts. int maybeSilenceBufferSize = - alignToBytePerFrameBoundary(durationUsToFrames(minimumSilenceDurationUs) / 2) * 2; + alignToBytePerFrameBoundary( + durationUsToFrames(minimumSilenceDurationUs) * bytesPerFrame / 2) + * 2; if (maybeSilenceBuffer.length != maybeSilenceBufferSize) { maybeSilenceBuffer = new byte[maybeSilenceBufferSize]; contiguousOutputBuffer = new byte[maybeSilenceBufferSize]; @@ -647,6 +653,7 @@ private void modifyVolume(byte[] sampleBuffer, int size, @VolumeChangeType int v return; } + int lastFrameIdx = (size / bytesPerFrame) - 1; for (int idx = 0; idx < size; idx += 2) { byte mostSignificantByte = sampleBuffer[idx + 1]; byte leastSignificantByte = sampleBuffer[idx]; @@ -655,10 +662,10 @@ private void modifyVolume(byte[] sampleBuffer, int size, @VolumeChangeType int v int volumeModificationPercentage; if (volumeChangeType == FADE_OUT) { volumeModificationPercentage = - calculateFadeOutPercentage(/* value= */ idx, /* max= */ size - 1); + calculateFadeOutPercentage(/* value= */ idx / bytesPerFrame, /* max= */ lastFrameIdx); } else if (volumeChangeType == FADE_IN) { volumeModificationPercentage = - calculateFadeInPercentage(/* value= */ idx, /* max= */ size - 1); + calculateFadeInPercentage(/* value= */ idx / bytesPerFrame, /* max= */ lastFrameIdx); } else { volumeModificationPercentage = minVolumeToKeepPercentageWhenMuting; } @@ -669,12 +676,18 @@ private void modifyVolume(byte[] sampleBuffer, int size, @VolumeChangeType int v } private int calculateFadeOutPercentage(int value, int max) { + if (max == 0) { + return minVolumeToKeepPercentageWhenMuting; + } return ((minVolumeToKeepPercentageWhenMuting - 100) * ((AVOID_TRUNCATION_FACTOR * value) / max)) / AVOID_TRUNCATION_FACTOR + 100; } private int calculateFadeInPercentage(int value, int max) { + if (max == 0) { + return 100; + } return (minVolumeToKeepPercentageWhenMuting + ((100 - minVolumeToKeepPercentageWhenMuting) * (AVOID_TRUNCATION_FACTOR * value) / max) / AVOID_TRUNCATION_FACTOR); @@ -753,4 +766,10 @@ private boolean isNoise(byte mostSignificantByte, byte leastSignificantByte) { return Math.abs(twoByteSampleToInt(mostSignificantByte, leastSignificantByte)) > silenceThresholdLevel; } + + @Override + @RestrictTo(LIBRARY_GROUP) + public void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_AUDIO_TEMPORAL_TRIM); + } } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/TrimmingAudioProcessor.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/TrimmingAudioProcessor.java index 5484439950e..cefffbac18c 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/TrimmingAudioProcessor.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/audio/TrimmingAudioProcessor.java @@ -15,10 +15,13 @@ */ package androidx.media3.exoplayer.audio; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; import static androidx.media3.common.util.Util.isEncodingLinearPcm; import static java.lang.Math.max; import static java.lang.Math.min; +import androidx.annotation.RestrictTo; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.audio.BaseAudioProcessor; import androidx.media3.common.util.UnstableApi; import androidx.media3.common.util.Util; @@ -26,7 +29,7 @@ /** Audio processor for trimming samples from the start/end of data. */ @UnstableApi -public final class TrimmingAudioProcessor extends BaseAudioProcessor { +public final class TrimmingAudioProcessor extends BaseAudioProcessor implements MetricsProvider { private int trimStartFrames; private int trimEndFrames; @@ -192,4 +195,10 @@ protected void onFlush(StreamMetadata streamMetadata) { protected void onReset() { endBuffer = Util.EMPTY_BYTE_ARRAY; } + + @Override + @RestrictTo(LIBRARY_GROUP) + public void populateMetrics(MetricConsumer consumer) { + consumer.setCategory(MetricConsumer.CATEGORY_AUDIO_TEMPORAL_TRIM); + } } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/drm/DefaultDrmSession.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/drm/DefaultDrmSession.java index ed2c33b5d86..92fc5322104 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/drm/DefaultDrmSession.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/drm/DefaultDrmSession.java @@ -691,9 +691,12 @@ public void handleMessage(Message msg) { synchronized (keyRequestInfoLock) { if (currentKeyRequestInfo != null && keyResponse.loadEventInfo != null) { currentKeyRequestInfo.addLoadInfo( - keyResponse.loadEventInfo.copyWithTaskIdAndDurationMs( - requestTask.taskId, - SystemClock.elapsedRealtime() - requestTask.startTimeMs)); + keyResponse + .loadEventInfo + .buildUpon() + .setLoadTaskId(requestTask.taskId) + .setLoadDurationMs(SystemClock.elapsedRealtime() - requestTask.startTimeMs) + .build()); } } break; @@ -730,14 +733,13 @@ private boolean maybeRetryRequest(Message originalMsg, MediaDrmCallbackException return false; } LoadEventInfo loadEventInfo = - new LoadEventInfo( - requestTask.taskId, - exception.dataSpec, - exception.uriAfterRedirects, - exception.responseHeaders, - SystemClock.elapsedRealtime(), - /* loadDurationMs= */ SystemClock.elapsedRealtime() - requestTask.startTimeMs, - exception.bytesLoaded); + new LoadEventInfo.Builder( + requestTask.taskId, exception.dataSpec, SystemClock.elapsedRealtime()) + .setUri(exception.uriAfterRedirects) + .setResponseHeaders(exception.responseHeaders) + .setLoadDurationMs(SystemClock.elapsedRealtime() - requestTask.startTimeMs) + .setBytesLoaded(exception.bytesLoaded) + .build(); MediaLoadData mediaLoadData = new MediaLoadData(C.DATA_TYPE_DRM); IOException loadErrorCause = exception.getCause() instanceof IOException diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/drm/DrmSessionEventListener.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/drm/DrmSessionEventListener.java index 1b8987761d3..edb2ed0150f 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/drm/DrmSessionEventListener.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/drm/DrmSessionEventListener.java @@ -98,7 +98,7 @@ default void onDrmKeysRemoved(int windowIndex, @Nullable MediaPeriodId mediaPeri default void onDrmSessionReleased(int windowIndex, @Nullable MediaPeriodId mediaPeriodId) {} /** Dispatches events to {@link DrmSessionEventListener DrmSessionEventListeners}. */ - class EventDispatcher { + final class EventDispatcher { /** The timeline window index reported with the events. */ public final int windowIndex; diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/drm/DrmUtil.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/drm/DrmUtil.java index 3b3430b013f..df2b696762d 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/drm/DrmUtil.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/drm/DrmUtil.java @@ -187,14 +187,14 @@ public static MediaDrmCallback.Response executePost( try { byte[] response = ByteStreams.toByteArray(inputStream); LoadEventInfo loadEventInfo = - new LoadEventInfo( - -1, // This will be replaced with the actual taskId from the request. - originalDataSpec, - statsDataSource.getLastOpenedUri(), - statsDataSource.getLastResponseHeaders(), - SystemClock.elapsedRealtime(), - /* loadDurationMs= */ 0, - response.length); + new LoadEventInfo.Builder( + -1, // This will be replaced with the actual taskId from the request. + originalDataSpec, + SystemClock.elapsedRealtime()) + .setUri(statsDataSource.getLastOpenedUri()) + .setResponseHeaders(statsDataSource.getLastResponseHeaders()) + .setBytesLoaded(response.length) + .build(); return new MediaDrmCallback.Response.Builder(response) .setLoadEventInfo(loadEventInfo) .build(); diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/AsynchronousMediaCodecAdapter.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/AsynchronousMediaCodecAdapter.java index 7ce18a39582..43b6c26395f 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/AsynchronousMediaCodecAdapter.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/AsynchronousMediaCodecAdapter.java @@ -33,7 +33,7 @@ import androidx.annotation.RequiresApi; import androidx.annotation.VisibleForTesting; import androidx.media3.common.C; -import androidx.media3.common.util.ExperimentalApi; +import androidx.media3.common.util.ThrowingRunnable; import androidx.media3.common.util.TraceUtil; import androidx.media3.decoder.CryptoInfo; import com.google.common.base.Supplier; @@ -58,7 +58,6 @@ public static final class Factory implements MediaCodecAdapter.Factory { private final Supplier queueingThreadSupplier; private boolean enableSynchronousBufferQueueingWithAsyncCryptoFlag; - private boolean enableAsyncCryptoSynchronization; /** * Creates an factory for {@link AsynchronousMediaCodecAdapter} instances. @@ -98,22 +97,6 @@ public void setAsyncCryptoFlagEnabled(boolean enableAsyncCryptoFlag) { enableSynchronousBufferQueueingWithAsyncCryptoFlag = enableAsyncCryptoFlag; } - /** - * Sets whether to force synchronization for queuing input buffers on API 31 and above for - * {@link AsynchronousMediaCodecAdapter} instances. - * - *

    A known bug in the Android framework (b/149908061) prior to API 31 can cause garbled video - * when audio and video are sharing the same DRM session. A workaround was implemented that - * forces synchronization for queuing input buffers. This workaround is disabled for devices - * with API level >= 31 but can be enabled using this method. - * - *

    The default is {@code false}. - */ - @ExperimentalApi // TODO: b/502930657 - Remove this method. - public void setAsyncCryptoSynchronizationEnabled(boolean enableAsyncCryptoSynchronization) { - this.enableAsyncCryptoSynchronization = enableAsyncCryptoSynchronization; - } - @Override public AsynchronousMediaCodecAdapter createAdapter(Configuration configuration) throws IOException { @@ -131,8 +114,7 @@ && useSynchronousBufferQueueingWithAsyncCryptoFlag()) { flags |= MediaCodec.CONFIGURE_FLAG_USE_CRYPTO_ASYNC; } else { bufferEnqueuer = - new AsynchronousMediaCodecBufferEnqueuer( - codec, queueingThreadSupplier.get(), enableAsyncCryptoSynchronization); + new AsynchronousMediaCodecBufferEnqueuer(codec, queueingThreadSupplier.get()); } codecAdapter = new AsynchronousMediaCodecAdapter( @@ -266,11 +248,11 @@ public ByteBuffer getInputBuffer(int index) { } @Override - public void useInputBuffer(Runnable runnable) { - asynchronousMediaCodecCallback.useInputBuffer( + public void useBuffer(ThrowingRunnable runnable) throws E { + asynchronousMediaCodecCallback.useBuffer( () -> { bufferEnqueuer.maybeThrowException(); - asynchronousMediaCodecCallback.useInputBuffer(runnable); + runnable.run(); }); } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/AsynchronousMediaCodecBufferEnqueuer.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/AsynchronousMediaCodecBufferEnqueuer.java index 618cb214f20..1eabca635c7 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/AsynchronousMediaCodecBufferEnqueuer.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/AsynchronousMediaCodecBufferEnqueuer.java @@ -57,7 +57,6 @@ private @MonotonicNonNull Handler handler; private final AtomicReference<@NullableType RuntimeException> pendingRuntimeException; private final ConditionVariable conditionVariable; - private final boolean asyncCryptoSynchronizationEnabled; private boolean started; /** @@ -66,25 +65,16 @@ * @param codec The {@link MediaCodec} to submit input buffers to. * @param queueingThread The {@link HandlerThread} to use for queueing buffers. */ - public AsynchronousMediaCodecBufferEnqueuer( - MediaCodec codec, HandlerThread queueingThread, boolean enableAsyncCryptoSynchronization) { - this( - codec, - queueingThread, - /* conditionVariable= */ new ConditionVariable(), - enableAsyncCryptoSynchronization); + public AsynchronousMediaCodecBufferEnqueuer(MediaCodec codec, HandlerThread queueingThread) { + this(codec, queueingThread, /* conditionVariable= */ new ConditionVariable()); } @VisibleForTesting /* package */ AsynchronousMediaCodecBufferEnqueuer( - MediaCodec codec, - HandlerThread handlerThread, - ConditionVariable conditionVariable, - boolean enableAsyncCryptoSynchronization) { + MediaCodec codec, HandlerThread handlerThread, ConditionVariable conditionVariable) { this.codec = codec; this.handlerThread = handlerThread; this.conditionVariable = conditionVariable; - this.asyncCryptoSynchronizationEnabled = enableAsyncCryptoSynchronization; pendingRuntimeException = new AtomicReference<>(); } @@ -237,7 +227,7 @@ private void doQueueSecureInputBuffer( // Below 31, synchronize calls to MediaCodec.queueSecureInputBuffer() to avoid race conditions // inside the crypto module when audio and video are sharing the same DRM session // (see [Internal: b/149908061]). - if (SDK_INT >= 31 && !asyncCryptoSynchronizationEnabled) { + if (SDK_INT >= 31) { codec.queueSecureInputBuffer(index, offset, info, presentationTimeUs, flags); } else { synchronized (QUEUE_SECURE_LOCK) { @@ -310,12 +300,9 @@ private static void copy( frameworkCryptoInfo.key = checkNotNull(copy(cryptoInfo.key, frameworkCryptoInfo.key)); frameworkCryptoInfo.iv = checkNotNull(copy(cryptoInfo.iv, frameworkCryptoInfo.iv)); frameworkCryptoInfo.mode = cryptoInfo.mode; - if (SDK_INT >= 24) { - android.media.MediaCodec.CryptoInfo.Pattern pattern = - new android.media.MediaCodec.CryptoInfo.Pattern( - cryptoInfo.encryptedBlocks, cryptoInfo.clearBlocks); - frameworkCryptoInfo.setPattern(pattern); - } + MediaCodec.CryptoInfo.Pattern pattern = + new MediaCodec.CryptoInfo.Pattern(cryptoInfo.encryptedBlocks, cryptoInfo.clearBlocks); + frameworkCryptoInfo.setPattern(pattern); } /** diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/AsynchronousMediaCodecCallback.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/AsynchronousMediaCodecCallback.java index 8055b2ac1d6..ae201d3e3cb 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/AsynchronousMediaCodecCallback.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/AsynchronousMediaCodecCallback.java @@ -26,6 +26,7 @@ import androidx.annotation.GuardedBy; import androidx.annotation.Nullable; import androidx.media3.common.util.CircularIntArray; +import androidx.media3.common.util.ThrowingRunnable; import androidx.media3.common.util.Util; import java.util.ArrayDeque; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; @@ -128,14 +129,17 @@ public void shutdown() { } /** - * Uses an acquired input buffer (e.g. to write data to it). + * Uses an acquired input or output buffer (e.g. to populate, drain, or write data to it). * - *

    This can be used for error handling, e.g. to guarantee no operation can invalidate the - * buffer while it's being used. + *

    This runs the provided {@link ThrowingRunnable} under the internal codec/callback monitor, + * ensuring that concurrent asynchronous framework callbacks (e.g., error reporting or pipeline + * invalidation) cannot pre-empt or interleave with active {@code ByteBuffer} memory traversals. * - * @param runnable The {@link Runnable} using an acquired input buffer. + * @param runnable The {@link ThrowingRunnable} utilizing an acquired buffer. + * @param The type of exception thrown by {@code runnable}. + * @throws E If {@code runnable} throws an exception. */ - public void useInputBuffer(Runnable runnable) { + public void useBuffer(ThrowingRunnable runnable) throws E { synchronized (lock) { // Run the Runnable under the buffer lock to ensure no new error callback can be handled // while the buffer is used (under the assumption that error callbacks invalidate the buffer). diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/DefaultMediaCodecAdapterFactory.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/DefaultMediaCodecAdapterFactory.java index a4f3612eb00..560b533f9ba 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/DefaultMediaCodecAdapterFactory.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/DefaultMediaCodecAdapterFactory.java @@ -24,7 +24,6 @@ import androidx.annotation.IntDef; import androidx.annotation.Nullable; import androidx.media3.common.MimeTypes; -import androidx.media3.common.util.ExperimentalApi; import androidx.media3.common.util.Log; import androidx.media3.common.util.UnstableApi; import androidx.media3.common.util.Util; @@ -66,7 +65,6 @@ public final class DefaultMediaCodecAdapterFactory implements MediaCodecAdapter. private @Mode int asynchronousMode; private boolean asyncCryptoFlagEnabled; - private boolean asyncCryptoSynchronizationEnabled; /** * @deprecated Use {@link #DefaultMediaCodecAdapterFactory(Context)} instead. @@ -143,27 +141,6 @@ public DefaultMediaCodecAdapterFactory setAsyncCryptoFlagEnabled(boolean enableA return this; } - /** - * Sets whether to force synchronization for queuing input buffers on API 31 and above for {@link - * AsynchronousMediaCodecAdapter} instances. - * - *

    A known bug in the Android framework (b/149908061) prior to API 31 can cause garbled video - * when audio and video are sharing the same DRM session. A workaround was implemented that forces - * synchronization for queuing input buffers. This workaround is disabled for devices with API - * level >= 31 but can be enabled using this method. - * - *

    The default is {@code false}. - * - * @return This factory, for convenience. - */ - @CanIgnoreReturnValue - @ExperimentalApi // TODO: b/502930657 - Remove this method. - public DefaultMediaCodecAdapterFactory setAsyncCryptoSynchronizationEnabled( - boolean enableAsyncCryptoSynchronization) { - asyncCryptoSynchronizationEnabled = enableAsyncCryptoSynchronization; - return this; - } - @Override public MediaCodecAdapter createAdapter(MediaCodecAdapter.Configuration configuration) throws IOException { @@ -180,7 +157,6 @@ public MediaCodecAdapter createAdapter(MediaCodecAdapter.Configuration configura callbackThreadSupplier, queueingThreadSupplier) : new AsynchronousMediaCodecAdapter.Factory(trackType); factory.setAsyncCryptoFlagEnabled(asyncCryptoFlagEnabled); - factory.setAsyncCryptoSynchronizationEnabled(asyncCryptoSynchronizationEnabled); return factory.createAdapter(configuration); } return new SynchronousMediaCodecAdapter.Factory().createAdapter(configuration); diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/ForwardingMediaCodecAdapter.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/ForwardingMediaCodecAdapter.java index 483d9a4c1a4..919e79546ee 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/ForwardingMediaCodecAdapter.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/ForwardingMediaCodecAdapter.java @@ -23,6 +23,7 @@ import android.view.Surface; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; +import androidx.media3.common.util.ThrowingRunnable; import androidx.media3.common.util.UnstableApi; import androidx.media3.decoder.CryptoInfo; import java.nio.ByteBuffer; @@ -65,8 +66,8 @@ public ByteBuffer getInputBuffer(int index) { } @Override - public void useInputBuffer(Runnable runnable) { - delegate.useInputBuffer(runnable); + public void useBuffer(ThrowingRunnable runnable) throws E { + delegate.useBuffer(runnable); } @Nullable diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/MediaCodecAdapter.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/MediaCodecAdapter.java index cda3b77483f..7dbf562a983 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/MediaCodecAdapter.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/MediaCodecAdapter.java @@ -27,6 +27,7 @@ import androidx.annotation.RequiresApi; import androidx.media3.common.C; import androidx.media3.common.Format; +import androidx.media3.common.util.ThrowingRunnable; import androidx.media3.common.util.UnstableApi; import androidx.media3.decoder.CryptoInfo; import java.io.IOException; @@ -208,14 +209,16 @@ default void onOutputBufferAvailable() {} ByteBuffer getInputBuffer(int index); /** - * Uses an acquired input buffer (e.g. to write data to it). + * Uses an acquired input or output buffer (e.g. to read from or write data to it). * *

    This can be used for error handling, e.g. to guarantee no operation can invalidate the - * buffer while it's being used. + * buffer while it's being used by the application or downstream components. * - * @param runnable The {@link Runnable} using an acquired input buffer. + * @param runnable The {@link ThrowingRunnable} utilizing an acquired buffer. + * @param The type of exception thrown by {@code runnable}. + * @throws E If {@code runnable} throws an exception. */ - default void useInputBuffer(Runnable runnable) { + default void useBuffer(ThrowingRunnable runnable) throws E { runnable.run(); } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/MediaCodecInfo.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/MediaCodecInfo.java index ccdc4a965d7..16845413b9c 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/MediaCodecInfo.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/MediaCodecInfo.java @@ -54,10 +54,15 @@ import androidx.media3.common.util.CodecSpecificDataUtil; import androidx.media3.common.util.CodecSpecificDataUtil.MediaCodecProfileAndLevel; import androidx.media3.common.util.Log; +import androidx.media3.common.util.NullableType; import androidx.media3.common.util.UnstableApi; import androidx.media3.common.util.Util; import androidx.media3.exoplayer.DecoderReuseEvaluation; import androidx.media3.exoplayer.DecoderReuseEvaluation.DecoderDiscardReasons; +import com.google.common.base.Strings; +import com.google.common.base.Supplier; +import com.google.common.base.Suppliers; +import com.google.common.primitives.Ints; import com.google.errorprone.annotations.InlineMe; import java.util.Objects; @@ -74,6 +79,10 @@ public final class MediaCodecInfo { */ public static final int MAX_SUPPORTED_INSTANCES_UNKNOWN = -1; + private static final Supplier<@NullableType Integer> VENDOR_API_LEVEL = + Suppliers.<@NullableType Integer>memoize( + () -> Ints.tryParse(Strings.nullToEmpty(Util.getSystemProperty("ro.vendor.api_level")))); + /** * The name of the decoder. * @@ -334,6 +343,11 @@ private boolean isCodecProfileAndLevelSupported( } if (codecProfileAndLevel == null) { + if (Objects.equals(format.sampleMimeType, MimeTypes.VIDEO_DOLBY_VISION)) { + // Do not assume unknown Dolby Vision profiles are supported, as decoders only support + // specific profiles. + return false; + } // If we don't know any better, we assume that the profile and level are supported. return true; } @@ -381,11 +395,6 @@ private boolean isCodecProfileAndLevelSupported( // in the codec capabilities. profileLevels = estimateLegacyAc4ProfileLevels(context, capabilities); } - if (SDK_INT == 23 && MimeTypes.VIDEO_VP9.equals(mimeType) && profileLevels.length == 0) { - // Some older devices don't report profile levels for VP9. Estimate them using other data in - // the codec capabilities. - profileLevels = estimateLegacyVp9ProfileLevels(capabilities); - } for (CodecProfileLevel profileLevel : profileLevels) { if (profileLevel.profile == profile @@ -902,9 +911,6 @@ private static boolean areSizeAndRateSupported( if (!capabilities.areSizeAndRateSupported(width, height, floorFrameRate)) { return false; } - if (SDK_INT < 24) { - return true; - } @Nullable Range achievableFrameRates = capabilities.getAchievableFrameRatesFor(width, height); if (achievableFrameRates == null) { @@ -956,56 +962,6 @@ private static CodecProfileLevel[] estimateLegacyAc4ProfileLevels( }; } - /** - * Called on devices with {@link Build.VERSION#SDK_INT} 23 and below, for VP9 decoders whose - * {@link CodecCapabilities} do not correctly report profile levels. The returned {@link - * CodecProfileLevel CodecProfileLevels} are estimated based on other data in the {@link - * CodecCapabilities}. - * - * @param capabilities The {@link CodecCapabilities} for a VP9 decoder, or {@code null} if not - * known. - * @return The estimated {@link CodecProfileLevel CodecProfileLevels} for the decoder. - */ - private static CodecProfileLevel[] estimateLegacyVp9ProfileLevels( - @Nullable CodecCapabilities capabilities) { - int maxBitrate = 0; - if (capabilities != null) { - @Nullable VideoCapabilities videoCapabilities = capabilities.getVideoCapabilities(); - if (videoCapabilities != null) { - maxBitrate = videoCapabilities.getBitrateRange().getUpper(); - } - } - - // Values taken from https://www.webmproject.org/vp9/levels. - int level; - if (maxBitrate >= 180_000_000) { - level = CodecProfileLevel.VP9Level52; - } else if (maxBitrate >= 120_000_000) { - level = CodecProfileLevel.VP9Level51; - } else if (maxBitrate >= 60_000_000) { - level = CodecProfileLevel.VP9Level5; - } else if (maxBitrate >= 30_000_000) { - level = CodecProfileLevel.VP9Level41; - } else if (maxBitrate >= 18_000_000) { - level = CodecProfileLevel.VP9Level4; - } else if (maxBitrate >= 12_000_000) { - level = CodecProfileLevel.VP9Level31; - } else if (maxBitrate >= 7_200_000) { - level = CodecProfileLevel.VP9Level3; - } else if (maxBitrate >= 3_600_000) { - level = CodecProfileLevel.VP9Level21; - } else if (maxBitrate >= 1_800_000) { - level = CodecProfileLevel.VP9Level2; - } else if (maxBitrate >= 800_000) { - level = CodecProfileLevel.VP9Level11; - } else { // Assume level 1 is always supported. - level = CodecProfileLevel.VP9Level1; - } - - // Since this method is for legacy devices only, assume that only profile 0 is supported. - return new CodecProfileLevel[] {createCodecProfileLevel(CodecProfileLevel.VP9Profile0, level)}; - } - /** * Returns whether the decoder is known to fail when an attempt is made to reconfigure it with a * new format's configuration data. @@ -1072,13 +1028,12 @@ private static boolean needsProfileExcludedWorkaround(String mimeType, int profi /** Returns whether the device is known to have issues with the detached surface mode. */ private static boolean needsDetachedSurfaceUnsupportedWorkaround() { - if (!MediaLibraryInfo.enableWorkarounds()) { - return false; - } - if (SDK_INT > 37) { - // After API 37, we assume that detached surface mode is supported and we don't apply the - // workaround. Devices should not declare support for FEATURE_DetachedSurface unless they - // actually support it. + // We deliberately don't check MediaLibraryInfo.enableWorkarounds() here, because this + // workaround needs to remain active even during certification testing (b/535214648). + Integer vendorApiLevel = VENDOR_API_LEVEL.get(); + if (vendorApiLevel == null || vendorApiLevel > 202604) { + // Devices with vendor API level higher than SDK_INT 37 must only declare support for + // FEATURE_DetachedSurface if they actually support it. return false; } return Build.MANUFACTURER.equals("Xiaomi") diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/MediaCodecRenderer.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/MediaCodecRenderer.java index cd731fd2d0f..762a835511b 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/MediaCodecRenderer.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/MediaCodecRenderer.java @@ -47,6 +47,7 @@ import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.Format; import androidx.media3.common.MediaLibraryInfo; import androidx.media3.common.MimeTypes; @@ -299,32 +300,11 @@ private static String buildCustomDiagnosticInfo(int errorCode) { /** The codec should be reinitialized. */ private static final int DRAIN_ACTION_REINITIALIZE = 3; - @Documented - @Retention(RetentionPolicy.SOURCE) - @Target(TYPE_USE) - @IntDef({ - ADAPTATION_WORKAROUND_MODE_NEVER, - ADAPTATION_WORKAROUND_MODE_SAME_RESOLUTION, - ADAPTATION_WORKAROUND_MODE_ALWAYS - }) - private @interface AdaptationWorkaroundMode {} - - /** The adaptation workaround is never used. */ - private static final int ADAPTATION_WORKAROUND_MODE_NEVER = 0; - - /** - * The adaptation workaround is used when adapting between formats of the same resolution only. - */ - private static final int ADAPTATION_WORKAROUND_MODE_SAME_RESOLUTION = 1; - - /** The adaptation workaround is always used when adapting between formats. */ - private static final int ADAPTATION_WORKAROUND_MODE_ALWAYS = 2; - /** * H.264/AVC buffer to queue when using the adaptation workaround (see {@link - * #codecAdaptationWorkaroundMode(String)}. Consists of three NAL units with start codes: Baseline - * sequence/picture parameter sets and a 32 * 32 pixel IDR slice. This stream can be queued to - * force a resolution change when adapting to a new format. + * #codecNeedsAdaptationWorkaround(String)}). Consists of three NAL units with start codes: + * Baseline sequence/picture parameter sets and a 32 * 32 pixel IDR slice. This stream can be + * queued to force a resolution change when adapting to a new format. */ private static final byte[] ADAPTATION_WORKAROUND_BUFFER = new byte[] { @@ -378,9 +358,8 @@ private static String buildCustomDiagnosticInfo(int errorCode) { @Nullable private ArrayDeque availableCodecInfos; @Nullable private DecoderInitializationException preferredDecoderInitializationException; @Nullable private MediaCodecInfo codecInfo; - private @AdaptationWorkaroundMode int codecAdaptationWorkaroundMode; + private boolean codecNeedsAdaptationWorkaround; private boolean codecNeedsSosFlushWorkaround; - private boolean codecNeedsEosFlushWorkaround; private boolean codecNeedsAdaptationWorkaroundBuffer; private boolean shouldSkipAdaptationWorkaroundOutputBuffer; private boolean codecNeedsEosPropagation; @@ -391,6 +370,7 @@ private static String buildCustomDiagnosticInfo(int errorCode) { private int outputIndex; @Nullable private ByteBuffer outputBuffer; private boolean isDecodeOnlyOutputBuffer; + private boolean isOutputBufferStale; private boolean bypassEnabled; private boolean bypassSampleBufferPending; private boolean bypassDrainAndReinitialize; @@ -412,9 +392,9 @@ private static String buildCustomDiagnosticInfo(int errorCode) { private OutputStreamInfo outputStreamInfo; private long lastProcessedOutputBufferTimeUs; private boolean needToNotifyOutputFormatChangeAfterStreamChange; - private boolean experimentalEnableProcessedStreamChangedAtStart; private boolean hasSkippedFlushAndWaitingForQueueInputBuffer; private long skippedFlushOffsetUs; + private long largestStaleModifiedPresentationTimeUs; private CodecParameters activeCodecParameters; private CodecParameters lastDispatchedCodecParameters; private ImmutableSet subscribedCodecParameterKeys; @@ -465,7 +445,6 @@ public MediaCodecRenderer( oggOpusAudioPacketizer = new OggOpusAudioPacketizer(); codecOperatingRate = CODEC_OPERATING_RATE_UNSET; - codecAdaptationWorkaroundMode = ADAPTATION_WORKAROUND_MODE_NEVER; codecReconfigurationState = RECONFIGURATION_STATE_NONE; inputIndex = C.INDEX_UNSET; outputIndex = C.INDEX_UNSET; @@ -479,6 +458,7 @@ public MediaCodecRenderer( decoderCounters = new DecoderCounters(); hasSkippedFlushAndWaitingForQueueInputBuffer = false; skippedFlushOffsetUs = 0; + largestStaleModifiedPresentationTimeUs = C.TIME_UNSET; this.subscribedCodecParameterKeys = ImmutableSet.of(); this.activeCodecParameters = CodecParameters.EMPTY; this.lastDispatchedCodecParameters = CodecParameters.EMPTY; @@ -530,17 +510,6 @@ public final long getDurationToProgressUs(long positionUs, long elapsedRealtimeU /* isOnBufferAvailableListenerRegistered= */ codecRegisteredOnBufferAvailableListener); } - /** - * Enables the renderer to invoke {@link #onProcessedStreamChange()} on the first stream. - * - *

    When not enabled, {@link #onProcessedStreamChange()} is invoked from the second stream - * onwards. - */ - @ExperimentalApi // TODO: b/470373575 - Enable this feature by default. - public void experimentalEnableProcessedStreamChangedAtStart() { - this.experimentalEnableProcessedStreamChangedAtStart = true; - } - /** * Returns minimum time playback must advance in order for the {@link #render} call to make * progress. @@ -773,7 +742,8 @@ protected void onStreamChanged( offsetUs, durationUs, streamFlags)); - if (experimentalEnableProcessedStreamChangedAtStart) { + if (shouldProcessStreamChangeAtStart() + || Flags.isEnabled(Flags.FLAG_PROCESSED_STREAM_CHANGED_AT_START)) { onProcessedStreamChange(); } } else if (pendingOutputStreamChanges.isEmpty() @@ -1057,9 +1027,7 @@ private boolean flushOrReleaseCodec() { */ protected boolean shouldReleaseCodecInsteadOfFlushing() { if (codecDrainAction == DRAIN_ACTION_REINITIALIZE - || (codecNeedsSosFlushWorkaround && !codecHasOutputMediaFormat) - || (codecNeedsEosFlushWorkaround - && (codecReceivedEos || codecDrainState == DRAIN_STATE_SIGNAL_END_OF_STREAM))) { + || (codecNeedsSosFlushWorkaround && !codecHasOutputMediaFormat)) { return true; } if (codecDrainAction == DRAIN_ACTION_FLUSH_AND_UPDATE_DRM_SESSION) { @@ -1127,6 +1095,7 @@ protected void resetCodecStateForFlush() { codecReceivedBuffers = false; codecNeedsAdaptationWorkaroundBuffer = false; shouldSkipAdaptationWorkaroundOutputBuffer = false; + isOutputBufferStale = false; isDecodeOnlyOutputBuffer = false; codecDrainState = DRAIN_STATE_NONE; codecDrainAction = DRAIN_ACTION_NONE; @@ -1137,6 +1106,7 @@ protected void resetCodecStateForFlush() { codecReconfigured ? RECONFIGURATION_STATE_WRITE_PENDING : RECONFIGURATION_STATE_NONE; hasSkippedFlushAndWaitingForQueueInputBuffer = false; skippedFlushOffsetUs = 0; + largestStaleModifiedPresentationTimeUs = C.TIME_UNSET; } /** @@ -1157,9 +1127,8 @@ protected void resetCodecStateForRelease() { codecOutputMediaFormatChanged = false; codecHasOutputMediaFormat = false; codecOperatingRate = CODEC_OPERATING_RATE_UNSET; - codecAdaptationWorkaroundMode = ADAPTATION_WORKAROUND_MODE_NEVER; + codecNeedsAdaptationWorkaround = false; codecNeedsSosFlushWorkaround = false; - codecNeedsEosFlushWorkaround = false; codecNeedsEosPropagation = false; codecRegisteredOnBufferAvailableListener = false; codecReconfigured = false; @@ -1435,15 +1404,13 @@ private void initCodec(MediaCodecInfo codecInfo, @Nullable MediaCrypto crypto) t this.codecOperatingRate = codecOperatingRate; codecInputFormat = inputFormat; - codecAdaptationWorkaroundMode = codecAdaptationWorkaroundMode(codecName); + codecNeedsAdaptationWorkaround = codecNeedsAdaptationWorkaround(codecName); codecNeedsSosFlushWorkaround = codecNeedsSosFlushWorkaround(codecName); - codecNeedsEosFlushWorkaround = codecNeedsEosFlushWorkaround(codecName); codecNeedsEosPropagation = codecNeedsEosPropagationWorkaround(codecInfo); if (checkNotNull(codec).needsReconfiguration()) { this.codecReconfigured = true; this.codecReconfigurationState = RECONFIGURATION_STATE_WRITE_PENDING; - this.codecNeedsAdaptationWorkaroundBuffer = - codecAdaptationWorkaroundMode != ADAPTATION_WORKAROUND_MODE_NEVER; + this.codecNeedsAdaptationWorkaroundBuffer = codecNeedsAdaptationWorkaround; } if (getState() == STATE_STARTED) { @@ -1557,7 +1524,7 @@ private boolean feedInputBuffer() throws ExoPlaybackException { FormatHolder formatHolder = getFormatHolder(); try { - codec.useInputBuffer( + codec.useBuffer( () -> readDataResultHolder.set(readSource(formatHolder, buffer, /* readFlags= */ 0))); } catch (InsufficientCapacityException e) { onCodecError(e); @@ -1659,6 +1626,18 @@ private boolean feedInputBuffer() throws ExoPlaybackException { lastStreamInfo.queuedBufferAfterReset = true; waitingForFirstSampleInFormat = false; } + + if (hasSkippedFlushAndWaitingForQueueInputBuffer) { + largestStaleModifiedPresentationTimeUs = + largestQueuedPresentationTimeUs + skippedFlushOffsetUs; + if (presentationTimeUs <= largestQueuedPresentationTimeUs) { + skippedFlushOffsetUs += largestQueuedPresentationTimeUs - presentationTimeUs + 1; + } + largestQueuedPresentationTimeUs = presentationTimeUs; + largestQueuedPresentationTimeWithinDurationUs = presentationTimeUs; + hasSkippedFlushAndWaitingForQueueInputBuffer = false; + } + largestQueuedPresentationTimeUs = max(largestQueuedPresentationTimeUs, presentationTimeUs); long streamEndPositionUs = getStreamEndPositionUs(); if (streamEndPositionUs == C.TIME_UNSET @@ -1675,15 +1654,6 @@ private boolean feedInputBuffer() throws ExoPlaybackException { handleInputBufferSupplementalData(buffer); } - if (hasSkippedFlushAndWaitingForQueueInputBuffer) { - if (presentationTimeUs <= largestQueuedPresentationTimeUs) { - skippedFlushOffsetUs += largestQueuedPresentationTimeUs - presentationTimeUs + 1; - } - largestQueuedPresentationTimeUs = presentationTimeUs; - largestQueuedPresentationTimeWithinDurationUs = presentationTimeUs; - hasSkippedFlushAndWaitingForQueueInputBuffer = false; - } - onQueueInputBuffer(buffer); int flags = getCodecBufferFlags(buffer); presentationTimeUs += skippedFlushOffsetUs; @@ -1809,13 +1779,16 @@ protected DecoderReuseEvaluation onInputFormatChanged(FormatHolder formatHolder) PlaybackException.ERROR_CODE_DECODING_FORMAT_UNSUPPORTED); } - // Remove the initialization data (CSD) when decoding AV1 or VP9, as it is not required by - // MediaCodec for these encodings, and some codec implementations fail if it's present. + // Remove the initialization data (CSD) when decoding AV1, VP9, E-AC-3, or E-AC-3 JOC, as + // it is not required by MediaCodec for these encodings, and some codec implementations fail + // if it's present. // * https://developer.android.com/reference/android/media/MediaCodec#CSD // * b/229399008#comment9 // * https://github.com/androidx/media/issues/2408 if ((Objects.equals(newFormat.sampleMimeType, MimeTypes.VIDEO_AV1) || Objects.equals(newFormat.sampleMimeType, MimeTypes.VIDEO_VP9) + || Objects.equals(newFormat.sampleMimeType, MimeTypes.AUDIO_E_AC3) + || Objects.equals(newFormat.sampleMimeType, MimeTypes.AUDIO_E_AC3_JOC) || (Objects.equals(newFormat.sampleMimeType, MimeTypes.VIDEO_DOLBY_VISION) && Objects.equals( CodecSpecificDataUtil.getDolbyVisionBaseLayerMimeType(newFormat), @@ -1885,11 +1858,7 @@ protected DecoderReuseEvaluation onInputFormatChanged(FormatHolder formatHolder) updateCodecOperatingRate(newFormat); codecReconfigured = true; codecReconfigurationState = RECONFIGURATION_STATE_WRITE_PENDING; - codecNeedsAdaptationWorkaroundBuffer = - codecAdaptationWorkaroundMode == ADAPTATION_WORKAROUND_MODE_ALWAYS - || (codecAdaptationWorkaroundMode == ADAPTATION_WORKAROUND_MODE_SAME_RESOLUTION - && newFormat.width == oldFormat.width - && newFormat.height == oldFormat.height); + codecNeedsAdaptationWorkaroundBuffer = codecNeedsAdaptationWorkaround; codecInputFormat = newFormat; if (drainAndUpdateCodecDrmSession && !drainAndUpdateCodecDrmSession()) { overridingDiscardReasons |= DISCARD_REASON_WORKAROUND; @@ -2043,6 +2012,17 @@ protected void onProcessedStreamChange() { // Do nothing. } + /** + * Returns whether {@link #onProcessedStreamChange()} should be invoked on the first stream. + * + *

    Subclasses can override this method if they need stream change processing for the initial + * stream even when {@linkplain Flags#FLAG_PROCESSED_STREAM_CHANGED_AT_START} is disabled. + */ + @ExperimentalApi // TODO: b/470373575 - Remove this method. + protected boolean shouldProcessStreamChangeAtStart() { + return false; + } + /** * Evaluates whether the existing {@link MediaCodec} can be kept for a new {@link Format}, and if * it can, whether it requires reconfiguration. @@ -2264,6 +2244,9 @@ private boolean drainOutputBuffer(long positionUs, long elapsedRealtimeUs) } // We've dequeued a buffer. + isOutputBufferStale = + largestStaleModifiedPresentationTimeUs != C.TIME_UNSET + && outputBufferInfo.presentationTimeUs <= largestStaleModifiedPresentationTimeUs; outputBufferInfo.presentationTimeUs -= skippedFlushOffsetUs; if (shouldSkipAdaptationWorkaroundOutputBuffer) { shouldSkipAdaptationWorkaroundOutputBuffer = false; @@ -2301,7 +2284,8 @@ private boolean drainOutputBuffer(long positionUs, long elapsedRealtimeUs) && outputBufferInfo.presentationTimeUs - getOutputStreamOffsetUs() >= outputStreamInfo.durationUs; isDecodeOnlyOutputBuffer = - hasSkippedFlushAndWaitingForQueueInputBuffer + isOutputBufferStale + || hasSkippedFlushAndWaitingForQueueInputBuffer || outputBufferInfo.presentationTimeUs < getLastResetPositionUs() || isStrictDurationExceeded; boolean isLastOutputBuffer = @@ -2342,7 +2326,7 @@ private boolean drainOutputBuffer(long positionUs, long elapsedRealtimeUs) private void processOutputMediaFormatChanged() { codecHasOutputMediaFormat = true; MediaFormat mediaFormat = checkNotNull(codec).getOutputFormat(); - if (codecAdaptationWorkaroundMode != ADAPTATION_WORKAROUND_MODE_NEVER + if (codecNeedsAdaptationWorkaround && mediaFormat.getInteger(MediaFormat.KEY_WIDTH) == ADAPTATION_WORKAROUND_SLICE_WIDTH_HEIGHT && mediaFormat.getInteger(MediaFormat.KEY_HEIGHT) == ADAPTATION_WORKAROUND_SLICE_WIDTH_HEIGHT) { @@ -2826,7 +2810,7 @@ private static boolean isMediaCodecException(IllegalStateException error) { } /** - * Returns a mode that specifies when the adaptation workaround should be enabled. + * Returns whether the adaptation workaround should be enabled. * *

    When enabled, the workaround queues and discards a blank frame with a resolution whose width * and height both equal {@link #ADAPTATION_WORKAROUND_SLICE_WIDTH_HEIGHT}, to reset the decoder's @@ -2836,29 +2820,18 @@ private static boolean isMediaCodecException(IllegalStateException error) { * href="https://github.com/google/ExoPlayer/issues/3257">GitHub issue #3257. * * @param name The name of the decoder. - * @return The mode specifying when the adaptation workaround should be enabled. + * @return Whether the adaptation workaround should be enabled. */ - private @AdaptationWorkaroundMode int codecAdaptationWorkaroundMode(String name) { + private static boolean codecNeedsAdaptationWorkaround(String name) { if (!MediaLibraryInfo.enableWorkarounds()) { - return ADAPTATION_WORKAROUND_MODE_NEVER; + return false; } - if (SDK_INT <= 25 + return SDK_INT <= 25 && "OMX.Exynos.avc.dec.secure".equals(name) && (Build.MODEL.startsWith("SM-T585") || Build.MODEL.startsWith("SM-A510") || Build.MODEL.startsWith("SM-A520") - || Build.MODEL.startsWith("SM-J700"))) { - return ADAPTATION_WORKAROUND_MODE_ALWAYS; - } else if (SDK_INT < 24 - && ("OMX.Nvidia.h264.decode".equals(name) || "OMX.Nvidia.h264.decode.secure".equals(name)) - && ("flounder".equals(Build.DEVICE) - || "flounder_lte".equals(Build.DEVICE) - || "grouper".equals(Build.DEVICE) - || "tilapia".equals(Build.DEVICE))) { - return ADAPTATION_WORKAROUND_MODE_SAME_RESOLUTION; - } else { - return ADAPTATION_WORKAROUND_MODE_NEVER; - } + || Build.MODEL.startsWith("SM-J700")); } /** @@ -2911,26 +2884,6 @@ private static boolean codecNeedsEosPropagationWorkaround(MediaCodecInfo codecIn || ("Amazon".equals(Build.MANUFACTURER) && "AFTS".equals(Build.MODEL) && codecInfo.secure); } - /** - * Returns whether the decoder is known to behave incorrectly if flushed after receiving an input - * buffer with {@link MediaCodec#BUFFER_FLAG_END_OF_STREAM} set. - * - *

    If true is returned, the renderer will work around the issue by instantiating a new decoder - * when this case occurs. - * - *

    See [Internal: b/8578467, b/23361053]. - * - * @param name The name of the decoder. - * @return True if the decoder is known to behave incorrectly if flushed after receiving an input - * buffer with {@link MediaCodec#BUFFER_FLAG_END_OF_STREAM} set. False otherwise. - */ - private static boolean codecNeedsEosFlushWorkaround(String name) { - if (!MediaLibraryInfo.enableWorkarounds()) { - return false; - } - return SDK_INT == 23 && "OMX.google.vorbis.decoder".equals(name); - } - private static final class OutputStreamInfo { private static final OutputStreamInfo UNSET = diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/MediaCodecUtil.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/MediaCodecUtil.java index 6881de1ecf2..6599f192c51 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/MediaCodecUtil.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/mediacodec/MediaCodecUtil.java @@ -163,20 +163,6 @@ public static synchronized List getDecoderInfos( MediaCodecListCompat mediaCodecList = new MediaCodecListCompatV21(secure, tunneling, specialCodec); ArrayList decoderInfos = getDecoderInfosInternal(key, mediaCodecList); - if (secure && decoderInfos.isEmpty() && SDK_INT == 23) { - // Some devices don't list secure decoders on API level 21 [Internal: b/18678462]. Try the - // legacy path. We also try this path on API levels 22 and 23 as a defensive measure. - mediaCodecList = new MediaCodecListCompatV16(); - decoderInfos = getDecoderInfosInternal(key, mediaCodecList); - if (!decoderInfos.isEmpty()) { - Log.w( - TAG, - "MediaCodecList API didn't list secure decoder for: " - + mimeType - + ". Assuming: " - + decoderInfos.get(0).name); - } - } applyWorkarounds(mimeType, decoderInfos); ImmutableList immutableDecoderInfos = ImmutableList.copyOf(decoderInfos); decoderInfosCache.put(key, immutableDecoderInfos); @@ -222,6 +208,56 @@ public static List getDecoderInfosSoftMatch( .build(); } + /** + * Returns a list of decoders that can decode media in the specified format, in the priority order + * specified by the {@link MediaCodecSelector}. Unlike {@link #getDecoderInfosSoftMatch}, this + * method may exclude decoders that match the sample MIME type but do not support relevant format + * details. + * + *

    This list is more complete than {@link #getDecoderInfos}, as it also considers alternative + * MIME types that are a close match using {@link #getAlternativeCodecMimeType}. + * + * @param context The application context. + * @param mediaCodecSelector The decoder selector. + * @param format The {@link Format} for which a decoder is required. + * @param requiresSecureDecoder Whether a secure decoder is required. + * @param requiresTunnelingDecoder Whether a tunneling decoder is required. + * @return A list of {@link MediaCodecInfo}s corresponding to decoders. May be empty. + * @throws DecoderQueryException Thrown if there was an error querying decoders. + */ + @RequiresNonNull("#3.sampleMimeType") + public static List getDecoderInfosSoftMatchFilteredByFormatSupport( + Context context, + MediaCodecSelector mediaCodecSelector, + Format format, + boolean requiresSecureDecoder, + boolean requiresTunnelingDecoder) + throws DecoderQueryException { + + List decoderInfos = + mediaCodecSelector.getDecoderInfos( + format.sampleMimeType, requiresSecureDecoder, requiresTunnelingDecoder); + + List filteredDecoderInfos = + ImmutableList.builder() + .addAll(filterDecoderInfosByFormatSupport(context, decoderInfos, format)) + .addAll( + getAlternativeDecoderInfosFilteredByFormatSupport( + context, + mediaCodecSelector, + format, + requiresSecureDecoder, + requiresTunnelingDecoder)) + .build(); + + if (!filteredDecoderInfos.isEmpty()) { + return filteredDecoderInfos; + } + + return getDecoderInfosSoftMatch( + mediaCodecSelector, format, requiresSecureDecoder, requiresTunnelingDecoder); + } + /** * Returns a list of decoders for {@linkplain #getAlternativeCodecMimeType alternative MIME types} * that can decode samples of the provided {@link Format}, in the priority order specified by the @@ -254,6 +290,48 @@ public static List getAlternativeDecoderInfos( alternativeMimeType, requiresSecureDecoder, requiresTunnelingDecoder); } + /** + * Returns a list of decoders for {@linkplain #getAlternativeCodecMimeType alternative MIME types} + * that can decode samples of the provided {@link Format}, in the priority order specified by the + * {@link MediaCodecSelector}. Unlike {@link #getAlternativeDecoderInfos}, this method excludes + * decoders that do not support relevant format details. + * + * @param context The application context. + * @param mediaCodecSelector The decoder selector. + * @param format The {@link Format} for which an alternative decoder is required. + * @param requiresSecureDecoder Whether a secure decoder is required. + * @param requiresTunnelingDecoder Whether a tunneling decoder is required. + * @return A list of {@link MediaCodecInfo}s corresponding to alternative decoders. May be empty. + * @throws DecoderQueryException Thrown if there was an error querying decoders. + */ + public static List getAlternativeDecoderInfosFilteredByFormatSupport( + Context context, + MediaCodecSelector mediaCodecSelector, + Format format, + boolean requiresSecureDecoder, + boolean requiresTunnelingDecoder) + throws DecoderQueryException { + @Nullable String alternativeMimeType = getAlternativeCodecMimeType(format); + if (alternativeMimeType == null) { + return ImmutableList.of(); + } + List alternativeDecoderInfos = + mediaCodecSelector.getDecoderInfos( + alternativeMimeType, requiresSecureDecoder, requiresTunnelingDecoder); + return filterDecoderInfosByFormatSupport(context, alternativeDecoderInfos, format); + } + + private static ImmutableList filterDecoderInfosByFormatSupport( + Context context, List decoderInfos, Format format) { + ImmutableList.Builder result = ImmutableList.builder(); + for (MediaCodecInfo decoderInfo : decoderInfos) { + if (decoderInfo.isFormatSupported(context, format)) { + result.add(decoderInfo); + } + } + return result.build(); + } + /** * Returns a copy of the provided decoder list sorted such that decoders with functional format * support are listed first. The returned list is modifiable for convenience. @@ -448,7 +526,6 @@ private static ArrayList getDecoderInfosInternal( ArrayList decoderInfos = new ArrayList<>(); String mimeType = key.mimeType; int numberOfCodecs = mediaCodecList.getCodecCount(); - boolean secureDecodersExplicit = mediaCodecList.secureDecodersExplicit(); // Note: MediaCodecList is sorted by the framework such that the best decoders come first. for (int i = 0; i < numberOfCodecs; i++) { android.media.MediaCodecInfo codecInfo = mediaCodecList.getCodecInfoAt(i); @@ -458,7 +535,7 @@ private static ArrayList getDecoderInfosInternal( continue; } String name = codecInfo.getName(); - if (!isCodecUsableDecoder(codecInfo, name, secureDecodersExplicit, mimeType)) { + if (codecInfo.isEncoder()) { continue; } @Nullable String codecMimeType = getCodecMimeType(codecInfo, name, mimeType); @@ -488,8 +565,7 @@ private static ArrayList getDecoderInfosInternal( boolean hardwareAccelerated = isHardwareAccelerated(codecInfo, mimeType); boolean softwareOnly = isSoftwareOnly(codecInfo, mimeType); boolean vendor = isVendor(codecInfo); - if ((secureDecodersExplicit && key.secure == secureSupported) - || (!secureDecodersExplicit && !key.secure)) { + if (key.secure == secureSupported) { decoderInfos.add( MediaCodecInfo.newInstance( name, @@ -501,31 +577,10 @@ private static ArrayList getDecoderInfosInternal( vendor, /* forceDisableAdaptive= */ false, /* forceSecure= */ false)); - } else if (!secureDecodersExplicit && secureSupported) { - decoderInfos.add( - MediaCodecInfo.newInstance( - name + ".secure", - mimeType, - codecMimeType, - capabilities, - hardwareAccelerated, - softwareOnly, - vendor, - /* forceDisableAdaptive= */ false, - /* forceSecure= */ true)); - // It only makes sense to have one synthesized secure decoder, return immediately. - return decoderInfos; } } catch (Exception e) { - if (SDK_INT == 23 && !decoderInfos.isEmpty()) { - // Suppress error querying secondary codec capabilities up to API level 23. - Log.e(TAG, "Skipping codec " + name + " (failed to query capabilities)"); - } else { - // Rethrow error querying primary codec capabilities, or secondary codec - // capabilities if API level is greater than 23. - Log.e(TAG, "Failed to query codec " + name + " (" + codecMimeType + ")"); - throw e; - } + Log.e(TAG, "Failed to query codec " + name + " (" + codecMimeType + ")"); + throw e; } } return decoderInfos; @@ -582,49 +637,6 @@ private static String getCodecMimeType( return null; } - /** - * Returns whether the specified codec is usable for decoding on the current device. - * - * @param info The codec information. - * @param name The name of the codec - * @param secureDecodersExplicit Whether secure decoders were explicitly listed, if present. - * @param mimeType The MIME type. - * @return Whether the specified codec is usable for decoding on the current device. - */ - private static boolean isCodecUsableDecoder( - android.media.MediaCodecInfo info, - String name, - boolean secureDecodersExplicit, - String mimeType) { - if (info.isEncoder() || (!secureDecodersExplicit && name.endsWith(".secure"))) { - return false; - } - - // Work around https://github.com/google/ExoPlayer/issues/3249. - if (SDK_INT < 24 - && ("OMX.SEC.aac.dec".equals(name) || "OMX.Exynos.AAC.Decoder".equals(name)) - && "samsung".equals(Build.MANUFACTURER) - && (Build.DEVICE.startsWith("zeroflte") // Galaxy S6 - || Build.DEVICE.startsWith("zerolte") // Galaxy S6 Edge - || Build.DEVICE.startsWith("zenlte") // Galaxy S6 Edge+ - || "SC-05G".equals(Build.DEVICE) // Galaxy S6 - || "marinelteatt".equals(Build.DEVICE) // Galaxy S6 Active - || "404SC".equals(Build.DEVICE) // Galaxy S6 Edge - || "SC-04G".equals(Build.DEVICE) - || "SCV31".equals(Build.DEVICE))) { - return false; - } - - // MTK AC3 decoder doesn't support decoding JOC streams in 2-D. See [Internal: b/69400041]. - if (SDK_INT == 23 - && MimeTypes.AUDIO_E_AC3_JOC.equals(mimeType) - && "OMX.MTK.AUDIO.DECODER.DSPAC3".equals(name)) { - return false; - } - - return true; - } - /** * Modifies a list of {@link MediaCodecInfo}s to apply workarounds where we know better than the * platform. @@ -825,9 +837,6 @@ private interface MediaCodecListCompat { */ android.media.MediaCodecInfo getCodecInfoAt(int index); - /** Returns whether secure decoders are explicitly listed, if present. */ - boolean secureDecodersExplicit(); - /** Whether the specified {@link CodecCapabilities} {@code feature} is supported. */ boolean isFeatureSupported(String feature, String mimeType, CodecCapabilities capabilities); @@ -861,11 +870,6 @@ public android.media.MediaCodecInfo getCodecInfoAt(int index) { return mediaCodecInfos[index]; } - @Override - public boolean secureDecodersExplicit() { - return true; - } - @Override public boolean isFeatureSupported( String feature, String mimeType, CodecCapabilities capabilities) { @@ -886,39 +890,6 @@ private void ensureMediaCodecInfosInitialized() { } } - private static final class MediaCodecListCompatV16 implements MediaCodecListCompat { - - @Override - public int getCodecCount() { - return MediaCodecList.getCodecCount(); - } - - @Override - public android.media.MediaCodecInfo getCodecInfoAt(int index) { - return MediaCodecList.getCodecInfoAt(index); - } - - @Override - public boolean secureDecodersExplicit() { - return false; - } - - @Override - public boolean isFeatureSupported( - String feature, String mimeType, CodecCapabilities capabilities) { - // Secure decoders weren't explicitly listed prior to API level 21. We assume that a secure - // H264 decoder exists. - return CodecCapabilities.FEATURE_SecurePlayback.equals(feature) - && MimeTypes.VIDEO_H264.equals(mimeType); - } - - @Override - public boolean isFeatureRequired( - String feature, String mimeType, CodecCapabilities capabilities) { - return false; - } - } - private static final class CodecKey { public final String mimeType; diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/metadata/MetadataRenderer.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/metadata/MetadataRenderer.java index bf64124e232..d87f4499198 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/metadata/MetadataRenderer.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/metadata/MetadataRenderer.java @@ -246,6 +246,13 @@ private void readMetadata(long positionUs) { int result = readSource( formatHolder, buffer, SampleStream.FLAG_PEEK | SampleStream.FLAG_OMIT_SAMPLE_DATA); + while (result == C.RESULT_FORMAT_READ) { + // Format reads are not repeated by a subsequent non-peek read, so handle it here. + subsampleOffsetUs = checkNotNull(formatHolder.format).subsampleOffsetUs; + result = + readSource( + formatHolder, buffer, SampleStream.FLAG_PEEK | SampleStream.FLAG_OMIT_SAMPLE_DATA); + } if (result == C.RESULT_NOTHING_READ && !buffer.isEndOfStream()) { // Nothing to consume yet. return; diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/offline/DownloadHelper.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/offline/DownloadHelper.java index ce63faf3fea..0c9932c24bf 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/offline/DownloadHelper.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/offline/DownloadHelper.java @@ -563,7 +563,7 @@ public DownloadHelper( this.rendererCapabilities = rendererCapabilities; this.debugLoggingEnabled = debugLoggingEnabled; this.scratchSet = new SparseIntArray(); - trackSelector.init(/* listener= */ () -> {}, BandwidthMeter.NO_OP); + trackSelector.init(/* listener= */ trackSelector::onParametersActivated, BandwidthMeter.NO_OP); callbackHandler = Util.createHandlerForCurrentOrMainLooper(); window = new Timeline.Window(); } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/offline/SegmentDownloader.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/offline/SegmentDownloader.java index fedfdd80dd8..5a18faa4cd1 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/offline/SegmentDownloader.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/offline/SegmentDownloader.java @@ -394,6 +394,29 @@ public final void remove() { */ protected final M getManifest(DataSource dataSource, DataSpec dataSpec, boolean removing) throws InterruptedException, IOException { + return getManifest(dataSource, manifestParser, dataSpec, removing); + } + + /** + * Loads and parses a manifest with a different parser than the base parser that parsed the + * initial manifest. + * + *

    This is useful for cases where the manifest needs to be loaded and parsed as a different + * type than the base parser. For example, HLS manifests can be parsed as media playlists that + * inherit variable declarations from the multivariant playlist. + * + * @param dataSource The source to use when loading the manifest. + * @param manifestParser The parser to use when parsing the manifest. + * @param dataSpec The manifest {@link DataSpec}. + * @param removing Whether the manifest is being loaded as part of the download being removed. + * @return The loaded manifest. + * @throws InterruptedException If the thread on which the method is called is interrupted. + * @throws IOException If an error occurs during execution. + */ + protected final M getManifest( + DataSource dataSource, Parser manifestParser, DataSpec dataSpec, boolean removing) + throws InterruptedException, IOException { + checkNotNull(manifestParser); return execute( () -> new RunnableFutureTask() { diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/scheduler/Requirements.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/scheduler/Requirements.java index 273d0318d84..99904217e96 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/scheduler/Requirements.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/scheduler/Requirements.java @@ -15,7 +15,6 @@ */ package androidx.media3.exoplayer.scheduler; -import static android.os.Build.VERSION.SDK_INT; import static com.google.common.base.Preconditions.checkNotNull; import static java.lang.annotation.ElementType.FIELD; import static java.lang.annotation.ElementType.LOCAL_VARIABLE; @@ -211,14 +210,6 @@ private boolean isStorageNotLow(Context context) { } private static boolean isInternetConnectivityValidated(ConnectivityManager connectivityManager) { - // It's possible to check NetworkCapabilities.NET_CAPABILITY_VALIDATED from API level 23, but - // RequirementsWatcher only fires an event to re-check the requirements when NetworkCapabilities - // change from API level 24. We assume that network capability is validated for API level 23 to - // keep in sync. - if (SDK_INT < 24) { - return true; - } - @Nullable Network activeNetwork = connectivityManager.getActiveNetwork(); if (activeNetwork == null) { return false; diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/scheduler/RequirementsWatcher.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/scheduler/RequirementsWatcher.java index 2ba0f9b6b8c..0e9fa92060f 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/scheduler/RequirementsWatcher.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/scheduler/RequirementsWatcher.java @@ -15,7 +15,6 @@ */ package androidx.media3.exoplayer.scheduler; -import static android.os.Build.VERSION.SDK_INT; import static com.google.common.base.Preconditions.checkNotNull; import android.content.BroadcastReceiver; @@ -29,7 +28,6 @@ import android.os.Looper; import android.os.PowerManager; import androidx.annotation.Nullable; -import androidx.annotation.RequiresApi; import androidx.media3.common.util.UnstableApi; import androidx.media3.common.util.Util; @@ -89,11 +87,7 @@ public RequirementsWatcher(Context context, Listener listener, Requirements requ IntentFilter filter = new IntentFilter(); if (requirements.isNetworkRequired()) { - if (SDK_INT >= 24) { - registerNetworkCallbackV24(); - } else { - filter.addAction(ConnectivityManager.CONNECTIVITY_ACTION); - } + registerNetworkCallback(); } if (requirements.isChargingRequired()) { filter.addAction(Intent.ACTION_POWER_CONNECTED); @@ -115,8 +109,8 @@ public RequirementsWatcher(Context context, Listener listener, Requirements requ public void stop() { context.unregisterReceiver(checkNotNull(receiver)); receiver = null; - if (SDK_INT >= 24 && networkCallback != null) { - unregisterNetworkCallbackV24(); + if (networkCallback != null) { + unregisterNetworkCallback(); } } @@ -125,16 +119,14 @@ public Requirements getRequirements() { return requirements; } - @RequiresApi(24) - private void registerNetworkCallbackV24() { + private void registerNetworkCallback() { ConnectivityManager connectivityManager = checkNotNull((ConnectivityManager) context.getSystemService(Context.CONNECTIVITY_SERVICE)); networkCallback = new NetworkCallback(); connectivityManager.registerDefaultNetworkCallback(networkCallback); } - @RequiresApi(24) - private void unregisterNetworkCallbackV24() { + private void unregisterNetworkCallback() { ConnectivityManager connectivityManager = checkNotNull((ConnectivityManager) context.getSystemService(Context.CONNECTIVITY_SERVICE)); connectivityManager.unregisterNetworkCallback(checkNotNull(networkCallback)); @@ -176,7 +168,6 @@ public void onReceive(Context context, Intent intent) { } } - @RequiresApi(24) private final class NetworkCallback extends ConnectivityManager.NetworkCallback { private boolean receivedCapabilitiesChange; diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ClippingMediaPeriod.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ClippingMediaPeriod.java index 4b7f554fa7d..ce2fb039836 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ClippingMediaPeriod.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ClippingMediaPeriod.java @@ -22,6 +22,7 @@ import androidx.annotation.Nullable; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; import androidx.media3.common.StreamKey; @@ -82,7 +83,7 @@ public ClippingMediaPeriod( enableInitialDiscontinuity, startUs, endUs, - /* enableClippingInMediaPeriod= */ false); + Flags.isEnabled(Flags.FLAG_ENABLE_CLIPPING_IN_MEDIA_PERIOD)); } /** @@ -90,16 +91,16 @@ public ClippingMediaPeriod( * MediaPeriod}'s sample streams. * *

    If the start point is guaranteed to be a key frame, pass {@code false} to {@code - * enableInitialPositionDiscontinuity} to suppress an initial discontinuity when the period is - * first read from. + * enableInitialDiscontinuity} to suppress an initial discontinuity when a period is first read + * from. * * @param mediaPeriod The media period to clip. * @param enableInitialDiscontinuity Whether the initial discontinuity should be enabled. * @param startUs The clipping start time, in microseconds. * @param endUs The clipping end time, in microseconds, or {@link C#TIME_END_OF_SOURCE} to * indicate the end of the period. - * @param enableClippingInMediaPeriod Whether the end clipping should be delegated to the {@code - * mediaPeriod}. + * @param enableClippingInMediaPeriod Whether to delegate end-position clipping to {@link + * MediaPeriod} instances. */ @ExperimentalApi // TODO: b/474538573 - Remove once clipping in media period is default. public ClippingMediaPeriod( @@ -351,17 +352,17 @@ private SeekParameters clipSeekParameters(long positionUs, SeekParameters seekPa } private static boolean shouldKeepInitialDiscontinuity( - long startUs, + long enablePositionUs, long requestedPositionUs, @NullableType ExoTrackSelection[] selections, int streamIndex) { - // If the source adjusted the start position to be before the requested position, we need to + // If the source adjusted the enable position to be before the requested position, we need to // report a discontinuity to ensure renderers decode-only the samples before the requested start // position. - if (startUs < requestedPositionUs) { + if (enablePositionUs < requestedPositionUs) { return true; } - // If the clipping start position is non-zero, the clipping sample streams will adjust + // If the enable position is non-zero, the clipping sample streams will adjust // timestamps on buffers they read from the unclipped sample streams. These adjusted buffer // timestamps can be negative, because sample streams provide buffers starting at a key-frame, // which may be before the clipping start point. When the renderer reads a buffer with a @@ -370,7 +371,7 @@ private static boolean shouldKeepInitialDiscontinuity( // discontinuity which resets the renderers before they read the clipping sample stream. // However, for tracks where all samples are sync samples, we assume they have random access // seek behaviour and do not need an initial discontinuity to reset the renderer. - if (startUs != 0) { + if (enablePositionUs != 0) { if (selections[streamIndex] != null) { Format selectedFormat = selections[streamIndex].getSelectedFormat(); return !MimeTypes.allSamplesAreSyncSamples( @@ -391,7 +392,7 @@ private static long enforceClippingRange( /** Wraps a {@link SampleStream} and clips its samples. */ private final class ClippingSampleStream implements SampleStream { - public final SampleStream childStream; + private final SampleStream childStream; private final boolean hasPreroll; private boolean sentEos; @@ -400,7 +401,7 @@ private ClippingSampleStream(SampleStream childStream, boolean hasPreroll) { this.hasPreroll = hasPreroll; } - public void clearSentEos() { + private void clearSentEos() { sentEos = false; } @@ -470,7 +471,12 @@ public int skipData(long positionUs) { if (endUs != C.TIME_END_OF_SOURCE) { flags |= FLAG_STRICT_DURATION; } - if (hasPreroll) { + // A non-zero clipping start introduces preroll the wrapped period cannot know about, because + // sample streams provide buffers starting at a key-frame that may precede the clipping start + // point. With a zero clipping start the samples delivered are exactly the samples the wrapped + // period delivers, so its own report is exact and must not be overwritten with the + // conservative assumption made by the enable-position heuristic. + if (startUs != 0 && hasPreroll) { flags &= ~FLAG_MAYBE_HAS_PREROLL; flags |= FLAG_HAS_PREROLL; } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ClippingMediaSource.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ClippingMediaSource.java index db329473cd1..4fac202a1c3 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ClippingMediaSource.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ClippingMediaSource.java @@ -26,6 +26,7 @@ import androidx.annotation.Nullable; import androidx.media3.common.AdPlaybackState; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.MediaItem; import androidx.media3.common.Timeline; import androidx.media3.common.util.ExperimentalApi; @@ -54,7 +55,6 @@ public static final class Builder { private final MediaSource mediaSource; private MediaItem.ClippingConfiguration.Builder config; - private boolean enableClippingInMediaPeriod; private boolean buildCalled; @@ -66,6 +66,7 @@ public static final class Builder { public Builder(MediaSource mediaSource) { this.mediaSource = checkNotNull(mediaSource); this.config = new MediaItem.ClippingConfiguration.Builder(); + enableClippingInMediaPeriod = Flags.isEnabled(Flags.FLAG_ENABLE_CLIPPING_IN_MEDIA_PERIOD); } /** @@ -236,13 +237,12 @@ public Builder setAllowUnseekableMedia(boolean allowUnseekableMedia) { } /** - * Sets whether an experimental setting to delegate end position clipping to the wrapped {@link - * MediaPeriod} is enabled. + * Sets whether to delegate end-position clipping to {@link MediaPeriod} instances. * - *

    The default value is {@code false}. + *

    The default value is determined by {@link Flags#FLAG_ENABLE_CLIPPING_IN_MEDIA_PERIOD}. * - * @param enableClippingInMediaPeriod Whether the end clipping should be delegated to the - * wrapped {@link MediaPeriod}. + * @param enableClippingInMediaPeriod Whether to delegate end-position clipping to {@link + * MediaPeriod} instances. * @return This builder. */ @ExperimentalApi // TODO: b/474538573 - Remove once clipping in media period is default. @@ -593,6 +593,9 @@ public Window getWindow(int windowIndex, Window window, long defaultPositionProj @Override public Period getPeriod(int periodIndex, Period period, boolean setIds) { timeline.getPeriod(/* periodIndex= */ 0, period, setIds); + if (period.originalDurationUs == C.TIME_UNSET) { + period.originalDurationUs = period.durationUs; + } long positionInClippedWindowUs = period.getPositionInWindowUs() - startUs; long periodDurationUs = durationUs == C.TIME_UNSET ? C.TIME_UNSET : durationUs - positionInClippedWindowUs; @@ -603,7 +606,8 @@ public Period getPeriod(int periodIndex, Period period, boolean setIds) { periodDurationUs, positionInClippedWindowUs, AdPlaybackState.NONE, - period.isPlaceholder); + period.isPlaceholder, + period.originalDurationUs); } } } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ConcatenatingMediaSource2.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ConcatenatingMediaSource2.java index 1ac41c17465..5b547685c5d 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ConcatenatingMediaSource2.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ConcatenatingMediaSource2.java @@ -619,6 +619,9 @@ public Period getPeriodByUid(Object periodUid, Period period) { timeline.getPeriodByUid(childPeriodUid, period); period.windowIndex = 0; period.positionInWindowUs = periodOffsetsInWindowUs.get(periodIndex); + if (period.originalDurationUs == C.TIME_UNSET) { + period.originalDurationUs = period.durationUs; + } period.durationUs = getPeriodDurationUs(period, periodIndex); period.uid = periodUid; return period; @@ -631,6 +634,9 @@ public Period getPeriod(int periodIndex, Period period, boolean setIds) { timelines.get(childIndex).getPeriod(periodIndex - firstPeriodIndexInChild, period, setIds); period.windowIndex = 0; period.positionInWindowUs = periodOffsetsInWindowUs.get(periodIndex); + if (period.originalDurationUs == C.TIME_UNSET) { + period.originalDurationUs = period.durationUs; + } period.durationUs = getPeriodDurationUs(period, periodIndex); if (setIds) { period.uid = getPeriodUid(childIndex, checkNotNull(period.uid)); diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/DefaultMediaSourceFactory.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/DefaultMediaSourceFactory.java index 1ee5c795ef2..caf5ccea0c0 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/DefaultMediaSourceFactory.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/DefaultMediaSourceFactory.java @@ -24,9 +24,11 @@ import androidx.annotation.Nullable; import androidx.media3.common.AdViewProvider; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.Format; import androidx.media3.common.MediaItem; import androidx.media3.common.MimeTypes; +import androidx.media3.common.Player; import androidx.media3.common.util.ExperimentalApi; import androidx.media3.common.util.Log; import androidx.media3.common.util.UnstableApi; @@ -128,6 +130,7 @@ public interface AdsLoaderProvider extends AdsLoader.Provider {} private boolean parseSubtitlesDuringExtraction; private boolean loadOnlySelectedTracks; private boolean enableClippingInMediaPeriod; + private boolean alwaysWrapInClippingMediaSource; /** * Creates a new instance. @@ -214,6 +217,8 @@ public DefaultMediaSourceFactory( liveMinSpeed = C.RATE_UNSET; liveMaxSpeed = C.RATE_UNSET; parseSubtitlesDuringExtraction = true; + loadOnlySelectedTracks = true; + enableClippingInMediaPeriod = Flags.isEnabled(Flags.FLAG_ENABLE_CLIPPING_IN_MEDIA_PERIOD); } @CanIgnoreReturnValue @@ -237,16 +242,6 @@ public DefaultMediaSourceFactory setSubtitleParserFactory( return this; } - @CanIgnoreReturnValue - @Override - @UnstableApi - public DefaultMediaSourceFactory experimentalSetCodecsToParseWithinGopSampleDependencies( - @C.VideoCodecFlags int codecsToParseWithinGopSampleDependencies) { - delegateFactoryLoader.setCodecsToParseWithinGopSampleDependencies( - codecsToParseWithinGopSampleDependencies); - return this; - } - /** * Sets the {@link AdsLoader.Provider} that provides {@link AdsLoader} instances for media items * that have {@link MediaItem.LocalConfiguration#adsConfiguration ads configurations}. @@ -482,10 +477,21 @@ public DefaultMediaSourceFactory setLoadErrorHandlingPolicy( } /** - * Sets whether to load only the tracks selected by the track selection policy. + * Sets whether to load only the video and image tracks selected by the track selection policy. + * + *

    Audio, text, and metadata tracks are always loaded regardless of track selection, as they + * have relatively low bitrate and are more likely to be toggled mid-playback (e.g., changing + * audio languages or subtitle tracks, or disabling audio during scrubbing mode). Always loading + * them allows seamlessly switching mid-playback without re-buffering or seeking. + * + *

    Note that while this saves memory, it may cause additional buffering or latency if + * unselected video or image tracks are enabled mid-playback, as the player may need to perform + * additional network requests to fetch the newly selected track data. * - * @param loadOnlySelectedTracks Whether to load only the tracks selected by the track selection - * policy, instead of loading all tracks. + *

    The default is {@code true}. + * + * @param loadOnlySelectedTracks Whether to load only the video and image tracks selected by the + * track selection policy, instead of loading all tracks. * @return This factory, for convenience. */ @CanIgnoreReturnValue @@ -497,37 +503,51 @@ public DefaultMediaSourceFactory setLoadOnlySelectedTracks(boolean loadOnlySelec } /** - * Sets whether to enable experimental HAGC (ST 2094-50) metadata playback support. When enabled, - * the player will automatically merge HAGC metadata tracks with the associated video track and - * deliver the metadata out-of-band to the decoder on API 37+. + * Sets whether to delegate end-position clipping to {@link MediaPeriod} instances. * - *

    The default value is {@code true}. + *

    The default value is determined by {@link Flags#FLAG_ENABLE_CLIPPING_IN_MEDIA_PERIOD}. * - * @param enableHagcPlayback Whether experimental HAGC metadata playback is enabled. + * @param enableClippingInMediaPeriod Whether to delegate end-position clipping to {@link + * MediaPeriod} instances. * @return This factory, for convenience. */ + @ExperimentalApi // TODO: b/474538573 - Remove once clipping in media period is default. @CanIgnoreReturnValue - @UnstableApi - public DefaultMediaSourceFactory setExperimentalEnableHagcPlayback(boolean enableHagcPlayback) { - delegateFactoryLoader.setExperimentalEnableHagcPlayback(enableHagcPlayback); + public DefaultMediaSourceFactory setEnableClippingInMediaPeriod( + boolean enableClippingInMediaPeriod) { + this.enableClippingInMediaPeriod = enableClippingInMediaPeriod; return this; } /** - * Sets whether an experimental setting to delegate end position clipping to a wrapped {@link - * MediaPeriod} is enabled. + * Sets whether to always wrap created media sources in a {@link ClippingMediaSource}, even when + * the {@link MediaItem} has no clipping configuration. + * + *

    By default a {@link ClippingMediaSource} is only added when {@link + * MediaItem#clippingConfiguration} is non-trivial. This means an item that starts unclipped has + * no {@link ClippingMediaSource} in its source structure, and a later {@link + * Player#replaceMediaItem} that adds a clipping configuration is silently ignored, because the + * existing media source reports that it can be updated in place. Enabling this ensures the + * wrapper is always present, so clipping configuration changes take effect. To apply them without + * re-preparing the media source, also enable {@link #setEnableClippingInMediaPeriod(boolean)}. + * + *

    This applies to all media sources created by this factory, except for items with {@link + * C#SSAI_SCHEME} URIs, which are never clipped. + * + *

    Note: {@link ClippingMediaSource} only supports single-period content, so enabling + * this makes playback of multi-period content fail, even if it isn't clipped. * *

    The default value is {@code false}. * - * @param enableClippingInMediaPeriod Whether the end clipping should be delegated to the wrapped - * {@link MediaPeriod}. + * @param alwaysWrapInClippingMediaSource Whether to always wrap created media sources in a {@link + * ClippingMediaSource}. * @return This factory, for convenience. */ - @ExperimentalApi // TODO: b/474538573 - Remove once clipping in media period is default. @CanIgnoreReturnValue - public DefaultMediaSourceFactory setEnableClippingInMediaPeriod( - boolean enableClippingInMediaPeriod) { - this.enableClippingInMediaPeriod = enableClippingInMediaPeriod; + @UnstableApi + public DefaultMediaSourceFactory setAlwaysWrapInClippingMediaSource( + boolean alwaysWrapInClippingMediaSource) { + this.alwaysWrapInClippingMediaSource = alwaysWrapInClippingMediaSource; return this; } @@ -603,10 +623,9 @@ public MediaSource createMediaSource(MediaItem mediaItem) { List subtitleConfigurations = castNonNull(mediaItem.localConfiguration).subtitleConfigurations; if (!subtitleConfigurations.isEmpty()) { - MediaSource[] mediaSources = new MediaSource[subtitleConfigurations.size() + 1]; - mediaSources[0] = mediaSource; - for (int i = 0; i < subtitleConfigurations.size(); i++) { - if (parseSubtitlesDuringExtraction) { + if (parseSubtitlesDuringExtraction) { + MediaSource[] subtitleMediaSources = new MediaSource[subtitleConfigurations.size()]; + for (int i = 0; i < subtitleConfigurations.size(); i++) { Format format = new Format.Builder() .setSampleMimeType(subtitleConfigurations.get(i).mimeType) @@ -641,10 +660,17 @@ public MediaSource createMediaSource(MediaItem mediaItem) { if (loadErrorHandlingPolicy != null) { progressiveMediaSourceFactory.setLoadErrorHandlingPolicy(loadErrorHandlingPolicy); } - mediaSources[i + 1] = + subtitleMediaSources[i] = progressiveMediaSourceFactory.createMediaSource( MediaItem.fromUri(subtitleConfigurations.get(i).uri.toString())); - } else { + } + mediaSource = + new SideloadedSubtitlesMediaSource( + mediaSource, subtitleConfigurations, subtitleMediaSources); + } else { + MediaSource[] mediaSources = new MediaSource[subtitleConfigurations.size() + 1]; + mediaSources[0] = mediaSource; + for (int i = 0; i < subtitleConfigurations.size(); i++) { SingleSampleMediaSource.Factory singleSampleMediaSourceFactory = new SingleSampleMediaSource.Factory(dataSourceFactory); if (loadErrorHandlingPolicy != null) { @@ -654,19 +680,24 @@ public MediaSource createMediaSource(MediaItem mediaItem) { singleSampleMediaSourceFactory.createMediaSource( subtitleConfigurations.get(i), /* durationUs= */ C.TIME_UNSET); } + mediaSource = new MergingMediaSource(mediaSources); } - - mediaSource = new MergingMediaSource(mediaSources); } return maybeWrapWithAdsMediaSource( - mediaItem, maybeClipMediaSource(mediaItem, mediaSource, enableClippingInMediaPeriod)); + mediaItem, + maybeClipMediaSource( + mediaItem, mediaSource, enableClippingInMediaPeriod, alwaysWrapInClippingMediaSource)); } // internal methods private static MediaSource maybeClipMediaSource( - MediaItem mediaItem, MediaSource mediaSource, boolean enableClippingInMediaPeriod) { - if (mediaItem.clippingConfiguration.startPositionUs == 0 + MediaItem mediaItem, + MediaSource mediaSource, + boolean enableClippingInMediaPeriod, + boolean alwaysWrapInClippingMediaSource) { + if (!alwaysWrapInClippingMediaSource + && mediaItem.clippingConfiguration.startPositionUs == 0 && mediaItem.clippingConfiguration.endPositionUs == C.TIME_END_OF_SOURCE && !mediaItem.clippingConfiguration.relativeToDefaultPosition) { return mediaSource; @@ -721,9 +752,7 @@ private static final class DelegateFactoryLoader { private DataSource.@MonotonicNonNull Factory dataSourceFactory; private boolean parseSubtitlesDuringExtraction; private SubtitleParser.Factory subtitleParserFactory; - private @C.VideoCodecFlags int codecsToParseWithinGopSampleDependencies; private boolean loadOnlySelectedTracks; - private boolean experimentalEnableHagcPlayback; @Nullable private CmcdConfiguration.Factory cmcdConfigurationFactory; @Nullable private DrmSessionManagerProvider drmSessionManagerProvider; @Nullable private LoadErrorHandlingPolicy loadErrorHandlingPolicy; @@ -736,8 +765,7 @@ public DelegateFactoryLoader( mediaSourceFactorySuppliers = new HashMap<>(); mediaSourceFactories = new HashMap<>(); parseSubtitlesDuringExtraction = true; - experimentalEnableHagcPlayback = true; - codecsToParseWithinGopSampleDependencies = C.VIDEO_CODEC_FLAG_H264 | C.VIDEO_CODEC_FLAG_H265; + loadOnlySelectedTracks = true; } public @C.ContentType int[] getSupportedTypes() { @@ -769,8 +797,6 @@ public MediaSource.Factory getMediaSourceFactory(@C.ContentType int contentType) } mediaSourceFactory.setSubtitleParserFactory(subtitleParserFactory); mediaSourceFactory.experimentalParseSubtitlesDuringExtraction(parseSubtitlesDuringExtraction); - mediaSourceFactory.experimentalSetCodecsToParseWithinGopSampleDependencies( - codecsToParseWithinGopSampleDependencies); mediaSourceFactories.put(contentType, mediaSourceFactory); return mediaSourceFactory; } @@ -802,13 +828,6 @@ public void setSubtitleParserFactory(SubtitleParser.Factory subtitleParserFactor } } - public void setCodecsToParseWithinGopSampleDependencies( - @C.VideoCodecFlags int codecsToParseWithinGopSampleDependencies) { - this.codecsToParseWithinGopSampleDependencies = codecsToParseWithinGopSampleDependencies; - extractorsFactory.experimentalSetCodecsToParseWithinGopSampleDependencies( - codecsToParseWithinGopSampleDependencies); - } - public void setCmcdConfigurationFactory(CmcdConfiguration.Factory cmcdConfigurationFactory) { this.cmcdConfigurationFactory = cmcdConfigurationFactory; for (MediaSource.Factory mediaSourceFactory : mediaSourceFactories.values()) { @@ -840,10 +859,6 @@ private void setLoadOnlySelectedTracks(boolean loadOnlySelectedTracks) { this.loadOnlySelectedTracks = loadOnlySelectedTracks; } - private void setExperimentalEnableHagcPlayback(boolean experimentalEnableHagcPlayback) { - this.experimentalEnableHagcPlayback = experimentalEnableHagcPlayback; - } - private void setHeifExtractorFlags(@HeifExtractor.Flags int flags) { if (this.extractorsFactory instanceof DefaultExtractorsFactory) { ((DefaultExtractorsFactory) this.extractorsFactory).setHeifExtractorFlags(flags); @@ -917,8 +932,7 @@ private Supplier loadSupplier(@C.ContentType int contentTyp mediaSourceFactorySupplier = () -> new ProgressiveMediaSource.Factory(dataSourceFactory, extractorsFactory) - .setLoadOnlySelectedTracks(loadOnlySelectedTracks) - .setExperimentalEnableHagcPlayback(experimentalEnableHagcPlayback); + .setLoadOnlySelectedTracks(loadOnlySelectedTracks); break; default: throw new IllegalArgumentException("Unrecognized contentType: " + contentType); diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ForwardingMediaSourceFactory.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ForwardingMediaSourceFactory.java index d47fd2a2604..5a1466ddb67 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ForwardingMediaSourceFactory.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ForwardingMediaSourceFactory.java @@ -55,13 +55,6 @@ public MediaSource.Factory experimentalParseSubtitlesDuringExtraction( return factory.experimentalParseSubtitlesDuringExtraction(parseSubtitlesDuringExtraction); } - @Override - public MediaSource.Factory experimentalSetCodecsToParseWithinGopSampleDependencies( - @C.VideoCodecFlags int codecsToParseWithinGopSampleDependencies) { - return factory.experimentalSetCodecsToParseWithinGopSampleDependencies( - codecsToParseWithinGopSampleDependencies); - } - @Override public @C.ContentType int[] getSupportedTypes() { return factory.getSupportedTypes(); diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/LoadEventInfo.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/LoadEventInfo.java index 0cd0f7e12a7..dde135c86b5 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/LoadEventInfo.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/LoadEventInfo.java @@ -17,9 +17,11 @@ import android.net.Uri; import android.os.SystemClock; +import androidx.annotation.Nullable; import androidx.media3.common.util.UnstableApi; import androidx.media3.datasource.DataSpec; -import java.util.Collections; +import com.google.common.collect.ImmutableMap; +import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.util.List; import java.util.Map; import java.util.concurrent.atomic.AtomicLong; @@ -28,6 +30,155 @@ @UnstableApi public final class LoadEventInfo { + /** Builder for {@link LoadEventInfo}. */ + public static final class Builder { + private long loadTaskId; + private DataSpec dataSpec; + private long elapsedRealtimeMs; + private Uri uri; + @Nullable private String steeredPathwayId; + private Map> responseHeaders; + private long loadDurationMs; + private long bytesLoaded; + + /** + * Creates a builder. + * + * @param loadTaskId See {@link LoadEventInfo#loadTaskId}. + * @param dataSpec See {@link LoadEventInfo#dataSpec}. + * @param elapsedRealtimeMs See {@link LoadEventInfo#elapsedRealtimeMs}. + */ + public Builder(long loadTaskId, DataSpec dataSpec, long elapsedRealtimeMs) { + this.loadTaskId = loadTaskId; + this.dataSpec = dataSpec; + this.uri = dataSpec.uri; + this.elapsedRealtimeMs = elapsedRealtimeMs; + this.responseHeaders = ImmutableMap.of(); + } + + /** + * Creates a builder with the values from the given {@link LoadEventInfo} instance. + * + * @param loadEventInfo The {@link LoadEventInfo} to build upon. + */ + private Builder(LoadEventInfo loadEventInfo) { + this.loadTaskId = loadEventInfo.loadTaskId; + this.dataSpec = loadEventInfo.dataSpec; + this.uri = loadEventInfo.uri; + this.responseHeaders = loadEventInfo.responseHeaders; + this.elapsedRealtimeMs = loadEventInfo.elapsedRealtimeMs; + this.loadDurationMs = loadEventInfo.loadDurationMs; + this.bytesLoaded = loadEventInfo.bytesLoaded; + } + + /** + * Sets the {@link LoadEventInfo#loadTaskId}. + * + * @param loadTaskId See {@link LoadEventInfo#loadTaskId}. + * @return This builder. + */ + @CanIgnoreReturnValue + public Builder setLoadTaskId(long loadTaskId) { + this.loadTaskId = loadTaskId; + return this; + } + + /** + * Sets the {@link LoadEventInfo#dataSpec}. + * + *

    This also sets the {@link LoadEventInfo#uri} to the {@link DataSpec#uri} of the given + * {@link DataSpec}. If you need to set the {@link LoadEventInfo#uri} to a different value, call + * {@link #setUri(Uri)} to override it. + * + * @param dataSpec See {@link LoadEventInfo#dataSpec}. + * @return This builder. + */ + @CanIgnoreReturnValue + public Builder setDataSpec(DataSpec dataSpec) { + this.dataSpec = dataSpec; + this.uri = dataSpec.uri; + return this; + } + + /** + * Sets the {@link LoadEventInfo#elapsedRealtimeMs}. + * + * @param elapsedRealtimeMs See {@link LoadEventInfo#elapsedRealtimeMs}. + * @return This builder. + */ + @CanIgnoreReturnValue + public Builder setElapsedRealtimeMs(long elapsedRealtimeMs) { + this.elapsedRealtimeMs = elapsedRealtimeMs; + return this; + } + + /** + * Sets the {@link LoadEventInfo#uri}. + * + * @param uri See {@link LoadEventInfo#uri}. The default is {@code dataSpec.uri}, where {@link + * DataSpec} is the one passed to the builder constructor or {@link #setDataSpec(DataSpec)}. + * @return This builder. + */ + @CanIgnoreReturnValue + public Builder setUri(Uri uri) { + this.uri = uri; + return this; + } + + /** + * Sets the {@link LoadEventInfo#steeredPathwayId}, the default is {@code null}. + * + * @param steeredPathwayId See {@link LoadEventInfo#steeredPathwayId}. + * @return This builder. + */ + @CanIgnoreReturnValue + public Builder setSteeredPathwayId(@Nullable String steeredPathwayId) { + this.steeredPathwayId = steeredPathwayId; + return this; + } + + /** + * Sets the {@link LoadEventInfo#responseHeaders}. + * + * @param responseHeaders See {@link LoadEventInfo#responseHeaders}. + * @return This builder. + */ + @CanIgnoreReturnValue + public Builder setResponseHeaders(Map> responseHeaders) { + this.responseHeaders = responseHeaders; + return this; + } + + /** + * Sets the {@link LoadEventInfo#loadDurationMs}. + * + * @param loadDurationMs See {@link LoadEventInfo#loadDurationMs}. The default is 0. + * @return This builder. + */ + @CanIgnoreReturnValue + public Builder setLoadDurationMs(long loadDurationMs) { + this.loadDurationMs = loadDurationMs; + return this; + } + + /** + * Sets the {@link LoadEventInfo#bytesLoaded}. + * + * @param bytesLoaded See {@link LoadEventInfo#bytesLoaded}. The default is 0. + * @return This builder. + */ + @CanIgnoreReturnValue + public Builder setBytesLoaded(long bytesLoaded) { + this.bytesLoaded = bytesLoaded; + return this; + } + + /** Builds the {@link LoadEventInfo}. */ + public LoadEventInfo build() { + return new LoadEventInfo(this); + } + } + /** Used for the generation of unique ids. */ private static final AtomicLong idSource = new AtomicLong(); @@ -49,6 +200,12 @@ public static long getNewId() { */ public final Uri uri; + /** + * The ID of the steered pathway from which data is being loaded, or {@code null} if not + * applicable. + */ + @Nullable public final String steeredPathwayId; + /** The response headers associated with the load, or an empty map if unavailable. */ public final Map> responseHeaders; @@ -62,32 +219,24 @@ public static long getNewId() { public final long bytesLoaded; /** - * Equivalent to {@link #LoadEventInfo(long, DataSpec, Uri, Map, long, long, long) - * LoadEventInfo(loadTaskId, dataSpec, dataSpec.uri, Collections.emptyMap(), elapsedRealtimeMs, 0, - * 0)}. + * @deprecated Use {@link Builder} instead. */ + @Deprecated public LoadEventInfo(long loadTaskId, DataSpec dataSpec, long elapsedRealtimeMs) { this( loadTaskId, dataSpec, dataSpec.uri, - Collections.emptyMap(), + ImmutableMap.of(), elapsedRealtimeMs, /* loadDurationMs= */ 0, /* bytesLoaded= */ 0); } /** - * Creates load event info. - * - * @param loadTaskId See {@link #loadTaskId}. - * @param dataSpec See {@link #dataSpec}. - * @param uri See {@link #uri}. - * @param responseHeaders See {@link #responseHeaders}. - * @param elapsedRealtimeMs See {@link #elapsedRealtimeMs}. - * @param loadDurationMs See {@link #loadDurationMs}. - * @param bytesLoaded See {@link #bytesLoaded}. + * @deprecated Use {@link Builder} instead. */ + @Deprecated public LoadEventInfo( long loadTaskId, DataSpec dataSpec, @@ -99,14 +248,39 @@ public LoadEventInfo( this.loadTaskId = loadTaskId; this.dataSpec = dataSpec; this.uri = uri; + this.steeredPathwayId = null; this.responseHeaders = responseHeaders; this.elapsedRealtimeMs = elapsedRealtimeMs; this.loadDurationMs = loadDurationMs; this.bytesLoaded = bytesLoaded; } + /** + * Creates a {@link LoadEventInfo} from a {@link Builder}. + * + * @param builder The builder. + */ + private LoadEventInfo(Builder builder) { + this.loadTaskId = builder.loadTaskId; + this.dataSpec = builder.dataSpec; + this.uri = builder.uri; + this.steeredPathwayId = builder.steeredPathwayId; + this.responseHeaders = builder.responseHeaders; + this.elapsedRealtimeMs = builder.elapsedRealtimeMs; + this.loadDurationMs = builder.loadDurationMs; + this.bytesLoaded = builder.bytesLoaded; + } + + /** Initializes a new {@link Builder} with the values from this instance. */ + public Builder buildUpon() { + return new Builder(this); + } + + /** + * @deprecated Use {@link #buildUpon()} instead. + */ + @Deprecated public LoadEventInfo copyWithTaskIdAndDurationMs(long loadTaskId, long loadDurationMs) { - return new LoadEventInfo( - loadTaskId, dataSpec, uri, responseHeaders, elapsedRealtimeMs, loadDurationMs, bytesLoaded); + return buildUpon().setLoadTaskId(loadTaskId).setLoadDurationMs(loadDurationMs).build(); } } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/MediaSource.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/MediaSource.java index 5fbecfb74f0..6fb0b22232d 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/MediaSource.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/MediaSource.java @@ -33,7 +33,6 @@ import androidx.media3.exoplayer.upstream.CmcdConfiguration; import androidx.media3.exoplayer.upstream.LoadErrorHandlingPolicy; import androidx.media3.exoplayer.util.ReleasableExecutor; -import androidx.media3.extractor.mp4.Mp4Extractor; import androidx.media3.extractor.text.SubtitleParser; import com.google.common.base.Supplier; import com.google.errorprone.annotations.CanIgnoreReturnValue; @@ -138,27 +137,6 @@ default Factory setSubtitleParserFactory(SubtitleParser.Factory subtitleParserFa return this; } - /** - * Sets the set of video codecs for which within GOP sample dependency information should be - * parsed as part of extraction. Defaults to H.264 and H.265. - * - *

    Having access to additional sample dependency information can speed up seeking. See {@link - * Mp4Extractor#FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES}. - * - *

    This method is experimental and will be renamed or removed in a future release. - * - * @param codecsToParseWithinGopSampleDependencies The set of codecs for which to parse within - * GOP sample dependency information. - * @return This factory, for convenience. - */ - @UnstableApi - @CanIgnoreReturnValue - @ExperimentalApi // TODO: b/470365670 - Remove method once config is enabled by default. - default Factory experimentalSetCodecsToParseWithinGopSampleDependencies( - @C.VideoCodecFlags int codecsToParseWithinGopSampleDependencies) { - return this; - } - /** * Sets a supplier for an {@link ReleasableExecutor} that is used for loading the media. * diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/MediaSourceEventListener.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/MediaSourceEventListener.java index e0835880dfc..996ac7b383f 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/MediaSourceEventListener.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/MediaSourceEventListener.java @@ -145,7 +145,7 @@ default void onDownstreamFormatChanged( int windowIndex, @Nullable MediaPeriodId mediaPeriodId, MediaLoadData mediaLoadData) {} /** Dispatches events to {@link MediaSourceEventListener MediaSourceEventListeners}. */ - class EventDispatcher { + final class EventDispatcher { /** The timeline window index reported with the events. */ public final int windowIndex; diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/MergingMediaSource.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/MergingMediaSource.java index b2e9c8f4b23..4ca84423fbd 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/MergingMediaSource.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/MergingMediaSource.java @@ -24,7 +24,6 @@ import androidx.media3.common.C; import androidx.media3.common.MediaItem; import androidx.media3.common.Timeline; -import androidx.media3.common.util.ExperimentalApi; import androidx.media3.common.util.UnstableApi; import androidx.media3.datasource.TransferListener; import androidx.media3.exoplayer.upstream.Allocator; @@ -97,7 +96,6 @@ public IllegalMergeException(@Reason int reason) { private long[][] periodTimeOffsetsUs; @Nullable private IllegalMergeException mergeError; - private boolean enableClippingInMediaPeriod; /** * Creates a merging media source. @@ -175,22 +173,6 @@ public MergingMediaSource( clippedMediaPeriods = MultimapBuilder.hashKeys().arrayListValues().build(); } - /** - * Sets whether an experimental setting to delegate end position clipping to a wrapped {@link - * MediaPeriod} is enabled. - * - *

    The default value is {@code false}. - * - *

    This method must be called immediately after creating the merging source. - * - * @param enableClippingInMediaPeriod Whether the end clipping should be delegated to the wrapped - * {@link MediaPeriod} instances. - */ - @ExperimentalApi // TODO: b/474538573 - Remove once clipping in media period is default. - public void setEnableClippingInMediaPeriod(boolean enableClippingInMediaPeriod) { - this.enableClippingInMediaPeriod = enableClippingInMediaPeriod; - } - @Override public MediaItem getMediaItem() { return mediaSources.length > 0 ? mediaSources[0].getMediaItem() : PLACEHOLDER_MEDIA_ITEM; @@ -242,8 +224,7 @@ public MediaPeriod createPeriod(MediaPeriodId id, Allocator allocator, long star mediaPeriod, /* enableInitialDiscontinuity= */ false, /* startUs= */ 0, - /* endUs= */ checkNotNull(clippedDurationsUs.get(id.periodUid)), - enableClippingInMediaPeriod); + /* endUs= */ checkNotNull(clippedDurationsUs.get(id.periodUid))); clippedMediaPeriods.put(id.periodUid, (ClippingMediaPeriod) mediaPeriod); } return mediaPeriod; @@ -405,6 +386,9 @@ public Window getWindow(int windowIndex, Window window, long defaultPositionProj @Override public Period getPeriod(int periodIndex, Period period, boolean setIds) { super.getPeriod(periodIndex, period, setIds); + if (period.originalDurationUs == C.TIME_UNSET) { + period.originalDurationUs = period.durationUs; + } period.durationUs = periodDurationsUs[periodIndex]; return period; } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/MergingMetadataSampleStream.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/MergingMetadataSampleStream.java index b13b652af56..9a1f0fdc9f4 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/MergingMetadataSampleStream.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/MergingMetadataSampleStream.java @@ -150,9 +150,13 @@ public int skipData(long positionUs) { /** * Reads from the metadata stream into the cache until a metadata sample with a PTS greater than * {@code videoPtsUs} is encountered, or the metadata stream has no more data. + * + *

    If the cache already contains a sample beyond {@code videoPtsUs}, no further reads are + * performed to avoid aggressive prefetching and premature cache eviction. */ private void populateMetadataCache(long videoPtsUs) { - if (metadataStreamEnded) { + if (metadataStreamEnded + || (lastMetadataPtsUs != C.TIME_UNSET && lastMetadataPtsUs > videoPtsUs)) { return; } while (true) { diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ProgressiveMediaPeriod.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ProgressiveMediaPeriod.java index 29ef6f6edb6..191f1fa1ae7 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ProgressiveMediaPeriod.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ProgressiveMediaPeriod.java @@ -23,15 +23,17 @@ import android.net.Uri; import android.os.Build; import android.os.Handler; +import androidx.annotation.IntDef; import androidx.annotation.Nullable; +import androidx.annotation.VisibleForTesting; import androidx.media3.common.C; -import androidx.media3.common.C.DataType; import androidx.media3.common.DataReader; import androidx.media3.common.Format; import androidx.media3.common.Metadata; import androidx.media3.common.MimeTypes; import androidx.media3.common.ParserException; import androidx.media3.common.TrackGroup; +import androidx.media3.common.util.CodecSpecificDataUtil; import androidx.media3.common.util.ConditionVariable; import androidx.media3.common.util.Log; import androidx.media3.common.util.NullableType; @@ -67,12 +69,19 @@ import androidx.media3.extractor.SeekMap; import androidx.media3.extractor.SeekMap.SeekPoints; import androidx.media3.extractor.SeekMap.Unseekable; +import androidx.media3.extractor.SeekPoint; +import androidx.media3.extractor.TrackAwareSeekMap; import androidx.media3.extractor.TrackOutput; import androidx.media3.extractor.metadata.icy.IcyHeaders; import com.google.common.collect.ImmutableMap; import com.google.common.net.HttpHeaders; import java.io.IOException; import java.io.InterruptedIOException; +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; @@ -80,7 +89,7 @@ import java.util.List; import java.util.Map; import java.util.Objects; -import java.util.concurrent.atomic.AtomicReference; +import java.util.concurrent.atomic.AtomicBoolean; import org.checkerframework.checker.nullness.qual.EnsuresNonNull; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; @@ -130,7 +139,6 @@ interface Listener { @Nullable private final String customCacheKey; private final long continueLoadingCheckIntervalBytes; private final boolean loadOnlySelectedTracks; - private final boolean experimentalEnableHagcPlayback; private final int singleTrackId; @Nullable private final Format singleTrackFormat; private final long singleSampleDurationUs; @@ -140,6 +148,8 @@ interface Listener { private final Runnable maybeFinishPrepareRunnable; private final Runnable onContinueLoadingRequestedRunnable; private final Handler handler; + private final List mergingSampleStreams; + private final LoadingStateMachine loadingStateMachine; @Nullable private Callback callback; @Nullable private IcyHeaders icyHeaders; @@ -157,23 +167,11 @@ interface Listener { private @MonotonicNonNull SeekMap seekMap; private long durationUs; private boolean isLive; - private @DataType int dataType; + private @C.DataType int dataType; private long endPositionUs; - - private boolean seenFirstTrackSelection; - private boolean notifyDiscontinuity; - private boolean pendingInitialDiscontinuity; - private boolean usesStreamPrerollFlags; private int enabledTrackCount; - private final List mergingSampleStreams; private boolean isLengthKnown; - - private long lastSeekPositionUs; - private long pendingResetPositionUs; - private boolean pendingDeferredRetry; - - private int extractedSamplesCountAtStartOfLoad; - private boolean loadingFinished; + private boolean isWaitingForContinueLoading; private boolean released; /** @@ -192,8 +190,8 @@ interface Listener { * indexing. May be null. * @param continueLoadingCheckIntervalBytes The number of bytes that should be loaded between each * invocation of {@link Callback#onContinueLoadingRequested(SequenceableLoader)}. - * @param loadOnlySelectedTracks Whether to load only the tracks selected by the track selection - * policy. + * @param loadOnlySelectedTracks Whether to load only the video and image tracks selected by the + * track selection policy. Audio, text, and metadata tracks are always loaded. * @param singleTrackId The ID of the track configured by {@code singleTrackFormat}. Ignored if * {@code singleTrackFormat} is null. * @param singleTrackFormat The format of the single track this period is known to emit, allowing @@ -217,7 +215,6 @@ public ProgressiveMediaPeriod( @Nullable String customCacheKey, int continueLoadingCheckIntervalBytes, boolean loadOnlySelectedTracks, - boolean experimentalEnableHagcPlayback, int singleTrackId, @Nullable Format singleTrackFormat, long singleSampleDurationUs, @@ -233,7 +230,6 @@ public ProgressiveMediaPeriod( this.customCacheKey = customCacheKey; this.continueLoadingCheckIntervalBytes = continueLoadingCheckIntervalBytes; this.loadOnlySelectedTracks = loadOnlySelectedTracks; - this.experimentalEnableHagcPlayback = experimentalEnableHagcPlayback; this.singleTrackId = singleTrackId; this.singleTrackFormat = singleTrackFormat; this.endPositionUs = C.TIME_END_OF_SOURCE; @@ -247,7 +243,8 @@ public ProgressiveMediaPeriod( maybeFinishPrepareRunnable = this::maybeFinishPrepare; onContinueLoadingRequestedRunnable = () -> { - if (!released) { + if (!released && !loadCondition.isOpen()) { + isWaitingForContinueLoading = true; checkNotNull(callback).onContinueLoadingRequested(ProgressiveMediaPeriod.this); } }; @@ -256,7 +253,7 @@ public ProgressiveMediaPeriod( sampleQueues = new SampleQueue[0]; controlledTrackOutputs = new ControlledTrackOutput[0]; mergingSampleStreams = new ArrayList<>(); - pendingResetPositionUs = C.TIME_UNSET; + loadingStateMachine = new LoadingStateMachine(); dataType = C.DATA_TYPE_MEDIA; } @@ -285,6 +282,7 @@ public void onLoaderReleased() { @Override public void prepare(Callback callback, long positionUs) { this.callback = callback; + loadingStateMachine.onPrepared(/* isSingleTrack= */ singleTrackFormat != null, positionUs); if (singleTrackFormat != null) { // track() and endTracks() are meant to be called on the loading thread, which doesn't exist // yet (we're on the playback thread here). Starting the loading thread will provide a memory @@ -297,9 +295,7 @@ public void prepare(Callback callback, long positionUs) { /* timesUs= */ new long[] {0}, /* durationUs= */ C.TIME_UNSET)); endTracks(); - pendingResetPositionUs = positionUs; } else { - loadCondition.open(); startLoading(); } } @@ -307,7 +303,7 @@ public void prepare(Callback callback, long positionUs) { @Override public void maybeThrowPrepareError() throws IOException { maybeThrowError(); - if (loadingFinished && !prepared) { + if (loadingStateMachine.isFinished() && !prepared) { throw ParserException.createForMalformedContainer( "Loading finished before preparation is complete.", /* cause= */ null); } @@ -378,8 +374,10 @@ public long selectTracks( // when we have a single sample only), or if we're making a selection having previously // disabled all tracks. boolean seekRequired = - seenFirstTrackSelection ? oldEnabledTrackCount == 0 : positionUs != 0 && !isSingleSample; - boolean hasPreroll = false; + loadingStateMachine.hasSeenFirstTrackSelection() + ? oldEnabledTrackCount == 0 + : positionUs != 0 && !isSingleSample; + boolean legacyHasPreroll = false; // Select new tracks. for (int i = 0; i < selections.length; i++) { if (streams[i] == null && selections[i] != null) { @@ -390,11 +388,10 @@ public long selectTracks( checkState(!trackEnabledStates[track]); enabledTrackCount++; trackEnabledStates[track] = true; - hasPreroll |= selection.getSelectedFormat().hasPrerollSamples; - SampleStream stream = - new SampleStreamImpl(track, selection.getSelectedFormat().hasPrerollSamples); - if (experimentalEnableHagcPlayback - && Build.VERSION.SDK_INT >= 37 + boolean trackHasPrerollAtStart = trackHasPreroll(track, positionUs); + legacyHasPreroll |= trackHasPreroll(track, /* positionUs= */ 0); + SampleStream stream = new SampleStreamImpl(track, trackHasPrerollAtStart); + if (Build.VERSION.SDK_INT >= 37 && !isT35TrackExplicitlySelected && MimeTypes.isVideo(selection.getSelectedFormat().sampleMimeType)) { // TODO: b/388762778 - The MP4 container has extra information (e.g. cdsc box) specifying @@ -402,45 +399,28 @@ public long selectTracks( // the correct mergingSampleStreams when multiple video/HAGC tracks exist. for (int j = 0; j < tracks.length; j++) { Format format = tracks.get(j).getFormat(0); - if (Objects.equals(format.sampleMimeType, MimeTypes.APPLICATION_ITUT_T35)) { - boolean isHagc = false; - if (!format.initializationData.isEmpty()) { - byte[] initData = format.initializationData.get(0); - if (initData.length >= 5 - && initData[0] == (byte) 0xB5 - && initData[1] == (byte) 0x00 - && initData[2] == (byte) 0x90 - && initData[3] == (byte) 0x00 - && initData[4] == (byte) 0x01) { - isHagc = true; - } - } - if (isHagc) { - checkState(!trackEnabledStates[j]); - enabledTrackCount++; - trackEnabledStates[j] = true; - if (loadOnlySelectedTracks) { - controlledTrackOutputs[j].updateSelectionState(true); - } - // HAGC it35 metadata samples are standalone and do not depend on - // previous samples, hence hasPreroll is not relevant and always false. - stream = - new MergingMetadataSampleStream( - stream, - new SampleStreamImpl(j, /* hasPreroll= */ false), - selection.getSelectedFormat()); - mergingSampleStreams.add((MergingMetadataSampleStream) stream); - break; - } + if (CodecSpecificDataUtil.isHagcTrack(format)) { + checkState(!trackEnabledStates[j]); + enabledTrackCount++; + trackEnabledStates[j] = true; + // HAGC it35 metadata samples are standalone and do not depend on + // previous samples, hence hasPreroll is not relevant and always false. + stream = + new MergingMetadataSampleStream( + stream, + new SampleStreamImpl(j, /* hasPreroll= */ false), + selection.getSelectedFormat()); + mergingSampleStreams.add((MergingMetadataSampleStream) stream); + break; } } } streams[i] = stream; streamResetFlags[i] = true; - if (loadOnlySelectedTracks) { - // If we reenable a track, it needs to be seeked since it was not previously loaded. - seekRequired |= seenFirstTrackSelection; + if (loadOnlySelectedTracks && !controlledTrackOutputs[track].isSelected()) { + // If we reenable a track that was not previously loaded, it needs to be seeked. + seekRequired |= loadingStateMachine.hasSeenFirstTrackSelection(); continue; } @@ -457,28 +437,33 @@ public long selectTracks( } } - if (pendingInitialDiscontinuity || !seenFirstTrackSelection) { - pendingInitialDiscontinuity = hasPreroll; + loadingStateMachine.onTrackSelection(legacyHasPreroll, enabledTrackCount); + + boolean hasEnabledAudioVideoTracks = false; + for (int i = 0; i < tracks.length; i++) { + if (trackEnabledStates[i] && trackState.trackIsAudioVideoFlags[i]) { + hasEnabledAudioVideoTracks = true; + break; + } } + trackState.hasEnabledAudioVideoTracks = hasEnabledAudioVideoTracks; if (loadOnlySelectedTracks) { + boolean[] tracksSelectedForLoading = getTracksSelectedForLoading(trackEnabledStates, tracks); for (int i = 0; i < controlledTrackOutputs.length; i++) { - controlledTrackOutputs[i].updateSelectionState(trackEnabledStates[i]); + controlledTrackOutputs[i].updateSelectionState(tracksSelectedForLoading[i]); } } if (enabledTrackCount == 0) { - pendingDeferredRetry = false; - notifyDiscontinuity = false; - pendingInitialDiscontinuity = false; - if (loader.isLoading()) { + if (loadingStateMachine.isCanceling()) { // Discard as much as we can synchronously. for (SampleQueue sampleQueue : sampleQueues) { sampleQueue.discardToEnd(); } loader.cancelLoading(); } else { - loadingFinished = false; + loader.clearFatalError(); for (SampleQueue sampleQueue : sampleQueues) { sampleQueue.reset(); } @@ -495,7 +480,6 @@ public long selectTracks( } } } - seenFirstTrackSelection = true; return positionUs; } @@ -506,7 +490,7 @@ public void discardBuffer(long positionUs, boolean toKeyframe) { return; } assertPrepared(); - if (isPendingReset()) { + if (loadingStateMachine.isPendingReset()) { return; } boolean[] trackEnabledStates = trackState.trackEnabledStates; @@ -518,31 +502,32 @@ public void discardBuffer(long positionUs, boolean toKeyframe) { @Override public void reevaluateBuffer(long positionUs) { - if (enabledTrackCount > 0 && !isPendingReset() && haveSampleQueuesReachedEndTimeUs()) { - loadingFinished = true; - // TODO: b/474538573 - Actually cancel ongoing load and restart later if required. + if (enabledTrackCount > 0 && haveSampleQueuesReachedEndTimeUs()) { + loadingStateMachine.onEndPositionReached(); + maybeCancelOrDiscardUpstreamBuffers(); } } @Override public boolean continueLoading(LoadingInfo loadingInfo) { - if (loadingFinished - || loader.hasFatalError() - || pendingDeferredRetry - || ((prepared || singleTrackFormat != null) && enabledTrackCount == 0)) { + if (!loadingStateMachine.canContinueLoading( + /* isPreparedOrSingleTrack= */ prepared || singleTrackFormat != null, enabledTrackCount)) { return false; } - boolean continuedLoading = loadCondition.open(); if (!loader.isLoading()) { startLoading(); - continuedLoading = true; + return true; + } + if (isWaitingForContinueLoading) { + isWaitingForContinueLoading = false; + return loadCondition.open(); } - return continuedLoading; + return false; } @Override public boolean isLoading() { - return !loadingFinished && loader.isLoading() && loadCondition.isOpen(); + return loadingStateMachine.isLoading() && loadCondition.isOpen(); } @Override @@ -552,31 +537,21 @@ public long getNextLoadPositionUs() { @Override public void setUsesStreamPrerollFlags() { - this.usesStreamPrerollFlags = true; + loadingStateMachine.setUsesStreamPrerollFlags(); } @Override public long readDiscontinuity() { - if (!usesStreamPrerollFlags && pendingInitialDiscontinuity) { - pendingInitialDiscontinuity = false; - return lastSeekPositionUs; - } - - if (notifyDiscontinuity - && (loadingFinished || getExtractedSamplesCount() > extractedSamplesCountAtStartOfLoad)) { - notifyDiscontinuity = false; - return lastSeekPositionUs; - } - return C.TIME_UNSET; + return loadingStateMachine.readDiscontinuity(getExtractedSamplesCount()); } @Override public long getBufferedPositionUs() { assertPrepared(); - if (loadingFinished || enabledTrackCount == 0) { + if (loadingStateMachine.isFinished() || enabledTrackCount == 0) { return C.TIME_END_OF_SOURCE; - } else if (isPendingReset()) { - return pendingResetPositionUs; + } else if (loadingStateMachine.isPendingReset()) { + return loadingStateMachine.getPendingResetPositionUs(); } long largestQueuedTimestampUs = Long.MAX_VALUE; if (haveAudioVideoTracks) { @@ -595,7 +570,7 @@ public long getBufferedPositionUs() { largestQueuedTimestampUs = getLargestQueuedTimestampUs(/* includeDisabledTracks= */ false); } return largestQueuedTimestampUs == Long.MIN_VALUE - ? lastSeekPositionUs + ? loadingStateMachine.getLastSeekPositionUs() : largestQueuedTimestampUs; } @@ -609,29 +584,20 @@ public long seekToUs(long positionUs) { // Treat all seeks into non-seekable media as being to t=0. positionUs = seekMap.isSeekable() ? positionUs : 0; - notifyDiscontinuity = false; - boolean isSameAsLastSeekPosition = lastSeekPositionUs == positionUs; - lastSeekPositionUs = positionUs; - if (isPendingReset()) { - // A reset is already pending. We only need to update its position. - pendingResetPositionUs = positionUs; - return positionUs; - } + boolean wasPendingReset = loadingStateMachine.isPendingReset(); + boolean isSameAsLastSeekPosition = loadingStateMachine.isLastSeekPosition(positionUs); + boolean canSeekInsideBuffer = + loadingStateMachine.canSeekInsideBuffer(dataType) + && seekInsideBufferUs(trackIsAudioVideoFlags, positionUs, isSameAsLastSeekPosition); + + loadingStateMachine.onSeek(positionUs, canSeekInsideBuffer); - // If we're not playing a live stream, and when loading will continue (or has finished), try - // and seek within the existing buffer instead of restarting the load. - if (dataType != C.DATA_TYPE_MEDIA_PROGRESSIVE_LIVE - && (loadingFinished || loader.isLoading()) - && seekInsideBufferUs(trackIsAudioVideoFlags, positionUs, isSameAsLastSeekPosition)) { + if (canSeekInsideBuffer || wasPendingReset) { return positionUs; } // We can't seek inside the buffer, and so need to reset. - pendingDeferredRetry = false; - pendingResetPositionUs = positionUs; - loadingFinished = false; - pendingInitialDiscontinuity = false; - if (loader.isLoading()) { + if (loadingStateMachine.isCanceling()) { // Discard as much as we can synchronously. for (SampleQueue sampleQueue : sampleQueues) { sampleQueue.discardToEnd(); @@ -660,17 +626,39 @@ public long getAdjustedSeekPositionUs(long positionUs, SeekParameters seekParame @Override public long setEndPositionUs(long endPositionUs) { + long oldEndPositionUs = this.endPositionUs; this.endPositionUs = endPositionUs; for (SampleQueue sampleQueue : sampleQueues) { sampleQueue.setReadEndTimeUs(endPositionUs); } + if (!prepared) { + return endPositionUs; + } + boolean isExtension = + oldEndPositionUs != C.TIME_END_OF_SOURCE + && (endPositionUs == C.TIME_END_OF_SOURCE || endPositionUs > oldEndPositionUs); + boolean haveSampleQueuesReachedEndTimeUs = haveSampleQueuesReachedEndTimeUs(); + if (isExtension && !haveSampleQueuesReachedEndTimeUs) { + loadingStateMachine.onEndPositionExtended(); + if (enabledTrackCount > 0) { + checkNotNull(callback).onContinueLoadingRequested(this); + } + } else if (haveSampleQueuesReachedEndTimeUs) { + loadingStateMachine.onEndPositionReached(); + if (loadingStateMachine.isCancelingForClipping()) { + loader.cancelLoading(); + } else if (loadingStateMachine.isClippedFinished()) { + maybeCancelOrDiscardUpstreamBuffers(); + } + } return endPositionUs; } // SampleStream methods. /* package */ boolean isReady(int track) { - return !suppressRead() && sampleQueues[track].isReady(loadingFinished); + return !loadingStateMachine.suppressRead() + && sampleQueues[track].isReady(loadingStateMachine.isFinished()); } /* package */ void maybeThrowError(int sampleQueueIndex) throws IOException { @@ -687,12 +675,13 @@ public long setEndPositionUs(long endPositionUs) { FormatHolder formatHolder, DecoderInputBuffer buffer, @ReadFlags int readFlags) { - if (suppressRead()) { + if (loadingStateMachine.suppressRead()) { return C.RESULT_NOTHING_READ; } maybeNotifyDownstreamFormat(sampleQueueIndex); int result = - sampleQueues[sampleQueueIndex].read(formatHolder, buffer, readFlags, loadingFinished); + sampleQueues[sampleQueueIndex].read( + formatHolder, buffer, readFlags, loadingStateMachine.isFinished()); if (result == C.RESULT_NOTHING_READ) { maybeStartDeferredRetry(sampleQueueIndex); } @@ -700,12 +689,12 @@ public long setEndPositionUs(long endPositionUs) { } /* package */ int skipData(int track, long positionUs) { - if (suppressRead()) { + if (loadingStateMachine.suppressRead()) { return 0; } maybeNotifyDownstreamFormat(track); SampleQueue sampleQueue = sampleQueues[track]; - int skipCount = sampleQueue.getSkipCount(positionUs, loadingFinished); + int skipCount = sampleQueue.getSkipCount(positionUs, loadingStateMachine.isFinished()); sampleQueue.skip(skipCount); if (skipCount == 0) { maybeStartDeferredRetry(track); @@ -714,19 +703,38 @@ public long setEndPositionUs(long endPositionUs) { } private boolean haveSampleQueuesReachedEndTimeUs() { - if (endPositionUs == C.TIME_END_OF_SOURCE) { + if (endPositionUs == C.TIME_END_OF_SOURCE || enabledTrackCount == 0) { return false; } + if (loadingStateMachine.getState() == LoadingStateMachine.STATE_FINISHED) { + return true; + } assertPrepared(); - boolean endPositionReached = true; for (int i = 0; i < sampleQueues.length; i++) { - // Ignore non-AV tracks, which may be sparse or poorly interleaved. + // Ignore non-AV tracks if AV tracks are enabled, as they may be sparse or poorly interleaved. if (trackState.trackEnabledStates[i] - && (trackState.trackIsAudioVideoFlags[i] || !haveAudioVideoTracks)) { - endPositionReached &= sampleQueues[i].hasQueuedTimestampsUpToReadEndTimeUs(); + && (trackState.trackIsAudioVideoFlags[i] || !trackState.hasEnabledAudioVideoTracks)) { + if (!sampleQueues[i].hasQueuedTimestampsUpToReadEndTimeUs()) { + return false; + } + } + } + return true; + } + + private void maybeCancelOrDiscardUpstreamBuffers() { + if (loadingStateMachine.isCancelingForClipping()) { + loader.cancelLoading(); + } else if (loadingStateMachine.isClippedFinished()) { + for (SampleQueue sampleQueue : sampleQueues) { + long largestReadTimestampUs = sampleQueue.getLargestReadTimestampUs(); + long discardFromUs = endPositionUs; + if (largestReadTimestampUs != Long.MIN_VALUE && largestReadTimestampUs >= endPositionUs) { + discardFromUs = largestReadTimestampUs + 1; + } + sampleQueue.discardUpstreamFrom(discardFromUs); } } - return endPositionReached; } private void maybeNotifyDownstreamFormat(int track) { @@ -739,53 +747,43 @@ private void maybeNotifyDownstreamFormat(int track) { trackFormat, C.SELECTION_REASON_UNKNOWN, /* trackSelectionData= */ null, - lastSeekPositionUs); + loadingStateMachine.getLastSeekPositionUs()); trackNotifiedDownstreamFormats[track] = true; } } private void maybeStartDeferredRetry(int track) { assertPrepared(); - if (!pendingDeferredRetry + if (!loadingStateMachine.isDeferredRetryPending() || (haveAudioVideoTracks && !trackState.trackIsAudioVideoFlags[track]) || sampleQueues[track].isReady(/* loadingFinished= */ false)) { return; } - pendingResetPositionUs = 0; - pendingDeferredRetry = false; - notifyDiscontinuity = true; - lastSeekPositionUs = 0; - extractedSamplesCountAtStartOfLoad = 0; + loadingStateMachine.onDeferredRetryStarted(); for (SampleQueue sampleQueue : sampleQueues) { sampleQueue.reset(); } checkNotNull(callback).onContinueLoadingRequested(this); } - private boolean suppressRead() { - return notifyDiscontinuity || isPendingReset(); - } - // Loader.Callback implementation. @Override public void onLoadStarted( ExtractingLoadable loadable, long elapsedRealtimeMs, long loadDurationMs, int retryCount) { StatsDataSource dataSource = loadable.dataSource; - LoadEventInfo loadEventInfo = - retryCount == 0 - ? new LoadEventInfo(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) - : new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - dataSource.getLastOpenedUri(), - dataSource.getLastResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - dataSource.getBytesRead()); + LoadEventInfo.Builder loadEventInfo = + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs); + if (retryCount != 0) { + loadEventInfo + .setUri(dataSource.getLastOpenedUri()) + .setResponseHeaders(dataSource.getLastResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(dataSource.getBytesRead()); + } mediaSourceEventDispatcher.loadStarted( - loadEventInfo, - C.DATA_TYPE_MEDIA, + loadEventInfo.build(), + dataType, C.TRACK_TYPE_UNKNOWN, /* trackFormat= */ null, C.SELECTION_REASON_UNKNOWN, @@ -809,25 +807,27 @@ public void onLoadCompleted( } StatsDataSource dataSource = loadable.dataSource; LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - dataSource.getLastOpenedUri(), - dataSource.getLastResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - dataSource.getBytesRead()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(dataSource.getLastOpenedUri()) + .setResponseHeaders(dataSource.getLastResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(dataSource.getBytesRead()) + .build(); loadErrorHandlingPolicy.onLoadTaskConcluded(loadable.loadTaskId); mediaSourceEventDispatcher.loadCompleted( loadEventInfo, - C.DATA_TYPE_MEDIA, + dataType, C.TRACK_TYPE_UNKNOWN, /* trackFormat= */ null, C.SELECTION_REASON_UNKNOWN, /* trackSelectionData= */ null, /* mediaStartTimeUs= */ loadable.seekTimeUs, durationUs); - loadingFinished = true; + loadingStateMachine.onLoadCompleted(); + if (haveSampleQueuesReachedEndTimeUs()) { + loadingStateMachine.onEndPositionReached(); + maybeCancelOrDiscardUpstreamBuffers(); + } checkNotNull(callback).onContinueLoadingRequested(this); } @@ -836,29 +836,32 @@ public void onLoadCanceled( ExtractingLoadable loadable, long elapsedRealtimeMs, long loadDurationMs, boolean released) { StatsDataSource dataSource = loadable.dataSource; LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - dataSource.getLastOpenedUri(), - dataSource.getLastResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - dataSource.getBytesRead()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(dataSource.getLastOpenedUri()) + .setResponseHeaders(dataSource.getLastResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(dataSource.getBytesRead()) + .build(); loadErrorHandlingPolicy.onLoadTaskConcluded(loadable.loadTaskId); mediaSourceEventDispatcher.loadCanceled( loadEventInfo, - C.DATA_TYPE_MEDIA, + dataType, C.TRACK_TYPE_UNKNOWN, /* trackFormat= */ null, C.SELECTION_REASON_UNKNOWN, /* trackSelectionData= */ null, /* mediaStartTimeUs= */ loadable.seekTimeUs, durationUs); + loadingStateMachine.onLoadCanceled(released, haveSampleQueuesReachedEndTimeUs()); if (!released) { - for (SampleQueue sampleQueue : sampleQueues) { - sampleQueue.reset(); + if (loadingStateMachine.isClippedFinished()) { + maybeCancelOrDiscardUpstreamBuffers(); + } else if (loadingStateMachine.isPendingReset()) { + for (SampleQueue sampleQueue : sampleQueues) { + sampleQueue.reset(); + } } - if (enabledTrackCount > 0) { + if (enabledTrackCount > 0 && !loadingStateMachine.isFinished()) { checkNotNull(callback).onContinueLoadingRequested(this); } } @@ -873,17 +876,15 @@ public LoadErrorAction onLoadError( int errorCount) { StatsDataSource dataSource = loadable.dataSource; LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - dataSource.getLastOpenedUri(), - dataSource.getLastResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - dataSource.getBytesRead()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(dataSource.getLastOpenedUri()) + .setResponseHeaders(dataSource.getLastResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(dataSource.getBytesRead()) + .build(); MediaLoadData mediaLoadData = new MediaLoadData( - C.DATA_TYPE_MEDIA, + dataType, C.TRACK_TYPE_UNKNOWN, /* trackFormat= */ null, C.SELECTION_REASON_UNKNOWN, @@ -891,14 +892,19 @@ public LoadErrorAction onLoadError( /* mediaStartTimeMs= */ Util.usToMs(loadable.seekTimeUs), Util.usToMs(durationUs)); LoadErrorAction loadErrorAction; + int extractedSamplesCount = getExtractedSamplesCount(); + boolean madeProgress = + loadingStateMachine.hasExtractedProgressSinceLoadStart(extractedSamplesCount); + int effectiveErrorCount = madeProgress ? 1 : errorCount; long retryDelayMs = - loadErrorHandlingPolicy.getRetryDelayMsFor( - new LoadErrorInfo(loadEventInfo, mediaLoadData, error, errorCount)); + effectiveErrorCount > loadErrorHandlingPolicy.getMinimumLoadableRetryCount(dataType) + ? C.TIME_UNSET + : loadErrorHandlingPolicy.getRetryDelayMsFor( + new LoadErrorInfo(loadEventInfo, mediaLoadData, error, effectiveErrorCount)); if (retryDelayMs == C.TIME_UNSET) { loadErrorAction = Loader.DONT_RETRY_FATAL; + loadingStateMachine.onFatalLoadError(); } else /* the load should be retried */ { - int extractedSamplesCount = getExtractedSamplesCount(); - boolean madeProgress = extractedSamplesCount > extractedSamplesCountAtStartOfLoad; loadErrorAction = configureRetry(loadable, extractedSamplesCount) ? Loader.createRetryAction(/* resetErrorCount= */ madeProgress, retryDelayMs) @@ -908,7 +914,7 @@ public LoadErrorAction onLoadError( boolean wasCanceled = !loadErrorAction.isRetry(); mediaSourceEventDispatcher.loadError( loadEventInfo, - C.DATA_TYPE_MEDIA, + dataType, C.TRACK_TYPE_UNKNOWN, /* trackFormat= */ null, C.SELECTION_REASON_UNKNOWN, @@ -1014,6 +1020,7 @@ private void maybeFinishPrepare() { } } loadCondition.close(); + isWaitingForContinueLoading = true; int trackCount = sampleQueues.length; int primaryTrackIndex = 0; @C.TrackType int primaryTrackIndexType = C.TRACK_TYPE_UNKNOWN; @@ -1085,71 +1092,84 @@ public long getDurationUs() { } private void startLoading() { + boolean wasPendingReset = loadingStateMachine.isPendingReset(); + long loadPositionUs = + wasPendingReset ? loadingStateMachine.getPendingResetPositionUs() : C.TIME_UNSET; + + loadingStateMachine.onStartLoading( + prepared, endPositionUs, durationUs, getExtractedSamplesCount()); + if (loadingStateMachine.isFinished()) { + return; + } + + isWaitingForContinueLoading = false; + loadCondition.open(); ExtractingLoadable loadable = new ExtractingLoadable( uri, dataSource, progressiveMediaExtractor, /* extractorOutput= */ this, loadCondition); if (prepared) { - checkState(isPendingReset()); - long maxLoadPositionUs = endPositionUs != C.TIME_END_OF_SOURCE ? endPositionUs : durationUs; - if (maxLoadPositionUs != C.TIME_UNSET && pendingResetPositionUs > maxLoadPositionUs) { - loadingFinished = true; - pendingResetPositionUs = C.TIME_UNSET; - return; - } - loadable.setLoadPosition( - checkNotNull(seekMap).getSeekPoints(pendingResetPositionUs).first.position, - pendingResetPositionUs); - for (SampleQueue sampleQueue : sampleQueues) { - sampleQueue.setStartTimeUs(pendingResetPositionUs); + if (wasPendingReset) { + loadable.setLoadPosition( + checkNotNull(seekMap).getSeekPoints(loadPositionUs).first.position, loadPositionUs); + for (SampleQueue sampleQueue : sampleQueues) { + sampleQueue.setStartTimeUs(loadPositionUs); + } + } else { + long resumePositionUs = calculateResumePositionUs(); + SeekPoint seekPoint = checkNotNull(seekMap).getSeekPoints(resumePositionUs).first; + loadable.setLoadPosition(seekPoint.position, seekPoint.timeUs); + for (SampleQueue sampleQueue : sampleQueues) { + sampleQueue.expectResume(); + } } - pendingResetPositionUs = C.TIME_UNSET; } - extractedSamplesCountAtStartOfLoad = getExtractedSamplesCount(); loader.startLoading( loadable, this, loadErrorHandlingPolicy.getMinimumLoadableRetryCount(dataType)); } + private long calculateResumePositionUs() { + assertPrepared(); + checkState(enabledTrackCount > 0); + long minLargestQueuedUs = Long.MAX_VALUE; + for (int i = 0; i < sampleQueues.length; i++) { + // Prioritize AV tracks to determine the resume point, preventing sparse non-AV tracks from + // causing unnecessary re-extraction from the start of the period. + if (trackState.trackEnabledStates[i] + && (trackState.trackIsAudioVideoFlags[i] || !trackState.hasEnabledAudioVideoTracks)) { + long largestQueuedUs = sampleQueues[i].getLargestQueuedTimestampUs(); + if (largestQueuedUs == Long.MIN_VALUE) { + // An enabled track is empty, so resume from the last seek position. + return loadingStateMachine.getLastSeekPositionUs(); + } + minLargestQueuedUs = min(minLargestQueuedUs, largestQueuedUs); + } + } + checkState(minLargestQueuedUs != Long.MAX_VALUE); + return minLargestQueuedUs; + } + /** * Called to configure a retry when a load error occurs. * * @param loadable The current loadable for which the error was encountered. - * @param currentExtractedSampleCount The current number of samples that have been extracted into + * @param currentExtractedSamplesCount The current number of samples that have been extracted into * the sample queues. * @return Whether the loader should retry with the current loadable. False indicates a deferred * retry. */ - private boolean configureRetry(ExtractingLoadable loadable, int currentExtractedSampleCount) { - if (isLengthKnown || (seekMap != null && seekMap.getDurationUs() != C.TIME_UNSET)) { - // We're playing an on-demand stream. Resume the current loadable, which will - // request data starting from the point it left off. - extractedSamplesCountAtStartOfLoad = currentExtractedSampleCount; - return true; - } else if (prepared && !suppressRead()) { - // We're playing a stream of unknown length and duration. Assume it's live, and therefore that - // the data at the uri is a continuously shifting window of the latest available media. For - // this case there's no way to continue loading from where a previous load finished, so it's - // necessary to load from the start whenever commencing a new load. Deferring the retry until - // we run out of buffered data makes for a much better user experience. See: - // https://github.com/google/ExoPlayer/issues/1606. - // Note that the suppressRead() check means only a single deferred retry can occur without - // progress being made. Any subsequent failures without progress will go through the else - // block below. - pendingDeferredRetry = true; - return false; - } else { - // This is the same case as above, except in this case there's no value in deferring the retry - // because there's no buffered data to be read. This case also covers an on-demand stream with - // unknown length that has yet to be prepared. This case cannot be disambiguated from the live - // stream case, so we have no option but to load from the start. - notifyDiscontinuity = prepared; - lastSeekPositionUs = 0; - extractedSamplesCountAtStartOfLoad = 0; + private boolean configureRetry(ExtractingLoadable loadable, int currentExtractedSamplesCount) { + boolean isLengthKnownOrHasDuration = + isLengthKnown || (seekMap != null && seekMap.getDurationUs() != C.TIME_UNSET); + loadingStateMachine.onLoadError( + isLengthKnownOrHasDuration, prepared, currentExtractedSamplesCount); + boolean retry = !loadingStateMachine.isDeferredRetryPending(); + if (retry && !isLengthKnownOrHasDuration) { for (SampleQueue sampleQueue : sampleQueues) { sampleQueue.reset(); } loadable.setLoadPosition(0, 0); - return true; } + return retry; } /** @@ -1171,13 +1191,29 @@ private boolean seekInsideBufferUs( continue; } - if (sampleQueue.getReadIndex() == 0 && isSameAsLastSeekPosition) { + if (isSingleSample) { + // Single sample media (like images) only have one sample, so seeking always succeeds + // by resetting to the first sample. + sampleQueue.seekTo(sampleQueue.getFirstIndex()); + continue; + } + long firstTimestampUs = sampleQueue.getFirstTimestampUs(); + if (firstTimestampUs != Long.MIN_VALUE + && positionUs <= firstTimestampUs + && sampleQueue.getFirstIndex() == 0 + && isSameAsLastSeekPosition) { + // If the seek target position is at or before the timestamp of the very first queued + // sample, + // and all samples loaded from the beginning of the stream are still present, reset to the + // first sample by index. This avoids falling through to a timestamp-based seek, which would + // reject the seek if the target position is strictly smaller than the first sample's + // timestamp (e.g., when seeking to time 0 on an audio track with encoder delay). + sampleQueue.seekTo(sampleQueue.getFirstIndex()); continue; } boolean seekInsideQueue = - isSingleSample - ? sampleQueue.seekTo(sampleQueue.getFirstIndex()) - : sampleQueue.seekTo(positionUs, /* allowTimeBeyondBuffer= */ loadingFinished); + sampleQueue.seekTo( + positionUs, /* allowTimeBeyondBuffer= */ loadingStateMachine.isFinished()); // If we have AV tracks then an in-buffer seek is successful if the seek into every AV queue // is successful. We ignore whether seeks within non-AV queues are successful in this case, as // they may be sparse or poorly interleaved. If we only have non-AV tracks then a seek is @@ -1208,10 +1244,6 @@ private long getLargestQueuedTimestampUs(boolean includeDisabledTracks) { return largestQueuedTimestampUs; } - private boolean isPendingReset() { - return pendingResetPositionUs != C.TIME_UNSET; - } - @EnsuresNonNull({"trackState", "seekMap"}) private void assertPrepared() { checkState(prepared); @@ -1219,6 +1251,43 @@ private void assertPrepared() { checkNotNull(seekMap); } + private static boolean[] getTracksSelectedForLoading( + boolean[] trackEnabledStates, TrackGroupArray tracks) { + boolean[] selectedForLoading = new boolean[trackEnabledStates.length]; + for (int i = 0; i < selectedForLoading.length; i++) { + int type = tracks.get(i).type; + selectedForLoading[i] = + (type != C.TRACK_TYPE_VIDEO && type != C.TRACK_TYPE_IMAGE) || trackEnabledStates[i]; + } + return selectedForLoading; + } + + private boolean trackHasPreroll(int trackIndex, long positionUs) { + if (isSingleSample) { + return false; + } + assertPrepared(); + Format format = trackState.tracks.get(trackIndex).getFormat(0); + if (SampleQueue.isDiscardingAllSamplesToStartTime(format)) { + return false; + } + if (seekMap.isSeekable()) { + int extractorTrackId = sampleQueueTrackIds[trackIndex].id; + SeekPoints seekPoints; + if (seekMap instanceof TrackAwareSeekMap) { + TrackAwareSeekMap trackAwareSeekMap = (TrackAwareSeekMap) seekMap; + seekPoints = + trackAwareSeekMap.isSeekable(extractorTrackId) + ? trackAwareSeekMap.getSeekPoints(positionUs, extractorTrackId) + : seekMap.getSeekPoints(positionUs); + } else { + seekPoints = seekMap.getSeekPoints(positionUs); + } + return seekPoints.first.timeUs < positionUs; + } + return false; + } + private final class SampleStreamImpl implements SampleStream { private final int track; @@ -1351,8 +1420,9 @@ public void load() throws IOException { long currentInputPosition = progressiveMediaExtractor.getCurrentInputPosition(); if (currentInputPosition > position + continueLoadingCheckIntervalBytes) { position = currentInputPosition; - loadCondition.close(); - handler.post(onContinueLoadingRequestedRunnable); + if (loadCondition.close()) { + handler.post(onContinueLoadingRequestedRunnable); + } } } } finally { @@ -1418,10 +1488,11 @@ private void setLoadPosition(long position, long timeUs) { /** Stores track state. */ private static final class TrackState { - public final TrackGroupArray tracks; - public final boolean[] trackIsAudioVideoFlags; - public final boolean[] trackEnabledStates; - public final boolean[] trackNotifiedDownstreamFormats; + private final TrackGroupArray tracks; + private final boolean[] trackIsAudioVideoFlags; + private final boolean[] trackEnabledStates; + private final boolean[] trackNotifiedDownstreamFormats; + private boolean hasEnabledAudioVideoTracks; public TrackState(TrackGroupArray tracks, boolean[] trackIsAudioVideoFlags) { this.tracks = tracks; @@ -1487,26 +1558,13 @@ private static int getTrackTypePriority(@C.TrackType int trackType) { private static class ControlledTrackOutput extends ForwardingTrackOutput { private final SampleQueue sampleQueue; private final DiscardingTrackOutput discardingTrackOutput; - private final AtomicReference outputMode; - - /** The mode of operation for the controlled track output. */ - static enum OutputMode { - /** Pass samples through to the downstream track output. */ - PASS_THROUGH, - /** - * Pass samples through to the downstream track output, but discard after the next sample - * metadata is received. - */ - DISCARD_AFTER_NEXT_SAMPLE_METADATA, - /** Discards all samples. */ - DISCARDING, - }; + private final AtomicBoolean isDiscarding; ControlledTrackOutput(SampleQueue sampleQueue) { super(sampleQueue); this.sampleQueue = sampleQueue; this.discardingTrackOutput = new DiscardingTrackOutput(); - this.outputMode = new AtomicReference<>(OutputMode.PASS_THROUGH); + this.isDiscarding = new AtomicBoolean(false); } @Override @@ -1540,10 +1598,6 @@ public void sampleMetadata( int offset, @Nullable CryptoData cryptoData) { getCurrentOutput().sampleMetadata(timeUs, flags, size, offset, cryptoData); - if (outputMode.get() == OutputMode.DISCARD_AFTER_NEXT_SAMPLE_METADATA) { - sampleQueue.reset(); - outputMode.set(OutputMode.DISCARDING); - } } /** @@ -1552,10 +1606,7 @@ public void sampleMetadata( * @param selected Whether the track is selected. */ void updateSelectionState(boolean selected) { - // Since there could still be some samples within the internal SampleDataQueue, we will be - // confident about releasing them all after the next sample metadata is received. - outputMode.set( - selected ? OutputMode.PASS_THROUGH : OutputMode.DISCARD_AFTER_NEXT_SAMPLE_METADATA); + isDiscarding.set(!selected); // In case the existing samples are taking too much memory, preventing further load, release // them optimistically. if (!selected) { @@ -1565,11 +1616,386 @@ void updateSelectionState(boolean selected) { /** Returns whether the track is selected. */ boolean isSelected() { - return outputMode.get() == OutputMode.PASS_THROUGH; + return !isDiscarding.get(); } private TrackOutput getCurrentOutput() { - return outputMode.get() == OutputMode.DISCARDING ? discardingTrackOutput : sampleQueue; + return isDiscarding.get() ? discardingTrackOutput : sampleQueue; + } + } + + /** State machine managing the loading state of a {@link ProgressiveMediaPeriod}. */ + @VisibleForTesting + /* package */ static final class LoadingStateMachine { + + /** State of the loading state machine. */ + @Documented + @Target(ElementType.TYPE_USE) + @Retention(RetentionPolicy.SOURCE) + @IntDef({ + STATE_IDLE, + STATE_LOADING, + STATE_CANCELING, + STATE_FINISHED, + STATE_DEFERRED_RETRY_PENDING, + STATE_ERROR, + STATE_CANCELING_FOR_CLIPPING, + STATE_CLIPPED_FINISHED, + }) + public @interface State {} + + /** + * The loader is idle (not currently extracting data), and no load error is pending deferred + * retry. + */ + public static final int STATE_IDLE = 1; + + /** An {@link ExtractingLoadable} is currently being loaded by the {@link Loader}. */ + public static final int STATE_LOADING = 2; + + /** + * A load is being canceled (e.g. due to an out-of-buffer seek or track deselection). Once the + * load is canceled, state will return to {@link #STATE_IDLE}. + */ + public static final int STATE_CANCELING = 3; + + /** Loading has finished because end-of-file (EOF) or the duration/end position was reached. */ + public static final int STATE_FINISHED = 4; + + /** + * A load error occurred on a live / unknown-length stream while buffered data remains. + * Re-loading is deferred until the buffered data is consumed. + */ + public static final int STATE_DEFERRED_RETRY_PENDING = 5; + + /** A fatal load error occurred that cannot be retried. */ + public static final int STATE_ERROR = 6; + + /** + * An active load is being canceled specifically to trim upstream buffers after the clip end + * position was shortened ({@code endPositionUs < oldEndPositionUs}). + */ + public static final int STATE_CANCELING_FOR_CLIPPING = 7; + + /** + * Loading finished because the clip end position ({@code endPositionUs}) was reached or the + * buffer was trimmed to the clip end position. + */ + public static final int STATE_CLIPPED_FINISHED = 8; + + private @State int state; + + /** + * The last seek position in microseconds, or 0 if reset by a live-stream or deferred retry. + * Returned as the discontinuity position by {@link #readDiscontinuity(int)}. + */ + private long lastSeekPositionUs; + + private long pendingResetPositionUs; + private int extractedSamplesCountAtStartOfLoad; + private boolean notifyDiscontinuity; + private boolean pendingInitialDiscontinuity; + private boolean seenFirstTrackSelection; + private boolean usesStreamPrerollFlags; + + /** Creates an instance. */ + public LoadingStateMachine() { + state = STATE_IDLE; + pendingResetPositionUs = C.TIME_UNSET; + } + + /** Returns the current {@link State}. */ + @VisibleForTesting + public @State int getState() { + return state; + } + + /** Returns whether loading has finished (e.g. EOF or end position reached). */ + public boolean isFinished() { + return state == STATE_FINISHED || state == STATE_CLIPPED_FINISHED; + } + + /** Returns whether loading has finished specifically due to clipping (end position reached). */ + public boolean isClippedFinished() { + return state == STATE_CLIPPED_FINISHED; + } + + /** Returns whether an active load is being canceled specifically for end position clipping. */ + public boolean isCancelingForClipping() { + return state == STATE_CANCELING_FOR_CLIPPING; + } + + /** Returns whether a load error is pending a deferred retry. */ + public boolean isDeferredRetryPending() { + return state == STATE_DEFERRED_RETRY_PENDING; + } + + /** Returns whether the given seek position matches the last requested seek position. */ + public boolean isLastSeekPosition(long positionUs) { + return lastSeekPositionUs == positionUs; + } + + /** + * Returns whether progress (more extracted samples) has been made since the current load task + * started. + * + * @param currentExtractedSamplesCount Current total sample count across all queues. + */ + public boolean hasExtractedProgressSinceLoadStart(int currentExtractedSamplesCount) { + return currentExtractedSamplesCount > extractedSamplesCountAtStartOfLoad; + } + + /** + * Returns whether a load reset (to {@link #getPendingResetPositionUs()}) is currently pending. + * + *

    A reset is only pending while in {@link #STATE_IDLE} or {@link #STATE_CANCELING}, awaiting + * a new load task to start. + */ + public boolean isPendingReset() { + return pendingResetPositionUs != C.TIME_UNSET; + } + + /** Returns the pending reset position in microseconds, or {@link C#TIME_UNSET} if none. */ + public long getPendingResetPositionUs() { + return pendingResetPositionUs; + } + + /** + * Returns the last seek position in microseconds (or {@code 0} if reset by a live-stream or + * deferred retry). + */ + public long getLastSeekPositionUs() { + return lastSeekPositionUs; + } + + /** Returns the number of extracted samples recorded when the current load started. */ + public int getExtractedSamplesCountAtStartOfLoad() { + return extractedSamplesCountAtStartOfLoad; + } + + /** Configures whether sample streams manage their own preroll flags. */ + public void setUsesStreamPrerollFlags() { + this.usesStreamPrerollFlags = true; + } + + /** Returns whether track selection has occurred at least once for this period. */ + public boolean hasSeenFirstTrackSelection() { + return seenFirstTrackSelection; + } + + /** + * Returns whether sample reading should be suppressed due to pending reset or discontinuity. + */ + public boolean suppressRead() { + return notifyDiscontinuity || isPendingReset(); + } + + /** Returns whether an active load is currently being canceled. */ + public boolean isCanceling() { + return state == STATE_CANCELING || state == STATE_CANCELING_FOR_CLIPPING; + } + + /** Returns whether the state machine is in a loading or canceling state. */ + public boolean isLoading() { + return state == STATE_LOADING || isCanceling(); + } + + /** Returns whether seeking inside the sample queues buffer is permitted. */ + public boolean canSeekInsideBuffer(@C.DataType int dataType) { + return !isPendingReset() + && dataType != C.DATA_TYPE_MEDIA_PROGRESSIVE_LIVE + && (isFinished() + || state == STATE_IDLE + || state == STATE_LOADING + || state == STATE_CANCELING_FOR_CLIPPING + || state == STATE_DEFERRED_RETRY_PENDING); + } + + /** Returns whether loading can be started or continued by the caller. */ + public boolean canContinueLoading(boolean isPreparedOrSingleTrack, int enabledTrackCount) { + return (state == STATE_IDLE || state == STATE_LOADING) + && (!isPreparedOrSingleTrack || enabledTrackCount > 0); + } + + /** Evaluates and returns a pending discontinuity position, or {@link C#TIME_UNSET} if none. */ + public long readDiscontinuity(int currentExtractedSamplesCount) { + if (!usesStreamPrerollFlags && pendingInitialDiscontinuity) { + pendingInitialDiscontinuity = false; + return lastSeekPositionUs; + } + if (notifyDiscontinuity + && (isFinished() || currentExtractedSamplesCount > extractedSamplesCountAtStartOfLoad)) { + notifyDiscontinuity = false; + return lastSeekPositionUs; + } + return C.TIME_UNSET; + } + + // --- State Transitions & Events --- + + /** + * Called when track selections are updated. + * + * @param hasPreroll Whether any selected track has preroll samples. + * @param enabledTrackCount Number of currently enabled tracks. + */ + public void onTrackSelection(boolean hasPreroll, int enabledTrackCount) { + if (enabledTrackCount == 0) { + notifyDiscontinuity = false; + pendingInitialDiscontinuity = false; + if (state == STATE_LOADING + || state == STATE_CANCELING + || state == STATE_CANCELING_FOR_CLIPPING) { + state = STATE_CANCELING; + } else { + state = STATE_IDLE; + } + } else if (pendingInitialDiscontinuity || !seenFirstTrackSelection) { + pendingInitialDiscontinuity = hasPreroll; + } + seenFirstTrackSelection = true; + } + + /** Configures initial reset position upon single-track period preparation. */ + public void onPrepared(boolean isSingleTrack, long positionUs) { + if (isSingleTrack) { + pendingResetPositionUs = positionUs; + } + } + + /** + * Called when a load attempt is initiated. + * + * @param isPrepared Whether the period is prepared. + * @param endPositionUs The clip end position in microseconds, or {@link C#TIME_END_OF_SOURCE} + * if not clipped. + * @param durationUs The stream duration in microseconds, or {@link C#TIME_UNSET} if unknown. + * @param currentExtractedSamplesCount Current total sample count across all queues. + */ + public void onStartLoading( + boolean isPrepared, long endPositionUs, long durationUs, int currentExtractedSamplesCount) { + if (isPrepared && isPendingReset()) { + boolean isClipped = endPositionUs != C.TIME_END_OF_SOURCE; + long maxLoadPositionUs = isClipped ? endPositionUs : durationUs; + if (maxLoadPositionUs != C.TIME_UNSET && pendingResetPositionUs > maxLoadPositionUs) { + pendingResetPositionUs = C.TIME_UNSET; + state = isClipped ? STATE_CLIPPED_FINISHED : STATE_FINISHED; + return; + } + } + extractedSamplesCountAtStartOfLoad = currentExtractedSamplesCount; + pendingResetPositionUs = C.TIME_UNSET; + state = STATE_LOADING; + } + + /** Called when a load completes naturally (e.g. reaches EOF). */ + public void onLoadCompleted() { + pendingResetPositionUs = C.TIME_UNSET; + state = STATE_FINISHED; + } + + /** Called when an active load is canceled by the {@link Loader}. */ + public void onLoadCanceled(boolean released, boolean haveSampleQueuesReachedEndTimeUs) { + if (!released) { + if (state == STATE_CANCELING_FOR_CLIPPING) { + state = + haveSampleQueuesReachedEndTimeUs && !isPendingReset() + ? STATE_CLIPPED_FINISHED + : STATE_IDLE; + } else if (state == STATE_CANCELING) { + state = STATE_IDLE; + } + } + } + + /** Called when a fatal load error occurs and loading will not be retried. */ + public void onFatalLoadError() { + state = STATE_ERROR; + } + + /** Evaluates state transition upon load error. */ + public void onLoadError( + boolean isLengthKnownOrHasDuration, boolean isPrepared, int currentExtractedSamplesCount) { + if (isLengthKnownOrHasDuration) { + // We're playing an on-demand stream. Resume the current loadable, which will + // request data starting from the point it left off. + extractedSamplesCountAtStartOfLoad = currentExtractedSamplesCount; + state = STATE_LOADING; + } else if (isPrepared && !suppressRead()) { + // We're playing a stream of unknown length and duration. Assume it's live, and therefore + // that the chance of the loadable managing to recover when retried immediately is low. + // Defer retrying until the buffer has been depleted. + state = STATE_DEFERRED_RETRY_PENDING; + } else { + // We're playing a stream of unknown length and duration, and the buffer is empty. + // Assume it's live, and that starting a new loadable (which will start from the + // beginning of the live stream) is the best option. + notifyDiscontinuity = isPrepared; + lastSeekPositionUs = 0; + extractedSamplesCountAtStartOfLoad = 0; + state = STATE_LOADING; + } + } + + /** Called when a deferred retry is initiated upon buffer exhaustion. */ + public void onDeferredRetryStarted() { + pendingResetPositionUs = 0; + notifyDiscontinuity = true; + lastSeekPositionUs = 0; + extractedSamplesCountAtStartOfLoad = 0; + state = STATE_IDLE; + } + + /** + * Called when a seek is requested. + * + * @param positionUs The seek target position in microseconds. + * @param canSeekInsideBuffer Whether in-buffer seek succeeded on sample queues. + */ + public void onSeek(long positionUs, boolean canSeekInsideBuffer) { + notifyDiscontinuity = false; + lastSeekPositionUs = positionUs; + + if (isPendingReset()) { + pendingResetPositionUs = positionUs; + return; + } + + if (canSeekInsideBuffer) { + return; + } + + pendingResetPositionUs = positionUs; + pendingInitialDiscontinuity = false; + if (state == STATE_LOADING + || state == STATE_CANCELING + || state == STATE_CANCELING_FOR_CLIPPING) { + state = STATE_CANCELING; + } else { + state = STATE_IDLE; + } + } + + /** Evaluates state transitions upon reaching the clip end position. */ + public void onEndPositionReached() { + if (state == STATE_CANCELING + || state == STATE_CANCELING_FOR_CLIPPING + || state == STATE_ERROR + || isPendingReset()) { + return; + } + if (state == STATE_LOADING) { + state = STATE_CANCELING_FOR_CLIPPING; + } else { + state = STATE_CLIPPED_FINISHED; + } + } + + /** Evaluates state transitions when the clip end position is extended. */ + public void onEndPositionExtended() { + if (state == STATE_CLIPPED_FINISHED) { + state = STATE_IDLE; + } } } } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ProgressiveMediaSource.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ProgressiveMediaSource.java index 87aaee8e097..c05d3c79169 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ProgressiveMediaSource.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ProgressiveMediaSource.java @@ -96,7 +96,6 @@ public static final class Factory implements MediaSourceFactory { private int singleTrackId; @Nullable private Format singleTrackFormat; private boolean loadOnlySelectedTracks; - private boolean experimentalEnableHagcPlayback; /** * Creates a new factory for {@link ProgressiveMediaSource}s. @@ -187,19 +186,7 @@ public Factory( this.drmSessionManagerProvider = drmSessionManagerProvider; this.loadErrorHandlingPolicy = loadErrorHandlingPolicy; this.continueLoadingCheckIntervalBytes = continueLoadingCheckIntervalBytes; - this.experimentalEnableHagcPlayback = true; - } - - /** - * Sets whether to enable experimental HAGC (ST 2094-50) metadata playback support. - * - * @param enableHagcPlayback Whether experimental HAGC metadata playback is enabled. - * @return This factory, for convenience. - */ - @CanIgnoreReturnValue - public Factory setExperimentalEnableHagcPlayback(boolean enableHagcPlayback) { - this.experimentalEnableHagcPlayback = enableHagcPlayback; - return this; + this.loadOnlySelectedTracks = true; } @CanIgnoreReturnValue @@ -287,10 +274,21 @@ public MediaSource.Factory setDownloadExecutor(Supplier down } /** - * Sets whether to load only the tracks selected by the track selection policy. + * Sets whether to load only the video and image tracks selected by the track selection policy. + * + *

    Audio, text, and metadata tracks are always loaded regardless of track selection, as they + * have relatively low bitrate and are more likely to be toggled mid-playback (e.g., changing + * audio languages or subtitle tracks, or disabling audio during scrubbing mode). Always loading + * them allows seamlessly switching mid-playback without re-buffering or seeking. + * + *

    Note that while this saves memory, it may cause additional buffering or latency if + * unselected video or image tracks are enabled mid-playback, as the player may need to perform + * additional network requests to fetch the newly selected track data. + * + *

    The default is {@code true}. * - * @param loadOnlySelectedTracks Whether to load only the tracks selected by the track selection - * policy, instead of loading all tracks. + * @param loadOnlySelectedTracks Whether to load only the video and image tracks selected by the + * track selection policy, instead of loading all tracks. * @return This factory, for convenience. */ @CanIgnoreReturnValue @@ -317,7 +315,6 @@ public ProgressiveMediaSource createMediaSource(MediaItem mediaItem) { loadErrorHandlingPolicy, continueLoadingCheckIntervalBytes, loadOnlySelectedTracks, - experimentalEnableHagcPlayback, singleTrackId, singleTrackFormat, downloadExecutorSupplier); @@ -341,7 +338,6 @@ public ProgressiveMediaSource createMediaSource(MediaItem mediaItem) { private final LoadErrorHandlingPolicy loadableLoadErrorHandlingPolicy; private final int continueLoadingCheckIntervalBytes; private final boolean loadOnlySelectedTracks; - private final boolean experimentalEnableHagcPlayback; /** * The ID passed to {@link Factory#enableLazyLoadingWithSingleTrack(int, Format)}. Only valid if @@ -377,7 +373,6 @@ private ProgressiveMediaSource( LoadErrorHandlingPolicy loadableLoadErrorHandlingPolicy, int continueLoadingCheckIntervalBytes, boolean loadOnlySelectedTracks, - boolean experimentalEnableHagcPlayback, int singleTrackId, @Nullable Format singleTrackFormat, @Nullable Supplier downloadExecutorSupplier) { @@ -388,7 +383,6 @@ private ProgressiveMediaSource( this.loadableLoadErrorHandlingPolicy = loadableLoadErrorHandlingPolicy; this.continueLoadingCheckIntervalBytes = continueLoadingCheckIntervalBytes; this.loadOnlySelectedTracks = loadOnlySelectedTracks; - this.experimentalEnableHagcPlayback = experimentalEnableHagcPlayback; this.singleTrackFormat = singleTrackFormat; this.singleTrackId = singleTrackId; this.timelineIsPlaceholder = true; @@ -450,7 +444,6 @@ public MediaPeriod createPeriod(MediaPeriodId id, Allocator allocator, long star localConfiguration.customCacheKey, continueLoadingCheckIntervalBytes, loadOnlySelectedTracks, - experimentalEnableHagcPlayback, singleTrackId, singleTrackFormat, Util.msToUs(localConfiguration.imageDurationMs), diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/SampleQueue.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/SampleQueue.java index 4b3edc50522..30da88dceab 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/SampleQueue.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/SampleQueue.java @@ -78,6 +78,8 @@ public interface UpstreamFormatChangedListener { */ private static final int DEFAULT_MAX_NUM_REORDER_SAMPLES = 16; + private static final long EXPECT_RESUME_MAX_TIMESTAMP_DRIFT_US = 1_000; + private final SampleDataQueue sampleDataQueue; private final SampleExtrasHolder extrasHolder; private final SpannedData sharedSampleMetadata; @@ -125,6 +127,9 @@ public interface UpstreamFormatChangedListener { private long sampleOffsetUs; private boolean pendingSplice; + private boolean pendingResume; + private long resumeEndTimestampUs; + private int resumeEndDuplicateCount; /** * Creates a sample queue without DRM resource management. @@ -172,6 +177,20 @@ public static SampleQueue createWithDrm( allocator, checkNotNull(drmSessionManager), checkNotNull(drmEventDispatcher)); } + /** + * Returns whether samples earlier than {@linkplain #setStartTimeUs the start time} will be + * discarded for the given format. + * + *

    This returns {@code true} for audio formats where {@linkplain + * MimeTypes#allSamplesAreSyncSamples all samples are sync samples}. + * + * @param format The {@link Format} to check. + * @return Whether all samples earlier than the start time will be discarded for the given format. + */ + public static boolean isDiscardingAllSamplesToStartTime(Format format) { + return canDiscardAllSamplesToStartTime(format.sampleMimeType, format.codecs); + } + protected SampleQueue( Allocator allocator, @Nullable DrmSessionManager drmSessionManager, @@ -236,6 +255,7 @@ public void reset(boolean resetUpstreamFormat) { largestDiscardedTimestampUs = Long.MIN_VALUE; largestQueuedTimestampUs = Long.MIN_VALUE; isLastSampleQueued = false; + pendingResume = false; sharedSampleMetadata.clear(); if (resetUpstreamFormat) { unadjustedUpstreamFormat = null; @@ -246,9 +266,9 @@ public void reset(boolean resetUpstreamFormat) { } /** - * Sets the start time for the queue. Samples with earlier timestamps will be discarded for audio - * formats if {@linkplain MimeTypes#allSamplesAreSyncSamples all samples are sync samples} in the - * given input format. + * Sets the start time for the queue. Samples with earlier timestamps will be discarded if + * {@linkplain #isDiscardingAllSamplesToStartTime(Format) discarding to start time is supported} + * for the input format. * * @param startTimeUs The start time, in microseconds. */ @@ -289,6 +309,34 @@ public final void splice() { pendingSplice = true; } + /** + * Prepares the queue to resume receiving samples from an earlier timestamp, matching duplicate + * overlap samples at the existing queue tail before appending new samples. + */ + public final void expectResume() { + if (length > 0) { + upstreamKeyframeRequired = false; + int relativeTailIndex = getRelativeIndex(length - 1); + resumeEndTimestampUs = timesUs[relativeTailIndex]; + + int duplicateCount = 0; + for (int i = length - 1; i >= 0; i--) { + int index = getRelativeIndex(i); + if (Math.abs(timesUs[index] - resumeEndTimestampUs) + <= EXPECT_RESUME_MAX_TIMESTAMP_DRIFT_US) { + duplicateCount++; + } else { + break; + } + } + resumeEndDuplicateCount = duplicateCount; + pendingResume = true; + } else { + pendingResume = false; + resumeEndDuplicateCount = 0; + } + } + /** Returns the current absolute write index. */ public final int getWriteIndex() { return absoluteFirstIndex + length; @@ -522,8 +570,9 @@ public final synchronized boolean seekTo(int sampleIndex) { /** * Attempts to seek the read position to the keyframe before or at the specified time. * - *

    For audio formats where {@linkplain MimeTypes#allSamplesAreSyncSamples all samples are sync - * samples}, it seeks the read position to the first sample at or after the specified time. + *

    For formats where {@linkplain #isDiscardingAllSamplesToStartTime(Format) discarding to start + * time is supported}, it seeks the read position to the first sample at or after the specified + * time. * * @param timeUs The time to seek to. * @param allowTimeBeyondBuffer Whether the operation can succeed if {@code timeUs} is beyond the @@ -679,17 +728,42 @@ public void sampleMetadata( format(checkNotNull(unadjustedUpstreamFormat)); } + timeUs += sampleOffsetUs; + + if (pendingResume) { + boolean isDuplicateMatch = + Math.abs(timeUs - resumeEndTimestampUs) <= EXPECT_RESUME_MAX_TIMESTAMP_DRIFT_US; + if (isDuplicateMatch) { + if (resumeEndDuplicateCount > 1) { + resumeEndDuplicateCount--; + } else { + pendingResume = false; + } + discardUpstreamSampleBytes(size, offset); + return; + } + + boolean isKeyframe = (flags & C.BUFFER_FLAG_KEY_FRAME) != 0; + if (isKeyframe && timeUs > largestQueuedTimestampUs + EXPECT_RESUME_MAX_TIMESTAMP_DRIFT_US) { + pendingResume = false; + } else { + discardUpstreamSampleBytes(size, offset); + return; + } + } + boolean isKeyframe = (flags & C.BUFFER_FLAG_KEY_FRAME) != 0; if (upstreamKeyframeRequired) { if (!isKeyframe) { + discardUpstreamSampleBytes(size, offset); return; } upstreamKeyframeRequired = false; } - timeUs += sampleOffsetUs; if (discardAllSamplesToStartTime) { if (timeUs < startTimeUs) { + discardUpstreamSampleBytes(size, offset); return; } if ((flags & C.BUFFER_FLAG_KEY_FRAME) == 0) { @@ -704,6 +778,7 @@ public void sampleMetadata( } if (pendingSplice) { if (!isKeyframe || !attemptSplice(timeUs)) { + discardUpstreamSampleBytes(size, offset); return; } pendingSplice = false; @@ -745,6 +820,12 @@ protected Format getAdjustedUpstreamFormat(Format format) { // Internal methods. + private void discardUpstreamSampleBytes(int size, int offset) { + if (offset == 0) { + sampleDataQueue.discardUpstreamSampleBytes(sampleDataQueue.getTotalBytesWritten() - size); + } + } + /** Rewinds the read position to the first sample in the queue. */ private synchronized void rewind() { readPosition = 0; @@ -1240,13 +1321,13 @@ private int getRelativeIndex(int offset) { } private static boolean canDiscardAllSamplesToStartTime( - @Nullable String mimeType, @Nullable String codec) { + @Nullable String mimeType, @Nullable String codecs) { // We can only discard up to the start time immediately if the samples are guaranteed to be all // sync samples. This optimization is also only possible for audio tracks where the inherent // duration of a sample is negligible, and it doesn't affect playback if a partial sample is // dropped. @C.TrackType int trackType = MimeTypes.getTrackType(mimeType); - return trackType == C.TRACK_TYPE_AUDIO && MimeTypes.allSamplesAreSyncSamples(mimeType, codec); + return trackType == C.TRACK_TYPE_AUDIO && MimeTypes.allSamplesAreSyncSamples(mimeType, codecs); } /** A holder for sample metadata not held by {@link DecoderInputBuffer}. */ diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/SideloadedSubtitlesMediaSource.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/SideloadedSubtitlesMediaSource.java new file mode 100644 index 00000000000..6389f7a4d56 --- /dev/null +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/SideloadedSubtitlesMediaSource.java @@ -0,0 +1,168 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.exoplayer.source; + +import static com.google.common.base.Preconditions.checkArgument; +import static com.google.common.base.Preconditions.checkNotNull; + +import androidx.annotation.Nullable; +import androidx.media3.common.MediaItem; +import androidx.media3.exoplayer.upstream.Allocator; +import com.google.common.collect.ImmutableList; +import java.util.ArrayList; +import java.util.List; + +/** + * A {@link MediaSource} merging a content source with its sideloaded subtitle sources, applying the + * {@linkplain MediaItem.SubtitleConfiguration.Builder#setTimeOffsetUs(long) time offsets} of the + * subtitle configurations and handling {@link MediaItem} updates that change them. + * + *

    Time offset changes are forwarded to the affected subtitle sources without interrupting + * playback. Updates that change the subtitle configurations in any other way are rejected from + * {@link #canUpdateMediaItem}, so that the player falls back to re-preparing the item. + */ +/* package */ final class SideloadedSubtitlesMediaSource extends WrappingMediaSource { + + private final TimeOffsetMediaSource[] subtitleSources; + + /** + * Creates the media source. + * + * @param contentMediaSource The content {@link MediaSource}. + * @param subtitleConfigurations The {@link MediaItem.SubtitleConfiguration} instances of the + * sideloaded subtitles. + * @param subtitleMediaSources The sideloaded subtitle {@link MediaSource} instances, in the same + * order as {@code subtitleConfigurations}. + */ + public SideloadedSubtitlesMediaSource( + MediaSource contentMediaSource, + List subtitleConfigurations, + MediaSource[] subtitleMediaSources) { + this(createTimeOffsetSources(subtitleConfigurations, subtitleMediaSources), contentMediaSource); + } + + private SideloadedSubtitlesMediaSource( + TimeOffsetMediaSource[] subtitleSources, MediaSource contentMediaSource) { + super(createMergingMediaSource(contentMediaSource, subtitleSources)); + this.subtitleSources = subtitleSources; + } + + @Override + public boolean canUpdateMediaItem(MediaItem mediaItem) { + return super.canUpdateMediaItem(mediaItem) && canUpdateSubtitleConfigurations(mediaItem); + } + + @Override + public void updateMediaItem(MediaItem mediaItem) { + super.updateMediaItem(mediaItem); + ImmutableList newSubtitleConfigurations = + checkNotNull(mediaItem.localConfiguration).subtitleConfigurations; + for (int i = 0; i < subtitleSources.length; i++) { + subtitleSources[i].setTimeOffsetUs(newSubtitleConfigurations.get(i).timeOffsetUs); + } + } + + private boolean canUpdateSubtitleConfigurations(MediaItem mediaItem) { + @Nullable MediaItem.LocalConfiguration localConfiguration = mediaItem.localConfiguration; + @Nullable + MediaItem.LocalConfiguration currentLocalConfiguration = getMediaItem().localConfiguration; + if (localConfiguration == null + || currentLocalConfiguration == null + || localConfiguration.subtitleConfigurations.size() + != currentLocalConfiguration.subtitleConfigurations.size()) { + return false; + } + for (int i = 0; i < currentLocalConfiguration.subtitleConfigurations.size(); i++) { + if (!equalsIgnoringTimeOffset( + localConfiguration.subtitleConfigurations.get(i), + currentLocalConfiguration.subtitleConfigurations.get(i))) { + return false; + } + } + return true; + } + + private static boolean equalsIgnoringTimeOffset( + MediaItem.SubtitleConfiguration subtitleConfiguration, + MediaItem.SubtitleConfiguration other) { + return subtitleConfiguration + .buildUpon() + .setTimeOffsetUs(other.timeOffsetUs) + .build() + .equals(other); + } + + private static TimeOffsetMediaSource[] createTimeOffsetSources( + List subtitleConfigurations, + MediaSource[] subtitleMediaSources) { + checkArgument(subtitleConfigurations.size() == subtitleMediaSources.length); + TimeOffsetMediaSource[] timeOffsetSources = + new TimeOffsetMediaSource[subtitleMediaSources.length]; + for (int i = 0; i < subtitleMediaSources.length; i++) { + timeOffsetSources[i] = + new TimeOffsetMediaSource( + subtitleMediaSources[i], subtitleConfigurations.get(i).timeOffsetUs); + } + return timeOffsetSources; + } + + private static MergingMediaSource createMergingMediaSource( + MediaSource contentMediaSource, TimeOffsetMediaSource[] subtitleSources) { + MediaSource[] mediaSources = new MediaSource[subtitleSources.length + 1]; + mediaSources[0] = contentMediaSource; + System.arraycopy(subtitleSources, 0, mediaSources, 1, subtitleSources.length); + return new MergingMediaSource(mediaSources); + } + + /** + * A {@link WrappingMediaSource} that applies a time offset to the media periods of the wrapped + * source. + */ + private static final class TimeOffsetMediaSource extends WrappingMediaSource { + + private final List activeMediaPeriods; + private long timeOffsetUs; + + private TimeOffsetMediaSource(MediaSource mediaSource, long timeOffsetUs) { + super(mediaSource); + this.timeOffsetUs = timeOffsetUs; + this.activeMediaPeriods = new ArrayList<>(); + } + + private void setTimeOffsetUs(long timeOffsetUs) { + this.timeOffsetUs = timeOffsetUs; + for (int i = 0; i < activeMediaPeriods.size(); i++) { + activeMediaPeriods.get(i).updateTimeOffsetUs(timeOffsetUs); + } + } + + @Override + public MediaPeriod createPeriod(MediaPeriodId id, Allocator allocator, long startPositionUs) { + TimeOffsetMediaPeriod mediaPeriod = + new TimeOffsetMediaPeriod( + super.createPeriod(id, allocator, startPositionUs - timeOffsetUs), timeOffsetUs); + activeMediaPeriods.add(mediaPeriod); + return mediaPeriod; + } + + @Override + public void releasePeriod(MediaPeriod mediaPeriod) { + TimeOffsetMediaPeriod timeOffsetMediaPeriod = (TimeOffsetMediaPeriod) mediaPeriod; + activeMediaPeriods.remove(timeOffsetMediaPeriod); + super.releasePeriod(timeOffsetMediaPeriod.getWrappedMediaPeriod()); + } + } +} diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/SingleSampleMediaPeriod.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/SingleSampleMediaPeriod.java index 11a9575f173..275bceeed83 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/SingleSampleMediaPeriod.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/SingleSampleMediaPeriod.java @@ -207,19 +207,17 @@ public long getAdjustedSeekPositionUs(long positionUs, SeekParameters seekParame public void onLoadStarted( SourceLoadable loadable, long elapsedRealtimeMs, long loadDurationMs, int retryCount) { StatsDataSource dataSource = loadable.dataSource; - LoadEventInfo loadEventInfo = - retryCount == 0 - ? new LoadEventInfo(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) - : new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - dataSource.getLastOpenedUri(), - dataSource.getLastResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - dataSource.getBytesRead()); + LoadEventInfo.Builder loadEventInfo = + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs); + if (retryCount != 0) { + loadEventInfo + .setUri(dataSource.getLastOpenedUri()) + .setResponseHeaders(dataSource.getLastResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(dataSource.getBytesRead()); + } eventDispatcher.loadStarted( - loadEventInfo, + loadEventInfo.build(), C.DATA_TYPE_MEDIA, C.TRACK_TYPE_UNKNOWN, format, @@ -238,14 +236,12 @@ public void onLoadCompleted( loadingFinished = true; StatsDataSource dataSource = loadable.dataSource; LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - dataSource.getLastOpenedUri(), - dataSource.getLastResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - sampleSize); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(dataSource.getLastOpenedUri()) + .setResponseHeaders(dataSource.getLastResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(sampleSize) + .build(); loadErrorHandlingPolicy.onLoadTaskConcluded(loadable.loadTaskId); eventDispatcher.loadCompleted( loadEventInfo, @@ -263,14 +259,12 @@ public void onLoadCanceled( SourceLoadable loadable, long elapsedRealtimeMs, long loadDurationMs, boolean released) { StatsDataSource dataSource = loadable.dataSource; LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - dataSource.getLastOpenedUri(), - dataSource.getLastResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - dataSource.getBytesRead()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(dataSource.getLastOpenedUri()) + .setResponseHeaders(dataSource.getLastResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(dataSource.getBytesRead()) + .build(); loadErrorHandlingPolicy.onLoadTaskConcluded(loadable.loadTaskId); eventDispatcher.loadCanceled( loadEventInfo, @@ -292,14 +286,12 @@ public LoadErrorAction onLoadError( int errorCount) { StatsDataSource dataSource = loadable.dataSource; LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - dataSource.getLastOpenedUri(), - dataSource.getLastResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - dataSource.getBytesRead()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(dataSource.getLastOpenedUri()) + .setResponseHeaders(dataSource.getLastResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(dataSource.getBytesRead()) + .build(); MediaLoadData mediaLoadData = new MediaLoadData( C.DATA_TYPE_MEDIA, @@ -314,8 +306,7 @@ public LoadErrorAction onLoadError( new LoadErrorInfo(loadEventInfo, mediaLoadData, error, errorCount)); boolean errorCanBePropagated = retryDelay == C.TIME_UNSET - || errorCount - >= loadErrorHandlingPolicy.getMinimumLoadableRetryCount(C.DATA_TYPE_MEDIA); + || errorCount > loadErrorHandlingPolicy.getMinimumLoadableRetryCount(C.DATA_TYPE_MEDIA); LoadErrorAction action; if (treatLoadErrorsAsEndOfStream && errorCanBePropagated) { @@ -324,7 +315,7 @@ public LoadErrorAction onLoadError( action = Loader.DONT_RETRY; } else { action = - retryDelay != C.TIME_UNSET + !errorCanBePropagated ? Loader.createRetryAction(/* resetErrorCount= */ false, retryDelay) : Loader.DONT_RETRY_FATAL; } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/TimeOffsetMediaPeriod.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/TimeOffsetMediaPeriod.java index 9c72c0cd7ba..8b680da99d5 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/TimeOffsetMediaPeriod.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/TimeOffsetMediaPeriod.java @@ -31,12 +31,12 @@ import java.util.List; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; -/** A {@link MediaPeriod} that applies a fixed time offset to all timestamps */ +/** A {@link MediaPeriod} that applies a time offset to all timestamps */ /* package */ final class TimeOffsetMediaPeriod implements MediaPeriod, MediaPeriod.Callback { private final MediaPeriod mediaPeriod; - private final long timeOffsetUs; + private long timeOffsetUs; private @MonotonicNonNull Callback callback; /** @@ -56,6 +56,21 @@ public MediaPeriod getWrappedMediaPeriod() { return mediaPeriod; } + /** + * Updates the offset that is applied to all timestamps coming from the wrapped period. + * + *

    The new offset applies to all future interactions with this period and its {@linkplain + * SampleStream sample streams}. Data already read with the previous offset is unaffected. To + * apply the new offset to the data at the current playback position, the tracks of this period + * need to be re-selected (or seeked) so that the wrapped period re-reads this data. + * + * @param timeOffsetUs The offset to apply to all timestamps coming from the wrapped period, in + * microseconds. + */ + public void updateTimeOffsetUs(long timeOffsetUs) { + this.timeOffsetUs = timeOffsetUs; + } + @Override public void prepare(Callback callback, long positionUs) { this.callback = callback; @@ -102,7 +117,7 @@ public long selectTracks( streams[i] = null; } else if (streams[i] == null || ((TimeOffsetSampleStream) streams[i]).getChildStream() != childStream) { - streams[i] = new TimeOffsetSampleStream(childStream, timeOffsetUs); + streams[i] = new TimeOffsetSampleStream(childStream); } } return startPositionUs + timeOffsetUs; @@ -192,17 +207,15 @@ public long setEndPositionUs(long endPositionUs) { : actualEndPositionUs + timeOffsetUs; } - private static final class TimeOffsetSampleStream implements SampleStream { + private final class TimeOffsetSampleStream implements SampleStream { private final SampleStream sampleStream; - private final long timeOffsetUs; - public TimeOffsetSampleStream(SampleStream sampleStream, long timeOffsetUs) { + private TimeOffsetSampleStream(SampleStream sampleStream) { this.sampleStream = sampleStream; - this.timeOffsetUs = timeOffsetUs; } - public SampleStream getChildStream() { + private SampleStream getChildStream() { return sampleStream; } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ads/AdTimeline.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ads/AdTimeline.java index 5437ac0b909..1d9c516190c 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ads/AdTimeline.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ads/AdTimeline.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2026 The Android Open Source Project + * Copyright 2026 The Android Open Source Project * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -28,11 +28,13 @@ * A custom {@link Timeline} for sources that have {@link AdPlaybackState} split among multiple * periods. * - *

    For each period a modified {@link AdPlaybackState} is created: + *

    The ad group times in the input {@link AdPlaybackState} must be relative to the start of the + * first {@link Timeline.Period} (i.e. {@code timeUs = 0} corresponds to the start of period 0). For + * each period a modified {@link AdPlaybackState} is created: * *

      - *
    • Ad group time is offset relative to period start time - *
    • Post-roll ad group is kept only for last period + *
    • Ad group time is offset relative to the period's start time + *
    • Post-roll ad group is kept only for the last period *
    • Ad group count and indices are kept unchanged *
    */ @@ -47,8 +49,8 @@ public final class AdTimeline extends ForwardingTimeline { * * @param contentTimeline The timeline of the content alongside which ads will be played. * @param adPlaybackState The state of the media's ads. The ad group times in this state must be - * relative to the start of the window (i.e. timeUs = 0 corresponds to the start of the - * window). + * relative to the start of the first {@link Timeline.Period} (i.e. {@code timeUs = 0} + * corresponds to the start of the first period). */ public AdTimeline(Timeline contentTimeline, AdPlaybackState adPlaybackState) { super(contentTimeline); @@ -57,12 +59,14 @@ public AdTimeline(Timeline contentTimeline, AdPlaybackState adPlaybackState) { this.adPlaybackStates = new AdPlaybackState[periodCount]; Timeline.Period period = new Timeline.Period(); + timeline.getPeriod(0, period); + long period0PositionInWindowUs = period.positionInWindowUs; for (int periodIndex = 0; periodIndex < periodCount; periodIndex++) { timeline.getPeriod(periodIndex, period); adPlaybackStates[periodIndex] = forPeriod( adPlaybackState, - period.positionInWindowUs, + period.positionInWindowUs - period0PositionInWindowUs, period.durationUs, /* isLastPeriod= */ periodIndex == periodCount - 1); } @@ -82,7 +86,8 @@ public Period getPeriod(int periodIndex, Period period, boolean setIds) { durationUs, period.getPositionInWindowUs(), adPlaybackStates[periodIndex], - period.isPlaceholder); + period.isPlaceholder, + period.originalDurationUs); return period; } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ads/AdsLoader.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ads/AdsLoader.java index 5d98c8223e4..25f2ee937e8 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ads/AdsLoader.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ads/AdsLoader.java @@ -77,7 +77,9 @@ interface EventListener { * AdPlaybackState#adGroupCount ad groups} may not change after the first call. * *

    The ad group times in the {@link AdPlaybackState} must be relative to the start of the - * timeline window (i.e. timeUs = 0 corresponds to the start of the window). + * first {@link Timeline.Period} of the content timeline (i.e. {@code timeUs = 0} corresponds to + * the start of the first period). For single-period content, this corresponds to the start of + * the period. * * @param adPlaybackState The new ad playback state. */ diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ads/AdsMediaSource.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ads/AdsMediaSource.java index fb67b28a619..64a3bc9c147 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ads/AdsMediaSource.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ads/AdsMediaSource.java @@ -64,8 +64,10 @@ * A {@link MediaSource} that inserts ads linearly into a provided content media source. * *

    The wrapped content media source can contain multiple {@linkplain Timeline.Period periods}. - * Note that multi-period ad insertion is not supported by all {@link AdsLoader} implementations - * (for example, the IMA extension's {@code ImaAdsLoader} only supports single-period Timelines). + * The ad group times in the {@link AdPlaybackState} provided by the {@link AdsLoader} must be + * relative to the start of the first period of the content timeline. Note that multi-period ad + * insertion is not supported by all {@link AdsLoader} implementations (for example, the IMA + * extension's {@code ImaAdsLoader} only supports single-period Timelines). */ @UnstableApi public final class AdsMediaSource extends CompositeMediaSource { @@ -162,7 +164,6 @@ public RuntimeException getRuntimeExceptionForUnexpected() { @Nullable private Timeline activeTimeline; @Nullable private AdPlaybackState adPlaybackState; private @NullableType AdMediaSourceHolder[][] adMediaSourceHolders; - @Nullable private Handler playerHandler; /** * Constructs a new source that inserts ads linearly with the content specified by {@code @@ -276,8 +277,7 @@ public void updateMediaItem(MediaItem mediaItem) { @Override protected void prepareSourceInternal(@Nullable TransferListener mediaTransferListener) { super.prepareSourceInternal(mediaTransferListener); - this.playerHandler = Util.createHandlerForCurrentLooper(); - ComponentListener componentListener = new ComponentListener(playerHandler); + ComponentListener componentListener = new ComponentListener(); this.componentListener = componentListener; contentTimeline = contentMediaSource.getTimeline(); prepareChildSource(CHILD_SOURCE_MEDIA_PERIOD_ID, contentMediaSource); @@ -376,7 +376,6 @@ protected void releaseSourceInternal() { super.releaseSourceInternal(); ComponentListener componentListener = checkNotNull(this.componentListener); this.componentListener = null; - this.playerHandler = null; componentListener.stop(); contentTimeline = null; activeTimeline = null; @@ -396,6 +395,7 @@ protected void onChildSourceInfoRefreshed( maybeUpdateSourceInfo(); } else { contentTimeline = newTimeline; + ComponentListener componentListener = checkNotNull(this.componentListener); mainHandler.post( () -> { boolean sourceInfoUpdated = adsLoader.handleContentTimelineChanged(this, newTimeline); @@ -403,7 +403,7 @@ protected void onChildSourceInfoRefreshed( checkState(!sourceInfoUpdated || !useLazyContentSourcePreparation); // If the source isn't updated by the ads loader we do, if not already published. if (!sourceInfoUpdated && !useLazyContentSourcePreparation) { - checkNotNull(playerHandler).post(this::maybeUpdateSourceInfo); + componentListener.onContentTimelineChanged(); } }); if (useLazyContentSourcePreparation) { @@ -593,7 +593,10 @@ private static MediaItem.AdsConfiguration getAdsConfiguration(MediaItem mediaIte : mediaItem.localConfiguration.adsConfiguration; } - /** Listener for component events. All methods are called on the main thread. */ + /** + * Listener for component events. All listener methods are called on the main thread, while the + * constructor and {@link #stop()} are called on the playback thread. + */ private final class ComponentListener implements AdsLoader.EventListener { private final Handler playerHandler; @@ -604,8 +607,8 @@ private final class ComponentListener implements AdsLoader.EventListener { * Creates new listener which forwards ad playback states on the creating thread and all other * events on the external event listener thread. */ - public ComponentListener(Handler playerHandler) { - this.playerHandler = playerHandler; + private ComponentListener() { + playerHandler = Util.createHandlerForCurrentLooper(); } /** Stops event delivery from this instance. */ @@ -614,6 +617,19 @@ public void stop() { playerHandler.removeCallbacksAndMessages(null); } + private void onContentTimelineChanged() { + if (stopped) { + return; + } + playerHandler.post( + () -> { + if (stopped) { + return; + } + AdsMediaSource.this.maybeUpdateSourceInfo(); + }); + } + @Override public void onAdPlaybackState(final AdPlaybackState adPlaybackState) { if (stopped) { @@ -635,10 +651,11 @@ public void onAdLoadError(final AdLoadException error, DataSpec dataSpec) { } createEventDispatcher(/* mediaPeriodId= */ null) .loadError( - new LoadEventInfo( - LoadEventInfo.getNewId(), - dataSpec, - /* elapsedRealtimeMs= */ SystemClock.elapsedRealtime()), + new LoadEventInfo.Builder( + LoadEventInfo.getNewId(), + dataSpec, + /* elapsedRealtimeMs= */ SystemClock.elapsedRealtime()) + .build(), C.DATA_TYPE_AD, error, /* wasCanceled= */ true); @@ -667,10 +684,11 @@ public void onPrepareComplete(MediaPeriodId mediaPeriodId) { public void onPrepareError(MediaPeriodId mediaPeriodId, IOException exception) { createEventDispatcher(mediaPeriodId) .loadError( - new LoadEventInfo( - LoadEventInfo.getNewId(), - new DataSpec(checkNotNull(adMediaItem.localConfiguration).uri), - /* elapsedRealtimeMs= */ SystemClock.elapsedRealtime()), + new LoadEventInfo.Builder( + LoadEventInfo.getNewId(), + new DataSpec(checkNotNull(adMediaItem.localConfiguration).uri), + /* elapsedRealtimeMs= */ SystemClock.elapsedRealtime()) + .build(), C.DATA_TYPE_AD, AdLoadException.createForAd(exception), /* wasCanceled= */ true); diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ads/ServerSideAdInsertionMediaSource.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ads/ServerSideAdInsertionMediaSource.java index 818ec73acb4..106ed20475f 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ads/ServerSideAdInsertionMediaSource.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/ads/ServerSideAdInsertionMediaSource.java @@ -890,7 +890,9 @@ public long selectTracks( streams[i] = null; lastDownstreamFormatChangeData[i] = null; } else if (streams[i] == null || streamResetFlags[i]) { - streams[i] = new SampleStreamImpl(mediaPeriod, /* streamIndex= */ i); + streams[i] = + new SampleStreamImpl( + mediaPeriod, /* streamIndex= */ i, /* isReusedStream= */ false); lastDownstreamFormatChangeData[i] = null; } } @@ -905,7 +907,8 @@ public long selectTracks( if (streamResetFlags[i]) { streams[i] = Objects.equals(trackSelections[i], selections[i]) - ? new SampleStreamImpl(mediaPeriod, /* streamIndex= */ i) + ? new SampleStreamImpl( + mediaPeriod, /* streamIndex= */ i, /* isReusedStream= */ true) : new EmptySampleStream(); } } else { @@ -1124,6 +1127,12 @@ public Period getPeriod(int periodIndex, Period period, boolean setIds) { getMediaPeriodPositionUsForContent( durationUs, /* nextAdGroupIndex= */ C.INDEX_UNSET, adPlaybackState); } + long originalDurationUs = period.originalDurationUs; + if (originalDurationUs != C.TIME_UNSET) { + originalDurationUs = + getMediaPeriodPositionUsForContent( + originalDurationUs, /* nextAdGroupIndex= */ C.INDEX_UNSET, adPlaybackState); + } long positionInWindowUs = 0; Period innerPeriod = new Period(); for (int i = 0; i < periodIndex + 1; i++) { @@ -1151,7 +1160,8 @@ public Period getPeriod(int periodIndex, Period period, boolean setIds) { durationUs, positionInWindowUs, adPlaybackState, - period.isPlaceholder); + period.isPlaceholder, + originalDurationUs); return period; } } @@ -1299,10 +1309,12 @@ private static final class SampleStreamImpl implements SampleStream { private final MediaPeriodImpl mediaPeriod; private final int streamIndex; + private final boolean isReusedStream; - public SampleStreamImpl(MediaPeriodImpl mediaPeriod, int streamIndex) { + private SampleStreamImpl(MediaPeriodImpl mediaPeriod, int streamIndex, boolean isReusedStream) { this.mediaPeriod = mediaPeriod; this.streamIndex = streamIndex; + this.isReusedStream = isReusedStream; } @Override @@ -1331,16 +1343,23 @@ public int skipData(long positionUs) { public @SampleStream.Flags int getFlags() { @Nullable SampleStream childStream = mediaPeriod.sharedPeriod.sampleStreams[streamIndex]; @SampleStream.Flags int childFlags = childStream == null ? 0 : childStream.getFlags(); - // The strict duration flag should only be applied to the very last stream of the content, - // not for in-stream transitions (e.g. ad to content or content to ad). - boolean isAtEndOfSourceStream = - mediaPeriod.mediaPeriodId.nextAdGroupIndex == C.INDEX_UNSET - && !mediaPeriod.mediaPeriodId.isAd(); - if (isAtEndOfSourceStream) { - return childFlags; - } else { - return childFlags & ~SampleStream.FLAG_STRICT_DURATION; + // The preroll flag should only be applied to the first stream of the wrapped period, not to + // reused streams across in-stream transitions (e.g. ad to content or content to ad). The + // strict duration flag should only be applied to the very last stream of the content. + if (isReusedStream) { + childFlags &= ~(SampleStream.FLAG_HAS_PREROLL | SampleStream.FLAG_MAYBE_HAS_PREROLL); } + if (!isAtEndOfSourceStream(mediaPeriod.mediaPeriodId)) { + childFlags &= ~SampleStream.FLAG_STRICT_DURATION; + } + return childFlags; + } + } + + private static boolean isAtEndOfSourceStream(MediaPeriodId mediaPeriodId) { + if (mediaPeriodId.isAd()) { + return false; } + return mediaPeriodId.nextAdGroupIndex == C.INDEX_UNSET; } } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/BaseMediaChunk.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/BaseMediaChunk.java index c7dc60b7910..387d14dd63e 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/BaseMediaChunk.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/BaseMediaChunk.java @@ -77,7 +77,8 @@ public BaseMediaChunk( trackSelectionData, startTimeUs, endTimeUs, - chunkIndex); + chunkIndex, + /* steeredPathwayId= */ null); this.clippedStartTimeUs = clippedStartTimeUs; this.clippedEndTimeUs = clippedEndTimeUs; } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/BundledChunkExtractor.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/BundledChunkExtractor.java index 5d56f2210e0..d91d687bf06 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/BundledChunkExtractor.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/BundledChunkExtractor.java @@ -82,11 +82,9 @@ public static final class Factory implements ChunkExtractor.Factory { private SubtitleParser.Factory subtitleParserFactory; private boolean parseSubtitlesDuringExtraction; - private @C.VideoCodecFlags int codecsToParseWithinGopSampleDependencies; public Factory() { subtitleParserFactory = new DefaultSubtitleParserFactory(); - codecsToParseWithinGopSampleDependencies = C.VIDEO_CODEC_FLAG_H264 | C.VIDEO_CODEC_FLAG_H265; } @CanIgnoreReturnValue @@ -104,14 +102,6 @@ public Factory experimentalParseSubtitlesDuringExtraction( return this; } - @Override - @CanIgnoreReturnValue - public Factory experimentalSetCodecsToParseWithinGopSampleDependencies( - @C.VideoCodecFlags int codecsToParseWithinGopSampleDependencies) { - this.codecsToParseWithinGopSampleDependencies = codecsToParseWithinGopSampleDependencies; - return this; - } - /** * {@inheritDoc} * @@ -177,9 +167,6 @@ public ChunkExtractor createProgressiveMediaExtractor( if (!parseSubtitlesDuringExtraction) { flags |= FragmentedMp4Extractor.FLAG_EMIT_RAW_SUBTITLE_DATA; } - flags |= - FragmentedMp4Extractor.codecsToParseWithinGopSampleDependenciesAsFlags( - codecsToParseWithinGopSampleDependencies); extractor = new FragmentedMp4Extractor( subtitleParserFactory, diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/Chunk.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/Chunk.java index 9e1de1ed649..df20030dd7c 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/Chunk.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/Chunk.java @@ -75,6 +75,12 @@ public abstract class Chunk implements Loadable { */ public final long endTimeUs; + /** + * The ID of the steered pathway from which data is being loaded, or {@code null} if not + * applicable. + */ + @Nullable public final String steeredPathwayId; + protected final StatsDataSource dataSource; /** @@ -86,6 +92,7 @@ public abstract class Chunk implements Loadable { * @param trackSelectionData See {@link #trackSelectionData}. * @param startTimeUs See {@link #startTimeUs}. * @param endTimeUs See {@link #endTimeUs}. + * @param steeredPathwayId See {@link #steeredPathwayId}. */ public Chunk( DataSource dataSource, @@ -95,7 +102,8 @@ public Chunk( @C.SelectionReason int trackSelectionReason, @Nullable Object trackSelectionData, long startTimeUs, - long endTimeUs) { + long endTimeUs, + @Nullable String steeredPathwayId) { this.dataSource = new StatsDataSource(dataSource); this.dataSpec = checkNotNull(dataSpec); this.type = type; @@ -104,6 +112,7 @@ public Chunk( this.trackSelectionData = trackSelectionData; this.startTimeUs = startTimeUs; this.endTimeUs = endTimeUs; + this.steeredPathwayId = steeredPathwayId; loadTaskId = LoadEventInfo.getNewId(); } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/ChunkExtractor.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/ChunkExtractor.java index e9cfcdeab71..a30c3f91fcb 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/ChunkExtractor.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/ChunkExtractor.java @@ -26,7 +26,6 @@ import androidx.media3.extractor.Extractor; import androidx.media3.extractor.ExtractorInput; import androidx.media3.extractor.TrackOutput; -import androidx.media3.extractor.mp4.FragmentedMp4Extractor; import androidx.media3.extractor.text.SubtitleParser; import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.io.IOException; @@ -75,26 +74,6 @@ default Factory experimentalParseSubtitlesDuringExtraction( return this; } - /** - * Sets the set of video codecs for which within GOP sample dependency information should be - * parsed as part of extraction. Defaults to H.264 and H.265. - * - *

    Having access to additional sample dependency information can speed up seeking. See {@link - * FragmentedMp4Extractor#FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES}. - * - *

    This method is experimental and will be renamed or removed in a future release. - * - * @param codecsToParseWithinGopSampleDependencies The set of codecs for which to parse within - * GOP sample dependency information. - * @return This factory, for convenience. - */ - @CanIgnoreReturnValue - @ExperimentalApi // TODO: b/470365670 - Remove method once config is enabled by default. - default Factory experimentalSetCodecsToParseWithinGopSampleDependencies( - @C.VideoCodecFlags int codecsToParseWithinGopSampleDependencies) { - return this; - } - /** * Returns the output {@link Format} of emitted {@linkplain C#TRACK_TYPE_TEXT text samples} * which were originally in {@code sourceFormat}. diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/ChunkSampleStream.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/ChunkSampleStream.java index dff4778efd9..5dc7cbde943 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/ChunkSampleStream.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/ChunkSampleStream.java @@ -484,19 +484,17 @@ public int skipData(long positionUs) { @Override public void onLoadStarted( Chunk loadable, long elapsedRealtimeMs, long loadDurationMs, int retryCount) { - LoadEventInfo loadEventInfo = - retryCount == 0 - ? new LoadEventInfo(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) - : new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + LoadEventInfo.Builder loadEventInfo = + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs); + if (retryCount != 0) { + loadEventInfo + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()); + } mediaSourceEventDispatcher.loadStarted( - loadEventInfo, + loadEventInfo.build(), loadable.type, primaryTrackType, loadable.trackFormat, @@ -512,14 +510,12 @@ public void onLoadCompleted(Chunk loadable, long elapsedRealtimeMs, long loadDur loadingChunk = null; chunkSource.onChunkLoadCompleted(loadable); LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(); loadErrorHandlingPolicy.onLoadTaskConcluded(loadable.loadTaskId); mediaSourceEventDispatcher.loadCompleted( loadEventInfo, @@ -539,14 +535,12 @@ public void onLoadCanceled( loadingChunk = null; canceledMediaChunk = null; LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(); loadErrorHandlingPolicy.onLoadTaskConcluded(loadable.loadTaskId); mediaSourceEventDispatcher.loadCanceled( loadEventInfo, @@ -584,14 +578,12 @@ public LoadErrorAction onLoadError( boolean cancelable = bytesLoaded == 0 || !isMediaChunk || !haveReadFromMediaChunk(lastChunkIndex); LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - bytesLoaded); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(bytesLoaded) + .build(); MediaLoadData mediaLoadData = new MediaLoadData( loadable.type, @@ -623,7 +615,10 @@ public LoadErrorAction onLoadError( if (loadErrorAction == null) { // The load was not cancelled. Either the load must be retried or the error propagated. - long retryDelayMs = loadErrorHandlingPolicy.getRetryDelayMsFor(loadErrorInfo); + long retryDelayMs = + errorCount > loadErrorHandlingPolicy.getMinimumLoadableRetryCount(loadable.type) + ? C.TIME_UNSET + : loadErrorHandlingPolicy.getRetryDelayMsFor(loadErrorInfo); loadErrorAction = retryDelayMs != C.TIME_UNSET ? Loader.createRetryAction(/* resetErrorCount= */ false, retryDelayMs) @@ -780,7 +775,7 @@ && haveReadFromMediaChunk(/* mediaChunkIndex= */ mediaChunks.size() - 1)) { public boolean mayHaveInitialDiscontinuity() { return (needToEvaluateInitialDiscontinuity || hasInitialDiscontinuity) && !loadingFinished - && !loader.hasFatalError(); + && !hasFatalError(); } /** @@ -808,7 +803,7 @@ public void setUsesStreamPrerollFlags() { @Override public @Flags int getFlags() { - if (needToEvaluateInitialDiscontinuity) { + if (needToEvaluateInitialDiscontinuity && !loadingFinished && !hasFatalError()) { return FLAG_MAYBE_HAS_PREROLL; } return hasInitialDiscontinuity ? FLAG_HAS_PREROLL : 0; @@ -860,6 +855,15 @@ public void discardUpstreamSamplesForClippedDuration(long clippedDurationUs) { primaryTrackType, clippedDurationUs, largestQueuedTimestampUs); } + private boolean hasFatalError() { + try { + maybeThrowError(); + return false; + } catch (IOException e) { + return true; + } + } + private void discardUpstream(int preferredQueueSize) { checkState(!loader.isLoading()); diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/DataChunk.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/DataChunk.java index 5e8707ad2c9..38d96e9ced8 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/DataChunk.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/DataChunk.java @@ -65,7 +65,8 @@ public DataChunk( trackSelectionReason, trackSelectionData, C.TIME_UNSET, - C.TIME_UNSET); + C.TIME_UNSET, + /* steeredPathwayId= */ null); this.data = data == null ? Util.EMPTY_BYTE_ARRAY : data; } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/InitializationChunk.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/InitializationChunk.java index 4ad794a88f2..25751dbcdf5 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/InitializationChunk.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/InitializationChunk.java @@ -66,7 +66,8 @@ public InitializationChunk( trackSelectionReason, trackSelectionData, C.TIME_UNSET, - C.TIME_UNSET); + C.TIME_UNSET, + /* steeredPathwayId= */ null); this.chunkExtractor = chunkExtractor; } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/MediaChunk.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/MediaChunk.java index 8817bece221..35ec7024514 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/MediaChunk.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/MediaChunk.java @@ -40,6 +40,7 @@ public abstract class MediaChunk extends Chunk { * @param startTimeUs The start time of the media contained by the chunk, in microseconds. * @param endTimeUs The end time of the media contained by the chunk, in microseconds. * @param chunkIndex The index of the chunk, or {@link C#INDEX_UNSET} if it is not known. + * @param steeredPathwayId See {@link #steeredPathwayId}. */ public MediaChunk( DataSource dataSource, @@ -49,7 +50,8 @@ public MediaChunk( @Nullable Object trackSelectionData, long startTimeUs, long endTimeUs, - long chunkIndex) { + long chunkIndex, + @Nullable String steeredPathwayId) { super( dataSource, dataSpec, @@ -58,7 +60,8 @@ public MediaChunk( trackSelectionReason, trackSelectionData, startTimeUs, - endTimeUs); + endTimeUs, + steeredPathwayId); checkNotNull(trackFormat); this.chunkIndex = chunkIndex; } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/MediaChunkIterator.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/MediaChunkIterator.java index 5641bc0ecab..420a2d1b6e0 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/MediaChunkIterator.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/chunk/MediaChunkIterator.java @@ -103,4 +103,16 @@ public void reset() { /** Resets the iterator to the initial position. */ void reset(); + + /** + * Returns the index in the steering priority list of the locations (e.g., DASH service location + * or HLS pathway) for the current media chunks, or {@code Integer.MAX_VALUE} if steering is + * inactive or unavailable. + * + *

    The returned index is only meaningful relatively among candidate tracks (i.e. a lower + * numerical value indicates a higher priority, with {@code 0} being the top priority). + */ + default int getLocationSteeringPriorityIndex() { + return Integer.MAX_VALUE; + } } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/preload/DefaultPreloadManager.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/preload/DefaultPreloadManager.java index c363aebcae5..91634a58db6 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/preload/DefaultPreloadManager.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/preload/DefaultPreloadManager.java @@ -648,7 +648,7 @@ private DefaultPreloadManager(Builder builder) { preloadLooperProvider = builder.preloadLooperProvider; trackSelector = builder.trackSelectorFactory.createTrackSelector(builder.context); BandwidthMeter bandwidthMeter = builder.bandwidthMeterSupplier.get(); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(trackSelector::onParametersActivated, bandwidthMeter); Looper preloadLooper = preloadLooperProvider.obtainLooper(); preloadMediaSourceFactory = new PreloadMediaSource.Factory( diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/preload/PreCacheHelper.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/preload/PreCacheHelper.java index 925accb01eb..632eb4e9541 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/preload/PreCacheHelper.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/preload/PreCacheHelper.java @@ -492,6 +492,11 @@ public void onPrepareError(DownloadHelper helper, IOException e) { public void maybeSubmitPendingDownloadRequest() { checkState(Looper.myLooper() == preCacheHandler.getLooper()); + synchronized (lock) { + if (isCanceled) { + return; + } + } if (pendingDownloadRequest != null) { downloader = downloaderFactory.createDownloader(pendingDownloadRequest); downloaderTask = diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/preload/PreloadMediaSource.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/preload/PreloadMediaSource.java index 20cedc7349d..7a176e5eb3e 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/preload/PreloadMediaSource.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/source/preload/PreloadMediaSource.java @@ -311,11 +311,13 @@ public void preload(long startPositionUs) { if (isUsedByPlayer()) { onUsedByPlayer(); } else { - this.preloadCalled = true; this.startPositionUs = startPositionUs; setPlayerId(PlayerId.PRELOAD); setBandwidthMeter(bandwidthMeter); - loadControl.onPrepared(PlayerId.PRELOAD); + if (!preloadCalled) { + loadControl.onPrepared(PlayerId.PRELOAD); + preloadCalled = true; + } prepareSourceInternal(bandwidthMeter.getTransferListener()); checkForPreloadError(); } @@ -357,6 +359,9 @@ protected void prepareSourceInternal() { protected void onChildSourceInfoRefreshed(Timeline newTimeline) { this.timeline = newTimeline; refreshSourceInfo(newTimeline); + if (!preloadCalled) { + return; + } preloadHandler.post( () -> { if (isUsedByPlayer() || onSourcePreparedNotified) { diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/text/TextRenderer.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/text/TextRenderer.java index 7a4611f010f..f224cfb67ba 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/text/TextRenderer.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/text/TextRenderer.java @@ -338,6 +338,13 @@ private boolean readAndDecodeCuesWithTiming(long positionUs) { formatHolder, cueDecoderInputBuffer, SampleStream.FLAG_PEEK | SampleStream.FLAG_OMIT_SAMPLE_DATA); + while (readResult == C.RESULT_FORMAT_READ) { + readResult = + readSource( + formatHolder, + cueDecoderInputBuffer, + SampleStream.FLAG_PEEK | SampleStream.FLAG_OMIT_SAMPLE_DATA); + } if (readResult == C.RESULT_NOTHING_READ && !cueDecoderInputBuffer.isEndOfStream()) { // Nothing to consume yet. return false; @@ -467,6 +474,13 @@ private void renderFromSubtitles(long positionUs) { formatHolder, nextInputBuffer, SampleStream.FLAG_PEEK | SampleStream.FLAG_OMIT_SAMPLE_DATA); + while (result == C.RESULT_FORMAT_READ) { + result = + readSource( + formatHolder, + nextInputBuffer, + SampleStream.FLAG_PEEK | SampleStream.FLAG_OMIT_SAMPLE_DATA); + } if (result == C.RESULT_NOTHING_READ && !nextInputBuffer.isEndOfStream()) { // Nothing to consume yet. return; diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/trackselection/AdaptiveTrackSelection.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/trackselection/AdaptiveTrackSelection.java index 1319e961aae..1b334b011c5 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/trackselection/AdaptiveTrackSelection.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/trackselection/AdaptiveTrackSelection.java @@ -342,6 +342,7 @@ protected AdaptiveTrackSelection createAdaptiveTrackSelection( private long lastBufferEvaluationMs; @Nullable private MediaChunk lastBufferEvaluationMediaChunk; private long latestBitrateEstimate; + private final int[] latestLocationSteeringPriority; /** * @param group The {@link TrackGroup}. @@ -498,6 +499,8 @@ protected AdaptiveTrackSelection( reason = C.SELECTION_REASON_UNKNOWN; lastBufferEvaluationMs = C.TIME_UNSET; latestBitrateEstimate = C.RATE_UNSET_INT; + latestLocationSteeringPriority = new int[length]; + Arrays.fill(latestLocationSteeringPriority, Integer.MAX_VALUE); } @CallSuper @@ -529,7 +532,12 @@ public void updateSelectedTrack( long nowMs = clock.elapsedRealtime(); long chunkDurationUs = getNextChunkDurationUs(mediaChunkIterators, queue); long effectiveBitrate = getAllocatedBandwidth(chunkDurationUs); - + for (int i = 0; i < length; i++) { + latestLocationSteeringPriority[i] = + i < mediaChunkIterators.length + ? mediaChunkIterators[i].getLocationSteeringPriorityIndex() + : Integer.MAX_VALUE; + } // Make initial selection if (reason == C.SELECTION_REASON_UNKNOWN) { reason = C.SELECTION_REASON_INITIAL; @@ -551,15 +559,20 @@ public void updateSelectedTrack( // Revert back to the previous selection if conditions are not suitable for switching. long minDurationForQualityIncreaseUs = minDurationForQualityIncreaseUs(availableDurationUs, chunkDurationUs); + int locationPriorityForPreviousSelectedIndex = + latestLocationSteeringPriority[previousSelectedIndex]; + int locationPriorityForNewSelectedIndex = latestLocationSteeringPriority[newSelectedIndex]; if (newSelectedIndex < previousSelectedIndex && bufferedDurationUs < minDurationForQualityIncreaseUs) { - // The selected track is higher priority, but we have insufficient buffer to safely switch - // up. Defer switching up for now. + // The selected track is higher in track format priority, but we have insufficient buffer to + // safely switch up. Defer switching up for now. newSelectedIndex = previousSelectedIndex; } else if (newSelectedIndex > previousSelectedIndex - && bufferedDurationUs >= maxDurationForQualityDecreaseUs) { - // The selected track is lower priority, but we have sufficient buffer to defer switching - // down for now. + && bufferedDurationUs >= maxDurationForQualityDecreaseUs + && locationPriorityForNewSelectedIndex >= locationPriorityForPreviousSelectedIndex) { + // The selected track is lower in track format priority, but we have sufficient buffer to + // defer switching down for now. Note that we only consider deferring if we are not + // improving the steering priority. newSelectedIndex = previousSelectedIndex; } } @@ -683,17 +696,24 @@ protected long getMinDurationToRetainAfterDiscardUs() { */ private int determineIdealSelectedIndex(long nowMs, long effectiveBitrate) { int lowestBitrateAllowedIndex = 0; + int bestIndex = C.INDEX_UNSET; + int bestLocationSteeringPriority = Integer.MAX_VALUE; for (int i = 0; i < length; i++) { if (nowMs == Long.MIN_VALUE || !isTrackExcluded(i, nowMs)) { Format format = getFormat(i); if (canSelectFormat(format, format.bitrate, effectiveBitrate)) { - return i; + int locationSteeringPriority = latestLocationSteeringPriority[i]; + if (bestIndex == C.INDEX_UNSET + || locationSteeringPriority < bestLocationSteeringPriority) { + bestLocationSteeringPriority = locationSteeringPriority; + bestIndex = i; + } } else { lowestBitrateAllowedIndex = i; } } } - return lowestBitrateAllowedIndex; + return bestIndex != C.INDEX_UNSET ? bestIndex : lowestBitrateAllowedIndex; } private long minDurationForQualityIncreaseUs(long availableDurationUs, long chunkDurationUs) { diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/trackselection/DefaultTrackSelector.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/trackselection/DefaultTrackSelector.java index b3824c00da1..3c96a84d47d 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/trackselection/DefaultTrackSelector.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/trackselection/DefaultTrackSelector.java @@ -38,7 +38,6 @@ import android.util.SparseArray; import android.util.SparseBooleanArray; import android.view.accessibility.CaptioningManager; -import androidx.annotation.GuardedBy; import androidx.annotation.IntDef; import androidx.annotation.Nullable; import androidx.media3.common.AudioAttributes; @@ -67,6 +66,7 @@ import androidx.media3.exoplayer.mediacodec.MediaCodecUtil; import androidx.media3.exoplayer.source.MediaSource.MediaPeriodId; import androidx.media3.exoplayer.source.TrackGroupArray; +import androidx.media3.exoplayer.upstream.BandwidthMeter; import androidx.media3.exoplayer.util.SpatializerWrapper; import com.google.common.base.Predicate; import com.google.common.collect.ComparisonChain; @@ -2455,16 +2455,13 @@ public static SelectionOverride fromBundle(Bundle bundle) { ? (second == Format.NO_VALUE ? 0 : -1) : (second == Format.NO_VALUE ? 1 : (first - second))); - private final Object lock; @Nullable public final Context context; private final ExoTrackSelection.Factory trackSelectionFactory; - @GuardedBy("lock") - private Parameters parameters; + private Parameters applicationParameters; + private Parameters playerParameters; - @GuardedBy("lock") - @Nullable - private Thread playbackThread; + @Nullable private volatile Thread playbackThread; @Nullable private SpatializerWrapper spatializer; private AudioAttributes audioAttributes; @@ -2531,28 +2528,46 @@ private DefaultTrackSelector( TrackSelectionParameters parameters, ExoTrackSelection.Factory trackSelectionFactory, @Nullable Context context) { - this.lock = new Object(); this.context = context != null ? context.getApplicationContext() : null; this.trackSelectionFactory = trackSelectionFactory; + Parameters initialParameters; if (parameters instanceof Parameters) { - this.parameters = (Parameters) parameters; + initialParameters = (Parameters) parameters; } else { - this.parameters = Parameters.DEFAULT.buildUpon().set(parameters).build(); + initialParameters = Parameters.DEFAULT.buildUpon().set(parameters).build(); } + this.applicationParameters = initialParameters; + this.playerParameters = initialParameters; this.audioAttributes = AudioAttributes.DEFAULT; - if (this.parameters.constrainAudioChannelCountToDeviceCapabilities && context == null) { + if (initialParameters.constrainAudioChannelCountToDeviceCapabilities && context == null) { Log.w(TAG, AUDIO_CHANNEL_COUNT_CONSTRAINTS_WARN_MESSAGE); } } + @Override + public void init(InvalidationListener listener, BandwidthMeter bandwidthMeter) { + super.init(listener, bandwidthMeter); + this.playerParameters = getParameters(); + } + + @Override + public void onParametersActivated(@Nullable TrackSelectionParameters parameters) { + if (parameters == null) { + return; + } + if (parameters instanceof Parameters) { + this.playerParameters = (Parameters) parameters; + } else { + this.playerParameters = new Parameters.Builder(this.playerParameters).set(parameters).build(); + } + } + @Override public void release() { - synchronized (lock) { - if (playbackThread != null) { - checkState( - playbackThread == Thread.currentThread(), - "DefaultTrackSelector is accessed on the wrong thread."); - } + if (playbackThread != null) { + checkState( + Thread.currentThread().equals(playbackThread), + "DefaultTrackSelector is accessed on the wrong thread."); } if (SDK_INT >= 32 && spatializer != null) { spatializer.release(); @@ -2563,9 +2578,7 @@ public void release() { @Override public Parameters getParameters() { - synchronized (lock) { - return parameters; - } + return applicationParameters; } @Override @@ -2622,17 +2635,14 @@ public Parameters.Builder buildUponParameters() { */ private void setParametersInternal(Parameters parameters) { checkNotNull(parameters); - boolean parametersChanged; - synchronized (lock) { - parametersChanged = !this.parameters.equals(parameters); - this.parameters = parameters; - } + boolean parametersChanged = !this.applicationParameters.equals(parameters); + this.applicationParameters = parameters; if (parametersChanged) { if (parameters.constrainAudioChannelCountToDeviceCapabilities && context == null) { Log.w(TAG, AUDIO_CHANNEL_COUNT_CONSTRAINTS_WARN_MESSAGE); } - invalidate(); + invalidate(parameters); } } @@ -2660,15 +2670,11 @@ public void onRendererCapabilitiesChanged(Renderer renderer) { MediaPeriodId mediaPeriodId, Timeline timeline) throws ExoPlaybackException { - Parameters parameters; - synchronized (lock) { - playbackThread = Thread.currentThread(); - parameters = this.parameters; - } + playbackThread = Thread.currentThread(); if (deviceIsTV == null && context != null) { deviceIsTV = Util.isTv(context); } - if (parameters.constrainAudioChannelCountToDeviceCapabilities + if (playerParameters.constrainAudioChannelCountToDeviceCapabilities && SDK_INT >= 32 && spatializer == null) { spatializer = @@ -2681,9 +2687,9 @@ public void onRendererCapabilitiesChanged(Renderer renderer) { new ExoTrackSelection.Definition[rendererCount]; // Apply overrides once so that selectAllTracks is aware of the overridden tracks. - applyTrackSelectionOverrides(mappedTrackInfo, parameters, definitions); - applyLegacyRendererOverrides(mappedTrackInfo, parameters, definitions); - applyRendererDisableOverrides(mappedTrackInfo, parameters, definitions); + applyTrackSelectionOverrides(mappedTrackInfo, playerParameters, definitions); + applyLegacyRendererOverrides(mappedTrackInfo, playerParameters, definitions); + applyRendererDisableOverrides(mappedTrackInfo, playerParameters, definitions); // Select remaining tracks based on parameters. selectAllTracks( @@ -2691,13 +2697,13 @@ public void onRendererCapabilitiesChanged(Renderer renderer) { mappedTrackInfo, rendererFormatSupports, rendererMixedMimeTypeAdaptationSupport, - parameters); + playerParameters); // Re-apply overrides to enforce them in case selectAllTracks changed them or added additional // tracks that shouldn't be enabled according to the overrides. - applyTrackSelectionOverrides(mappedTrackInfo, parameters, definitions); - applyLegacyRendererOverrides(mappedTrackInfo, parameters, definitions); - applyRendererDisableOverrides(mappedTrackInfo, parameters, definitions); + applyTrackSelectionOverrides(mappedTrackInfo, playerParameters, definitions); + applyLegacyRendererOverrides(mappedTrackInfo, playerParameters, definitions); + applyRendererDisableOverrides(mappedTrackInfo, playerParameters, definitions); @NullableType ExoTrackSelection[] rendererTrackSelections = @@ -2711,7 +2717,8 @@ public void onRendererCapabilitiesChanged(Renderer renderer) { for (int i = 0; i < rendererCount; i++) { @C.TrackType int rendererType = mappedTrackInfo.getRendererType(i); boolean forceRendererDisabled = - parameters.getRendererDisabled(i) || parameters.disabledTrackTypes.contains(rendererType); + playerParameters.getRendererDisabled(i) + || playerParameters.disabledTrackTypes.contains(rendererType); boolean rendererEnabled = !forceRendererDisabled && (mappedTrackInfo.getRendererType(i) == C.TRACK_TYPE_NONE @@ -2720,15 +2727,15 @@ public void onRendererCapabilitiesChanged(Renderer renderer) { } // Configure audio and video renderers to use tunneling if appropriate. - if (parameters.tunnelingEnabled) { + if (playerParameters.tunnelingEnabled) { maybeConfigureRenderersForTunneling( mappedTrackInfo, rendererFormatSupports, rendererConfigurations, rendererTrackSelections); } // Configure audio renderer to use offload if appropriate. - if (parameters.audioOffloadPreferences.audioOffloadMode != AUDIO_OFFLOAD_MODE_DISABLED) { + if (playerParameters.audioOffloadPreferences.audioOffloadMode != AUDIO_OFFLOAD_MODE_DISABLED) { maybeConfigureRendererForOffload( - parameters, + playerParameters, mappedTrackInfo, rendererFormatSupports, rendererConfigurations, @@ -3258,25 +3265,18 @@ private > Pair sel } private void maybeInvalidateForAudioChannelCountConstraints() { - boolean shouldInvalidate; - synchronized (lock) { - shouldInvalidate = - parameters.constrainAudioChannelCountToDeviceCapabilities - && SDK_INT >= 32 - && spatializer != null - && spatializer.isSpatializationSupported(); - } + boolean shouldInvalidate = + playerParameters.constrainAudioChannelCountToDeviceCapabilities + && SDK_INT >= 32 + && spatializer != null + && spatializer.isSpatializationSupported(); if (shouldInvalidate) { - invalidate(); + invalidate(/* parameters= */ null); } } private void maybeInvalidateForRendererCapabilitiesChange(Renderer renderer) { - boolean shouldInvalidate; - synchronized (lock) { - shouldInvalidate = parameters.allowInvalidateSelectionsOnRendererCapabilitiesChange; - } - if (shouldInvalidate) { + if (playerParameters.allowInvalidateSelectionsOnRendererCapabilitiesChange) { invalidateForRendererCapabilitiesChange(renderer); } } @@ -3937,7 +3937,7 @@ public VideoTrackInfo( == RendererCapabilities.HARDWARE_ACCELERATION_SUPPORTED; this.resolvedMimeType = resolvedMimeType; codecPreferenceScore = getVideoCodecPreferenceScore(resolvedMimeType); - isHdr = usesPrimaryDecoder && ColorInfo.isTransferHdr(format.colorInfo); + isHdr = usesPrimaryOrFallbackDecoder && ColorInfo.isTransferHdr(format.colorInfo); selectionEligibility = evaluateSelectionEligibility(formatSupport, requiredAdaptiveSupport); } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/trackselection/TrackSelector.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/trackselection/TrackSelector.java index 671fa256dab..db42a091c61 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/trackselection/TrackSelector.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/trackselection/TrackSelector.java @@ -61,20 +61,28 @@ * the media during the 30 second gap. The player indicates to the track selector when a * selection it has previously made becomes active by calling {@link * #onSelectionActivated(Object)}. - *

  • If the track selector wishes to indicate to the player that selections it has previously - * made are invalid, it can do so by calling {@link - * InvalidationListener#onTrackSelectionsInvalidated()} on the {@link InvalidationListener} - * that was passed to {@link #init(InvalidationListener, BandwidthMeter)}. A track selector - * may wish to do this if its configuration has changed, for example if it now wishes to - * prefer audio tracks in a particular language. This will trigger the player to make new - * track selections. Note that the player will have to re-buffer in the case that the new - * track selection for the currently playing period differs from the one that was invalidated. - * Implementing subclasses can trigger invalidation by calling {@link #invalidate()}, which - * will call {@link InvalidationListener#onTrackSelectionsInvalidated()}. + *
  • When the track selector's configuration changes (for example, if it now wishes to prefer + * audio tracks in a particular language) or its previous selections become invalid, it can + * trigger invalidation by calling {@link #invalidate(TrackSelectionParameters)}. This invokes + * {@link InvalidationListener#onTrackSelectionsInvalidated(TrackSelectionParameters)} on the + * {@link InvalidationListener} that was passed to {@link #init(InvalidationListener, + * BandwidthMeter)}. When handling this callback, if {@code parameters} is not {@code null}, + * the listener must call {@link #onParametersActivated(TrackSelectionParameters)} to activate + * the updated parameters before triggering the player to make new track selections. Note that + * the player will have to re-buffer if the new track selection for the currently playing + * period differs from the one that was invalidated. *
  • When the player is {@linkplain Player#release() released}, it will release the track * selector by calling {@link #release()}. * * + *

    Track selection parameters

    + * + *

    Track selection parameters can be retrieved and modified on the application thread by calling + * {@link #getParameters()} and {@link #setParameters(TrackSelectionParameters)} respectively (if + * supported, as indicated by {@link #isSetParametersSupported()}). Modifying parameters on the + * application thread triggers the invalidation flow described above, ensuring that the updated + * parameters are activated and applied on the playback thread. + * *

    Renderer configuration

    * * The {@link TrackSelectorResult} returned by {@link #selectTracks(RendererCapabilities[], @@ -119,9 +127,16 @@ public interface InvalidationListener { /** * Called by a {@link TrackSelector} to indicate that selections it has previously made are no - * longer valid. May be called from any thread. + * longer valid, or that track selection parameters have changed. + * + *

    If {@code parameters} is not {@code null}, the receiver must call {@link + * TrackSelector#onParametersActivated(TrackSelectionParameters)} to activate the new + * parameters. + * + * @param parameters The new track selection parameters, or {@code null} if selections need to + * be re-evaluated without applying new parameters. */ - void onTrackSelectionsInvalidated(); + void onTrackSelectionsInvalidated(@Nullable TrackSelectionParameters parameters); /** * Called by a {@link TrackSelector} to indicate that selections it has previously made may no @@ -187,6 +202,18 @@ public abstract TrackSelectorResult selectTracks( */ public abstract void onSelectionActivated(@Nullable Object info); + /** + * Called by the player when track selection parameters become active for subsequent track + * selections made by {@link #selectTracks(RendererCapabilities[], TrackGroupArray, MediaPeriodId, + * Timeline)}. + * + * @param parameters The activated track selection parameters, or {@code null} if no new + * parameters need to be activated. + */ + public void onParametersActivated(@Nullable TrackSelectionParameters parameters) { + // Default implementation is no-op. + } + /** * Returns the current parameters for track selection. * @@ -237,14 +264,17 @@ public RendererCapabilities.Listener getRendererCapabilitiesListener() { } /** - * Calls {@link InvalidationListener#onTrackSelectionsInvalidated()} to invalidate all previously - * generated track selections. + * Calls {@link InvalidationListener#onTrackSelectionsInvalidated(TrackSelectionParameters)} to + * invalidate all previously generated track selections. * *

    This method can be called from both the application and the playback thread. + * + * @param parameters The track selection parameters, or {@code null} to re-evaluate selections + * without applying new parameters. */ - protected final void invalidate() { + protected final void invalidate(@Nullable TrackSelectionParameters parameters) { if (listener != null) { - listener.onTrackSelectionsInvalidated(); + listener.onTrackSelectionsInvalidated(parameters); } } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/upstream/ParsingLoadable.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/upstream/ParsingLoadable.java index d027f57a304..fece377fae6 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/upstream/ParsingLoadable.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/upstream/ParsingLoadable.java @@ -29,6 +29,7 @@ import androidx.media3.datasource.StatsDataSource; import androidx.media3.exoplayer.source.LoadEventInfo; import androidx.media3.exoplayer.upstream.Loader.Loadable; +import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.io.IOException; import java.io.InputStream; import java.util.List; @@ -57,39 +58,66 @@ public interface Parser { T parse(Uri uri, InputStream inputStream) throws IOException; } - /** - * Loads a single parsable object. - * - * @param dataSource The {@link DataSource} through which the object should be read. - * @param parser The {@link Parser} to parse the object from the response. - * @param uri The {@link Uri} of the object to read. - * @param type The type of the data. One of the {@link C}{@code DATA_TYPE_*} constants. - * @return The parsed object - * @throws IOException Thrown if there is an error while loading or parsing. - */ - public static T load(DataSource dataSource, Parser parser, Uri uri, int type) - throws IOException { - ParsingLoadable loadable = new ParsingLoadable<>(dataSource, uri, type, parser); - loadable.load(); - return checkNotNull(loadable.getResult()); - } + /** Builder for {@link ParsingLoadable}. */ + public static final class Builder { + private final DataSource dataSource; + private final DataSpec dataSpec; + private final @C.DataType int type; + private final Parser parser; + @Nullable private String steeredPathwayId; - /** - * Loads a single parsable object. - * - * @param dataSource The {@link DataSource} through which the object should be read. - * @param parser The {@link Parser} to parse the object from the response. - * @param dataSpec The {@link DataSpec} of the object to read. - * @param type The type of the data. One of the {@link C}{@code DATA_TYPE_*} constants. - * @return The parsed object - * @throws IOException Thrown if there is an error while loading or parsing. - */ - public static T load( - DataSource dataSource, Parser parser, DataSpec dataSpec, int type) - throws IOException { - ParsingLoadable loadable = new ParsingLoadable<>(dataSource, dataSpec, type, parser); - loadable.load(); - return checkNotNull(loadable.getResult()); + /** + * Creates a builder. + * + * @param dataSource A {@link DataSource} to use when loading the data. + * @param uri The {@link Uri} from which the object should be loaded. + * @param type The {@linkplain C.DataType type} of the data. + * @param parser Parses the object from the response. + */ + public Builder( + DataSource dataSource, Uri uri, @C.DataType int type, Parser parser) { + this( + dataSource, + new DataSpec.Builder().setUri(uri).setFlags(DataSpec.FLAG_ALLOW_GZIP).build(), + type, + parser); + } + + /** + * Creates a builder. + * + * @param dataSource A {@link DataSource} to use when loading the data. + * @param dataSpec The {@link DataSpec} from which the object should be loaded. + * @param type The {@linkplain C.DataType type} of the data. + * @param parser Parses the object from the response. + */ + public Builder( + DataSource dataSource, + DataSpec dataSpec, + @C.DataType int type, + Parser parser) { + this.dataSource = dataSource; + this.dataSpec = dataSpec; + this.type = type; + this.parser = parser; + } + + /** + * Sets the {@link ParsingLoadable#steeredPathwayId}, the default is {@code null}. + * + * @param steeredPathwayId See {@link ParsingLoadable#steeredPathwayId}. + * @return This builder. + */ + @CanIgnoreReturnValue + public Builder setSteeredPathwayId(@Nullable String steeredPathwayId) { + this.steeredPathwayId = steeredPathwayId; + return this; + } + + /** Builds the {@link ParsingLoadable}. */ + public ParsingLoadable build() { + return new ParsingLoadable<>(this); + } } /** Identifies the load task for this loadable. */ @@ -98,11 +126,14 @@ public static T load( /** The {@link DataSpec} that defines the data to be loaded. */ public final DataSpec dataSpec; + /** The {@linkplain C.DataType type} of the data. For reporting only. */ + public final @C.DataType int type; + /** - * The type of the data. One of the {@code DATA_TYPE_*} constants defined in {@link C}. For - * reporting only. + * The ID of the steered pathway from which data is being loaded, or {@code null} if not + * applicable. */ - public final int type; + @Nullable public final String steeredPathwayId; private final StatsDataSource dataSource; private final Parser parser; @@ -110,12 +141,11 @@ public static T load( @Nullable private volatile T result; /** - * @param dataSource A {@link DataSource} to use when loading the data. - * @param uri The {@link Uri} from which the object should be loaded. - * @param type See {@link #type}. - * @param parser Parses the object from the response. + * @deprecated Use {@link Builder} instead. */ - public ParsingLoadable(DataSource dataSource, Uri uri, int type, Parser parser) { + @Deprecated + public ParsingLoadable( + DataSource dataSource, Uri uri, @C.DataType int type, Parser parser) { this( dataSource, new DataSpec.Builder().setUri(uri).setFlags(DataSpec.FLAG_ALLOW_GZIP).build(), @@ -124,20 +154,85 @@ public ParsingLoadable(DataSource dataSource, Uri uri, int type, Parser parser) { + DataSource dataSource, DataSpec dataSpec, @C.DataType int type, Parser parser) { this.dataSource = new StatsDataSource(dataSource); this.dataSpec = dataSpec; this.type = type; this.parser = parser; + this.steeredPathwayId = null; loadTaskId = LoadEventInfo.getNewId(); } + private ParsingLoadable(Builder builder) { + this.dataSource = new StatsDataSource(builder.dataSource); + this.dataSpec = builder.dataSpec; + this.type = builder.type; + this.parser = builder.parser; + this.steeredPathwayId = builder.steeredPathwayId; + loadTaskId = LoadEventInfo.getNewId(); + } + + /** + * Loads a single parsable object. + * + * @param dataSource The {@link DataSource} through which the object should be read. + * @param parser The {@link Parser} to parse the object from the response. + * @param uri The {@link Uri} of the object to read. + * @param type The {@linkplain C.DataType type} of the data. + * @return The parsed object + * @throws IOException Thrown if there is an error while loading or parsing. + */ + public static T load( + DataSource dataSource, Parser parser, Uri uri, @C.DataType int type) + throws IOException { + ParsingLoadable loadable = + new ParsingLoadable.Builder(dataSource, uri, type, parser).build(); + loadable.load(); + return checkNotNull(loadable.getResult()); + } + + /** + * Loads a single parsable object. + * + * @param dataSource The {@link DataSource} through which the object should be read. + * @param parser The {@link Parser} to parse the object from the response. + * @param dataSpec The {@link DataSpec} of the object to read. + * @param type The {@linkplain C.DataType type} of the data. + * @return The parsed object + * @throws IOException Thrown if there is an error while loading or parsing. + */ + public static T load( + DataSource dataSource, Parser parser, DataSpec dataSpec, @C.DataType int type) + throws IOException { + ParsingLoadable loadable = + new ParsingLoadable.Builder(dataSource, dataSpec, type, parser).build(); + loadable.load(); + return checkNotNull(loadable.getResult()); + } + + @Override + public final void load() throws IOException { + // We always load from the beginning, so reset bytesRead to 0. + dataSource.resetBytesRead(); + DataSourceInputStream inputStream = new DataSourceInputStream(dataSource, dataSpec); + try { + inputStream.open(); + Uri dataSourceUri = checkNotNull(dataSource.getUri()); + result = parser.parse(dataSourceUri, inputStream); + } finally { + Util.closeQuietly(inputStream); + } + } + + @Override + public final void cancelLoad() { + // Do nothing. + } + /** Returns the loaded object, or null if an object has not been loaded. */ @Nullable public final T getResult() { @@ -168,23 +263,4 @@ public Uri getUri() { public Map> getResponseHeaders() { return dataSource.getLastResponseHeaders(); } - - @Override - public final void cancelLoad() { - // Do nothing. - } - - @Override - public final void load() throws IOException { - // We always load from the beginning, so reset bytesRead to 0. - dataSource.resetBytesRead(); - DataSourceInputStream inputStream = new DataSourceInputStream(dataSource, dataSpec); - try { - inputStream.open(); - Uri dataSourceUri = checkNotNull(dataSource.getUri()); - result = parser.parse(dataSourceUri, inputStream); - } finally { - Util.closeQuietly(inputStream); - } - } } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/upstream/contentsteering/BaseContentSteeringTracker.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/upstream/contentsteering/BaseContentSteeringTracker.java new file mode 100644 index 00000000000..9d3f0c31d60 --- /dev/null +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/upstream/contentsteering/BaseContentSteeringTracker.java @@ -0,0 +1,229 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.exoplayer.upstream.contentsteering; + +import static com.google.common.base.Preconditions.checkNotNull; +import static com.google.common.base.Preconditions.checkState; + +import android.net.Uri; +import androidx.annotation.Nullable; +import androidx.media3.common.util.UnstableApi; +import androidx.media3.datasource.DataSource; +import androidx.media3.exoplayer.source.MediaSourceEventListener; +import androidx.media3.exoplayer.util.ReleasableExecutor; +import com.google.common.base.Supplier; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import java.io.IOException; +import java.util.Map; +import java.util.TreeMap; + +/** + * Abstract base class for Content Steering tracker implementations. + * + *

    It provides common state management for steering manifest tracking, pathway evaluation, and + * pathway cloning. + */ +@UnstableApi +public abstract class BaseContentSteeringTracker { + + private final SteeringManifestTracker steeringManifestTracker; + @Nullable private ImmutableList currentPathwayPriority; + private boolean isActive; + + /** + * Creates a {@link BaseContentSteeringTracker}. + * + * @param dataSourceFactory The {@link DataSource.Factory} to load steering manifests. + * @param downloadExecutorSupplier A supplier for a {@link ReleasableExecutor} to download + * steering manifests. + */ + protected BaseContentSteeringTracker( + DataSource.Factory dataSourceFactory, + @Nullable Supplier downloadExecutorSupplier) { + this.steeringManifestTracker = + new SteeringManifestTracker(dataSourceFactory, downloadExecutorSupplier); + } + + /** + * Starts the Content Steering tracker. + * + * @param initialSteeringManifestUri The initial {@link Uri} of the steering manifest. + * @param initialPathwayIds The IDs of the initial pathways to use before the first steering + * manifest is loaded. + * @param eventDispatcher A {@link MediaSourceEventListener.EventDispatcher} for reporting load + * events. + */ + public final void start( + Uri initialSteeringManifestUri, + ImmutableList initialPathwayIds, + MediaSourceEventListener.EventDispatcher eventDispatcher) { + checkState(!isActive()); + verifyInitialPathwayIdsAvailable(initialPathwayIds); + isActive = true; + onStart(initialPathwayIds); + steeringManifestTracker.start( + initialSteeringManifestUri, new SteeringManifestTrackerCallback(), eventDispatcher); + } + + private void verifyInitialPathwayIdsAvailable(ImmutableList initialPathwayIds) { + for (String pathwayId : initialPathwayIds) { + checkState( + isPathwayAvailable(pathwayId), "The pathway with ID: " + pathwayId + " is not available"); + } + } + + /** + * Returns whether the Content Steering tracker is active. + * + *

    If this method returns {@code false}, the caller of the tracker should behave as if content + * steering is absent. + */ + public final boolean isActive() { + return isActive; + } + + /** + * Releases the Content Steering tracker. + * + *

    Once released, the tracker cannot be used again. + */ + public final void release() { + stopInternal(); + } + + /** + * Called when the tracker is started. + * + * @param initialPathwayIds The IDs of the initial pathways to be used before the first steering + * manifest is loaded. + */ + protected abstract void onStart(ImmutableList initialPathwayIds); + + /** + * Returns whether the given pathway is currently available. + * + * @param pathwayId The ID of the pathway to check. + * @return Whether the given pathway is available. + */ + protected abstract boolean isPathwayAvailable(String pathwayId); + + /** Returns the query parameters to include in the steering manifest request. */ + protected abstract ImmutableMap getSteeringQueryParameters(); + + /** + * Performs a pathway evaluation based on the passed pathway priority. + * + * @param pathwayPriority The {@linkplain SteeringManifest#pathwayPriority pathway priority} to + * use for evaluation. + */ + protected abstract void performPathwayEvaluation(ImmutableList pathwayPriority); + + /** + * Performs a pathway clone defined in the steering manifest. + * + * @param pathwayClone The {@link SteeringManifest.PathwayClone} to perform. + */ + protected abstract void performPathwayClone(SteeringManifest.PathwayClone pathwayClone); + + /** + * Called when tracking is stopped. + * + *

    This can be either externally called by {@link #release()} or internally due to an + * unrecoverable error. + */ + protected abstract void onStop(); + + /** + * Returns the current pathway priority. + * + * @return The current pathway priority, or {@code null} if the tracker is not active. + */ + @Nullable + protected final ImmutableList getCurrentPathwayPriority() { + return currentPathwayPriority; + } + + /** + * Applies a {@link SteeringManifest.UriReplacement} to a base URI. + * + * @param baseUri The base {@link Uri}. + * @param uriReplacement The {@link SteeringManifest.UriReplacement}. + * @return The updated {@link Uri}. + */ + protected static Uri applyUriReplacement( + Uri baseUri, SteeringManifest.UriReplacement uriReplacement) { + Uri.Builder newUrlBuilder = baseUri.buildUpon().clearQuery(); + if (uriReplacement.host != null) { + newUrlBuilder.authority(uriReplacement.host); + } + // Combine existing and new query parameters, giving precedence to new ones. + Map combinedParams = new TreeMap<>(); + // Add existing parameters. + for (String existingParamName : baseUri.getQueryParameterNames()) { + combinedParams.put( + existingParamName, checkNotNull(baseUri.getQueryParameter(existingParamName))); + } + // Add new parameters, overwriting existing ones if keys clash. + combinedParams.putAll(uriReplacement.params); + // Append all parameters from the sorted map. + for (Map.Entry param : combinedParams.entrySet()) { + newUrlBuilder.appendQueryParameter(param.getKey(), param.getValue()); + } + return newUrlBuilder.build(); + } + + private void stopInternal() { + steeringManifestTracker.stop(); + isActive = false; + currentPathwayPriority = null; + onStop(); + } + + private void performPathwayClones(ImmutableList pathwayClones) { + for (SteeringManifest.PathwayClone pathwayClone : pathwayClones) { + if (!isPathwayAvailable(pathwayClone.baseId) || isPathwayAvailable(pathwayClone.id)) { + continue; + } + performPathwayClone(pathwayClone); + } + } + + private class SteeringManifestTrackerCallback implements SteeringManifestTracker.Callback { + + @Override + public ImmutableMap getSteeringQueryParameters() { + checkState(isActive()); + return BaseContentSteeringTracker.this.getSteeringQueryParameters(); + } + + @Override + public void onSteeringManifestUpdated(SteeringManifest steeringManifest) { + checkState(isActive()); + performPathwayClones(steeringManifest.pathwayClones); + currentPathwayPriority = steeringManifest.pathwayPriority; + performPathwayEvaluation(currentPathwayPriority); + } + + @Override + public void onSteeringManifestLoadError(IOException error, boolean canceled) { + checkState(isActive()); + if (canceled && currentPathwayPriority == null) { + stopInternal(); + } + } + } +} diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/upstream/contentsteering/ContentSteeringTracker.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/upstream/contentsteering/ContentSteeringTracker.java deleted file mode 100644 index 56d784b9899..00000000000 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/upstream/contentsteering/ContentSteeringTracker.java +++ /dev/null @@ -1,60 +0,0 @@ -/* - * Copyright 2026 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * http://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package androidx.media3.exoplayer.upstream.contentsteering; - -import android.net.Uri; -import androidx.annotation.Nullable; -import androidx.media3.common.util.UnstableApi; -import androidx.media3.exoplayer.source.MediaSourceEventListener; - -/** Tracks the content steering states for an adaptive stream. */ -@UnstableApi -public interface ContentSteeringTracker { - - /** - * Starts the {@link ContentSteeringTracker}. - * - * @param initialSteeringManifestUri The initial {@link Uri} of the steering manifest. - * @param initialPathwayId The ID of the initial pathway to use before the first steering manifest - * is loaded, or {@code null} to allow the tracker to pick any available pathway before the - * first steering manifest is loaded. - * @param eventDispatcher A {@link MediaSourceEventListener.EventDispatcher} for reporting load - * events. - */ - void start( - Uri initialSteeringManifestUri, - @Nullable String initialPathwayId, - MediaSourceEventListener.EventDispatcher eventDispatcher); - - /** - * Excludes the current pathway for the given duration, in milliseconds. - * - * @param excludeDurationMs The duration for which to exclude the current pathway. - * @return Whether the exclusion was successful. - */ - boolean excludeCurrentPathway(long excludeDurationMs); - - /** - * Returns whether the {@link ContentSteeringTracker} is active. - * - *

    If this method returns {@code false}, the caller of the {@link ContentSteeringTracker} - * should behave as if content steering is absent. - */ - boolean isActive(); - - /** Stops the {@link ContentSteeringTracker}. */ - void stop(); -} diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/upstream/contentsteering/SteeringManifestTracker.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/upstream/contentsteering/SteeringManifestTracker.java index 00fa6d06861..0bf08823586 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/upstream/contentsteering/SteeringManifestTracker.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/upstream/contentsteering/SteeringManifestTracker.java @@ -179,11 +179,12 @@ private void loadSteeringManifestImmediately() { DataSpec dataSpec = new DataSpec.Builder().setUri(checkNotNull(steeringManifestUrlBuilder.build())).build(); ParsingLoadable steeringManifestLoadable = - new ParsingLoadable<>( - dataSourceFactory.createDataSource(), - dataSpec, - C.DATA_TYPE_STEERING_MANIFEST, - new SteeringManifestParser()); + new ParsingLoadable.Builder<>( + dataSourceFactory.createDataSource(), + dataSpec, + C.DATA_TYPE_STEERING_MANIFEST, + new SteeringManifestParser()) + .build(); checkNotNull(steeringManifestLoader) .startLoading( steeringManifestLoadable, @@ -208,14 +209,12 @@ private static Uri getSteeringManifestUrl( private static LoadEventInfo buildLoadEventInfo( ParsingLoadable loadable, long elapsedRealtimeMs, long loadDurationMs) { - return new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + return new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(); } private class SteeringManifestLoaderCallback diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/Av1ObuUtil.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/Av1ObuUtil.java index 9182b4fd78e..5722ab0a6e3 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/Av1ObuUtil.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/Av1ObuUtil.java @@ -15,33 +15,98 @@ */ package androidx.media3.exoplayer.video; +import androidx.annotation.Nullable; +import androidx.media3.common.util.CodecSpecificDataUtil; +import androidx.media3.common.util.Log; import androidx.media3.common.util.UnstableApi; import androidx.media3.container.ObuParser; import java.nio.BufferUnderflowException; import java.nio.ByteBuffer; +import java.util.ArrayList; import java.util.List; /** Utility methods for AV1 OBUs. */ @UnstableApi public final class Av1ObuUtil { - // Expected magic bytes for HDR10+ metadata in AV1 OBU. Spec: - // https://aomediacodec.github.io/av1-hdr10plus/#hdr10plus-metadata - private static final byte[] expectedAv1Hdr10PlusPayloadPrefix = { - // itu_t_t35_country_code == 0xB5 - (byte) 0xB5, - // itu_t_t35_terminal_provider_code == 0x003C - (byte) 0x00, - (byte) 0x3C, - // itu_t_t35_terminal_provider_oriented_code == 0x0001 - (byte) 0x00, - (byte) 0x01, - // application_identifier == 0x04 - (byte) 0x04, - }; + private static final String TAG = "Av1ObuUtil"; private Av1ObuUtil() {} + /** + * Extracts HAGC (ST 2094-50) metadata from the first valid HAGC ITU-T T.35 metadata OBU in the + * buffer. If one is found, all ITU-T T.35 metadata OBUs in the buffer (including HAGC and HDR10+) + * are rewritten as unknown metadata OBUs so that the decoder does not prefer any in-band T.35 + * metadata over the returned HAGC metadata. + * + *

    If no valid HAGC metadata OBU is found, {@code buffer} remains unchanged. In either case, + * the {@code position()} and {@code limit()} of {@code buffer} remain unchanged. + * + * @param buffer The {@link ByteBuffer} containing AV1 OBUs. + * @return The HAGC payload without the ITU-T T.35 header and the AV1 OBU trailing bits ({@code + * 0x80} followed by any zero padding bytes), or {@code null} if no valid HAGC metadata OBU is + * found. + */ + @Nullable + public static byte[] extractHagcMetadataAndStripAllT35Metadata(ByteBuffer buffer) { + @Nullable byte[] hagcData = null; + List t35MetadataTypeIndices = new ArrayList<>(); + for (ObuParser.Obu obu : ObuParser.split(buffer)) { + if (obu.type != ObuParser.OBU_METADATA) { + continue; + } + ObuParser.Metadata metadata; + try { + metadata = ObuParser.Metadata.parse(obu); + } catch (BufferUnderflowException e) { + // Malformed metadata OBU, do not attempt to process it and let the underlying decoder deal + // with any potential errors. + continue; + } + if (metadata.type != ObuParser.Metadata.METADATA_TYPE_ITUT_T35) { + continue; + } + t35MetadataTypeIndices.add(obu.payload.position()); + if (!CodecSpecificDataUtil.isHagcMetadata(metadata.payload)) { + continue; + } + if (hagcData != null) { + Log.w(TAG, "Found multiple HAGC metadata OBUs in the same temporal unit. Using the first."); + continue; + } + hagcData = readHagcPayload(metadata.payload); + } + if (hagcData != null) { + for (int index : t35MetadataTypeIndices) { + // Mark as an unknown metadata OBU. See stripT35Metadata for details. + buffer.put(index, (byte) 0x1F); + } + } + return hagcData; + } + + /** + * Returns the HAGC payload in {@code payload} without the ITU-T T.35 header and the AV1 OBU + * trailing bits, or {@code null} if the payload is empty or the trailing bits are missing. + */ + @Nullable + private static byte[] readHagcPayload(ByteBuffer payload) { + int startIndex = payload.position() + CodecSpecificDataUtil.HAGC_T35_HEADER_LENGTH; + int lastNonZeroIndex = payload.limit() - 1; + while (lastNonZeroIndex >= startIndex && payload.get(lastNonZeroIndex) == 0) { + lastNonZeroIndex--; + } + int length = lastNonZeroIndex - startIndex; + if (length <= 0 || (payload.get(lastNonZeroIndex) & 0xFF) != 0x80) { + // Malformed OBU or empty HAGC payload. + return null; + } + byte[] hagcData = new byte[length]; + payload.position(startIndex); + payload.get(hagcData); + return hagcData; + } + /** * Rewrites all ITU-T T35 metadata OBUs from the buffer. This is done to prevent the decoder from * prioritizing in-band T.35 metadata (like HDR10+, Dolby Vision, etc.) over out-of-band metadata. @@ -84,7 +149,7 @@ private static void stripT35Metadata(ByteBuffer buffer, boolean keepHdr10Plus) { if (metadata.type != ObuParser.Metadata.METADATA_TYPE_ITUT_T35) { continue; } - if (!keepHdr10Plus || !isHdr10PlusMetadata(metadata.payload)) { + if (!keepHdr10Plus || !CodecSpecificDataUtil.isHdr10PlusMetadata(metadata.payload)) { // This is a metadata OBU with metadata type ITUT-T35, that we want to rewrite. Set the // first byte of the metadata type leb128 in the OBU payload to 0x1F to mark it as an // unknown metadata OBU. 0x1F is the leb128() encoding of the value 31 (which according to @@ -94,17 +159,4 @@ private static void stripT35Metadata(ByteBuffer buffer, boolean keepHdr10Plus) { } } } - - private static boolean isHdr10PlusMetadata(ByteBuffer payload) { - if (payload.remaining() < expectedAv1Hdr10PlusPayloadPrefix.length) { - return false; - } - int position = payload.position(); - for (int i = 0; i < expectedAv1Hdr10PlusPayloadPrefix.length; i++) { - if (payload.get(position + i) != expectedAv1Hdr10PlusPayloadPrefix[i]) { - return false; - } - } - return true; - } } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/HevcSeiUtil.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/HevcSeiUtil.java new file mode 100644 index 00000000000..87ba165bd7c --- /dev/null +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/HevcSeiUtil.java @@ -0,0 +1,192 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.exoplayer.video; + +import androidx.media3.common.util.CodecSpecificDataUtil; +import androidx.media3.common.util.UnstableApi; +import androidx.media3.container.NalUnitUtil; +import java.nio.ByteBuffer; + +/** Utility methods for handling HEVC SEI messages. */ +@UnstableApi +public final class HevcSeiUtil { + + private static final int SEI_PAYLOAD_TYPE_ITU_T_T35 = 4; + /* package */ static final int SEI_PAYLOAD_TYPE_UNSPECIFIED = 254; + + private HevcSeiUtil() {} + + /** + * Modifies the provided {@link ByteBuffer} in place to strip all ITU-T T.35 SEI messages that are + * not HDR10+. + * + *

    This is achieved by masking the payload type of any non-HDR10+ T.35 SEI message to an + * unspecified payload type ({@link #SEI_PAYLOAD_TYPE_UNSPECIFIED}). + * + * @param buffer The {@link ByteBuffer} containing the HEVC bitstream with Annex B start codes. + */ + public static void stripNonHdr10PlusT35Metadata(ByteBuffer buffer) { + stripT35Metadata(buffer, /* keepHdr10Plus= */ true); + } + + /** + * Modifies the provided {@link ByteBuffer} in place to strip all ITU-T T.35 SEI messages. + * + *

    This is achieved by masking the payload type of any T.35 SEI message to an unspecified + * payload type ({@link #SEI_PAYLOAD_TYPE_UNSPECIFIED}). The NAL unit lengths and message lengths + * remain unchanged, so the operation can be done in-place with zero memory allocations, avoiding + * the need to fully parse emulation prevention bytes or copy the buffer. + * + * @param buffer The {@link ByteBuffer} containing the HEVC bitstream with Annex B start codes. + */ + public static void stripAllT35Metadata(ByteBuffer buffer) { + stripT35Metadata(buffer, /* keepHdr10Plus= */ false); + } + + private static void stripT35Metadata(ByteBuffer buffer, boolean keepHdr10Plus) { + int position = buffer.position(); + int limit = buffer.limit(); + if (limit - position < 4) { + return; + } + + int searchOffset = position; + + while (searchOffset < limit) { + int nalUnitOffset = findNalUnit(buffer, searchOffset, limit); + if (nalUnitOffset == limit) { + break; // No more NAL units + } + + if (nalUnitOffset + 3 >= limit) { + break; // Not enough bytes for a NAL header. + } + + int nalUnitType = (buffer.get(nalUnitOffset + 3) & 0x7E) >> 1; + + // NAL unit types 0 to 31 indicate VCL NAL units. + // Prefix SEI messages always precede VCL NAL units, so we can safely stop parsing + // once we hit a VCL NAL unit to avoid scanning the encoded video data. + if (nalUnitType >= 0 && nalUnitType <= 31) { + break; + } + + if (nalUnitType == NalUnitUtil.H265_NAL_UNIT_TYPE_PREFIX_SEI) { + int seiOffset = nalUnitOffset + 3 + 2; // Skip 3-byte start code and 2-byte NAL header + + int nextNalUnitOffset = findNalUnit(buffer, seiOffset, limit); + int seiLimit = nextNalUnitOffset; // The SEI payload ends where the next NAL unit starts + + int consecutiveZeros = 0; // State tracked across the entire SEI NAL unit + + while (seiOffset < seiLimit) { + // Read payloadType + int payloadType = 0; + int t35PayloadTypeOffset = -1; + while (seiOffset < seiLimit) { + int b = buffer.get(seiOffset) & 0xFF; + if (b == 0x03 && consecutiveZeros >= 2) { + consecutiveZeros = 0; + seiOffset++; + continue; + } + consecutiveZeros = (b == 0x00) ? consecutiveZeros + 1 : 0; + + int currentOffset = seiOffset; + seiOffset++; + payloadType += b; + if (b != 0xFF) { + if (payloadType == SEI_PAYLOAD_TYPE_ITU_T_T35) { + t35PayloadTypeOffset = currentOffset; + } + break; + } + } + + if (seiOffset >= seiLimit) { + if (t35PayloadTypeOffset != -1) { + buffer.put(t35PayloadTypeOffset, (byte) SEI_PAYLOAD_TYPE_UNSPECIFIED); + } + break; + } + + // Read payloadSize + int payloadSize = 0; + while (seiOffset < seiLimit) { + int b = buffer.get(seiOffset) & 0xFF; + if (b == 0x03 && consecutiveZeros >= 2) { + consecutiveZeros = 0; + seiOffset++; + continue; + } + consecutiveZeros = (b == 0x00) ? consecutiveZeros + 1 : 0; + + seiOffset++; + payloadSize += b; + if (b != 0xFF) { + break; + } + } + + if (t35PayloadTypeOffset != -1) { + if (!keepHdr10Plus + || !CodecSpecificDataUtil.isHdr10PlusMetadata( + buffer, seiOffset, Math.min(seiOffset + payloadSize, seiLimit))) { + buffer.put(t35PayloadTypeOffset, (byte) SEI_PAYLOAD_TYPE_UNSPECIFIED); + } + } + + // Skip the logical payloadSize bytes, taking into account physical emulation + // prevention bytes (0x03) that might be in the payload. + for (int i = 0; i < payloadSize; i++) { + if (seiOffset >= seiLimit) { + break; + } + int b = buffer.get(seiOffset++) & 0xFF; + if (b == 0x03 && consecutiveZeros >= 2) { + // Emulation prevention byte. Doesn't count towards logical payloadSize. + consecutiveZeros = 0; + i--; + } else { + consecutiveZeros = (b == 0x00) ? consecutiveZeros + 1 : 0; + } + } + } + searchOffset = nextNalUnitOffset; + } else { + searchOffset = nalUnitOffset + 3; + } + } + } + + /** + * Finds the first NAL unit start code (0x00 00 01) in the buffer and returns the offset of the + * first zero byte of the start code. Returns {@code limit} if no start code is found. + */ + private static int findNalUnit(ByteBuffer buffer, int offset, int limit) { + int consecutiveZeros = 0; + int currentOffset = offset; + while (currentOffset < limit) { + int b = buffer.get(currentOffset) & 0xFF; + if (consecutiveZeros >= 2 && b == 0x01) { + return currentOffset - 2; + } + consecutiveZeros = (b == 0x00) ? consecutiveZeros + 1 : 0; + currentOffset++; + } + return limit; + } +} diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/MediaCodecVideoRenderer.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/MediaCodecVideoRenderer.java index bac9447aad9..30d77966963 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/MediaCodecVideoRenderer.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/MediaCodecVideoRenderer.java @@ -17,6 +17,7 @@ import static android.os.Build.VERSION.SDK_INT; import static android.view.Display.DEFAULT_DISPLAY; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; import static androidx.media3.exoplayer.DecoderReuseEvaluation.DISCARD_REASON_MAX_INPUT_SIZE_EXCEEDED; import static androidx.media3.exoplayer.DecoderReuseEvaluation.DISCARD_REASON_VIDEO_FRAME_RATE_CHANGED; import static androidx.media3.exoplayer.DecoderReuseEvaluation.DISCARD_REASON_VIDEO_MAX_RESOLUTION_EXCEEDED; @@ -53,9 +54,12 @@ import androidx.annotation.CallSuper; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; +import androidx.annotation.RestrictTo; import androidx.media3.common.C; +import androidx.media3.common.ColorInfo; import androidx.media3.common.DrmInitData; import androidx.media3.common.Effect; +import androidx.media3.common.Flags; import androidx.media3.common.Format; import androidx.media3.common.MediaLibraryInfo; import androidx.media3.common.MimeTypes; @@ -81,7 +85,6 @@ import androidx.media3.exoplayer.ExoPlayer; import androidx.media3.exoplayer.FormatHolder; import androidx.media3.exoplayer.PlayerMessage.Target; -import androidx.media3.exoplayer.Renderer; import androidx.media3.exoplayer.RendererCapabilities; import androidx.media3.exoplayer.ScrubbingModeParameters; import androidx.media3.exoplayer.mediacodec.MediaCodecAdapter; @@ -96,6 +99,7 @@ import com.google.common.collect.ImmutableList; import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.nio.ByteBuffer; +import java.util.Arrays; import java.util.List; import java.util.PriorityQueue; import org.checkerframework.checker.initialization.qual.Initialized; @@ -158,9 +162,6 @@ public class MediaCodecVideoRenderer extends MediaCodecRenderer private static final String KEY_CROP_BOTTOM = "crop-bottom"; private static final String KEY_CROP_TOP = "crop-top"; - // TODO: b/388762778 - Replace with MediaFormat.KEY_HDR_ST2094_50_INFO once compile SDK is 37. - private static final String KEY_HDR_ST2094_50_INFO = "hdr-st2094-50-info"; - // Long edge length in pixels for standard video formats, in decreasing in order. private static final int[] STANDARD_LONG_EDGE_VIDEO_PX = new int[] {1920, 1600, 1440, 1280, 960, 854, 640, 540, 480}; @@ -222,7 +223,7 @@ public class MediaCodecVideoRenderer extends MediaCodecRenderer private final VideoFrameReleaseControl videoFrameReleaseControl; private final VideoFrameReleaseControl.FrameReleaseInfo videoFrameReleaseInfo; private final FixedFrameRateEstimator frameRateEstimator; - @Nullable private final Av1SampleDependencyParser av1SampleDependencyParser; + private final Av1SampleDependencyParser av1SampleDependencyParser; /** * The earliest time threshold, in microseconds, after which decoder input buffers may be dropped. @@ -232,8 +233,6 @@ public class MediaCodecVideoRenderer extends MediaCodecRenderer @Nullable private final VideoFrameReleaseEarlyTimeForecaster videoFrameReleaseEarlyTimeForecaster; private final PriorityQueue discardedDecoderInputBufferTimestamps; - private final boolean enableMediaCodecBufferDecodeOnlyFlag; - private final boolean enableDurationToProgressUs; private @MonotonicNonNull CodecMaxValues codecMaxValues; private boolean codecNeedsSetOutputSurfaceWorkaround; @@ -259,7 +258,6 @@ public class MediaCodecVideoRenderer extends MediaCodecRenderer private boolean isFlushRequired; private long totalVideoFrameProcessingOffsetUs; private int videoFrameProcessingOffsetCount; - private long lastFrameReleaseTimeNs; private VideoSize decodedVideoSize; @Nullable private VideoSize reportedVideoSize; private int rendererPriority; @@ -287,11 +285,9 @@ public static final class Builder { private int maxDroppedFramesToNotify; private float assumedMinimumCodecOperatingRate; @Nullable private VideoSink videoSink; - private boolean parseAv1SampleDependencies; private long lateThresholdToDropDecoderInputUs; - private boolean enableMediaCodecBufferDecodeOnlyFlag; - private boolean enableDurationToProgressUs; private long earlySchedulingThresholdUs; + private boolean skipBuffersWithIdenticalReleaseTime; /** * Creates a new builder. @@ -303,9 +299,9 @@ public Builder(Context context) { this.mediaCodecSelector = MediaCodecSelector.DEFAULT; this.codecAdapterFactory = MediaCodecAdapter.Factory.getDefault(context); this.assumedMinimumCodecOperatingRate = 0; - this.parseAv1SampleDependencies = true; this.lateThresholdToDropDecoderInputUs = DEFAULT_LATE_THRESHOLD_TO_DROP_DECODER_INPUT_US; this.earlySchedulingThresholdUs = DEFAULT_EARLY_SCHEDULING_THRESHOLD_US; + this.skipBuffersWithIdenticalReleaseTime = true; } /** Sets the {@link MediaCodecSelector decoder selector}. */ @@ -403,22 +399,6 @@ public Builder setVideoSink(@Nullable VideoSink videoSink) { return this; } - /** - * Sets whether {@link MimeTypes#VIDEO_AV1} bitstream parsing for sample dependency information - * is enabled. Knowing which input frames are not depended on can speed up seeking and reduce - * dropped frames. - * - *

    Defaults to {@code true}. - * - *

    This method is experimental and will be renamed or removed in a future release. - */ - @CanIgnoreReturnValue - @ExperimentalApi // TODO: b/470365670 - Remove method once config is enabled by default. - public Builder experimentalSetParseAv1SampleDependencies(boolean parseAv1SampleDependencies) { - this.parseAv1SampleDependencies = parseAv1SampleDependencies; - return this; - } - /** * Sets the late threshold for rendered output buffers, in microseconds, after which decoder * input buffers may be dropped. @@ -441,50 +421,6 @@ public Builder experimentalSetLateThresholdToDropDecoderInputUs( return this; } - /** - * Sets whether the {@link MediaCodec#BUFFER_FLAG_DECODE_ONLY} flag will be included when - * queuing decode-only input buffers to the decoder. - * - *

    If {@code false}, then only if the decoder is set up in tunneling mode will decode-only - * input buffers be queued with the {@link MediaCodec#BUFFER_FLAG_DECODE_ONLY} flag. The default - * value is {@code false}. - * - *

    Requires API 34. - * - *

    This method is experimental and will be renamed or removed in a future release. - */ - @RequiresApi(34) - @CanIgnoreReturnValue - @ExperimentalApi // TODO: b/470367414 - Run experiments and enable by default. - public Builder experimentalSetEnableMediaCodecBufferDecodeOnlyFlag( - boolean enableMediaCodecBufferDecodeOnlyFlag) { - this.enableMediaCodecBufferDecodeOnlyFlag = enableMediaCodecBufferDecodeOnlyFlag; - return this; - } - - /** - * Sets whether the {@link #getDurationToProgressUs} is enabled. - * - *

    When ExoPlayer's {@link ExoPlayer.Builder#experimentalSetDynamicSchedulingEnabled dynamic - * scheduling} is enabled, ExoPlayer uses {@link Renderer#getDurationToProgressUs} to better - * align when it wakes the CPU with when player progress can be made. - * - *

    If {@code true}, then if the {@link MediaCodec} decoder is set up in asynchronous mode - * with a registered {@link MediaCodec.Callback} listener, {@link #getDurationToProgressUs} will - * return durations based on the next output frame's presentation time. This will increase CPU - * Idle time thereby reducing power consumption. The default value is {@code false}. - * - *

    This method is experimental and will be renamed or removed in a future release. - * - * @see ExoPlayer.Builder#experimentalSetDynamicSchedulingEnabled(boolean) - */ - @CanIgnoreReturnValue - @ExperimentalApi // TODO: b/369523131 - Remove once experiment is complete. - public Builder setEnableDurationToProgressUs(boolean enableDurationToProgressUs) { - this.enableDurationToProgressUs = enableDurationToProgressUs; - return this; - } - /** * Sets the threshold for how early a frame may be scheduled for release on the surface. * @@ -502,6 +438,21 @@ public Builder setEarlySchedulingThresholdUs(long earlySchedulingThresholdUs) { return this; } + /** + * Sets whether to skip buffers that have an identical release time as the previous released + * buffer. + * + * @param skipBuffersWithIdenticalReleaseTime Whether to skip buffers with identical release + * time. + * @return This builder. + */ + @CanIgnoreReturnValue + public Builder setSkipBuffersWithIdenticalReleaseTime( + boolean skipBuffersWithIdenticalReleaseTime) { + this.skipBuffersWithIdenticalReleaseTime = skipBuffersWithIdenticalReleaseTime; + return this; + } + /** * Builds the {@link MediaCodecVideoRenderer}. Must only be called once per Builder instance. * @@ -681,7 +632,10 @@ protected MediaCodecVideoRenderer(Builder builder) { VideoFrameReleaseControl.@Initialized FrameTimingEvaluator thisRef = this; videoFrameReleaseControl = new VideoFrameReleaseControl( - this.context, /* frameTimingEvaluator= */ thisRef, builder.allowedJoiningTimeMs); + this.context, + /* frameTimingEvaluator= */ thisRef, + builder.allowedJoiningTimeMs, + builder.skipBuffersWithIdenticalReleaseTime); videoFrameReleaseControl.setEarlySchedulingThresholdUs(builder.earlySchedulingThresholdUs); videoFrameReleaseInfo = new VideoFrameReleaseControl.FrameReleaseInfo(); frameRateEstimator = new FixedFrameRateEstimator(this::onFrameRateEstimateChanged); @@ -694,8 +648,7 @@ protected MediaCodecVideoRenderer(Builder builder) { reportedVideoSize = null; rendererPriority = C.PRIORITY_PLAYBACK; startPositionUs = C.TIME_UNSET; - av1SampleDependencyParser = - builder.parseAv1SampleDependencies ? new Av1SampleDependencyParser() : null; + av1SampleDependencyParser = new Av1SampleDependencyParser(); discardedDecoderInputBufferTimestamps = new PriorityQueue<>(); if (builder.lateThresholdToDropDecoderInputUs != C.TIME_UNSET) { minEarlyUsToDropDecoderInput = -builder.lateThresholdToDropDecoderInputUs; @@ -705,8 +658,6 @@ protected MediaCodecVideoRenderer(Builder builder) { minEarlyUsToDropDecoderInput = C.TIME_UNSET; videoFrameReleaseEarlyTimeForecaster = null; } - enableMediaCodecBufferDecodeOnlyFlag = builder.enableMediaCodecBufferDecodeOnlyFlag; - enableDurationToProgressUs = builder.enableDurationToProgressUs; nextOutputBufferToProcessPresentationTimeUs = C.TIME_UNSET; scrubbingModeParameters = null; } @@ -883,10 +834,10 @@ protected List getDecoderInfos( /** * Returns a list of decoders that can decode media in the specified format, in the priority order - * specified by the {@link MediaCodecSelector}. Note that since the {@link MediaCodecSelector} - * only has access to {@link Format#sampleMimeType}, the list is not ordered to account for - * whether each decoder supports the details of the format (e.g., taking into account the format's - * profile, level, resolution and so on). {@link + * specified by the {@link MediaCodecSelector}. Note that for formats other than Dolby Vision, + * since the {@link MediaCodecSelector} only has access to {@link Format#sampleMimeType}, the list + * is not filtered or ordered to account for whether each decoder supports the details of the + * format (e.g., taking into account the format's profile, level, resolution and so on). {@link * MediaCodecUtil#getDecoderInfosSortedByFormatSupport} can be used to further sort the list into * an order where decoders that fully support the format come first. * @@ -909,15 +860,24 @@ private static List getDecoderInfos( return ImmutableList.of(); } if (SDK_INT >= 26 - && MimeTypes.VIDEO_DOLBY_VISION.equals(format.sampleMimeType) + && format.sampleMimeType.equals(MimeTypes.VIDEO_DOLBY_VISION) && !Api26.doesDisplaySupportDolbyVision(context)) { List alternativeDecoderInfos = - MediaCodecUtil.getAlternativeDecoderInfos( - mediaCodecSelector, format, requiresSecureDecoder, requiresTunnelingDecoder); + MediaCodecUtil.getAlternativeDecoderInfosFilteredByFormatSupport( + context, mediaCodecSelector, format, requiresSecureDecoder, requiresTunnelingDecoder); + if (alternativeDecoderInfos.isEmpty()) { + alternativeDecoderInfos = + MediaCodecUtil.getAlternativeDecoderInfos( + mediaCodecSelector, format, requiresSecureDecoder, requiresTunnelingDecoder); + } if (!alternativeDecoderInfos.isEmpty()) { return alternativeDecoderInfos; } } + if (format.sampleMimeType.equals(MimeTypes.VIDEO_DOLBY_VISION)) { + return MediaCodecUtil.getDecoderInfosSoftMatchFilteredByFormatSupport( + context, mediaCodecSelector, format, requiresSecureDecoder, requiresTunnelingDecoder); + } return MediaCodecUtil.getDecoderInfosSoftMatch( mediaCodecSelector, format, requiresSecureDecoder, requiresTunnelingDecoder); } @@ -981,7 +941,6 @@ protected void onEnabled(boolean joining, boolean mayRenderStartOfStream) mayRenderStartOfStream ? RELEASE_FIRST_FRAME_IMMEDIATELY : RELEASE_FIRST_FRAME_WHEN_STARTED; - experimentalEnableProcessedStreamChangedAtStart(); } else { videoFrameReleaseControl.setClock(getClock()); int firstFrameReleaseInstruction = @@ -1053,6 +1012,7 @@ public void onError(VideoSink.VideoSinkException videoSinkException) { } /** Creates a {@link PlaybackVideoGraphWrapper} instance. */ + @RestrictTo(LIBRARY_GROUP) protected PlaybackVideoGraphWrapper createPlaybackVideoGraphWrapper( Context context, VideoFrameReleaseControl videoFrameReleaseControl) { // TODO: b/391109644 - Add a more explicit API to enable replaying. @@ -1462,9 +1422,7 @@ protected void resetCodecStateForFlush() { consecutiveDroppedInputBufferCount = 0; isFlushRequired = false; nextOutputBufferToProcessPresentationTimeUs = C.TIME_UNSET; - if (av1SampleDependencyParser != null) { - av1SampleDependencyParser.reset(); - } + av1SampleDependencyParser.reset(); } @Override @@ -1559,7 +1517,8 @@ public static int getCodecMaxInputSize(MediaCodecInfo codecInfo, Format format) @Override protected long getDurationToProgressUs( long positionUs, long elapsedRealtimeUs, boolean isOnBufferAvailableListenerRegistered) { - if (!enableDurationToProgressUs || !isOnBufferAvailableListenerRegistered) { + if (!Flags.isEnabled(Flags.FLAG_VIDEO_RENDERER_DURATION_TO_PROGRESS) + || !isOnBufferAvailableListenerRegistered) { return super.getDurationToProgressUs( positionUs, elapsedRealtimeUs, isOnBufferAvailableListenerRegistered); } @@ -1583,31 +1542,24 @@ protected long getDurationToProgressUs( return max(DEFAULT_DURATION_TO_PROGRESS_US, durationUs); } - try { - @VideoFrameReleaseControl.FrameReleaseAction - int frameReleaseAction = - videoFrameReleaseControl.getFrameReleaseAction( - nextOutputBufferToProcessPresentationTimeUs, - positionUs, - elapsedRealtimeUs, - getOutputStreamStartPositionUs(), - /* isDecodeOnlyFrame= */ false, - /* isLastFrame= */ false, - frameRateEstimator.getFrameDurationNs(), - frameRateEstimator.getFrameIndex(), - videoFrameReleaseInfo); - if (frameReleaseAction != VideoFrameReleaseControl.FRAME_RELEASE_TRY_AGAIN_LATER) { - return 0; - } - long durationUs = videoFrameReleaseInfo.getEarlyUs(); - // Rebase duration to the start of this iteration of the rendering loop. - durationUs += Util.msToUs(getClock().elapsedRealtime()) - elapsedRealtimeUs; - // Set a lower bound of waking within 1.5*VSync rate at 60 fps(25 ms). - return max(0, durationUs - 25_000); - } catch (ExoPlaybackException exoPlaybackException) { - Log.w(TAG, "Error while evaluating frame release action"); + @VideoFrameReleaseControl.FrameReleaseAction + int frameReleaseAction = + videoFrameReleaseControl.probeFrameReleaseAction( + nextOutputBufferToProcessPresentationTimeUs, + positionUs, + elapsedRealtimeUs, + getOutputStreamStartPositionUs(), + frameRateEstimator.getFrameDurationNs(), + frameRateEstimator.getFrameIndex(), + videoFrameReleaseInfo); + if (frameReleaseAction != VideoFrameReleaseControl.FRAME_RELEASE_TRY_AGAIN_LATER) { + return 0; } - return DEFAULT_DURATION_TO_PROGRESS_US; + long durationUs = videoFrameReleaseInfo.getEarlyUs(); + // Rebase duration to the start of this iteration of the rendering loop. + durationUs += Util.msToUs(getClock().elapsedRealtime()) - elapsedRealtimeUs; + // Set a lower bound of waking within 1.5*VSync rate at 60 fps(25 ms). + return max(0, durationUs - 25_000); } @Override @@ -1693,7 +1645,9 @@ protected void onCodecInitialized( codecHandlesHdr10PlusOutOfBandMetadata = checkNotNull(getCodecInfo()).isHdr10PlusOutOfBandMetadataSupported(); codecNeedsStripNonHdr10PlusT35Workaround = - SDK_INT < 37 && checkNotNull(getCodecInfo()).mimeType.equals(MimeTypes.VIDEO_AV1); + SDK_INT < 37 + && (checkNotNull(getCodecInfo()).mimeType.equals(MimeTypes.VIDEO_AV1) + || checkNotNull(getCodecInfo()).mimeType.equals(MimeTypes.VIDEO_H265)); maybeSetupTunnelingForFirstFrame(); } @@ -1713,6 +1667,7 @@ protected final boolean shouldReleaseCodecInsteadOfFlushing() { MediaCodecInfo codecInfo = getCodecInfo(); if (videoSink != null && codecInfo != null + && MediaLibraryInfo.enableWorkarounds() && (codecInfo.name.equals("c2.mtk.avc.decoder") || codecInfo.name.equals("c2.mtk.hevc.decoder") || codecInfo.name.equals("c2.mtk.vp9.decoder"))) { @@ -1773,22 +1728,48 @@ protected DecoderReuseEvaluation onInputFormatChanged(FormatHolder formatHolder) @CallSuper @Override protected void onQueueInputBuffer(DecoderInputBuffer buffer) throws ExoPlaybackException { - if (checkNotNull(getCodecInfo()).mimeType.equals(MimeTypes.VIDEO_AV1) && buffer.data != null) { + String mimeType = checkNotNull(getCodecInfo()).mimeType; + if (buffer.data != null) { ByteBuffer bufferData = buffer.data; - Format codecInputFormat = getCodecInputFormat(); - boolean isAv1HighBitdepth = - codecInputFormat != null - && codecInputFormat.colorInfo != null - && codecInputFormat.colorInfo.lumaBitdepth > 8; - if (isApplyingContainerHagcMetadata) { - // Strip all T.35 metadata to prevent in-band metadata from overriding container HAGC. - Av1ObuUtil.stripAllT35Metadata(bufferData); - } else if (isAv1HighBitdepth && codecNeedsStripNonHdr10PlusT35Workaround) { - // For AV1 streams with bitdepth > 8, try to rewrite OBUs to skip non-HDR10+ metadata on - // SDKs prior to 37. - Av1ObuUtil.stripNonHdr10PlusT35Metadata(bufferData); + boolean isAv1 = mimeType.equals(MimeTypes.VIDEO_AV1); + if (!bufferData.isReadOnly()) { + Format codecInputFormat = getCodecInputFormat(); + boolean isHighBitdepth = + codecInputFormat != null + && codecInputFormat.colorInfo != null + && codecInputFormat.colorInfo.lumaBitdepth > 8; + boolean isHevc = mimeType.equals(MimeTypes.VIDEO_H265); + if (isApplyingContainerHagcMetadata) { + // Strip all AV1/HEVC T.35 metadata to prevent in-band metadata from overriding container + // HAGC. + if (isAv1) { + Av1ObuUtil.stripAllT35Metadata(bufferData); + } else if (isHevc) { + HevcSeiUtil.stripAllT35Metadata(bufferData); + } + } else if (isHighBitdepth && codecNeedsStripNonHdr10PlusT35Workaround) { + // For AV1/HEVC streams with bitdepth > 8, rewrite OBUs/SEIs to skip non-HDR10+ metadata + // on SDKs prior to 37. + if (isAv1) { + Av1ObuUtil.stripNonHdr10PlusT35Metadata(bufferData); + } else if (isHevc) { + HevcSeiUtil.stripNonHdr10PlusT35Metadata(bufferData); + } + } else if (SDK_INT >= 37 && isHighBitdepth && isAv1) { + // For AV1 streams with bitdepth > 8, extract the HAGC metadata (if any) from the OBUs and + // set it explicitly on the MediaCodec instance on SDKs >= 37. This way the platform will + // use the metadata irrespective of which underlying AV1 decoder is being used. All T.35 + // metadata is also stripped so that the decoder does not prefer it over the HAGC + // metadata set explicitly below. + byte[] hagcData = Av1ObuUtil.extractHagcMetadataAndStripAllT35Metadata(bufferData); + if (hagcData != null) { + Bundle codecParameters = new Bundle(); + codecParameters.putByteArray(MediaFormat.KEY_HDR_ST2094_50_INFO, hagcData); + checkNotNull(getCodec()).setParameters(codecParameters); + } + } } - if (av1SampleDependencyParser != null && buffer.isKeyFrame()) { + if (isAv1 && buffer.isKeyFrame()) { av1SampleDependencyParser.queueInputBuffer(bufferData); } } @@ -1804,7 +1785,7 @@ protected void onQueueInputBuffer(DecoderInputBuffer buffer) throws ExoPlaybackE @Override protected int getCodecBufferFlags(DecoderInputBuffer buffer) { if (SDK_INT >= 34 - && (enableMediaCodecBufferDecodeOnlyFlag + && (Flags.isEnabled(Flags.FLAG_ENABLE_MEDIACODEC_BUFFER_DECODE_ONLY) || (scrubbingModeParameters != null && scrubbingModeParameters.useDecodeOnlyFlag) || tunneling) && isBufferBeforeStartTime(buffer) @@ -1841,8 +1822,7 @@ protected boolean shouldDiscardDecoderInputBuffer(DecoderInputBuffer buffer) { if (buffer.notDependedOn()) { bufferDiscarded = true; buffer.clear(); - } else if (av1SampleDependencyParser != null - && checkNotNull(getCodecInfo()).mimeType.equals(MimeTypes.VIDEO_AV1) + } else if (checkNotNull(getCodecInfo()).mimeType.equals(MimeTypes.VIDEO_AV1) && buffer.data != null) { boolean skipFrameHeaders = isBufferBeforeStartTime @@ -2012,14 +1992,11 @@ protected void handleInputBufferSupplementalData(DecoderInputBuffer buffer) data.position(0); setHdr10PlusInfoV29(checkNotNull(getCodec()), hdr10PlusInfo); } - } else if (SDK_INT >= 37 - && ituTT35CountryCode == (byte) 0xB5 - && ituTT35TerminalProviderCode == 0x0090 - && ituTT35TerminalProviderOrientedCode == 0x0001) { + } else if (SDK_INT >= 37 && CodecSpecificDataUtil.isHagcMetadata(data)) { // HAGC (ST 2094-50) metadata from out-of-band track. // Android's MediaCodec API for ST 2094-50 expects the payload without the 5-byte T.35 // header. - data.position(5); + data.position(CodecSpecificDataUtil.HAGC_T35_HEADER_LENGTH); byte[] hagcData = new byte[data.remaining()]; data.get(hagcData); data.position(0); @@ -2027,7 +2004,7 @@ protected void handleInputBufferSupplementalData(DecoderInputBuffer buffer) isApplyingContainerHagcMetadata = hagcData.length > 0; if (isApplyingContainerHagcMetadata) { Bundle codecParameters = new Bundle(); - codecParameters.putByteArray(KEY_HDR_ST2094_50_INFO, hagcData); + codecParameters.putByteArray(MediaFormat.KEY_HDR_ST2094_50_INFO, hagcData); checkNotNull(getCodec()).setParameters(codecParameters); } } @@ -2061,6 +2038,12 @@ protected boolean processOutputBuffer( skipOutputBuffer(codec, bufferIndex, presentationTimeUs); return true; } + if (pendingVideoSinkInputStreamChange) { + onOutputFormatChanged(format, codec.getOutputFormat()); + if (pendingVideoSinkInputStreamChange) { + return false; + } + } return videoSink.handleInputFrame( bufferPresentationTimeUs, new VideoSink.VideoFrameHandler() { @@ -2101,8 +2084,7 @@ public void skip() { positionUs, isDecodeOnlyBuffer, isLastBuffer, - videoFrameReleaseInfo, - lastFrameReleaseTimeNs); + videoFrameReleaseInfo); } nextOutputBufferToProcessPresentationTimeUs = frameReleaseAction == VideoFrameReleaseControl.FRAME_RELEASE_TRY_AGAIN_LATER @@ -2169,18 +2151,9 @@ private void releaseFrame( MediaCodecAdapter codec, int bufferIndex, long presentationTimeUs, Format format) { long releaseTimeNs = videoFrameReleaseInfo.getReleaseTimeNs(); long earlyUs = videoFrameReleaseInfo.getEarlyUs(); - if (shouldSkipBuffersWithIdenticalReleaseTime() && releaseTimeNs == lastFrameReleaseTimeNs) { - // This frame should be displayed on the same vsync with the previous released frame. We - // are likely rendering frames at a rate higher than the screen refresh rate. Skip - // this buffer so that it's returned to MediaCodec sooner otherwise MediaCodec may not - // be able to keep decoding with this rate [b/263454203]. - skipOutputBuffer(codec, bufferIndex, presentationTimeUs); - } else { - notifyFrameMetadataListener(presentationTimeUs, releaseTimeNs, format); - renderOutputBufferV21(codec, bufferIndex, presentationTimeUs, releaseTimeNs); - } + notifyFrameMetadataListener(presentationTimeUs, releaseTimeNs, format); + renderOutputBufferV21(codec, bufferIndex, presentationTimeUs, releaseTimeNs); updateVideoFrameProcessingOffsetCounters(earlyUs); - lastFrameReleaseTimeNs = releaseTimeNs; } private void notifyFrameMetadataListener( @@ -2214,6 +2187,12 @@ protected void onProcessedOutputBuffer(long presentationTimeUs) { } } + @Override + @ExperimentalApi // TODO: b/470373575 - Remove this method. + protected boolean shouldProcessStreamChangeAtStart() { + return videoSink != null; + } + @Override protected void onProcessedStreamChange() { super.onProcessedStreamChange(); @@ -2260,14 +2239,6 @@ protected boolean shouldDropBuffersToKeyframe( return earlyUs < MIN_EARLY_US_VERY_LATE_THRESHOLD && !isLastBuffer; } - /** - * Returns whether to skip buffers that have an identical release time as the previous released - * buffer. - */ - protected boolean shouldSkipBuffersWithIdenticalReleaseTime() { - return true; - } - /** * Returns whether to force rendering an output buffer. * @@ -2684,7 +2655,7 @@ protected MediaFormat getMediaFormat( MediaFormatUtil.maybeSetInteger( mediaFormat, MediaFormat.KEY_MAX_INPUT_SIZE, codecMaxValues.inputSize); // Set codec configuration values. - mediaFormat.setInteger(MediaFormat.KEY_PRIORITY, 0 /* realtime priority */); + mediaFormat.setInteger(MediaFormat.KEY_PRIORITY, C.MEDIA_CODEC_PRIORITY_REALTIME); if (codecOperatingRate != CODEC_OPERATING_RATE_UNSET) { mediaFormat.setFloat(MediaFormat.KEY_OPERATING_RATE, codecOperatingRate); } @@ -2736,9 +2707,11 @@ protected CodecMaxValues getCodecMaxValues( } boolean haveUnknownDimensions = false; for (Format streamFormat : streamFormats) { - if (format.colorInfo != null && streamFormat.colorInfo == null) { - // streamFormat likely has incomplete color information. Copy the complete color information - // from format to avoid codec re-use being ruled out for only this reason. + if (format.colorInfo != null + && isPartialColorInfoCompatible(format.colorInfo, streamFormat.colorInfo)) { + // streamFormat likely has incomplete color information (for example, when parsed from a + // manifest without container-level metadata). Copy the complete color information from + // format to avoid codec re-use being ruled out for only this reason. streamFormat = streamFormat.buildUpon().setColorInfo(format.colorInfo).build(); } if (codecInfo.canReuseCodec(format, streamFormat).result != REUSE_RESULT_NO) { @@ -2766,6 +2739,25 @@ protected CodecMaxValues getCodecMaxValues( return new CodecMaxValues(maxWidth, maxHeight, maxInputSize); } + private static boolean isPartialColorInfoCompatible( + ColorInfo fullColorInfo, @Nullable ColorInfo partialColorInfo) { + if (partialColorInfo == null) { + return true; + } + return (partialColorInfo.colorSpace == Format.NO_VALUE + || partialColorInfo.colorSpace == fullColorInfo.colorSpace) + && (partialColorInfo.colorRange == Format.NO_VALUE + || partialColorInfo.colorRange == fullColorInfo.colorRange) + && (partialColorInfo.colorTransfer == Format.NO_VALUE + || partialColorInfo.colorTransfer == fullColorInfo.colorTransfer) + && (partialColorInfo.hdrStaticInfo == null + || Arrays.equals(partialColorInfo.hdrStaticInfo, fullColorInfo.hdrStaticInfo)) + && (partialColorInfo.lumaBitdepth == Format.NO_VALUE + || partialColorInfo.lumaBitdepth == fullColorInfo.lumaBitdepth) + && (partialColorInfo.chromaBitdepth == Format.NO_VALUE + || partialColorInfo.chromaBitdepth == fullColorInfo.chromaBitdepth); + } + @Override protected MediaCodecDecoderException createDecoderException( Throwable cause, @Nullable MediaCodecInfo codecInfo) { @@ -2914,8 +2906,7 @@ private static void debugLogForBufferRelease( long positionUs, boolean isDecodeOnlyBuffer, boolean isLastBuffer, - VideoFrameReleaseControl.FrameReleaseInfo videoFrameReleaseInfo, - long lastFrameReleaseTimeNs) { + VideoFrameReleaseControl.FrameReleaseInfo videoFrameReleaseInfo) { if (frameReleaseAction == VideoFrameReleaseControl.FRAME_RELEASE_TRY_AGAIN_LATER) { return; } @@ -2948,9 +2939,6 @@ private static void debugLogForBufferRelease( case VideoFrameReleaseControl.FRAME_RELEASE_SCHEDULED: long releaseTimeNs = videoFrameReleaseInfo.getReleaseTimeNs(); debugString += ", release=" + (releaseTimeNs / 1000); - if (lastFrameReleaseTimeNs != 0) { - debugString += " (+" + (releaseTimeNs - lastFrameReleaseTimeNs) / 1000 + ")"; - } break; case VideoFrameReleaseControl.FRAME_RELEASE_TRY_AGAIN_LATER: default: diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/PlaybackVideoGraphWrapper.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/PlaybackVideoGraphWrapper.java index fede5452832..0f53bd48c04 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/PlaybackVideoGraphWrapper.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/PlaybackVideoGraphWrapper.java @@ -50,6 +50,7 @@ import androidx.media3.common.util.ExperimentalApi; import androidx.media3.common.util.GlUtil; import androidx.media3.common.util.GlUtil.GlException; +import androidx.media3.common.util.HandlerExecutor; import androidx.media3.common.util.HandlerWrapper; import androidx.media3.common.util.Log; import androidx.media3.common.util.Size; @@ -600,6 +601,8 @@ private boolean registerInput(Format sourceFormat, int inputIndex) throw new VideoSink.VideoSinkException(e, sourceFormat); } handler = clock.createHandler(checkNotNull(Looper.myLooper()), /* callback= */ null); + Executor listenerExecutor = + new HandlerExecutor(handler, e -> onError(VideoFrameProcessingException.from(e))); try { // TODO: b/412585856 - Allow setting CompositorSetting and CompositionEffects dynamically. videoGraph = @@ -608,7 +611,7 @@ private boolean registerInput(Format sourceFormat, int inputIndex) outputColorInfo, DebugViewProvider.NONE, /* listener= */ this, - /* listenerExecutor= */ handler::post, + listenerExecutor, /* initialTimestampOffsetUs= */ 0, /* renderFramesAutomatically= */ false); videoGraph.setCompositionEffects(compositionEffects); @@ -623,7 +626,7 @@ private boolean registerInput(Format sourceFormat, int inputIndex) maybeSetOutputSurfaceInfo(surface, size.getWidth(), size.getHeight()); } defaultVideoSink.initialize(sourceFormat); - defaultVideoSink.setListener(new DefaultVideoSinkListener(), /* executor= */ handler::post); + defaultVideoSink.setListener(new DefaultVideoSinkListener(), listenerExecutor); state = STATE_INITIALIZED; } else { if (!isInitialized()) { @@ -1286,7 +1289,11 @@ public VideoGraph create( .newInstance(videoFrameProcessorFactory); // LINT.ThenChange( // ../../../../../../../proguard-rules.txt, - // ../ExoPlayerImpl.java:set_video_effects) + // ../ExoPlayerImpl.java:set_video_effects, + // + // ../../../../../../../../test_proguard_minimal/src/androidTest/proguard-checkdiscard-rules.pgcfg, + // + // ../../../../../../../../test_proguard_minimal/src/androidTest/java/androidx/media3/test/proguard/minimal/ExoPlayerMinimalModuleProguardTest.java) } catch (Exception e) { throw new IllegalStateException(e); } @@ -1327,7 +1334,12 @@ private static final class ReflectiveDefaultVideoFrameProcessorFactory .append("$Factory$Builder") .toString(); return Class.forName(className); - // LINT.ThenChange(../../../../../../../proguard-rules.txt) + // LINT.ThenChange( + // ../../../../../../../proguard-rules.txt, + // + // ../../../../../../../../test_proguard_minimal/src/androidTest/proguard-checkdiscard-rules.pgcfg, + // + // ../../../../../../../../test_proguard_minimal/src/androidTest/java/androidx/media3/test/proguard/minimal/ExoPlayerMinimalModuleProguardTest.java) } catch (Exception e) { throw new IllegalStateException(e); } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/VideoFrameReleaseControl.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/VideoFrameReleaseControl.java index f61396f44a9..89a967c6541 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/VideoFrameReleaseControl.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/VideoFrameReleaseControl.java @@ -30,6 +30,7 @@ import androidx.annotation.FloatRange; import androidx.annotation.IntDef; import androidx.annotation.Nullable; +import androidx.annotation.VisibleForTesting; import androidx.media3.common.C; import androidx.media3.common.util.Clock; import androidx.media3.common.util.ExperimentalApi; @@ -47,7 +48,8 @@ public final class VideoFrameReleaseControl { /** * The frame release action returned by {@link #getFrameReleaseAction(long, long, long, long, - * boolean, boolean, long, long, FrameReleaseInfo)}. + * boolean, boolean, long, long, FrameReleaseInfo)} or {@link #probeFrameReleaseAction(long, long, + * long, long, long, long, FrameReleaseInfo)}. * *

    One of {@link #FRAME_RELEASE_IMMEDIATELY}, {@link #FRAME_RELEASE_SCHEDULED}, {@link * #FRAME_RELEASE_DROP}, {@link #FRAME_RELEASE_IGNORE}, {@link #FRAME_RELEASE_SKIP} or {@link @@ -88,6 +90,17 @@ public final class VideoFrameReleaseControl { /** Signals that a frame should not be released and the caller should try again later. */ public static final int FRAME_RELEASE_TRY_AGAIN_LATER = 5; + /** + * The fractional threshold of a vsync duration allowed when detecting frame starvation. + * + *

    When checking if starvation caused a frame to miss its intended vsync, the gap between frame + * releases is expected to exceed the intended presentation gap by at least one vsync duration. + * Applying this threshold reduces the required gap to {@code STARVATION_TOLERANCE * + * vsyncDurationNs} to account for minor timing fluctuations in vsync arrival or release + * execution. + */ + private static final float STARVATION_TOLERANCE = 0.85f; + /** Per {@link FrameReleaseAction} metadata. */ public static class FrameReleaseInfo { private long earlyUs; @@ -173,6 +186,7 @@ boolean shouldIgnoreFrame( private final FrameTimingEvaluator frameTimingEvaluator; private final VideoFrameReleaseHelper frameReleaseHelper; private final long allowedJoiningTimeMs; + private final boolean skipBuffersWithIdenticalReleaseTime; private boolean started; private @C.FirstFrameState int firstFrameState; @@ -186,6 +200,10 @@ boolean shouldIgnoreFrame( private boolean frameReadyWithoutSurface; private boolean disableAdvancingTimestampChecks; private boolean requiresOutputSurface; + private long lastFrameReleaseTimeNs; + private long prevFrameReleaseTimeNs; + private long lastFramePresentationTimeUs; + private long prevFramePresentationTimeUs; /** * Creates an instance. @@ -196,13 +214,17 @@ boolean shouldIgnoreFrame( * FrameReleaseInfo) frame release actions}. * @param allowedJoiningTimeMs The maximum duration in milliseconds for which the caller can * attempt to seamlessly join an ongoing playback. + * @param skipBuffersWithIdenticalReleaseTime Whether to skip buffers that have an identical + * release time as the previous released buffer. */ public VideoFrameReleaseControl( Context applicationContext, FrameTimingEvaluator frameTimingEvaluator, - long allowedJoiningTimeMs) { + long allowedJoiningTimeMs, + boolean skipBuffersWithIdenticalReleaseTime) { this.frameTimingEvaluator = frameTimingEvaluator; this.allowedJoiningTimeMs = allowedJoiningTimeMs; + this.skipBuffersWithIdenticalReleaseTime = skipBuffersWithIdenticalReleaseTime; frameReleaseHelper = new VideoFrameReleaseHelper(applicationContext); firstFrameState = C.FIRST_FRAME_NOT_RENDERED_ONLY_ALLOWED_IF_STARTED; initialPositionUs = C.TIME_UNSET; @@ -211,6 +233,7 @@ public VideoFrameReleaseControl( clock = Clock.DEFAULT; requiresOutputSurface = true; earlySchedulingThresholdUs = DEFAULT_EARLY_SCHEDULING_THRESHOLD_US; + resetReleasedFrameState(); } /** @@ -266,6 +289,7 @@ public void onStopped() { started = false; joiningDeadlineMs = C.TIME_UNSET; frameReleaseHelper.onStopped(); + resetReleasedFrameState(); } /** Called when the display surface changed. */ @@ -396,67 +420,60 @@ public boolean isJoining() { long frameIndex, FrameReleaseInfo frameReleaseInfo) throws ExoPlaybackException { - frameReleaseInfo.reset(); - - if (started && initialPositionUs == C.TIME_UNSET) { - initialPositionUs = positionUs; - } - - frameReleaseInfo.earlyUs = - calculateEarlyTimeUs(positionUs, elapsedRealtimeUs, presentationTimeUs); + return getFrameReleaseActionInternal( + presentationTimeUs, + positionUs, + elapsedRealtimeUs, + outputStreamStartPositionUs, + isDecodeOnlyFrame, + isLastFrame, + frameDurationNs, + frameIndex, + /* onlyProbe= */ false, + frameReleaseInfo); + } - if (isDecodeOnlyFrame && !isLastFrame) { - return FRAME_RELEASE_SKIP; - } - if (!hasOutputSurface && requiresOutputSurface) { - // Skip frames in sync with playback, so we'll be at the right frame if a surface is set. - if (frameTimingEvaluator.shouldIgnoreFrame( - frameReleaseInfo.earlyUs, + /** + * Probes the expected {@link FrameReleaseAction} and populates {@code frameReleaseInfo} for a + * pending video frame without mutating release state or invoking {@link + * FrameTimingEvaluator#shouldIgnoreFrame}. + * + * @param presentationTimeUs The presentation time of the video frame, in microseconds. + * @param positionUs The current playback position, in microseconds. + * @param elapsedRealtimeUs {@link android.os.SystemClock#elapsedRealtime()} in microseconds, + * taken approximately at the time the playback position was {@code positionUs}. + * @param outputStreamStartPositionUs The stream's start position, in microseconds. + * @param frameDurationNs The estimated fixed frame duration in nanoseconds, or {@link + * C#TIME_UNSET} if unknown. + * @param frameIndex A monotonically increasing index for the frame, or {@link C#INDEX_UNSET} if + * unknown. + * @param frameReleaseInfo A {@link FrameReleaseInfo} that will be filled with timing details. + * @return The expected {@link FrameReleaseAction} for the frame. + */ + public @FrameReleaseAction int probeFrameReleaseAction( + long presentationTimeUs, + long positionUs, + long elapsedRealtimeUs, + long outputStreamStartPositionUs, + long frameDurationNs, + long frameIndex, + FrameReleaseInfo frameReleaseInfo) { + try { + return getFrameReleaseActionInternal( + presentationTimeUs, positionUs, elapsedRealtimeUs, - isLastFrame, - /* treatDroppedBuffersAsSkipped= */ true)) { - return FRAME_RELEASE_IGNORE; - } - if (started && frameReleaseInfo.earlyUs < 30_000) { - return FRAME_RELEASE_SKIP; - } - frameReadyWithoutSurface = true; - return FRAME_RELEASE_TRY_AGAIN_LATER; - } - if (!requiresOutputSurface) { - frameReadyWithoutSurface = true; - } - if (shouldForceRelease(positionUs, frameReleaseInfo.earlyUs, outputStreamStartPositionUs)) { - return FRAME_RELEASE_IMMEDIATELY; - } - if (!started || positionUs == initialPositionUs) { - return FRAME_RELEASE_TRY_AGAIN_LATER; - } - - // Calculate release time and adjust earlyUs to screen vsync. - long systemTimeNs = clock.nanoTime(); - frameReleaseInfo.releaseTimeNs = - frameReleaseHelper.adjustReleaseTime( - systemTimeNs + (frameReleaseInfo.earlyUs * 1_000), - presentationTimeUs, - frameDurationNs, - frameIndex); - frameReleaseInfo.earlyUs = (frameReleaseInfo.releaseTimeNs - systemTimeNs) / 1_000; - // While joining, late frames are skipped while we catch up with the playback position. - boolean treatDropAsSkip = - joiningDeadlineMs != C.TIME_UNSET && !joiningRenderNextFrameImmediately; - if (frameTimingEvaluator.shouldIgnoreFrame( - frameReleaseInfo.earlyUs, positionUs, elapsedRealtimeUs, isLastFrame, treatDropAsSkip)) { - return FRAME_RELEASE_IGNORE; - } else if (frameTimingEvaluator.shouldDropFrame( - frameReleaseInfo.earlyUs, elapsedRealtimeUs, isLastFrame)) { - // While joining, dropped buffers are considered skipped. - return treatDropAsSkip ? FRAME_RELEASE_SKIP : FRAME_RELEASE_DROP; - } else if (frameReleaseInfo.earlyUs > earlySchedulingThresholdUs) { - return FRAME_RELEASE_TRY_AGAIN_LATER; + outputStreamStartPositionUs, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + frameDurationNs, + frameIndex, + /* onlyProbe= */ true, + frameReleaseInfo); + } catch (ExoPlaybackException e) { + // Never thrown when onlyProbe is true because shouldIgnoreFrame is not called. + throw new IllegalStateException(e); } - return FRAME_RELEASE_SCHEDULED; } /** Resets the release control. */ @@ -466,6 +483,18 @@ public void reset() { lowerFirstFrameState(C.FIRST_FRAME_NOT_RENDERED); joiningDeadlineMs = C.TIME_UNSET; frameReadyWithoutSurface = false; + resetReleasedFrameState(); + } + + /** + * Sets the vsync timing data. + * + * @param vsyncSampleTimeNs The vsync sample time in nanoseconds. + * @param vsyncDurationNs The vsync duration in nanoseconds. + */ + @VisibleForTesting + public void setVsyncData(long vsyncSampleTimeNs, long vsyncDurationNs) { + frameReleaseHelper.setVsyncData(vsyncSampleTimeNs, vsyncDurationNs); } /** @@ -552,4 +581,148 @@ private boolean shouldForceRelease( throw new IllegalStateException(); } } + + /** + * Returns a {@link FrameReleaseAction} for a video frame which instructs the caller what to do + * with the frame. + * + *

    When {@code onlyProbe} is {@code true}, this method evaluates the release action and + * populates {@code frameReleaseInfo} without mutating release state or calling {@link + * FrameTimingEvaluator#shouldIgnoreFrame}. + */ + private @FrameReleaseAction int getFrameReleaseActionInternal( + long presentationTimeUs, + long positionUs, + long elapsedRealtimeUs, + long outputStreamStartPositionUs, + boolean isDecodeOnlyFrame, + boolean isLastFrame, + long frameDurationNs, + long frameIndex, + boolean onlyProbe, + FrameReleaseInfo frameReleaseInfo) + throws ExoPlaybackException { + frameReleaseInfo.reset(); + + if (!onlyProbe && started && initialPositionUs == C.TIME_UNSET) { + initialPositionUs = positionUs; + } + + frameReleaseInfo.earlyUs = + calculateEarlyTimeUs(positionUs, elapsedRealtimeUs, presentationTimeUs); + + if (isDecodeOnlyFrame && !isLastFrame) { + return FRAME_RELEASE_SKIP; + } + if (!hasOutputSurface && requiresOutputSurface) { + // Skip frames in sync with playback, so we'll be at the right frame if a surface is set. + if (!onlyProbe + && frameTimingEvaluator.shouldIgnoreFrame( + frameReleaseInfo.earlyUs, + positionUs, + elapsedRealtimeUs, + isLastFrame, + /* treatDroppedBuffersAsSkipped= */ true)) { + return FRAME_RELEASE_IGNORE; + } + if (started && frameReleaseInfo.earlyUs < 30_000) { + return FRAME_RELEASE_SKIP; + } + if (!onlyProbe) { + frameReadyWithoutSurface = true; + } + return FRAME_RELEASE_TRY_AGAIN_LATER; + } + if (!onlyProbe && !requiresOutputSurface) { + frameReadyWithoutSurface = true; + } + if (shouldForceRelease(positionUs, frameReleaseInfo.earlyUs, outputStreamStartPositionUs)) { + if (!onlyProbe) { + updateReleasedFrameState(clock.nanoTime(), presentationTimeUs); + } + return FRAME_RELEASE_IMMEDIATELY; + } + if (!started || initialPositionUs == C.TIME_UNSET || positionUs == initialPositionUs) { + return FRAME_RELEASE_TRY_AGAIN_LATER; + } + + long systemTimeNs = clock.nanoTime(); + long unadjustedReleaseTimeNs = systemTimeNs + (frameReleaseInfo.earlyUs * 1_000); + frameReleaseInfo.releaseTimeNs = + onlyProbe + ? frameReleaseHelper.probeAdjustedReleaseTime( + unadjustedReleaseTimeNs, presentationTimeUs, frameDurationNs, frameIndex) + : frameReleaseHelper.adjustReleaseTime( + unadjustedReleaseTimeNs, presentationTimeUs, frameDurationNs, frameIndex); + frameReleaseInfo.earlyUs = (frameReleaseInfo.releaseTimeNs - systemTimeNs) / 1_000; + + // While joining, late frames are skipped while we catch up with the playback position. + boolean treatDropAsSkip = + joiningDeadlineMs != C.TIME_UNSET && !joiningRenderNextFrameImmediately; + if (!onlyProbe + && frameTimingEvaluator.shouldIgnoreFrame( + frameReleaseInfo.earlyUs, + positionUs, + elapsedRealtimeUs, + isLastFrame, + treatDropAsSkip)) { + return FRAME_RELEASE_IGNORE; + } + if (frameTimingEvaluator.shouldDropFrame( + frameReleaseInfo.earlyUs, elapsedRealtimeUs, isLastFrame)) { + // While joining, dropped buffers are considered skipped. + return treatDropAsSkip ? FRAME_RELEASE_SKIP : FRAME_RELEASE_DROP; + } else if (frameReleaseInfo.earlyUs > earlySchedulingThresholdUs) { + return FRAME_RELEASE_TRY_AGAIN_LATER; + } else if (skipBuffersWithIdenticalReleaseTime + && frameReleaseInfo.releaseTimeNs == lastFrameReleaseTimeNs) { + return treatSameReleaseTimeAsDropped() ? FRAME_RELEASE_DROP : FRAME_RELEASE_SKIP; + } + if (!onlyProbe) { + updateReleasedFrameState(frameReleaseInfo.releaseTimeNs, presentationTimeUs); + } + return FRAME_RELEASE_SCHEDULED; + } + + private void resetReleasedFrameState() { + lastFrameReleaseTimeNs = C.TIME_UNSET; + prevFrameReleaseTimeNs = C.TIME_UNSET; + lastFramePresentationTimeUs = C.TIME_UNSET; + prevFramePresentationTimeUs = C.TIME_UNSET; + } + + private void updateReleasedFrameState(long releaseTimeNs, long presentationTimeUs) { + prevFrameReleaseTimeNs = lastFrameReleaseTimeNs; + lastFrameReleaseTimeNs = releaseTimeNs; + prevFramePresentationTimeUs = lastFramePresentationTimeUs; + lastFramePresentationTimeUs = presentationTimeUs; + } + + /** + * Evaluates whether two buffers scheduled for identical release times collided because of slow + * decoding starvation (DROP) or intentional high-FPS subsampling (SKIP). + * + *

    If the actual release timestamp gap between the previous two rendered frames exceeds the + * intended presentation timestamp gap by roughly a vsync duration, the renderer fell behind due + * to slow decoding and missed a vsync (return {@code true} -> DROP). Otherwise, the renderer is + * keeping up with playback speed and duplicate vsync assignments occur due to high content frame + * rate exceeding display refresh rate (return {@code false} -> SKIP). + */ + private boolean treatSameReleaseTimeAsDropped() { + long vsyncDurationNs = frameReleaseHelper.getVsyncDurationNs(); + if (lastFrameReleaseTimeNs == C.TIME_UNSET + || prevFrameReleaseTimeNs == C.TIME_UNSET + || lastFramePresentationTimeUs == C.TIME_UNSET + || prevFramePresentationTimeUs == C.TIME_UNSET + || vsyncDurationNs == C.TIME_UNSET) { + return true; + } + + long actualReleaseGapNs = lastFrameReleaseTimeNs - prevFrameReleaseTimeNs; + long presentationTimeGapUs = lastFramePresentationTimeUs - prevFramePresentationTimeUs; + long intendedReleaseGapNs = (long) ((presentationTimeGapUs * 1000) / (double) playbackSpeed); + long toleranceNs = (long) (vsyncDurationNs * STARVATION_TOLERANCE); + + return actualReleaseGapNs >= intendedReleaseGapNs + toleranceNs; + } } diff --git a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/VideoFrameReleaseHelper.java b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/VideoFrameReleaseHelper.java index 767b5cc06f7..f1b2c0d3cb7 100644 --- a/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/VideoFrameReleaseHelper.java +++ b/libraries/exoplayer/src/main/java/androidx/media3/exoplayer/video/VideoFrameReleaseHelper.java @@ -69,16 +69,12 @@ public final class VideoFrameReleaseHelper { private final Context context; - private boolean vsyncSampleBuilt; + private boolean vsyncSamplerBuilt; @Nullable private VSyncSampler vsyncSampler; private boolean started; @Nullable private Surface surface; - /** - * The media frame rate used to calculate the playback frame rate of the {@link Surface}. This may - * be different to {@link #formatFrameRate} if {@link #formatFrameRate} is unspecified or - * inaccurate. - */ + /** The media frame rate used to calculate the playback frame rate of the {@link Surface}. */ private float surfaceMediaFrameRate; /** The playback frame rate set on the {@link Surface}. */ @@ -133,12 +129,12 @@ public void setChangeFrameRateStrategy( public void onStarted() { started = true; resetAdjustment(); - if (!vsyncSampleBuilt) { + if (!vsyncSamplerBuilt) { vsyncSampler = VSyncSampler.maybeBuildInstance(context); if (vsyncSampler != null) { vsyncSampler.listener = vsyncSamplerListener; } - vsyncSampleBuilt = true; + vsyncSamplerBuilt = true; } if (vsyncSampler != null) { vsyncSampler.register(); @@ -266,6 +262,30 @@ public long adjustReleaseTime( return snappedTimeNs - (vsyncDurationNs * VSYNC_OFFSET_PERCENTAGE) / 100; } + /* package */ long probeAdjustedReleaseTime( + long releaseTimeNs, long presentationTimeUs, long frameDurationNs, long frameIndex) { + long savedLastAdjustedFrameIndex = lastAdjustedFrameIndex; + long savedLastAdjustedReleaseTimeNs = lastAdjustedReleaseTimeNs; + long savedLastAdjustedPresentationTimeUs = lastAdjustedPresentationTimeUs; + long savedLastVsyncHysteresisOffsetNs = lastVsyncHysteresisOffsetNs; + long savedPendingLastAdjustedFrameIndex = pendingLastAdjustedFrameIndex; + long savedPendingLastAdjustedReleaseTimeNs = pendingLastAdjustedReleaseTimeNs; + long savedPendingLastPresentationTimeUs = pendingLastPresentationTimeUs; + long savedPendingVsyncHysteresisOffsetNs = pendingVsyncHysteresisOffsetNs; + try { + return adjustReleaseTime(releaseTimeNs, presentationTimeUs, frameDurationNs, frameIndex); + } finally { + lastAdjustedFrameIndex = savedLastAdjustedFrameIndex; + lastAdjustedReleaseTimeNs = savedLastAdjustedReleaseTimeNs; + lastAdjustedPresentationTimeUs = savedLastAdjustedPresentationTimeUs; + lastVsyncHysteresisOffsetNs = savedLastVsyncHysteresisOffsetNs; + pendingLastAdjustedFrameIndex = savedPendingLastAdjustedFrameIndex; + pendingLastAdjustedReleaseTimeNs = savedPendingLastAdjustedReleaseTimeNs; + pendingLastPresentationTimeUs = savedPendingLastPresentationTimeUs; + pendingVsyncHysteresisOffsetNs = savedPendingVsyncHysteresisOffsetNs; + } + } + @VisibleForTesting public void setVsyncData(long vsyncSampleTimeNs, long vsyncDurationNs) { checkNotNull(vsyncSampler).sampledVsyncTimeNs = vsyncSampleTimeNs; @@ -280,6 +300,11 @@ public void setVsyncData(long vsyncSampleTimeNs, long vsyncDurationNs) { } } + /** Returns the vsync duration in nanoseconds, or {@link C#TIME_UNSET} if unknown. */ + public long getVsyncDurationNs() { + return vsyncSampler != null ? vsyncSampler.vsyncDurationNs : C.TIME_UNSET; + } + private void resetAdjustment() { lastAdjustedFrameIndex = C.INDEX_UNSET; pendingLastAdjustedFrameIndex = C.INDEX_UNSET; diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerAdPlaybackEnforcementTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerAdPlaybackEnforcementTest.java index 956b8105d00..123ff274ad7 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerAdPlaybackEnforcementTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerAdPlaybackEnforcementTest.java @@ -237,7 +237,9 @@ public void onTimelineChanged(Timeline timeline, int reason) { advance(player).untilTimelineChanges(); AdPlaybackState updatedAdPlaybackState = emptyAdPlaybackState - .withNewAdGroup(/* adGroupIndex= */ 0, /* adGroupTimeUs= */ 0) + .withNewAdGroup( + /* adGroupIndex= */ 0, + /* adGroupTimeUs= */ DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US) .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) .withAvailableAdMediaItem( /* adGroupIndex= */ 0, @@ -379,7 +381,7 @@ public void adInsertedAtCurrentPosition_notEnforcedWithPreRoll_preRollPlaying() TestExoPlayerBuilder playerBuilder = new TestExoPlayerBuilder(ApplicationProvider.getApplicationContext()); ExoPlayer player = playerBuilder.setEnforceAdPlaybackOnTimelineRefresh(false).build(); - long firstSampleTimeUs = 0; + long firstSampleTimeUs = DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US; AdPlaybackState initialAdPlaybackState = new AdPlaybackState("adsId", firstSampleTimeUs) .withAvailableAdMediaItem( @@ -463,7 +465,7 @@ public void onPlaybackStateChanged(int playbackState) { TestExoPlayerBuilder playerBuilder = new TestExoPlayerBuilder(ApplicationProvider.getApplicationContext()); ExoPlayer player = playerBuilder.setEnforceAdPlaybackOnTimelineRefresh(false).build(); - long firstSampleTimeUs = 0; + long firstSampleTimeUs = DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US; AdPlaybackState initialAdPlaybackState = new AdPlaybackState("adsId", firstSampleTimeUs + 1_000_000L) .withAvailableAdMediaItem( @@ -634,7 +636,7 @@ public void adInsertedAtCurrentPosition_notEnforcedNonPlaceholderSameUid_keepsAd TestExoPlayerBuilder playerBuilder = new TestExoPlayerBuilder(ApplicationProvider.getApplicationContext()); ExoPlayer player = playerBuilder.setEnforceAdPlaybackOnTimelineRefresh(false).build(); - long firstSampleTimeUs = 0; + long firstSampleTimeUs = DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US; AdPlaybackState initialAdPlaybackState = new AdPlaybackState("adsId", firstSampleTimeUs) .withAvailableAdMediaItem( diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerAdTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerAdTest.java index 1ec5d3fc57a..cf385c5781b 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerAdTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerAdTest.java @@ -25,10 +25,12 @@ import static androidx.media3.test.utils.robolectric.TestPlayerRunHelper.advance; import static androidx.media3.test.utils.robolectric.TestPlayerRunHelper.play; import static com.google.common.truth.Truth.assertThat; +import static com.google.common.truth.Truth.assertWithMessage; import static org.mockito.AdditionalMatchers.not; import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.anyInt; import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.atLeast; import static org.mockito.Mockito.mock; import static org.mockito.Mockito.never; import static org.mockito.Mockito.times; @@ -40,6 +42,8 @@ import androidx.annotation.Nullable; import androidx.media3.common.AdPlaybackState; import androidx.media3.common.C; +import androidx.media3.common.Flags; +import androidx.media3.common.Format; import androidx.media3.common.MediaItem; import androidx.media3.common.PlaybackException; import androidx.media3.common.Player; @@ -49,29 +53,42 @@ import androidx.media3.common.Timeline; import androidx.media3.common.Timeline.Period; import androidx.media3.common.Timeline.Window; +import androidx.media3.common.util.HandlerWrapper; import androidx.media3.common.util.SystemClock; import androidx.media3.common.util.Util; import androidx.media3.datasource.DataSpec; import androidx.media3.datasource.TransferListener; import androidx.media3.exoplayer.ExoPlayer.PreloadConfiguration; +import androidx.media3.exoplayer.analytics.PlaybackStatsListener; import androidx.media3.exoplayer.drm.DrmSessionEventListener; import androidx.media3.exoplayer.drm.DrmSessionManager; +import androidx.media3.exoplayer.drm.DrmSessionManagerProvider; import androidx.media3.exoplayer.source.MediaPeriod; +import androidx.media3.exoplayer.source.MediaSource; +import androidx.media3.exoplayer.source.MediaSource.MediaPeriodId; import androidx.media3.exoplayer.source.MediaSourceEventListener; +import androidx.media3.exoplayer.source.SampleStream; import androidx.media3.exoplayer.source.TrackGroupArray; import androidx.media3.exoplayer.source.ads.AdsMediaSource; import androidx.media3.exoplayer.source.ads.ServerSideAdInsertionMediaSource; import androidx.media3.exoplayer.upstream.Allocator; +import androidx.media3.exoplayer.upstream.LoadErrorHandlingPolicy; +import androidx.media3.exoplayer.video.VideoRendererEventListener; import androidx.media3.test.utils.ActionSchedule; import androidx.media3.test.utils.ActionSchedule.PlayerRunnable; +import androidx.media3.test.utils.BindFlag; import androidx.media3.test.utils.ExoPlayerTestRunner; import androidx.media3.test.utils.FakeAdsLoader; import androidx.media3.test.utils.FakeMediaPeriod; +import androidx.media3.test.utils.FakeMediaPeriod.TrackDataFactory; import androidx.media3.test.utils.FakeMediaSource; import androidx.media3.test.utils.FakeMediaSourceFactory; +import androidx.media3.test.utils.FakeSampleStream; +import androidx.media3.test.utils.FakeSampleStream.FakeSampleStreamItem; import androidx.media3.test.utils.FakeTimeline; import androidx.media3.test.utils.FakeTimeline.TimelineWindowDefinition; import androidx.media3.test.utils.FakeVideoRenderer; +import androidx.media3.test.utils.Media3FlagsRule; import androidx.media3.test.utils.TestExoPlayerBuilder; import androidx.media3.test.utils.robolectric.RobolectricUtil; import androidx.media3.test.utils.robolectric.ShadowMediaCodecConfig; @@ -109,17 +126,30 @@ public class ExoPlayerAdTest { */ private static final int TIMEOUT_MS = 10_000; - @Parameters(name = "preload={0}") + /** The number of content samples per second produced by the content source. */ + private static final float CONTENT_SAMPLES_PER_SECOND = 10; + + /** The duration of a content "segment", i.e. the spacing between content sync samples. */ + private static final long CONTENT_SEGMENT_DURATION_US = 10_000_000L; + + private static final int CONTENT_SAMPLES_PER_SEGMENT = + (int) (CONTENT_SAMPLES_PER_SECOND * CONTENT_SEGMENT_DURATION_US / C.MICROS_PER_SECOND); + + @Parameters(name = "preload={0} perStream={2}") public static ImmutableList params() { return ImmutableList.of( - new Object[] {false, new PreloadConfiguration(C.TIME_UNSET)}, - new Object[] {true, new PreloadConfiguration(5_000_000L)}); + new Object[] {false, new PreloadConfiguration(C.TIME_UNSET), false}, + new Object[] {false, new PreloadConfiguration(C.TIME_UNSET), true}, + new Object[] {true, new PreloadConfiguration(5_000_000L), false}, + new Object[] {true, new PreloadConfiguration(5_000_000L), true}); } @Rule - public ShadowMediaCodecConfig mediaCodecConfig = + public final ShadowMediaCodecConfig mediaCodecConfig = ShadowMediaCodecConfig.withAllDefaultSupportedCodecs(); + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); + // The explicit boolean parameter is only used to give clear test names. @Parameter(0) public boolean unusedIsPreloadEnabled; @@ -127,12 +157,18 @@ public static ImmutableList params() { @Parameter(1) public ExoPlayer.PreloadConfiguration preloadConfiguration; + // TODO: b/510217604 - Remove parameterization. + @Parameter(2) + @BindFlag(Flags.FLAG_PER_STREAM_MEDIA_PROGRESSION) + public boolean perStreamMediaProgressionEnabled; + private Context context; @Before public void setUp() { + Flags.disableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); + Flags.enableFlag(Flags.FLAG_ENABLE_STUCK_PLAYING_DETECTION); context = ApplicationProvider.getApplicationContext(); - ExoPlayer.Builder.experimentalEnableStuckPlayingDetection = true; } private TestExoPlayerBuilder parameterizeTestExoPlayerBuilder(TestExoPlayerBuilder builder) { @@ -346,7 +382,7 @@ public void playAds_midRollsWithAndWithoutDuration_clippedOrNotClippedAccordingl Timeline primaryContentTimeline = new FakeTimeline(new TimelineWindowDefinition.Builder().setDurationUs(60_000_000L).build()); AdPlaybackState adPlaybackState = - new AdPlaybackState("adsId", 10_000_000L, 20_000_000L) + new AdPlaybackState("adsId", 133_000_000L, 143_000_000L) .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) .withAvailableAdMediaItem( /* adGroupIndex= */ 0, @@ -1057,6 +1093,66 @@ public void adInMovingLiveWindow_keepsContentPosition() throws Exception { assertThat(contentPositionAfterLiveWindowUpdateMs).isEqualTo(2000); } + @Test + public void timelineRefresh_movingLiveDefaultPositionPastPlayingSsaiAd_keepsPlayingAd() + throws Exception { + // Live window with three 20s periods: |- p0 content -|- p1 ad -|- p2 content -|. + Object adsId = new Object(); + TimelineWindowDefinition liveWindowDefinition = + new TimelineWindowDefinition.Builder() + .setDynamic(true) + .setLive(true) + .setSeekable(true) + .setPeriodCount(3) + .setDurationUs(60_000_000) + .setWindowStartTimeUs(1_720_000_000_000_000L) + .setWindowPositionInFirstPeriodUs(0) + .setDefaultPositionUs(30_000_000) + .build(); + Timeline initialContentTimeline = new FakeTimeline(liveWindowDefinition); + // p1 is entirely covered by a server-side inserted ad. + AdPlaybackState contentOnlyAdPlaybackState = new AdPlaybackState(adsId); + AdPlaybackState adPeriodAdPlaybackState = + addAdGroupToAdPlaybackState( + contentOnlyAdPlaybackState, + /* fromPositionUs= */ 0, + /* contentResumeOffsetUs= */ 20_000_000, + /* adDurationsUs...= */ 20_000_000); + // ServerSideAdInsertionMediaSource requires an AdPlaybackState for every period. + ImmutableMap adPlaybackStates = + ImmutableMap.of( + initialContentTimeline.getUidOfPeriod(/* periodIndex= */ 0), + contentOnlyAdPlaybackState, + initialContentTimeline.getUidOfPeriod(/* periodIndex= */ 1), + adPeriodAdPlaybackState, + initialContentTimeline.getUidOfPeriod(/* periodIndex= */ 2), + contentOnlyAdPlaybackState); + FakeMediaSource contentMediaSource = new FakeMediaSource(initialContentTimeline); + ServerSideAdInsertionMediaSource mediaSource = + new ServerSideAdInsertionMediaSource( + contentMediaSource, /* adPlaybackStateUpdater= */ contentTimeline -> false); + mediaSource.setAdPlaybackStates(adPlaybackStates, initialContentTimeline); + ExoPlayer player = parameterizeTestExoPlayerBuilder(new TestExoPlayerBuilder(context)).build(); + + // Join the live stream while the ad in p1 is on air. + player.setMediaSource(mediaSource); + player.prepare(); + advance(player).untilState(Player.STATE_READY); + boolean isPlayingAdAfterJoining = player.isPlayingAd(); + // Refresh the live timeline with a default position that moved past the ad, into p2. + contentMediaSource.setNewSourceInfo( + new FakeTimeline( + liveWindowDefinition.buildUpon().setDefaultPositionUs(45_000_000).build())); + advance(player).untilPendingCommandsAreFullyHandled(); + boolean isPlayingAdAfterRefresh = player.isPlayingAd(); + @Nullable PlaybackException error = player.getPlayerError(); + player.release(); + + assertThat(isPlayingAdAfterJoining).isTrue(); + assertThat(error).isNull(); + assertThat(isPlayingAdAfterRefresh).isTrue(); + } + @Test public void addMediaSource_whilePlayingAd_correctMasking() throws Exception { long contentDurationMs = 10_000; @@ -2362,6 +2458,42 @@ public void newServerSideInsertedAdAtPlaybackPosition_keepsRenderersEnabled() th .isEqualTo(2); } + @Test + public void playAds_returningFromSegmentAlignedMidRoll_keepsRenderersEnabled() throws Exception { + PrerollMeasurement measurement = + playAdsWithMidRolls( + /* firstAdGroupTimeUs= */ 20_000_000L, /* secondAdGroupTimeUs= */ 40_000_000L); + + assertWithMessage("expected the content source to deliver no preroll") + .that(measurement.prerollSamplesSkipped + measurement.prerollSamplesRendered) + .isEqualTo(0); + // With per-stream media progression enabled, the renderer avoids resets because the period has + // no preroll. + int expectedEnabledCount = perStreamMediaProgressionEnabled ? 1 : 2; + assertWithMessage("[renderer enable count, preroll samples rendered]") + .that(ImmutableList.of(measurement.enabledCount, measurement.prerollSamplesRendered)) + .isEqualTo(ImmutableList.of(expectedEnabledCount, 0)); + } + + @Test + public void playAds_returningFromMidSegmentMidRoll_skipsPrerollOfClippedPeriodsOnly() + throws Exception { + PrerollMeasurement measurement = + playAdsWithMidRolls( + /* firstAdGroupTimeUs= */ 25_000_000L, /* secondAdGroupTimeUs= */ 45_000_000L); + + assertWithMessage("expected the content source to deliver preroll samples") + .that(measurement.prerollSamplesSkipped + measurement.prerollSamplesRendered) + .isGreaterThan(0); + // With per-stream media progression enabled, the preroll of the unclipped last + // content period is also treated as decode-only at the cost of one additional renderer enable. + int expectedEnabledCount = perStreamMediaProgressionEnabled ? 3 : 2; + int expectedPrerollSamplesRendered = perStreamMediaProgressionEnabled ? 0 : 50; + assertWithMessage("[renderer enable count, preroll samples rendered]") + .that(ImmutableList.of(measurement.enabledCount, measurement.prerollSamplesRendered)) + .isEqualTo(ImmutableList.of(expectedEnabledCount, expectedPrerollSamplesRendered)); + } + @Test public void playAds_midRollsInMultiPeriodContent_playedSuccessfully() throws PlaybackException, TimeoutException { @@ -2460,4 +2592,353 @@ public void playAds_midRollsInMultiPeriodContent_playedSuccessfully() assertThat(newPositions.get(4).positionMs).isEqualTo(40_000L); assertThat(newPositions.get(4).contentPositionMs).isEqualTo(40_000L); } + + @Test + public void playAds_liveStreamWithWindowOffset_adPlaysAtCorrectPosition() throws Exception { + Timeline liveContentTimeline = + new FakeTimeline( + new TimelineWindowDefinition.Builder() + .setPeriodCount(1) + .setLive(true) + .setDynamic(true) + .setDurationUs(60_000_000L) + .setWindowPositionInFirstPeriodUs(10_000_000L) + .build()); + AdPlaybackState adPlaybackState = + new AdPlaybackState("adsId", 15_000_000L) + .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + MediaItem.fromUri("http://example.com/ad_0_0")); + + FakeAdsLoader fakeAdsLoader = new FakeAdsLoader(); + AdsMediaSource adsMediaSource = + new AdsMediaSource( + new FakeMediaSource(liveContentTimeline, ExoPlayerTestRunner.VIDEO_FORMAT), + new DataSpec(Uri.EMPTY), + "adsId", + /* adMediaSourceFactory= */ new FakeMediaSourceFactory( + new TimelineWindowDefinition.Builder() + .setDurationUs(5_000_000L) + .setWindowPositionInFirstPeriodUs(0)), + fakeAdsLoader, + /* adViewProvider= */ () -> null, + /* useLazyContentSourcePreparation= */ true, + /* useAdMediaSourceClipping= */ true); + + ExoPlayer player = parameterizeTestExoPlayerBuilder(new TestExoPlayerBuilder(context)).build(); + Player.Listener mockListener = mock(Player.Listener.class); + player.addListener(mockListener); + player.setMediaSource(adsMediaSource); + player.prepare(); + advance(player) + .untilBackgroundThreadCondition( + (Supplier) () -> fakeAdsLoader.eventListeners.get("adsId") != null); + fakeAdsLoader.eventListeners.get("adsId").onAdPlaybackState(adPlaybackState); + + player.play(); + // Play past the ad at window position 5s (period position 15s) and ad duration (5s). + advance(player).untilPositionAtLeast(/* mediaItemIndex= */ 0, /* positionMs= */ 6_000L); + player.release(); + + ArgumentCaptor oldPositionsCaptor = ArgumentCaptor.forClass(PositionInfo.class); + ArgumentCaptor newPositionsCaptor = ArgumentCaptor.forClass(PositionInfo.class); + ArgumentCaptor reasonsCaptor = ArgumentCaptor.forClass(Integer.class); + verify(mockListener, atLeast(2)) + .onPositionDiscontinuity( + oldPositionsCaptor.capture(), newPositionsCaptor.capture(), reasonsCaptor.capture()); + + List oldPositions = oldPositionsCaptor.getAllValues(); + List newPositions = newPositionsCaptor.getAllValues(); + + // Content to ad transition at window position 5s (period position 15s) + assertThat(oldPositions.get(0).adGroupIndex).isEqualTo(C.INDEX_UNSET); + assertThat(oldPositions.get(0).positionMs).isEqualTo(5_000L); + assertThat(oldPositions.get(0).contentPositionMs).isEqualTo(5_000L); + assertThat(newPositions.get(0).adGroupIndex).isEqualTo(0); + assertThat(newPositions.get(0).positionMs).isEqualTo(0L); + + // Ad to content transition at ad completion (window position 5s) + assertThat(oldPositions.get(1).adGroupIndex).isEqualTo(0); + assertThat(oldPositions.get(1).positionMs).isEqualTo(5_000L); // Ad duration + assertThat(oldPositions.get(1).contentPositionMs).isEqualTo(5_000L); + assertThat(newPositions.get(1).adGroupIndex).isEqualTo(C.INDEX_UNSET); + assertThat(newPositions.get(1).positionMs).isEqualTo(5_000L); + assertThat(newPositions.get(1).contentPositionMs).isEqualTo(5_000L); + } + + @Test + public void stop_duringLiveStreamPrerollAdWithPlaybackStatsListener_doesNotCrash() + throws Exception { + AdPlaybackState adPlaybackState = + FakeTimeline.createAdPlaybackState( + /* adsPerAdGroup= */ 1, + /* adGroupTimesUs...= */ DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US); + Timeline fakeTimeline = + new FakeTimeline( + new TimelineWindowDefinition.Builder() + .setLive(true) + .setAdPlaybackStates(ImmutableList.of(adPlaybackState)) + .build()); + FakeMediaSource fakeMediaSource = + new FakeMediaSource(fakeTimeline, ExoPlayerTestRunner.VIDEO_FORMAT); + ExoPlayer player = parameterizeTestExoPlayerBuilder(new TestExoPlayerBuilder(context)).build(); + player.addAnalyticsListener( + new PlaybackStatsListener(/* keepHistory= */ false, /* callback= */ null)); + player.setMediaSource(fakeMediaSource); + player.prepare(); + player.play(); + advance(player).untilState(Player.STATE_READY); + assertThat(player.isPlayingAd()).isTrue(); + + player.stop(); + advance(player).untilPendingCommandsAreFullyHandled(); + player.setVolume(0f); + advance(player).untilPendingCommandsAreFullyHandled(); + player.release(); + } + + /** + * Plays content with two mid-roll ad groups ('A a B b C') and measures renderer enables and + * preroll samples across transitions. + */ + private PrerollMeasurement playAdsWithMidRolls(long firstAdGroupTimeUs, long secondAdGroupTimeUs) + throws Exception { + AtomicReference videoRenderer = new AtomicReference<>(); + RenderersFactory renderersFactory = + (handler, videoListener, audioListener, textOutput, metadataOutput) -> { + videoRenderer.set( + new PrerollTrackingVideoRenderer( + SystemClock.DEFAULT.createHandler(handler.getLooper(), /* callback= */ null), + videoListener)); + return new Renderer[] {videoRenderer.get()}; + }; + Timeline primaryContentTimeline = + new FakeTimeline( + new TimelineWindowDefinition.Builder() + .setWindowPositionInFirstPeriodUs(0L) + .setDurationUs(60_000_000L) + .build()); + // Two mid-rolls, placed so that the content period between them has a next ad group while the + // content period after the last ad group has none. + AdPlaybackState adPlaybackState = + new AdPlaybackState( + "adsId", /* adGroupTimesUs...= */ firstAdGroupTimeUs, secondAdGroupTimeUs) + .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) + .withAdDurationsUs(/* adGroupIndex= */ 0, 5_000_000L) + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + MediaItem.fromUri("http://example.com/ad_0_0")) + .withAdCount(/* adGroupIndex= */ 1, /* adCount= */ 1) + .withAdDurationsUs(/* adGroupIndex= */ 1, 5_000_000L) + .withAvailableAdMediaItem( + /* adGroupIndex= */ 1, + /* adIndexInAdGroup= */ 0, + MediaItem.fromUri("http://example.com/ad_1_0")); + // The ads use the same format as the content, so that the renderer can be kept enabled across + // the content/ad transitions. + MediaSource.Factory adMediaSourceFactory = + new MediaSource.Factory() { + @Override + public MediaSource.Factory setDrmSessionManagerProvider( + DrmSessionManagerProvider drmSessionManagerProvider) { + throw new UnsupportedOperationException(); + } + + @Override + public MediaSource.Factory setLoadErrorHandlingPolicy( + LoadErrorHandlingPolicy loadErrorHandlingPolicy) { + throw new UnsupportedOperationException(); + } + + @Override + public @C.ContentType int[] getSupportedTypes() { + return new int[] {C.CONTENT_TYPE_OTHER}; + } + + @Override + public MediaSource createMediaSource(MediaItem mediaItem) { + return new FakeMediaSource( + new FakeTimeline( + new TimelineWindowDefinition.Builder() + .setMediaItem(mediaItem) + .setDurationUs(5_000_000L) + .build()), + ExoPlayerTestRunner.VIDEO_FORMAT); + } + }; + FakeAdsLoader fakeAdsLoader = new FakeAdsLoader(); + AdsMediaSource adsMediaSource = + new AdsMediaSource( + new PrerollEmittingMediaSource( + primaryContentTimeline, + TrackDataFactory.samplesWithRateDurationAndKeyframeInterval( + /* initialSampleTimeUs= */ 0, + /* sampleRate= */ CONTENT_SAMPLES_PER_SECOND, + /* durationUs= */ 60_000_000L, + /* keyFrameInterval= */ CONTENT_SAMPLES_PER_SEGMENT), + ExoPlayerTestRunner.VIDEO_FORMAT), + new DataSpec(Uri.EMPTY), + "adsId", + /* adMediaSourceFactory= */ adMediaSourceFactory, + fakeAdsLoader, + /* adViewProvider= */ () -> null, + /* useLazyContentSourcePreparation= */ true, + /* useAdMediaSourceClipping= */ true); + ExoPlayer player = + parameterizeTestExoPlayerBuilder( + new TestExoPlayerBuilder(context).setRenderersFactory(renderersFactory)) + .build(); + player.setMediaSource(adsMediaSource); + player.prepare(); + advance(player) + .untilBackgroundThreadCondition( + (Supplier) () -> fakeAdsLoader.eventListeners.get("adsId") != null); + fakeAdsLoader.eventListeners.get("adsId").onAdPlaybackState(adPlaybackState); + + player.play(); + advance(player).untilState(Player.STATE_ENDED); + int enabledCount = videoRenderer.get().enabledCount; + int prerollSamplesSkipped = videoRenderer.get().prerollSamplesSkipped; + int prerollSamplesRendered = videoRenderer.get().prerollSamplesRendered; + player.release(); + + return new PrerollMeasurement(enabledCount, prerollSamplesSkipped, prerollSamplesRendered); + } + + /** Returns the start of the segment containing {@code positionUs}. */ + private static long contentSegmentStartUs(long positionUs) { + return (positionUs / CONTENT_SEGMENT_DURATION_US) * CONTENT_SEGMENT_DURATION_US; + } + + /** The result of one {@link #playAdsWithMidRolls} playback. */ + private static final class PrerollMeasurement { + private final int enabledCount; + private final int prerollSamplesSkipped; + private final int prerollSamplesRendered; + + private PrerollMeasurement( + int enabledCount, int prerollSamplesSkipped, int prerollSamplesRendered) { + this.enabledCount = enabledCount; + this.prerollSamplesSkipped = prerollSamplesSkipped; + this.prerollSamplesRendered = prerollSamplesRendered; + } + } + + /** + * A {@link FakeMediaSource} whose periods deliver samples from the start of the segment + * containing the period's start position, rather than from exactly that position. + */ + private static final class PrerollEmittingMediaSource extends FakeMediaSource { + + private final TrackDataFactory trackDataFactory; + + private PrerollEmittingMediaSource( + Timeline timeline, TrackDataFactory trackDataFactory, Format... formats) { + super(timeline, formats); + this.trackDataFactory = trackDataFactory; + } + + @Override + protected MediaPeriod createMediaPeriod( + MediaPeriodId id, + TrackGroupArray trackGroupArray, + Allocator allocator, + MediaSourceEventListener.EventDispatcher mediaSourceEventDispatcher, + DrmSessionManager drmSessionManager, + DrmSessionEventListener.EventDispatcher drmEventDispatcher, + @Nullable TransferListener transferListener) { + return new FakeMediaPeriod( + trackGroupArray, + allocator, + trackDataFactory, + mediaSourceEventDispatcher, + drmSessionManager, + drmEventDispatcher, + /* deferOnPrepared= */ false) { + + private long preparePositionUs; + + @Override + public synchronized void prepare(MediaPeriod.Callback callback, long positionUs) { + preparePositionUs = positionUs; + super.prepare(callback, positionUs); + } + + @Override + protected FakeSampleStream createSampleStream( + Allocator allocator, + @Nullable MediaSourceEventListener.EventDispatcher mediaSourceEventDispatcher, + DrmSessionManager drmSessionManager, + DrmSessionEventListener.EventDispatcher drmEventDispatcher, + Format initialFormat, + List fakeSampleStreamItems) { + long segmentStartUs = contentSegmentStartUs(preparePositionUs); + boolean hasPreroll = segmentStartUs < preparePositionUs; + return new FakeSampleStream( + allocator, + mediaSourceEventDispatcher, + drmSessionManager, + drmEventDispatcher, + initialFormat, + fakeSampleStreamItems) { + @Override + public void writeData(long startPositionUs) { + // Deliver from the start of the containing segment, not from startPositionUs. + super.writeData(segmentStartUs); + } + + @Override + public @SampleStream.Flags int getFlags() { + // Report preroll exactly, as a real chunk-loading source does. + return super.getFlags() | (hasPreroll ? SampleStream.FLAG_HAS_PREROLL : 0); + } + }; + } + }; + } + } + + /** + * A {@link FakeVideoRenderer} that counts preroll samples, split by whether they would have been + * treated as decode-only. + */ + private static final class PrerollTrackingVideoRenderer extends FakeVideoRenderer { + + /** Preroll samples correctly treated as decode-only. */ + private int prerollSamplesSkipped; + + /** Preroll samples that would have been rendered, i.e. dropped or shown as stale content. */ + private int prerollSamplesRendered; + + private long streamStartPositionUs; + + private PrerollTrackingVideoRenderer( + HandlerWrapper handler, VideoRendererEventListener eventListener) { + super(handler, eventListener); + } + + @Override + protected void onStreamChanged( + Format[] formats, long startPositionUs, long offsetUs, MediaPeriodId mediaPeriodId) + throws ExoPlaybackException { + super.onStreamChanged(formats, startPositionUs, offsetUs, mediaPeriodId); + streamStartPositionUs = startPositionUs; + } + + @Override + protected boolean shouldProcessBuffer(long bufferTimeUs, long playbackPositionUs) { + boolean shouldProcess = super.shouldProcessBuffer(bufferTimeUs, playbackPositionUs); + if (shouldProcess && bufferTimeUs < streamStartPositionUs) { + if (bufferTimeUs < getLastResetPositionUs()) { + prerollSamplesSkipped++; + } else { + prerollSamplesRendered++; + } + } + return shouldProcess; + } + } } diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerScrubbingTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerScrubbingTest.java index 2b2962e121c..c46a0b15dd7 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerScrubbingTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerScrubbingTest.java @@ -44,12 +44,12 @@ import android.graphics.SurfaceTexture; import android.media.MediaFormat; import android.os.Bundle; -import android.os.Handler; import android.os.Looper; import android.util.Pair; import android.view.Surface; import androidx.annotation.Nullable; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.Format; import androidx.media3.common.MediaItem; import androidx.media3.common.MimeTypes; @@ -66,10 +66,8 @@ import androidx.media3.exoplayer.image.ImageRenderer; import androidx.media3.exoplayer.mediacodec.ForwardingMediaCodecAdapter; import androidx.media3.exoplayer.mediacodec.MediaCodecAdapter; -import androidx.media3.exoplayer.mediacodec.MediaCodecSelector; -import androidx.media3.exoplayer.video.MediaCodecVideoRenderer; +import androidx.media3.exoplayer.source.MergingMediaSource; import androidx.media3.exoplayer.video.VideoFrameMetadataListener; -import androidx.media3.exoplayer.video.VideoRendererEventListener; import androidx.media3.test.utils.ExoPlayerTestRunner; import androidx.media3.test.utils.FakeAudioRenderer; import androidx.media3.test.utils.FakeClock; @@ -79,6 +77,7 @@ import androidx.media3.test.utils.FakeTimeline; import androidx.media3.test.utils.FakeTimeline.TimelineWindowDefinition; import androidx.media3.test.utils.FakeVideoRenderer; +import androidx.media3.test.utils.Media3FlagsRule; import androidx.media3.test.utils.TestExoPlayerBuilder; import androidx.media3.test.utils.robolectric.IdlingMediaCodecAdapterFactory; import androidx.media3.test.utils.robolectric.ShadowMediaCodecConfig; @@ -94,7 +93,6 @@ import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicInteger; import java.util.concurrent.atomic.AtomicReference; -import org.junit.Ignore; import org.junit.Rule; import org.junit.Test; import org.junit.runner.RunWith; @@ -112,6 +110,8 @@ public final class ExoPlayerScrubbingTest { @Rule public Expect expect = Expect.create(); + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); + @Test public void scrubbingMode_getterWorks() throws Exception { ExoPlayer player = @@ -864,139 +864,73 @@ protected MediaCodecAdapter.Factory getCodecAdapterFactory() { } @Test - @Config(minSdk = 31) // Relies on async MediaCodec mode, which is only the default on API 31+. - @Ignore("Flaky: b/515127273") public void dynamicSchedulingInScrubbingMode_renderCalledMoreFrequentlyThan10ms() throws Exception { + Flags.disableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); Context context = ApplicationProvider.getApplicationContext(); - FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); - IdlingMediaCodecAdapterFactory codecAdapterFactory = - new IdlingMediaCodecAdapterFactory(context, clock); AtomicInteger renderCounter = new AtomicInteger(); - RenderersFactory renderersFactory = - new DefaultRenderersFactory(context) { - @Override - protected void buildVideoRenderers( - Context context, - @ExtensionRendererMode int extensionRendererMode, - MediaCodecSelector mediaCodecSelector, - boolean enableDecoderFallback, - Handler eventHandler, - VideoRendererEventListener eventListener, - long allowedVideoJoiningTimeMs, - ArrayList out) { - MediaCodecVideoRenderer videoRenderer = - new MediaCodecVideoRenderer.Builder(context) - .setCodecAdapterFactory(codecAdapterFactory) - .setMediaCodecSelector(mediaCodecSelector) - .setAllowedJoiningTimeMs(allowedVideoJoiningTimeMs) - .setEnableDecoderFallback(enableDecoderFallback) - .setEventHandler(eventHandler) - .setEventListener(eventListener) - .build(); - out.add(new RenderCountingRenderer(videoRenderer, renderCounter)); - } - }; - + ForwardingDurationToProgressRenderer fakeRenderer = + new ForwardingDurationToProgressRenderer( + new FakeRenderer(C.TRACK_TYPE_VIDEO), + /* durationToProgressUs= */ 2_000L, + renderCounter); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); ExoPlayer player = - new TestExoPlayerBuilder(context) - .setRenderersFactory(renderersFactory) - .setDynamicSchedulingEnabled(false) - .setClock(clock) - .setStuckSuppressedDetectionTimeoutMs(Integer.MAX_VALUE) - .build(); - player.setMediaSource(create30Fps2sGop10sDurationVideoSource()); - Surface surface = new Surface(new SurfaceTexture(1)); - player.setVideoSurface(surface); + new TestExoPlayerBuilder(context).setClock(clock).setRenderers(fakeRenderer).build(); + player.setMediaSource( + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT)); player.prepare(); player.play(); - advance(player).untilState(Player.STATE_READY); + player.setScrubbingModeEnabled(true); advance(player).untilPendingCommandsAreFullyHandled(); - long playerReadyTimeMs = clock.currentTimeMillis(); - renderCounter.set(0); - - // This seeks to near the end of a GoP, requiring decoding 58 frames. - player.seekTo(3950); - advance(player) - .untilBackgroundThreadCondition(() -> clock.currentTimeMillis() - playerReadyTimeMs >= 500); + advance(player).untilBackgroundThreadCondition(() -> clock.currentTimeMillis() >= 500); + renderCounter.set(0); + advance(player).untilBackgroundThreadCondition(() -> clock.currentTimeMillis() >= 800); - // With dynamic scheduling enabled, and lots of decoding work to do for the seeks, we should - // be triggering the renderer more frequently than every 10ms. - assertThat(renderCounter.get()).isGreaterThan(55); + // With dynamic scheduling enabled in scrubbing mode, the 2ms renderer duration should be used + // (150 render calls in 300ms). + assertThat(renderCounter.get()).isEqualTo(150); player.release(); - surface.release(); } @Test - @Config(minSdk = 31) // TODO: b/511055213 - Run on all API levels when Robolectric is fixed. - @Ignore("Flaky: b/515127273") - public void dynamicSchedulingDisabledInScrubbingMode_renderCalledEvery10ms() throws Exception { + public void dynamicSchedulingDisabledInScrubbingMode_usesDefaultIdleSchedulingInterval() + throws Exception { + Flags.disableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); Context context = ApplicationProvider.getApplicationContext(); - FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); - IdlingMediaCodecAdapterFactory codecAdapterFactory = - new IdlingMediaCodecAdapterFactory(context, clock); AtomicInteger renderCounter = new AtomicInteger(); - RenderersFactory renderersFactory = - new DefaultRenderersFactory(context) { - @Override - protected void buildVideoRenderers( - Context context, - @ExtensionRendererMode int extensionRendererMode, - MediaCodecSelector mediaCodecSelector, - boolean enableDecoderFallback, - Handler eventHandler, - VideoRendererEventListener eventListener, - long allowedVideoJoiningTimeMs, - ArrayList out) { - MediaCodecVideoRenderer videoRenderer = - new MediaCodecVideoRenderer.Builder(context) - .setCodecAdapterFactory(codecAdapterFactory) - .setMediaCodecSelector(mediaCodecSelector) - .setAllowedJoiningTimeMs(allowedVideoJoiningTimeMs) - .setEnableDecoderFallback(enableDecoderFallback) - .setEventHandler(eventHandler) - .setEventListener(eventListener) - .build(); - out.add(new RenderCountingRenderer(videoRenderer, renderCounter)); - } - }; - + ForwardingDurationToProgressRenderer fakeRenderer = + new ForwardingDurationToProgressRenderer( + new FakeRenderer(C.TRACK_TYPE_VIDEO), + /* durationToProgressUs= */ 2_000L, + renderCounter); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); ExoPlayer player = - new TestExoPlayerBuilder(context) - .setRenderersFactory(renderersFactory) - .setDynamicSchedulingEnabled(false) - .setClock(clock) - .setStuckSuppressedDetectionTimeoutMs(Integer.MAX_VALUE) - .build(); - player.setMediaSource(create30Fps2sGop10sDurationVideoSource()); - Surface surface = new Surface(new SurfaceTexture(1)); - player.setVideoSurface(surface); + new TestExoPlayerBuilder(context).setClock(clock).setRenderers(fakeRenderer).build(); + player.setMediaSource( + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT)); player.setScrubbingModeParameters( new ScrubbingModeParameters.Builder().setShouldEnableDynamicScheduling(false).build()); player.prepare(); player.play(); - advance(player).untilState(Player.STATE_READY); + player.setScrubbingModeEnabled(true); advance(player).untilPendingCommandsAreFullyHandled(); - long playerReadyTimeMs = clock.currentTimeMillis(); - renderCounter.set(0); - // This seeks to near the end of a GoP, requiring decoding 58 frames. - player.seekTo(3950); - - advance(player) - .untilBackgroundThreadCondition(() -> clock.currentTimeMillis() - playerReadyTimeMs >= 500); + advance(player).untilBackgroundThreadCondition(() -> clock.currentTimeMillis() >= 1000); + renderCounter.set(0); + advance(player).untilBackgroundThreadCondition(() -> clock.currentTimeMillis() >= 4000); - // Expect about one render call every 10ms. - assertThat(renderCounter.get()).isWithin(5).of(50); + // Dynamic scheduling is disabled, so the 2ms renderer duration is ignored and the default + // static idle scheduling interval (1000ms) is used (3 render calls in 3000ms). + assertThat(renderCounter.get()).isEqualTo(3); player.release(); - surface.release(); } @Test @@ -1157,6 +1091,74 @@ public void onRenderedFirstFrame() { .of(58); } + @Test + public void + scrubbingMode_seekIntoAudioOnlyPortionOfMergingMediaSource_doesNotHangSubsequentSeeks() + throws Exception { + ExoPlayer player = + new TestExoPlayerBuilder(ApplicationProvider.getApplicationContext()) + .setStuckPlayingDetectionTimeoutMs(Integer.MAX_VALUE) + .setStuckSuppressedDetectionTimeoutMs(Integer.MAX_VALUE) + .build(); + Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); + player.setVideoSurface(surface); + // Create a source where the video track ends early to provide an 'audio-only' portion. + FakeMediaSource audioSource = + createFakeMediaSource(ExoPlayerTestRunner.AUDIO_FORMAT, /* durationUs= */ 1_000_000L); + FakeMediaSource videoSource1 = + createFakeMediaSource(ExoPlayerTestRunner.VIDEO_FORMAT, /* durationUs= */ 1_000_000L); + FakeMediaSource videoSource2 = + createFakeMediaSource(ExoPlayerTestRunner.VIDEO_FORMAT, /* durationUs= */ 500_000L); + MergingMediaSource mergingSource1 = + new MergingMediaSource( + /* adjustPeriodTimeOffsets= */ false, + /* clipDurations= */ false, + videoSource1, + audioSource); + MergingMediaSource mergingSource2 = + new MergingMediaSource( + /* adjustPeriodTimeOffsets= */ false, + /* clipDurations= */ false, + videoSource2, + audioSource); + player.setMediaSources(ImmutableList.of(mergingSource1, mergingSource2)); + player.prepare(); + advance(player).untilState(Player.STATE_READY); + List renderedFrames = new ArrayList<>(); + player.setVideoFrameMetadataListener( + (presentationTimeUs, releaseTimeNs, format, mediaFormat) -> + renderedFrames.add(presentationTimeUs)); + + player.setScrubbingModeEnabled(true); + advance(player).untilPendingCommandsAreFullyHandled(); + player.seekTo(/* mediaItemIndex= */ 1, /* positionMs= */ 800); + advance(player).untilPendingCommandsAreFullyHandled(); + player.seekTo(/* mediaItemIndex= */ 0, /* positionMs= */ 500); + advance(player).untilPendingCommandsAreFullyHandled(); + advance(player).untilBackgroundThreadCondition(() -> renderedFrames.contains(500_000L)); + + player.release(); + surface.release(); + } + + private static FakeMediaSource createFakeMediaSource(Format format, long durationUs) { + return new FakeMediaSource.Builder() + .setTimeline( + new FakeTimeline( + new TimelineWindowDefinition.Builder() + .setWindowPositionInFirstPeriodUs(0) + .setDurationUs(durationUs) + .build())) + .setFormats(format) + .setTrackDataFactory( + TrackDataFactory.samplesWithRateDurationAndKeyframeInterval( + /* initialSampleTimeUs= */ 0, + /* sampleRate= */ 30, + /* durationUs= */ durationUs, + /* keyFrameInterval= */ 1)) + .build(); + } + private static FakeMediaSource create30Fps2sGop10sDurationVideoSource() { return new FakeMediaSource.Builder() .setTimeline( @@ -1201,14 +1203,22 @@ private static FakeMediaSource create60sDurationImageSource() { .build(); } - private static class RenderCountingRenderer extends ForwardingRenderer { + private static class ForwardingDurationToProgressRenderer extends ForwardingRenderer { + private final long durationToProgressUs; private final AtomicInteger renderCounter; - public RenderCountingRenderer(Renderer renderer, AtomicInteger renderCounter) { + private ForwardingDurationToProgressRenderer( + Renderer renderer, long durationToProgressUs, AtomicInteger renderCounter) { super(renderer); + this.durationToProgressUs = durationToProgressUs; this.renderCounter = renderCounter; } + @Override + public long getDurationToProgressUs(long positionUs, long elapsedRealtimeUs) { + return durationToProgressUs; + } + @Override public void render(long positionUs, long elapsedRealtimeUs) throws ExoPlaybackException { super.render(positionUs, elapsedRealtimeUs); diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerStuckPlayerDetectionTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerStuckPlayerDetectionTest.java index 3b4c372030c..c5fc3c1e93b 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerStuckPlayerDetectionTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerStuckPlayerDetectionTest.java @@ -26,6 +26,7 @@ import android.media.AudioManager; import androidx.annotation.Nullable; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.PlaybackException; import androidx.media3.common.PlaybackParameters; import androidx.media3.common.Player; @@ -52,6 +53,7 @@ import androidx.media3.test.utils.FakeMediaSource; import androidx.media3.test.utils.FakeRenderer; import androidx.media3.test.utils.FakeTimeline; +import androidx.media3.test.utils.Media3FlagsRule; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.collect.ImmutableList; @@ -59,6 +61,7 @@ import java.util.List; import java.util.Random; import org.junit.Before; +import org.junit.Rule; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.annotation.Config; @@ -70,6 +73,8 @@ @RunWith(AndroidJUnit4.class) public class ExoPlayerStuckPlayerDetectionTest { + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); + private static final int TIMEOUT_MS = 10_000; private Context context; @@ -77,7 +82,7 @@ public class ExoPlayerStuckPlayerDetectionTest { @Before public void setUp() { context = ApplicationProvider.getApplicationContext(); - ExoPlayer.Builder.experimentalEnableStuckPlayingDetection = true; + Flags.enableFlag(Flags.FLAG_ENABLE_STUCK_PLAYING_DETECTION); } @Test diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerTargetLiveOffsetTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerTargetLiveOffsetTest.java index 9181a1f82f5..6de02161f1f 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerTargetLiveOffsetTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerTargetLiveOffsetTest.java @@ -27,6 +27,7 @@ import android.net.Uri; import androidx.media3.common.AdPlaybackState; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.MediaItem; import androidx.media3.common.PlaybackParameters; import androidx.media3.common.Player; @@ -37,12 +38,15 @@ import androidx.media3.test.utils.FakeMediaSource; import androidx.media3.test.utils.FakeTimeline; import androidx.media3.test.utils.FakeTimeline.TimelineWindowDefinition; +import androidx.media3.test.utils.Media3FlagsRule; import androidx.media3.test.utils.TestExoPlayerBuilder; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.collect.ImmutableList; import com.google.common.collect.Range; import org.junit.Before; +import org.junit.Ignore; +import org.junit.Rule; import org.junit.Test; import org.junit.runner.RunWith; @@ -50,10 +54,13 @@ @RunWith(AndroidJUnit4.class) public final class ExoPlayerTargetLiveOffsetTest { + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); + private Context context; @Before public void setUp() { + Flags.disableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); context = ApplicationProvider.getApplicationContext(); } @@ -248,6 +255,7 @@ public void targetLiveOffsetInMedia_withUserSeekOutsideMaxLivOffset_adjustsLiveO } @Test + @Ignore("Flaky on CI: b/568098904") public void targetLiveOffsetInMedia_withTimelineUpdate_adjustsLiveOffsetToLatestTimeline() throws Exception { long windowStartUnixTimeMs = 987_654_321_000L; diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerTest.java index ff90dff2a99..0b58b2dcb5c 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerTest.java @@ -50,6 +50,7 @@ import static androidx.media3.common.Player.COMMAND_SET_VIDEO_SURFACE; import static androidx.media3.common.Player.COMMAND_SET_VOLUME; import static androidx.media3.common.Player.COMMAND_STOP; +import static androidx.media3.common.Player.DISCONTINUITY_REASON_SEEK; import static androidx.media3.common.Player.DISCONTINUITY_REASON_SEEK_ADJUSTMENT; import static androidx.media3.test.utils.FakeSampleStream.FakeSampleStreamItem.END_OF_STREAM_ITEM; import static androidx.media3.test.utils.FakeSampleStream.FakeSampleStreamItem.oneByteSample; @@ -104,6 +105,7 @@ import androidx.media3.common.AudioAttributes; import androidx.media3.common.C; import androidx.media3.common.DeviceInfo; +import androidx.media3.common.Flags; import androidx.media3.common.Format; import androidx.media3.common.MediaItem; import androidx.media3.common.MediaMetadata; @@ -175,6 +177,7 @@ import androidx.media3.test.utils.ActionSchedule; import androidx.media3.test.utils.ActionSchedule.PlayerRunnable; import androidx.media3.test.utils.ActionSchedule.PlayerTarget; +import androidx.media3.test.utils.BindFlag; import androidx.media3.test.utils.EmptyMediaSource; import androidx.media3.test.utils.ExoPlayerTestRunner; import androidx.media3.test.utils.FakeAudioRenderer; @@ -194,6 +197,7 @@ import androidx.media3.test.utils.FakeTrackSelection; import androidx.media3.test.utils.FakeTrackSelector; import androidx.media3.test.utils.FakeVideoRenderer; +import androidx.media3.test.utils.Media3FlagsRule; import androidx.media3.test.utils.ReleaseListener; import androidx.media3.test.utils.TestExoPlayerBuilder; import androidx.media3.test.utils.robolectric.IdlingMediaCodecAdapterFactory; @@ -267,11 +271,14 @@ public static List params() { public ShadowMediaCodecConfig mediaCodecConfig = ShadowMediaCodecConfig.withAllDefaultSupportedCodecs(); + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); + @Parameter(0) public boolean isPreloadEnabled; // TODO: b/510217604 - Remove parameterization. @Parameter(1) + @BindFlag(Flags.FLAG_PER_STREAM_MEDIA_PROGRESSION) public boolean perStreamMediaProgressionEnabled; private Context context; @@ -282,20 +289,17 @@ public void setUp() { context = ApplicationProvider.getApplicationContext(); placeholderTimeline = new MaskingMediaSource.PlaceholderTimeline(FAKE_MEDIA_ITEM.buildUpon().setTag(0).build()); - ExoPlayer.Builder.experimentalEnableStuckPlayingDetection = true; + Flags.enableFlag(Flags.FLAG_ENABLE_STUCK_PLAYING_DETECTION); + Flags.disableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); } private TestExoPlayerBuilder parameterizeTestExoPlayerBuilder(TestExoPlayerBuilder builder) { - return builder - .setPreloadConfiguration(preloadConfigurationsList.get(isPreloadEnabled ? 1 : 0)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled); + return builder.setPreloadConfiguration(preloadConfigurationsList.get(isPreloadEnabled ? 1 : 0)); } private ExoPlayerTestRunner.Builder parameterizeExoPlayerTestRunnerBuilder( ExoPlayerTestRunner.Builder builder) { - return builder - .setPreloadConfiguration(preloadConfigurationsList.get(isPreloadEnabled ? 1 : 0)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled); + return builder.setPreloadConfiguration(preloadConfigurationsList.get(isPreloadEnabled ? 1 : 0)); } /** @@ -1010,6 +1014,464 @@ protected void onStreamChanged( player.release(); } + @Test + public void + rendererError_inAdvancedPeriodWithPerStreamMediaProgressionEnabled_doesNotDisruptCurrentPlayback() + throws Exception { + if (!perStreamMediaProgressionEnabled) { + return; + } + + AtomicBoolean allowVideoEndOfStream = new AtomicBoolean(false); + AtomicInteger audioPeriod2ErrorCount = new AtomicInteger(0); + Format textFormat = new Format.Builder().setSampleMimeType(MimeTypes.TEXT_VTT).build(); + RenderersFactory renderersFactory = + (handler, videoListener, audioListener, textOutput, metadataOutput) -> + new Renderer[] { + // Video Renderer remains active on Period 1 until allowVideoEndOfStream is true. + new FakeVideoRenderer( + SystemClock.DEFAULT.createHandler(handler.getLooper(), /* callback= */ null), + videoListener) { + @Override + public boolean isEnded() { + return allowVideoEndOfStream.get() && super.isEnded(); + } + }, + // Text Renderer finishes Period 1 at 0ms and advances early to Period 2. + new FakeRenderer(C.TRACK_TYPE_TEXT), + // Audio Renderer throws an error when its reading period advances to Period 2. + new FakeAudioRenderer( + SystemClock.DEFAULT.createHandler(handler.getLooper(), /* callback= */ null), + audioListener) { + @Override + protected void onStreamChanged( + Format[] formats, + long startPositionUs, + long offsetUs, + MediaSource.MediaPeriodId mediaPeriodId) + throws ExoPlaybackException { + super.onStreamChanged(formats, startPositionUs, offsetUs, mediaPeriodId); + int periodIndex = getTimeline().getIndexOfPeriod(mediaPeriodId.periodUid); + if (periodIndex == 1) { + audioPeriod2ErrorCount.incrementAndGet(); + throw ExoPlaybackException.createForRenderer( + new Exception("Audio error while reading ahead into Period 2"), + getName(), + getIndex(), + formats[0], + C.FORMAT_HANDLED, + mediaPeriodId, + /* isRecoverable= */ false, + PlaybackException.ERROR_CODE_DECODER_INIT_FAILED); + } + } + } + }; + ExoPlayer player = + parameterizeTestExoPlayerBuilder( + new TestExoPlayerBuilder(context).setRenderersFactory(renderersFactory)) + .build(); + // Period 1: Text and Audio reach end-of-stream immediately at 0ms, while Video stays active in + // Period 1 until allowVideoEndOfStream is set to true. + TrackGroupArray trackGroupArray = + new TrackGroupArray( + new TrackGroup(ExoPlayerTestRunner.AUDIO_FORMAT), + new TrackGroup(ExoPlayerTestRunner.VIDEO_FORMAT), + new TrackGroup(textFormat)); + MediaSource mediaSource1 = + new FakeMediaSource( + new FakeTimeline(), + DrmSessionManager.DRM_UNSUPPORTED, + /* trackDataFactory= */ null, + /* syncSampleTimesUs= */ null, + trackGroupArray) { + @Override + protected MediaPeriod createMediaPeriod( + MediaPeriodId id, + TrackGroupArray trackGroupArray, + Allocator allocator, + MediaSourceEventListener.EventDispatcher mediaSourceEventDispatcher, + DrmSessionManager drmSessionManager, + DrmSessionEventListener.EventDispatcher drmEventDispatcher, + TransferListener transferListener) { + return new FakeMediaPeriod( + trackGroupArray, + allocator, + FakeMediaPeriod.TrackDataFactory.singleSampleWithTimeUs(0), + mediaSourceEventDispatcher, + drmSessionManager, + drmEventDispatcher, + /* deferOnPrepared= */ false) { + @Override + protected FakeSampleStream createSampleStream( + Allocator allocator, + MediaSourceEventListener.EventDispatcher mediaSourceEventDispatcher, + DrmSessionManager drmSessionManager, + DrmSessionEventListener.EventDispatcher drmEventDispatcher, + Format initialFormat, + List fakeSampleStreamItems) { + return new FakeSampleStream( + allocator, + mediaSourceEventDispatcher, + drmSessionManager, + drmEventDispatcher, + initialFormat, + fakeSampleStreamItems) { + @Override + public boolean isReady() { + return true; + } + + @Override + public int readData( + FormatHolder formatHolder, + DecoderInputBuffer buffer, + @ReadFlags int readFlags) { + int result = super.readData(formatHolder, buffer, readFlags); + if (MimeTypes.isVideo(initialFormat.sampleMimeType) + && !allowVideoEndOfStream.get() + && result == C.RESULT_BUFFER_READ + && buffer.isEndOfStream()) { + buffer.clear(); + return C.RESULT_NOTHING_READ; + } + return result; + } + }; + } + }; + } + }; + MediaSource mediaSource2 = + new FakeMediaSource( + new FakeTimeline(), + ExoPlayerTestRunner.VIDEO_FORMAT, + ExoPlayerTestRunner.AUDIO_FORMAT, + textFormat); + player.setMediaSources(ImmutableList.of(mediaSource1, mediaSource2)); + player.prepare(); + player.play(); + + // Wait until Audio advances early to Period 2 and throws the error, which rolls back the + // early-reading streams without disrupting Period 1's active video playback. + runMainLooperUntil(() -> audioPeriod2ErrorCount.get() >= 1); + advance(player).untilPendingCommandsAreFullyHandled(); + assertThat(audioPeriod2ErrorCount.get()).isEqualTo(1); + + // Period 1 playback continues uninterrupted in STATE_READY. + assertThat(player.getPlaybackState()).isEqualTo(Player.STATE_READY); + assertThat(player.getPlayerError()).isNull(); + assertThat(player.getCurrentMediaItemIndex()).isEqualTo(0); + + // Once Video finishes reading Period 1, all streams advance to Period 2 and the error on + // Period 2 is reported at the transition point. + allowVideoEndOfStream.set(true); + ExoPlaybackException error = runUntilError(player); + assertThat(audioPeriod2ErrorCount.get()).isEqualTo(2); + assertThat(player.getPlaybackState()).isEqualTo(Player.STATE_IDLE); + assertThat(player.getCurrentMediaItemIndex()).isEqualTo(1); + assertThat(error.errorCode).isEqualTo(PlaybackException.ERROR_CODE_DECODER_INIT_FAILED); + + player.release(); + } + + @Test + public void + rendererError_inThirdPeriodWhileEarliestReadingPeriodIsSecondPeriod_doesNotDisruptFirstOrSecondPeriodPlayback() + throws Exception { + if (!perStreamMediaProgressionEnabled) { + return; + } + + AtomicBoolean allowPeriod2VideoEndOfStream = new AtomicBoolean(false); + AtomicInteger audioPeriod3ErrorCount = new AtomicInteger(0); + RenderersFactory renderersFactory = + (handler, videoListener, audioListener, textOutput, metadataOutput) -> + new Renderer[] { + new FakeVideoRenderer( + SystemClock.DEFAULT.createHandler(handler.getLooper(), /* callback= */ null), + videoListener), + new FakeAudioRenderer( + SystemClock.DEFAULT.createHandler(handler.getLooper(), /* callback= */ null), + audioListener) { + @Override + protected void onStreamChanged( + Format[] formats, + long startPositionUs, + long offsetUs, + MediaSource.MediaPeriodId mediaPeriodId) + throws ExoPlaybackException { + super.onStreamChanged(formats, startPositionUs, offsetUs, mediaPeriodId); + int periodIndex = getTimeline().getIndexOfPeriod(mediaPeriodId.periodUid); + if (periodIndex == 2) { + audioPeriod3ErrorCount.incrementAndGet(); + throw ExoPlaybackException.createForRenderer( + new Exception("Audio error while reading ahead into Period 3"), + getName(), + getIndex(), + formats[0], + C.FORMAT_HANDLED, + mediaPeriodId, + /* isRecoverable= */ false, + PlaybackException.ERROR_CODE_DECODER_INIT_FAILED); + } + } + } + }; + ExoPlayer player = + parameterizeTestExoPlayerBuilder( + new TestExoPlayerBuilder(context).setRenderersFactory(renderersFactory)) + .build(); + TrackGroupArray trackGroupArray = + new TrackGroupArray( + new TrackGroup(ExoPlayerTestRunner.AUDIO_FORMAT), + new TrackGroup(ExoPlayerTestRunner.VIDEO_FORMAT)); + // Period 1 (5s): Both Video and Audio reach end-of-stream immediately at 0ms and advance their + // reading periods to Period 2 while Period 1 is still playing. + MediaSource mediaSource1 = + new FakeMediaSource( + new FakeTimeline( + new TimelineWindowDefinition.Builder() + .setDurationUs(5 * C.MICROS_PER_SECOND) + .build()), + ExoPlayerTestRunner.VIDEO_FORMAT, + ExoPlayerTestRunner.AUDIO_FORMAT); + // Period 2 (2s): Audio reaches end-of-stream immediately and advances to Period 3 (at 7s, + // within the 10s reading-ahead threshold), while Video stays reading Period 2 until + // allowPeriod2VideoEndOfStream is set to true. + MediaSource mediaSource2 = + new FakeMediaSource( + new FakeTimeline( + new TimelineWindowDefinition.Builder() + .setDurationUs(2 * C.MICROS_PER_SECOND) + .build()), + DrmSessionManager.DRM_UNSUPPORTED, + /* trackDataFactory= */ null, + /* syncSampleTimesUs= */ null, + trackGroupArray) { + @Override + protected MediaPeriod createMediaPeriod( + MediaPeriodId id, + TrackGroupArray trackGroupArray, + Allocator allocator, + MediaSourceEventListener.EventDispatcher mediaSourceEventDispatcher, + DrmSessionManager drmSessionManager, + DrmSessionEventListener.EventDispatcher drmEventDispatcher, + TransferListener transferListener) { + return new FakeMediaPeriod( + trackGroupArray, + allocator, + FakeMediaPeriod.TrackDataFactory.singleSampleWithTimeUs(0), + mediaSourceEventDispatcher, + drmSessionManager, + drmEventDispatcher, + /* deferOnPrepared= */ false) { + @Override + protected FakeSampleStream createSampleStream( + Allocator allocator, + MediaSourceEventListener.EventDispatcher mediaSourceEventDispatcher, + DrmSessionManager drmSessionManager, + DrmSessionEventListener.EventDispatcher drmEventDispatcher, + Format initialFormat, + List fakeSampleStreamItems) { + return new FakeSampleStream( + allocator, + mediaSourceEventDispatcher, + drmSessionManager, + drmEventDispatcher, + initialFormat, + fakeSampleStreamItems) { + @Override + public boolean isReady() { + return true; + } + + @Override + public int readData( + FormatHolder formatHolder, + DecoderInputBuffer buffer, + @ReadFlags int readFlags) { + int result = super.readData(formatHolder, buffer, readFlags); + if (MimeTypes.isVideo(initialFormat.sampleMimeType) + && !allowPeriod2VideoEndOfStream.get() + && result == C.RESULT_BUFFER_READ + && buffer.isEndOfStream()) { + buffer.clear(); + return C.RESULT_NOTHING_READ; + } + return result; + } + }; + } + }; + } + }; + MediaSource mediaSource3 = + new FakeMediaSource( + new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT, ExoPlayerTestRunner.AUDIO_FORMAT); + player.setMediaSources(ImmutableList.of(mediaSource1, mediaSource2, mediaSource3)); + player.prepare(); + player.play(); + + // Wait until Audio advances to Period 3 and throws the error while Period 1 is still playing + // and Video is reading Period 2 (earliestReadingPeriod == Period 2). + runMainLooperUntil(() -> audioPeriod3ErrorCount.get() >= 1); + advance(player).untilPendingCommandsAreFullyHandled(); + assertThat(audioPeriod3ErrorCount.get()).isEqualTo(1); + assertThat(player.getPlaybackState()).isEqualTo(Player.STATE_READY); + assertThat(player.getPlayerError()).isNull(); + assertThat(player.getCurrentMediaItemIndex()).isEqualTo(0); + + // Playback transitions smoothly from Period 1 to Period 2 without error. + runMainLooperUntil(() -> player.getCurrentMediaItemIndex() == 1); + advance(player).untilPendingCommandsAreFullyHandled(); + assertThat(player.getPlaybackState()).isEqualTo(Player.STATE_READY); + assertThat(player.getPlayerError()).isNull(); + assertThat(audioPeriod3ErrorCount.get()).isEqualTo(1); + + // Once Video finishes reading Period 2, all streams advance to Period 3 and report the error. + allowPeriod2VideoEndOfStream.set(true); + ExoPlaybackException error = runUntilError(player); + assertThat(audioPeriod3ErrorCount.get()).isEqualTo(2); + assertThat(player.getPlaybackState()).isEqualTo(Player.STATE_IDLE); + assertThat(player.getCurrentMediaItemIndex()).isEqualTo(2); + assertThat(error.errorCode).isEqualTo(PlaybackException.ERROR_CODE_DECODER_INIT_FAILED); + + player.release(); + } + + @Test + public void + play_withNonPreparedPeriodAndPartialRendererTransitionBeforePrepared_enablesAllRenderersOncePrepared() + throws Exception { + AtomicBoolean allowVideoEndOfStream = new AtomicBoolean(false); + AtomicReference audioRendererRef = new AtomicReference<>(); + RenderersFactory renderersFactory = + (handler, videoListener, audioListener, textOutput, metadataOutput) -> { + FakeVideoRenderer videoRenderer = + new FakeVideoRenderer( + SystemClock.DEFAULT.createHandler(handler.getLooper(), /* callback= */ null), + videoListener) { + @Override + public boolean isEnded() { + return allowVideoEndOfStream.get() && super.isEnded(); + } + }; + FakeAudioRenderer audioRenderer = + new FakeAudioRenderer( + SystemClock.DEFAULT.createHandler(handler.getLooper(), /* callback= */ null), + audioListener); + audioRendererRef.set(audioRenderer); + return new Renderer[] {videoRenderer, audioRenderer}; + }; + ExoPlayer player = + parameterizeTestExoPlayerBuilder( + new TestExoPlayerBuilder(context).setRenderersFactory(renderersFactory)) + .build(); + MediaSource mediaSource1 = + new FakeMediaSource( + new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT, ExoPlayerTestRunner.AUDIO_FORMAT); + FakeMediaSource mediaSource2 = + new FakeMediaSource( + new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT, ExoPlayerTestRunner.AUDIO_FORMAT); + mediaSource2.setPeriodDefersOnPreparedCallback(true); + player.setMediaSources(ImmutableList.of(mediaSource1, mediaSource2)); + player.prepare(); + player.play(); + + // Advance until Period 1 reaches its end position (10s) and Audio renderer has ended and been + // disabled while Period 2 is still unprepared and Video renderer has not yet reported + // isEnded(). + advance(player).untilPositionAtLeast(/* mediaItemIndex= */ 0, /* positionMs= */ 10_000); + runMainLooperUntil(() -> audioRendererRef.get().getState() == Renderer.STATE_DISABLED); + FakeMediaPeriod deferredPeriod2 = + (FakeMediaPeriod) mediaSource2.getLastCreatedActiveMediaPeriod(); + + // Complete preparation of Period 2 while playingPeriod is still Period 1. + deferredPeriod2.setPreparationComplete(); + advance(player).untilPendingCommandsAreFullyHandled(); + + // Allow Video renderer to finish Period 1 so playback transitions to Period 2. + allowVideoEndOfStream.set(true); + advance(player).untilStartOfMediaItem(/* mediaItemIndex= */ 1); + + // Verify that the Audio renderer (which ended and disabled while Period 2 was unprepared) is + // properly re-enabled for Period 2. + assertThat(audioRendererRef.get().getState()).isNotEqualTo(Renderer.STATE_DISABLED); + assertThat(audioRendererRef.get().getStream()).isNotNull(); + assertThat(audioRendererRef.get().enabledCount).isEqualTo(2); + + advance(player).untilState(Player.STATE_ENDED); + + player.release(); + } + + @Test + public void + play_withNonPreparedPeriodAndCompleteRendererTransitionBeforePrepared_enablesAllRenderersOncePrepared() + throws Exception { + AtomicBoolean allowVideoEndOfStream = new AtomicBoolean(false); + AtomicReference audioRendererRef = new AtomicReference<>(); + RenderersFactory renderersFactory = + (handler, videoListener, audioListener, textOutput, metadataOutput) -> { + FakeVideoRenderer videoRenderer = + new FakeVideoRenderer( + SystemClock.DEFAULT.createHandler(handler.getLooper(), /* callback= */ null), + videoListener) { + @Override + public boolean isEnded() { + return allowVideoEndOfStream.get() && super.isEnded(); + } + }; + FakeAudioRenderer audioRenderer = + new FakeAudioRenderer( + SystemClock.DEFAULT.createHandler(handler.getLooper(), /* callback= */ null), + audioListener); + audioRendererRef.set(audioRenderer); + return new Renderer[] {videoRenderer, audioRenderer}; + }; + ExoPlayer player = + parameterizeTestExoPlayerBuilder( + new TestExoPlayerBuilder(context).setRenderersFactory(renderersFactory)) + .build(); + MediaSource mediaSource1 = + new FakeMediaSource( + new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT, ExoPlayerTestRunner.AUDIO_FORMAT); + FakeMediaSource mediaSource2 = + new FakeMediaSource( + new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT, ExoPlayerTestRunner.AUDIO_FORMAT); + mediaSource2.setPeriodDefersOnPreparedCallback(true); + player.setMediaSources(ImmutableList.of(mediaSource1, mediaSource2)); + player.prepare(); + player.play(); + + // Advance until Period 1 reaches its end position (10s) and Audio renderer has ended and been + // disabled while Period 2 is still unprepared and Video renderer has not yet reported + // isEnded(). + advance(player).untilPositionAtLeast(/* mediaItemIndex= */ 0, /* positionMs= */ 10_000); + runMainLooperUntil(() -> audioRendererRef.get().getState() == Renderer.STATE_DISABLED); + FakeMediaPeriod deferredPeriod2 = + (FakeMediaPeriod) mediaSource2.getLastCreatedActiveMediaPeriod(); + + // Now allow Video renderer to also finish Period 1 while Period 2 is STILL unprepared. + allowVideoEndOfStream.set(true); + advance(player).untilState(Player.STATE_BUFFERING); + assertThat(player.getCurrentMediaItemIndex()).isEqualTo(1); + + // Complete preparation of Period 2 after playingPeriod has already advanced to Period 2. + deferredPeriod2.setPreparationComplete(); + advance(player).untilState(Player.STATE_READY); + + // Verify that the Audio renderer is properly re-enabled for Period 2. + assertThat(audioRendererRef.get().getState()).isNotEqualTo(Renderer.STATE_DISABLED); + assertThat(audioRendererRef.get().getStream()).isNotNull(); + assertThat(audioRendererRef.get().enabledCount).isEqualTo(2); + + advance(player).untilState(Player.STATE_ENDED); + + player.release(); + } + @Test public void playLiveStreams_withPerStreamMediaProgressionEnabled_doesNotAdvanceReadingPeriodPerStream() @@ -1577,6 +2039,52 @@ public void allActivatedTrackSelectionAreReleasedForSinglePeriod() throws Except assertThat(numSelectionsEnabled).isEqualTo(2); } + @Test + public void setTrackSelectionParameters_rapidUpdate_notifiesSequentially() throws Exception { + MediaSource mediaSource = + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT); + List parameterHistory = new ArrayList<>(); + FakeTrackSelector trackSelector = + new FakeTrackSelector() { + @Override + protected void selectAllTracks( + ExoTrackSelection.@NullableType Definition[] definitions, + MappedTrackInfo mappedTrackInfo, + int[][][] rendererFormatSupports, + int[] rendererMixedMimeTypeAdaptationSupports, + Parameters params) { + super.selectAllTracks( + definitions, + mappedTrackInfo, + rendererFormatSupports, + rendererMixedMimeTypeAdaptationSupports, + params); + parameterHistory.add(params); + } + }; + ExoPlayer player = + parameterizeTestExoPlayerBuilder( + new TestExoPlayerBuilder(context).setTrackSelector(trackSelector)) + .build(); + player.setMediaSource(mediaSource); + player.prepare(); + advance(player).untilState(Player.STATE_READY); + // Initial track selection happens during prepare/ready. Clear history to focus on updates. + parameterHistory.clear(); + // Trigger two parameter changes in rapid succession on the application thread. + DefaultTrackSelector.Parameters params1 = + trackSelector.buildUponParameters().setPreferredAudioLanguage("eng").build(); + DefaultTrackSelector.Parameters params2 = + trackSelector.buildUponParameters().setPreferredAudioLanguage("deu").build(); + + player.setTrackSelectionParameters(params1); + player.setTrackSelectionParameters(params2); + + advance(player).untilPendingCommandsAreFullyHandled(); + player.release(); + assertThat(parameterHistory).containsExactly(params1, params2).inOrder(); + } + @Test public void allActivatedTrackSelectionAreReleasedForMultiPeriods() throws Exception { Timeline timeline = new FakeTimeline(/* windowCount= */ 2); @@ -8403,6 +8911,18 @@ public void maybeThrowError() throws IOException { assertThat(renderer.sampleBufferReadCount).isEqualTo(3); } + @Test + public void seekTo_withEmptyTimeline_emitsPositionDiscontinuityEvent() throws Exception { + ExoPlayer player = parameterizeTestExoPlayerBuilder(new TestExoPlayerBuilder(context)).build(); + Player.Listener mockListener = mock(Player.Listener.class); + player.addListener(mockListener); + + player.seekTo(/* positionMs= */ 2_000L); + + verify(mockListener).onPositionDiscontinuity(any(), any(), eq(DISCONTINUITY_REASON_SEEK)); + player.release(); + } + @Test public void seekTo_whileReady_callsOnIsPlayingChanged() throws Exception { ActionSchedule actionSchedule = @@ -9876,6 +10396,7 @@ protected void onEnabled(boolean joining, boolean mayRenderStartOfStream) @Test public void play_withDynamicSchedulingEnabled_usesRendererDurationSchedulingInterval() throws Exception { + Flags.enableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); AtomicInteger renderCounter = new AtomicInteger(); ForwardingDurationToProgressRenderer fakeRenderer = new ForwardingDurationToProgressRenderer( @@ -9884,11 +10405,7 @@ public void play_withDynamicSchedulingEnabled_usesRendererDurationSchedulingInte renderCounter); FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); ExoPlayer player = - new TestExoPlayerBuilder(context) - .setClock(clock) - .setDynamicSchedulingEnabled(true) - .setRenderers(fakeRenderer) - .build(); + new TestExoPlayerBuilder(context).setClock(clock).setRenderers(fakeRenderer).build(); player.setMediaSource( new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT)); player.prepare(); @@ -9934,6 +10451,7 @@ public void play_withDynamicSchedulingDisabled_usesDefaultSchedulingInterval() t public void play_withDynamicSchedulingEnabledAndMultipleRenderers_usesMinimumDurationSchedulingInterval() throws Exception { + Flags.enableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); AtomicInteger audioRenderCounter = new AtomicInteger(); ForwardingDurationToProgressRenderer fakeAudioRenderer = new ForwardingDurationToProgressRenderer( @@ -9949,7 +10467,6 @@ public void play_withDynamicSchedulingDisabled_usesDefaultSchedulingInterval() t ExoPlayer player = new TestExoPlayerBuilder(context) .setClock(clock) - .setDynamicSchedulingEnabled(true) .setRenderers(fakeAudioRenderer, fakeVideoRenderer) .build(); player.setMediaSource( @@ -9973,6 +10490,7 @@ public void play_withDynamicSchedulingDisabled_usesDefaultSchedulingInterval() t @Test public void prepareOnly_withDynamicSchedulingEnabled_usesDefaultIdleSchedulingInterval() throws Exception { + Flags.enableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); AtomicInteger renderCounter = new AtomicInteger(); FakeRenderer fakeRenderer = new FakeRenderer(C.TRACK_TYPE_AUDIO) { @@ -9984,11 +10502,7 @@ public void render(long positionUs, long elapsedRealtimeUs) throws ExoPlaybackEx }; FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); ExoPlayer player = - new TestExoPlayerBuilder(context) - .setClock(clock) - .setDynamicSchedulingEnabled(true) - .setRenderers(fakeRenderer) - .build(); + new TestExoPlayerBuilder(context).setClock(clock).setRenderers(fakeRenderer).build(); player.setMediaSource( new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT)); player.prepare(); @@ -10005,6 +10519,7 @@ public void render(long positionUs, long elapsedRealtimeUs) throws ExoPlaybackEx @Test public void play_withDynamicSchedulingEnabledAndInBufferingState_usesBufferingSchedulingInterval() throws Exception { + Flags.enableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); AtomicInteger renderCounter = new AtomicInteger(); AtomicBoolean allowStreamRead = new AtomicBoolean(); ForwardingDurationToProgressRenderer fakeRenderer = @@ -10021,11 +10536,7 @@ public boolean isReady() { }; FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); ExoPlayer player = - new TestExoPlayerBuilder(context) - .setClock(clock) - .setDynamicSchedulingEnabled(true) - .setRenderers(fakeRenderer) - .build(); + new TestExoPlayerBuilder(context).setClock(clock).setRenderers(fakeRenderer).build(); // Prevent reading any samples to keep player in a buffering state. FakeDelayedMediaSource fakeDelayedMediaSource = new FakeDelayedMediaSource( @@ -10047,6 +10558,7 @@ public boolean isReady() { @Test @Config(minSdk = 31) public void play_withDynamicSchedulingEnabled_wakesUpExoPlayerForWork() throws Exception { + Flags.enableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); AtomicInteger renderCounter = new AtomicInteger(); AtomicBoolean queuedInputBuffer = new AtomicBoolean(); AtomicBoolean allowProgressInRenderBeyondFirstBuffer = new AtomicBoolean(); @@ -10094,7 +10606,6 @@ protected void onQueueInputBuffer(DecoderInputBuffer buffer) ExoPlayer player = new TestExoPlayerBuilder(context) .setRenderersFactory(renderersFactory) - .setDynamicSchedulingEnabled(true) .setClock(clock) .build(); player.setMediaSource( @@ -10114,6 +10625,325 @@ protected void onQueueInputBuffer(DecoderInputBuffer buffer) assertThat(renderCounter.get()).isLessThan(40); } + @Test + public void getCurrentPosition_withDynamicSchedulingEnabled_calculatesUsingEstimatedPosition() + throws Exception { + Flags.enableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); + AtomicInteger renderCounter = new AtomicInteger(); + ForwardingDurationToProgressRenderer fakeRenderer = + new ForwardingDurationToProgressRenderer( + new FakeRenderer(C.TRACK_TYPE_AUDIO), + /* durationToProgressUs= */ 500_000L, + renderCounter); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); + ExoPlayer player = + new TestExoPlayerBuilder(context).setClock(clock).setRenderers(fakeRenderer).build(); + player.setMediaSource( + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT)); + player.prepare(); + player.play(); + advance(player).untilState(Player.STATE_READY); + + long positionAtT0 = player.getCurrentPosition(); + clock.advanceTime(100); + long positionAtT1 = player.getCurrentPosition(); + + assertThat(positionAtT1).isEqualTo(positionAtT0 + 100); + + player.release(); + } + + @Test + public void + getCurrentPosition_withDynamicSchedulingEnabledAndPaused_calculatedPositionIsNotExtrapolated() + throws Exception { + Flags.enableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); + AtomicInteger renderCounter = new AtomicInteger(); + ForwardingDurationToProgressRenderer fakeRenderer = + new ForwardingDurationToProgressRenderer( + new FakeRenderer(C.TRACK_TYPE_AUDIO), + /* durationToProgressUs= */ 500_000L, + renderCounter); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); + ExoPlayer player = + new TestExoPlayerBuilder(context).setClock(clock).setRenderers(fakeRenderer).build(); + player.setMediaSource( + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT)); + player.prepare(); + player.play(); + advance(player).untilState(Player.STATE_READY); + + long positionAtT0 = player.getCurrentPosition(); + player.pause(); + clock.advanceTime(100); + long positionAtT1 = player.getCurrentPosition(); + + assertThat(positionAtT1).isEqualTo(positionAtT0); + + player.release(); + } + + @Test + public void + getCurrentPosition_withDynamicSchedulingEnabledAndPlayThenPaused_returnsMaskedEstimatedPosition() + throws Exception { + Flags.enableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); + AtomicInteger renderCounter = new AtomicInteger(); + ForwardingDurationToProgressRenderer fakeRenderer = + new ForwardingDurationToProgressRenderer( + new FakeRenderer(C.TRACK_TYPE_AUDIO), + /* durationToProgressUs= */ 500_000L, + renderCounter); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); + ExoPlayer player = + new TestExoPlayerBuilder(context).setClock(clock).setRenderers(fakeRenderer).build(); + player.setMediaSource( + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT)); + player.prepare(); + player.play(); + advance(player).untilState(Player.STATE_READY); + + long positionAtT0 = player.getCurrentPosition(); + clock.advanceTime(100); + player.pause(); + long positionAtT1Masked = player.getCurrentPosition(); + advance(player).untilPendingCommandsAreFullyHandled(); + long positionAtT1Synced = player.getCurrentPosition(); + + assertThat(positionAtT1Masked).isEqualTo(positionAtT0 + 100); + assertThat(positionAtT1Synced).isEqualTo(positionAtT0 + 100); + + player.release(); + } + + @Test + public void + getCurrentPosition_withDynamicSchedulingEnabledAndStopped_returnsMaskedEstimatedPosition() + throws Exception { + Flags.enableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); + AtomicInteger renderCounter = new AtomicInteger(); + ForwardingDurationToProgressRenderer fakeRenderer = + new ForwardingDurationToProgressRenderer( + new FakeRenderer(C.TRACK_TYPE_AUDIO), + /* durationToProgressUs= */ 500_000L, + renderCounter); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); + ExoPlayer player = + new TestExoPlayerBuilder(context).setClock(clock).setRenderers(fakeRenderer).build(); + player.setMediaSource( + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT)); + player.prepare(); + player.play(); + advance(player).untilState(Player.STATE_READY); + + long positionAtT0 = player.getCurrentPosition(); + clock.advanceTime(100); + player.stop(); + long positionAtT1Masked = player.getCurrentPosition(); + advance(player).untilPendingCommandsAreFullyHandled(); + long positionAtT1Synced = player.getCurrentPosition(); + + assertThat(positionAtT1Masked).isEqualTo(positionAtT0 + 100); + assertThat(positionAtT1Synced).isEqualTo(positionAtT0 + 100); + + player.release(); + } + + @Test + public void + getCurrentPosition_withDynamicSchedulingEnabledAndSpeedChangedBeforeClockAdvance_calculatesCorrectPositionEstimate() + throws Exception { + Flags.enableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); + AtomicInteger renderCounter = new AtomicInteger(); + ForwardingDurationToProgressRenderer fakeRenderer = + new ForwardingDurationToProgressRenderer( + new FakeRenderer(C.TRACK_TYPE_AUDIO), + /* durationToProgressUs= */ 500_000L, + renderCounter); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); + ExoPlayer player = + new TestExoPlayerBuilder(context).setClock(clock).setRenderers(fakeRenderer).build(); + player.setMediaSource( + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT)); + player.prepare(); + player.play(); + advance(player).untilState(Player.STATE_READY); + + player.setPlaybackParameters(new PlaybackParameters(2.0f)); + advance(player).untilPendingCommandsAreFullyHandled(); + long positionAtT0 = player.getCurrentPosition(); + clock.advanceTime(100); + long positionAtT1 = player.getCurrentPosition(); + + assertThat(positionAtT1).isEqualTo(positionAtT0 + 200); + + player.release(); + } + + @Test + public void + getCurrentPosition_withDynamicSchedulingEnabledAndSpeedChangedAfterClockAdvance_calculatesCorrectPositionEstimate() + throws Exception { + Flags.enableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); + AtomicInteger renderCounter = new AtomicInteger(); + ForwardingDurationToProgressRenderer fakeRenderer = + new ForwardingDurationToProgressRenderer( + new FakeRenderer(C.TRACK_TYPE_AUDIO), + /* durationToProgressUs= */ 500_000L, + renderCounter); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); + ExoPlayer player = + new TestExoPlayerBuilder(context).setClock(clock).setRenderers(fakeRenderer).build(); + player.setMediaSource( + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT)); + player.prepare(); + player.play(); + advance(player).untilState(Player.STATE_READY); + + long positionAtT0 = player.getCurrentPosition(); + clock.advanceTime(100); + player.setPlaybackParameters(new PlaybackParameters(2.0f)); + long positionAtT1Masked = player.getCurrentPosition(); + advance(player).untilPendingCommandsAreFullyHandled(); + long positionAtT1Synced = player.getCurrentPosition(); + + assertThat(positionAtT1Masked).isEqualTo(positionAtT0 + 100); + assertThat(positionAtT1Synced).isEqualTo(positionAtT0 + 100); + + player.release(); + } + + @Test + public void getCurrentPosition_withDynamicSchedulingEnabled_enforcesMonotonicPositionTracking() + throws Exception { + Flags.enableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); + AtomicInteger renderCounter = new AtomicInteger(); + AtomicLong rendererPositionUs = new AtomicLong(0); + ForwardingDurationToProgressRenderer fakeRenderer = + new ForwardingDurationToProgressRenderer( + new FakeMediaClockRenderer(C.TRACK_TYPE_AUDIO) { + @Override + public long getPositionUs() { + return rendererPositionUs.get(); + } + + @Override + public void setPlaybackParameters(PlaybackParameters playbackParameters) {} + + @Override + public PlaybackParameters getPlaybackParameters() { + return PlaybackParameters.DEFAULT; + } + }, + /* durationToProgressUs= */ 500_000L, // 500 ms sleep interval + renderCounter); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); + ExoPlayer player = + new TestExoPlayerBuilder(context).setClock(clock).setRenderers(fakeRenderer).build(); + player.setMediaSource( + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT)); + player.prepare(); + player.play(); + advance(player).untilState(Player.STATE_READY); + long initialPosition = player.getCurrentPosition(); + // Set renderer position 10ms ahead to guarantee using DefaultMediaClock. + rendererPositionUs.set(Util.msToUs(initialPosition + 10)); + clock.advanceTime(500); + advance(player).untilPendingCommandsAreFullyHandled(); + + long positionAtT0 = player.getCurrentPosition(); + clock.advanceTime(100); + long positionAtT1 = player.getCurrentPosition(); + assertThat(positionAtT1).isEqualTo(positionAtT0 + 100); + + // Advance the hardware renderer by only 50 ms to simulate slower progress slower than realtime. + rendererPositionUs.addAndGet(Util.msToUs(50)); + clock.advanceTime(400); + advance(player).untilPendingCommandsAreFullyHandled(); + + // Verify the monotonic clamp prevents the position from dropping to T0 + 50. + long positionAtT2 = player.getCurrentPosition(); + assertThat(positionAtT2).isEqualTo(positionAtT1); + + // Verify seeking resets the monotonic clamp. + player.seekTo(0); + long positionAfterSeek = player.getCurrentPosition(); + assertThat(positionAfterSeek).isEqualTo(0); + assertThat(positionAfterSeek).isLessThan(positionAtT1); + + player.release(); + } + + @Test + public void + getCurrentPosition_withDynamicSchedulingEnabled_playAfterPauseDoesNotExtrapolateBeforeInternalThreadCatchesUp() + throws Exception { + Flags.enableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); + AtomicInteger renderCounter = new AtomicInteger(); + ForwardingDurationToProgressRenderer fakeRenderer = + new ForwardingDurationToProgressRenderer( + new FakeRenderer(C.TRACK_TYPE_AUDIO), + /* durationToProgressUs= */ 500_000L, + renderCounter); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); + ExoPlayer player = + new TestExoPlayerBuilder(context).setClock(clock).setRenderers(fakeRenderer).build(); + player.setMediaSource( + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT)); + + player.prepare(); + advance(player).untilState(Player.STATE_READY); + long positionAtT0 = player.getCurrentPosition(); + player.play(); + // Simulates the UI thread advancing time (e.g., during layout) before the background thread + // processes the play command. + clock.advanceTime(100); + long positionAtT1 = player.getCurrentPosition(); + + assertThat(positionAtT1).isEqualTo(positionAtT0); + + player.release(); + } + + @Test + public void play_backwardSeekWithDynamicScheduling_reportsCorrectTargetInDiscontinuityCallback() + throws Exception { + Flags.enableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); + AtomicInteger renderCounter = new AtomicInteger(); + AtomicLong reportedNewPositionMs = new AtomicLong(C.TIME_UNSET); + ForwardingDurationToProgressRenderer fakeRenderer = + new ForwardingDurationToProgressRenderer( + new FakeRenderer(C.TRACK_TYPE_AUDIO), + /* durationToProgressUs= */ 500_000L, + renderCounter); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); + ExoPlayer player = + new TestExoPlayerBuilder(context).setClock(clock).setRenderers(fakeRenderer).build(); + player.setMediaSource( + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT)); + player.prepare(); + player.play(); + advance(player).untilState(Player.STATE_READY); + clock.advanceTime(100); + assertThat(player.getCurrentPosition()).isEqualTo(100); + player.addListener( + new Player.Listener() { + @Override + public void onPositionDiscontinuity( + PositionInfo oldPosition, PositionInfo newPosition, int reason) { + if (reason == Player.DISCONTINUITY_REASON_SEEK) { + reportedNewPositionMs.set(newPosition.positionMs); + } + } + }); + + player.seekTo(20); + + assertThat(reportedNewPositionMs.get()).isEqualTo(20); + + player.release(); + } + @Test public void enablingOffload_withAudioOnly_playerSleeps() throws Exception { FakeSleepRenderer sleepRenderer = new FakeSleepRenderer(C.TRACK_TYPE_AUDIO); @@ -14323,6 +15153,7 @@ protected void buildAudioRenderers( } }; + Flags.disableFlag(Flags.FLAG_PER_STREAM_MEDIA_PROGRESSION); ExoPlayer player = new TestExoPlayerBuilder(context).setRenderersFactory(renderersFactory).build(); Format formatA = @@ -14805,6 +15636,7 @@ public void render(long positionUs, long elapsedRealtimeUs) throws ExoPlaybackEx super.render(positionUs, elapsedRealtimeUs); } }; + Flags.enableFlag(Flags.FLAG_DYNAMIC_SCHEDULING); ExoPlayer player = new ExoPlayer.Builder(context) .setClock(clock) @@ -14814,7 +15646,6 @@ public void render(long positionUs, long elapsedRealtimeUs) throws ExoPlaybackEx audioRendererEventListener, textRendererOutput, metadataRendererOutput) -> new Renderer[] {fakeRenderer}) - .experimentalSetDynamicSchedulingEnabled(true) .build(); player.setMediaSource( new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT)); diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerVideoFrameMetadataListenerTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerVideoFrameMetadataListenerTest.java new file mode 100644 index 00000000000..b2b09005edd --- /dev/null +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerVideoFrameMetadataListenerTest.java @@ -0,0 +1,150 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.exoplayer; + +import static androidx.media3.test.utils.robolectric.TestPlayerRunHelper.play; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyLong; +import static org.mockito.Mockito.atLeastOnce; +import static org.mockito.Mockito.clearInvocations; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.verify; + +import android.content.Context; +import android.graphics.SurfaceTexture; +import android.view.Surface; +import androidx.media3.exoplayer.video.VideoFrameMetadataListener; +import androidx.media3.test.utils.ExoPlayerTestRunner; +import androidx.media3.test.utils.FakeMediaSource; +import androidx.media3.test.utils.FakeTimeline; +import androidx.media3.test.utils.TestExoPlayerBuilder; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link ExoPlayer} video frame metadata listeners. */ +@RunWith(AndroidJUnit4.class) +public final class ExoPlayerVideoFrameMetadataListenerTest { + + private Surface surface; + private ExoPlayer player; + private VideoFrameMetadataListener listener1; + private VideoFrameMetadataListener listener2; + + @Before + public void setUp() { + Context context = ApplicationProvider.getApplicationContext(); + surface = new Surface(new SurfaceTexture(/* texName= */ 1)); + player = new TestExoPlayerBuilder(context).build(); + player.setVideoSurface(surface); + player.setMediaSource( + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT)); + listener1 = mock(VideoFrameMetadataListener.class); + listener2 = mock(VideoFrameMetadataListener.class); + } + + @After + public void tearDown() { + player.release(); + surface.release(); + } + + @Test + public void addAndRemoveVideoFrameMetadataListener_deliversEventsToRegisteredListeners() + throws Exception { + player.addVideoFrameMetadataListener(listener1); + player.addVideoFrameMetadataListener(listener2); + player.prepare(); + + play(player).untilPosition(0, 100); + + verify(listener1, atLeastOnce()) + .onVideoFrameAboutToBeRendered(anyLong(), anyLong(), any(), any()); + verify(listener2, atLeastOnce()) + .onVideoFrameAboutToBeRendered(anyLong(), anyLong(), any(), any()); + + clearInvocations(listener1, listener2); + player.removeVideoFrameMetadataListener(listener1); + player.seekTo(0); + + play(player).untilPosition(0, 100); + + verify(listener1, never()).onVideoFrameAboutToBeRendered(anyLong(), anyLong(), any(), any()); + verify(listener2, atLeastOnce()) + .onVideoFrameAboutToBeRendered(anyLong(), anyLong(), any(), any()); + } + + @Test + @SuppressWarnings("deprecation") // Testing deprecated setVideoFrameMetadataListener. + public void setAndClearVideoFrameMetadataListener_deliversEventsAndUnregistersCorrectly() + throws Exception { + player.setVideoFrameMetadataListener(listener1); + player.setVideoFrameMetadataListener(listener2); + player.prepare(); + + play(player).untilPosition(0, 100); + + verify(listener1, never()).onVideoFrameAboutToBeRendered(anyLong(), anyLong(), any(), any()); + verify(listener2, atLeastOnce()) + .onVideoFrameAboutToBeRendered(anyLong(), anyLong(), any(), any()); + + clearInvocations(listener1, listener2); + player.clearVideoFrameMetadataListener(listener2); + player.seekTo(0); + + play(player).untilPosition(0, 100); + + verify(listener2, never()).onVideoFrameAboutToBeRendered(anyLong(), anyLong(), any(), any()); + } + + @Test + @SuppressWarnings("deprecation") // Testing interaction with deprecated methods. + public void mixedSetAndAddCalls_deliversEventsAndUnregistersCorrectly() throws Exception { + player.setVideoFrameMetadataListener(listener1); + player.addVideoFrameMetadataListener(listener2); + player.prepare(); + + play(player).untilPosition(0, 100); + + verify(listener1, atLeastOnce()) + .onVideoFrameAboutToBeRendered(anyLong(), anyLong(), any(), any()); + verify(listener2, atLeastOnce()) + .onVideoFrameAboutToBeRendered(anyLong(), anyLong(), any(), any()); + + clearInvocations(listener1, listener2); + player.removeVideoFrameMetadataListener(listener1); + player.seekTo(0); + + play(player).untilPosition(0, 100); + + verify(listener1, never()).onVideoFrameAboutToBeRendered(anyLong(), anyLong(), any(), any()); + verify(listener2, atLeastOnce()) + .onVideoFrameAboutToBeRendered(anyLong(), anyLong(), any(), any()); + + clearInvocations(listener1, listener2); + player.clearVideoFrameMetadataListener(listener2); + player.seekTo(0); + + play(player).untilPosition(0, 100); + + verify(listener1, never()).onVideoFrameAboutToBeRendered(anyLong(), anyLong(), any(), any()); + verify(listener2, never()).onVideoFrameAboutToBeRendered(anyLong(), anyLong(), any(), any()); + } +} diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerWithPrewarmingRenderersTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerWithPrewarmingRenderersTest.java index eac9780553a..706978bc86b 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerWithPrewarmingRenderersTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/ExoPlayerWithPrewarmingRenderersTest.java @@ -33,6 +33,7 @@ import android.util.Pair; import androidx.annotation.Nullable; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.Format; import androidx.media3.common.PlaybackException; import androidx.media3.common.Player; @@ -56,6 +57,7 @@ import androidx.media3.exoplayer.trackselection.ExoTrackSelection; import androidx.media3.exoplayer.upstream.Allocator; import androidx.media3.exoplayer.video.VideoRendererEventListener; +import androidx.media3.test.utils.BindFlag; import androidx.media3.test.utils.ExoPlayerTestRunner; import androidx.media3.test.utils.FakeAudioRenderer; import androidx.media3.test.utils.FakeClock; @@ -64,6 +66,7 @@ import androidx.media3.test.utils.FakeSampleStream; import androidx.media3.test.utils.FakeTimeline; import androidx.media3.test.utils.FakeVideoRenderer; +import androidx.media3.test.utils.Media3FlagsRule; import androidx.media3.test.utils.TestExoPlayerBuilder; import androidx.media3.test.utils.robolectric.ShadowMediaCodecConfig; import androidx.test.core.app.ApplicationProvider; @@ -72,7 +75,6 @@ import java.util.List; import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicReference; -import org.junit.Before; import org.junit.Rule; import org.junit.Test; import org.junit.runner.RunWith; @@ -82,7 +84,7 @@ @RunWith(ParameterizedRobolectricTestRunner.class) // TODO: b/510217604 - Remove parameterization. public class ExoPlayerWithPrewarmingRenderersTest { - private Context context; + private final Context context = ApplicationProvider.getApplicationContext(); @ParameterizedRobolectricTestRunner.Parameters(name = "perStream={0}") public static ImmutableList params() { @@ -90,16 +92,14 @@ public static ImmutableList params() { } @ParameterizedRobolectricTestRunner.Parameter(0) + @BindFlag(Flags.FLAG_PER_STREAM_MEDIA_PROGRESSION) public Boolean perStreamMediaProgressionEnabled; @Rule public ShadowMediaCodecConfig mediaCodecConfig = ShadowMediaCodecConfig.withAllDefaultSupportedCodecs(); - @Before - public void setUp() { - context = ApplicationProvider.getApplicationContext(); - } + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); @Test public void play_withTwoItemsAndPrewarming_secondaryRendererisEnabledButNotStarted() @@ -110,7 +110,6 @@ public void play_withTwoItemsAndPrewarming_secondaryRendererisEnabledButNotStart .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer videoRenderer = player.getRenderer(/* index= */ 0); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); @@ -155,7 +154,6 @@ public void play_withTwoItemsAndPrewarming_secondaryRendererisEnabledButNotStart .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer videoRenderer = player.getRenderer(/* index= */ 0); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); @@ -180,7 +178,7 @@ public void play_withTwoItemsAndPrewarming_secondaryRendererisEnabledButNotStart () -> secondaryVideoRenderer.getState() == Renderer.STATE_ENABLED); @Renderer.State int videoState1 = videoRenderer.getState(); @Renderer.State int secondaryVideoState1 = secondaryVideoRenderer.getState(); - // Play until until the primary renderer is being pre-warmed. + // Play until the primary renderer is being pre-warmed. advance(player) .untilBackgroundThreadCondition(() -> videoRenderer.getState() == Renderer.STATE_ENABLED); @Renderer.State int videoState2 = videoRenderer.getState(); @@ -200,6 +198,62 @@ public void play_withTwoItemsAndPrewarming_secondaryRendererisEnabledButNotStart assertThat(secondaryVideoState3).isEqualTo(Renderer.STATE_DISABLED); } + @Test + public void + play_multipleItemPlaylistAndSecondaryRendererLastActive_prewarmsAndUsesSecondaryRenderer() + throws Exception { + Clock fakeClock = new FakeClock(/* isAutoAdvancing= */ true); + ExoPlayer player = + new TestExoPlayerBuilder(context) + .setClock(fakeClock) + .setRenderersFactory( + new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) + .build(); + Renderer videoRenderer = player.getRenderer(/* index= */ 0); + Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); + // Set a playlist with 2 videos, 2 audio-only items, and 1 final video item. + // Secondary-renderer should be last used and therefore pre-warmed for final item. + player.setMediaSources( + ImmutableList.of( + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT), + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT), + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT), + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.AUDIO_FORMAT), + new FakeMediaSource(new FakeTimeline(), ExoPlayerTestRunner.VIDEO_FORMAT))); + player.prepare(); + player.play(); + // Play until the secondary video renderer is started (playing the second video item). + advance(player) + .untilBackgroundThreadCondition( + () -> secondaryVideoRenderer.getState() == Renderer.STATE_STARTED); + // Play to the audio items. + advance(player) + .untilBackgroundThreadCondition( + () -> + secondaryVideoRenderer.getState() == Renderer.STATE_DISABLED + && videoRenderer.getState() == Renderer.STATE_DISABLED); + + // Play until the secondary video renderer is enabled for pre-warming the final video item. + advance(player) + .untilBackgroundThreadCondition( + () -> secondaryVideoRenderer.getState() == Renderer.STATE_ENABLED); + @Renderer.State int videoState1 = videoRenderer.getState(); + @Renderer.State int secondaryVideoState1 = secondaryVideoRenderer.getState(); + // Play until the final media item is started. + advance(player) + .untilBackgroundThreadCondition( + () -> secondaryVideoRenderer.getState() == Renderer.STATE_STARTED); + @Renderer.State int videoState2 = videoRenderer.getState(); + @Renderer.State int secondaryVideoState2 = secondaryVideoRenderer.getState(); + + player.release(); + + assertThat(videoState1).isEqualTo(Renderer.STATE_DISABLED); + assertThat(secondaryVideoState1).isEqualTo(Renderer.STATE_ENABLED); + assertThat(videoState2).isEqualTo(Renderer.STATE_DISABLED); + assertThat(secondaryVideoState2).isEqualTo(Renderer.STATE_STARTED); + } + @Test public void prepare_withPeriodBetweenPlayingAndPrewarmingPeriods_playerSuccessfullyPrewarms() throws Exception { @@ -209,7 +263,6 @@ public void prepare_withPeriodBetweenPlayingAndPrewarmingPeriods_playerSuccessfu .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); // Set a playlist that allows a new renderer to be enabled early. @@ -246,7 +299,6 @@ public void prepare_withPeriodBetweenPlayingAndPrewarmingPeriods_playerSuccessfu .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); // Set a playlist that would prevent pre-warming at normal speed with duration limit for reading @@ -285,7 +337,6 @@ public void prepare_withPeriodBetweenPlayingAndPrewarmingPeriods_playerSuccessfu .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); // Set a playlist that would prevent pre-warming at normal speed with duration limit for reading @@ -324,7 +375,6 @@ public void prepare_withPeriodBetweenPlayingAndPrewarmingPeriods_playerSuccessfu .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer videoRenderer = player.getRenderer(/* index= */ 0); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); @@ -384,7 +434,6 @@ public void prepare_withPeriodBetweenPlayingAndPrewarmingPeriods_playerSuccessfu .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer videoRenderer = player.getRenderer(/* index= */ 0); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); @@ -428,7 +477,6 @@ public void prepare_withPeriodBetweenPlayingAndPrewarmingPeriods_playerSuccessfu .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer videoRenderer = player.getRenderer(/* index= */ 0); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); @@ -496,7 +544,6 @@ public void prepare_withPeriodBetweenPlayingAndPrewarmingPeriods_playerSuccessfu .setTrackSelector(defaultTrackSelector) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer videoRenderer = player.getRenderer(/* index= */ 0); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); @@ -548,7 +595,6 @@ public void prepare_withPeriodBetweenPlayingAndPrewarmingPeriods_playerSuccessfu .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer videoRenderer = player.getRenderer(/* index= */ 0); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); @@ -604,7 +650,6 @@ public void prepare_withPeriodBetweenPlayingAndPrewarmingPeriods_playerSuccessfu .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer videoRenderer = player.getRenderer(/* index= */ 0); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); @@ -681,7 +726,6 @@ protected Pair selectAudioTrack( .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) .setTrackSelector(trackSelector) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Format audioFormat = ExoPlayerTestRunner.AUDIO_FORMAT.buildUpon().setAverageBitrate(70_000).build(); @@ -732,7 +776,6 @@ protected Pair selectAudioTrack( .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer videoRenderer = player.getRenderer(/* index= */ 0); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); @@ -784,7 +827,6 @@ protected Pair selectAudioTrack( .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer videoRenderer = player.getRenderer(/* index= */ 0); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); @@ -832,7 +874,6 @@ public void seek_intoCurrentPeriodWithSecondaryAndReadingPeriodAdvanced_swapsToP .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer videoRenderer = player.getRenderer(/* index= */ 0); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); @@ -882,7 +923,6 @@ public void seek_pastReadingPeriodWithSecondaryRendererOnPlayingPeriod_swapsToPr .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer videoRenderer = player.getRenderer(/* index= */ 0); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); @@ -926,7 +966,6 @@ public void seek_pastReadingPeriodWithSecondaryRendererOnPlayingPeriod_swapsToPr .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer videoRenderer = player.getRenderer(/* index= */ 0); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); @@ -978,7 +1017,6 @@ public void seek_pastReadingPeriodWithSecondaryRendererOnPlayingPeriod_swapsToPr .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer videoRenderer = player.getRenderer(/* index= */ 0); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); @@ -1031,7 +1069,6 @@ public void seek_pastReadingPeriodWithSecondaryRendererOnPlayingPeriod_swapsToPr .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer videoRenderer = player.getRenderer(/* index= */ 0); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); @@ -1088,7 +1125,6 @@ public void seek_pastReadingPeriodWithSecondaryRendererOnPlayingPeriod_swapsToPr .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer videoRenderer = player.getRenderer(/* index= */ 0); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); @@ -1138,7 +1174,6 @@ public void seek_pastReadingPeriodWithSecondaryRendererOnPlayingPeriod_swapsToPr .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer videoRenderer = player.getRenderer(/* index= */ 0); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); @@ -1184,7 +1219,6 @@ public void seek_pastReadingPeriodWithSecondaryRendererOnPlayingPeriod_swapsToPr .setClock(fakeClock) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRenderer(fakeClock)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer videoRenderer = player.getRenderer(/* index= */ 0); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); @@ -1239,7 +1273,6 @@ public void seek_pastReadingPeriodWithSecondaryRendererOnPlayingPeriod_swapsToPr .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRendererThatThrowsOnRender( fakeClock, attemptedRenderWithSecondaryRenderer, shouldSecondaryRendererThrow)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); player.addListener(listener); Renderer videoRenderer = player.getRenderer(/* index= */ 0); @@ -1280,7 +1313,6 @@ public void seek_pastReadingPeriodWithSecondaryRendererOnPlayingPeriod_swapsToPr .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRendererThatThrowsOnRender( fakeClock, attemptedRenderWithSecondaryRenderer, shouldSecondaryRendererThrow)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); player.addListener(listener); Renderer videoRenderer = player.getRenderer(/* index= */ 0); @@ -1330,7 +1362,6 @@ public void play_errorByPrewarmingSecondaryRenderer_primaryRendererIsUsedOnSubse .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRendererThatThrowsOnRender( fakeClock, attemptedRenderWithSecondaryRenderer, shouldSecondaryRendererThrow)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); player.addListener(listener); Renderer videoRenderer = player.getRenderer(/* index= */ 0); @@ -1379,7 +1410,6 @@ public void play_errorByPrewarmingSecondaryRenderer_primaryRendererIsUsedOnSubse .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRendererThatThrowsOnRender( fakeClock, attemptedRenderWithSecondaryRenderer, shouldSecondaryRendererThrow)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); player.addListener(listener); Renderer videoRenderer = player.getRenderer(/* index= */ 0); @@ -1468,7 +1498,6 @@ public void render(long positionUs, long elapsedRealtimeUs) }; } }) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); player.addListener(listener); Renderer videoRenderer = player.getRenderer(/* index= */ 0); @@ -1551,7 +1580,6 @@ public void render(long positionUs, long elapsedRealtimeUs) }; } }) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); player.addListener(listener); Renderer videoRenderer = player.getRenderer(/* index= */ 0); @@ -1608,7 +1636,6 @@ public void render(long positionUs, long elapsedRealtimeUs) .setRenderersFactory( new FakeRenderersFactorySupportingSecondaryVideoRendererThatThrowsOnRender( fakeClock, attemptedRenderWithSecondaryRenderer, shouldSecondaryRendererThrow)) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); player.addListener(listener); Renderer videoRenderer = player.getRenderer(/* index= */ 0); @@ -1694,7 +1721,6 @@ public void render(long positionUs, long elapsedRealtimeUs) }; } }) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); player.addListener(listener); Renderer videoRenderer = player.getRenderer(/* index= */ 0); @@ -1786,7 +1812,6 @@ public void render(long positionUs, long elapsedRealtimeUs) new TestExoPlayerBuilder(context) .setClock(fakeClock) .setRenderersFactory(renderersFactoryWithNoSampleRenderer) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); Renderer secondaryVideoRenderer = player.getSecondaryRenderer(/* index= */ 0); Renderer noSampleRenderer = player.getRenderer(/* index= */ 2); diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/MediaPeriodQueueTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/MediaPeriodQueueTest.java index af3b31d6bbe..1f3423153d2 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/MediaPeriodQueueTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/MediaPeriodQueueTest.java @@ -33,6 +33,7 @@ import static org.robolectric.Shadows.shadowOf; import android.os.Looper; +import android.os.SystemClock; import android.util.Pair; import androidx.annotation.Nullable; import androidx.media3.common.AdPlaybackState; @@ -1981,6 +1982,7 @@ public void resolveMediaPeriodIdForAdsAfterPeriodPositionChange_enforceAdPlaybac /* requestedContentPositionUs= */ C.TIME_UNSET, /* discontinuityStartPositionUs= */ 0, /* totalBufferedDurationUs= */ 0, + SystemClock.elapsedRealtime(), /* trackGroups= */ null, /* trackSelectorResult= */ null, /* staticMetadata= */ ImmutableList.of()); @@ -2016,6 +2018,7 @@ public void resolveMediaPeriodIdForAdsAfterPeriodPositionChange_enforceAdPlaybac /* requestedContentPositionUs= */ C.TIME_UNSET, /* discontinuityStartPositionUs= */ 0, /* totalBufferedDurationUs= */ 0, + SystemClock.elapsedRealtime(), /* trackGroups= */ null, /* trackSelectorResult= */ null, /* staticMetadata= */ ImmutableList.of()); @@ -2050,6 +2053,7 @@ public void resolveMediaPeriodIdForAdsAfterPeriodPositionChange_enforceAdPlaybac /* requestedContentPositionUs= */ C.TIME_UNSET, /* discontinuityStartPositionUs= */ 0, /* totalBufferedDurationUs= */ 0, + SystemClock.elapsedRealtime(), /* trackGroups= */ null, /* trackSelectorResult= */ null, /* staticMetadata= */ ImmutableList.of()); @@ -2087,6 +2091,7 @@ public void resolveMediaPeriodIdForAdsAfterPeriodPositionChange_enforceAdPlaybac /* requestedContentPositionUs= */ C.TIME_UNSET, /* discontinuityStartPositionUs= */ 0, /* totalBufferedDurationUs= */ 0, + SystemClock.elapsedRealtime(), /* trackGroups= */ null, /* trackSelectorResult= */ null, /* staticMetadata= */ ImmutableList.of()); @@ -2144,7 +2149,8 @@ public void resolveMediaPeriodIdForAdsAfterPeriodPositionChange_enforceAdPlaybac /* totalBufferedDurationUs= */ 0, /* positionUs= */ 0, /* positionUpdateTimeMs= */ 0, - /* sleepingForOffload= */ false); + /* sleepingForOffload= */ false, + /* useEstimatedPosition= */ false); MediaPeriodId mediaPeriodId = mediaPeriodQueue.resolveMediaPeriodIdForAdsAfterPeriodPositionChange( @@ -2524,6 +2530,59 @@ public void setPreloadConfiguration_enablePreloading_preloadHolderCreated() { .inOrder(); } + @Test + public void + removeAfter_withEarliestReadingPeriodAheadOfPlayingPeriod_clampsRemovedReadingPeriodToNewTail() { + AnalyticsCollector analyticsCollector = new DefaultAnalyticsCollector(Clock.DEFAULT); + analyticsCollector.setPlayer( + new ExoPlayer.Builder(ApplicationProvider.getApplicationContext()).build(), + Looper.getMainLooper()); + HandlerWrapper handler = + Clock.DEFAULT.createHandler(Looper.getMainLooper(), /* callback= */ null); + mediaPeriodQueue = + new MediaPeriodQueue( + analyticsCollector, + handler, + (info, rendererPositionOffsetUs) -> + new MediaPeriodHolder( + rendererCapabilities, + rendererPositionOffsetUs, + trackSelector, + allocator, + mediaSourceList, + info, + new TrackSelectorResult( + new RendererConfiguration[0], + new ExoTrackSelection[0], + Tracks.EMPTY, + /* info= */ null), + /* targetPreloadBufferDurationUs= */ 5_000_000L, + /* usesStreamPrerollFlags= */ false), + PreloadConfiguration.DEFAULT, + /* rendererCount= */ 2); + setupTimelines(new FakeTimeline(), new FakeTimeline(), new FakeTimeline()); + MediaPeriodHolder period1 = + mediaPeriodQueue.enqueueNextMediaPeriodHolder(getNextMediaPeriodInfo()); + MediaPeriodHolder period2 = + mediaPeriodQueue.enqueueNextMediaPeriodHolder(getNextMediaPeriodInfo()); + MediaPeriodHolder period3 = + mediaPeriodQueue.enqueueNextMediaPeriodHolder(getNextMediaPeriodInfo()); + // Renderer 0 advances to period2; Renderer 1 advances to period3. + assertThat(mediaPeriodQueue.advanceReadingPeriod(/* index= */ 0)).isEqualTo(period2); + assertThat(mediaPeriodQueue.advanceReadingPeriod(/* index= */ 1)).isEqualTo(period2); + assertThat(mediaPeriodQueue.advanceReadingPeriod(/* index= */ 1)).isEqualTo(period3); + assertThat(mediaPeriodQueue.getPlayingPeriod()).isEqualTo(period1); + assertThat(mediaPeriodQueue.getEarliestReadingPeriod()).isEqualTo(period2); + + mediaPeriodQueue.removeAfter(period2); + + assertThat(mediaPeriodQueue.getPlayingPeriod()).isEqualTo(period1); + assertThat(mediaPeriodQueue.getLoadingPeriod()).isEqualTo(period2); + assertThat(mediaPeriodQueue.getEarliestReadingPeriod()).isEqualTo(period2); + assertThat(mediaPeriodQueue.getReadingPeriod(/* index= */ 0)).isEqualTo(period2); + assertThat(mediaPeriodQueue.getReadingPeriod(/* index= */ 1)).isEqualTo(period2); + } + private void setupAdTimeline(long... adGroupTimesUs) { adPlaybackState = new AdPlaybackState(/* adsId= */ new Object(), adGroupTimesUs) @@ -2579,7 +2638,8 @@ private void setupMediaSources(FakeMediaSource... mediaSources) { /* totalBufferedDurationUs= */ 0, /* positionUs= */ 0, /* positionUpdateTimeMs= */ 0, - /* sleepingForOffload= */ false); + /* sleepingForOffload= */ false, + /* useEstimatedPosition= */ false); } private void advance() { @@ -2611,6 +2671,7 @@ private void clear() { /* requestedContentPositionUs= */ C.TIME_UNSET, /* discontinuityStartPositionUs= */ 0, /* totalBufferedDurationUs= */ 0, + SystemClock.elapsedRealtime(), /* trackGroups= */ null, /* trackSelectorResult= */ null, /* staticMetadata= */ ImmutableList.of()); diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/analytics/DefaultAnalyticsCollectorTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/analytics/DefaultAnalyticsCollectorTest.java index 06fd1e7911b..c171ae32bd7 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/analytics/DefaultAnalyticsCollectorTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/analytics/DefaultAnalyticsCollectorTest.java @@ -96,6 +96,7 @@ import androidx.media3.common.util.Clock; import androidx.media3.common.util.ConditionVariable; import androidx.media3.common.util.HandlerWrapper; +import androidx.media3.common.util.Util; import androidx.media3.exoplayer.DecoderCounters; import androidx.media3.exoplayer.DecoderReuseEvaluation; import androidx.media3.exoplayer.ExoPlaybackException; @@ -2079,6 +2080,98 @@ public void onVideoDisabled(EventTime eventTime, DecoderCounters decoderCounters assertThat(releasedEventTime.getValue().realtimeMs).isGreaterThan(videoDisableTimeMs); } + @Test + public void onVolumeChanged_afterStopDuringLiveStreamPrerollAd_reportsEventTimeWithAdTimeline() + throws Exception { + AdPlaybackState adPlaybackState = + FakeTimeline.createAdPlaybackState( + /* adsPerAdGroup= */ 1, + /* adGroupTimesUs...= */ TimelineWindowDefinition + .DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US); + Timeline liveTimelineWithPreroll = + new FakeTimeline( + new TimelineWindowDefinition.Builder() + .setLive(true) + .setAdPlaybackStates(ImmutableList.of(adPlaybackState)) + .build()); + ExoPlayer player = setupPlayer(); + AnalyticsListener analyticsListener = mock(AnalyticsListener.class); + player.addAnalyticsListener(analyticsListener); + player.setMediaSource( + new FakeMediaSource(liveTimelineWithPreroll, ExoPlayerTestRunner.VIDEO_FORMAT)); + player.prepare(); + player.play(); + advance(player).untilState(Player.STATE_READY); + assertThat(player.isPlayingAd()).isTrue(); + + player.stop(); + advance(player).untilPendingCommandsAreFullyHandled(); + player.setVolume(0f); + advance(player).untilPendingCommandsAreFullyHandled(); + player.release(); + + ArgumentCaptor eventTimeCaptor = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(analyticsListener).onVolumeChanged(eventTimeCaptor.capture(), eq(0f)); + AnalyticsListener.EventTime eventTime = eventTimeCaptor.getValue(); + assertThat(eventTime.mediaPeriodId).isNotNull(); + assertThat(eventTime.mediaPeriodId.isAd()).isTrue(); + Timeline.Period period = + eventTime.timeline.getPeriodByUid(eventTime.mediaPeriodId.periodUid, new Timeline.Period()); + assertThat(period.getAdGroupCount()).isEqualTo(1); + assertThat(period.getAdGroupTimeUs(eventTime.mediaPeriodId.adGroupIndex)) + .isEqualTo(TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US); + } + + @Test + public void + onVolumeChanged_afterStopDuringLiveStreamContentBeforeMidrollAd_reportsEventTimeWithAdTimeline() + throws Exception { + long midrollAdGroupTimeUs = + TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US + 5 * C.MICROS_PER_SECOND; + AdPlaybackState adPlaybackState = + FakeTimeline.createAdPlaybackState( + /* adsPerAdGroup= */ 1, /* adGroupTimesUs...= */ midrollAdGroupTimeUs); + Timeline liveTimelineWithMidroll = + new FakeTimeline( + new TimelineWindowDefinition.Builder() + .setLive(true) + .setAdPlaybackStates(ImmutableList.of(adPlaybackState)) + .build()); + ExoPlayer player = setupPlayer(); + AnalyticsListener analyticsListener = mock(AnalyticsListener.class); + player.addAnalyticsListener(analyticsListener); + player.setMediaSource( + new FakeMediaSource(liveTimelineWithMidroll, ExoPlayerTestRunner.VIDEO_FORMAT)); + player.prepare(); + player.play(); + advance(player).untilState(Player.STATE_READY); + assertThat(player.isPlayingAd()).isFalse(); + + player.stop(); + advance(player).untilPendingCommandsAreFullyHandled(); + player.setVolume(0f); + advance(player).untilPendingCommandsAreFullyHandled(); + player.release(); + + ArgumentCaptor eventTimeCaptor = + ArgumentCaptor.forClass(AnalyticsListener.EventTime.class); + verify(analyticsListener).onVolumeChanged(eventTimeCaptor.capture(), eq(0f)); + // Verify that the volume changed event was reported for a content period pointing to the + // upcoming ad group. + AnalyticsListener.EventTime eventTime = eventTimeCaptor.getValue(); + assertThat(eventTime.mediaPeriodId).isNotNull(); + assertThat(eventTime.mediaPeriodId.isAd()).isFalse(); + assertThat(eventTime.mediaPeriodId.nextAdGroupIndex).isEqualTo(0); + assertThat(eventTime.eventPlaybackPositionMs).isLessThan(Util.usToMs(midrollAdGroupTimeUs)); + // Verify that the ad group time matches the original timeline. + Timeline.Period period = + eventTime.timeline.getPeriodByUid(eventTime.mediaPeriodId.periodUid, new Timeline.Period()); + assertThat(period.getAdGroupCount()).isEqualTo(1); + assertThat(period.getAdGroupTimeUs(eventTime.mediaPeriodId.nextAdGroupIndex)) + .isEqualTo(midrollAdGroupTimeUs); + } + private void populateEventIds(Timeline timeline) { period0 = new EventWindowAndPeriodId( diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/analytics/MediaMetricsListenerTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/analytics/MediaMetricsListenerTest.java new file mode 100644 index 00000000000..fc5c22d3f64 --- /dev/null +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/analytics/MediaMetricsListenerTest.java @@ -0,0 +1,74 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.exoplayer.analytics; + +import static com.google.common.truth.Truth.assertThat; + +import android.media.metrics.PlaybackMetrics; +import androidx.media3.common.Flags; +import androidx.media3.test.utils.Media3FlagsRule; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import org.junit.Rule; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.annotation.Config; + +/** Unit tests for {@link MediaMetricsListener}. */ +@RunWith(AndroidJUnit4.class) +@Config(minSdk = 31) +public final class MediaMetricsListenerTest { + + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); + + private static final @Flags.Flag int TEST_FLAG_1 = 1_000_000; + private static final @Flags.Flag int TEST_FLAG_2 = 1_000_001; + + @Test + public void addFlagExperimentIds_defaultCanaryMode_addsNoExperimentIds() { + PlaybackMetrics.Builder builder = new PlaybackMetrics.Builder(); + + MediaMetricsListener.addFlagExperimentIds(builder); + + PlaybackMetrics metrics = builder.build(); + assertThat(metrics.getExperimentIds()).isEmpty(); + } + + @Test + public void addFlagExperimentIds_canaryModeDisabled_addsCanaryModeDisabledExperimentId() { + Flags.setCanaryModeEnabled(false); + PlaybackMetrics.Builder builder = new PlaybackMetrics.Builder(); + + MediaMetricsListener.addFlagExperimentIds(builder); + + PlaybackMetrics metrics = builder.build(); + assertThat(metrics.getExperimentIds()).asList().containsExactly(7_000_000_000_000_000_000L); + } + + @Test + public void addFlagExperimentIds_withFlagOverrides_addsCorrectExperimentIds() { + Flags.enableFlag(TEST_FLAG_1); + Flags.disableFlag(TEST_FLAG_2); + PlaybackMetrics.Builder builder = new PlaybackMetrics.Builder(); + + MediaMetricsListener.addFlagExperimentIds(builder); + + PlaybackMetrics metrics = builder.build(); + assertThat(metrics.getExperimentIds()) + .asList() + .containsExactly( + 7_001_000_000_000_000_000L + TEST_FLAG_1, 7_002_000_000_000_000_000L + TEST_FLAG_2); + } +} diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/AudioCapabilitiesReceiverWrongThreadTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/AudioCapabilitiesReceiverWrongThreadTest.java new file mode 100644 index 00000000000..7db6479505b --- /dev/null +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/AudioCapabilitiesReceiverWrongThreadTest.java @@ -0,0 +1,146 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.exoplayer.audio; + +import static com.google.common.truth.Truth.assertThat; +import static org.robolectric.Shadows.shadowOf; + +import android.media.AudioDeviceInfo; +import android.os.Looper; +import androidx.media3.common.AudioAttributes; +import androidx.media3.common.C; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.Sets; +import java.util.Set; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.atomic.AtomicInteger; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.annotation.Config; + +/** Tests that raw OEM audio-device callbacks are serialized on the receiver handler. */ +@RunWith(AndroidJUnit4.class) +public final class AudioCapabilitiesReceiverWrongThreadTest { + + private static final AudioCapabilities OVERRIDDEN_AUDIO_CAPABILITIES = + new AudioCapabilities( + new int[] {C.ENCODING_AC3}, + /* maxChannelCount= */ 6, + /* speakerLayoutChannelMasks= */ ImmutableList.of(), + /* spatializerChannelMasks= */ ImmutableList.of()); + + private ExecutorService wrongThreadExecutor; + + @Before + public void setUp() { + wrongThreadExecutor = Executors.newSingleThreadExecutor(); + } + + @After + public void tearDown() { + wrongThreadExecutor.shutdownNow(); + } + + @Test + @Config(maxSdk = 37) // The workaround isn't needed from API 38+ + public void onAudioDevicesAdded_fromBackgroundThreadWithoutLooper_notifiesListenerOnLooperThread() + throws Exception { + TrackingListener listener = new TrackingListener(); + AudioCapabilitiesReceiver audioCapabilitiesReceiver = + new AudioCapabilitiesReceiver( + ApplicationProvider.getApplicationContext(), + listener, + new AudioAttributes.Builder().setContentType(C.AUDIO_CONTENT_TYPE_MUSIC).build(), + /* routedDevice= */ null); + AudioCapabilities unused = audioCapabilitiesReceiver.register(); + audioCapabilitiesReceiver.overrideCapabilities(OVERRIDDEN_AUDIO_CAPABILITIES); + + wrongThreadExecutor + .submit( + () -> + audioCapabilitiesReceiver.audioDeviceCallback.onAudioDevicesAdded( + new AudioDeviceInfo[0])) + .get(); + shadowOf(Looper.getMainLooper()).idle(); + + assertThat(listener.invocationCount.get()).isEqualTo(3); + assertThat(listener.looperlessThreads).isEmpty(); + } + + @Test + @Config(maxSdk = 37) // The workaround isn't needed from API 38+ + public void + onAudioDevicesRemoved_fromBackgroundThreadWithoutLooper_notifiesListenerOnLooperThread() + throws Exception { + TrackingListener listener = new TrackingListener(); + AudioCapabilitiesReceiver audioCapabilitiesReceiver = + new AudioCapabilitiesReceiver( + ApplicationProvider.getApplicationContext(), + listener, + new AudioAttributes.Builder().setContentType(C.AUDIO_CONTENT_TYPE_MUSIC).build(), + /* routedDevice= */ null); + AudioCapabilities unused = audioCapabilitiesReceiver.register(); + audioCapabilitiesReceiver.overrideCapabilities(OVERRIDDEN_AUDIO_CAPABILITIES); + + wrongThreadExecutor + .submit( + () -> + audioCapabilitiesReceiver.audioDeviceCallback.onAudioDevicesRemoved( + new AudioDeviceInfo[0])) + .get(); + shadowOf(Looper.getMainLooper()).idle(); + + assertThat(listener.invocationCount.get()).isEqualTo(3); + assertThat(listener.looperlessThreads).isEmpty(); + } + + @Test + public void onAudioDevicesAdded_onCorrectLooperThread_doesntRePost() throws Exception { + TrackingListener listener = new TrackingListener(); + AudioCapabilitiesReceiver audioCapabilitiesReceiver = + new AudioCapabilitiesReceiver( + ApplicationProvider.getApplicationContext(), + listener, + new AudioAttributes.Builder().setContentType(C.AUDIO_CONTENT_TYPE_MUSIC).build(), + /* routedDevice= */ null); + AudioCapabilities unused = audioCapabilitiesReceiver.register(); + audioCapabilitiesReceiver.overrideCapabilities(OVERRIDDEN_AUDIO_CAPABILITIES); + + audioCapabilitiesReceiver.audioDeviceCallback.onAudioDevicesAdded(new AudioDeviceInfo[0]); + + // Assert the listener is invoked immediately without needing to idle the main looper. + assertThat(listener.invocationCount.get()).isEqualTo(3); + assertThat(listener.looperlessThreads).isEmpty(); + } + + private static final class TrackingListener implements AudioCapabilitiesReceiver.Listener { + private final Set looperlessThreads = Sets.newConcurrentHashSet(); + private final AtomicInteger invocationCount = new AtomicInteger(); + + @Override + public void onAudioCapabilitiesChanged(AudioCapabilities audioCapabilities) { + invocationCount.incrementAndGet(); + if (Looper.myLooper() == null) { + looperlessThreads.add(Thread.currentThread()); + } + } + } +} diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/AudioTrackAudioOutputTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/AudioTrackAudioOutputTest.java index 9b77e26e277..e6be5278dcc 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/AudioTrackAudioOutputTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/AudioTrackAudioOutputTest.java @@ -19,22 +19,32 @@ import static com.google.common.truth.Truth.assertThat; import android.media.AudioFormat; +import android.media.AudioManager; import android.media.AudioTrack; import androidx.annotation.Nullable; import androidx.media3.common.AudioAttributes; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.exoplayer.audio.AudioOutputProvider.OutputConfig; import androidx.media3.test.utils.FakeClock; +import androidx.media3.test.utils.Media3FlagsRule; +import androidx.media3.test.utils.robolectric.RobolectricUtil; import androidx.test.ext.junit.runners.AndroidJUnit4; import java.nio.ByteBuffer; import java.nio.ByteOrder; +import java.util.concurrent.atomic.AtomicBoolean; +import org.junit.Rule; import org.junit.Test; import org.junit.runner.RunWith; +import org.robolectric.annotation.Config; +import org.robolectric.shadows.ShadowAudioSystem; /** Unit tests for {@link AudioTrackAudioOutput}. */ @RunWith(AndroidJUnit4.class) public final class AudioTrackAudioOutputTest { + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); + private AudioTrack audioTrack; private AudioTrackAudioOutput audioTrackAudioOutput; private final FakeClock clock = @@ -73,6 +83,122 @@ public void write_withPcmData_positionAdvances() throws Exception { assertThat(audioTrackAudioOutput.getPositionUs()).isEqualTo(2_000_000L); } + @Test + @Config(minSdk = 29) + public void release_fromApi29WithSkipFlushFlagEnabled_skipsFlushBeforeRelease() throws Exception { + Flags.enableFlag(Flags.FLAG_SKIP_AUDIO_TRACK_FLUSH_BEFORE_RELEASE); + initializeAudioTrackAudioOutput(); + audioTrackAudioOutput.play(); + boolean unused = + audioTrackAudioOutput.write( + createByteBuffer(ONE_SECOND_BUFFER), + /* encodedAccessUnitCount= */ 1, + /* presentationTimeUs= */ 0); + assertThat(audioTrack.getPlaybackHeadPosition()).isEqualTo(44100); + + releaseAndAwaitCompletion(audioTrackAudioOutput); + + assertThat(audioTrack.getState()).isEqualTo(AudioTrack.STATE_UNINITIALIZED); + assertThat(audioTrack.getPlaybackHeadPosition()).isEqualTo(44100); + } + + @Test + @Config(minSdk = 29) + public void release_fromApi29WithSkipFlushFlagDisabled_flushesBeforeRelease() throws Exception { + Flags.disableFlag(Flags.FLAG_SKIP_AUDIO_TRACK_FLUSH_BEFORE_RELEASE); + initializeAudioTrackAudioOutput(); + audioTrackAudioOutput.play(); + boolean unused = + audioTrackAudioOutput.write( + createByteBuffer(ONE_SECOND_BUFFER), + /* encodedAccessUnitCount= */ 1, + /* presentationTimeUs= */ 0); + assertThat(audioTrack.getPlaybackHeadPosition()).isEqualTo(44100); + + releaseAndAwaitCompletion(audioTrackAudioOutput); + + assertThat(audioTrack.getState()).isEqualTo(AudioTrack.STATE_UNINITIALIZED); + assertThat(audioTrack.getPlaybackHeadPosition()).isEqualTo(0); + } + + @Test + @Config(maxSdk = 28) + public void release_upToApi28WithSkipFlushFlagEnabled_flushesBeforeRelease() throws Exception { + Flags.enableFlag(Flags.FLAG_SKIP_AUDIO_TRACK_FLUSH_BEFORE_RELEASE); + initializeAudioTrackAudioOutput(); + audioTrackAudioOutput.play(); + boolean unused = + audioTrackAudioOutput.write( + createByteBuffer(ONE_SECOND_BUFFER), + /* encodedAccessUnitCount= */ 1, + /* presentationTimeUs= */ 0); + assertThat(audioTrack.getPlaybackHeadPosition()).isEqualTo(44100); + + releaseAndAwaitCompletion(audioTrackAudioOutput); + + assertThat(audioTrack.getState()).isEqualTo(AudioTrack.STATE_UNINITIALIZED); + assertThat(audioTrack.getPlaybackHeadPosition()).isEqualTo(0); + } + + @Test + @Config(minSdk = 29) + public void release_offloadedPlaybackWithSkipFlushFlagDisabled_skipsFlushBeforeRelease() + throws Exception { + Flags.disableFlag(Flags.FLAG_SKIP_AUDIO_TRACK_FLUSH_BEFORE_RELEASE); + android.media.AudioAttributes platformAttributes = + new android.media.AudioAttributes.Builder() + .setContentType(android.media.AudioAttributes.CONTENT_TYPE_MUSIC) + .setUsage(android.media.AudioAttributes.USAGE_MEDIA) + .build(); + AudioFormat platformFormat = + new AudioFormat.Builder() + .setEncoding(AudioFormat.ENCODING_AAC_LC) + .setSampleRate(44100) + .setChannelMask(AudioFormat.CHANNEL_OUT_STEREO) + .build(); + ShadowAudioSystem.setOffloadSupported( + platformFormat, platformAttributes, /* supported= */ true); + ShadowAudioSystem.setOffloadPlaybackSupport( + platformFormat, platformAttributes, AudioManager.PLAYBACK_OFFLOAD_SUPPORTED); + ShadowAudioSystem.setDirectPlaybackSupport( + platformFormat, platformAttributes, AudioManager.DIRECT_PLAYBACK_OFFLOAD_SUPPORTED); + audioTrack = + new AudioTrack.Builder() + .setAudioAttributes(platformAttributes) + .setAudioFormat(platformFormat) + .setBufferSizeInBytes(1024) + .setOffloadedPlayback(true) + .build(); + audioTrackAudioOutput = + new AudioTrackAudioOutput( + audioTrack, + new OutputConfig.Builder() + .setEncoding(C.ENCODING_AAC_LC) + .setSampleRate(44100) + .setChannelMask(AudioFormat.CHANNEL_OUT_STEREO) + .setBufferSize(1024) + .setIsOffload(true) + .setAudioAttributes( + new AudioAttributes.Builder() + .setContentType(C.AUDIO_CONTENT_TYPE_MUSIC) + .setUsage(C.USAGE_MEDIA) + .build()) + .build(), + /* capabilityChangeListener= */ null, + MAX_PLAYBACK_SPEED, + /* clock= */ clock); + audioTrackAudioOutput.play(); + boolean unused = + audioTrackAudioOutput.write( + createByteBuffer(256), /* encodedAccessUnitCount= */ 1, /* presentationTimeUs= */ 0); + assertThat(audioTrack.getPlaybackHeadPosition()).isEqualTo(256); + + releaseAndAwaitCompletion(audioTrackAudioOutput); + + assertThat(audioTrack.getState()).isEqualTo(AudioTrack.STATE_UNINITIALIZED); + assertThat(audioTrack.getPlaybackHeadPosition()).isEqualTo(256); + } + private void initializeAudioTrackAudioOutput() { initializeAudioTrackAudioOutput( /* listener= */ null, @@ -122,6 +248,31 @@ private void initializeAudioTrackAudioOutput( /* clock= */ clock); } + private static void releaseAndAwaitCompletion(AudioTrackAudioOutput output) throws Exception { + AtomicBoolean released = new AtomicBoolean(); + output.addListener( + new AudioOutput.Listener() { + @Override + public void onPositionAdvancing(long playoutStartSystemTimeMs) {} + + @Override + public void onOffloadDataRequest() {} + + @Override + public void onOffloadPresentationEnded() {} + + @Override + public void onUnderrun() {} + + @Override + public void onReleased() { + released.set(true); + } + }); + output.release(); + RobolectricUtil.runMainLooperUntil(released::get); + } + private static ByteBuffer createByteBuffer(int size) { return ByteBuffer.allocateDirect(size).order(ByteOrder.nativeOrder()); } diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/AudioTrackPositionTrackerTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/AudioTrackPositionTrackerTest.java index 24bc38163a2..c38115d154e 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/AudioTrackPositionTrackerTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/AudioTrackPositionTrackerTest.java @@ -22,6 +22,7 @@ import static org.mockito.Mockito.never; import static org.mockito.Mockito.times; import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; import static org.robolectric.Shadows.shadowOf; import android.media.AudioFormat; @@ -64,8 +65,7 @@ public class AudioTrackPositionTrackerTest { private final AudioTrack audioTrack = createDefaultAudioTrack(); @Test - public void - getCurrentPositionUs_withoutExpectRawPlaybackHeadReset_returnsPositionWithWrappedValue() { + public void getCurrentPositionUs_withoutExpectRawPlaybackHeadReset_doesNotReturnWrappedValue() { AudioTrackPositionTracker audioTrackPositionTracker = new AudioTrackPositionTracker( mock(AudioTrackPositionTracker.Listener.class), @@ -76,18 +76,18 @@ public class AudioTrackPositionTrackerTest { MIN_BUFFER_SIZE); audioTrackPositionTracker.start(); audioTrack.play(); - // Advance and write to audio track at least twice to move rawHeadPosition past wrap point. + // Advance and write to audio track at least twice. for (int i = 0; i < 2; i++) { writeBytesAndAdvanceTime(audioTrack); long unused = audioTrackPositionTracker.getCurrentPositionUs(LARGE_WRITTEN_FRAMES); } - // Reset audio track and write bytes to simulate position overflow. + // Reset audio track and write bytes without expecting playback head reset. audioTrack.flush(); writeBytesAndAdvanceTime(audioTrack); assertThat(audioTrackPositionTracker.getCurrentPositionUs(LARGE_WRITTEN_FRAMES)) - .isGreaterThan(4294967296L); + .isLessThan(Util.sampleCountToDurationUs(1L << 32, SAMPLE_RATE)); } @Test @@ -191,8 +191,13 @@ public void getCurrentPositionUs_withAudioTrackFlushAndResettingTracker_returnsC } @Test - public void getCurrentPositionUs_withoutAudioTrackResetsExpectPosition_returnsWrappedValue() { - // Set tracker to expect playback head reset to simulate expected track transition. + public void getCurrentPositionUs_withPlaybackHeadPositionWrapAround_returnsWrappedValue() { + AudioTrack audioTrack = mock(AudioTrack.class); + when(audioTrack.getSampleRate()).thenReturn(SAMPLE_RATE); + when(audioTrack.getPlayState()).thenReturn(AudioTrack.PLAYSTATE_PLAYING); + when(audioTrack.getPlaybackHeadPosition()) + .thenReturn(-1) // 0xFFFFFFFFL (4,294,967,295 frames) + .thenReturn(SAMPLE_RATE); // 44100 frames (wrapped around past 2^32) AudioTrackPositionTracker audioTrackPositionTracker = new AudioTrackPositionTracker( mock(AudioTrackPositionTracker.Listener.class), @@ -202,19 +207,102 @@ public void getCurrentPositionUs_withoutAudioTrackResetsExpectPosition_returnsWr OUTPUT_PCM_FRAME_SIZE, MIN_BUFFER_SIZE); audioTrackPositionTracker.start(); - audioTrack.play(); - // Advance and write to audio track at least twice to move rawHeadPosition past wrap point. - for (int i = 0; i < 2; i++) { - writeBytesAndAdvanceTime(audioTrack); - long unused = audioTrackPositionTracker.getCurrentPositionUs(LARGE_WRITTEN_FRAMES); - } - // Reset audio track and write bytes to simulate position overflow. - audioTrack.flush(); - writeBytesAndAdvanceTime(audioTrack); + long writtenFrames = 5_000_000_000L; + // Advance and sample to establish rawHeadPosition near 2^32. + long unused = audioTrackPositionTracker.getCurrentPositionUs(writtenFrames); + clock.advanceTime(TIME_TO_ADVANCE_MS); + long positionUs = audioTrackPositionTracker.getCurrentPositionUs(writtenFrames); - assertThat(audioTrackPositionTracker.getCurrentPositionUs(LARGE_WRITTEN_FRAMES)) - .isGreaterThan(4294967296L); + assertThat(positionUs).isGreaterThan(Util.sampleCountToDurationUs(1L << 32, SAMPLE_RATE)); + } + + @Test + public void getCurrentPositionUs_withDecreasingPlaybackHeadPosition_doesNotReturnWrappedValue() { + AudioTrack audioTrack = mock(AudioTrack.class); + when(audioTrack.getSampleRate()).thenReturn(SAMPLE_RATE); + when(audioTrack.getPlayState()).thenReturn(AudioTrack.PLAYSTATE_PLAYING); + when(audioTrack.getPlaybackHeadPosition()).thenReturn(SAMPLE_RATE * 2).thenReturn(SAMPLE_RATE); + AudioTrackPositionTracker audioTrackPositionTracker = + new AudioTrackPositionTracker( + mock(AudioTrackPositionTracker.Listener.class), + clock, + audioTrack, + C.ENCODING_PCM_16BIT, + OUTPUT_PCM_FRAME_SIZE, + MIN_BUFFER_SIZE); + audioTrackPositionTracker.start(); + + long unused = audioTrackPositionTracker.getCurrentPositionUs(LARGE_WRITTEN_FRAMES); + clock.advanceTime(TIME_TO_ADVANCE_MS); + long positionUs = audioTrackPositionTracker.getCurrentPositionUs(LARGE_WRITTEN_FRAMES); + + assertThat(positionUs).isLessThan(Util.sampleCountToDurationUs(1L << 32, SAMPLE_RATE)); + } + + @Test + public void + getCurrentPositionUs_withResidualPlaybackHeadPositionResettingToZero_doesNotReturnWrappedValue() { + AudioTrack audioTrack = mock(AudioTrack.class); + when(audioTrack.getSampleRate()).thenReturn(SAMPLE_RATE); + when(audioTrack.getPlayState()).thenReturn(AudioTrack.PLAYSTATE_PLAYING); + // Simulate initial residual head position from a prior track on direct/passthrough HAL stream, + // which then resets to 0. + when(audioTrack.getPlaybackHeadPosition()).thenReturn(24_576).thenReturn(0); + AudioTrackPositionTracker audioTrackPositionTracker = + new AudioTrackPositionTracker( + mock(AudioTrackPositionTracker.Listener.class), + clock, + audioTrack, + C.ENCODING_PCM_16BIT, + OUTPUT_PCM_FRAME_SIZE, + MIN_BUFFER_SIZE); + audioTrackPositionTracker.start(); + + long unused = audioTrackPositionTracker.getCurrentPositionUs(LARGE_WRITTEN_FRAMES); + clock.advanceTime(TIME_TO_ADVANCE_MS); + long positionUs = audioTrackPositionTracker.getCurrentPositionUs(LARGE_WRITTEN_FRAMES); + + assertThat(positionUs).isLessThan(Util.sampleCountToDurationUs(1L << 32, SAMPLE_RATE)); + // Verify that the position is within normal elapsed time (< 5s), not wrapped past 2^32 frames + // (~27 hours) or clamped to LARGE_WRITTEN_FRAMES (~6.3 hours). + assertThat(positionUs).isLessThan(Util.msToUs(5000)); + } + + @Test + public void + getCurrentPositionUs_withUnexpectedDecreasingPlaybackHeadPosition_clearsStaleOffsets() { + AudioTrack audioTrack = mock(AudioTrack.class); + when(audioTrack.getSampleRate()).thenReturn(SAMPLE_RATE); + when(audioTrack.getPlayState()).thenReturn(AudioTrack.PLAYSTATE_PLAYING); + when(audioTrack.getPlaybackHeadPosition()) + .thenReturn(SAMPLE_RATE * 2) + .thenReturn(SAMPLE_RATE) + .thenReturn(SAMPLE_RATE + (SAMPLE_RATE / 2)); + AudioTrackPositionTracker audioTrackPositionTracker = + new AudioTrackPositionTracker( + mock(AudioTrackPositionTracker.Listener.class), + clock, + audioTrack, + C.ENCODING_PCM_16BIT, + OUTPUT_PCM_FRAME_SIZE, + MIN_BUFFER_SIZE); + audioTrackPositionTracker.start(); + + // Sample initial head position to populate playhead offset. + long initialPositionUs = audioTrackPositionTracker.getCurrentPositionUs(LARGE_WRITTEN_FRAMES); + assertThat(initialPositionUs).isEqualTo(2_000_000L); + + clock.advanceTime(TIME_TO_ADVANCE_MS); + // Head position unexpectedly drops to SAMPLE_RATE (1 second). Tracker should reset sync params + // and report 1 second instead of averaging with or extrapolating from the stale playhead offset + // (which would report 2 seconds). + long positionUs = audioTrackPositionTracker.getCurrentPositionUs(LARGE_WRITTEN_FRAMES); + assertThat(positionUs).isEqualTo(1_000_000L); + + clock.advanceTime(500L); + long postDropPositionUs = audioTrackPositionTracker.getCurrentPositionUs(LARGE_WRITTEN_FRAMES); + assertThat(postDropPositionUs).isEqualTo(1_500_000L); } @Test diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/DefaultAudioSinkTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/DefaultAudioSinkTest.java index d2e9a42517f..fb6e5bdb05f 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/DefaultAudioSinkTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/DefaultAudioSinkTest.java @@ -63,6 +63,7 @@ import java.util.concurrent.TimeUnit; import java.util.concurrent.TimeoutException; import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicLong; import java.util.concurrent.atomic.AtomicReference; import org.junit.After; import org.junit.Assert; @@ -1530,6 +1531,301 @@ public void setPlaybackParameters(PlaybackParameters playbackParams) { assertThat(actualPlaybackParameters.get()).isEqualTo(expectedMaxSpeedParameters); } + @Config(minSdk = 30) + @Test + public void hasPendingData_withPartialOffloadEncodedFrameWrite_returnsTrue() throws Exception { + Context context = ApplicationProvider.getApplicationContext(); + // Create a custom AudioOutputProvider that simulates a partial write of the compressed content. + AudioOutputProvider audioOutputProvider = + new ForwardingAudioOutputProvider( + new AudioTrackAudioOutputProvider.Builder(context).build()) { + @Override + public AudioOutput getAudioOutput(OutputConfig config) throws InitializationException { + return new ForwardingAudioOutput(super.getAudioOutput(config)) { + @Override + public boolean write( + ByteBuffer buffer, int encodedAccessUnitCount, long presentationTimeUs) + throws WriteException { + // Mock a partial write of 1024 bytes. + int bytesWritten = Math.min(buffer.remaining(), 1024); + buffer.position(buffer.position() + bytesWritten); + return false; + } + }; + } + }; + defaultAudioSink = + new DefaultAudioSink.Builder(context).setAudioOutputProvider(audioOutputProvider).build(); + AudioFormat audioFormat = + new AudioFormat.Builder() + .setSampleRate(SAMPLE_RATE_44_1) + .setChannelMask(AudioFormat.CHANNEL_OUT_STEREO) + .setEncoding(AudioFormat.ENCODING_AAC_LC) + .build(); + AudioAttributes audioAttributes = + new AudioAttributes.Builder() + .setContentType(C.AUDIO_CONTENT_TYPE_UNKNOWN) + .setUsage(C.USAGE_MEDIA) + .setAllowedCapturePolicy(C.ALLOW_CAPTURE_BY_ALL) + .build(); + ShadowAudioSystem.setOffloadSupported( + audioFormat, audioAttributes.getPlatformAudioAttributes(), true); + ShadowAudioSystem.setOffloadPlaybackSupport( + audioFormat, + audioAttributes.getPlatformAudioAttributes(), + AudioManager.PLAYBACK_OFFLOAD_SUPPORTED); + ShadowAudioSystem.setDirectPlaybackSupport( + audioFormat, + audioAttributes.getPlatformAudioAttributes(), + AudioManager.DIRECT_PLAYBACK_OFFLOAD_SUPPORTED); + configureDefaultAudioSinkWithOffload(); + + ByteBuffer largeBuffer = ByteBuffer.allocateDirect(11 * 1024).order(ByteOrder.nativeOrder()); + boolean handled = + defaultAudioSink.handleBuffer( + largeBuffer, /* presentationTimeUs= */ 0, /* encodedAccessUnitCount= */ 32); + assertThat(handled).isFalse(); + + assertThat(defaultAudioSink.hasPendingData()).isTrue(); + } + + @Config(minSdk = 30) + @Test + public void hasPendingData_withZeroBytesOffloadEncodedFrameWritten_returnsFalse() + throws Exception { + Context context = ApplicationProvider.getApplicationContext(); + // Create a custom AudioOutputProvider that blocks all writes. + AudioOutputProvider audioOutputProvider = + new ForwardingAudioOutputProvider( + new AudioTrackAudioOutputProvider.Builder(context).build()) { + @Override + public AudioOutput getAudioOutput(OutputConfig config) throws InitializationException { + return new ForwardingAudioOutput(super.getAudioOutput(config)) { + @Override + public boolean write( + ByteBuffer buffer, int encodedAccessUnitCount, long presentationTimeUs) + throws WriteException { + return false; + } + }; + } + }; + defaultAudioSink = + new DefaultAudioSink.Builder(context).setAudioOutputProvider(audioOutputProvider).build(); + AudioFormat audioFormat = + new AudioFormat.Builder() + .setSampleRate(SAMPLE_RATE_44_1) + .setChannelMask(AudioFormat.CHANNEL_OUT_STEREO) + .setEncoding(AudioFormat.ENCODING_AAC_LC) + .build(); + AudioAttributes audioAttributes = + new AudioAttributes.Builder() + .setContentType(C.AUDIO_CONTENT_TYPE_UNKNOWN) + .setUsage(C.USAGE_MEDIA) + .setAllowedCapturePolicy(C.ALLOW_CAPTURE_BY_ALL) + .build(); + ShadowAudioSystem.setOffloadSupported( + audioFormat, audioAttributes.getPlatformAudioAttributes(), true); + ShadowAudioSystem.setOffloadPlaybackSupport( + audioFormat, + audioAttributes.getPlatformAudioAttributes(), + AudioManager.PLAYBACK_OFFLOAD_SUPPORTED); + ShadowAudioSystem.setDirectPlaybackSupport( + audioFormat, + audioAttributes.getPlatformAudioAttributes(), + AudioManager.DIRECT_PLAYBACK_OFFLOAD_SUPPORTED); + configureDefaultAudioSinkWithOffload(); + + ByteBuffer largeBuffer = ByteBuffer.allocateDirect(11 * 1024).order(ByteOrder.nativeOrder()); + boolean handled = + defaultAudioSink.handleBuffer( + largeBuffer, /* presentationTimeUs= */ 0, /* encodedAccessUnitCount= */ 32); + assertThat(handled).isFalse(); + + assertThat(defaultAudioSink.hasPendingData()).isFalse(); + } + + @Test + public void hasPendingData_withUnwrittenPendingOutputBuffer_returnsFalse() throws Exception { + Context context = ApplicationProvider.getApplicationContext(); + AudioOutputProvider audioOutputProvider = + new ForwardingAudioOutputProvider( + new AudioTrackAudioOutputProvider.Builder(context).build()) { + @Override + public AudioOutput getAudioOutput(OutputConfig config) throws InitializationException { + return new ForwardingAudioOutput(super.getAudioOutput(config)) { + @Override + public boolean write( + ByteBuffer buffer, int encodedAccessUnitCount, long presentationTimeUs) + throws WriteException { + return false; + } + }; + } + }; + defaultAudioSink = + new DefaultAudioSink.Builder(context).setAudioOutputProvider(audioOutputProvider).build(); + configureDefaultAudioSink(CHANNEL_COUNT_STEREO); + + boolean handledFirstBuffer = + defaultAudioSink.handleBuffer( + create1Sec44100HzSilenceBuffer(), + /* presentationTimeUs= */ 0, + /* encodedAccessUnitCount= */ 1); + boolean handledSecondBuffer = + defaultAudioSink.handleBuffer( + create1Sec44100HzSilenceBuffer(), + /* presentationTimeUs= */ 1_000_000, + /* encodedAccessUnitCount= */ 1); + + assertThat(handledFirstBuffer).isTrue(); + assertThat(handledSecondBuffer).isFalse(); + assertThat(defaultAudioSink.hasPendingData()).isFalse(); + } + + @Test + public void hasPendingData_afterWrittenFramesPlayedWithUnwrittenPendingOutputBuffer_returnsFalse() + throws Exception { + Context context = ApplicationProvider.getApplicationContext(); + AtomicBoolean blockWrites = new AtomicBoolean(false); + AtomicLong positionUs = new AtomicLong(0); + AudioOutputProvider audioOutputProvider = + new ForwardingAudioOutputProvider( + new AudioTrackAudioOutputProvider.Builder(context).build()) { + @Override + public AudioOutput getAudioOutput(OutputConfig config) throws InitializationException { + return new ForwardingAudioOutput(super.getAudioOutput(config)) { + @Override + public boolean write( + ByteBuffer buffer, int encodedAccessUnitCount, long presentationTimeUs) + throws WriteException { + if (blockWrites.get()) { + return false; + } + buffer.position(buffer.limit()); + return true; + } + + @Override + public long getPositionUs() { + return positionUs.get(); + } + }; + } + }; + defaultAudioSink = + new DefaultAudioSink.Builder(context) + .setAudioOutputProvider(audioOutputProvider) + .setAudioProcessors(new AudioProcessor[] {new TeeAudioProcessor(arrayAudioBufferSink)}) + .build(); + configureDefaultAudioSink(CHANNEL_COUNT_STEREO); + boolean handledFirstBuffer = + defaultAudioSink.handleBuffer( + create1Sec44100HzSilenceBuffer(), + /* presentationTimeUs= */ 0, + /* encodedAccessUnitCount= */ 1); + assertThat(handledFirstBuffer).isTrue(); + assertThat(defaultAudioSink.hasPendingData()).isTrue(); + blockWrites.set(true); + + boolean handledSecondBuffer = + defaultAudioSink.handleBuffer( + create1Sec44100HzSilenceBuffer(), + /* presentationTimeUs= */ 1_000_000, + /* encodedAccessUnitCount= */ 1); + positionUs.set(1_000_000); + + assertThat(handledSecondBuffer).isTrue(); + assertThat(defaultAudioSink.hasPendingData()).isFalse(); + } + + @Config(minSdk = 30) + @Test + public void hasPendingData_withGaplessPartialOffloadEncodedFrameWrite_returnsTrue() + throws Exception { + Context context = ApplicationProvider.getApplicationContext(); + AtomicBoolean simulatePartialWrite = new AtomicBoolean(false); + AtomicLong mockPositionUs = new AtomicLong(0); + AudioOutputProvider audioOutputProvider = + new ForwardingAudioOutputProvider( + new AudioTrackAudioOutputProvider.Builder(context).build()) { + @Override + public AudioOutput getAudioOutput(OutputConfig config) throws InitializationException { + return new ForwardingAudioOutput(super.getAudioOutput(config)) { + @Override + public boolean write( + ByteBuffer buffer, int encodedAccessUnitCount, long presentationTimeUs) + throws WriteException { + if (simulatePartialWrite.get()) { + int bytesWritten = Math.min(buffer.remaining(), 1024); + buffer.position(buffer.position() + bytesWritten); + return false; + } + buffer.position(buffer.limit()); + return true; + } + + @Override + public long getPositionUs() { + return mockPositionUs.get(); + } + }; + } + }; + defaultAudioSink = + new DefaultAudioSink.Builder(context).setAudioOutputProvider(audioOutputProvider).build(); + AudioFormat audioFormat = + new AudioFormat.Builder() + .setSampleRate(SAMPLE_RATE_44_1) + .setChannelMask(AudioFormat.CHANNEL_OUT_STEREO) + .setEncoding(AudioFormat.ENCODING_AAC_LC) + .build(); + AudioAttributes audioAttributes = + new AudioAttributes.Builder() + .setContentType(C.AUDIO_CONTENT_TYPE_UNKNOWN) + .setUsage(C.USAGE_MEDIA) + .setAllowedCapturePolicy(C.ALLOW_CAPTURE_BY_ALL) + .build(); + ShadowAudioSystem.setOffloadSupported( + audioFormat, audioAttributes.getPlatformAudioAttributes(), true); + ShadowAudioSystem.setOffloadPlaybackSupport( + audioFormat, + audioAttributes.getPlatformAudioAttributes(), + AudioManager.PLAYBACK_OFFLOAD_SUPPORTED); + ShadowAudioSystem.setDirectPlaybackSupport( + audioFormat, + audioAttributes.getPlatformAudioAttributes(), + AudioManager.DIRECT_PLAYBACK_OFFLOAD_SUPPORTED); + configureDefaultAudioSinkWithOffload(); + ByteBuffer bufferA = ByteBuffer.allocateDirect(11 * 1024).order(ByteOrder.nativeOrder()); + boolean handledA = + defaultAudioSink.handleBuffer( + bufferA, /* presentationTimeUs= */ 0, /* encodedAccessUnitCount= */ 32); + assertThat(handledA).isTrue(); + + // Reconfigure the sink for Track B (gapless reuse transition). + Format formatB = + new Format.Builder() + .setSampleMimeType(MimeTypes.AUDIO_AAC) + .setChannelCount(2) + .setSampleRate(SAMPLE_RATE_44_1) + .setCodecs("mp4a.40.02") + .build(); + defaultAudioSink.configure(new AudioSink.AudioSinkConfig.Builder(formatB).build()); + simulatePartialWrite.set(true); + + ByteBuffer bufferB = ByteBuffer.allocateDirect(11 * 1024).order(ByteOrder.nativeOrder()); + boolean handledB = + defaultAudioSink.handleBuffer( + bufferB, /* presentationTimeUs= */ 3_000_000, /* encodedAccessUnitCount= */ 32); + assertThat(handledB).isFalse(); + + // Advance the mock playhead past Track A's duration. + mockPositionUs.set(750_000); + + assertThat(defaultAudioSink.hasPendingData()).isTrue(); + } + private void configureDefaultAudioSink(int channelCount) throws AudioSink.ConfigurationException { configureDefaultAudioSink(channelCount, /* trimStartFrames= */ 0, /* trimEndFrames= */ 0); } diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/PcmAudioUtilTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/PcmAudioUtilTest.java index eb0c092c80f..dab5af10c1e 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/PcmAudioUtilTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/PcmAudioUtilTest.java @@ -33,7 +33,7 @@ public final class PcmAudioUtilTest { @Test public void readAs32BitIntPcm_read8Bit_returnsExpectedValues() { ByteBuffer buffer = ByteBuffer.allocate(5); - buffer.put(hexToBytes("80" + "AB" + "00" + "12" + "7F")); + buffer.put(hexToBytes("00" + "2B" + "80" + "92" + "FF")); buffer.flip(); assertThat(PcmAudioUtil.readAs32BitIntPcm(buffer, C.ENCODING_PCM_8BIT)) @@ -197,7 +197,7 @@ public void write32BitIntPcm_write8Bit_returnsExpectedValues() { PcmAudioUtil.write32BitIntPcm(buffer, Integer.MAX_VALUE, C.ENCODING_PCM_8BIT); buffer.flip(); - assertThat(byteBufferToHex(buffer)).isEqualTo("80" + "AB" + "00" + "12" + "7F"); + assertThat(byteBufferToHex(buffer)).isEqualTo("00" + "2B" + "80" + "92" + "FF"); } @Test diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/SilenceSkippingAudioProcessorTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/SilenceSkippingAudioProcessorTest.java index d37de3cd098..11cc7208932 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/SilenceSkippingAudioProcessorTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/audio/SilenceSkippingAudioProcessorTest.java @@ -22,8 +22,10 @@ import androidx.media3.common.C; import androidx.media3.common.audio.AudioProcessor.AudioFormat; +import androidx.media3.common.audio.AudioProcessor.StreamMetadata; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.collect.Range; +import java.io.ByteArrayOutputStream; import java.nio.ByteBuffer; import java.nio.ByteOrder; import java.nio.ShortBuffer; @@ -95,7 +97,7 @@ public void skipInSilentSignal_skipsEverything() throws Exception { // When processing the entire signal. silenceSkippingAudioProcessor.setEnabled(true); silenceSkippingAudioProcessor.configure(AUDIO_FORMAT); - silenceSkippingAudioProcessor.flush(); + silenceSkippingAudioProcessor.flush(StreamMetadata.DEFAULT); assertThat(silenceSkippingAudioProcessor.isActive()).isTrue(); long totalOutputFrames = process(silenceSkippingAudioProcessor, inputBufferProvider, INPUT_BUFFER_SIZE); @@ -118,7 +120,7 @@ public void skipInNoisySignal_skipsNothing() throws Exception { new SilenceSkippingAudioProcessor(); silenceSkippingAudioProcessor.setEnabled(true); silenceSkippingAudioProcessor.configure(AUDIO_FORMAT); - silenceSkippingAudioProcessor.flush(); + silenceSkippingAudioProcessor.flush(StreamMetadata.DEFAULT); assertThat(silenceSkippingAudioProcessor.isActive()).isTrue(); long totalOutputFrames = process(silenceSkippingAudioProcessor, inputBufferProvider, INPUT_BUFFER_SIZE); @@ -142,7 +144,7 @@ public void skipInNoisySignalWithShortSilences_skipsNothing() throws Exception { new SilenceSkippingAudioProcessor(); silenceSkippingAudioProcessor.setEnabled(true); silenceSkippingAudioProcessor.configure(AUDIO_FORMAT); - silenceSkippingAudioProcessor.flush(); + silenceSkippingAudioProcessor.flush(StreamMetadata.DEFAULT); assertThat(silenceSkippingAudioProcessor.isActive()).isTrue(); long totalOutputFrames = process(silenceSkippingAudioProcessor, inputBufferProvider, INPUT_BUFFER_SIZE); @@ -166,13 +168,14 @@ public void skipInAlternatingTestSignal_hasCorrectOutputAndSkippedFrameCounts() new SilenceSkippingAudioProcessor(); silenceSkippingAudioProcessor.setEnabled(true); silenceSkippingAudioProcessor.configure(AUDIO_FORMAT); - silenceSkippingAudioProcessor.flush(); + silenceSkippingAudioProcessor.flush(StreamMetadata.DEFAULT); assertThat(silenceSkippingAudioProcessor.isActive()).isTrue(); long totalOutputFrames = process(silenceSkippingAudioProcessor, inputBufferProvider, INPUT_BUFFER_SIZE); - // The output has 50000 frames of noise, plus 50 * 0.2 * 1000 padding (plus rounding errors). - assertThat(totalOutputFrames).isIn(Range.closed(60000L - 500L, 60000L + 500L)); + // The output has 50000 frames of noise, plus 50 * (100 + 0.2 * 900) frames of silence (plus + // rounding errors). + assertThat(totalOutputFrames).isIn(Range.closed(64000L - 1500L, 64000L + 1500L)); assertThat(silenceSkippingAudioProcessor.getSkippedFrames()) .isEqualTo(TEST_SIGNAL_FRAME_COUNT - totalOutputFrames); } @@ -191,7 +194,7 @@ public void skipInAlternatingTestSignal_hasCorrectOutputAndSkippedFrameCounts() new SilenceSkippingAudioProcessor(); silenceSkippingAudioProcessor.setEnabled(true); silenceSkippingAudioProcessor.configure(AUDIO_FORMAT); - silenceSkippingAudioProcessor.flush(); + silenceSkippingAudioProcessor.flush(StreamMetadata.DEFAULT); // Early configure the next format without flushing yet (this format should be ignored). silenceSkippingAudioProcessor.configure( @@ -200,8 +203,9 @@ public void skipInAlternatingTestSignal_hasCorrectOutputAndSkippedFrameCounts() long totalOutputFrames = process(silenceSkippingAudioProcessor, inputBufferProvider, INPUT_BUFFER_SIZE); - // The output has 50000 frames of noise, plus 50 * 0.2 * 1000 padding (plus rounding errors). - assertThat(totalOutputFrames).isIn(Range.closed(60000L - 500L, 60000L + 500L)); + // The output has 50000 frames of noise, plus 50 * (100 + 0.2 * 900) frames of silence (plus + // rounding errors). + assertThat(totalOutputFrames).isIn(Range.closed(64000L - 1500L, 64000L + 1500L)); assertThat(silenceSkippingAudioProcessor.getSkippedFrames()) .isEqualTo(TEST_SIGNAL_FRAME_COUNT - totalOutputFrames); } @@ -221,13 +225,14 @@ public void skipWithSmallerInputBufferSize_hasCorrectOutputAndSkippedFrameCounts new SilenceSkippingAudioProcessor(); silenceSkippingAudioProcessor.setEnabled(true); silenceSkippingAudioProcessor.configure(AUDIO_FORMAT); - silenceSkippingAudioProcessor.flush(); + silenceSkippingAudioProcessor.flush(StreamMetadata.DEFAULT); assertThat(silenceSkippingAudioProcessor.isActive()).isTrue(); long totalOutputFrames = process(silenceSkippingAudioProcessor, inputBufferProvider, /* inputBufferSize= */ 80); - // The output has 50000 frames of noise, plus 50 * 0.2 * 1000 padding (plus rounding errors). - assertThat(totalOutputFrames).isIn(Range.closed(60000L - 500L, 60000L + 500L)); + // The output has 50000 frames of noise, plus 50 * (100 + 0.2 * 900) frames of silence (plus + // rounding errors). + assertThat(totalOutputFrames).isIn(Range.closed(64000L - 1500L, 64000L + 1500L)); assertThat(silenceSkippingAudioProcessor.getSkippedFrames()) .isEqualTo(TEST_SIGNAL_FRAME_COUNT - totalOutputFrames); } @@ -247,13 +252,14 @@ public void skipWithLargerInputBufferSize_hasCorrectOutputAndSkippedFrameCounts( new SilenceSkippingAudioProcessor(); silenceSkippingAudioProcessor.setEnabled(true); silenceSkippingAudioProcessor.configure(AUDIO_FORMAT); - silenceSkippingAudioProcessor.flush(); + silenceSkippingAudioProcessor.flush(StreamMetadata.DEFAULT); assertThat(silenceSkippingAudioProcessor.isActive()).isTrue(); long totalOutputFrames = process(silenceSkippingAudioProcessor, inputBufferProvider, /* inputBufferSize= */ 120); - // The output has 50000 frames of noise, plus 50 * 0.2 * 1000 padding (plus rounding errors). - assertThat(totalOutputFrames).isIn(Range.closed(60000L - 500L, 60000L + 500L)); + // The output has 50000 frames of noise, plus 50 * (100 + 0.2 * 900) frames of silence (plus + // rounding errors). + assertThat(totalOutputFrames).isIn(Range.closed(64000L - 1500L, 64000L + 1500L)); assertThat(silenceSkippingAudioProcessor.getSkippedFrames()) .isEqualTo(TEST_SIGNAL_FRAME_COUNT - totalOutputFrames); } @@ -277,13 +283,14 @@ public void customSilenceRetentionValue_hasCorrectOutputAndSkippedFrameCounts() SilenceSkippingAudioProcessor.DEFAULT_SILENCE_THRESHOLD_LEVEL); silenceSkippingAudioProcessor.setEnabled(true); silenceSkippingAudioProcessor.configure(AUDIO_FORMAT); - silenceSkippingAudioProcessor.flush(); + silenceSkippingAudioProcessor.flush(StreamMetadata.DEFAULT); assertThat(silenceSkippingAudioProcessor.isActive()).isTrue(); long totalOutputFrames = process(silenceSkippingAudioProcessor, inputBufferProvider, /* inputBufferSize= */ 120); - // The output has 50000 frames of noise, plus 50 * 0.05 * 1000 padding (plus rounding errors). - assertThat(totalOutputFrames).isIn(Range.closed(52500L - 500L, 52500L + 500L)); + // The output has 50000 frames of noise, plus 50 * (100 + 0.05 * 900) frames of silence (plus + // rounding errors). + assertThat(totalOutputFrames).isIn(Range.closed(56800L - 1500L, 56800L + 1500L)); assertThat(silenceSkippingAudioProcessor.getSkippedFrames()) .isEqualTo(TEST_SIGNAL_FRAME_COUNT - totalOutputFrames); } @@ -302,15 +309,113 @@ public void skipThenFlush_resetsSkippedFrameCount() throws Exception { new SilenceSkippingAudioProcessor(); silenceSkippingAudioProcessor.setEnabled(true); silenceSkippingAudioProcessor.configure(AUDIO_FORMAT); - silenceSkippingAudioProcessor.flush(); + silenceSkippingAudioProcessor.flush(StreamMetadata.DEFAULT); assertThat(silenceSkippingAudioProcessor.isActive()).isTrue(); process(silenceSkippingAudioProcessor, inputBufferProvider, INPUT_BUFFER_SIZE); - silenceSkippingAudioProcessor.flush(); + silenceSkippingAudioProcessor.flush(StreamMetadata.DEFAULT); // The skipped frame count is zero. assertThat(silenceSkippingAudioProcessor.getSkippedFrames()).isEqualTo(0); } + @Test + public void process_withSingleFrameFadeBuffer_processesWithoutError() throws Exception { + // 2ms at 1000Hz results in a 2-frame buffer (1-frame fade out/in, max == 0). + SilenceSkippingAudioProcessor silenceSkippingAudioProcessor = + new SilenceSkippingAudioProcessor( + /* minimumSilenceDurationUs= */ 2_000, + SilenceSkippingAudioProcessor.DEFAULT_SILENCE_RETENTION_RATIO, + SilenceSkippingAudioProcessor.DEFAULT_MAX_SILENCE_TO_KEEP_DURATION_US, + SilenceSkippingAudioProcessor.DEFAULT_MIN_VOLUME_TO_KEEP_PERCENTAGE, + SilenceSkippingAudioProcessor.DEFAULT_SILENCE_THRESHOLD_LEVEL); + silenceSkippingAudioProcessor.setEnabled(true); + silenceSkippingAudioProcessor.configure(AUDIO_FORMAT); + silenceSkippingAudioProcessor.flush(StreamMetadata.DEFAULT); + + // Provide 1 frame of noise (20000), 2 frames of silence (1000), and 1 frame of noise (20000). + ByteBuffer inputBuffer = + ByteBuffer.allocate(4 * AUDIO_FORMAT.bytesPerFrame).order(ByteOrder.nativeOrder()); + inputBuffer.putShort((short) 20000); + inputBuffer.putShort((short) 20000); + inputBuffer.putShort((short) 1000); + inputBuffer.putShort((short) 1000); + inputBuffer.putShort((short) 1000); + inputBuffer.putShort((short) 1000); + inputBuffer.putShort((short) 20000); + inputBuffer.putShort((short) 20000); + inputBuffer.flip(); + + ByteArrayOutputStream outStream = new ByteArrayOutputStream(); + while (inputBuffer.hasRemaining()) { + silenceSkippingAudioProcessor.queueInput(inputBuffer); + ByteBuffer outputBuffer = silenceSkippingAudioProcessor.getOutput(); + byte[] bytes = new byte[outputBuffer.remaining()]; + outputBuffer.get(bytes); + outStream.write(bytes); + } + silenceSkippingAudioProcessor.queueEndOfStream(); + while (!silenceSkippingAudioProcessor.isEnded()) { + ByteBuffer outputBuffer = silenceSkippingAudioProcessor.getOutput(); + byte[] bytes = new byte[outputBuffer.remaining()]; + outputBuffer.get(bytes); + outStream.write(bytes); + } + + ShortBuffer outputShortBuffer = + ByteBuffer.wrap(outStream.toByteArray()).order(ByteOrder.nativeOrder()).asShortBuffer(); + + // 1st frame: Noise (20000) + assertThat(outputShortBuffer.get()).isEqualTo((short) 20000); + assertThat(outputShortBuffer.get()).isEqualTo((short) 20000); + + // 2nd frame: 1-frame Fade-Out with max == 0 (scaled to 10% of 1000 = 100) + assertThat(outputShortBuffer.get()).isEqualTo((short) 100); + assertThat(outputShortBuffer.get()).isEqualTo((short) 100); + + // 3rd frame: 1-frame Fade-In with max == 0 (scaled to 100% of 1000 = 1000) + assertThat(outputShortBuffer.get()).isEqualTo((short) 1000); + assertThat(outputShortBuffer.get()).isEqualTo((short) 1000); + + // 4th frame: Noise (20000) + assertThat(outputShortBuffer.get()).isEqualTo((short) 20000); + assertThat(outputShortBuffer.get()).isEqualTo((short) 20000); + } + + @Test + public void queueInput_stereoFadedSilence_scalesBothChannelsEqually() throws Exception { + SilenceSkippingAudioProcessor silenceSkippingAudioProcessor = + new SilenceSkippingAudioProcessor(); + silenceSkippingAudioProcessor.setEnabled(true); + silenceSkippingAudioProcessor.configure(AUDIO_FORMAT); + silenceSkippingAudioProcessor.flush(StreamMetadata.DEFAULT); + + // Create a buffer with noise followed by silence (with non-zero sub-threshold level 500 on both + // L and R). + int frameCount = 2000; + ByteBuffer inputBuffer = + ByteBuffer.allocate(frameCount * AUDIO_FORMAT.bytesPerFrame).order(ByteOrder.nativeOrder()); + for (int i = 0; i < 500; i++) { + inputBuffer.putShort(MAX_VALUE); + inputBuffer.putShort(MAX_VALUE); + } + for (int i = 0; i < 1500; i++) { + inputBuffer.putShort((short) 500); + inputBuffer.putShort((short) 500); + } + inputBuffer.flip(); + + silenceSkippingAudioProcessor.queueInput(inputBuffer); + silenceSkippingAudioProcessor.queueEndOfStream(); + + ByteBuffer outputBuffer = silenceSkippingAudioProcessor.getOutput(); + ShortBuffer outputShortBuffer = outputBuffer.asShortBuffer(); + while (outputShortBuffer.hasRemaining()) { + short leftChannel = outputShortBuffer.get(); + short rightChannel = outputShortBuffer.get(); + assertThat(leftChannel).isEqualTo(rightChannel); + } + } + /** * Processes the entire stream provided by {@code inputBufferProvider} in chunks of {@code * inputBufferSize} and returns the total number of output frames. @@ -410,7 +515,7 @@ public Pcm16BitAudioBuilder(int channelCount, int frameCount) { public void appendFrames(int count, short... channelLevels) { checkState(!built); checkState(channelLevels.length == channelCount); - for (int i = 0; i < count; i += channelCount) { + for (int i = 0; i < count; i++) { for (short channelLevel : channelLevels) { buffer.put(channelLevel); } diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/ClippingPlaylistPlaybackTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/ClippingPlaylistPlaybackTest.java index 04a0dcc33c3..5eb57b0278b 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/ClippingPlaylistPlaybackTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/ClippingPlaylistPlaybackTest.java @@ -19,12 +19,15 @@ import android.graphics.SurfaceTexture; import android.view.Surface; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.MediaItem; import androidx.media3.common.Player; import androidx.media3.exoplayer.ExoPlayer; import androidx.media3.exoplayer.source.ConcatenatingMediaSource2; +import androidx.media3.test.utils.BindFlag; import androidx.media3.test.utils.DumpFileAsserts; import androidx.media3.test.utils.FakeClock; +import androidx.media3.test.utils.Media3FlagsRule; import androidx.media3.test.utils.robolectric.CapturingRenderersFactory; import androidx.media3.test.utils.robolectric.PlaybackOutput; import androidx.media3.test.utils.robolectric.ShadowMediaCodecConfig; @@ -70,12 +73,15 @@ public static List configs() { // TODO: b/510217604 - Remove parameterization. @ParameterizedRobolectricTestRunner.Parameter(2) + @BindFlag(Flags.FLAG_PER_STREAM_MEDIA_PROGRESSION) public boolean perStreamMediaPeriodProgressionEnabled; @Rule public ShadowMediaCodecConfig mediaCodecConfig = ShadowMediaCodecConfig.withAllDefaultSupportedCodecs(); + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); + @Test public void playbackWithClippingMediaSources() throws Exception { Context applicationContext = ApplicationProvider.getApplicationContext(); @@ -85,7 +91,6 @@ public void playbackWithClippingMediaSources() throws Exception { ExoPlayer player = new ExoPlayer.Builder(applicationContext, capturingRenderersFactory) .setClock(clock) - .enablePerStreamMediaProgression(perStreamMediaPeriodProgressionEnabled) .build(); Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); player.setVideoSurface(surface); @@ -135,7 +140,6 @@ public void playbackWithClippingMediaSourcesInConcatenatingMediaSource2() throws ExoPlayer player = new ExoPlayer.Builder(applicationContext, capturingRenderersFactory) .setClock(clock) - .enablePerStreamMediaProgression(perStreamMediaPeriodProgressionEnabled) .build(); Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); player.setVideoSurface(surface); diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/ClippingProgressivePlaybackTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/ClippingProgressivePlaybackTest.java index bc97a6c4783..ba398124744 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/ClippingProgressivePlaybackTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/ClippingProgressivePlaybackTest.java @@ -16,14 +16,17 @@ package androidx.media3.exoplayer.e2etest; import static androidx.media3.test.utils.robolectric.TestPlayerRunHelper.advance; +import static org.junit.Assume.assumeTrue; import android.content.Context; import android.graphics.SurfaceTexture; import android.net.Uri; +import android.os.ConditionVariable; import android.util.Pair; import android.view.Surface; import androidx.annotation.Nullable; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.MediaItem; import androidx.media3.common.Player; import androidx.media3.datasource.DefaultDataSource; @@ -40,19 +43,21 @@ import androidx.media3.extractor.ForwardingExtractorsFactory; import androidx.media3.extractor.ForwardingTrackOutput; import androidx.media3.extractor.TrackOutput; +import androidx.media3.test.utils.BindFlag; import androidx.media3.test.utils.DumpFileAsserts; import androidx.media3.test.utils.Dumper; import androidx.media3.test.utils.FakeClock; +import androidx.media3.test.utils.Media3FlagsRule; import androidx.media3.test.utils.robolectric.CapturingRenderersFactory; import androidx.media3.test.utils.robolectric.PlaybackOutput; import androidx.media3.test.utils.robolectric.ShadowMediaCodecConfig; import androidx.test.core.app.ApplicationProvider; +import com.google.common.base.Predicate; import com.google.common.collect.ImmutableList; import com.google.errorprone.annotations.CanIgnoreReturnValue; import com.google.testing.junit.testparameterinjector.TestParameter; import java.util.List; import java.util.Map; -import org.junit.Ignore; import org.junit.Rule; import org.junit.Test; import org.junit.runner.RunWith; @@ -63,13 +68,17 @@ // TODO: b/511055213 - Fails on API < 31 due to sync vs async MediaCodec behavior differences in // Robolectric. @Config(minSdk = 31) -@Ignore("Flaky: b/491840995") @RunWith(RobolectricTestParameterInjector.class) public final class ClippingProgressivePlaybackTest { private static final String TEST_MP4_URI = "asset:///media/mp4/sample.mp4"; + private static final String TEST_LONG_MP4_URI = "asset:///media/mp4/midroll-5s.mp4"; - @TestParameter private boolean enableMediaPeriodClipping; + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); + + @TestParameter + @BindFlag(Flags.FLAG_ENABLE_CLIPPING_IN_MEDIA_PERIOD) + private boolean enableMediaPeriodClipping; @Rule public ShadowMediaCodecConfig mediaCodecConfig = @@ -133,12 +142,327 @@ public void playback_clippedWithSeek() throws Exception { "playbackdumps/clipping/clipped_seek_period_clipping_" + dumpFileSuffix + ".dump"); } + @Test + public void replaceMediaItem_reduceEndPositionAfterLoadingFinished_playsToNewEndPosition() + throws Exception { + assumeTrue(enableMediaPeriodClipping); + Pair setupData = setUpPlayerAndCapturingOutputForClippingTest(); + ExoPlayer player = setupData.first; + PlaybackOutput playbackOutput = setupData.second; + MediaItem mediaItem = + new MediaItem.Builder() + .setUri(TEST_LONG_MP4_URI) + .setClippingConfiguration( + new MediaItem.ClippingConfiguration.Builder() + .setStartPositionMs(100) + .setEndPositionMs(3000) + .build()) + .build(); + player.setMediaItem(mediaItem); + player.prepare(); + advance(player).untilFullyBuffered(); + + player.replaceMediaItem( + /* index= */ 0, + mediaItem + .buildUpon() + .setClippingConfiguration( + mediaItem.clippingConfiguration.buildUpon().setEndPositionMs(2000).build()) + .build()); + advance(player).untilPendingCommandsAreFullyHandled(); + player.play(); + advance(player).untilState(Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + ApplicationProvider.getApplicationContext(), + playbackOutput, + "playbackdumps/clipping/replace_reduce_after_load_finished_enabled.dump"); + } + + @Test + public void + replaceMediaItem_reduceEndPositionBeforeLoadingFinishedToNotLoadedValue_playsToNewEndPosition() + throws Exception { + assumeTrue(enableMediaPeriodClipping); + ConditionVariable blockLoadingCondition = new ConditionVariable(); + CapturingExtractorsFactory[] factoryHolder = new CapturingExtractorsFactory[1]; + Pair setupData = + setUpPlayerAndCapturingOutputForClippingTest( + blockLoadingCondition, + /* blockLoadingFilter= */ sampleMetadata -> sampleMetadata.timeUs >= 1_000_000, + factoryHolder); + ExoPlayer player = setupData.first; + PlaybackOutput playbackOutput = setupData.second; + MediaItem mediaItem = + new MediaItem.Builder() + .setUri(TEST_LONG_MP4_URI) + .setClippingConfiguration( + new MediaItem.ClippingConfiguration.Builder() + .setStartPositionMs(100) + .setEndPositionMs(3000) + .build()) + .build(); + player.setMediaItem(mediaItem); + player.prepare(); + advance(player) + .untilBackgroundThreadCondition( + () -> factoryHolder[0] != null && factoryHolder[0].isBlocked()); + + player.replaceMediaItem( + /* index= */ 0, + mediaItem + .buildUpon() + .setClippingConfiguration( + mediaItem.clippingConfiguration.buildUpon().setEndPositionMs(2000).build()) + .build()); + advance(player).untilPendingCommandsAreFullyHandled(); + blockLoadingCondition.open(); + player.play(); + advance(player).untilState(Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + ApplicationProvider.getApplicationContext(), + playbackOutput, + "playbackdumps/clipping/replace_reduce_before_load_finished_to_not_loaded_value_enabled.dump"); + } + + @Test + public void + replaceMediaItem_reduceEndPositionBeforeLoadingFinishedToAlreadyLoadedValue_playsToNewEndPosition() + throws Exception { + assumeTrue(enableMediaPeriodClipping); + ConditionVariable blockLoadingCondition = new ConditionVariable(); + CapturingExtractorsFactory[] factoryHolder = new CapturingExtractorsFactory[1]; + Pair setupData = + setUpPlayerAndCapturingOutputForClippingTest( + blockLoadingCondition, + /* blockLoadingFilter= */ sampleMetadata -> sampleMetadata.timeUs >= 2_500_000, + factoryHolder); + ExoPlayer player = setupData.first; + PlaybackOutput playbackOutput = setupData.second; + MediaItem mediaItem = + new MediaItem.Builder() + .setUri(TEST_LONG_MP4_URI) + .setClippingConfiguration( + new MediaItem.ClippingConfiguration.Builder() + .setStartPositionMs(100) + .setEndPositionMs(4000) + .build()) + .build(); + player.setMediaItem(mediaItem); + player.prepare(); + advance(player) + .untilBackgroundThreadCondition( + () -> factoryHolder[0] != null && factoryHolder[0].isBlocked()); + + player.replaceMediaItem( + /* index= */ 0, + mediaItem + .buildUpon() + .setClippingConfiguration( + mediaItem.clippingConfiguration.buildUpon().setEndPositionMs(2000).build()) + .build()); + advance(player).untilPendingCommandsAreFullyHandled(); + blockLoadingCondition.open(); + player.play(); + advance(player).untilState(Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + ApplicationProvider.getApplicationContext(), + playbackOutput, + "playbackdumps/clipping/replace_reduce_before_load_finished_to_already_loaded_value_enabled.dump"); + } + + @Test + public void replaceMediaItem_extendEndPositionAfterLoadingFinished_playsToNewEndPosition() + throws Exception { + assumeTrue(enableMediaPeriodClipping); + Pair setupData = setUpPlayerAndCapturingOutputForClippingTest(); + ExoPlayer player = setupData.first; + PlaybackOutput playbackOutput = setupData.second; + MediaItem mediaItem = + new MediaItem.Builder() + .setUri(TEST_LONG_MP4_URI) + .setClippingConfiguration( + new MediaItem.ClippingConfiguration.Builder() + .setStartPositionMs(100) + .setEndPositionMs(2000) + .build()) + .build(); + player.setMediaItem(mediaItem); + player.prepare(); + advance(player).untilFullyBuffered(); + + player.replaceMediaItem( + /* index= */ 0, + mediaItem + .buildUpon() + .setClippingConfiguration( + mediaItem.clippingConfiguration.buildUpon().setEndPositionMs(3000).build()) + .build()); + advance(player).untilPendingCommandsAreFullyHandled(); + player.play(); + advance(player).untilState(Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + ApplicationProvider.getApplicationContext(), + playbackOutput, + "playbackdumps/clipping/replace_extend_after_load_finished_enabled.dump"); + } + + @Test + public void replaceMediaItem_extendEndPositionBeforeLoadingFinished_playsToNewEndPosition() + throws Exception { + assumeTrue(enableMediaPeriodClipping); + ConditionVariable blockLoadingCondition = new ConditionVariable(); + CapturingExtractorsFactory[] factoryHolder = new CapturingExtractorsFactory[1]; + Pair setupData = + setUpPlayerAndCapturingOutputForClippingTest( + blockLoadingCondition, + /* blockLoadingFilter= */ sampleMetadata -> sampleMetadata.timeUs >= 1_500_000, + factoryHolder); + ExoPlayer player = setupData.first; + PlaybackOutput playbackOutput = setupData.second; + MediaItem mediaItem = + new MediaItem.Builder() + .setUri(TEST_LONG_MP4_URI) + .setClippingConfiguration( + new MediaItem.ClippingConfiguration.Builder() + .setStartPositionMs(100) + .setEndPositionMs(2000) + .build()) + .build(); + player.setMediaItem(mediaItem); + player.prepare(); + advance(player) + .untilBackgroundThreadCondition( + () -> factoryHolder[0] != null && factoryHolder[0].isBlocked()); + + player.replaceMediaItem( + /* index= */ 0, + mediaItem + .buildUpon() + .setClippingConfiguration( + mediaItem.clippingConfiguration.buildUpon().setEndPositionMs(3000).build()) + .build()); + advance(player).untilPendingCommandsAreFullyHandled(); + blockLoadingCondition.open(); + player.play(); + advance(player).untilState(Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + ApplicationProvider.getApplicationContext(), + playbackOutput, + "playbackdumps/clipping/replace_extend_before_load_finished_enabled.dump"); + } + + @Test + public void + replaceMediaItem_extendEndPositionToEndOfSourceAfterLoadingFinished_playsToEndOfSource() + throws Exception { + assumeTrue(enableMediaPeriodClipping); + Pair setupData = setUpPlayerAndCapturingOutputForClippingTest(); + ExoPlayer player = setupData.first; + PlaybackOutput playbackOutput = setupData.second; + MediaItem mediaItem = + new MediaItem.Builder() + .setUri(TEST_LONG_MP4_URI) + .setClippingConfiguration( + new MediaItem.ClippingConfiguration.Builder() + .setStartPositionMs(100) + .setEndPositionMs(2000) + .build()) + .build(); + player.setMediaItem(mediaItem); + player.prepare(); + advance(player).untilFullyBuffered(); + + player.replaceMediaItem( + /* index= */ 0, + mediaItem + .buildUpon() + .setClippingConfiguration( + mediaItem + .clippingConfiguration + .buildUpon() + .setEndPositionMs(C.TIME_END_OF_SOURCE) + .build()) + .build()); + advance(player).untilPendingCommandsAreFullyHandled(); + player.play(); + advance(player).untilState(Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + ApplicationProvider.getApplicationContext(), + playbackOutput, + "playbackdumps/clipping/replace_extend_to_end_of_source_after_load_finished_enabled.dump"); + } + + @Test + public void + replaceMediaItem_reduceEndPositionFromEndOfSourceAfterLoadingFinished_playsToNewEndPosition() + throws Exception { + assumeTrue(enableMediaPeriodClipping); + Pair setupData = setUpPlayerAndCapturingOutputForClippingTest(); + ExoPlayer player = setupData.first; + PlaybackOutput playbackOutput = setupData.second; + MediaItem mediaItem = + new MediaItem.Builder() + .setUri(TEST_LONG_MP4_URI) + .setClippingConfiguration( + new MediaItem.ClippingConfiguration.Builder() + .setStartPositionMs(100) + .setEndPositionMs(C.TIME_END_OF_SOURCE) + .build()) + .build(); + player.setMediaItem(mediaItem); + player.prepare(); + advance(player).untilFullyBuffered(); + + player.replaceMediaItem( + /* index= */ 0, + mediaItem + .buildUpon() + .setClippingConfiguration( + mediaItem.clippingConfiguration.buildUpon().setEndPositionMs(2000).build()) + .build()); + advance(player).untilPendingCommandsAreFullyHandled(); + player.play(); + advance(player).untilState(Player.STATE_ENDED); + player.release(); + + DumpFileAsserts.assertOutput( + ApplicationProvider.getApplicationContext(), + playbackOutput, + "playbackdumps/clipping/replace_reduce_from_end_of_source_after_load_finished_enabled.dump"); + } + private Pair setUpPlayerAndCapturingOutputForClippingTest() { + return setUpPlayerAndCapturingOutputForClippingTest( + /* blockLoadingCondition= */ null, + /* blockLoadingFilter= */ null, + /* factoryHolder= */ null); + } + + private Pair setUpPlayerAndCapturingOutputForClippingTest( + @Nullable ConditionVariable blockLoadingCondition, + @Nullable Predicate blockLoadingFilter, + @Nullable CapturingExtractorsFactory[] factoryHolder) { Context context = ApplicationProvider.getApplicationContext(); FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); // Capture the extracted samples in addition to the decoded samples to capture the loading // pattern and verify that loading stops once all required samples are available. - CapturingExtractorsFactory capturingExtractorsFactory = new CapturingExtractorsFactory(); + CapturingExtractorsFactory capturingExtractorsFactory = + new CapturingExtractorsFactory(blockLoadingCondition, blockLoadingFilter); + if (factoryHolder != null && factoryHolder.length > 0) { + factoryHolder[0] = capturingExtractorsFactory; + } CapturingRenderersFactory capturingRenderersFactory = new CapturingRenderersFactory(context, clock) { @Override @@ -163,12 +487,20 @@ public MediaSource createMediaSource(MediaItem mediaItem) { return new ClippingMediaSource.Builder(progressiveSource) .setStartPositionUs(mediaItem.clippingConfiguration.startPositionUs) .setEndPositionUs(mediaItem.clippingConfiguration.endPositionUs) - .setEnableClippingInMediaPeriod(enableMediaPeriodClipping) .build(); } }) .setClock(clock) .build(); + player.addListener( + new Player.Listener() { + @Override + public void onPlaybackStateChanged(@Player.State int playbackState) { + if (playbackState == Player.STATE_READY || playbackState == Player.STATE_ENDED) { + capturingExtractorsFactory.startedPlayback(); + } + } + }); Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); player.setVideoSurface(surface); PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory); @@ -178,15 +510,36 @@ public MediaSource createMediaSource(MediaItem mediaItem) { private static final class CapturingExtractorsFactory extends ForwardingExtractorsFactory implements Dumper.Dumpable { - private final ImmutableList.Builder samples; + @Nullable private final ConditionVariable blockLoadingCondition; + @Nullable private final Predicate blockLoadingFilter; + + private ImmutableList.Builder samples; + private volatile boolean isBlocked; + private boolean startedPlayback; private int nextSampleIndex; private CapturingExtractorsFactory() { + this(/* blockLoadingCondition= */ null, /* blockLoadingFilter= */ null); + } + + private CapturingExtractorsFactory( + @Nullable ConditionVariable blockLoadingCondition, + @Nullable Predicate blockLoadingFilter) { super(new DefaultExtractorsFactory()); + this.blockLoadingCondition = blockLoadingCondition; + this.blockLoadingFilter = blockLoadingFilter; samples = ImmutableList.builder(); } + boolean isBlocked() { + return isBlocked; + } + + synchronized void startedPlayback() { + startedPlayback = true; + } + @Override public Extractor[] createExtractors() { return wrapExtractors(super.createExtractors()); @@ -198,7 +551,7 @@ public Extractor[] createExtractors(Uri uri, Map> responseH } @Override - public void dump(Dumper dumper) { + public synchronized void dump(Dumper dumper) { dumper.startBlock("extracted samples"); for (DumpableSampleMetadata sampleMetadata : samples.build()) { sampleMetadata.dump(dumper); @@ -211,11 +564,23 @@ private Extractor[] wrapExtractors(Extractor[] extractors) { for (int i = 0; i < extractors.length; i++) { extractors[i] = new ForwardingExtractor(extractors[i]) { - boolean seenClippedSeek = false; + private boolean seenInitialPreparationSeek; @Override public void seek(long position, long timeUs) { - seenClippedSeek |= timeUs != 0; + if (!seenInitialPreparationSeek) { + seenInitialPreparationSeek = true; + } else if (timeUs != 0) { + // A seek with non-zero timeUs before playback starts indicates that the initial + // preparation load was cancelled and restarted for the clipped start position. + // Clear the discarded samples from the aborted initial load. + synchronized (CapturingExtractorsFactory.this) { + if (!startedPlayback) { + samples = ImmutableList.builder(); + nextSampleIndex = 0; + } + } + } super.seek(position, timeUs); } @@ -233,12 +598,18 @@ public void sampleMetadata( int size, int offset, @Nullable CryptoData cryptoData) { - if (seenClippedSeek) { - // Ignore samples from initial preparation before applying the seek to - // the clipped start position. - samples.add( + DumpableSampleMetadata metadata; + synchronized (CapturingExtractorsFactory.this) { + metadata = new DumpableSampleMetadata( - nextSampleIndex++, id, timeUs, flags, size)); + nextSampleIndex++, id, timeUs, flags, size); + samples.add(metadata); + } + if (blockLoadingCondition != null + && blockLoadingFilter != null + && blockLoadingFilter.apply(metadata)) { + isBlocked = true; + blockLoadingCondition.block(); } super.sampleMetadata(timeUs, flags, size, offset, cryptoData); } diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/EndToEndGaplessTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/EndToEndGaplessTest.java index ee721064d70..61afc195332 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/EndToEndGaplessTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/EndToEndGaplessTest.java @@ -92,8 +92,8 @@ public void testPlayback_twoIdenticalMp3Files() throws Exception { int bytesPerFrame = audioTrackListener.getAudioTrackOutputFormat().getFrameSizeInBytes(); int paddingBytes = max(0, playerAudioFormat.encoderPadding) * bytesPerFrame; int delayBytes = max(0, playerAudioFormat.encoderDelay) * bytesPerFrame; - assertThat(paddingBytes).isEqualTo(1750); - assertThat(delayBytes).isEqualTo(2210); + assertThat(paddingBytes).isEqualTo(2808); + assertThat(delayBytes).isEqualTo(1152); byte[] decoderOutputBytes = Bytes.concat(mp3Decoder.getAllOutputBytes().toArray(new byte[0][])); int bytesPerAudioFile = decoderOutputBytes.length / 2; diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/MergingPlaylistPlaybackTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/MergingPlaylistPlaybackTest.java index a28e3dc2d49..5dc96c3714b 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/MergingPlaylistPlaybackTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/MergingPlaylistPlaybackTest.java @@ -26,6 +26,7 @@ import android.graphics.SurfaceTexture; import android.view.Surface; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.MediaItem; import androidx.media3.common.Player; import androidx.media3.exoplayer.ExoPlayer; @@ -34,8 +35,10 @@ import androidx.media3.exoplayer.source.FilteringMediaSource; import androidx.media3.exoplayer.source.MediaSource; import androidx.media3.exoplayer.source.MergingMediaSource; +import androidx.media3.test.utils.BindFlag; import androidx.media3.test.utils.DumpFileAsserts; import androidx.media3.test.utils.FakeClock; +import androidx.media3.test.utils.Media3FlagsRule; import androidx.media3.test.utils.robolectric.CapturingRenderersFactory; import androidx.media3.test.utils.robolectric.PlaybackOutput; import androidx.media3.test.utils.robolectric.ShadowMediaCodecConfig; @@ -90,12 +93,15 @@ public static List configs() { // TODO: b/510217604 - Remove parameterization. @ParameterizedRobolectricTestRunner.Parameter(5) + @BindFlag(Flags.FLAG_PER_STREAM_MEDIA_PROGRESSION) public Boolean perStreamMediaProgressionEnabled; @Rule public ShadowMediaCodecConfig mediaCodecConfig = ShadowMediaCodecConfig.withAllDefaultSupportedCodecs(); + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); + @Test public void transitionBetweenDifferentMergeConfigurations() throws Exception { Context applicationContext = ApplicationProvider.getApplicationContext(); @@ -105,7 +111,6 @@ public void transitionBetweenDifferentMergeConfigurations() throws Exception { ExoPlayer player = new ExoPlayer.Builder(applicationContext, capturingRenderersFactory) .setClock(clock) - .enablePerStreamMediaProgression(perStreamMediaProgressionEnabled) .build(); Player.Listener listener = mock(Player.Listener.class); player.addListener(listener); @@ -153,7 +158,6 @@ public void multipleRepetitionsOfSameMergeConfiguration() throws Exception { ExoPlayer player = new ExoPlayer.Builder(applicationContext, capturingRenderersFactory) .setClock(clock) - .enablePerStreamMediaProgression(perStreamMediaProgressionEnabled) .build(); Player.Listener listener = mock(Player.Listener.class); player.addListener(listener); diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/ParseAv1SampleDependenciesPlaybackTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/ParseAv1SampleDependenciesPlaybackTest.java index 498e2a0269b..2ba23b4cf55 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/ParseAv1SampleDependenciesPlaybackTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/ParseAv1SampleDependenciesPlaybackTest.java @@ -67,7 +67,6 @@ public void playback_withClippedMediaItem_skipNonReferenceInputSamples() throws FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); CapturingRenderersFactory renderersFactory = new CapturingRenderersFactory(applicationContext, clock); - renderersFactory.experimentalSetParseAv1SampleDependencies(true); ExoPlayer player = new ExoPlayer.Builder(applicationContext, renderersFactory).setClock(clock).build(); Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); @@ -99,7 +98,6 @@ public void playback_withConcatenatedMediaSource2_skipNonReferenceInputSamples() FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); CapturingRenderersFactory renderersFactory = new CapturingRenderersFactory(applicationContext, clock); - renderersFactory.experimentalSetParseAv1SampleDependencies(true); ExoPlayer player = new ExoPlayer.Builder(applicationContext, renderersFactory).setClock(clock).build(); Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); @@ -198,7 +196,6 @@ public Renderer[] createRenderers( /* enableDecoderFallback= */ false, eventHandler, videoRendererEventListener, - /* parseAv1SampleDependencies= */ true, /* lateThresholdToDropDecoderInputUs= */ -100_000_000L) }; } @@ -215,7 +212,6 @@ private CapturingMediaCodecVideoRenderer( boolean enableDecoderFallback, @Nullable Handler eventHandler, @Nullable VideoRendererEventListener eventListener, - boolean parseAv1SampleDependencies, long lateThresholdToDropDecoderInputUs) { super( new Builder(context) @@ -226,9 +222,8 @@ private CapturingMediaCodecVideoRenderer( .setEventHandler(eventHandler) .setEventListener(eventListener) .setMaxDroppedFramesToNotify(1) - .experimentalSetParseAv1SampleDependencies(parseAv1SampleDependencies) - .experimentalSetLateThresholdToDropDecoderInputUs( - lateThresholdToDropDecoderInputUs)); + .experimentalSetLateThresholdToDropDecoderInputUs(lateThresholdToDropDecoderInputUs) + .setSkipBuffersWithIdenticalReleaseTime(false)); } @Override @@ -245,12 +240,6 @@ protected boolean shouldDropBuffersToKeyframe( return false; } - @Override - protected boolean shouldSkipBuffersWithIdenticalReleaseTime() { - // Do not skip buffers with identical vsync times as we can't control this from tests. - return false; - } - @Override protected boolean shouldForceRenderOutputBuffer(long earlyUs, long elapsedSinceLastRenderUs) { // An auto-advancing FakeClock can make a lot of progress before diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/PrewarmingRendererPlaybackTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/PrewarmingRendererPlaybackTest.java index ea5aa9886da..639289eb9e5 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/PrewarmingRendererPlaybackTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/PrewarmingRendererPlaybackTest.java @@ -22,6 +22,7 @@ import android.os.Handler; import android.view.Surface; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.MediaItem; import androidx.media3.common.Player; import androidx.media3.common.util.Clock; @@ -33,8 +34,10 @@ import androidx.media3.exoplayer.text.TextOutput; import androidx.media3.exoplayer.video.MediaCodecVideoRenderer; import androidx.media3.exoplayer.video.VideoRendererEventListener; +import androidx.media3.test.utils.BindFlag; import androidx.media3.test.utils.DumpFileAsserts; import androidx.media3.test.utils.FakeClock; +import androidx.media3.test.utils.Media3FlagsRule; import androidx.media3.test.utils.robolectric.CapturingRenderersFactory; import androidx.media3.test.utils.robolectric.PlaybackOutput; import androidx.media3.test.utils.robolectric.ShadowMediaCodecConfig; @@ -61,12 +64,15 @@ public static ImmutableList params() { } @ParameterizedRobolectricTestRunner.Parameter(0) + @BindFlag(Flags.FLAG_PER_STREAM_MEDIA_PROGRESSION) public Boolean perStreamMediaProgressionEnabled; @Rule public ShadowMediaCodecConfig mediaCodecConfig = ShadowMediaCodecConfig.withAllDefaultSupportedCodecs(); + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); + @Test public void playback_withTwoMediaItemsAndSecondaryVideoRenderer_dumpsCorrectOutput() throws Exception { @@ -77,7 +83,6 @@ public void playback_withTwoMediaItemsAndSecondaryVideoRenderer_dumpsCorrectOutp ExoPlayer player = new ExoPlayer.Builder(applicationContext, capturingRenderersFactory) .setClock(clock) - .enablePerStreamMediaProgression(perStreamMediaProgressionEnabled) .build(); Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); player.setVideoSurface(surface); @@ -112,7 +117,6 @@ public void playback_withThreeItemsAndSecondaryVideoRenderer_dumpsCorrectOutput( ExoPlayer player = new ExoPlayer.Builder(applicationContext, capturingRenderersFactory) .setClock(clock) - .enablePerStreamMediaProgression(perStreamMediaProgressionEnabled) .build(); Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); player.setVideoSurface(surface); @@ -158,7 +162,6 @@ public void playback_withStopDuringPlaybackWithSecondaryVideoRenderer_dumpsCorre ExoPlayer player = new ExoPlayer.Builder(applicationContext, capturingRenderersFactory) .setClock(clock) - .enablePerStreamMediaProgression(perStreamMediaProgressionEnabled) .build(); Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); player.setVideoSurface(surface); @@ -218,7 +221,6 @@ public void playback_withMultipleMediaItemsWithClippingConfigurations_dumpsCorre ExoPlayer player = new ExoPlayer.Builder(applicationContext, capturingRenderersFactory) .setClock(clock) - .enablePerStreamMediaProgression(perStreamMediaProgressionEnabled) .build(); Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); player.setVideoSurface(surface); @@ -277,7 +279,6 @@ public void playback_withPrewarmingNonTransitioningRenderer_dumpsCorrectOutput() ExoPlayer player = new ExoPlayer.Builder(applicationContext, capturingRenderersFactory) .setClock(clock) - .enablePerStreamMediaProgression(perStreamMediaProgressionEnabled) .build(); Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); player.setVideoSurface(surface); @@ -318,7 +319,6 @@ public void playback_withImageVideoPlaylistAndSecondaryVideoRendererOnly_dumpsCo ExoPlayer player = new ExoPlayer.Builder(applicationContext, capturingRenderersFactory) .setClock(clock) - .enablePerStreamMediaProgression(perStreamMediaProgressionEnabled) .build(); Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); player.setVideoSurface(surface); @@ -344,6 +344,79 @@ public void playback_withImageVideoPlaylistAndSecondaryVideoRendererOnly_dumpsCo + ".dump"); } + @Test + public void playback_withMultipleVideoImagePlaylist_dumpsCorrectOutput() throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); + CapturingRenderersWithSecondaryVideoAndImageRenderersFactory capturingRenderersFactory = + new CapturingRenderersWithSecondaryVideoAndImageRenderersFactory(applicationContext, clock); + ExoPlayer player = + new ExoPlayer.Builder(applicationContext, capturingRenderersFactory) + .setClock(clock) + .build(); + Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); + player.setVideoSurface(surface); + PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory); + player.addMediaItems( + ImmutableList.of( + new MediaItem.Builder().setUri(TEST_MP4_URI).build(), + new MediaItem.Builder().setUri(TEST_MP4_URI).build(), + new MediaItem.Builder().setUri(TEST_IMAGE_URI).setImageDurationMs(1000).build(), + new MediaItem.Builder().setUri(TEST_IMAGE_URI).setImageDurationMs(1000).build(), + new MediaItem.Builder().setUri(TEST_MP4_URI).build())); + + player.prepare(); + advance(player).untilState(Player.STATE_READY); + player.play(); + advance(player).untilState(Player.STATE_ENDED); + player.release(); + surface.release(); + + DumpFileAsserts.assertOutput( + applicationContext, + playbackOutput, + /* dumpFile= */ "playbackdumps/prewarmingRenderer/" + + "multipleVideoImagePlaylist-withSecondaryRenderers" + + ".dump"); + } + + @Test + public void playback_withVideoVideoImagePlaylistThenSeekToStart_dumpsCorrectOutput() + throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); + CapturingRenderersWithSecondaryVideoAndImageRenderersFactory capturingRenderersFactory = + new CapturingRenderersWithSecondaryVideoAndImageRenderersFactory(applicationContext, clock); + ExoPlayer player = + new ExoPlayer.Builder(applicationContext, capturingRenderersFactory) + .setClock(clock) + .build(); + Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); + player.setVideoSurface(surface); + PlaybackOutput playbackOutput = PlaybackOutput.register(player, capturingRenderersFactory); + player.addMediaItems( + ImmutableList.of( + new MediaItem.Builder().setUri(TEST_MP4_URI).build(), + new MediaItem.Builder().setUri(TEST_MP4_URI).build(), + new MediaItem.Builder().setUri(TEST_IMAGE_URI).setImageDurationMs(1000).build())); + + player.prepare(); + player.play(); + advance(player).untilState(Player.STATE_ENDED); + player.seekTo(/* mediaItemIndex= */ 0, /* positionMs= */ 0); + advance(player).untilState(Player.STATE_ENDED); + + player.release(); + surface.release(); + + DumpFileAsserts.assertOutput( + applicationContext, + playbackOutput, + /* dumpFile= */ "playbackdumps/prewarmingRenderer/" + + "videoVideoImagePlaylist-seekToStart-withSecondaryRenderers" + + ".dump"); + } + /** This class extends {@link CapturingRenderersFactory} to provide a secondary video renderer. */ private static final class CapturingRenderersWithSecondaryVideoRendererFactory extends CapturingRenderersFactory { diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/ScrubbingPlaybackTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/ScrubbingPlaybackTest.java index 4d5e4e895c0..542635526cf 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/ScrubbingPlaybackTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/ScrubbingPlaybackTest.java @@ -470,8 +470,7 @@ protected MediaCodecVideoRenderer createMediaCodecVideoRenderer( /* enableDecoderFallback= */ false, eventHandler, videoRendererEventListener, - DefaultRenderersFactory.MAX_DROPPED_VIDEO_FRAME_COUNT_TO_NOTIFY, - /* parseAv1SampleDependencies= */ false) { + DefaultRenderersFactory.MAX_DROPPED_VIDEO_FRAME_COUNT_TO_NOTIFY) { @Override protected boolean processOutputBuffer( long positionUs, diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/SubtitlePlaybackTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/SubtitlePlaybackTest.java index 30b2a7038dd..57718ec33bf 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/SubtitlePlaybackTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/e2etest/SubtitlePlaybackTest.java @@ -16,6 +16,7 @@ package androidx.media3.exoplayer.e2etest; import static androidx.media3.test.utils.robolectric.TestPlayerRunHelper.advance; +import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.truth.Truth.assertThat; import android.content.Context; @@ -28,6 +29,7 @@ import androidx.media3.common.MimeTypes; import androidx.media3.common.ParserException; import androidx.media3.common.Player; +import androidx.media3.common.text.CueGroup; import androidx.media3.exoplayer.ExoPlayer; import androidx.media3.exoplayer.analytics.AnalyticsListener; import androidx.media3.exoplayer.source.DefaultMediaSourceFactory; @@ -62,6 +64,267 @@ public class SubtitlePlaybackTest { public ShadowMediaCodecConfig mediaCodecConfig = ShadowMediaCodecConfig.withAllDefaultSupportedCodecs(); + @Test + public void sideloadedSubtitle_withPositiveTimeOffset_cuesShiftedLater() throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); + CapturingRenderersFactory capturingRenderersFactory = + new CapturingRenderersFactory(applicationContext, clock); + ExoPlayer player = + new ExoPlayer.Builder(applicationContext, capturingRenderersFactory) + .setClock(clock) + .build(); + Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); + player.setVideoSurface(surface); + List cueChangeTimesUs = new ArrayList<>(); + List cueTexts = new ArrayList<>(); + player.addListener(createNonEmptyCueGroupCollectingListener(cueChangeTimesUs, cueTexts)); + MediaItem mediaItem = + new MediaItem.Builder() + .setUri("asset:///media/mp4/preroll-5s.mp4") + .setSubtitleConfigurations( + ImmutableList.of( + new MediaItem.SubtitleConfiguration.Builder( + Uri.parse("asset:///media/webvtt/typical")) + .setMimeType(MimeTypes.TEXT_VTT) + .setLanguage("en") + .setSelectionFlags(C.SELECTION_FLAG_DEFAULT) + .setTimeOffsetUs(300_000) + .build())) + .build(); + + player.setMediaItem(mediaItem); + player.prepare(); + advance(player).untilState(Player.STATE_READY); + advance(player).untilFullyBuffered(); + player.play(); + advance(player).untilState(Player.STATE_ENDED); + player.release(); + surface.release(); + + assertThat(cueChangeTimesUs).containsExactly(300_000L, 2_645_000L).inOrder(); + assertThat(cueTexts) + .containsExactly("This is the first subtitle.", "This is the second subtitle.") + .inOrder(); + } + + @Test + public void sideloadedSubtitle_withNegativeTimeOffset_cuesShiftedEarlier() throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); + CapturingRenderersFactory capturingRenderersFactory = + new CapturingRenderersFactory(applicationContext, clock); + ExoPlayer player = + new ExoPlayer.Builder(applicationContext, capturingRenderersFactory) + .setClock(clock) + .build(); + Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); + player.setVideoSurface(surface); + List cueChangeTimesUs = new ArrayList<>(); + List cueTexts = new ArrayList<>(); + player.addListener(createNonEmptyCueGroupCollectingListener(cueChangeTimesUs, cueTexts)); + MediaItem mediaItem = + new MediaItem.Builder() + .setUri("asset:///media/mp4/preroll-5s.mp4") + .setSubtitleConfigurations( + ImmutableList.of( + new MediaItem.SubtitleConfiguration.Builder( + Uri.parse("asset:///media/webvtt/typical")) + .setMimeType(MimeTypes.TEXT_VTT) + .setLanguage("en") + .setSelectionFlags(C.SELECTION_FLAG_DEFAULT) + .setTimeOffsetUs(-2_000_000) + .build())) + .build(); + + player.setMediaItem(mediaItem); + player.prepare(); + advance(player).untilState(Player.STATE_READY); + advance(player).untilFullyBuffered(); + player.play(); + advance(player).untilState(Player.STATE_ENDED); + player.release(); + surface.release(); + + // The first cue ends before the start of the media, so only the second one is shown. + assertThat(cueChangeTimesUs).containsExactly(345_000L); + assertThat(cueTexts).containsExactly("This is the second subtitle."); + } + + @Test + public void + sideloadedSubtitle_timeOffsetUpdatedDuringPlaybackWithTextTrackReenabled_playbackContinuesWithShiftedCues() + throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); + CapturingRenderersFactory capturingRenderersFactory = + new CapturingRenderersFactory(applicationContext, clock); + ExoPlayer player = + new ExoPlayer.Builder(applicationContext, capturingRenderersFactory) + .setClock(clock) + .build(); + Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); + player.setVideoSurface(surface); + List cueChangeTimesUs = new ArrayList<>(); + List cueTexts = new ArrayList<>(); + player.addListener(createNonEmptyCueGroupCollectingListener(cueChangeTimesUs, cueTexts)); + MediaItem.SubtitleConfiguration subtitleConfiguration = + new MediaItem.SubtitleConfiguration.Builder(Uri.parse("asset:///media/webvtt/typical")) + .setMimeType(MimeTypes.TEXT_VTT) + .setLanguage("en") + .setSelectionFlags(C.SELECTION_FLAG_DEFAULT) + .build(); + MediaItem mediaItem = + new MediaItem.Builder() + .setUri("asset:///media/mp4/preroll-5s.mp4") + .setSubtitleConfigurations(ImmutableList.of(subtitleConfiguration)) + .build(); + + player.setMediaItem(mediaItem); + player.prepare(); + advance(player).untilState(Player.STATE_READY); + advance(player).untilFullyBuffered(); + advance(player).untilPositionAtLeast(/* positionMs= */ 2000); + List playbackStatesAfterUpdate = new ArrayList<>(); + player.addListener( + new Player.Listener() { + @Override + public void onPlaybackStateChanged(@Player.State int playbackState) { + playbackStatesAfterUpdate.add(playbackState); + } + }); + // Shift the subtitles two seconds later and re-enable the text track to apply the new offset + // to the cues around the current position. + player.replaceMediaItem( + /* index= */ 0, + mediaItem + .buildUpon() + .setSubtitleConfigurations( + ImmutableList.of( + subtitleConfiguration.buildUpon().setTimeOffsetUs(2_000_000).build())) + .build()); + player.setTrackSelectionParameters( + player + .getTrackSelectionParameters() + .buildUpon() + .setTrackTypeDisabled(C.TRACK_TYPE_TEXT, /* disabled= */ true) + .build()); + player.setTrackSelectionParameters( + player + .getTrackSelectionParameters() + .buildUpon() + .setTrackTypeDisabled(C.TRACK_TYPE_TEXT, /* disabled= */ false) + .build()); + player.play(); + advance(player).untilState(Player.STATE_ENDED); + long updatedTimeOffsetUs = + checkNotNull(player.getCurrentMediaItem().localConfiguration) + .subtitleConfigurations + .get(0) + .timeOffsetUs; + player.release(); + surface.release(); + + assertThat(updatedTimeOffsetUs).isEqualTo(2_000_000); + // The first cue is shown with the initial zero offset, then again when the text track is + // re-enabled at two seconds with the new offset, followed by the shifted second cue. + assertThat(cueChangeTimesUs).containsExactly(0L, 2_000_000L, 4_345_000L).inOrder(); + assertThat(cueTexts) + .containsExactly( + "This is the first subtitle.", + "This is the first subtitle.", + "This is the second subtitle.") + .inOrder(); + // The media item update must not interrupt playback with a re-preparation. + assertThat(playbackStatesAfterUpdate).containsExactly(Player.STATE_ENDED); + } + + @Test + public void + sideloadedSubtitle_timeOffsetUpdatedDuringPlaybackWithoutTextTrackReenabled_appliesToFutureCuesOnly() + throws Exception { + Context applicationContext = ApplicationProvider.getApplicationContext(); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); + CapturingRenderersFactory capturingRenderersFactory = + new CapturingRenderersFactory(applicationContext, clock); + ExoPlayer player = + new ExoPlayer.Builder(applicationContext, capturingRenderersFactory) + .setClock(clock) + .build(); + Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); + player.setVideoSurface(surface); + List cueChangeTimesUs = new ArrayList<>(); + List cueTexts = new ArrayList<>(); + player.addListener(createNonEmptyCueGroupCollectingListener(cueChangeTimesUs, cueTexts)); + MediaItem.SubtitleConfiguration subtitleConfiguration = + new MediaItem.SubtitleConfiguration.Builder(Uri.parse("asset:///media/webvtt/typical")) + .setMimeType(MimeTypes.TEXT_VTT) + .setLanguage("en") + .setSelectionFlags(C.SELECTION_FLAG_DEFAULT) + .build(); + MediaItem mediaItem = + new MediaItem.Builder() + .setUri("asset:///media/mp4/preroll-5s.mp4") + .setSubtitleConfigurations(ImmutableList.of(subtitleConfiguration)) + .build(); + + player.setMediaItem(mediaItem); + player.prepare(); + advance(player).untilState(Player.STATE_READY); + advance(player).untilFullyBuffered(); + advance(player).untilPositionAtLeast(/* positionMs= */ 1000); + List playbackStatesAfterUpdate = new ArrayList<>(); + player.addListener( + new Player.Listener() { + @Override + public void onPlaybackStateChanged(@Player.State int playbackState) { + playbackStatesAfterUpdate.add(playbackState); + } + }); + // Shift the subtitles one second later. Without re-enabling the text track, the new offset + // only applies to future cues that haven't been read by the renderer yet. + player.replaceMediaItem( + /* index= */ 0, + mediaItem + .buildUpon() + .setSubtitleConfigurations( + ImmutableList.of( + subtitleConfiguration.buildUpon().setTimeOffsetUs(1_000_000).build())) + .build()); + player.play(); + advance(player).untilState(Player.STATE_ENDED); + long updatedTimeOffsetUs = + checkNotNull(player.getCurrentMediaItem().localConfiguration) + .subtitleConfigurations + .get(0) + .timeOffsetUs; + player.release(); + surface.release(); + + assertThat(updatedTimeOffsetUs).isEqualTo(1_000_000); + // The first cue is shown with the initial zero offset, and the second cue is shown with the + // new shifted offset. + assertThat(cueChangeTimesUs).containsExactly(0L, 3_345_000L).inOrder(); + assertThat(cueTexts) + .containsExactly("This is the first subtitle.", "This is the second subtitle.") + .inOrder(); + // The media item update must not interrupt playback with a re-preparation. + assertThat(playbackStatesAfterUpdate).containsExactly(Player.STATE_ENDED); + } + + private static Player.Listener createNonEmptyCueGroupCollectingListener( + List cueChangeTimesUs, List cueTexts) { + return new Player.Listener() { + @Override + public void onCues(CueGroup cueGroup) { + if (!cueGroup.cues.isEmpty()) { + cueChangeTimesUs.add(cueGroup.presentationTimeUs); + cueTexts.add(String.valueOf(cueGroup.cues.get(0).text)); + } + } + }; + } + // https://github.com/androidx/media/issues/1721 @Test public void multipleSideloadedSubtitles_noneSelected_noneLoaded() throws Exception { diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/mediacodec/AsynchronousMediaCodecBufferEnqueuerTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/mediacodec/AsynchronousMediaCodecBufferEnqueuerTest.java index 02a3bcf4b05..615eb9ce992 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/mediacodec/AsynchronousMediaCodecBufferEnqueuerTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/mediacodec/AsynchronousMediaCodecBufferEnqueuerTest.java @@ -57,11 +57,7 @@ public void setUp() throws IOException { codec.start(); handlerThread = new TestHandlerThread("TestHandlerThread"); enqueuer = - new AsynchronousMediaCodecBufferEnqueuer( - codec, - handlerThread, - mockConditionVariable, - /* enableAsyncCryptoSynchronization= */ false); + new AsynchronousMediaCodecBufferEnqueuer(codec, handlerThread, mockConditionVariable); } @After diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/mediacodec/MediaCodecInfoTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/mediacodec/MediaCodecInfoTest.java index 45ba1af4f2c..bf8830bcf4f 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/mediacodec/MediaCodecInfoTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/mediacodec/MediaCodecInfoTest.java @@ -388,6 +388,35 @@ public void canReuseCodec_dolbyVisionWithDifferentCodecs_returnsNo() { /* discardReasons= */ DISCARD_REASON_WORKAROUND)); } + @Test + public void isFormatSupported_dolbyVisionWithNoCodecsString_returnsFalse() { + Format formatDolbyVisionUnknownProfile = + new Format.Builder() + .setSampleMimeType(VIDEO_DOLBY_VISION) + .setWidth(1920) + .setHeight(1080) + .build(); + MediaCodecInfo codecInfo = buildDolbyVisionCodecInfo(); + Context context = ApplicationProvider.getApplicationContext(); + + assertThat(codecInfo.isFormatSupported(context, formatDolbyVisionUnknownProfile)).isFalse(); + } + + @Test + public void isFormatSupported_dolbyVisionProfile20_returnsFalse() { + Format formatDolbyVisionProfile20 = + new Format.Builder() + .setSampleMimeType(VIDEO_DOLBY_VISION) + .setCodecs("dvh1.20.01") + .setWidth(1920) + .setHeight(1080) + .build(); + MediaCodecInfo codecInfo = buildDolbyVisionCodecInfo(); + Context context = ApplicationProvider.getApplicationContext(); + + assertThat(codecInfo.isFormatSupported(context, formatDolbyVisionProfile20)).isFalse(); + } + @Test public void canReuseCodec_eac3_returnsYesWithoutReconfiguration() { MediaCodecInfo codecInfo = diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/mediacodec/MediaCodecRendererTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/mediacodec/MediaCodecRendererTest.java index 290fa718217..ea6b4c5614b 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/mediacodec/MediaCodecRendererTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/mediacodec/MediaCodecRendererTest.java @@ -47,10 +47,12 @@ import android.view.Surface; import androidx.annotation.Nullable; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; import androidx.media3.common.util.Clock; import androidx.media3.common.util.MediaFormatUtil; +import androidx.media3.common.util.ThrowingRunnable; import androidx.media3.decoder.CryptoInfo; import androidx.media3.exoplayer.CodecParameters; import androidx.media3.exoplayer.DecoderReuseEvaluation; @@ -64,6 +66,7 @@ import androidx.media3.exoplayer.source.MediaSource; import androidx.media3.exoplayer.upstream.DefaultAllocator; import androidx.media3.test.utils.FakeSampleStream; +import androidx.media3.test.utils.Media3FlagsRule; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.base.Supplier; @@ -73,6 +76,8 @@ import java.nio.ByteBuffer; import java.util.Collections; import java.util.List; +import org.junit.Before; +import org.junit.Rule; import org.junit.Test; import org.junit.runner.RunWith; import org.mockito.ArgumentCaptor; @@ -83,6 +88,8 @@ @RunWith(AndroidJUnit4.class) public class MediaCodecRendererTest { + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); + private static final Format AUDIO_AAC = new Format.Builder() .setSampleMimeType(MimeTypes.AUDIO_AAC) @@ -96,6 +103,11 @@ public class MediaCodecRendererTest { .setSampleRate(48000) .build(); + @Before + public void setUp() { + Flags.enableFlag(Flags.FLAG_PROCESSED_STREAM_CHANGED_AT_START); + } + @Test public void render_withReplaceStream_triggersOutputCallbacksInCorrectOrder() throws Exception { Format format1 = @@ -947,11 +959,11 @@ public void resetPosition_withBuffersReceived_flushesCodec() throws Exception { MediaCodecAdapter mockCodecAdapter = mock(MediaCodecAdapter.class); doAnswer( invocation -> { - ((Runnable) invocation.getArgument(0)).run(); + ((ThrowingRunnable) invocation.getArgument(0)).run(); return null; }) .when(mockCodecAdapter) - .useInputBuffer(any()); + .useBuffer(any()); when(mockCodecAdapter.dequeueInputBufferIndex()) .thenReturn(0) .thenReturn(MediaCodec.INFO_TRY_AGAIN_LATER); @@ -1106,7 +1118,6 @@ public TestRenderer(MediaCodecAdapter.Factory mediaCodecAdapterFactory) { /* forceSecure= */ false)), /* enableDecoderFallback= */ false, /* assumedMinimumCodecOperatingRate= */ 0); - experimentalEnableProcessedStreamChangedAtStart(); } @Override diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/mediacodec/MediaCodecUtilTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/mediacodec/MediaCodecUtilTest.java index 215bea1213d..96329fe2a0d 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/mediacodec/MediaCodecUtilTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/mediacodec/MediaCodecUtilTest.java @@ -15,16 +15,28 @@ */ package androidx.media3.exoplayer.mediacodec; +import static android.media.MediaCodecInfo.CodecProfileLevel.DolbyVisionLevelFhd30; +import static android.media.MediaCodecInfo.CodecProfileLevel.DolbyVisionProfileDvheDtr; +import static android.media.MediaCodecInfo.CodecProfileLevel.DolbyVisionProfileDvheSt; +import static android.media.MediaCodecInfo.CodecProfileLevel.HEVCHighTierLevel51; +import static android.media.MediaCodecInfo.CodecProfileLevel.HEVCMainTierLevel4; +import static android.media.MediaCodecInfo.CodecProfileLevel.HEVCMainTierLevel41; +import static android.media.MediaCodecInfo.CodecProfileLevel.HEVCProfileMain; +import static android.media.MediaCodecInfo.CodecProfileLevel.HEVCProfileMain10; +import static androidx.media3.exoplayer.mediacodec.MediaCodecUtil.createCodecProfileLevel; import static com.google.common.truth.Truth.assertThat; -import android.media.MediaCodecInfo; +import android.media.MediaCodecInfo.CodecCapabilities; +import android.media.MediaCodecInfo.CodecProfileLevel; import androidx.media3.common.C; import androidx.media3.common.ColorInfo; import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; import androidx.media3.common.util.CodecSpecificDataUtil.MediaCodecProfileAndLevel; +import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.collect.ImmutableList; +import java.util.List; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.shadows.ShadowBuild; @@ -212,10 +224,7 @@ public void getHevcBaseLayerCodecProfileAndLevel_handlesFallbackFromMvHevc() { .setCodecs("hvc1.6.40.L120.BF.80") .setInitializationData(ImmutableList.of(CSD0, CSD1)) .build(); - assertHevcBaseLayerCodecProfileAndLevelForFormat( - format, - MediaCodecInfo.CodecProfileLevel.HEVCProfileMain, - MediaCodecInfo.CodecProfileLevel.HEVCMainTierLevel4); + assertHevcBaseLayerCodecProfileAndLevelForFormat(format, HEVCProfileMain, HEVCMainTierLevel4); } @Test @@ -260,6 +269,24 @@ public void getAlternativeCodecMimeType_withNonFallbackCompatibleFormat_returnsN .isEqualTo(MimeTypes.VIDEO_AV1); } + @Test + public void getAlternativeCodecMimeType_dolbyVisionProfile20_returnsNull() { + Format format = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION) + .setCodecs("dvh1.20.01") + .build(); + + assertThat(MediaCodecUtil.getAlternativeCodecMimeType(format)).isNull(); + } + + @Test + public void getAlternativeCodecMimeType_dolbyVisionWithNullCodecs_returnsNull() { + Format format = new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION).build(); + + assertThat(MediaCodecUtil.getAlternativeCodecMimeType(format)).isNull(); + } + @Test public void getAlternativeCodecMimeType_withEac3JocFormatOnNonGoogleDevice_returnsEac3() { ShadowBuild.setManufacturer("Samsung"); @@ -274,6 +301,177 @@ public void getAlternativeCodecMimeType_withEac3JocFormatOnGoogleDevice_returnsN assertThat(MediaCodecUtil.getAlternativeCodecMimeType(format)).isNull(); } + @Test + public void + getDecoderInfosSoftMatchFilteredByFormatSupport_withDecoderNotMatchingFormatSupport_excludesDecoder() + throws Exception { + Format formatDvProfile8 = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION) + .setCodecs("dvhe.08.01") + .build(); + CodecCapabilities capabilitiesDolbyProfile4 = + createCodecCapabilities(DolbyVisionProfileDvheDtr, DolbyVisionLevelFhd30); + CodecCapabilities capabilitiesDolbyProfile8 = + createCodecCapabilities(DolbyVisionProfileDvheSt, DolbyVisionLevelFhd30); + MediaCodecInfo decoderProfile4 = + MediaCodecInfo.newInstance( + /* name= */ "dv-p4-codec", + /* mimeType= */ MimeTypes.VIDEO_DOLBY_VISION, + /* codecMimeType= */ MimeTypes.VIDEO_DOLBY_VISION, + /* capabilities= */ capabilitiesDolbyProfile4, + /* hardwareAccelerated= */ true, + /* softwareOnly= */ false, + /* vendor= */ false, + /* forceDisableAdaptive= */ false, + /* forceSecure= */ false); + MediaCodecInfo decoderProfile8 = + MediaCodecInfo.newInstance( + /* name= */ "dv-p8-codec", + /* mimeType= */ MimeTypes.VIDEO_DOLBY_VISION, + /* codecMimeType= */ MimeTypes.VIDEO_DOLBY_VISION, + /* capabilities= */ capabilitiesDolbyProfile8, + /* hardwareAccelerated= */ true, + /* softwareOnly= */ false, + /* vendor= */ false, + /* forceDisableAdaptive= */ false, + /* forceSecure= */ false); + MediaCodecSelector mediaCodecSelector = + (mimeType, requiresSecureDecoder, requiresTunnelingDecoder) -> { + if (mimeType.equals(MimeTypes.VIDEO_DOLBY_VISION)) { + return ImmutableList.of(decoderProfile4, decoderProfile8); + } + return ImmutableList.of(); + }; + + List decoderInfos = + MediaCodecUtil.getDecoderInfosSoftMatchFilteredByFormatSupport( + ApplicationProvider.getApplicationContext(), + mediaCodecSelector, + formatDvProfile8, + /* requiresSecureDecoder= */ false, + /* requiresTunnelingDecoder= */ false); + + assertThat(decoderInfos).containsExactly(decoderProfile8); + } + + @Test + public void + getAlternativeDecoderInfosFilteredByFormatSupport_withDecoderNotMatchingFormatSupport_excludesDecoder() + throws Exception { + Format formatDvProfile8 = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION) + .setCodecs("dvhe.08.01") + .build(); + CodecCapabilities capabilitiesHevcMain = + createCodecCapabilities(HEVCProfileMain, HEVCMainTierLevel41); + CodecCapabilities capabilitiesHevcMain10 = + createCodecCapabilities(HEVCProfileMain10, HEVCHighTierLevel51); + MediaCodecInfo decoderHevcMain = + MediaCodecInfo.newInstance( + /* name= */ "hevc-main-codec", + /* mimeType= */ MimeTypes.VIDEO_H265, + /* codecMimeType= */ MimeTypes.VIDEO_H265, + /* capabilities= */ capabilitiesHevcMain, + /* hardwareAccelerated= */ true, + /* softwareOnly= */ false, + /* vendor= */ false, + /* forceDisableAdaptive= */ false, + /* forceSecure= */ false); + MediaCodecInfo decoderHevcMain10 = + MediaCodecInfo.newInstance( + /* name= */ "hevc-main10-codec", + /* mimeType= */ MimeTypes.VIDEO_H265, + /* codecMimeType= */ MimeTypes.VIDEO_H265, + /* capabilities= */ capabilitiesHevcMain10, + /* hardwareAccelerated= */ true, + /* softwareOnly= */ false, + /* vendor= */ false, + /* forceDisableAdaptive= */ false, + /* forceSecure= */ false); + MediaCodecSelector mediaCodecSelector = + (mimeType, requiresSecureDecoder, requiresTunnelingDecoder) -> { + if (mimeType.equals(MimeTypes.VIDEO_H265)) { + return ImmutableList.of(decoderHevcMain, decoderHevcMain10); + } + return ImmutableList.of(); + }; + + List decoderInfos = + MediaCodecUtil.getAlternativeDecoderInfosFilteredByFormatSupport( + ApplicationProvider.getApplicationContext(), + mediaCodecSelector, + formatDvProfile8, + /* requiresSecureDecoder= */ false, + /* requiresTunnelingDecoder= */ false); + + assertThat(decoderInfos).containsExactly(decoderHevcMain10); + } + + @Test + public void + getDecoderInfosSoftMatchFilteredByFormatSupport_withNoMatchingDecoders_fallsBackToUnfilteredList() + throws Exception { + Format formatDvProfile8 = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION) + .setCodecs("dvhe.08.01") + .build(); + CodecCapabilities capabilitiesDvProfile4 = + createCodecCapabilities(DolbyVisionProfileDvheDtr, DolbyVisionLevelFhd30); + CodecCapabilities capabilitiesHevcMain = + createCodecCapabilities(HEVCProfileMain, HEVCMainTierLevel41); + MediaCodecInfo decoderProfile4 = + MediaCodecInfo.newInstance( + /* name= */ "dv-profile4-codec", + /* mimeType= */ MimeTypes.VIDEO_DOLBY_VISION, + /* codecMimeType= */ MimeTypes.VIDEO_DOLBY_VISION, + /* capabilities= */ capabilitiesDvProfile4, + /* hardwareAccelerated= */ true, + /* softwareOnly= */ false, + /* vendor= */ false, + /* forceDisableAdaptive= */ false, + /* forceSecure= */ false); + MediaCodecInfo decoderHevcMain = + MediaCodecInfo.newInstance( + /* name= */ "hevc-main-codec", + /* mimeType= */ MimeTypes.VIDEO_H265, + /* codecMimeType= */ MimeTypes.VIDEO_H265, + /* capabilities= */ capabilitiesHevcMain, + /* hardwareAccelerated= */ true, + /* softwareOnly= */ false, + /* vendor= */ false, + /* forceDisableAdaptive= */ false, + /* forceSecure= */ false); + MediaCodecSelector mediaCodecSelector = + (mimeType, requiresSecureDecoder, requiresTunnelingDecoder) -> { + if (mimeType.equals(MimeTypes.VIDEO_DOLBY_VISION)) { + return ImmutableList.of(decoderProfile4); + } + if (mimeType.equals(MimeTypes.VIDEO_H265)) { + return ImmutableList.of(decoderHevcMain); + } + return ImmutableList.of(); + }; + + List decoderInfos = + MediaCodecUtil.getDecoderInfosSoftMatchFilteredByFormatSupport( + ApplicationProvider.getApplicationContext(), + mediaCodecSelector, + formatDvProfile8, + /* requiresSecureDecoder= */ false, + /* requiresTunnelingDecoder= */ false); + + assertThat(decoderInfos).containsExactly(decoderProfile4, decoderHevcMain).inOrder(); + } + + private static CodecCapabilities createCodecCapabilities(int profile, int level) { + CodecCapabilities capabilities = new CodecCapabilities(); + capabilities.profileLevels = new CodecProfileLevel[] {createCodecProfileLevel(profile, level)}; + return capabilities; + } + private static void assertHevcBaseLayerCodecProfileAndLevelForFormat( Format format, int profile, int level) { MediaCodecProfileAndLevel codecProfileAndLevel = diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/metadata/MetadataRendererTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/metadata/MetadataRendererTest.java index 10e6f106d9f..e389143897f 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/metadata/MetadataRendererTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/metadata/MetadataRendererTest.java @@ -368,6 +368,8 @@ public void renderMetadata_doesNotReadAheadTooFar() throws Exception { ImmutableList.of( sample(/* timeUs= */ 100_000, C.BUFFER_FLAG_KEY_FRAME, encodedEmsg), sample(/* timeUs= */ 450_000, C.BUFFER_FLAG_KEY_FRAME, encodedEmsg), + FakeSampleStream.FakeSampleStreamItem.format( + EMSG_FORMAT.buildUpon().setId("2").build()), sample(/* timeUs= */ 2_000_000, C.BUFFER_FLAG_KEY_FRAME, encodedEmsg), END_OF_STREAM_ITEM)); fakeSampleStream.writeData(/* startPositionUs= */ 0); @@ -470,12 +472,17 @@ private static List runRenderer(byte[] input) throws ExoPlaybackExcept private static FakeSampleStream createFakeSampleStream( ImmutableList samples) { + return createFakeSampleStream(EMSG_FORMAT, samples); + } + + private static FakeSampleStream createFakeSampleStream( + Format format, ImmutableList samples) { return new FakeSampleStream( new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), /* mediaSourceEventDispatcher= */ null, DrmSessionManager.DRM_UNSUPPORTED, new DrmSessionEventListener.EventDispatcher(), - EMSG_FORMAT, + format, samples); } @@ -576,4 +583,59 @@ public void renderMetadata_withNextAdGroup_doesReadEndOfStream() throws Exceptio assertThat(renderer.getReadingPositionUs()).isEqualTo(C.TIME_END_OF_SOURCE); assertThat(renderer.isEnded()).isTrue(); } + + @Test + public void renderMetadata_withSubsampleOffsetUs_propagatesOffsetToDecoder() throws Exception { + Format scte35Format1 = + new Format.Builder() + .setSampleMimeType(MimeTypes.APPLICATION_SCTE35) + .setSubsampleOffsetUs(500_000) + .build(); + Format scte35Format2 = + new Format.Builder() + .setSampleMimeType(MimeTypes.APPLICATION_SCTE35) + .setSubsampleOffsetUs(800_000) + .build(); + List metadata = new ArrayList<>(); + MetadataRenderer renderer = new MetadataRenderer(metadata::add, /* outputLooper= */ null); + FakeSampleStream fakeSampleStream = + createFakeSampleStream( + scte35Format1, + ImmutableList.of( + sample(/* timeUs= */ 2_000_000, C.BUFFER_FLAG_KEY_FRAME, SCTE35_TIME_SIGNAL_BYTES), + FakeSampleStream.FakeSampleStreamItem.format(scte35Format2), + sample(/* timeUs= */ 4_000_000, C.BUFFER_FLAG_KEY_FRAME, SCTE35_TIME_SIGNAL_BYTES), + END_OF_STREAM_ITEM)); + fakeSampleStream.writeData(/* startPositionUs= */ 0); + renderer.enable( + RendererConfiguration.DEFAULT, + new Format[] {scte35Format1}, + fakeSampleStream, + /* positionUs= */ 0, + /* joining= */ false, + /* mayRenderStartOfStream= */ true, + /* startPositionUs= */ 0, + /* offsetUs= */ 1_000_000, + new MediaSource.MediaPeriodId(new Object())); + renderer.start(); + + // First render peeks scte35Format1 and defers reading the sample at 3_000_000. + renderer.render(/* positionUs= */ 1_000_000, /* elapsedRealtimeUs= */ 0); + // Second render consumes the first sample, peeks scte35Format2, and defers the second sample. + renderer.render(/* positionUs= */ 3_000_000, /* elapsedRealtimeUs= */ 0); + // Third render consumes the second sample. + renderer.render(/* positionUs= */ 5_000_000, /* elapsedRealtimeUs= */ 0); + + assertThat(metadata).hasSize(2); + TimeSignalCommand timeSignalCommand1 = (TimeSignalCommand) metadata.get(0).get(0); + assertThat(timeSignalCommand1.ptsTime).isEqualTo(0x5203028fL); + // Expected playbackPositionUs is ptsToUs(0x5203028f) + subsampleOffsetUs (500_000) + offsetUs + // (1_000_000) = 15_288_099_722 + 1_500_000 = 15_289_599_722. + assertThat(timeSignalCommand1.playbackPositionUs).isEqualTo(15_289_599_722L); + TimeSignalCommand timeSignalCommand2 = (TimeSignalCommand) metadata.get(1).get(0); + assertThat(timeSignalCommand2.ptsTime).isEqualTo(0x5203028fL); + // Expected playbackPositionUs is ptsToUs(0x5203028f) + subsampleOffsetUs (800_000) + offsetUs + // (1_000_000) = 15_288_099_722 + 1_800_000 = 15_289_899_722. + assertThat(timeSignalCommand2.playbackPositionUs).isEqualTo(15_289_899_722L); + } } diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ClippingMediaPeriodTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ClippingMediaPeriodTest.java index b31051d09ae..99aef119169 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ClippingMediaPeriodTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ClippingMediaPeriodTest.java @@ -16,6 +16,7 @@ package androidx.media3.exoplayer.source; import static androidx.media3.exoplayer.source.SampleStream.FLAG_HAS_PREROLL; +import static androidx.media3.exoplayer.source.SampleStream.FLAG_MAYBE_HAS_PREROLL; import static androidx.media3.exoplayer.source.SampleStream.FLAG_STRICT_DURATION; import static androidx.media3.test.utils.FakeSampleStream.FakeSampleStreamItem.END_OF_STREAM_ITEM; import static androidx.media3.test.utils.FakeSampleStream.FakeSampleStreamItem.oneByteSample; @@ -157,7 +158,9 @@ public void readDiscontinuity_prepareFromNonZero_returnsPreparePosition() throws assertThat(sampleStreams[0]).isNotNull(); assertThat(sampleStreams[0].getFlags()).isEqualTo(FLAG_STRICT_DURATION); - assertThat(sampleStreams[1].getFlags()).isEqualTo(FLAG_HAS_PREROLL | FLAG_STRICT_DURATION); + // The clip only sets an end position, so the wrapped period's report of its own preroll is + // passed through rather than being replaced by the enable-position heuristic. + assertThat(sampleStreams[1].getFlags()).isEqualTo(FLAG_STRICT_DURATION); assertThat(discontinuityPositionUs).isEqualTo(250); } @@ -264,7 +267,9 @@ public void readDiscontinuity_repeatedCalls_returnsUnset() throws Exception { long discontinuityPositionUs = clippingMediaPeriod.readDiscontinuity(); assertThat(sampleStreams[0].getFlags()).isEqualTo(FLAG_STRICT_DURATION); - assertThat(sampleStreams[1].getFlags()).isEqualTo(FLAG_HAS_PREROLL | FLAG_STRICT_DURATION); + // The clip only sets an end position, so the wrapped period's report of its own preroll is + // passed through rather than being replaced by the enable-position heuristic. + assertThat(sampleStreams[1].getFlags()).isEqualTo(FLAG_STRICT_DURATION); assertThat(discontinuityPositionUs).isEqualTo(C.TIME_UNSET); } @@ -305,7 +310,9 @@ public void readDiscontinuity_afterInitialSeek_returnsUnset() throws Exception { clippingMediaPeriod.seekToUs(400); long discontinuityPositionUs = clippingMediaPeriod.readDiscontinuity(); - assertThat(sampleStreams[1].getFlags()).isEqualTo(FLAG_STRICT_DURATION | FLAG_HAS_PREROLL); + // The clip only sets an end position, so the wrapped period's report of its own preroll is + // passed through rather than being replaced by the enable-position heuristic. + assertThat(sampleStreams[1].getFlags()).isEqualTo(FLAG_STRICT_DURATION); assertThat(discontinuityPositionUs).isEqualTo(C.TIME_UNSET); } @@ -509,4 +516,116 @@ public int getFlags() { assertThat(sampleStreams[0].getFlags()).isEqualTo(SampleStream.FLAG_STRICT_DURATION); } + + @Test + public void selectTracks_withEndClippingAndChildStreamMaybeHasPreroll_passesThroughChildFlags() + throws Exception { + TrackGroupArray trackGroups = new TrackGroupArray(VIDEO_TRACK_GROUP); + FakeMediaPeriod mediaPeriod = + new FakeMediaPeriod( + trackGroups, + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), + /* trackDataFactory= */ (format, mediaPeriodId) -> ImmutableList.of(), + new MediaSourceEventListener.EventDispatcher() + .withParameters( + /* windowIndex= */ 0, + new MediaSource.MediaPeriodId(/* periodUid= */ new Object())), + DrmSessionManager.DRM_UNSUPPORTED, + new DrmSessionEventListener.EventDispatcher(), + /* deferOnPrepared= */ false) { + @Override + protected FakeSampleStream createSampleStream( + Allocator allocator, + @Nullable MediaSourceEventListener.EventDispatcher mediaSourceEventDispatcher, + DrmSessionManager drmSessionManager, + DrmSessionEventListener.EventDispatcher drmEventDispatcher, + Format initialFormat, + List fakeSampleStreamItems) { + return new FakeSampleStream( + allocator, + mediaSourceEventDispatcher, + drmSessionManager, + drmEventDispatcher, + initialFormat, + fakeSampleStreamItems) { + @Override + public int getFlags() { + return FLAG_MAYBE_HAS_PREROLL; + } + }; + } + }; + + ClippingMediaPeriod clippingMediaPeriod = + new ClippingMediaPeriod( + mediaPeriod, + /* enableInitialDiscontinuity= */ true, + /* startUs= */ 0, + /* endUs= */ 500); + + SampleStream[] sampleStreams = + prepareMediaPeriodAndSelectTracks( + clippingMediaPeriod, /* preparePositionUs= */ 250, trackGroups); + + // The clip only sets an end position, so it adds no preroll of its own and the child's + // FLAG_MAYBE_HAS_PREROLL must survive. The flag tells the player to defer the reading period + // transition until the child resolves its preroll, instead of resetting the renderer now. + assertThat(sampleStreams[0].getFlags()) + .isEqualTo(FLAG_MAYBE_HAS_PREROLL | FLAG_STRICT_DURATION); + } + + @Test + public void selectTracks_withStartAndEndClippingAndChildStreamMaybeHasPreroll_setsFlagHasPreroll() + throws Exception { + TrackGroupArray trackGroups = new TrackGroupArray(VIDEO_TRACK_GROUP); + FakeMediaPeriod mediaPeriod = + new FakeMediaPeriod( + trackGroups, + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), + /* trackDataFactory= */ (format, mediaPeriodId) -> ImmutableList.of(), + new MediaSourceEventListener.EventDispatcher() + .withParameters( + /* windowIndex= */ 0, + new MediaSource.MediaPeriodId(/* periodUid= */ new Object())), + DrmSessionManager.DRM_UNSUPPORTED, + new DrmSessionEventListener.EventDispatcher(), + /* deferOnPrepared= */ false) { + @Override + protected FakeSampleStream createSampleStream( + Allocator allocator, + @Nullable MediaSourceEventListener.EventDispatcher mediaSourceEventDispatcher, + DrmSessionManager drmSessionManager, + DrmSessionEventListener.EventDispatcher drmEventDispatcher, + Format initialFormat, + List fakeSampleStreamItems) { + return new FakeSampleStream( + allocator, + mediaSourceEventDispatcher, + drmSessionManager, + drmEventDispatcher, + initialFormat, + fakeSampleStreamItems) { + @Override + public int getFlags() { + return FLAG_MAYBE_HAS_PREROLL; + } + }; + } + }; + + ClippingMediaPeriod clippingMediaPeriod = + new ClippingMediaPeriod( + mediaPeriod, + /* enableInitialDiscontinuity= */ true, + /* startUs= */ 250, + /* endUs= */ 500); + + SampleStream[] sampleStreams = + prepareMediaPeriodAndSelectTracks( + clippingMediaPeriod, /* preparePositionUs= */ 250, trackGroups); + + // The clip sets a non-zero start position, so it introduces preroll and overrides the child's + // FLAG_MAYBE_HAS_PREROLL with FLAG_HAS_PREROLL. + assertThat(sampleStreams[0].getFlags()).isEqualTo(FLAG_HAS_PREROLL | FLAG_STRICT_DURATION); + } } diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ClippingMediaSourceTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ClippingMediaSourceTest.java index 43924fe9e2e..f9d4e761f59 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ClippingMediaSourceTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ClippingMediaSourceTest.java @@ -20,6 +20,7 @@ import android.net.Uri; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.MediaItem; import androidx.media3.common.Player; import androidx.media3.common.Timeline; @@ -33,6 +34,7 @@ import androidx.media3.test.utils.FakeMediaSource; import androidx.media3.test.utils.FakeTimeline; import androidx.media3.test.utils.FakeTimeline.TimelineWindowDefinition; +import androidx.media3.test.utils.Media3FlagsRule; import androidx.media3.test.utils.MediaSourceTestRunner; import androidx.media3.test.utils.TestUtil; import androidx.media3.test.utils.TimelineAsserts; @@ -41,6 +43,7 @@ import java.io.IOException; import java.util.concurrent.atomic.AtomicReference; import org.junit.Before; +import org.junit.Rule; import org.junit.Test; import org.junit.runner.RunWith; @@ -48,6 +51,8 @@ @RunWith(AndroidJUnit4.class) public final class ClippingMediaSourceTest { + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); + private static final long TEST_PERIOD_DURATION_US = 1_000_000; private static final long TEST_CLIP_AMOUNT_US = 300_000; @@ -79,6 +84,8 @@ public void noClipping_returnsExpectedTimeline() throws IOException { .isEqualTo(TEST_PERIOD_DURATION_US); assertThat(clippedTimeline.getPeriod(0, period).getDurationUs()) .isEqualTo(TEST_PERIOD_DURATION_US); + assertThat(clippedTimeline.getPeriod(0, period).getOriginalDurationUs()) + .isEqualTo(TEST_PERIOD_DURATION_US); } @Test @@ -191,6 +198,8 @@ public void clipping_startOnly_returnsExpectedTimeline() throws IOException { .isEqualTo(TEST_PERIOD_DURATION_US - TEST_CLIP_AMOUNT_US); assertThat(clippedTimeline.getPeriod(0, period).getDurationUs()) .isEqualTo(TEST_PERIOD_DURATION_US); + assertThat(clippedTimeline.getPeriod(0, period).getOriginalDurationUs()) + .isEqualTo(TEST_PERIOD_DURATION_US); } @Test @@ -210,6 +219,15 @@ public void clipping_endOnly_returnsExpectedTimeline() throws IOException { .isEqualTo(TEST_PERIOD_DURATION_US - TEST_CLIP_AMOUNT_US); assertThat(clippedTimeline.getPeriod(0, period).getDurationUs()) .isEqualTo(TEST_PERIOD_DURATION_US - TEST_CLIP_AMOUNT_US); + assertThat(clippedTimeline.getPeriod(0, period).getOriginalDurationUs()) + .isEqualTo(TEST_PERIOD_DURATION_US); + + Timeline nestedClippedTimeline = + getClippedTimeline(clippedTimeline, 0, TEST_PERIOD_DURATION_US - TEST_CLIP_AMOUNT_US * 2); + assertThat(nestedClippedTimeline.getPeriod(0, period).getDurationUs()) + .isEqualTo(TEST_PERIOD_DURATION_US - TEST_CLIP_AMOUNT_US * 2); + assertThat(nestedClippedTimeline.getPeriod(0, period).getOriginalDurationUs()) + .isEqualTo(TEST_PERIOD_DURATION_US); } @Test @@ -227,6 +245,8 @@ public void clipping_startAndEndWithInitialPlaceHolderTimeline_returnsExpectedTi .isEqualTo(TEST_PERIOD_DURATION_US - TEST_CLIP_AMOUNT_US * 3); assertThat(clippedTimeline.getPeriod(0, period).getDurationUs()) .isEqualTo(TEST_PERIOD_DURATION_US - TEST_CLIP_AMOUNT_US * 2); + assertThat(clippedTimeline.getPeriod(0, period).getOriginalDurationUs()) + .isEqualTo(C.TIME_UNSET); } @Test @@ -311,6 +331,8 @@ public void clipping_startAndEnd_returnsExpectedTimeline() throws IOException { .isEqualTo(TEST_PERIOD_DURATION_US - TEST_CLIP_AMOUNT_US * 3); assertThat(clippedTimeline.getPeriod(0, period).getDurationUs()) .isEqualTo(TEST_PERIOD_DURATION_US - TEST_CLIP_AMOUNT_US * 2); + assertThat(clippedTimeline.getPeriod(0, period).getOriginalDurationUs()) + .isEqualTo(TEST_PERIOD_DURATION_US); } @Test @@ -624,11 +646,9 @@ public void canUpdateMediaItem_withChangedClippingConfiguration_returnsFalse() { FakeMediaSource fakeMediaSource = new FakeMediaSource(); fakeMediaSource.setCanUpdateMediaItems(true); fakeMediaSource.updateMediaItem(initialMediaItem); + Flags.enableFlag(Flags.FLAG_ENABLE_CLIPPING_IN_MEDIA_PERIOD); ClippingMediaSource mediaSource = - new ClippingMediaSource.Builder(fakeMediaSource) - .setEnableClippingInMediaPeriod(true) - .setStartPositionMs(1) - .build(); + new ClippingMediaSource.Builder(fakeMediaSource).setStartPositionMs(1).build(); boolean canUpdateMediaItem = mediaSource.canUpdateMediaItem(updatedMediaItem); @@ -670,9 +690,9 @@ public void updateMediaItem_withStartAndEndPositionChange_updatesTimelineAndMedi MediaItem.fromUri(Uri.EMPTY)); FakeMediaSource fakeMediaSource = new FakeMediaSource(timeline); fakeMediaSource.setCanUpdateMediaItems(true); + Flags.enableFlag(Flags.FLAG_ENABLE_CLIPPING_IN_MEDIA_PERIOD); ClippingMediaSource mediaSource = new ClippingMediaSource.Builder(fakeMediaSource) - .setEnableClippingInMediaPeriod(true) .setStartPositionUs(100_000) .setEndPositionUs(900_000) .build(); @@ -725,11 +745,9 @@ public void updateMediaItem_withDefaultPositionChange_updatesPeriodsCorrectly() FakeMediaSource fakeMediaSource = new FakeMediaSource(timeline); fakeMediaSource.setCanUpdateMediaItems(true); // Initial: start=100ms relative to window (so 100ms in period if not relative to default) + Flags.enableFlag(Flags.FLAG_ENABLE_CLIPPING_IN_MEDIA_PERIOD); ClippingMediaSource mediaSource = - new ClippingMediaSource.Builder(fakeMediaSource) - .setEnableClippingInMediaPeriod(true) - .setStartPositionUs(100_000) - .build(); + new ClippingMediaSource.Builder(fakeMediaSource).setStartPositionUs(100_000).build(); MediaSourceTestRunner testRunner = new MediaSourceTestRunner(mediaSource); try { @@ -779,9 +797,9 @@ public void updateMediaItem_withRelativeToLiveWindowChange_updatesPeriodsCorrect FakeMediaSource fakeMediaSource = new FakeMediaSource(timeline); fakeMediaSource.setCanUpdateMediaItems(true); // Initial: relativeToLiveWindow = true + Flags.enableFlag(Flags.FLAG_ENABLE_CLIPPING_IN_MEDIA_PERIOD); ClippingMediaSource mediaSource = new ClippingMediaSource.Builder(fakeMediaSource) - .setEnableClippingInMediaPeriod(true) .setAllowDynamicClippingUpdates(true) .setStartPositionUs(100_000) .build(); @@ -851,9 +869,9 @@ public void updateMediaItem_toEndOfSource_updatesPeriodsCorrectly() throws Excep MediaItem.fromUri(Uri.EMPTY)); FakeMediaSource fakeMediaSource = new FakeMediaSource(timeline); fakeMediaSource.setCanUpdateMediaItems(true); + Flags.enableFlag(Flags.FLAG_ENABLE_CLIPPING_IN_MEDIA_PERIOD); ClippingMediaSource mediaSource = new ClippingMediaSource.Builder(fakeMediaSource) - .setEnableClippingInMediaPeriod(true) .setStartPositionUs(100_000) .setEndPositionUs(900_000) .build(); diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/DefaultMediaSourceFactoryTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/DefaultMediaSourceFactoryTest.java index 1ba66100260..57c880e5298 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/DefaultMediaSourceFactoryTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/DefaultMediaSourceFactoryTest.java @@ -29,7 +29,7 @@ import androidx.media3.exoplayer.source.ads.AdsMediaSource; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; -import java.util.Arrays; +import com.google.common.collect.ImmutableList; import java.util.List; import org.junit.Test; import org.junit.runner.RunWith; @@ -75,11 +75,11 @@ public void createMediaSource_withPath_progressiveSource() { } @Test - public void createMediaSource_withSubtitle_isMergingMediaSource() { + public void createMediaSource_withSubtitle_isSideloadedSubtitlesMediaSource() { DefaultMediaSourceFactory defaultMediaSourceFactory = new DefaultMediaSourceFactory((Context) ApplicationProvider.getApplicationContext()); List subtitleConfigurations = - Arrays.asList( + ImmutableList.of( new MediaItem.SubtitleConfiguration.Builder(Uri.parse(URI_TEXT)) .setMimeType(MimeTypes.APPLICATION_TTML) .setLanguage("en") @@ -97,7 +97,7 @@ public void createMediaSource_withSubtitle_isMergingMediaSource() { MediaSource mediaSource = defaultMediaSourceFactory.createMediaSource(mediaItem); - assertThat(mediaSource).isInstanceOf(MergingMediaSource.class); + assertThat(mediaSource).isInstanceOf(SideloadedSubtitlesMediaSource.class); } @Test @@ -168,6 +168,18 @@ public void createMediaSource_defaultToEnd_isNotClippingMediaSource() { assertThat(mediaSource).isInstanceOf(ProgressiveMediaSource.class); } + @Test + public void createMediaSource_alwaysWrapWithoutClipping_isClippingMediaSource() { + DefaultMediaSourceFactory defaultMediaSourceFactory = + new DefaultMediaSourceFactory((Context) ApplicationProvider.getApplicationContext()) + .setAlwaysWrapInClippingMediaSource(true); + MediaItem mediaItem = new MediaItem.Builder().setUri(URI_MEDIA).build(); + + MediaSource mediaSource = defaultMediaSourceFactory.createMediaSource(mediaItem); + + assertThat(mediaSource).isInstanceOf(ClippingMediaSource.class); + } + @Test public void getSupportedTypes_coreModule_onlyOther() { int[] supportedTypes = diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/MergingMediaSourceTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/MergingMediaSourceTest.java index aa93d847b43..b7f2db0cc8e 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/MergingMediaSourceTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/MergingMediaSourceTest.java @@ -91,7 +91,13 @@ public void prepare_withDurationClipping_usesDurationOfShortestSource() throws I Timeline mergedTimeline = prepareMergingMediaSource(/* clipDurations= */ true, timeline1, timeline2, timeline3); - assertThat(mergedTimeline).isEqualTo(timeline3); + Timeline.Window window = mergedTimeline.getWindow(/* windowIndex= */ 0, new Timeline.Window()); + Timeline.Period period = mergedTimeline.getPeriod(/* periodIndex= */ 0, new Timeline.Period()); + assertThat(window.durationUs).isEqualTo(10); + assertThat(period.durationUs) + .isEqualTo(timeline3.getPeriod(/* periodIndex= */ 0, new Timeline.Period()).durationUs); + assertThat(period.originalDurationUs) + .isEqualTo(timeline1.getPeriod(/* periodIndex= */ 0, new Timeline.Period()).durationUs); } @Test diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/MergingMetadataSampleStreamTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/MergingMetadataSampleStreamTest.java index f4e13d15401..8d5f714eb42 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/MergingMetadataSampleStreamTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/MergingMetadataSampleStreamTest.java @@ -28,6 +28,7 @@ import androidx.media3.decoder.DecoderInputBuffer; import androidx.media3.exoplayer.FormatHolder; import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.util.concurrent.atomic.AtomicInteger; import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; @@ -55,66 +56,122 @@ public void setUp() { public void readData_attachesMetadataAsSupplementalData() { configureMockMetadataStream( metadataStream, new MetadataSample(/* timeUs= */ 0, metadataPayload)); - configureVideoStreamToReturnBuffer(videoStream, /* timeUs= */ 0); - + configureVideoStreamToReturnBuffers(videoStream, /* timesUs...= */ 0); MergingMetadataSampleStream stream = new MergingMetadataSampleStream(videoStream, metadataStream, VIDEO_FORMAT); - FormatHolder formatHolder = new FormatHolder(); - DecoderInputBuffer buffer = - new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); - int result = stream.readData(formatHolder, buffer, /* readFlags= */ 0); - assertThat(result).isEqualTo(RESULT_BUFFER_READ); - buffer.flip(); - assertThat(buffer.supplementalData).isNotNull(); - byte[] supplemental = new byte[buffer.supplementalData.remaining()]; - buffer.supplementalData.get(supplemental); - assertThat(supplemental).isEqualTo(metadataPayload); + DecoderInputBuffer buffer = readBuffer(stream, formatHolder); + + assertThat(getSupplementalData(buffer)).isEqualTo(metadataPayload); } @Test public void readData_sparseMetadata_appliesSameMetadataToMultipleFrames() { configureMockMetadataStream( metadataStream, new MetadataSample(/* timeUs= */ 0, metadataPayload)); - + configureVideoStreamToReturnBuffers(videoStream, /* timesUs...= */ 0, 33_000); MergingMetadataSampleStream stream = new MergingMetadataSampleStream(videoStream, metadataStream, VIDEO_FORMAT); FormatHolder formatHolder = new FormatHolder(); // First frame at PTS=0. - configureVideoStreamToReturnBuffer(videoStream, /* timeUs= */ 0); - DecoderInputBuffer buffer1 = - new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); - int result1 = stream.readData(formatHolder, buffer1, /* readFlags= */ 0); - assertThat(result1).isEqualTo(RESULT_BUFFER_READ); - + DecoderInputBuffer buffer1 = readBuffer(stream, formatHolder); // Second frame at PTS=33000 - same metadata should still apply. - configureVideoStreamToReturnBuffer(videoStream, /* timeUs= */ 33_000); - DecoderInputBuffer buffer2 = - new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); - int result2 = stream.readData(formatHolder, buffer2, /* readFlags= */ 0); - assertThat(result2).isEqualTo(RESULT_BUFFER_READ); - buffer2.flip(); - assertThat(buffer2.supplementalData).isNotNull(); - byte[] supplemental = new byte[buffer2.supplementalData.remaining()]; - buffer2.supplementalData.get(supplemental); - assertThat(supplemental).isEqualTo(metadataPayload); + DecoderInputBuffer buffer2 = readBuffer(stream, formatHolder); + + assertThat(getSupplementalData(buffer1)).isEqualTo(metadataPayload); + assertThat(getSupplementalData(buffer2)).isEqualTo(metadataPayload); + } + + @Test + public void + readData_multipleMetadataSamplesWithZeroReorderSamples_doesNotEvictFutureMetadataEarly() { + Format zeroReorderVideoFormat = VIDEO_FORMAT.buildUpon().setMaxNumReorderSamples(0).build(); + byte[] payload0 = {0x00}; + byte[] payload1 = {0x01}; + byte[] payload2 = {0x02}; + byte[] payload3 = {0x03}; + byte[] payload4 = {0x04}; + configureMockMetadataStream( + metadataStream, + new MetadataSample(/* timeUs= */ 0, payload0), + new MetadataSample(/* timeUs= */ 1_000_000, payload1), + new MetadataSample(/* timeUs= */ 2_000_000, payload2), + new MetadataSample(/* timeUs= */ 3_000_000, payload3), + new MetadataSample(/* timeUs= */ 4_000_000, payload4)); + configureVideoStreamToReturnBuffers( + videoStream, + /* timesUs...= */ 0, + 33_000, + 67_000, + 100_000, + 1_000_000, + 2_000_000, + 3_000_000, + 4_000_000); + MergingMetadataSampleStream stream = + new MergingMetadataSampleStream(videoStream, metadataStream, zeroReorderVideoFormat); + FormatHolder formatHolder = new FormatHolder(); + + DecoderInputBuffer frame000ms = readBuffer(stream, formatHolder); + DecoderInputBuffer frame033ms = readBuffer(stream, formatHolder); + DecoderInputBuffer frame067ms = readBuffer(stream, formatHolder); + DecoderInputBuffer frame100ms = readBuffer(stream, formatHolder); + DecoderInputBuffer frame1000ms = readBuffer(stream, formatHolder); + DecoderInputBuffer frame2000ms = readBuffer(stream, formatHolder); + DecoderInputBuffer frame3000ms = readBuffer(stream, formatHolder); + DecoderInputBuffer frame4000ms = readBuffer(stream, formatHolder); + + assertThat(getSupplementalData(frame000ms)).isEqualTo(payload0); + assertThat(getSupplementalData(frame033ms)).isEqualTo(payload0); + assertThat(getSupplementalData(frame067ms)).isEqualTo(payload0); + assertThat(getSupplementalData(frame100ms)).isEqualTo(payload0); + assertThat(getSupplementalData(frame1000ms)).isEqualTo(payload1); + assertThat(getSupplementalData(frame2000ms)).isEqualTo(payload2); + assertThat(getSupplementalData(frame3000ms)).isEqualTo(payload3); + assertThat(getSupplementalData(frame4000ms)).isEqualTo(payload4); + } + + @Test + public void readData_withBframeReordering_attachesCorrectMetadata() { + Format reorderVideoFormat = VIDEO_FORMAT.buildUpon().setMaxNumReorderSamples(2).build(); + byte[] payload0 = {0x01}; + byte[] payload1 = {0x02}; + configureMockMetadataStream( + metadataStream, + new MetadataSample(/* timeUs= */ 0, payload0), + new MetadataSample(/* timeUs= */ 100_000, payload1)); + // Decode order: I(0), P(133_000), B(66_000), B(33_000), B(100_000). + configureVideoStreamToReturnBuffers( + videoStream, /* timesUs...= */ 0, 133_000, 66_000, 33_000, 100_000); + MergingMetadataSampleStream stream = + new MergingMetadataSampleStream(videoStream, metadataStream, reorderVideoFormat); + FormatHolder formatHolder = new FormatHolder(); + + DecoderInputBuffer frameI0 = readBuffer(stream, formatHolder); + DecoderInputBuffer frameP133ms = readBuffer(stream, formatHolder); + DecoderInputBuffer frameB66ms = readBuffer(stream, formatHolder); + DecoderInputBuffer frameB33ms = readBuffer(stream, formatHolder); + DecoderInputBuffer frameB100ms = readBuffer(stream, formatHolder); + + assertThat(getSupplementalData(frameI0)).isEqualTo(payload0); + assertThat(getSupplementalData(frameP133ms)).isEqualTo(payload1); + assertThat(getSupplementalData(frameB66ms)).isEqualTo(payload0); + assertThat(getSupplementalData(frameB33ms)).isEqualTo(payload0); + assertThat(getSupplementalData(frameB100ms)).isEqualTo(payload1); } @Test public void readData_noMetadataAvailable_doesNotAttachSupplementalData() { when(metadataStream.readData(any(), any(), anyInt())).thenReturn(RESULT_NOTHING_READ); - configureVideoStreamToReturnBuffer(videoStream, /* timeUs= */ 0); - + configureVideoStreamToReturnBuffers(videoStream, /* timesUs...= */ 0); MergingMetadataSampleStream stream = new MergingMetadataSampleStream(videoStream, metadataStream, VIDEO_FORMAT); FormatHolder formatHolder = new FormatHolder(); - DecoderInputBuffer buffer = - new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); - int result = stream.readData(formatHolder, buffer, /* readFlags= */ 0); - assertThat(result).isEqualTo(RESULT_BUFFER_READ); + DecoderInputBuffer buffer = readBuffer(stream, formatHolder); + assertThat(buffer.supplementalData).isNull(); } @@ -122,26 +179,18 @@ public void readData_noMetadataAvailable_doesNotAttachSupplementalData() { public void reset_clearsMetadataCache() { configureMockMetadataStream( metadataStream, new MetadataSample(/* timeUs= */ 0, metadataPayload)); - configureVideoStreamToReturnBuffer(videoStream, /* timeUs= */ 0); - + configureVideoStreamToReturnBuffers(videoStream, /* timesUs...= */ 0, 100_000); MergingMetadataSampleStream stream = new MergingMetadataSampleStream(videoStream, metadataStream, VIDEO_FORMAT); FormatHolder formatHolder = new FormatHolder(); - DecoderInputBuffer buffer = - new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); - - int result = stream.readData(formatHolder, buffer, /* readFlags= */ 0); - assertThat(result).isEqualTo(RESULT_BUFFER_READ); + DecoderInputBuffer buffer = readBuffer(stream, formatHolder); assertThat(buffer.supplementalData).isNotNull(); // Reset and read again with no new metadata. stream.reset(); when(metadataStream.readData(any(), any(), anyInt())).thenReturn(RESULT_NOTHING_READ); - configureVideoStreamToReturnBuffer(videoStream, /* timeUs= */ 100_000); - DecoderInputBuffer buffer2 = - new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); - int result2 = stream.readData(formatHolder, buffer2, /* readFlags= */ 0); - assertThat(result2).isEqualTo(RESULT_BUFFER_READ); + DecoderInputBuffer buffer2 = readBuffer(stream, formatHolder); + assertThat(buffer2.supplementalData).isNull(); } @@ -153,40 +202,23 @@ public void readData_decreasingMetadataPts_invalidatesMetadataTrack() { new MetadataSample(/* timeUs= */ 100_000, metadataPayload), new MetadataSample(/* timeUs= */ 120_000, metadataPayload), new MetadataSample(/* timeUs= */ 50_000, otherMetadataPayload)); // Decreasing PTS - + configureVideoStreamToReturnBuffers(videoStream, /* timesUs...= */ 100_000, 120_000, 120_000); MergingMetadataSampleStream stream = new MergingMetadataSampleStream(videoStream, metadataStream, VIDEO_FORMAT); FormatHolder formatHolder = new FormatHolder(); - DecoderInputBuffer buffer = - new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); // Read video frame at 100_000. Metadata at 100_000 should be attached. - configureVideoStreamToReturnBuffer(videoStream, /* timeUs= */ 100_000); - int result1 = stream.readData(formatHolder, buffer, /* readFlags= */ 0); - assertThat(result1).isEqualTo(RESULT_BUFFER_READ); - buffer.flip(); - assertThat(buffer.supplementalData).isNotNull(); - byte[] supplemental1 = new byte[buffer.supplementalData.remaining()]; - buffer.supplementalData.get(supplemental1); - assertThat(supplemental1).isEqualTo(metadataPayload); - + DecoderInputBuffer buffer = readBuffer(stream, formatHolder); // Read video frame at 120_000. This will cause the stream to try and read the next metadata // sample. The next metadata sample has a PTS of 50_000, which is less than the current // buffered metadata (120_000). This should invalidate the metadata track. - configureVideoStreamToReturnBuffer(videoStream, /* timeUs= */ 120_000); - DecoderInputBuffer buffer2 = - new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); - int result2 = stream.readData(formatHolder, buffer2, /* readFlags= */ 0); - assertThat(result2).isEqualTo(RESULT_BUFFER_READ); - assertThat(buffer2.supplementalData).isNull(); // Supplemental data should be null. - + DecoderInputBuffer buffer2 = readBuffer(stream, formatHolder); // Read video frame at 120_000. The metadata track is invalidated, so no supplemental data // should be attached. - configureVideoStreamToReturnBuffer(videoStream, /* timeUs= */ 120_000); - DecoderInputBuffer buffer3 = - new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); - int result3 = stream.readData(formatHolder, buffer3, /* readFlags= */ 0); - assertThat(result3).isEqualTo(RESULT_BUFFER_READ); + DecoderInputBuffer buffer3 = readBuffer(stream, formatHolder); + + assertThat(getSupplementalData(buffer)).isEqualTo(metadataPayload); + assertThat(buffer2.supplementalData).isNull(); // Supplemental data should be null. assertThat(buffer3.supplementalData).isNull(); } @@ -225,19 +257,20 @@ private static final class MetadataSample { } private static void configureMockMetadataStream(SampleStream stream, MetadataSample... samples) { - final int[] callCount = {0}; + AtomicInteger callCount = new AtomicInteger(); when(stream.readData(any(), any(), anyInt())) .thenAnswer( (Answer) invocation -> { - if (callCount[0] >= samples.length) { + int index = callCount.getAndIncrement(); + if (index >= samples.length) { return RESULT_NOTHING_READ; } DecoderInputBuffer buf = invocation.getArgument(1); if (buf == null) { return RESULT_BUFFER_READ; } - MetadataSample sample = samples[callCount[0]++]; + MetadataSample sample = samples[index]; buf.timeUs = sample.timeUs; buf.ensureSpaceForWrite(sample.data.length); buf.data.put(sample.data); @@ -245,16 +278,35 @@ private static void configureMockMetadataStream(SampleStream stream, MetadataSam }); } - private static void configureVideoStreamToReturnBuffer(SampleStream stream, long timeUs) { + private static void configureVideoStreamToReturnBuffers(SampleStream stream, long... timesUs) { + AtomicInteger callCount = new AtomicInteger(); when(stream.readData(any(), any(), anyInt())) .thenAnswer( (Answer) invocation -> { DecoderInputBuffer buf = invocation.getArgument(1); - if (buf != null) { - buf.timeUs = timeUs; + if (buf != null && timesUs.length > 0) { + int index = Math.min(callCount.getAndIncrement(), timesUs.length - 1); + buf.timeUs = timesUs[index]; } return RESULT_BUFFER_READ; }); } + + private static DecoderInputBuffer readBuffer( + MergingMetadataSampleStream stream, FormatHolder formatHolder) { + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + assertThat(stream.readData(formatHolder, buffer, /* readFlags= */ 0)) + .isEqualTo(RESULT_BUFFER_READ); + return buffer; + } + + private static byte[] getSupplementalData(DecoderInputBuffer buffer) { + buffer.flip(); + assertThat(buffer.supplementalData).isNotNull(); + byte[] supplemental = new byte[buffer.supplementalData.remaining()]; + buffer.supplementalData.get(supplemental); + return supplemental; + } } diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ProgressiveMediaPeriodStateMachineTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ProgressiveMediaPeriodStateMachineTest.java new file mode 100644 index 00000000000..e208373ba0a --- /dev/null +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ProgressiveMediaPeriodStateMachineTest.java @@ -0,0 +1,912 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.exoplayer.source; + +import static androidx.media3.exoplayer.source.ProgressiveMediaPeriod.LoadingStateMachine.STATE_CANCELING; +import static androidx.media3.exoplayer.source.ProgressiveMediaPeriod.LoadingStateMachine.STATE_CANCELING_FOR_CLIPPING; +import static androidx.media3.exoplayer.source.ProgressiveMediaPeriod.LoadingStateMachine.STATE_CLIPPED_FINISHED; +import static androidx.media3.exoplayer.source.ProgressiveMediaPeriod.LoadingStateMachine.STATE_DEFERRED_RETRY_PENDING; +import static androidx.media3.exoplayer.source.ProgressiveMediaPeriod.LoadingStateMachine.STATE_ERROR; +import static androidx.media3.exoplayer.source.ProgressiveMediaPeriod.LoadingStateMachine.STATE_FINISHED; +import static androidx.media3.exoplayer.source.ProgressiveMediaPeriod.LoadingStateMachine.STATE_IDLE; +import static androidx.media3.exoplayer.source.ProgressiveMediaPeriod.LoadingStateMachine.STATE_LOADING; +import static com.google.common.truth.Truth.assertThat; + +import androidx.media3.common.C; +import androidx.media3.exoplayer.source.ProgressiveMediaPeriod.LoadingStateMachine; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link ProgressiveMediaPeriod.LoadingStateMachine}. */ +@RunWith(AndroidJUnit4.class) +public class ProgressiveMediaPeriodStateMachineTest { + + private LoadingStateMachine stateMachine; + + @Before + public void setUp() { + stateMachine = new LoadingStateMachine(); + } + + @Test + public void initialState_isIdle() { + int state = stateMachine.getState(); + + assertThat(state).isEqualTo(STATE_IDLE); + assertThat(stateMachine.isPendingReset()).isFalse(); + } + + @Test + public void onStartLoading_idleState_transitionsToLoading() { + stateMachine.onStartLoading( + /* isPrepared= */ false, + /* endPositionUs= */ C.TIME_END_OF_SOURCE, + /* durationUs= */ C.TIME_UNSET, + /* currentExtractedSamplesCount= */ 0); + + assertThat(stateMachine.getState()).isEqualTo(STATE_LOADING); + assertThat(stateMachine.isPendingReset()).isFalse(); + } + + @Test + public void onLoadCompleted_loadingState_transitionsToFinished() { + startLoading(); + + stateMachine.onLoadCompleted(); + + assertThat(stateMachine.getState()).isEqualTo(STATE_FINISHED); + } + + @Test + public void onSeek_loadingState_outsideBuffer_transitionsToCancelingForSeek() { + startLoading(); + + stateMachine.onSeek(/* positionUs= */ 5000, /* canSeekInsideBuffer= */ false); + + assertThat(stateMachine.getState()).isEqualTo(STATE_CANCELING); + assertThat(stateMachine.isPendingReset()).isTrue(); + assertThat(stateMachine.getPendingResetPositionUs()).isEqualTo(5000); + } + + @Test + public void onSeek_loadingState_insideBuffer_remainsInLoadingState() { + startLoading(); + + stateMachine.onSeek(/* positionUs= */ 5000, /* canSeekInsideBuffer= */ true); + + assertThat(stateMachine.getState()).isEqualTo(STATE_LOADING); + assertThat(stateMachine.isPendingReset()).isFalse(); + } + + @Test + public void onLoadCanceled_cancelingForSeekState_transitionsToIdle() { + startLoading(); + stateMachine.onSeek(/* positionUs= */ 5000, /* canSeekInsideBuffer= */ false); + + stateMachine.onLoadCanceled( + /* released= */ false, /* haveSampleQueuesReachedEndTimeUs= */ false); + + assertThat(stateMachine.getState()).isEqualTo(STATE_IDLE); + assertThat(stateMachine.isPendingReset()).isTrue(); + assertThat(stateMachine.getPendingResetPositionUs()).isEqualTo(5000); + } + + @Test + public void onLoadError_isLengthKnown_resumesFromCurrentPosition() { + startLoading(/* currentExtractedSamplesCount= */ 10); + + stateMachine.onLoadError( + /* isLengthKnownOrHasDuration= */ true, + /* isPrepared= */ true, + /* currentExtractedSamplesCount= */ 25); + + assertThat(stateMachine.isDeferredRetryPending()).isFalse(); + assertThat(stateMachine.getState()).isEqualTo(STATE_LOADING); + assertThat(stateMachine.getExtractedSamplesCountAtStartOfLoad()).isEqualTo(25); + } + + @Test + public void onLoadError_unknownLengthAndPrepared_defersRetry() { + startLoading(/* currentExtractedSamplesCount= */ 10); + + stateMachine.onLoadError( + /* isLengthKnownOrHasDuration= */ false, + /* isPrepared= */ true, + /* currentExtractedSamplesCount= */ 10); + + assertThat(stateMachine.isDeferredRetryPending()).isTrue(); + assertThat(stateMachine.getState()).isEqualTo(STATE_DEFERRED_RETRY_PENDING); + } + + @Test + public void onLoadError_unknownLengthAndUnprepared_retriesFromStart() { + stateMachine.onTrackSelection(/* hasPreroll= */ false, /* enabledTrackCount= */ 1); + startLoading(/* currentExtractedSamplesCount= */ 10); + + stateMachine.onLoadError( + /* isLengthKnownOrHasDuration= */ false, + /* isPrepared= */ false, + /* currentExtractedSamplesCount= */ 10); + + assertThat(stateMachine.isDeferredRetryPending()).isFalse(); + assertThat(stateMachine.getState()).isEqualTo(STATE_LOADING); + assertThat(stateMachine.readDiscontinuity(/* currentExtractedSamplesCount= */ 0)) + .isEqualTo(C.TIME_UNSET); + assertThat(stateMachine.getLastSeekPositionUs()).isEqualTo(0); + assertThat(stateMachine.getExtractedSamplesCountAtStartOfLoad()).isEqualTo(0); + } + + @Test + public void onLoadError_unknownLengthAndReadingSuppressed_retriesFromStart() { + startLoading(); + stateMachine.onSeek(/* positionUs= */ 5000, /* canSeekInsideBuffer= */ false); + + stateMachine.onLoadError( + /* isLengthKnownOrHasDuration= */ false, + /* isPrepared= */ true, + /* currentExtractedSamplesCount= */ 0); + + assertThat(stateMachine.isDeferredRetryPending()).isFalse(); + assertThat(stateMachine.getState()).isEqualTo(STATE_LOADING); + assertThat(stateMachine.getLastSeekPositionUs()).isEqualTo(0); + } + + @Test + public void onDeferredRetryStarted_deferredRetryState_transitionsToIdleAndSetsDiscontinuity() { + startLoading(); + stateMachine.onLoadError( + /* isLengthKnownOrHasDuration= */ false, + /* isPrepared= */ true, + /* currentExtractedSamplesCount= */ 0); + + stateMachine.onDeferredRetryStarted(); + + assertThat(stateMachine.getState()).isEqualTo(STATE_IDLE); + assertThat(stateMachine.getPendingResetPositionUs()).isEqualTo(0); + assertThat(stateMachine.readDiscontinuity(/* currentExtractedSamplesCount= */ 1)).isEqualTo(0); + } + + @Test + public void onPrepared_singleTrack_configuresPendingResetPosition() { + stateMachine.onPrepared(/* isSingleTrack= */ true, /* positionUs= */ 1000); + + assertThat(stateMachine.getState()).isEqualTo(STATE_IDLE); + assertThat(stateMachine.isPendingReset()).isTrue(); + assertThat(stateMachine.getPendingResetPositionUs()).isEqualTo(1000); + } + + @Test + public void onPrepared_nonSingleTrack_doesNotConfigurePendingResetPosition() { + stateMachine.onPrepared(/* isSingleTrack= */ false, /* positionUs= */ 1000); + + assertThat(stateMachine.getState()).isEqualTo(STATE_IDLE); + assertThat(stateMachine.isPendingReset()).isFalse(); + assertThat(stateMachine.getPendingResetPositionUs()).isEqualTo(C.TIME_UNSET); + } + + @Test + public void onLoadCompleted_resetsPendingResetPositionUs() { + startLoading(); + stateMachine.onSeek(/* positionUs= */ 5000, /* canSeekInsideBuffer= */ false); + + stateMachine.onLoadCompleted(); + + assertThat(stateMachine.getState()).isEqualTo(STATE_FINISHED); + assertThat(stateMachine.getPendingResetPositionUs()).isEqualTo(C.TIME_UNSET); + assertThat(stateMachine.isPendingReset()).isFalse(); + } + + @Test + public void onSeek_whenAlreadyPendingReset_updatesPendingPosition() { + startLoading(); + stateMachine.onSeek(/* positionUs= */ 5000, /* canSeekInsideBuffer= */ false); + + stateMachine.onSeek(/* positionUs= */ 8000, /* canSeekInsideBuffer= */ false); + + assertThat(stateMachine.getPendingResetPositionUs()).isEqualTo(8000); + assertThat(stateMachine.getLastSeekPositionUs()).isEqualTo(8000); + } + + @Test + public void onSeek_loaderNotLoading_transitionsToIdle() { + stateMachine.onSeek(/* positionUs= */ 3000, /* canSeekInsideBuffer= */ false); + + assertThat(stateMachine.getState()).isEqualTo(STATE_IDLE); + assertThat(stateMachine.isPendingReset()).isTrue(); + assertThat(stateMachine.getPendingResetPositionUs()).isEqualTo(3000); + } + + @Test + public void onSeek_whenDeferredRetryPending_transitionsToIdle() { + startLoading(); + stateMachine.onLoadError( + /* isLengthKnownOrHasDuration= */ false, + /* isPrepared= */ true, + /* currentExtractedSamplesCount= */ 0); + + stateMachine.onSeek(/* positionUs= */ 3000, /* canSeekInsideBuffer= */ false); + + assertThat(stateMachine.getState()).isEqualTo(STATE_IDLE); + assertThat(stateMachine.isPendingReset()).isTrue(); + } + + @Test + public void + onTrackSelection_enabledTracksZeroAndLoading_transitionsToCancelingAndClearsDiscontinuity() { + startLoading(); + stateMachine.onTrackSelection(/* hasPreroll= */ true, /* enabledTrackCount= */ 1); + + stateMachine.onTrackSelection(/* hasPreroll= */ false, /* enabledTrackCount= */ 0); + + assertThat(stateMachine.getState()).isEqualTo(STATE_CANCELING); + assertThat(stateMachine.readDiscontinuity(/* currentExtractedSamplesCount= */ 0)) + .isEqualTo(C.TIME_UNSET); + } + + @Test + public void onTrackSelection_enabledTracksZeroAndNotLoading_transitionsToIdle() { + stateMachine.onTrackSelection(/* hasPreroll= */ false, /* enabledTrackCount= */ 0); + + assertThat(stateMachine.getState()).isEqualTo(STATE_IDLE); + } + + @Test + public void onEndPositionReached_whenLoading_transitionsToCancelingForClipping() { + startLoading(); + + stateMachine.onEndPositionReached(); + + assertThat(stateMachine.getState()).isEqualTo(STATE_CANCELING_FOR_CLIPPING); + assertThat(stateMachine.isCancelingForClipping()).isTrue(); + } + + @Test + public void onEndPositionReached_whenIdle_transitionsToClippedFinished() { + stateMachine.onEndPositionReached(); + + assertThat(stateMachine.getState()).isEqualTo(STATE_CLIPPED_FINISHED); + assertThat(stateMachine.isFinished()).isTrue(); + } + + @Test + public void onEndPositionReached_whenAlreadyCancelingForSeek_remainsInStateCanceling() { + startLoading(); + stateMachine.onSeek(/* positionUs= */ 5000, /* canSeekInsideBuffer= */ false); + + stateMachine.onEndPositionReached(); + + assertThat(stateMachine.getState()).isEqualTo(STATE_CANCELING); + assertThat(stateMachine.isCancelingForClipping()).isFalse(); + } + + @Test + public void onEndPositionReached_withPendingReset_doesNotTransitionToClippedFinished() { + stateMachine.onPrepared(/* isSingleTrack= */ true, /* positionUs= */ 1000); + + stateMachine.onEndPositionReached(); + + assertThat(stateMachine.getState()).isEqualTo(STATE_IDLE); + } + + @Test + public void onEndPositionReached_inErrorState_doesNotTransitionToClippedFinished() { + startLoading(); + stateMachine.onFatalLoadError(); + + stateMachine.onEndPositionReached(); + + assertThat(stateMachine.getState()).isEqualTo(STATE_ERROR); + } + + @Test + public void onFatalLoadError_transitionsToError() { + startLoading(); + + stateMachine.onFatalLoadError(); + + assertThat(stateMachine.getState()).isEqualTo(STATE_ERROR); + } + + @Test + public void onStartLoading_clearsPendingResetPositionUs() { + stateMachine.onPrepared(/* isSingleTrack= */ true, /* positionUs= */ 2000); + + stateMachine.onStartLoading( + /* isPrepared= */ true, + /* endPositionUs= */ C.TIME_END_OF_SOURCE, + /* durationUs= */ C.TIME_UNSET, + /* currentExtractedSamplesCount= */ 5); + + assertThat(stateMachine.getState()).isEqualTo(STATE_LOADING); + assertThat(stateMachine.isPendingReset()).isFalse(); + assertThat(stateMachine.getPendingResetPositionUs()).isEqualTo(C.TIME_UNSET); + assertThat(stateMachine.getExtractedSamplesCountAtStartOfLoad()).isEqualTo(5); + } + + @Test + public void readDiscontinuity_beforeTrackSelection_returnsTimeUnset() { + long discontinuity = stateMachine.readDiscontinuity(/* currentExtractedSamplesCount= */ 0); + + assertThat(discontinuity).isEqualTo(C.TIME_UNSET); + } + + @Test + public void readDiscontinuity_afterTrackSelectionWithPreroll_handlesInitialDiscontinuity() { + stateMachine.onTrackSelection(/* hasPreroll= */ true, /* enabledTrackCount= */ 1); + + long discontinuity = stateMachine.readDiscontinuity(/* currentExtractedSamplesCount= */ 0); + + assertThat(discontinuity).isEqualTo(0); + } + + @Test + public void readDiscontinuity_afterDiscontinuityReadOnce_returnsTimeUnsetOnSubsequentRead() { + stateMachine.onTrackSelection(/* hasPreroll= */ true, /* enabledTrackCount= */ 1); + long unused = stateMachine.readDiscontinuity(/* currentExtractedSamplesCount= */ 0); + + long discontinuity = stateMachine.readDiscontinuity(/* currentExtractedSamplesCount= */ 0); + + assertThat(discontinuity).isEqualTo(C.TIME_UNSET); + } + + @Test + public void readDiscontinuity_usesStreamPrerollFlags_ignoresInitialDiscontinuity() { + stateMachine.onTrackSelection(/* hasPreroll= */ true, /* enabledTrackCount= */ 1); + + stateMachine.setUsesStreamPrerollFlags(); + + assertThat(stateMachine.readDiscontinuity(/* currentExtractedSamplesCount= */ 0)) + .isEqualTo(C.TIME_UNSET); + } + + @Test + public void readDiscontinuity_afterDeferredRetryWithoutProgress_returnsTimeUnset() { + stateMachine.onTrackSelection(/* hasPreroll= */ false, /* enabledTrackCount= */ 1); + startLoading(/* currentExtractedSamplesCount= */ 10); + stateMachine.onLoadError( + /* isLengthKnownOrHasDuration= */ false, + /* isPrepared= */ true, + /* currentExtractedSamplesCount= */ 10); + + stateMachine.onDeferredRetryStarted(); + + assertThat(stateMachine.readDiscontinuity(/* currentExtractedSamplesCount= */ 0)) + .isEqualTo(C.TIME_UNSET); + } + + @Test + public void readDiscontinuity_afterDeferredRetryWithProgress_returnsDiscontinuityOnce() { + stateMachine.onTrackSelection(/* hasPreroll= */ false, /* enabledTrackCount= */ 1); + startLoading(/* currentExtractedSamplesCount= */ 10); + stateMachine.onLoadError( + /* isLengthKnownOrHasDuration= */ false, + /* isPrepared= */ true, + /* currentExtractedSamplesCount= */ 10); + stateMachine.onDeferredRetryStarted(); + + long discontinuity = stateMachine.readDiscontinuity(/* currentExtractedSamplesCount= */ 1); + + assertThat(discontinuity).isEqualTo(0); + } + + @Test + public void canContinueLoading_idleUnprepared_returnsTrue() { + boolean canContinue = + stateMachine.canContinueLoading( + /* isPreparedOrSingleTrack= */ false, /* enabledTrackCount= */ 0); + + assertThat(canContinue).isTrue(); + } + + @Test + public void canContinueLoading_preparedZeroTracks_returnsFalse() { + boolean canContinue = + stateMachine.canContinueLoading( + /* isPreparedOrSingleTrack= */ true, /* enabledTrackCount= */ 0); + + assertThat(canContinue).isFalse(); + } + + @Test + public void canContinueLoading_preparedOneTrack_returnsTrue() { + boolean canContinue = + stateMachine.canContinueLoading( + /* isPreparedOrSingleTrack= */ true, /* enabledTrackCount= */ 1); + + assertThat(canContinue).isTrue(); + } + + @Test + public void canContinueLoading_cancelingState_returnsFalse() { + startLoading(); + stateMachine.onSeek(/* positionUs= */ 5000, /* canSeekInsideBuffer= */ false); + + boolean canContinue = + stateMachine.canContinueLoading( + /* isPreparedOrSingleTrack= */ true, /* enabledTrackCount= */ 1); + + assertThat(canContinue).isFalse(); + } + + @Test + public void canContinueLoading_deferredRetryPending_returnsFalse() { + startLoading(); + stateMachine.onLoadError( + /* isLengthKnownOrHasDuration= */ false, + /* isPrepared= */ true, + /* currentExtractedSamplesCount= */ 0); + + boolean canContinue = + stateMachine.canContinueLoading( + /* isPreparedOrSingleTrack= */ true, /* enabledTrackCount= */ 1); + + assertThat(canContinue).isFalse(); + } + + @Test + public void canContinueLoading_errorState_returnsFalse() { + startLoading(); + stateMachine.onFatalLoadError(); + + boolean canContinue = + stateMachine.canContinueLoading( + /* isPreparedOrSingleTrack= */ true, /* enabledTrackCount= */ 1); + + assertThat(canContinue).isFalse(); + } + + @Test + public void canContinueLoading_finishedState_returnsFalse() { + startLoading(); + stateMachine.onLoadCompleted(); + + boolean canContinue = + stateMachine.canContinueLoading( + /* isPreparedOrSingleTrack= */ true, /* enabledTrackCount= */ 1); + + assertThat(canContinue).isFalse(); + } + + @Test + public void canContinueLoading_clippedFinishedState_returnsFalse() { + stateMachine.onEndPositionReached(); + + boolean canContinue = + stateMachine.canContinueLoading( + /* isPreparedOrSingleTrack= */ true, /* enabledTrackCount= */ 1); + + assertThat(canContinue).isFalse(); + } + + @Test + public void canContinueLoading_cancelingForClippingState_returnsFalse() { + startLoading(); + stateMachine.onEndPositionReached(); + + boolean canContinue = + stateMachine.canContinueLoading( + /* isPreparedOrSingleTrack= */ true, /* enabledTrackCount= */ 1); + + assertThat(canContinue).isFalse(); + } + + @Test + public void canSeekInsideBuffer_loadingAndNotLive_returnsTrue() { + startLoading(); + + boolean canSeek = stateMachine.canSeekInsideBuffer(C.DATA_TYPE_MEDIA); + + assertThat(canSeek).isTrue(); + } + + @Test + public void canSeekInsideBuffer_liveStream_returnsFalse() { + startLoading(); + + boolean canSeek = stateMachine.canSeekInsideBuffer(C.DATA_TYPE_MEDIA_PROGRESSIVE_LIVE); + + assertThat(canSeek).isFalse(); + } + + @Test + public void canSeekInsideBuffer_pendingReset_returnsFalse() { + startLoading(); + stateMachine.onSeek(/* positionUs= */ 5000, /* canSeekInsideBuffer= */ false); + + boolean canSeek = stateMachine.canSeekInsideBuffer(C.DATA_TYPE_MEDIA); + + assertThat(canSeek).isFalse(); + } + + @Test + public void canSeekInsideBuffer_cancelingForClipping_returnsTrue() { + startLoading(); + stateMachine.onEndPositionReached(); + assertThat(stateMachine.getState()).isEqualTo(STATE_CANCELING_FOR_CLIPPING); + + boolean canSeek = stateMachine.canSeekInsideBuffer(C.DATA_TYPE_MEDIA); + + assertThat(canSeek).isTrue(); + } + + @Test + public void isFinished_inIdleState_returnsFalse() { + boolean finished = stateMachine.isFinished(); + + assertThat(finished).isFalse(); + } + + @Test + public void isFinished_inFinishedState_returnsTrue() { + startLoading(); + + stateMachine.onLoadCompleted(); + + assertThat(stateMachine.isFinished()).isTrue(); + } + + @Test + public void isLoading_inLoadingAndCancelingStates_returnsTrue() { + assertThat(stateMachine.isLoading()).isFalse(); + + startLoading(); + assertThat(stateMachine.isLoading()).isTrue(); + + stateMachine.onEndPositionReached(); + assertThat(stateMachine.getState()).isEqualTo(STATE_CANCELING_FOR_CLIPPING); + assertThat(stateMachine.isLoading()).isTrue(); + + stateMachine.onLoadCanceled( + /* released= */ false, /* haveSampleQueuesReachedEndTimeUs= */ true); + assertThat(stateMachine.getState()).isEqualTo(STATE_CLIPPED_FINISHED); + assertThat(stateMachine.isLoading()).isFalse(); + } + + @Test + public void isDeferredRetryPending_inDeferredRetryState_returnsTrue() { + startLoading(); + + stateMachine.onLoadError( + /* isLengthKnownOrHasDuration= */ false, + /* isPrepared= */ true, + /* currentExtractedSamplesCount= */ 0); + + assertThat(stateMachine.isDeferredRetryPending()).isTrue(); + } + + @Test + public void isLastSeekPosition_matchesLastSeekPosition() { + stateMachine.onSeek(/* positionUs= */ 5000, /* canSeekInsideBuffer= */ true); + + boolean matches = stateMachine.isLastSeekPosition(5000); + + assertThat(matches).isTrue(); + assertThat(stateMachine.isLastSeekPosition(1000)).isFalse(); + } + + @Test + public void hasExtractedProgressSinceLoadStart_evaluatesProgress() { + startLoading(/* currentExtractedSamplesCount= */ 10); + + boolean progressMade = stateMachine.hasExtractedProgressSinceLoadStart(11); + + assertThat(progressMade).isTrue(); + assertThat(stateMachine.hasExtractedProgressSinceLoadStart(10)).isFalse(); + } + + @Test + public void onTrackSelection_noPreroll_clearsPendingInitialDiscontinuity() { + stateMachine.onTrackSelection(/* hasPreroll= */ false, /* enabledTrackCount= */ 1); + + long discontinuity = stateMachine.readDiscontinuity(/* currentExtractedSamplesCount= */ 0); + + assertThat(discontinuity).isEqualTo(C.TIME_UNSET); + } + + @Test + public void onTrackSelection_withPreroll_retainsPendingInitialDiscontinuity() { + stateMachine.onTrackSelection(/* hasPreroll= */ true, /* enabledTrackCount= */ 1); + + long discontinuity = stateMachine.readDiscontinuity(/* currentExtractedSamplesCount= */ 0); + + assertThat(discontinuity).isEqualTo(0); + } + + @Test + public void onTrackSelection_subsequentCallsDoNotOverwriteClearedDiscontinuity() { + stateMachine.onTrackSelection(/* hasPreroll= */ false, /* enabledTrackCount= */ 1); + + stateMachine.onTrackSelection(/* hasPreroll= */ true, /* enabledTrackCount= */ 1); + + assertThat(stateMachine.readDiscontinuity(/* currentExtractedSamplesCount= */ 0)) + .isEqualTo(C.TIME_UNSET); + } + + @Test + public void onTrackSelection_tracksCleared_clearsInitialDiscontinuity() { + stateMachine.onTrackSelection(/* hasPreroll= */ true, /* enabledTrackCount= */ 1); + + stateMachine.onTrackSelection(/* hasPreroll= */ false, /* enabledTrackCount= */ 0); + + assertThat(stateMachine.readDiscontinuity(/* currentExtractedSamplesCount= */ 0)) + .isEqualTo(C.TIME_UNSET); + } + + @Test + public void onTrackSelection_zeroTracks_setsSeenFirstTrackSelectionTrue() { + stateMachine.onTrackSelection(/* hasPreroll= */ false, /* enabledTrackCount= */ 0); + + boolean seenFirstSelection = stateMachine.hasSeenFirstTrackSelection(); + + assertThat(seenFirstSelection).isTrue(); + } + + @Test + public void onTrackSelection_nonZeroTracks_setsSeenFirstTrackSelectionTrue() { + stateMachine.onTrackSelection(/* hasPreroll= */ false, /* enabledTrackCount= */ 2); + + boolean seenFirstSelection = stateMachine.hasSeenFirstTrackSelection(); + + assertThat(seenFirstSelection).isTrue(); + } + + @Test + public void onLoadCanceled_cancelingForClippingState_transitionsToClippedFinished() { + startLoading(); + stateMachine.onEndPositionReached(); + + stateMachine.onLoadCanceled( + /* released= */ false, /* haveSampleQueuesReachedEndTimeUs= */ true); + + assertThat(stateMachine.getState()).isEqualTo(STATE_CLIPPED_FINISHED); + assertThat(stateMachine.isFinished()).isTrue(); + } + + @Test + public void onLoadCanceled_loadingState_remainsInLoadingState() { + startLoading(); + + stateMachine.onLoadCanceled( + /* released= */ false, /* haveSampleQueuesReachedEndTimeUs= */ false); + + assertThat(stateMachine.getState()).isEqualTo(STATE_LOADING); + } + + @Test + public void onEndPositionReached_whenFinished_transitionsToClippedFinished() { + startLoading(); + stateMachine.onLoadCompleted(); + assertThat(stateMachine.getState()).isEqualTo(STATE_FINISHED); + + stateMachine.onEndPositionReached(); + + assertThat(stateMachine.getState()).isEqualTo(STATE_CLIPPED_FINISHED); + assertThat(stateMachine.isFinished()).isTrue(); + } + + @Test + public void onLoadCanceled_cancelingForClippingState_notReachedEndTime_transitionsToIdle() { + startLoading(); + stateMachine.onEndPositionReached(); + + stateMachine.onLoadCanceled( + /* released= */ false, /* haveSampleQueuesReachedEndTimeUs= */ false); + + assertThat(stateMachine.getState()).isEqualTo(STATE_IDLE); + assertThat(stateMachine.isFinished()).isFalse(); + } + + @Test + public void onSeek_cancelingForClippingState_convertsToCancelingForSeek() { + startLoading(); + stateMachine.onEndPositionReached(); + + stateMachine.onSeek(/* positionUs= */ 2_000_000, /* canSeekInsideBuffer= */ false); + + assertThat(stateMachine.getState()).isEqualTo(STATE_CANCELING); + assertThat(stateMachine.isCancelingForClipping()).isFalse(); + assertThat(stateMachine.isPendingReset()).isTrue(); + assertThat(stateMachine.getPendingResetPositionUs()).isEqualTo(2_000_000); + } + + @Test + public void onEndPositionExtended_whenClippedFinished_transitionsToIdle() { + stateMachine.onEndPositionReached(); + assertThat(stateMachine.getState()).isEqualTo(STATE_CLIPPED_FINISHED); + + stateMachine.onEndPositionExtended(); + + assertThat(stateMachine.getState()).isEqualTo(STATE_IDLE); + assertThat(stateMachine.isFinished()).isFalse(); + } + + @Test + public void onEndPositionExtended_whenFinished_remainsFinished() { + stateMachine.onLoadCompleted(); + assertThat(stateMachine.getState()).isEqualTo(STATE_FINISHED); + + stateMachine.onEndPositionExtended(); + + assertThat(stateMachine.getState()).isEqualTo(STATE_FINISHED); + assertThat(stateMachine.isFinished()).isTrue(); + } + + @Test + public void onTrackSelection_zeroTracks_whenCancelingForClipping_transitionsToCanceling() { + startLoading(); + stateMachine.onEndPositionReached(); + assertThat(stateMachine.getState()).isEqualTo(STATE_CANCELING_FOR_CLIPPING); + + stateMachine.onTrackSelection(/* hasPreroll= */ false, /* enabledTrackCount= */ 0); + + assertThat(stateMachine.getState()).isEqualTo(STATE_CANCELING); + } + + @Test + public void onTrackSelection_zeroTracks_whenClippedFinished_transitionsToIdle() { + stateMachine.onEndPositionReached(); + assertThat(stateMachine.getState()).isEqualTo(STATE_CLIPPED_FINISHED); + + stateMachine.onTrackSelection(/* hasPreroll= */ false, /* enabledTrackCount= */ 0); + + assertThat(stateMachine.getState()).isEqualTo(STATE_IDLE); + } + + @Test + public void onSeek_outsideBuffer_whenClippedFinished_transitionsToIdle() { + stateMachine.onEndPositionReached(); + assertThat(stateMachine.getState()).isEqualTo(STATE_CLIPPED_FINISHED); + + stateMachine.onSeek(/* positionUs= */ 5000, /* canSeekInsideBuffer= */ false); + + assertThat(stateMachine.getState()).isEqualTo(STATE_IDLE); + assertThat(stateMachine.isPendingReset()).isTrue(); + assertThat(stateMachine.getPendingResetPositionUs()).isEqualTo(5000); + } + + @Test + public void onStartLoading_seekBeyondClippedEnd_transitionsToClippedFinished() { + stateMachine.onSeek(/* positionUs= */ 5000, /* canSeekInsideBuffer= */ false); + assertThat(stateMachine.isPendingReset()).isTrue(); + + stateMachine.onStartLoading( + /* isPrepared= */ true, + /* endPositionUs= */ 3000, + /* durationUs= */ C.TIME_UNSET, + /* currentExtractedSamplesCount= */ 0); + + assertThat(stateMachine.getState()).isEqualTo(STATE_CLIPPED_FINISHED); + assertThat(stateMachine.isFinished()).isTrue(); + assertThat(stateMachine.isPendingReset()).isFalse(); + assertThat(stateMachine.getPendingResetPositionUs()).isEqualTo(C.TIME_UNSET); + assertThat(stateMachine.suppressRead()).isFalse(); + } + + @Test + public void onStartLoading_seekBeyondDuration_transitionsToFinished() { + stateMachine.onSeek(/* positionUs= */ 5000, /* canSeekInsideBuffer= */ false); + assertThat(stateMachine.isPendingReset()).isTrue(); + + stateMachine.onStartLoading( + /* isPrepared= */ true, + /* endPositionUs= */ C.TIME_END_OF_SOURCE, + /* durationUs= */ 3000, + /* currentExtractedSamplesCount= */ 0); + + assertThat(stateMachine.getState()).isEqualTo(STATE_FINISHED); + assertThat(stateMachine.isFinished()).isTrue(); + assertThat(stateMachine.isPendingReset()).isFalse(); + assertThat(stateMachine.getPendingResetPositionUs()).isEqualTo(C.TIME_UNSET); + assertThat(stateMachine.suppressRead()).isFalse(); + } + + @Test + public void onStartLoading_seekWithinLoadableRange_transitionsToLoading() { + stateMachine.onSeek(/* positionUs= */ 2000, /* canSeekInsideBuffer= */ false); + assertThat(stateMachine.isPendingReset()).isTrue(); + + stateMachine.onStartLoading( + /* isPrepared= */ true, + /* endPositionUs= */ 3000, + /* durationUs= */ C.TIME_UNSET, + /* currentExtractedSamplesCount= */ 0); + + assertThat(stateMachine.getState()).isEqualTo(STATE_LOADING); + assertThat(stateMachine.isFinished()).isFalse(); + assertThat(stateMachine.isPendingReset()).isFalse(); + assertThat(stateMachine.getPendingResetPositionUs()).isEqualTo(C.TIME_UNSET); + } + + @Test + public void canSeekInsideBuffer_clippedFinished_returnsTrue() { + stateMachine.onEndPositionReached(); + assertThat(stateMachine.getState()).isEqualTo(STATE_CLIPPED_FINISHED); + + assertThat(stateMachine.canSeekInsideBuffer(C.DATA_TYPE_MEDIA)).isTrue(); + } + + @Test + public void canContinueLoading_cancelingForClipping_returnsFalse() { + startLoading(); + stateMachine.onEndPositionReached(); + assertThat(stateMachine.getState()).isEqualTo(STATE_CANCELING_FOR_CLIPPING); + + assertThat( + stateMachine.canContinueLoading( + /* isPreparedOrSingleTrack= */ true, /* enabledTrackCount= */ 1)) + .isFalse(); + } + + @Test + public void canContinueLoading_clippedFinished_returnsFalse() { + stateMachine.onEndPositionReached(); + assertThat(stateMachine.getState()).isEqualTo(STATE_CLIPPED_FINISHED); + + assertThat( + stateMachine.canContinueLoading( + /* isPreparedOrSingleTrack= */ true, /* enabledTrackCount= */ 1)) + .isFalse(); + } + + @Test + public void onFatalLoadError_whenCancelingForClipping_transitionsToError() { + startLoading(); + stateMachine.onEndPositionReached(); + assertThat(stateMachine.getState()).isEqualTo(STATE_CANCELING_FOR_CLIPPING); + + stateMachine.onFatalLoadError(); + + assertThat(stateMachine.getState()).isEqualTo(STATE_ERROR); + } + + @Test + public void onEndPositionReached_whenDeferredRetryPending_transitionsToClippedFinished() { + startLoading(); + stateMachine.onLoadError( + /* isLengthKnownOrHasDuration= */ false, + /* isPrepared= */ true, + /* currentExtractedSamplesCount= */ 5); + assertThat(stateMachine.getState()).isEqualTo(STATE_DEFERRED_RETRY_PENDING); + + stateMachine.onEndPositionReached(); + + assertThat(stateMachine.getState()).isEqualTo(STATE_CLIPPED_FINISHED); + } + + @Test + public void onEndPositionReached_whenError_remainsError() { + stateMachine.onFatalLoadError(); + assertThat(stateMachine.getState()).isEqualTo(STATE_ERROR); + + stateMachine.onEndPositionReached(); + + assertThat(stateMachine.getState()).isEqualTo(STATE_ERROR); + } + + private void startLoading() { + startLoading(/* currentExtractedSamplesCount= */ 0); + } + + private void startLoading(int currentExtractedSamplesCount) { + stateMachine.onStartLoading( + /* isPrepared= */ false, + /* endPositionUs= */ C.TIME_END_OF_SOURCE, + /* durationUs= */ C.TIME_UNSET, + currentExtractedSamplesCount); + } +} diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ProgressiveMediaPeriodTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ProgressiveMediaPeriodTest.java index 64516e6bb99..ef2d5e9676a 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ProgressiveMediaPeriodTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ProgressiveMediaPeriodTest.java @@ -16,19 +16,28 @@ package androidx.media3.exoplayer.source; import static android.os.Build.VERSION.SDK_INT; +import static androidx.media3.common.util.Util.createHandlerForCurrentLooper; import static androidx.media3.test.utils.robolectric.RobolectricUtil.runMainLooperUntil; import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; import static org.junit.Assume.assumeTrue; +import static org.robolectric.Shadows.shadowOf; import android.net.Uri; +import android.os.Looper; import androidx.annotation.Nullable; import androidx.media3.common.C; import androidx.media3.common.DataReader; import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; +import androidx.media3.common.ParserException; +import androidx.media3.common.util.ConditionVariable; import androidx.media3.common.util.Consumer; import androidx.media3.common.util.NullableType; import androidx.media3.datasource.AssetDataSource; +import androidx.media3.datasource.DataSource; +import androidx.media3.datasource.DataSpec; +import androidx.media3.datasource.TransferListener; import androidx.media3.decoder.DecoderInputBuffer; import androidx.media3.exoplayer.FormatHolder; import androidx.media3.exoplayer.LoadingInfo; @@ -39,6 +48,8 @@ import androidx.media3.exoplayer.trackselection.ExoTrackSelection; import androidx.media3.exoplayer.upstream.DefaultAllocator; import androidx.media3.exoplayer.upstream.DefaultLoadErrorHandlingPolicy; +import androidx.media3.exoplayer.upstream.LoadErrorHandlingPolicy; +import androidx.media3.exoplayer.upstream.Loader; import androidx.media3.exoplayer.util.ReleasableExecutor; import androidx.media3.extractor.DefaultExtractorsFactory; import androidx.media3.extractor.Extractor; @@ -46,18 +57,24 @@ import androidx.media3.extractor.ExtractorsFactory; import androidx.media3.extractor.PositionHolder; import androidx.media3.extractor.SeekMap; +import androidx.media3.extractor.SeekPoint; +import androidx.media3.extractor.TrackAwareSeekMap; import androidx.media3.extractor.mp4.Mp4Extractor; import androidx.media3.extractor.png.PngExtractor; import androidx.media3.extractor.text.SubtitleParser; import androidx.media3.test.utils.FakeTrackSelection; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.io.IOException; +import java.util.ArrayList; +import java.util.Arrays; import java.util.List; import java.util.Map; import java.util.concurrent.Executor; import java.util.concurrent.Executors; import java.util.concurrent.TimeoutException; import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicInteger; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.annotation.Config; @@ -126,7 +143,2480 @@ public void readData_forUnselectedTrack_returnsFormatAndNothingElse() throws Exc @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; boolean[] streamResetFlags = new boolean[trackGroups.length]; - // Select only track 0. + // Select only track 1 (audio). Track 0 (video) is left unselected. + selections[1] = + new FakeTrackSelection(trackGroups.get(1), new int[] {0}, /* selectedIndex= */ 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + // Run loader until source has finished loading. + boolean unusedResult = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil(() -> mediaPeriod.getBufferedPositionUs() == C.TIME_END_OF_SOURCE); + + // Read from stream 1 (selected audio) to check we successfully read the format and samples. + assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 1, buffer)) + .isEqualTo(C.RESULT_FORMAT_READ); + assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 1, buffer)) + .isEqualTo(C.RESULT_BUFFER_READ); + assertThat(buffer.isEndOfStream()).isFalse(); + // Read from stream 0 (unselected video) to check we get no samples (or at most in-flight + // samples from before track selection). + assertThat(readProgressiveStreamUntilEndOfStream(mediaPeriod, /* trackIndex= */ 0, buffer)) + .isAtMost(1); + mediaPeriod.release(); + } + + @Test + public void readData_unselectedTextTrack_returnsSamples() throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample_with_vobsub.mp4")); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + assertThat(trackGroups.length).isEqualTo(3); + assertThat(trackGroups.get(0).type).isEqualTo(C.TRACK_TYPE_VIDEO); + assertThat(trackGroups.get(1).type).isEqualTo(C.TRACK_TYPE_AUDIO); + assertThat(trackGroups.get(2).type).isEqualTo(C.TRACK_TYPE_TEXT); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[1] = + new FakeTrackSelection(trackGroups.get(1), new int[] {0}, /* selectedIndex= */ 0); + + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + assertThat(streams[0]).isNull(); + assertThat(streams[1]).isNotNull(); + assertThat(streams[2]).isNull(); + boolean unusedResult = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil(() -> mediaPeriod.getBufferedPositionUs() == C.TIME_END_OF_SOURCE); + + // Audio track (selected) returns samples. + assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 1, buffer)) + .isEqualTo(C.RESULT_FORMAT_READ); + assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 1, buffer)) + .isEqualTo(C.RESULT_BUFFER_READ); + assertThat(buffer.isEndOfStream()).isFalse(); + // Text track (unselected) returns samples because audio, text, and metadata tracks are always + // loaded. + assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 2, buffer)) + .isEqualTo(C.RESULT_FORMAT_READ); + assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 2, buffer)) + .isEqualTo(C.RESULT_BUFFER_READ); + assertThat(buffer.isEndOfStream()).isFalse(); + // Video track (unselected) returns no samples (or at most in-flight samples from before track + // selection). + assertThat(readProgressiveStreamUntilEndOfStream(mediaPeriod, /* trackIndex= */ 0, buffer)) + .isAtMost(1); + mediaPeriod.release(); + } + + @Test + public void readData_unselectedAudioAndTextTracks_returnsSamples() throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample_with_vobsub.mp4")); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + assertThat(trackGroups.length).isEqualTo(3); + assertThat(trackGroups.get(0).type).isEqualTo(C.TRACK_TYPE_VIDEO); + assertThat(trackGroups.get(1).type).isEqualTo(C.TRACK_TYPE_AUDIO); + assertThat(trackGroups.get(2).type).isEqualTo(C.TRACK_TYPE_TEXT); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + assertThat(streams[0]).isNotNull(); + assertThat(streams[1]).isNull(); + assertThat(streams[2]).isNull(); + boolean unusedResult = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil(() -> mediaPeriod.getBufferedPositionUs() == C.TIME_END_OF_SOURCE); + + // Video track (selected) returns samples. + assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 0, buffer)) + .isEqualTo(C.RESULT_FORMAT_READ); + assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 0, buffer)) + .isEqualTo(C.RESULT_BUFFER_READ); + assertThat(buffer.isEndOfStream()).isFalse(); + // Audio track (unselected) returns samples because audio tracks are always loaded. + assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 1, buffer)) + .isEqualTo(C.RESULT_FORMAT_READ); + assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 1, buffer)) + .isEqualTo(C.RESULT_BUFFER_READ); + assertThat(buffer.isEndOfStream()).isFalse(); + // Text track (unselected) returns samples because text tracks are always loaded. + assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 2, buffer)) + .isEqualTo(C.RESULT_FORMAT_READ); + assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 2, buffer)) + .isEqualTo(C.RESULT_BUFFER_READ); + assertThat(buffer.isEndOfStream()).isFalse(); + mediaPeriod.release(); + } + + @Test + public void selectTracks_disablingAllTracksAfterFatalError_clearsFatalError() throws Exception { + FatalErrorDataSource dataSource = + new FatalErrorDataSource(new AssetDataSource(ApplicationProvider.getApplicationContext())); + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod( + Uri.parse("asset://android_asset/media/mp4/sample.mp4"), + dataSource, + new DefaultLoadErrorHandlingPolicy()); + + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + + // Select a track so continueLoading starts sample loading. + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + // Wait until initial loading is complete before testing fatal error behavior. + runMainLooperUntil(() -> !mediaPeriod.isLoading()); + + // Reset track selection when idle so continueLoading can start a fresh load. + Arrays.fill(selections, null); + unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + + dataSource.setThrowErrorAfterPrepare(true); + unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil( + () -> { + try { + mediaPeriod.maybeThrowPrepareError(); + return false; + } catch (IOException e) { + return true; + } + }); + assertThrows(ParserException.class, mediaPeriod::maybeThrowPrepareError); + assertThat( + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build())) + .isFalse(); + + // Disabling all tracks clears the fatal error state on the loader. + Arrays.fill(selections, null); + unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + mediaPeriod.maybeThrowPrepareError(); + dataSource.setThrowErrorAfterPrepare(false); + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + + assertThat( + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build())) + .isTrue(); + mediaPeriod.release(); + } + + @Test + public void seekToUs_afterFatalError_clearsFatalError() throws Exception { + FatalErrorDataSource dataSource = + new FatalErrorDataSource(new AssetDataSource(ApplicationProvider.getApplicationContext())); + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod( + Uri.parse("asset://android_asset/media/mp4/sample.mp4"), + dataSource, + new DefaultLoadErrorHandlingPolicy()); + + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + + // Select a track so continueLoading starts sample loading. + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + // Wait until initial loading is complete before testing fatal error behavior. + runMainLooperUntil(() -> !mediaPeriod.isLoading()); + + // Reset track selection when idle so continueLoading can start a fresh load. + Arrays.fill(selections, null); + unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + + dataSource.setThrowErrorAfterPrepare(true); + unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil( + () -> { + try { + mediaPeriod.maybeThrowPrepareError(); + return false; + } catch (IOException e) { + return true; + } + }); + assertThrows(ParserException.class, mediaPeriod::maybeThrowPrepareError); + assertThat( + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build())) + .isFalse(); + + // Seeking outside the buffer clears the fatal error state on the loader. + long unusedSeek = mediaPeriod.seekToUs(1000); + mediaPeriod.maybeThrowPrepareError(); + dataSource.setThrowErrorAfterPrepare(false); + + assertThat( + mediaPeriod.continueLoading( + new LoadingInfo.Builder().setPlaybackPositionUs(1000).build())) + .isTrue(); + mediaPeriod.release(); + } + + @Test + public void selectTracks_disablingAllTracksWhenIdle_resetsLoadingFinished() throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); + + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + + // Select a track so continueLoading starts sample loading. + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil(() -> !mediaPeriod.isLoading()); + + // When idle (loading finished), continueLoading returns false. + assertThat( + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build())) + .isFalse(); + + // Disabling all tracks when idle resets loadingFinished. + Arrays.fill(selections, null); + unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + + assertThat( + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build())) + .isTrue(); + mediaPeriod.release(); + } + + @Test + public void selectTracks_disablingAllTracksWhenLoading_cancelsLoading() throws Exception { + ConditionVariable blockLoadingCondition = new ConditionVariable(); + ConditionVariable loadingStartedCondition = new ConditionVariable(); + blockLoadingCondition.open(); + AssetDataSource assetDataSource = + new AssetDataSource(ApplicationProvider.getApplicationContext()); + DataSource dataSource = + new DataSource() { + @Override + public void addTransferListener(TransferListener transferListener) { + assetDataSource.addTransferListener(transferListener); + } + + @Override + public long open(DataSpec dataSpec) throws IOException { + return assetDataSource.open(dataSpec); + } + + @Override + public int read(byte[] buffer, int offset, int length) throws IOException { + loadingStartedCondition.open(); + blockLoadingCondition.blockUninterruptible(); + return assetDataSource.read(buffer, offset, length); + } + + @Nullable + @Override + public Uri getUri() { + return assetDataSource.getUri(); + } + + @Override + public Map> getResponseHeaders() { + return assetDataSource.getResponseHeaders(); + } + + @Override + public void close() throws IOException { + assetDataSource.close(); + } + }; + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod( + Uri.parse("asset://android_asset/media/mp4/sample.mp4"), + dataSource, + new DefaultLoadErrorHandlingPolicy()); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[1] = + new FakeTrackSelection(trackGroups.get(1), new int[] {0}, /* selectedIndex= */ 0); + + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + blockLoadingCondition.close(); + assertThat( + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build())) + .isTrue(); + loadingStartedCondition.block(); + assertThat(mediaPeriod.isLoading()).isTrue(); + + // Disabling all tracks while loading starts canceling. + Arrays.fill(selections, null); + unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + + assertThat( + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build())) + .isFalse(); + + blockLoadingCondition.open(); + runMainLooperUntil(() -> !mediaPeriod.isLoading()); + selections[1] = + new FakeTrackSelection(trackGroups.get(1), new int[] {0}, /* selectedIndex= */ 0); + unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + + assertThat( + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build())) + .isTrue(); + mediaPeriod.release(); + } + + @Test + public void getTrackGroups_multipleTracks_setsExpectedPrimaryTrackGroupId() throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + + assertThat(trackGroups.length).isEqualTo(2); + assertThat(trackGroups.get(0).type).isEqualTo(C.TRACK_TYPE_VIDEO); + assertThat(trackGroups.get(1).type).isEqualTo(C.TRACK_TYPE_AUDIO); + assertThat(trackGroups.get(0).getFormat(0).primaryTrackGroupId).isNull(); + assertThat(trackGroups.get(1).getFormat(0).primaryTrackGroupId) + .isEqualTo(trackGroups.get(0).id); + + mediaPeriod = + createMediaPeriod( + Uri.parse("asset://android_asset/media/mp4/h265_with_metadata_track.mp4")); + trackGroups = mediaPeriod.getTrackGroups(); + + assertThat(trackGroups.length).isEqualTo(3); + assertThat(trackGroups.get(0).type).isEqualTo(C.TRACK_TYPE_METADATA); + assertThat(trackGroups.get(1).type).isEqualTo(C.TRACK_TYPE_AUDIO); + assertThat(trackGroups.get(2).type).isEqualTo(C.TRACK_TYPE_VIDEO); + assertThat(trackGroups.get(0).getFormat(0).primaryTrackGroupId) + .isEqualTo(trackGroups.get(2).id); + assertThat(trackGroups.get(1).getFormat(0).primaryTrackGroupId) + .isEqualTo(trackGroups.get(2).id); + assertThat(trackGroups.get(2).getFormat(0).primaryTrackGroupId).isNull(); + + mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample_with_vobsub.mp4")); + trackGroups = mediaPeriod.getTrackGroups(); + + assertThat(trackGroups.length).isEqualTo(3); + assertThat(trackGroups.get(0).type).isEqualTo(C.TRACK_TYPE_VIDEO); + assertThat(trackGroups.get(1).type).isEqualTo(C.TRACK_TYPE_AUDIO); + assertThat(trackGroups.get(2).type).isEqualTo(C.TRACK_TYPE_TEXT); + assertThat(trackGroups.get(0).getFormat(0).primaryTrackGroupId).isNull(); + assertThat(trackGroups.get(1).getFormat(0).primaryTrackGroupId) + .isEqualTo(trackGroups.get(0).id); + assertThat(trackGroups.get(2).getFormat(0).primaryTrackGroupId) + .isEqualTo(trackGroups.get(0).id); + + mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/ts/sample_with_id3.adts")); + trackGroups = mediaPeriod.getTrackGroups(); + + assertThat(trackGroups.length).isEqualTo(2); + assertThat(trackGroups.get(0).type).isEqualTo(C.TRACK_TYPE_AUDIO); + assertThat(trackGroups.get(1).type).isEqualTo(C.TRACK_TYPE_METADATA); + assertThat(trackGroups.get(0).getFormat(0).primaryTrackGroupId).isNull(); + assertThat(trackGroups.get(1).getFormat(0).primaryTrackGroupId) + .isEqualTo(trackGroups.get(0).id); + } + + @Test + public void selectTracks_joiningStreamWithPrerollSamples_doesNotIncurExtraDiscontinuity() + throws Exception { + ProgressiveMediaExtractor extractor = + new ProgressiveMediaExtractor() { + @Override + public void init( + DataReader dataReader, + Uri uri, + Map> responseHeaders, + long position, + long length, + ExtractorOutput output) { + // Provide custom non-sync formats and a SeekMap whose first seek point precedes 0 to + // indicate pre-roll samples. + output + .track(0, C.TRACK_TYPE_AUDIO) + .format( + new Format.Builder() + .setSampleMimeType(MimeTypes.AUDIO_AAC) + .setCodecs("mp4a.40.42") + .build()); + output + .track(1, C.TRACK_TYPE_VIDEO) + .format(new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build()); + output.endTracks(); + output.seekMap( + new SeekMap() { + @Override + public boolean isSeekable() { + return true; + } + + @Override + public long getDurationUs() { + return C.TIME_UNSET; + } + + @Override + public SeekPoints getSeekPoints(long timeUs) { + return new SeekPoints(new SeekPoint(/* timeUs= */ -100_000, /* position= */ 0)); + } + }); + } + + @Override + public void release() {} + + @Override + public void disableSeekingOnMp3Streams() {} + + @Override + public long getCurrentInputPosition() { + return 0; + } + + @Override + public void seek(long position, long timeUs) {} + + @Override + public int read(PositionHolder positionHolder) { + return Extractor.RESULT_END_OF_INPUT; + } + }; + + // Create the period using the custom extractor. + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod( + Uri.parse("asset://android_asset/media/mp4/sample.mp4"), + extractor, + /* imageDurationUs= */ C.TIME_UNSET, + /* executor= */ null, + /* executorReleased= */ null); + + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + + // 1. Initially select only the audio track (track 0). + selections[0] = new FakeTrackSelection(trackGroups.get(0), 0); + long unused = + mediaPeriod.selectTracks( + selections, + /* mayRetainStreamFlags= */ new boolean[2], + streams, + /* streamResetFlags= */ new boolean[2], + /* positionUs= */ 0); + + // Verify initial discontinuity and stream preroll flag for first track selection. + assertThat(streams[0].getFlags()).isEqualTo(SampleStream.FLAG_HAS_PREROLL); + assertThat(mediaPeriod.readDiscontinuity()).isEqualTo(0); + + // 2. Simulate seeking to a non-zero position. + long seekPositionUs = 5 * C.MICROS_PER_SECOND; + unused = mediaPeriod.seekToUs(seekPositionUs); + + // 3. Join the video stream (track 1) at the non-zero position. + selections[1] = new FakeTrackSelection(trackGroups.get(1), 0); + unused = + mediaPeriod.selectTracks( + selections, + /* mayRetainStreamFlags= */ new boolean[2], + streams, + /* streamResetFlags= */ new boolean[2], + /* positionUs= */ seekPositionUs); + + // Verify no extra discontinuity when selecting the second stream, while the joined stream + // reports its own preroll flag. + assertThat(streams[1].getFlags()).isEqualTo(SampleStream.FLAG_HAS_PREROLL); + assertThat(mediaPeriod.readDiscontinuity()).isEqualTo(C.TIME_UNSET); + + mediaPeriod.release(); + } + + @Test + public void selectTracks_mp4WithEditListPreroll_reportsPrerollForVideoAndZeroForSyncAudio() + throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample_edit_list.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + selections[0] = new FakeTrackSelection(trackGroups.get(0), 0); + selections[1] = new FakeTrackSelection(trackGroups.get(1), 0); + + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + new boolean[trackGroups.length], + /* positionUs= */ 0); + + assertThat(streams[0].getFlags()).isEqualTo(SampleStream.FLAG_HAS_PREROLL); + assertThat(streams[1].getFlags()).isEqualTo(0); + assertThat(mediaPeriod.readDiscontinuity()).isEqualTo(0); + mediaPeriod.release(); + } + + @Test + public void selectTracks_withHagcTrack_createsMergingMetadataSampleStream() throws Exception { + assumeTrue("Skipping HAGC test on SDK < 37", SDK_INT >= 37); + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample_with_it35_track.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + SampleStream[] streams = new SampleStream[trackGroups.length]; + // Select the video track (which is track 0 in sample_with_it35_track.mp4) + selections[0] = new FakeTrackSelection(trackGroups.get(0), 0); + + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + new boolean[trackGroups.length], + /* positionUs= */ 0); + + assertThat(streams[0]).isInstanceOf(MergingMetadataSampleStream.class); + mediaPeriod.release(); + } + + @Test + public void selectTracks_midGopPositionOnVideoTrack_returnsFlagHasPreroll() throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + // Select video track (track 0). + selections[0] = new FakeTrackSelection(trackGroups.get(0), 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + new boolean[trackGroups.length], + /* positionUs= */ 500_000); + + assertThat(streams[0].getFlags()).isEqualTo(SampleStream.FLAG_HAS_PREROLL); + mediaPeriod.release(); + } + + @Test + public void selectTracks_onSyncAudioTrack_returnsZero() throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + // Select audio track (track 1), which has AAC sync samples. + selections[1] = new FakeTrackSelection(trackGroups.get(1), 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + new boolean[trackGroups.length], + /* positionUs= */ 500_000); + + // Sync audio discards samples before start time and never delivers preroll. + assertThat(streams[1].getFlags()).isEqualTo(0); + mediaPeriod.release(); + } + + @Test + public void selectTracks_withSeekMap_keyframeMatchesSeekPosition_returnsZero() throws Exception { + ProgressiveMediaExtractor extractor = + new ProgressiveMediaExtractor() { + @Override + public void init( + DataReader dataReader, + Uri uri, + Map> responseHeaders, + long position, + long length, + ExtractorOutput output) { + output + .track(0, C.TRACK_TYPE_VIDEO) + .format(new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build()); + output.endTracks(); + // SeekMap with keyframes at 0 and 1_000_000 Us. + output.seekMap( + new SeekMap() { + @Override + public boolean isSeekable() { + return true; + } + + @Override + public long getDurationUs() { + return 2_000_000; + } + + @Override + public SeekPoints getSeekPoints(long timeUs) { + SeekPoint seekPoint = + timeUs >= 1_000_000 ? new SeekPoint(1_000_000, 100) : new SeekPoint(0, 0); + return new SeekPoints(seekPoint); + } + }); + } + + @Override + public void release() {} + + @Override + public void disableSeekingOnMp3Streams() {} + + @Override + public long getCurrentInputPosition() { + return 0; + } + + @Override + public void seek(long position, long timeUs) {} + + @Override + public int read(PositionHolder positionHolder) { + return Extractor.RESULT_END_OF_INPUT; + } + }; + + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod( + Uri.parse("asset://android_asset/media/mp4/sample.mp4"), + extractor, + /* imageDurationUs= */ C.TIME_UNSET, + /* executor= */ null, + /* executorReleased= */ null); + + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + selections[0] = new FakeTrackSelection(trackGroups.get(0), 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + new boolean[trackGroups.length], + /* positionUs= */ 1_000_000); + + // Exact keyframe (1_000_000) results in seek point at 1_000_000 == 1_000_000 -> no preroll. + assertThat(streams[0].getFlags()).isEqualTo(0); + + mediaPeriod.release(); + } + + @Test + public void selectTracks_withTrackAwareSeekMap_reportsPrerollForTrack() throws Exception { + ProgressiveMediaExtractor extractor = + new ProgressiveMediaExtractor() { + @Override + public void init( + DataReader dataReader, + Uri uri, + Map> responseHeaders, + long position, + long length, + ExtractorOutput output) { + output + .track(0, C.TRACK_TYPE_VIDEO) + .format(new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build()); + output.endTracks(); + output.seekMap( + new TrackAwareSeekMap() { + @Override + public boolean isSeekable() { + return true; + } + + @Override + public boolean isSeekable(int trackId) { + return trackId == 0; + } + + @Override + public long getDurationUs() { + return 2_000_000; + } + + @Override + public SeekPoints getSeekPoints(long timeUs) { + return new SeekPoints(new SeekPoint(0, 0)); + } + + @Override + public SeekPoints getSeekPoints(long timeUs, int trackId) { + SeekPoint seekPoint = + timeUs >= 1_000_000 ? new SeekPoint(1_000_000, 100) : new SeekPoint(0, 0); + return new SeekPoints(seekPoint); + } + }); + } + + @Override + public void release() {} + + @Override + public void disableSeekingOnMp3Streams() {} + + @Override + public long getCurrentInputPosition() { + return 0; + } + + @Override + public void seek(long position, long timeUs) {} + + @Override + public int read(PositionHolder positionHolder) { + return Extractor.RESULT_END_OF_INPUT; + } + }; + + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod( + Uri.parse("asset://android_asset/media/mp4/sample.mp4"), + extractor, + /* imageDurationUs= */ C.TIME_UNSET, + /* executor= */ null, + /* executorReleased= */ null); + + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + selections[0] = new FakeTrackSelection(trackGroups.get(0), 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + new boolean[trackGroups.length], + /* positionUs= */ 500_000); + + // Mid-GOP (500_000) results in seek point at 0 < 500_000 -> preroll. + assertThat(streams[0].getFlags()).isEqualTo(SampleStream.FLAG_HAS_PREROLL); + + mediaPeriod.release(); + } + + @Test + public void selectTracks_toNegativePosition_evaluatesPrerollFromSeekMap() throws Exception { + ProgressiveMediaExtractor extractor = + new ProgressiveMediaExtractor() { + @Override + public void init( + DataReader dataReader, + Uri uri, + Map> responseHeaders, + long position, + long length, + ExtractorOutput output) { + output + .track(0, C.TRACK_TYPE_VIDEO) + .format(new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build()); + output.endTracks(); + // SeekMap with keyframes at -1_000_000 and -500_000 Us. + output.seekMap( + new SeekMap() { + @Override + public boolean isSeekable() { + return true; + } + + @Override + public long getDurationUs() { + return 2_000_000; + } + + @Override + public SeekPoints getSeekPoints(long timeUs) { + SeekPoint seekPoint = + timeUs >= -500_000 + ? new SeekPoint(-500_000, 100) + : new SeekPoint(-1_000_000, 0); + return new SeekPoints(seekPoint); + } + }); + } + + @Override + public void release() {} + + @Override + public void disableSeekingOnMp3Streams() {} + + @Override + public long getCurrentInputPosition() { + return 0; + } + + @Override + public void seek(long position, long timeUs) {} + + @Override + public int read(PositionHolder positionHolder) { + return Extractor.RESULT_END_OF_INPUT; + } + }; + + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod( + Uri.parse("asset://android_asset/media/mp4/sample.mp4"), + extractor, + /* imageDurationUs= */ C.TIME_UNSET, + /* executor= */ null, + /* executorReleased= */ null); + + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + selections[0] = new FakeTrackSelection(trackGroups.get(0), 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + new boolean[trackGroups.length], + /* positionUs= */ -250_000); + + // Position -250_000 results in seek point at -500_000 < -250_000 -> preroll. + assertThat(streams[0].getFlags()).isEqualTo(SampleStream.FLAG_HAS_PREROLL); + + mediaPeriod.release(); + } + + @Test + public void selectTracks_switchingToTrackOfSameType_retainsBuffer() throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample_ac3_aac.mp4")); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + assertThat(trackGroups.length).isEqualTo(2); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + boolean unusedResult = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil(() -> mediaPeriod.getBufferedPositionUs() == C.TIME_END_OF_SOURCE); + + assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 0, buffer)) + .isEqualTo(C.RESULT_FORMAT_READ); + assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 0, buffer)) + .isEqualTo(C.RESULT_BUFFER_READ); + assertThat(buffer.isEndOfStream()).isFalse(); + + selections[0] = null; + selections[1] = + new FakeTrackSelection(trackGroups.get(1), new int[] {0}, /* selectedIndex= */ 0); + unused = + mediaPeriod.selectTracks( + selections, + new boolean[] {true, false}, + streams, + streamResetFlags, + /* positionUs= */ 0); + + assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 1, buffer)) + .isEqualTo(C.RESULT_FORMAT_READ); + assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 1, buffer)) + .isEqualTo(C.RESULT_BUFFER_READ); + assertThat(buffer.isEndOfStream()).isFalse(); + mediaPeriod.release(); + } + + private static @SampleStream.ReadDataResult int readProgressiveStream( + ProgressiveMediaPeriod mediaPeriod, int trackIndex, DecoderInputBuffer buffer) { + return mediaPeriod.readData(trackIndex, new FormatHolder(), buffer, /* readFlags= */ 0); + } + + private static int readProgressiveStreamUntilEndOfStream( + ProgressiveMediaPeriod mediaPeriod, int trackIndex, DecoderInputBuffer buffer) { + int readResult; + int sampleCount = 0; + do { + readResult = readProgressiveStream(mediaPeriod, trackIndex, buffer); + if (readResult == C.RESULT_BUFFER_READ && !buffer.isEndOfStream()) { + sampleCount++; + } + } while (readResult != C.RESULT_BUFFER_READ || !buffer.isEndOfStream()); + return sampleCount; + } + + private static ProgressiveMediaPeriod createMediaPeriod( + ProgressiveMediaExtractor extractor, + long imageDurationUs, + @Nullable Executor executor, + @Nullable Consumer executorReleased) + throws TimeoutException { + return createMediaPeriod( + Uri.parse("asset://android_asset/media/mp4/sample.mp4"), + extractor, + imageDurationUs, + executor, + executorReleased); + } + + private static ProgressiveMediaPeriod createMediaPeriod(Uri uri) throws TimeoutException { + return createMediaPeriod( + uri, + new AssetDataSource(ApplicationProvider.getApplicationContext()), + new DefaultLoadErrorHandlingPolicy()); + } + + private static ProgressiveMediaPeriod createMediaPeriod( + Uri uri, + ProgressiveMediaExtractor extractor, + long imageDurationUs, + @Nullable Executor executor, + @Nullable Consumer executorReleased) + throws TimeoutException { + return createMediaPeriod( + uri, + new AssetDataSource(ApplicationProvider.getApplicationContext()), + new DefaultLoadErrorHandlingPolicy(), + extractor, + imageDurationUs, + executor, + executorReleased); + } + + private static ProgressiveMediaPeriod createMediaPeriod( + Uri uri, DataSource dataSource, LoadErrorHandlingPolicy loadErrorHandlingPolicy) + throws TimeoutException { + return createMediaPeriod( + uri, + dataSource, + loadErrorHandlingPolicy, + new BundledExtractorsAdapter(new DefaultExtractorsFactory()), + /* imageDurationUs= */ C.TIME_UNSET, + /* executor= */ null, + /* executorReleased= */ null); + } + + private static ProgressiveMediaPeriod createMediaPeriod(Uri uri, MediaPeriod.Callback callback) + throws TimeoutException { + return createMediaPeriod( + uri, + new AssetDataSource(ApplicationProvider.getApplicationContext()), + new DefaultLoadErrorHandlingPolicy(), + new BundledExtractorsAdapter(new DefaultExtractorsFactory()), + /* imageDurationUs= */ C.TIME_UNSET, + /* executor= */ null, + /* executorReleased= */ null, + callback); + } + + private static ProgressiveMediaPeriod createMediaPeriod( + Uri uri, + DataSource dataSource, + LoadErrorHandlingPolicy loadErrorHandlingPolicy, + ProgressiveMediaExtractor extractor, + long imageDurationUs, + @Nullable Executor executor, + @Nullable Consumer executorReleased) + throws TimeoutException { + return createMediaPeriod( + uri, + dataSource, + loadErrorHandlingPolicy, + extractor, + imageDurationUs, + executor, + executorReleased, + new MediaPeriod.Callback() { + @Override + public void onPrepared(MediaPeriod mediaPeriod) {} + + @Override + public void onContinueLoadingRequested(MediaPeriod source) { + source.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + } + }); + } + + private static ProgressiveMediaPeriod createMediaPeriod( + Uri uri, + DataSource dataSource, + LoadErrorHandlingPolicy loadErrorHandlingPolicy, + ProgressiveMediaExtractor extractor, + long imageDurationUs, + @Nullable Executor executor, + @Nullable Consumer executorReleased, + MediaPeriod.Callback callback) + throws TimeoutException { + AtomicBoolean sourceInfoRefreshCalled = new AtomicBoolean(false); + ProgressiveMediaPeriod.Listener sourceInfoRefreshListener = + (durationUs, seekMap, isLive) -> sourceInfoRefreshCalled.set(true); + MediaPeriodId mediaPeriodId = new MediaPeriodId(/* periodUid= */ new Object()); + ProgressiveMediaPeriod mediaPeriod = + new ProgressiveMediaPeriod( + uri, + dataSource, + extractor, + DrmSessionManager.DRM_UNSUPPORTED, + new DrmSessionEventListener.EventDispatcher() + .withParameters(/* windowIndex= */ 0, mediaPeriodId), + loadErrorHandlingPolicy, + new MediaSourceEventListener.EventDispatcher() + .withParameters(/* windowIndex= */ 0, mediaPeriodId), + sourceInfoRefreshListener, + new DefaultAllocator(/* trimOnReset= */ true, C.DEFAULT_BUFFER_SEGMENT_SIZE), + /* customCacheKey= */ null, + ProgressiveMediaSource.DEFAULT_LOADING_CHECK_INTERVAL_BYTES, + /* loadOnlySelectedTracks= */ true, + /* singleTrackId= */ 0, + /* singleTrackFormat= */ null, + imageDurationUs, + executor != null ? ReleasableExecutor.from(executor, executorReleased) : null); + + AtomicBoolean prepareCallbackCalled = new AtomicBoolean(false); + AtomicBoolean sourceInfoRefreshCalledBeforeOnPrepared = new AtomicBoolean(false); + mediaPeriod.prepare( + new MediaPeriod.Callback() { + @Override + public void onPrepared(MediaPeriod mediaPeriod) { + sourceInfoRefreshCalledBeforeOnPrepared.set(sourceInfoRefreshCalled.get()); + prepareCallbackCalled.set(true); + callback.onPrepared(mediaPeriod); + } + + @Override + public void onContinueLoadingRequested(MediaPeriod source) { + callback.onContinueLoadingRequested(source); + } + }, + /* positionUs= */ 0); + runMainLooperUntil(prepareCallbackCalled::get); + + assertThat(sourceInfoRefreshCalledBeforeOnPrepared.get()).isTrue(); + return mediaPeriod; + } + + private static void testExtractorsUpdatesSourceInfoBeforeOnPreparedCallback( + ProgressiveMediaExtractor extractor, long imageDurationUs) throws TimeoutException { + testExtractorsUpdatesSourceInfoBeforeOnPreparedCallback( + extractor, imageDurationUs, /* executor= */ null, /* executorReleased= */ null); + } + + private static void testExtractorsUpdatesSourceInfoBeforeOnPreparedCallback( + ProgressiveMediaExtractor extractor, + long imageDurationUs, + @Nullable Executor executor, + @Nullable Consumer executorReleased) + throws TimeoutException { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(extractor, imageDurationUs, executor, executorReleased); + mediaPeriod.release(); + } + + @Test + public void setEndPositionUs_beforePreparation_setsEndPositionAndReturnsEarly() throws Exception { + MediaPeriodId mediaPeriodId = new MediaPeriodId(/* periodUid= */ new Object()); + ProgressiveMediaPeriod mediaPeriod = + new ProgressiveMediaPeriod( + Uri.parse("asset://android_asset/media/mp4/sample.mp4"), + new AssetDataSource(ApplicationProvider.getApplicationContext()), + new BundledExtractorsAdapter(new DefaultExtractorsFactory()), + DrmSessionManager.DRM_UNSUPPORTED, + new DrmSessionEventListener.EventDispatcher() + .withParameters(/* windowIndex= */ 0, mediaPeriodId), + new DefaultLoadErrorHandlingPolicy(), + new MediaSourceEventListener.EventDispatcher() + .withParameters(/* windowIndex= */ 0, mediaPeriodId), + /* listener= */ null, + new DefaultAllocator(/* trimOnReset= */ true, C.DEFAULT_BUFFER_SEGMENT_SIZE), + /* customCacheKey= */ null, + ProgressiveMediaSource.DEFAULT_LOADING_CHECK_INTERVAL_BYTES, + /* loadOnlySelectedTracks= */ true, + /* singleTrackId= */ 0, + /* singleTrackFormat= */ null, + /* singleSampleDurationUs= */ C.TIME_UNSET, + /* downloadExecutor= */ null); + + long resultEndPositionUs = mediaPeriod.setEndPositionUs(500_000); + + assertThat(resultEndPositionUs).isEqualTo(500_000); + mediaPeriod.release(); + } + + @Test + public void setEndPositionUs_loweredMidPlayback_discardsUpstreamBuffers() throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil(() -> !mediaPeriod.isLoading()); + + long unusedEndPosition = mediaPeriod.setEndPositionUs(300_000); + + FormatHolder formatHolder = new FormatHolder(); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + int readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + assertThat(readResult).isEqualTo(C.RESULT_FORMAT_READ); + long lastReadTimeUs = C.TIME_UNSET; + while (true) { + buffer.clear(); + readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + if (readResult == C.RESULT_BUFFER_READ && !buffer.isEndOfStream()) { + lastReadTimeUs = buffer.timeUs; + } else { + break; + } + } + assertThat(lastReadTimeUs).isAtMost(300_000); + assertThat(buffer.isEndOfStream()).isTrue(); + mediaPeriod.release(); + } + + @Test + public void setEndPositionUs_loweredWhileLoading_defersUpstreamDiscardUntilLoadCanceled() + throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + assertThat(mediaPeriod.isLoading()).isTrue(); + + long unusedEndPosition = mediaPeriod.setEndPositionUs(300_000); + runMainLooperUntil(() -> !mediaPeriod.isLoading()); + + assertThat(mediaPeriod.isLoading()).isFalse(); + FormatHolder formatHolder = new FormatHolder(); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + int readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + assertThat(readResult).isEqualTo(C.RESULT_FORMAT_READ); + long lastReadTimeUs = C.TIME_UNSET; + while (true) { + buffer.clear(); + readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + if (readResult == C.RESULT_BUFFER_READ && !buffer.isEndOfStream()) { + lastReadTimeUs = buffer.timeUs; + } else { + break; + } + } + assertThat(lastReadTimeUs).isAtMost(300_000); + assertThat(buffer.isEndOfStream()).isTrue(); + mediaPeriod.release(); + } + + @Test + public void setEndPositionUs_beforeTrackSelection_doesNotPreventLoadingWhenTracksSelectedLater() + throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + + // Set end position when prepared but enabledTrackCount == 0. + long unusedEndPosition = mediaPeriod.setEndPositionUs(300_000); + + // Later select a track and start loading. + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + + assertThat( + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build())) + .isTrue(); + runMainLooperUntil(() -> !mediaPeriod.isLoading()); + + FormatHolder formatHolder = new FormatHolder(); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + int readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + assertThat(readResult).isEqualTo(C.RESULT_FORMAT_READ); + long lastReadTimeUs = C.TIME_UNSET; + while (true) { + buffer.clear(); + readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + if (readResult == C.RESULT_BUFFER_READ && !buffer.isEndOfStream()) { + lastReadTimeUs = buffer.timeUs; + } else { + break; + } + } + assertThat(lastReadTimeUs).isAtMost(300_000); + assertThat(buffer.isEndOfStream()).isTrue(); + mediaPeriod.release(); + } + + @Test + public void setEndPositionUs_afterFatalError_doesNotClearFatalError() throws Exception { + FatalErrorDataSource dataSource = + new FatalErrorDataSource(new AssetDataSource(ApplicationProvider.getApplicationContext())); + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod( + Uri.parse("asset://android_asset/media/mp4/sample.mp4"), + dataSource, + new DefaultLoadErrorHandlingPolicy()); + + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil(() -> !mediaPeriod.isLoading()); + + // Reset track selection when idle so continueLoading can start a fresh load. + Arrays.fill(selections, null); + unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + + dataSource.setThrowErrorAfterPrepare(true); + unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil( + () -> { + try { + mediaPeriod.maybeThrowPrepareError(); + return false; + } catch (IOException e) { + return true; + } + }); + assertThrows(ParserException.class, mediaPeriod::maybeThrowPrepareError); + assertThat( + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build())) + .isFalse(); + + // Setting end position while in fatal error should NOT transition to CLIPPED_FINISHED. + long unusedEnd = mediaPeriod.setEndPositionUs(300_000); + assertThat( + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build())) + .isFalse(); + mediaPeriod.release(); + } + + @Test + public void setEndPositionUs_onlyNonAvTrackSelected_loadsUntilEndPosition() throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample_with_vobsub.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + // Verify sample_with_vobsub.mp4 has Video, Audio, and Text tracks. + assertThat(trackGroups.get(0).type).isEqualTo(C.TRACK_TYPE_VIDEO); + assertThat(trackGroups.get(1).type).isEqualTo(C.TRACK_TYPE_AUDIO); + assertThat(trackGroups.get(2).type).isEqualTo(C.TRACK_TYPE_TEXT); + + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + // Select ONLY the text track (index 2). + selections[2] = + new FakeTrackSelection(trackGroups.get(2), new int[] {0}, /* selectedIndex= */ 0); + + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + assertThat(streams[2]).isNotNull(); + + // Clip at 300ms. Since only a non-AV track is selected, it must NOT immediately assume the + // end position is reached, and should load until 300ms. + long unusedEnd = mediaPeriod.setEndPositionUs(300_000); + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil( + () -> { + mediaPeriod.reevaluateBuffer(/* positionUs= */ 0); + return !mediaPeriod.isLoading(); + }); + + FormatHolder formatHolder = new FormatHolder(); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + int readResult = streams[2].readData(formatHolder, buffer, /* readFlags= */ 0); + assertThat(readResult).isEqualTo(C.RESULT_FORMAT_READ); + long lastReadTimeUs = C.TIME_UNSET; + while (true) { + buffer.clear(); + readResult = streams[2].readData(formatHolder, buffer, /* readFlags= */ 0); + if (readResult == C.RESULT_BUFFER_READ && !buffer.isEndOfStream()) { + lastReadTimeUs = buffer.timeUs; + } else { + break; + } + } + assertThat(lastReadTimeUs).isAtMost(300_000); + assertThat(buffer.isEndOfStream()).isTrue(); + mediaPeriod.release(); + } + + @Test + public void setEndPositionUs_reducedTwice_discardsUpstreamBuffersToLatestEndPosition() + throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil(() -> !mediaPeriod.isLoading()); + + // Reduce end position first to 600ms, then further to 300ms. + long unusedEndPosition1 = mediaPeriod.setEndPositionUs(600_000); + long unusedEndPosition2 = mediaPeriod.setEndPositionUs(300_000); + + FormatHolder formatHolder = new FormatHolder(); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + int readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + assertThat(readResult).isEqualTo(C.RESULT_FORMAT_READ); + long lastReadTimeUs = C.TIME_UNSET; + while (true) { + buffer.clear(); + readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + if (readResult == C.RESULT_BUFFER_READ && !buffer.isEndOfStream()) { + lastReadTimeUs = buffer.timeUs; + } else { + break; + } + } + assertThat(lastReadTimeUs).isAtMost(300_000); + assertThat(buffer.isEndOfStream()).isTrue(); + mediaPeriod.release(); + } + + @Test + public void setEndPositionUs_extendedAfterFinished_calculatesMinAvResumePositionAndResumes() + throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + long unusedEndPosition1 = mediaPeriod.setEndPositionUs(300_000); + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil( + () -> { + mediaPeriod.reevaluateBuffer(/* positionUs= */ 0); + return mediaPeriod.getBufferedPositionUs() == C.TIME_END_OF_SOURCE; + }); + shadowOf(Looper.getMainLooper()).idle(); + + // Extend end position after loading finished at 300_000us. + long unusedEndPosition2 = mediaPeriod.setEndPositionUs(600_000); + boolean unusedLoad2 = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil( + () -> { + mediaPeriod.reevaluateBuffer(/* positionUs= */ 0); + return mediaPeriod.getBufferedPositionUs() == C.TIME_END_OF_SOURCE; + }); + shadowOf(Looper.getMainLooper()).idle(); + + FormatHolder formatHolder = new FormatHolder(); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + int readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + assertThat(readResult).isEqualTo(C.RESULT_FORMAT_READ); + long lastReadTimeUs = C.TIME_UNSET; + while (true) { + buffer.clear(); + readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + if (readResult == C.RESULT_BUFFER_READ && !buffer.isEndOfStream()) { + lastReadTimeUs = buffer.timeUs; + } else { + break; + } + } + assertThat(lastReadTimeUs).isGreaterThan(300_000); + assertThat(lastReadTimeUs).isAtMost(600_000); + assertThat(buffer.isEndOfStream()).isTrue(); + mediaPeriod.release(); + } + + @Test + public void setEndPositionUs_extendedWhileCancelingForClipping_resumesLoadingToNewEnd() + throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + assertThat(mediaPeriod.isLoading()).isTrue(); + + // Lower end position while loading to trigger cancellation for clipping. + long unusedEndPosition1 = mediaPeriod.setEndPositionUs(300_000); + // Extend end position before cancellation completes. + long unusedEndPosition2 = mediaPeriod.setEndPositionUs(600_000); + + runMainLooperUntil( + () -> { + mediaPeriod.reevaluateBuffer(/* positionUs= */ 0); + return mediaPeriod.getBufferedPositionUs() == C.TIME_END_OF_SOURCE; + }); + shadowOf(Looper.getMainLooper()).idle(); + + FormatHolder formatHolder = new FormatHolder(); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + int readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + assertThat(readResult).isEqualTo(C.RESULT_FORMAT_READ); + long lastReadTimeUs = C.TIME_UNSET; + while (true) { + buffer.clear(); + readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + if (readResult == C.RESULT_BUFFER_READ && !buffer.isEndOfStream()) { + lastReadTimeUs = buffer.timeUs; + } else { + break; + } + } + assertThat(lastReadTimeUs).isGreaterThan(300_000); + assertThat(lastReadTimeUs).isAtMost(600_000); + assertThat(buffer.isEndOfStream()).isTrue(); + mediaPeriod.release(); + } + + @Test + public void + setEndPositionUs_extendedAfterFinished_loadErrorDuringResumption_retriesFromFailurePosition() + throws Exception { + AtomicInteger openCount = new AtomicInteger(); + DataSource underlyingDataSource = + new AssetDataSource(ApplicationProvider.getApplicationContext()); + DataSource dataSource = + new DataSource() { + @Override + public void addTransferListener(TransferListener transferListener) { + underlyingDataSource.addTransferListener(transferListener); + } + + @Override + public long open(DataSpec dataSpec) throws IOException { + if (openCount.incrementAndGet() == 2) { + throw new IOException("Simulated transient network error on resumption"); + } + return underlyingDataSource.open(dataSpec); + } + + @Override + public int read(byte[] buffer, int offset, int length) throws IOException { + return underlyingDataSource.read(buffer, offset, length); + } + + @Override + public Uri getUri() { + return underlyingDataSource.getUri(); + } + + @Override + public void close() throws IOException { + underlyingDataSource.close(); + } + }; + + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod( + Uri.parse("asset://android_asset/media/mp4/sample.mp4"), + dataSource, + new DefaultLoadErrorHandlingPolicy(/* minimumLoadableRetryCount= */ 3) { + @Override + public long getRetryDelayMsFor(LoadErrorInfo loadErrorInfo) { + return 0; // Retry immediately + } + }); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + long unusedEndPosition1 = mediaPeriod.setEndPositionUs(300_000); + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil( + () -> { + mediaPeriod.reevaluateBuffer(/* positionUs= */ 0); + return mediaPeriod.getBufferedPositionUs() == C.TIME_END_OF_SOURCE; + }); + shadowOf(Looper.getMainLooper()).idle(); + + assertThat(openCount.get()).isEqualTo(1); + + // Extend end position after loading finished at 300_000us. + long unusedEndPosition2 = mediaPeriod.setEndPositionUs(600_000); + boolean unusedLoad2 = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil( + () -> { + mediaPeriod.reevaluateBuffer(/* positionUs= */ 0); + return openCount.get() >= 3 + && mediaPeriod.getBufferedPositionUs() == C.TIME_END_OF_SOURCE; + }); + shadowOf(Looper.getMainLooper()).idle(); + + // Verified that openCount reached at least 3 (1 initial, 2 simulated failure, 3 retry success). + assertThat(openCount.get()).isAtLeast(3); + + FormatHolder formatHolder = new FormatHolder(); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + int readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + assertThat(readResult).isEqualTo(C.RESULT_FORMAT_READ); + long lastReadTimeUs = C.TIME_UNSET; + while (true) { + buffer.clear(); + readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + if (readResult == C.RESULT_BUFFER_READ && !buffer.isEndOfStream()) { + lastReadTimeUs = buffer.timeUs; + } else { + break; + } + } + assertThat(lastReadTimeUs).isGreaterThan(300_000); + assertThat(lastReadTimeUs).isAtMost(600_000); + assertThat(buffer.isEndOfStream()).isTrue(); + mediaPeriod.release(); + } + + @Test + public void setEndPositionUs_extendedToEndOfSourceAfterFinished_resumesLoadingToEnd() + throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + long unusedEndPosition1 = mediaPeriod.setEndPositionUs(300_000); + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil( + () -> { + mediaPeriod.reevaluateBuffer(/* positionUs= */ 0); + return mediaPeriod.getBufferedPositionUs() == C.TIME_END_OF_SOURCE; + }); + shadowOf(Looper.getMainLooper()).idle(); + + // Extend end position to C.TIME_END_OF_SOURCE after loading finished at 300_000us. + long unusedEndPosition2 = mediaPeriod.setEndPositionUs(C.TIME_END_OF_SOURCE); + boolean unusedLoad2 = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil( + () -> { + mediaPeriod.reevaluateBuffer(/* positionUs= */ 0); + return mediaPeriod.getBufferedPositionUs() == C.TIME_END_OF_SOURCE + && !mediaPeriod.isLoading(); + }); + shadowOf(Looper.getMainLooper()).idle(); + + FormatHolder formatHolder = new FormatHolder(); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + int readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + assertThat(readResult).isEqualTo(C.RESULT_FORMAT_READ); + long lastReadTimeUs = C.TIME_UNSET; + while (true) { + buffer.clear(); + readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + if (readResult == C.RESULT_BUFFER_READ && !buffer.isEndOfStream()) { + lastReadTimeUs = buffer.timeUs; + } else { + break; + } + } + assertThat(lastReadTimeUs).isGreaterThan(300_000); + assertThat(buffer.isEndOfStream()).isTrue(); + mediaPeriod.release(); + } + + @Test + public void setEndPositionUs_extendedToEndOfSourceWhileCancelingForClipping_resumesLoadingToEnd() + throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + assertThat(mediaPeriod.isLoading()).isTrue(); + + // Lower end position while loading to trigger cancellation for clipping. + long unusedEndPosition1 = mediaPeriod.setEndPositionUs(300_000); + // Extend end position to C.TIME_END_OF_SOURCE before cancellation completes. + long unusedEndPosition2 = mediaPeriod.setEndPositionUs(C.TIME_END_OF_SOURCE); + + runMainLooperUntil(() -> !mediaPeriod.isLoading()); + + FormatHolder formatHolder = new FormatHolder(); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + int readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + assertThat(readResult).isEqualTo(C.RESULT_FORMAT_READ); + long lastReadTimeUs = C.TIME_UNSET; + while (true) { + buffer.clear(); + readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + if (readResult == C.RESULT_BUFFER_READ && !buffer.isEndOfStream()) { + lastReadTimeUs = buffer.timeUs; + } else { + break; + } + } + assertThat(lastReadTimeUs).isGreaterThan(300_000); + assertThat(buffer.isEndOfStream()).isTrue(); + mediaPeriod.release(); + } + + @Test + public void seekToUs_beyondEndPositionUs_signalsEndOfStream() throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + + // Initial load up to 300ms. + long unusedEndPosition = mediaPeriod.setEndPositionUs(300_000); + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil( + () -> { + mediaPeriod.reevaluateBuffer(/* positionUs= */ 0); + return mediaPeriod.getBufferedPositionUs() == C.TIME_END_OF_SOURCE; + }); + shadowOf(Looper.getMainLooper()).idle(); + + // Seek to 500ms (beyond clip end of 300ms). + long seekTimeUs = mediaPeriod.seekToUs(500_000); + assertThat(seekTimeUs).isEqualTo(500_000); + runMainLooperUntil( + () -> { + boolean unused2 = + mediaPeriod.continueLoading( + new LoadingInfo.Builder().setPlaybackPositionUs(500_000).build()); + mediaPeriod.reevaluateBuffer(/* positionUs= */ 500_000); + return !mediaPeriod.isLoading(); + }); + shadowOf(Looper.getMainLooper()).idle(); + assertThat(mediaPeriod.isLoading()).isFalse(); + + // Verify stream signals EOS without stalling. + FormatHolder formatHolder = new FormatHolder(); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + int readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + if (readResult == C.RESULT_FORMAT_READ) { + buffer.clear(); + readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + } + while (readResult == C.RESULT_BUFFER_READ && !buffer.isEndOfStream()) { + buffer.clear(); + readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + } + assertThat(readResult).isEqualTo(C.RESULT_BUFFER_READ); + assertThat(buffer.isEndOfStream()).isTrue(); + mediaPeriod.release(); + } + + @Test + public void setEndPositionUs_reducedThenExtendedThenReduced_discardsToFinalEndPosition() + throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + assertThat(mediaPeriod.isLoading()).isTrue(); + + // 1. Lower end position to 300ms while loading. + long unusedEndPosition1 = mediaPeriod.setEndPositionUs(300_000); + // 2. Extend end position to 600ms before cancellation finishes. + long unusedEndPosition2 = mediaPeriod.setEndPositionUs(600_000); + // 3. Reduce end position again to 450ms. + long unusedEndPosition3 = mediaPeriod.setEndPositionUs(450_000); + + runMainLooperUntil( + () -> { + mediaPeriod.reevaluateBuffer(/* positionUs= */ 0); + return !mediaPeriod.isLoading(); + }); + shadowOf(Looper.getMainLooper()).idle(); + + FormatHolder formatHolder = new FormatHolder(); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + int readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + assertThat(readResult).isEqualTo(C.RESULT_FORMAT_READ); + long lastReadTimeUs = C.TIME_UNSET; + while (true) { + buffer.clear(); + readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + if (readResult == C.RESULT_BUFFER_READ && !buffer.isEndOfStream()) { + lastReadTimeUs = buffer.timeUs; + } else { + break; + } + } + assertThat(lastReadTimeUs).isAtMost(450_000); + assertThat(buffer.isEndOfStream()).isTrue(); + mediaPeriod.release(); + } + + @Test + public void setEndPositionUs_extendedAfterNaturalEofReached_resumesLoading() throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + + // Initial load until entire file reaches natural EOF (STATE_FINISHED). + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil(() -> mediaPeriod.getBufferedPositionUs() == C.TIME_END_OF_SOURCE); + shadowOf(Looper.getMainLooper()).idle(); + + // Clip the stream at 300ms. State transitions FINISHED -> CLIPPED_FINISHED and discards + // upstream buffers. + long unusedEndPosition1 = mediaPeriod.setEndPositionUs(300_000); + + // Later extend the clip to 600ms. State transitions CLIPPED_FINISHED -> IDLE and loading + // resumes. + long unusedEndPosition2 = mediaPeriod.setEndPositionUs(600_000); + runMainLooperUntil( + () -> { + mediaPeriod.reevaluateBuffer(/* positionUs= */ 0); + return !mediaPeriod.isLoading(); + }); + shadowOf(Looper.getMainLooper()).idle(); + + FormatHolder formatHolder = new FormatHolder(); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + int readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + assertThat(readResult).isEqualTo(C.RESULT_FORMAT_READ); + long lastReadTimeUs = C.TIME_UNSET; + while (true) { + buffer.clear(); + readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + if (readResult == C.RESULT_BUFFER_READ && !buffer.isEndOfStream()) { + lastReadTimeUs = buffer.timeUs; + } else { + break; + } + } + assertThat(lastReadTimeUs).isGreaterThan(300_000); + assertThat(lastReadTimeUs).isAtMost(600_000); + assertThat(buffer.isEndOfStream()).isTrue(); + mediaPeriod.release(); + } + + @Test + public void seekToUs_toStreamStartWithPositiveFirstSampleTimestamp_seeksInsideBuffer() + throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[1] = + new FakeTrackSelection(trackGroups.get(1), new int[] {0}, /* selectedIndex= */ 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + // Initial load until buffered. + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil( + () -> { + mediaPeriod.reevaluateBuffer(/* positionUs= */ 0); + return mediaPeriod.getBufferedPositionUs() == C.TIME_END_OF_SOURCE; + }); + shadowOf(Looper.getMainLooper()).idle(); + // Read first sample to advance read index past 0. + FormatHolder formatHolder = new FormatHolder(); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + int readResult = streams[1].readData(formatHolder, buffer, /* readFlags= */ 0); + if (readResult == C.RESULT_FORMAT_READ) { + buffer.clear(); + readResult = streams[1].readData(formatHolder, buffer, /* readFlags= */ 0); + } + assertThat(readResult).isEqualTo(C.RESULT_BUFFER_READ); + long firstSampleTimeUs = buffer.timeUs; + assertThat(firstSampleTimeUs).isGreaterThan(0); + + long seekTimeUs = mediaPeriod.seekToUs(0); + + assertThat(seekTimeUs).isEqualTo(0); + // Verify in-buffer seek was successful and loading is not restarted. + assertThat(mediaPeriod.isLoading()).isFalse(); + // Verify reading starts again from the first sample. + buffer.clear(); + readResult = streams[1].readData(formatHolder, buffer, /* readFlags= */ 0); + if (readResult == C.RESULT_FORMAT_READ) { + buffer.clear(); + readResult = streams[1].readData(formatHolder, buffer, /* readFlags= */ 0); + } + assertThat(readResult).isEqualTo(C.RESULT_BUFFER_READ); + assertThat(buffer.timeUs).isEqualTo(firstSampleTimeUs); + mediaPeriod.release(); + } + + @Test + public void onLoadCompleted_withEndPositionSet_immediatelyTransitionsToClippedFinished() + throws Exception { + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + + // Configure clip end at 300ms before load finishes. + long unusedEndPosition = mediaPeriod.setEndPositionUs(300_000); + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + + // Run until the load completes without explicitly calling reevaluateBuffer. + runMainLooperUntil(() -> !mediaPeriod.isLoading()); + shadowOf(Looper.getMainLooper()).idle(); + + // Verify upstream buffers beyond 300ms were immediately discarded on load completion. + FormatHolder formatHolder = new FormatHolder(); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + int readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + assertThat(readResult).isEqualTo(C.RESULT_FORMAT_READ); + long lastReadTimeUs = C.TIME_UNSET; + while (true) { + buffer.clear(); + readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + if (readResult == C.RESULT_BUFFER_READ && !buffer.isEndOfStream()) { + lastReadTimeUs = buffer.timeUs; + } else { + break; + } + } + assertThat(lastReadTimeUs).isAtMost(300_000); + assertThat(buffer.isEndOfStream()).isTrue(); + mediaPeriod.release(); + } + + @Test + public void seekToUs_inIdleStateAfterClipExtension_seeksInsideBuffer() throws Exception { + AtomicBoolean allowContinueLoading = new AtomicBoolean(true); + ProgressiveMediaPeriod mediaPeriod = + createMediaPeriod( + Uri.parse("asset://android_asset/media/mp4/sample.mp4"), + new MediaPeriod.Callback() { + @Override + public void onPrepared(MediaPeriod mediaPeriod) {} + + @Override + public void onContinueLoadingRequested(MediaPeriod source) { + if (allowContinueLoading.get()) { + source.continueLoading( + new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + } + } + }); + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; + selections[0] = + new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); + long unused = + mediaPeriod.selectTracks( + selections, + new boolean[trackGroups.length], + streams, + streamResetFlags, + /* positionUs= */ 0); + + // Initial load clipped at 300ms until loading finishes. + long unusedEndPosition1 = mediaPeriod.setEndPositionUs(300_000); + boolean unusedLoad = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil( + () -> { + mediaPeriod.reevaluateBuffer(/* positionUs= */ 0); + return mediaPeriod.getBufferedPositionUs() == C.TIME_END_OF_SOURCE; + }); + shadowOf(Looper.getMainLooper()).idle(); + + // Extend end position to 600ms (transitions state to STATE_IDLE). + allowContinueLoading.set(false); + long unusedEndPosition2 = mediaPeriod.setEndPositionUs(600_000); + assertThat(mediaPeriod.isLoading()).isFalse(); + + // Seek to 0 while in STATE_IDLE. + long seekTimeUs = mediaPeriod.seekToUs(0); + assertThat(seekTimeUs).isEqualTo(0); + // Verify in-buffer seek was successful and buffered samples were not discarded. + assertThat(mediaPeriod.getBufferedPositionUs()).isAtLeast(300_000); + assertThat(mediaPeriod.isLoading()).isFalse(); + + // Continue loading to new end position and verify stream plays up to 600ms without losing + // samples. + allowContinueLoading.set(true); + boolean unusedLoad2 = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + runMainLooperUntil( + () -> { + mediaPeriod.reevaluateBuffer(/* positionUs= */ 0); + return mediaPeriod.getBufferedPositionUs() == C.TIME_END_OF_SOURCE; + }); + shadowOf(Looper.getMainLooper()).idle(); + + FormatHolder formatHolder = new FormatHolder(); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + int readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + assertThat(readResult).isEqualTo(C.RESULT_FORMAT_READ); + long lastReadTimeUs = C.TIME_UNSET; + while (true) { + buffer.clear(); + readResult = streams[0].readData(formatHolder, buffer, /* readFlags= */ 0); + if (readResult == C.RESULT_BUFFER_READ && !buffer.isEndOfStream()) { + lastReadTimeUs = buffer.timeUs; + } else { + break; + } + } + assertThat(lastReadTimeUs).isGreaterThan(300_000); + assertThat(lastReadTimeUs).isAtMost(600_000); + assertThat(buffer.isEndOfStream()).isTrue(); + mediaPeriod.release(); + } + + @Test + public void onLoadError_vod_returnsDontRetryFatalAndReportsCanceledWhenRetriesExhausted() + throws Exception { + MediaSourceEventListener.EventDispatcher eventDispatcher = + new MediaSourceEventListener.EventDispatcher(); + List dataTypes = new ArrayList<>(); + List wasCanceledEvents = new ArrayList<>(); + eventDispatcher.addEventListener( + createHandlerForCurrentLooper(), + new MediaSourceEventListener() { + @Override + public void onLoadError( + int windowIndex, + @Nullable MediaPeriodId mediaPeriodId, + LoadEventInfo loadEventInfo, + MediaLoadData mediaLoadData, + IOException error, + boolean wasCanceled) { + dataTypes.add(mediaLoadData.dataType); + wasCanceledEvents.add(wasCanceled); + } + }); + DataSource dataSource = new AssetDataSource(ApplicationProvider.getApplicationContext()); + ProgressiveMediaExtractor extractor = + new BundledExtractorsAdapter(Mp4Extractor.newFactory(SubtitleParser.Factory.UNSUPPORTED)); + ProgressiveMediaPeriod mediaPeriod = + new ProgressiveMediaPeriod( + Uri.parse("asset://android_asset/media/mp4/sample.mp4"), + dataSource, + extractor, + DrmSessionManager.DRM_UNSUPPORTED, + new DrmSessionEventListener.EventDispatcher(), + new DefaultLoadErrorHandlingPolicy(), + eventDispatcher, + (durationUs, seekMap, isLive) -> {}, + new DefaultAllocator(/* trimOnReset= */ true, C.DEFAULT_BUFFER_SEGMENT_SIZE), + /* customCacheKey= */ null, + ProgressiveMediaSource.DEFAULT_LOADING_CHECK_INTERVAL_BYTES, + /* loadOnlySelectedTracks= */ true, + /* singleTrackId= */ 0, + /* singleTrackFormat= */ null, + /* singleSampleDurationUs= */ C.TIME_UNSET, + /* downloadExecutor= */ null); + mediaPeriod.seekMap(new SeekMap.Unseekable(/* durationUs= */ 10_000_000)); + shadowOf(Looper.getMainLooper()).idle(); + ProgressiveMediaPeriod.ExtractingLoadable loadable = + mediaPeriod + .new ExtractingLoadable( + Uri.parse("asset://android_asset/media/mp4/sample.mp4"), + dataSource, + extractor, + mediaPeriod, + new ConditionVariable()); + + Loader.LoadErrorAction actionWithinLimit = + mediaPeriod.onLoadError( + loadable, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + new IOException("Transient error"), + /* errorCount= */ 3); + Loader.LoadErrorAction actionExhausted = + mediaPeriod.onLoadError( + loadable, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + new IOException("Transient error"), + /* errorCount= */ 4); + boolean continueLoadingAfterFatalError = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + mediaPeriod.release(); + + assertThat(actionWithinLimit.isRetry()).isTrue(); + assertThat(actionExhausted).isEqualTo(Loader.DONT_RETRY_FATAL); + assertThat(dataTypes).containsExactly(C.DATA_TYPE_MEDIA, C.DATA_TYPE_MEDIA); + assertThat(wasCanceledEvents).containsExactly(false, true).inOrder(); + assertThat(continueLoadingAfterFatalError).isFalse(); + } + + @Test + public void onLoadError_live_usesLiveDataTypeAndMinimumRetryCount() throws Exception { + MediaSourceEventListener.EventDispatcher eventDispatcher = + new MediaSourceEventListener.EventDispatcher(); + List dataTypes = new ArrayList<>(); + List wasCanceledEvents = new ArrayList<>(); + eventDispatcher.addEventListener( + createHandlerForCurrentLooper(), + new MediaSourceEventListener() { + @Override + public void onLoadError( + int windowIndex, + @Nullable MediaPeriodId mediaPeriodId, + LoadEventInfo loadEventInfo, + MediaLoadData mediaLoadData, + IOException error, + boolean wasCanceled) { + dataTypes.add(mediaLoadData.dataType); + wasCanceledEvents.add(wasCanceled); + } + }); + DataSource dataSource = new AssetDataSource(ApplicationProvider.getApplicationContext()); + ProgressiveMediaExtractor extractor = + new BundledExtractorsAdapter(Mp4Extractor.newFactory(SubtitleParser.Factory.UNSUPPORTED)); + ProgressiveMediaPeriod mediaPeriod = + new ProgressiveMediaPeriod( + Uri.parse("asset://android_asset/media/mp4/sample.mp4"), + dataSource, + extractor, + DrmSessionManager.DRM_UNSUPPORTED, + new DrmSessionEventListener.EventDispatcher(), + new DefaultLoadErrorHandlingPolicy(), + eventDispatcher, + (durationUs, seekMap, isLive) -> {}, + new DefaultAllocator(/* trimOnReset= */ true, C.DEFAULT_BUFFER_SEGMENT_SIZE), + /* customCacheKey= */ null, + ProgressiveMediaSource.DEFAULT_LOADING_CHECK_INTERVAL_BYTES, + /* loadOnlySelectedTracks= */ true, + /* singleTrackId= */ 0, + /* singleTrackFormat= */ null, + /* singleSampleDurationUs= */ C.TIME_UNSET, + /* downloadExecutor= */ null); + mediaPeriod.seekMap(new SeekMap.Unseekable(/* durationUs= */ C.TIME_UNSET)); + shadowOf(Looper.getMainLooper()).idle(); + ProgressiveMediaPeriod.ExtractingLoadable loadable = + mediaPeriod + .new ExtractingLoadable( + Uri.parse("asset://android_asset/media/mp4/sample.mp4"), + dataSource, + extractor, + mediaPeriod, + new ConditionVariable()); + + Loader.LoadErrorAction actionWithinLimit = + mediaPeriod.onLoadError( + loadable, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + new IOException("Transient error"), + /* errorCount= */ 6); + Loader.LoadErrorAction actionExhausted = + mediaPeriod.onLoadError( + loadable, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + new IOException("Transient error"), + /* errorCount= */ 7); + boolean continueLoadingAfterFatalError = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + mediaPeriod.release(); + + assertThat(actionWithinLimit.isRetry()).isTrue(); + assertThat(actionExhausted).isEqualTo(Loader.DONT_RETRY_FATAL); + assertThat(dataTypes) + .containsExactly(C.DATA_TYPE_MEDIA_PROGRESSIVE_LIVE, C.DATA_TYPE_MEDIA_PROGRESSIVE_LIVE); + assertThat(wasCanceledEvents).containsExactly(false, true).inOrder(); + assertThat(continueLoadingAfterFatalError).isFalse(); + } + + @Test + public void onLoadError_whenMadeProgress_resetsEffectiveErrorCountAndRetries() throws Exception { + MediaSourceEventListener.EventDispatcher eventDispatcher = + new MediaSourceEventListener.EventDispatcher(); + List dataTypes = new ArrayList<>(); + List wasCanceledEvents = new ArrayList<>(); + eventDispatcher.addEventListener( + createHandlerForCurrentLooper(), + new MediaSourceEventListener() { + @Override + public void onLoadError( + int windowIndex, + @Nullable MediaPeriodId mediaPeriodId, + LoadEventInfo loadEventInfo, + MediaLoadData mediaLoadData, + IOException error, + boolean wasCanceled) { + dataTypes.add(mediaLoadData.dataType); + wasCanceledEvents.add(wasCanceled); + } + }); + DataSource dataSource = new AssetDataSource(ApplicationProvider.getApplicationContext()); + ProgressiveMediaExtractor extractor = + new BundledExtractorsAdapter(Mp4Extractor.newFactory(SubtitleParser.Factory.UNSUPPORTED)); + ProgressiveMediaPeriod mediaPeriod = + new ProgressiveMediaPeriod( + Uri.parse("asset://android_asset/media/mp4/sample.mp4"), + dataSource, + extractor, + DrmSessionManager.DRM_UNSUPPORTED, + new DrmSessionEventListener.EventDispatcher(), + new DefaultLoadErrorHandlingPolicy(), + eventDispatcher, + (durationUs, seekMap, isLive) -> {}, + new DefaultAllocator(/* trimOnReset= */ true, C.DEFAULT_BUFFER_SEGMENT_SIZE), + /* customCacheKey= */ null, + ProgressiveMediaSource.DEFAULT_LOADING_CHECK_INTERVAL_BYTES, + /* loadOnlySelectedTracks= */ true, + /* singleTrackId= */ 0, + /* singleTrackFormat= */ null, + /* singleSampleDurationUs= */ C.TIME_UNSET, + /* downloadExecutor= */ null); + mediaPeriod.prepare( + new MediaPeriod.Callback() { + @Override + public void onPrepared(MediaPeriod period) {} + + @Override + public void onContinueLoadingRequested(MediaPeriod source) {} + }, + /* positionUs= */ 0); + runMainLooperUntil(() -> !mediaPeriod.isLoading()); + // Select track 0 so continueLoading triggers startLoading() with initial sample count recorded. + TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); + @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; + @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; + boolean[] streamResetFlags = new boolean[trackGroups.length]; selections[0] = new FakeTrackSelection(trackGroups.get(0), new int[] {0}, /* selectedIndex= */ 0); long unused = @@ -136,84 +2626,47 @@ public void readData_forUnselectedTrack_returnsFormatAndNothingElse() throws Exc streams, streamResetFlags, /* positionUs= */ 0); - // Run loader until source has finished loading. - boolean unusedResult = + // Extract samples by running until loading finishes, so extractedSamplesCount > initial count. + boolean unusedLoad = mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); - runMainLooperUntil(() -> mediaPeriod.getBufferedPositionUs() == C.TIME_END_OF_SOURCE); - - // Read from stream 0 (selected) to check we successfully read the format and samples. - assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 0, buffer)) - .isEqualTo(C.RESULT_FORMAT_READ); - assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 0, buffer)) - .isEqualTo(C.RESULT_BUFFER_READ); - assertThat(buffer.isEndOfStream()).isFalse(); - // Read from stream 1 (unselected) to check we get no samples. - assertThat(readProgressiveStream(mediaPeriod, /* trackIndex= */ 1, buffer)) - .isEqualTo(C.RESULT_BUFFER_READ); - assertThat(buffer.isEndOfStream()).isTrue(); + runMainLooperUntil(() -> !mediaPeriod.isLoading()); + ProgressiveMediaPeriod.ExtractingLoadable loadable = + mediaPeriod + .new ExtractingLoadable( + Uri.parse("asset://android_asset/media/mp4/sample.mp4"), + dataSource, + extractor, + mediaPeriod, + new ConditionVariable()); + // Even with errorCount=4 (exceeding default minRetryCount=3), because madeProgress is true, + // effectiveErrorCount becomes 1 and the load should be retried. + Loader.LoadErrorAction action = + mediaPeriod.onLoadError( + loadable, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + new IOException("Transient error after progress"), + /* errorCount= */ 4); mediaPeriod.release(); - } - - @Test - public void getTrackGroups_multipleTracks_setsExpectedPrimaryTrackGroupId() throws Exception { - ProgressiveMediaPeriod mediaPeriod = - createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample.mp4")); - TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); - - assertThat(trackGroups.length).isEqualTo(2); - assertThat(trackGroups.get(0).type).isEqualTo(C.TRACK_TYPE_VIDEO); - assertThat(trackGroups.get(1).type).isEqualTo(C.TRACK_TYPE_AUDIO); - assertThat(trackGroups.get(0).getFormat(0).primaryTrackGroupId).isNull(); - assertThat(trackGroups.get(1).getFormat(0).primaryTrackGroupId) - .isEqualTo(trackGroups.get(0).id); - - mediaPeriod = - createMediaPeriod( - Uri.parse("asset://android_asset/media/mp4/h265_with_metadata_track.mp4")); - trackGroups = mediaPeriod.getTrackGroups(); - - assertThat(trackGroups.length).isEqualTo(3); - assertThat(trackGroups.get(0).type).isEqualTo(C.TRACK_TYPE_METADATA); - assertThat(trackGroups.get(1).type).isEqualTo(C.TRACK_TYPE_AUDIO); - assertThat(trackGroups.get(2).type).isEqualTo(C.TRACK_TYPE_VIDEO); - assertThat(trackGroups.get(0).getFormat(0).primaryTrackGroupId) - .isEqualTo(trackGroups.get(2).id); - assertThat(trackGroups.get(1).getFormat(0).primaryTrackGroupId) - .isEqualTo(trackGroups.get(2).id); - assertThat(trackGroups.get(2).getFormat(0).primaryTrackGroupId).isNull(); - - mediaPeriod = - createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample_with_vobsub.mp4")); - trackGroups = mediaPeriod.getTrackGroups(); - - assertThat(trackGroups.length).isEqualTo(3); - assertThat(trackGroups.get(0).type).isEqualTo(C.TRACK_TYPE_VIDEO); - assertThat(trackGroups.get(1).type).isEqualTo(C.TRACK_TYPE_AUDIO); - assertThat(trackGroups.get(2).type).isEqualTo(C.TRACK_TYPE_TEXT); - assertThat(trackGroups.get(0).getFormat(0).primaryTrackGroupId).isNull(); - assertThat(trackGroups.get(1).getFormat(0).primaryTrackGroupId) - .isEqualTo(trackGroups.get(0).id); - assertThat(trackGroups.get(2).getFormat(0).primaryTrackGroupId) - .isEqualTo(trackGroups.get(0).id); - - mediaPeriod = - createMediaPeriod(Uri.parse("asset://android_asset/media/ts/sample_with_id3.adts")); - trackGroups = mediaPeriod.getTrackGroups(); - assertThat(trackGroups.length).isEqualTo(2); - assertThat(trackGroups.get(0).type).isEqualTo(C.TRACK_TYPE_AUDIO); - assertThat(trackGroups.get(1).type).isEqualTo(C.TRACK_TYPE_METADATA); - assertThat(trackGroups.get(0).getFormat(0).primaryTrackGroupId).isNull(); - assertThat(trackGroups.get(1).getFormat(0).primaryTrackGroupId) - .isEqualTo(trackGroups.get(0).id); + assertThat(action.isRetry()).isTrue(); + assertThat(dataTypes).containsExactly(C.DATA_TYPE_MEDIA); + assertThat(wasCanceledEvents).containsExactly(false); } @Test - public void selectTracks_joiningStreamWithPrerollSamples_doesNotIncurExtraDiscontinuity() - throws Exception { + public void + continueLoading_beforeContinueLoadingRequestedRunnableRuns_returnsFalseAndDoesNotUnblockLoader() + throws Exception { + ConditionVariable allowFirstReadCondition = new ConditionVariable(); + ConditionVariable chunkPausedCondition = new ConditionVariable(); + AtomicInteger initCount = new AtomicInteger(); + AtomicInteger readCount = new AtomicInteger(); ProgressiveMediaExtractor extractor = new ProgressiveMediaExtractor() { + private long currentInputPosition; + @Override public void init( DataReader dataReader, @@ -222,23 +2675,15 @@ public void init( long position, long length, ExtractorOutput output) { - // Provide custom formats that have pre-roll samples. - output - .track(0, C.TRACK_TYPE_AUDIO) - .format( - new Format.Builder() - .setSampleMimeType(MimeTypes.AUDIO_AAC) - .setHasPrerollSamples(true) - .build()); - output - .track(1, C.TRACK_TYPE_VIDEO) - .format( - new Format.Builder() - .setSampleMimeType(MimeTypes.VIDEO_H264) - .setHasPrerollSamples(true) - .build()); - output.endTracks(); - output.seekMap(new SeekMap.Unseekable(C.TIME_UNSET)); + if (initCount.incrementAndGet() == 1) { + output + .track(0, C.TRACK_TYPE_VIDEO) + .format(new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build()); + output.endTracks(); + output.seekMap(new SeekMap.Unseekable(/* durationUs= */ 2_000_000)); + } else { + chunkPausedCondition.open(); + } } @Override @@ -249,7 +2694,7 @@ public void disableSeekingOnMp3Streams() {} @Override public long getCurrentInputPosition() { - return 0; + return currentInputPosition; } @Override @@ -257,11 +2702,16 @@ public void seek(long position, long timeUs) {} @Override public int read(PositionHolder positionHolder) { + if (readCount.incrementAndGet() == 1) { + allowFirstReadCondition.blockUninterruptible(); + currentInputPosition = + ProgressiveMediaSource.DEFAULT_LOADING_CHECK_INTERVAL_BYTES + 1; + positionHolder.position = 0; + return Extractor.RESULT_SEEK; + } return Extractor.RESULT_END_OF_INPUT; } }; - - // Create the period using the custom extractor. ProgressiveMediaPeriod mediaPeriod = createMediaPeriod( Uri.parse("asset://android_asset/media/mp4/sample.mp4"), @@ -269,55 +2719,10 @@ public int read(PositionHolder positionHolder) { /* imageDurationUs= */ C.TIME_UNSET, /* executor= */ null, /* executorReleased= */ null); - TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); @NullableType ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; @NullableType SampleStream[] streams = new SampleStream[trackGroups.length]; - - // 1. Initially select only the audio track (track 0). - selections[0] = new FakeTrackSelection(trackGroups.get(0), 0); - long unused = - mediaPeriod.selectTracks( - selections, - /* mayRetainStreamFlags= */ new boolean[2], - streams, - /* streamResetFlags= */ new boolean[2], - /* positionUs= */ 0); - - // Verify initial discontinuity for first track selection. - assertThat(mediaPeriod.readDiscontinuity()).isEqualTo(0); - - // 2. Simulate seeking to a non-zero position. - long seekPositionUs = 5 * C.MICROS_PER_SECOND; - unused = mediaPeriod.seekToUs(seekPositionUs); - - // 3. Join the video stream (track 1) at the non-zero position. - selections[1] = new FakeTrackSelection(trackGroups.get(1), 0); - unused = - mediaPeriod.selectTracks( - selections, - /* mayRetainStreamFlags= */ new boolean[2], - streams, - /* streamResetFlags= */ new boolean[2], - /* positionUs= */ seekPositionUs); - - // Verify no extra discontinuity when selecting the second stream. - assertThat(mediaPeriod.readDiscontinuity()).isEqualTo(C.TIME_UNSET); - - mediaPeriod.release(); - } - - @Test - public void selectTracks_withHagcTrack_createsMergingMetadataSampleStream() throws Exception { - assumeTrue("Skipping HAGC test on SDK < 37", SDK_INT >= 37); - ProgressiveMediaPeriod mediaPeriod = - createMediaPeriod(Uri.parse("asset://android_asset/media/mp4/sample_with_it35_track.mp4")); - TrackGroupArray trackGroups = mediaPeriod.getTrackGroups(); - ExoTrackSelection[] selections = new ExoTrackSelection[trackGroups.length]; - SampleStream[] streams = new SampleStream[trackGroups.length]; - // Select the video track (which is track 0 in sample_with_it35_track.mp4) selections[0] = new FakeTrackSelection(trackGroups.get(0), 0); - long unused = mediaPeriod.selectTracks( selections, @@ -325,123 +2730,85 @@ public void selectTracks_withHagcTrack_createsMergingMetadataSampleStream() thro streams, new boolean[trackGroups.length], /* positionUs= */ 0); + boolean initialContinueLoading = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + allowFirstReadCondition.open(); + chunkPausedCondition.block(); - assertThat(streams[0]).isInstanceOf(MergingMetadataSampleStream.class); + boolean continuedBeforeRunnable = + mediaPeriod.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + boolean isLoadingBeforeRunnable = mediaPeriod.isLoading(); + runMainLooperUntil(() -> readCount.get() == 2 && !mediaPeriod.isLoading()); mediaPeriod.release(); - } - private static @SampleStream.ReadDataResult int readProgressiveStream( - ProgressiveMediaPeriod mediaPeriod, int trackIndex, DecoderInputBuffer buffer) { - return mediaPeriod.readData(trackIndex, new FormatHolder(), buffer, /* readFlags= */ 0); + assertThat(initialContinueLoading).isTrue(); + assertThat(continuedBeforeRunnable).isFalse(); + assertThat(isLoadingBeforeRunnable).isFalse(); + assertThat(readCount.get()).isEqualTo(2); } - private static ProgressiveMediaPeriod createMediaPeriod( - ProgressiveMediaExtractor extractor, - long imageDurationUs, - @Nullable Executor executor, - @Nullable Consumer executorReleased) - throws TimeoutException { - return createMediaPeriod( - Uri.parse("asset://android_asset/media/mp4/sample.mp4"), - extractor, - imageDurationUs, - executor, - executorReleased); - } + private static final class ExecutionTrackingThread extends Thread { + private final AtomicBoolean hasRun; - private static ProgressiveMediaPeriod createMediaPeriod(Uri uri) throws TimeoutException { - return createMediaPeriod( - uri, - new BundledExtractorsAdapter(new DefaultExtractorsFactory()), - /* imageDurationUs= */ C.TIME_UNSET, - /* executor= */ null, - /* executorReleased= */ null); + public ExecutionTrackingThread(Runnable runnable, AtomicBoolean hasRun) { + super(runnable, "TestExecutionTrackingThread"); + this.hasRun = hasRun; + } + + @Override + public void run() { + hasRun.set(true); + super.run(); + } } - private static ProgressiveMediaPeriod createMediaPeriod( - Uri uri, - ProgressiveMediaExtractor extractor, - long imageDurationUs, - @Nullable Executor executor, - @Nullable Consumer executorReleased) - throws TimeoutException { - AtomicBoolean sourceInfoRefreshCalled = new AtomicBoolean(false); - ProgressiveMediaPeriod.Listener sourceInfoRefreshListener = - (durationUs, seekMap, isLive) -> sourceInfoRefreshCalled.set(true); - MediaPeriodId mediaPeriodId = new MediaPeriodId(/* periodUid= */ new Object()); - ProgressiveMediaPeriod mediaPeriod = - new ProgressiveMediaPeriod( - uri, - new AssetDataSource(ApplicationProvider.getApplicationContext()), - extractor, - DrmSessionManager.DRM_UNSUPPORTED, - new DrmSessionEventListener.EventDispatcher() - .withParameters(/* windowIndex= */ 0, mediaPeriodId), - new DefaultLoadErrorHandlingPolicy(), - new MediaSourceEventListener.EventDispatcher() - .withParameters(/* windowIndex= */ 0, mediaPeriodId), - sourceInfoRefreshListener, - new DefaultAllocator(/* trimOnReset= */ true, C.DEFAULT_BUFFER_SEGMENT_SIZE), - /* customCacheKey= */ null, - ProgressiveMediaSource.DEFAULT_LOADING_CHECK_INTERVAL_BYTES, - /* loadOnlySelectedTracks= */ true, - /* experimentalEnableHagcPlayback= */ true, - /* singleTrackId= */ 0, - /* singleTrackFormat= */ null, - imageDurationUs, - executor != null ? ReleasableExecutor.from(executor, executorReleased) : null); + private static final class FatalErrorDataSource implements DataSource { + private final DataSource delegate; + private boolean throwErrorAfterPrepare; - AtomicBoolean prepareCallbackCalled = new AtomicBoolean(false); - AtomicBoolean sourceInfoRefreshCalledBeforeOnPrepared = new AtomicBoolean(false); - mediaPeriod.prepare( - new MediaPeriod.Callback() { - @Override - public void onPrepared(MediaPeriod mediaPeriod) { - sourceInfoRefreshCalledBeforeOnPrepared.set(sourceInfoRefreshCalled.get()); - prepareCallbackCalled.set(true); - } + public FatalErrorDataSource(DataSource delegate) { + this.delegate = delegate; + } - @Override - public void onContinueLoadingRequested(MediaPeriod source) { - source.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); - } - }, - /* positionUs= */ 0); - runMainLooperUntil(prepareCallbackCalled::get); + public void setThrowErrorAfterPrepare(boolean throwErrorAfterPrepare) { + this.throwErrorAfterPrepare = throwErrorAfterPrepare; + } - assertThat(sourceInfoRefreshCalledBeforeOnPrepared.get()).isTrue(); - return mediaPeriod; - } + @Override + public void addTransferListener(TransferListener transferListener) { + delegate.addTransferListener(transferListener); + } - private static void testExtractorsUpdatesSourceInfoBeforeOnPreparedCallback( - ProgressiveMediaExtractor extractor, long imageDurationUs) throws TimeoutException { - testExtractorsUpdatesSourceInfoBeforeOnPreparedCallback( - extractor, imageDurationUs, /* executor= */ null, /* executorReleased= */ null); - } + @Override + public long open(DataSpec dataSpec) throws IOException { + if (throwErrorAfterPrepare) { + throw ParserException.createForMalformedContainer("Fatal error", /* cause= */ null); + } + return delegate.open(dataSpec); + } - private static void testExtractorsUpdatesSourceInfoBeforeOnPreparedCallback( - ProgressiveMediaExtractor extractor, - long imageDurationUs, - @Nullable Executor executor, - @Nullable Consumer executorReleased) - throws TimeoutException { - ProgressiveMediaPeriod mediaPeriod = - createMediaPeriod(extractor, imageDurationUs, executor, executorReleased); - mediaPeriod.release(); - } + @Override + public int read(byte[] buffer, int offset, int length) throws IOException { + if (throwErrorAfterPrepare) { + throw ParserException.createForMalformedContainer("Fatal error", /* cause= */ null); + } + return delegate.read(buffer, offset, length); + } - private static final class ExecutionTrackingThread extends Thread { - private final AtomicBoolean hasRun; + @Override + @Nullable + public Uri getUri() { + return delegate.getUri(); + } - public ExecutionTrackingThread(Runnable runnable, AtomicBoolean hasRun) { - super(runnable, "TestExecutionTrackingThread"); - this.hasRun = hasRun; + @Override + public Map> getResponseHeaders() { + return delegate.getResponseHeaders(); } @Override - public void run() { - hasRun.set(true); - super.run(); + public void close() throws IOException { + delegate.close(); } } } diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/SampleQueueTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/SampleQueueTest.java index 088e79c28ac..75f9ebf776b 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/SampleQueueTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/SampleQueueTest.java @@ -1049,6 +1049,28 @@ public void setStartTimeUs_notAllSamplesAreSyncSamples_discardsOnReadSide() { /* sampleOffsetUs= */ 0); } + @Test + public void isDiscardingAllSamplesToStartTime() { + Format syncAudioAacWithCodecs = + new Format.Builder().setSampleMimeType(MimeTypes.AUDIO_AAC).setCodecs("mp4a.40.2").build(); + Format syncAudioAacWithoutCodecs = + new Format.Builder().setSampleMimeType(MimeTypes.AUDIO_AAC).build(); + Format syncAudioAacInvalidCodecs = + new Format.Builder().setSampleMimeType(MimeTypes.AUDIO_AAC).setCodecs("invalid").build(); + Format syncAudioRaw = new Format.Builder().setSampleMimeType(MimeTypes.AUDIO_RAW).build(); + Format syncVideoApv = new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_APV).build(); + Format formatWithoutMimeType = new Format.Builder().build(); + + assertThat(SampleQueue.isDiscardingAllSamplesToStartTime(FORMAT_SYNC_SAMPLE_ONLY_1)).isTrue(); + assertThat(SampleQueue.isDiscardingAllSamplesToStartTime(FORMAT_1)).isFalse(); + assertThat(SampleQueue.isDiscardingAllSamplesToStartTime(syncAudioAacWithCodecs)).isTrue(); + assertThat(SampleQueue.isDiscardingAllSamplesToStartTime(syncAudioAacWithoutCodecs)).isFalse(); + assertThat(SampleQueue.isDiscardingAllSamplesToStartTime(syncAudioAacInvalidCodecs)).isFalse(); + assertThat(SampleQueue.isDiscardingAllSamplesToStartTime(syncAudioRaw)).isTrue(); + assertThat(SampleQueue.isDiscardingAllSamplesToStartTime(syncVideoApv)).isFalse(); + assertThat(SampleQueue.isDiscardingAllSamplesToStartTime(formatWithoutMimeType)).isFalse(); + } + @Test public void discardToEnd() { writeTestData(); @@ -1724,6 +1746,429 @@ public void setReadEndTimeUsTwice() { assertReadEndOfStream(/* formatRequired= */ false); } + @Test + public void + sampleMetadata_droppedNonKeyframeOnEmptyQueueAfterDownstreamDiscard_unwindsCorrectlyWithoutCorruption() { + sampleQueue.format(FORMAT_SYNC_SAMPLE_ONLY_1); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 0, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + assertReadFormat(false, FORMAT_SYNC_SAMPLE_ONLY_1); + assertReadSample( + /* timeUs= */ 0, + /* isKeyFrame= */ true, + /* isEncrypted= */ false, + DATA, + /* offset= */ 0, + ALLOCATION_SIZE); + sampleQueue.discardToRead(); + sampleQueue.setStartTimeUs(2000); + + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 1000, + /* flags= */ 0, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + + assertAllocationCount(0); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 2000, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + assertReadSample( + /* timeUs= */ 2000, + /* isKeyFrame= */ true, + /* isEncrypted= */ false, + DATA, + /* offset= */ 0, + ALLOCATION_SIZE); + } + + @Test + public void sampleMetadata_droppedPrerollSample_unwindsSampleDataQueue() { + sampleQueue.format(FORMAT_SYNC_SAMPLE_ONLY_1); + sampleQueue.setStartTimeUs(1000); + + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 0, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + + assertAllocationCount(0); + assertReadFormat(false, FORMAT_SYNC_SAMPLE_ONLY_1); + assertNoSamplesToRead(FORMAT_SYNC_SAMPLE_ONLY_1); + } + + @Test + public void sampleMetadata_droppedSpliceSample_unwindsSampleDataQueue() { + sampleQueue.format(FORMAT_1); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 1000, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + assertReadFormat(false, FORMAT_1); + assertReadSample( + /* timeUs= */ 1000, + /* isKeyFrame= */ true, + /* isEncrypted= */ false, + DATA, + /* offset= */ 0, + ALLOCATION_SIZE); + sampleQueue.splice(); + + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 500, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + + assertAllocationCount(1); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 2000, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + assertReadSample( + /* timeUs= */ 2000, + /* isKeyFrame= */ true, + /* isEncrypted= */ false, + DATA, + /* offset= */ 0, + ALLOCATION_SIZE); + } + + @Test + public void + sampleMetadata_droppedSampleSpanningMultipleAllocationNodes_freesAllocationsAndResetsWritePosition() { + sampleQueue.format(FORMAT_SYNC_SAMPLE_ONLY_1); + sampleQueue.setStartTimeUs(1000); + + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE * 3); + sampleQueue.sampleMetadata( + /* timeUs= */ 0, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME, + /* size= */ ALLOCATION_SIZE * 3, + /* offset= */ 0, + /* cryptoData= */ null); + + assertAllocationCount(0); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE * 2); + sampleQueue.sampleMetadata( + /* timeUs= */ 1000, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME, + /* size= */ ALLOCATION_SIZE * 2, + /* offset= */ 0, + /* cryptoData= */ null); + assertAllocationCount(2); + assertReadFormat(false, FORMAT_SYNC_SAMPLE_ONLY_1); + assertReadSample( + /* timeUs= */ 1000, + /* isKeyFrame= */ true, + /* isEncrypted= */ false, + DATA, + /* offset= */ 0, + ALLOCATION_SIZE * 2); + } + + @Test + public void + sampleMetadata_droppedSampleWithNonZeroOffset_skipsUnwindingToPreserveTrailingBytes() { + sampleQueue.format(FORMAT_SYNC_SAMPLE_ONLY_1); + sampleQueue.setStartTimeUs(1000); + + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE * 2); + sampleQueue.sampleMetadata( + /* timeUs= */ 0, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME, + /* size= */ ALLOCATION_SIZE, + /* offset= */ ALLOCATION_SIZE, + /* cryptoData= */ null); + + assertAllocationCount(2); + sampleQueue.sampleMetadata( + /* timeUs= */ 1000, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + assertReadFormat(false, FORMAT_SYNC_SAMPLE_ONLY_1); + assertReadSample( + /* timeUs= */ 1000, + /* isKeyFrame= */ true, + /* isEncrypted= */ false, + DATA, + /* offset= */ ALLOCATION_SIZE, + ALLOCATION_SIZE); + } + + @Test + public void expectResume_traverseOverlap_unwindsOverlapBytesAndAppendsNewSamplesCleanly() { + sampleQueue.format(FORMAT_1); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 1000000, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 2000000, + /* flags= */ 0, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + assertAllocationCount(2); + + sampleQueue.expectResume(); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 1000000, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + assertAllocationCount(2); + + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 2000000, + /* flags= */ 0, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + assertAllocationCount(2); + + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 3000000, + /* flags= */ 0, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + + assertAllocationCount(3); + assertReadFormat(false, FORMAT_1); + assertReadSample( + /* timeUs= */ 1000000, + /* isKeyFrame= */ true, + /* isEncrypted= */ false, + DATA, + /* offset= */ 0, + ALLOCATION_SIZE); + assertReadSample( + /* timeUs= */ 2000000, + /* isKeyFrame= */ false, + /* isEncrypted= */ false, + DATA, + /* offset= */ 0, + ALLOCATION_SIZE); + assertReadSample( + /* timeUs= */ 3000000, + /* isKeyFrame= */ false, + /* isEncrypted= */ false, + DATA, + /* offset= */ 0, + ALLOCATION_SIZE); + } + + @Test + public void expectResume_audioTimestampDrift_clearsPendingResumeAndAppendsCleanly() { + sampleQueue.format(FORMAT_1); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 1000000, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + + sampleQueue.expectResume(); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 1000001, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + assertAllocationCount(1); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 1020000, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + + assertAllocationCount(2); + assertReadFormat(false, FORMAT_1); + assertReadSample( + /* timeUs= */ 1000000, + /* isKeyFrame= */ true, + /* isEncrypted= */ false, + DATA, + /* offset= */ 0, + ALLOCATION_SIZE); + assertReadSample( + /* timeUs= */ 1020000, + /* isKeyFrame= */ true, + /* isEncrypted= */ false, + DATA, + /* offset= */ 0, + ALLOCATION_SIZE); + } + + @Test + public void expectResume_duplicateTimestampsAtTail_dropsExactDuplicateCountAndAppends() { + sampleQueue.format(FORMAT_1); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 1000000, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 2000000, + /* flags= */ 0, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 2000000, + /* flags= */ 0, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + assertAllocationCount(3); + + sampleQueue.expectResume(); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 2000000, + /* flags= */ 0, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + assertAllocationCount(3); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 2000000, + /* flags= */ 0, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + assertAllocationCount(3); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 3000000, + /* flags= */ 0, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + + assertAllocationCount(4); + assertReadFormat(false, FORMAT_1); + assertReadSample( + /* timeUs= */ 1000000, + /* isKeyFrame= */ true, + /* isEncrypted= */ false, + DATA, + /* offset= */ 0, + ALLOCATION_SIZE); + assertReadSample( + /* timeUs= */ 2000000, + /* isKeyFrame= */ false, + /* isEncrypted= */ false, + DATA, + /* offset= */ 0, + ALLOCATION_SIZE); + assertReadSample( + /* timeUs= */ 2000000, + /* isKeyFrame= */ false, + /* isEncrypted= */ false, + DATA, + /* offset= */ 0, + ALLOCATION_SIZE); + assertReadSample( + /* timeUs= */ 3000000, + /* isKeyFrame= */ false, + /* isEncrypted= */ false, + DATA, + /* offset= */ 0, + ALLOCATION_SIZE); + } + + @Test + public void expectResume_withNonZeroSampleOffset_unwindsOverlapAndAppendsCleanly() { + sampleQueue.format(FORMAT_1); + sampleQueue.setSampleOffsetUs(500000); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 500000, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + assertAllocationCount(1); + + sampleQueue.expectResume(); + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 500000, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + assertAllocationCount(1); + + sampleQueue.sampleData(new ParsableByteArray(DATA), ALLOCATION_SIZE); + sampleQueue.sampleMetadata( + /* timeUs= */ 1500000, + /* flags= */ 0, + /* size= */ ALLOCATION_SIZE, + /* offset= */ 0, + /* cryptoData= */ null); + + assertAllocationCount(2); + assertReadFormat(false, FORMAT_1.buildUpon().setSubsampleOffsetUs(500000).build()); + assertReadSample( + /* timeUs= */ 1000000, + /* isKeyFrame= */ true, + /* isEncrypted= */ false, + DATA, + /* offset= */ 0, + ALLOCATION_SIZE); + assertReadSample( + /* timeUs= */ 2000000, + /* isKeyFrame= */ false, + /* isEncrypted= */ false, + DATA, + /* offset= */ 0, + ALLOCATION_SIZE); + } + // Internal methods. /** Writes standard test data to {@code sampleQueue}. */ diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/SideloadedSubtitlesMediaSourceTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/SideloadedSubtitlesMediaSourceTest.java new file mode 100644 index 00000000000..fe83233f25a --- /dev/null +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/SideloadedSubtitlesMediaSourceTest.java @@ -0,0 +1,103 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.exoplayer.source; + +import static com.google.common.truth.Truth.assertThat; + +import android.net.Uri; +import androidx.media3.common.MediaItem; +import androidx.media3.common.MimeTypes; +import androidx.media3.test.utils.FakeMediaSource; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link SideloadedSubtitlesMediaSource}. */ +@RunWith(AndroidJUnit4.class) +public final class SideloadedSubtitlesMediaSourceTest { + + private static final String URI_MEDIA = "http://example.com/media.mp4"; + private static final String URI_TEXT = "http://example.com/subtitles.vtt"; + + @Test + public void canUpdateMediaItem_withChangedSubtitleTimeOffset_returnsTrue() { + MediaItem.SubtitleConfiguration subtitleConfiguration = + new MediaItem.SubtitleConfiguration.Builder(Uri.parse(URI_TEXT)) + .setMimeType(MimeTypes.APPLICATION_TTML) + .setLanguage("en") + .build(); + MediaItem mediaItem = + new MediaItem.Builder() + .setUri(URI_MEDIA) + .setSubtitleConfigurations(ImmutableList.of(subtitleConfiguration)) + .build(); + FakeMediaSource contentMediaSource = new FakeMediaSource(); + contentMediaSource.setCanUpdateMediaItems(true); + contentMediaSource.updateMediaItem(mediaItem); + MediaSource sideloadedSubtitlesMediaSource = + new SideloadedSubtitlesMediaSource( + contentMediaSource, + ImmutableList.of(subtitleConfiguration), + new MediaSource[] {new FakeMediaSource()}); + + MediaItem updatedMediaItem = + mediaItem + .buildUpon() + .setSubtitleConfigurations( + ImmutableList.of( + subtitleConfiguration.buildUpon().setTimeOffsetUs(1_000_000).build())) + .build(); + + assertThat(sideloadedSubtitlesMediaSource.canUpdateMediaItem(updatedMediaItem)).isTrue(); + } + + @Test + public void canUpdateMediaItem_withStructurallyChangedSubtitleConfigurations_returnsFalse() { + MediaItem.SubtitleConfiguration subtitleConfiguration = + new MediaItem.SubtitleConfiguration.Builder(Uri.parse(URI_TEXT)) + .setMimeType(MimeTypes.APPLICATION_TTML) + .setLanguage("en") + .build(); + MediaItem mediaItem = + new MediaItem.Builder() + .setUri(URI_MEDIA) + .setSubtitleConfigurations(ImmutableList.of(subtitleConfiguration)) + .build(); + FakeMediaSource contentMediaSource = new FakeMediaSource(); + contentMediaSource.setCanUpdateMediaItems(true); + contentMediaSource.updateMediaItem(mediaItem); + MediaSource sideloadedSubtitlesMediaSource = + new SideloadedSubtitlesMediaSource( + contentMediaSource, + ImmutableList.of(subtitleConfiguration), + new MediaSource[] {new FakeMediaSource()}); + + MediaItem mediaItemWithChangedLanguage = + mediaItem + .buildUpon() + .setSubtitleConfigurations( + ImmutableList.of(subtitleConfiguration.buildUpon().setLanguage("de").build())) + .build(); + MediaItem mediaItemWithoutSubtitles = + mediaItem.buildUpon().setSubtitleConfigurations(ImmutableList.of()).build(); + + assertThat(sideloadedSubtitlesMediaSource.canUpdateMediaItem(mediaItemWithChangedLanguage)) + .isFalse(); + assertThat(sideloadedSubtitlesMediaSource.canUpdateMediaItem(mediaItemWithoutSubtitles)) + .isFalse(); + } +} diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/SingleSampleMediaPeriodTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/SingleSampleMediaPeriodTest.java new file mode 100644 index 00000000000..641485f4d99 --- /dev/null +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/SingleSampleMediaPeriodTest.java @@ -0,0 +1,163 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.exoplayer.source; + +import static androidx.media3.common.util.Util.createHandlerForCurrentLooper; +import static com.google.common.truth.Truth.assertThat; + +import android.net.Uri; +import androidx.annotation.Nullable; +import androidx.media3.common.Format; +import androidx.media3.common.MimeTypes; +import androidx.media3.datasource.DataSpec; +import androidx.media3.exoplayer.source.MediaSourceEventListener.EventDispatcher; +import androidx.media3.exoplayer.upstream.DefaultLoadErrorHandlingPolicy; +import androidx.media3.exoplayer.upstream.Loader; +import androidx.media3.test.utils.FakeDataSource; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit test for {@link SingleSampleMediaPeriod}. */ +@RunWith(AndroidJUnit4.class) +public final class SingleSampleMediaPeriodTest { + + @Test + public void onLoadError_treatLoadErrorsAsEndOfStreamTrue_stopsRetryingAfterMinimumRetryCount() { + EventDispatcher eventDispatcher = new EventDispatcher(); + List wasCanceledEvents = new ArrayList<>(); + eventDispatcher.addEventListener( + createHandlerForCurrentLooper(), + new MediaSourceEventListener() { + @Override + public void onLoadError( + int windowIndex, + @Nullable MediaSource.MediaPeriodId mediaPeriodId, + LoadEventInfo loadEventInfo, + MediaLoadData mediaLoadData, + IOException error, + boolean wasCanceled) { + wasCanceledEvents.add(wasCanceled); + } + }); + DataSpec dataSpec = new DataSpec(Uri.parse("http://example.com/subtitle.vtt")); + FakeDataSource fakeDataSource = new FakeDataSource(); + SingleSampleMediaPeriod mediaPeriod = + new SingleSampleMediaPeriod( + dataSpec, + () -> fakeDataSource, + /* transferListener= */ null, + new Format.Builder().setSampleMimeType(MimeTypes.TEXT_VTT).build(), + /* durationUs= */ 10_000_000L, + new DefaultLoadErrorHandlingPolicy(/* minimumLoadableRetryCount= */ 3), + eventDispatcher, + /* treatLoadErrorsAsEndOfStream= */ true, + /* downloadExecutor= */ null); + SingleSampleMediaPeriod.SourceLoadable loadable = + new SingleSampleMediaPeriod.SourceLoadable(dataSpec, fakeDataSource); + + // At errorCount == 3 (minimumLoadableRetryCount), it should still retry. + Loader.LoadErrorAction actionAtMinRetryCount = + mediaPeriod.onLoadError( + loadable, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + new IOException("Transient error"), + /* errorCount= */ 3); + boolean loadingFinishedAtMinRetry = mediaPeriod.loadingFinished; + // At errorCount == 4 (> minimumLoadableRetryCount), it should treat as end of stream. + Loader.LoadErrorAction actionExhausted = + mediaPeriod.onLoadError( + loadable, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + new IOException("Transient error"), + /* errorCount= */ 4); + boolean loadingFinishedAfterExhausted = mediaPeriod.loadingFinished; + mediaPeriod.release(); + + assertThat(actionAtMinRetryCount.isRetry()).isTrue(); + assertThat(loadingFinishedAtMinRetry).isFalse(); + assertThat(actionExhausted).isEqualTo(Loader.DONT_RETRY); + assertThat(loadingFinishedAfterExhausted).isTrue(); + assertThat(wasCanceledEvents).containsExactly(false, true).inOrder(); + } + + @Test + public void + onLoadError_treatLoadErrorsAsEndOfStreamFalse_returnsDontRetryFatalAfterMinimumRetryCount() { + EventDispatcher eventDispatcher = new EventDispatcher(); + List wasCanceledEvents = new ArrayList<>(); + eventDispatcher.addEventListener( + createHandlerForCurrentLooper(), + new MediaSourceEventListener() { + @Override + public void onLoadError( + int windowIndex, + @Nullable MediaSource.MediaPeriodId mediaPeriodId, + LoadEventInfo loadEventInfo, + MediaLoadData mediaLoadData, + IOException error, + boolean wasCanceled) { + wasCanceledEvents.add(wasCanceled); + } + }); + DataSpec dataSpec = new DataSpec(Uri.parse("http://example.com/subtitle.vtt")); + FakeDataSource fakeDataSource = new FakeDataSource(); + SingleSampleMediaPeriod mediaPeriod = + new SingleSampleMediaPeriod( + dataSpec, + () -> fakeDataSource, + /* transferListener= */ null, + new Format.Builder().setSampleMimeType(MimeTypes.TEXT_VTT).build(), + /* durationUs= */ 10_000_000L, + new DefaultLoadErrorHandlingPolicy(/* minimumLoadableRetryCount= */ 3), + eventDispatcher, + /* treatLoadErrorsAsEndOfStream= */ false, + /* downloadExecutor= */ null); + SingleSampleMediaPeriod.SourceLoadable loadable = + new SingleSampleMediaPeriod.SourceLoadable(dataSpec, fakeDataSource); + + // At errorCount == 3 (minimumLoadableRetryCount), it should still retry. + Loader.LoadErrorAction actionAtMinRetryCount = + mediaPeriod.onLoadError( + loadable, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + new IOException("Transient error"), + /* errorCount= */ 3); + boolean loadingFinishedAtMinRetry = mediaPeriod.loadingFinished; + // At errorCount == 4 (> minimumLoadableRetryCount), it should return DONT_RETRY_FATAL. + Loader.LoadErrorAction actionExhausted = + mediaPeriod.onLoadError( + loadable, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + new IOException("Transient error"), + /* errorCount= */ 4); + boolean loadingFinishedAfterExhausted = mediaPeriod.loadingFinished; + mediaPeriod.release(); + + assertThat(actionAtMinRetryCount.isRetry()).isTrue(); + assertThat(loadingFinishedAtMinRetry).isFalse(); + assertThat(actionExhausted).isEqualTo(Loader.DONT_RETRY_FATAL); + assertThat(loadingFinishedAfterExhausted).isFalse(); + assertThat(wasCanceledEvents).containsExactly(false, true).inOrder(); + } +} diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/TimeOffsetMediaPeriodTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/TimeOffsetMediaPeriodTest.java index d4dc94ebebb..a2c07db78e1 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/TimeOffsetMediaPeriodTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/TimeOffsetMediaPeriodTest.java @@ -83,6 +83,38 @@ public void selectTracks_createsSampleStreamCorrectingOffset() throws Exception assertThat(readEndOfStreamBuffer).isTrue(); } + @Test + public void updateTimeOffsetUs_appliesNewOffsetToExistingSampleStreamsAndSeeks() + throws Exception { + FakeMediaPeriod fakeMediaPeriod = + createFakeMediaPeriod( + ImmutableList.of( + oneByteSample(/* timeUs= */ 8000, C.BUFFER_FLAG_KEY_FRAME), + oneByteSample(/* timeUs= */ 12000, C.BUFFER_FLAG_KEY_FRAME), + END_OF_STREAM_ITEM)); + MediaPeriod spyPeriod = spy(fakeMediaPeriod); + TimeOffsetMediaPeriod timeOffsetMediaPeriod = + new TimeOffsetMediaPeriod(spyPeriod, /* timeOffsetUs= */ -3000); + prepareMediaPeriodSync(timeOffsetMediaPeriod, /* positionUs= */ 0); + FormatHolder formatHolder = new FormatHolder(); + DecoderInputBuffer inputBuffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + SampleStream sampleStream = selectTracksOnMediaPeriodAndTriggerLoading(timeOffsetMediaPeriod); + sampleStream.readData(formatHolder, inputBuffer, FLAG_REQUIRE_FORMAT); + sampleStream.readData(formatHolder, inputBuffer, /* readFlags= */ 0); + long firstReadBufferTimeUs = inputBuffer.timeUs; + + timeOffsetMediaPeriod.updateTimeOffsetUs(/* timeOffsetUs= */ 2000); + sampleStream.readData(formatHolder, inputBuffer, /* readFlags= */ 0); + long secondReadBufferTimeUs = inputBuffer.timeUs; + long seekResultTimeUs = timeOffsetMediaPeriod.seekToUs(/* positionUs= */ 7000); + + assertThat(firstReadBufferTimeUs).isEqualTo(5000); + assertThat(secondReadBufferTimeUs).isEqualTo(14000); + verify(spyPeriod).seekToUs(5000); + assertThat(seekResultTimeUs).isEqualTo(7000); + } + @Test public void setEndPositionUs_isForwardedWithTimeOffset() throws Exception { FakeMediaPeriod fakeMediaPeriod = diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ads/AdTimelineTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ads/AdTimelineTest.java index f1a2bd10690..dd94eecc63c 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ads/AdTimelineTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ads/AdTimelineTest.java @@ -1,5 +1,5 @@ /* - * Copyright (C) 2026 The Android Open Source Project + * Copyright 2026 The Android Open Source Project * * Licensed under the Apache License, Version 2.0 (the "License"); * you may not use this file except in compliance with the License. @@ -190,6 +190,60 @@ public void getPeriod_lastPeriodUnsetDuration_fallsBackToAdPlaybackStateContentD // Content duration should fall back to adPlaybackState.contentDurationUs (50s) because period // duration is unset. assertThat(period.durationUs).isEqualTo(50 * MICROS_PER_SECOND); + assertThat(period.originalDurationUs).isEqualTo(TIME_UNSET); assertThat(period.adPlaybackState.contentDurationUs).isEqualTo(50 * MICROS_PER_SECOND); } + + @Test + public void getPeriod_singlePeriodWithWindowOffset_doesNotShiftAdGroupTimes() { + // Single period live stream with window offset 10s into the period. + FakeTimeline contentTimeline = + new FakeTimeline( + new FakeTimeline.TimelineWindowDefinition.Builder() + .setPeriodCount(1) + .setWindowPositionInFirstPeriodUs(10 * MICROS_PER_SECOND) + .setDurationUs(20 * MICROS_PER_SECOND) + .build()); + + long adGroupTimeUs = 15 * MICROS_PER_SECOND; + AdPlaybackState adPlaybackState = new AdPlaybackState("adsId", adGroupTimeUs); + + AdTimeline adTimeline = new AdTimeline(contentTimeline, adPlaybackState); + Period period = new Period(); + + adTimeline.getPeriod(/* periodIndex= */ 0, period); + + // Ad group time is unchanged relative to period 0. + assertThat(period.adPlaybackState.getAdGroup(0).timeUs).isEqualTo(adGroupTimeUs); + } + + @Test + public void getPeriod_multiPeriodWithWindowOffset_offsetsAdGroupTimesRelativeToPeriod0() { + // Multi-period live stream: Period 0 (20s duration), Period 1 (20s duration), + // Window starts 10s into Period 0. + FakeTimeline contentTimeline = + new FakeTimeline( + new FakeTimeline.TimelineWindowDefinition.Builder() + .setPeriodCount(2) + .setWindowPositionInFirstPeriodUs(10 * MICROS_PER_SECOND) + .setDurationUs(40 * MICROS_PER_SECOND) + .build()); + + long adGroup0TimeUs = 5 * MICROS_PER_SECOND; // In Period 0 + long adGroup1TimeUs = + 35 * MICROS_PER_SECOND; // In Period 1 (5s into Period 1, which starts at 30s) + AdPlaybackState adPlaybackState = new AdPlaybackState("adsId", adGroup0TimeUs, adGroup1TimeUs); + + AdTimeline adTimeline = new AdTimeline(contentTimeline, adPlaybackState); + Period period0 = new Period(); + Period period1 = new Period(); + + adTimeline.getPeriod(/* periodIndex= */ 0, period0); + adTimeline.getPeriod(/* periodIndex= */ 1, period1); + + // Period 0 retains ad group 0 at 5s. + assertThat(period0.adPlaybackState.getAdGroup(0).timeUs).isEqualTo(adGroup0TimeUs); + // Period 1 offsets ad group 1 by Period 0 duration (30s), so 35s - 30s = 5s. + assertThat(period1.adPlaybackState.getAdGroup(1).timeUs).isEqualTo(5 * MICROS_PER_SECOND); + } } diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ads/AdsMediaSourceTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ads/AdsMediaSourceTest.java index 0a2ad5a2282..4198033f4fb 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ads/AdsMediaSourceTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ads/AdsMediaSourceTest.java @@ -1339,6 +1339,62 @@ public void onAdPlaybackState_withClipping_adDurationsClippedIfAdDurationSet() { .inOrder(); } + @Test + public void + onChildSourceInfoRefreshed_liveStreamWithWindowOffset_adGroupTimesUnmodifiedInPeriod() { + Timeline liveTimeline = + new FakeTimeline( + new FakeTimeline.TimelineWindowDefinition.Builder() + .setPeriodCount(1) + .setLive(true) + .setDynamic(true) + .setDurationUs(60_000_000L) + .setWindowPositionInFirstPeriodUs(10_000_000L) + .build()); + AdPlaybackState adPlaybackState = + new AdPlaybackState("adsId", 15_000_000L) + .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1); + + setAdPlaybackState(adPlaybackState); + contentMediaSource.setNewSourceInfo(liveTimeline); + adsMediaSource.createPeriod( + new MediaPeriodId(liveTimeline.getUidOfPeriod(0), /* windowSequenceNumber= */ 0), + mock(Allocator.class), + /* startPositionUs= */ 0); + shadowOf(Looper.getMainLooper()).idle(); + + ArgumentCaptor timelineCaptor = ArgumentCaptor.forClass(Timeline.class); + verify(mockMediaSourceCaller, times(2)) + .onSourceInfoRefreshed(eq(adsMediaSource), timelineCaptor.capture()); + + Timeline publishedTimeline = timelineCaptor.getValue(); + Period period = new Period(); + publishedTimeline.getPeriod(0, period); + assertThat(period.adPlaybackState.getAdGroup(0).timeUs).isEqualTo(15_000_000L); + } + + @Test + public void releaseSource_afterChildSourceInfoRefreshed_doesNotThrow() { + FakeMediaSource contentMediaSource = new FakeMediaSource(CONTENT_TIMELINE); + AdsMediaSource adsMediaSource = + new AdsMediaSource( + contentMediaSource, + TEST_ADS_DATA_SPEC, + TEST_ADS_ID, + new DefaultMediaSourceFactory((Context) ApplicationProvider.getApplicationContext()), + new NoOpAdsLoader(), + /* adViewProvider= */ () -> null, + /* useLazyContentSourcePreparation= */ false, + /* useAdMediaSourceClipping= */ false); + MediaSourceCaller mediaSourceCaller = (source, timeline) -> {}; + adsMediaSource.prepareSource(mediaSourceCaller, PlayerId.UNSET, BandwidthMeter.NO_OP); + + adsMediaSource.releaseSource(mediaSourceCaller); + shadowOf(Looper.getMainLooper()).idle(); + + contentMediaSource.assertReleased(); + } + private static class NoOpAdsLoader implements AdsLoader { @Override diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ads/ServerSideAdInsertionMediaSourceTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ads/ServerSideAdInsertionMediaSourceTest.java index 883ffc862f0..e903e19a49a 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ads/ServerSideAdInsertionMediaSourceTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/ads/ServerSideAdInsertionMediaSourceTest.java @@ -44,6 +44,7 @@ import androidx.annotation.Nullable; import androidx.media3.common.AdPlaybackState; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.Format; import androidx.media3.common.MediaItem; import androidx.media3.common.Player; @@ -76,6 +77,7 @@ import androidx.media3.exoplayer.upstream.Allocator; import androidx.media3.exoplayer.upstream.BandwidthMeter; import androidx.media3.exoplayer.upstream.DefaultAllocator; +import androidx.media3.test.utils.BindFlag; import androidx.media3.test.utils.CapturingAudioSink; import androidx.media3.test.utils.DumpFileAsserts; import androidx.media3.test.utils.FakeClock; @@ -83,6 +85,7 @@ import androidx.media3.test.utils.FakeMediaSource; import androidx.media3.test.utils.FakeSampleStream; import androidx.media3.test.utils.FakeTimeline; +import androidx.media3.test.utils.Media3FlagsRule; import androidx.media3.test.utils.robolectric.CapturingRenderersFactory; import androidx.media3.test.utils.robolectric.PlaybackOutput; import androidx.media3.test.utils.robolectric.RobolectricUtil; @@ -112,12 +115,15 @@ public static ImmutableList params() { } @ParameterizedRobolectricTestRunner.Parameter(0) + @BindFlag(Flags.FLAG_PER_STREAM_MEDIA_PROGRESSION) public Boolean perStreamMediaProgressionEnabled; @Rule public ShadowMediaCodecConfig mediaCodecConfig = ShadowMediaCodecConfig.withAllDefaultSupportedCodecs(); + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); + private static final String TEST_ASSET = "asset:///media/mp4/sample.mp4"; @Test @@ -419,11 +425,7 @@ public void playbackWithPredefinedAds_playsSuccessfulWithoutRendererResets() thr Context context = ApplicationProvider.getApplicationContext(); FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); CapturingRenderersFactory renderersFactory = new CapturingRenderersFactory(context, clock); - ExoPlayer player = - new ExoPlayer.Builder(context, renderersFactory) - .setClock(clock) - .enablePerStreamMediaProgression(perStreamMediaProgressionEnabled) - .build(); + ExoPlayer player = new ExoPlayer.Builder(context, renderersFactory).setClock(clock).build(); Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); player.setVideoSurface(surface); PlaybackOutput playbackOutput = PlaybackOutput.register(player, renderersFactory); @@ -498,11 +500,7 @@ public void playbackWithNewlyInsertedAds_playsSuccessfulWithoutRendererResets() CapturingRenderersFactory renderersFactory = new CapturingRenderersFactory( context, clock, DiscontinuitySkippingCapturingAudioSink.create()); - ExoPlayer player = - new ExoPlayer.Builder(context, renderersFactory) - .setClock(clock) - .enablePerStreamMediaProgression(perStreamMediaProgressionEnabled) - .build(); + ExoPlayer player = new ExoPlayer.Builder(context, renderersFactory).setClock(clock).build(); Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); player.setVideoSurface(surface); PlaybackOutput playbackOutput = PlaybackOutput.register(player, renderersFactory); @@ -583,11 +581,7 @@ public void playbackWithAdditionalAdsInAdGroup_playsSuccessfulWithoutRendererRes CapturingRenderersFactory renderersFactory = new CapturingRenderersFactory( context, clock, DiscontinuitySkippingCapturingAudioSink.create()); - ExoPlayer player = - new ExoPlayer.Builder(context, renderersFactory) - .setClock(clock) - .enablePerStreamMediaProgression(perStreamMediaProgressionEnabled) - .build(); + ExoPlayer player = new ExoPlayer.Builder(context, renderersFactory).setClock(clock).build(); Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); player.setVideoSurface(surface); PlaybackOutput playbackOutput = PlaybackOutput.register(player, renderersFactory); @@ -663,10 +657,7 @@ public void playbackWithAdditionalAdsInAdGroup_playsSuccessfulWithoutRendererRes public void playbackWithSeek_isHandledCorrectly() throws Exception { Context context = ApplicationProvider.getApplicationContext(); ExoPlayer player = - new ExoPlayer.Builder(context) - .setClock(new FakeClock(/* isAutoAdvancing= */ true)) - .enablePerStreamMediaProgression(perStreamMediaProgressionEnabled) - .build(); + new ExoPlayer.Builder(context).setClock(new FakeClock(/* isAutoAdvancing= */ true)).build(); Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); player.setVideoSurface(surface); @@ -736,8 +727,11 @@ public void playbackWithSeek_isHandledCorrectly() throws Exception { verify(listener, times(12)).onDownstreamFormatChanged(any(), any()); assertThat(contentPositionAfterSeekMs).isEqualTo(1_600); assertThat(positionAfterSeekMs).isEqualTo(0); // Beginning of second ad. - // Assert renderers played through without reset, except for the seek. - verify(listener, times(2)).onVideoEnabled(any(), any()); + // Assert renderers played through without reset, except for the seek. With per-stream media + // progression, the content period after the seek reports a preroll because of the 1-second gap + // in contentResumeOffsetUs, resetting the video renderer once more. Audio has no preroll and is + // not reset. + verify(listener, times(perStreamMediaProgressionEnabled ? 3 : 2)).onVideoEnabled(any(), any()); verify(listener, times(2)).onAudioEnabled(any(), any()); // Assert playback progression was smooth (=no unexpected delays that cause audio to underrun) verify(listener, never()).onAudioUnderrun(any(), anyInt(), anyLong(), anyLong()); @@ -866,6 +860,315 @@ public void onContinueLoadingRequested(MediaPeriod source) { assertThat(readSamples).containsExactly(0L, 200L, 400L, 600L, 800L).inOrder(); } + @Test + public void getFlags_firstStreamWithAlreadyPlayedPrerollAdGroup_keepsPrerollFlag() + throws Exception { + TrackGroup trackGroup = new TrackGroup(new Format.Builder().build()); + Timeline timeline = new FakeTimeline(); + Object periodUid = timeline.getUidOfPeriod(/* periodIndex= */ 0); + AdPlaybackState adPlaybackState = + createAdPlaybackStateWithPlayedPrerollAndMidroll(/* adsId= */ new Object()); + DefaultAllocator allocator = + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024); + ServerSideAdInsertionMediaSource mediaSource = + prepareSourceWithPrerollFlagInChildStreams( + timeline, periodUid, adPlaybackState, trackGroup, allocator); + // The already played preroll ad group is skipped, so the first media period of the stream is a + // content period whose next ad group is the midroll at index 1. + MediaSource.MediaPeriodId contentPeriodId = + new MediaSource.MediaPeriodId( + periodUid, /* windowSequenceNumber= */ 0, /* nextAdGroupIndex= */ 1); + MediaPeriod mediaPeriod = + mediaSource.createPeriod(contentPeriodId, allocator, /* startPositionUs= */ 0); + prepareMediaPeriod(mediaPeriod, /* positionUs= */ 0); + + SampleStream[] streams = selectSingleTrack(mediaPeriod, trackGroup, /* positionUs= */ 0); + + assertThat(streams[0].getFlags()).isEqualTo(SampleStream.FLAG_HAS_PREROLL); + } + + @Test + public void getFlags_firstStreamStartingAfterSourceStreamStart_keepsPrerollFlag() + throws Exception { + TrackGroup trackGroup = new TrackGroup(new Format.Builder().build()); + Timeline timeline = new FakeTimeline(); + Object periodUid = timeline.getUidOfPeriod(/* periodIndex= */ 0); + AdPlaybackState adPlaybackState = + createAdPlaybackStateWithPlayedPrerollAndMidroll(/* adsId= */ new Object()); + DefaultAllocator allocator = + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024); + ServerSideAdInsertionMediaSource mediaSource = + prepareSourceWithPrerollFlagInChildStreams( + timeline, periodUid, adPlaybackState, trackGroup, allocator); + // The content period starts at a non-zero position and its next ad group is the midroll at + // index 1, but it still triggers the track selection of the wrapped period at that position, so + // any preroll the wrapped period reports belongs to this stream. + MediaSource.MediaPeriodId contentPeriodId = + new MediaSource.MediaPeriodId( + periodUid, /* windowSequenceNumber= */ 0, /* nextAdGroupIndex= */ 1); + MediaPeriod mediaPeriod = + mediaSource.createPeriod(contentPeriodId, allocator, /* startPositionUs= */ 100_000); + prepareMediaPeriod(mediaPeriod, /* positionUs= */ 100_000); + + SampleStream[] streams = selectSingleTrack(mediaPeriod, trackGroup, /* positionUs= */ 100_000); + + assertThat(streams[0].getFlags()).isEqualTo(SampleStream.FLAG_HAS_PREROLL); + } + + @Test + public void getFlags_streamOfPeriodAfterInStreamTransition_removesPrerollFlag() throws Exception { + TrackGroup trackGroup = new TrackGroup(new Format.Builder().build()); + Timeline timeline = new FakeTimeline(); + Object periodUid = timeline.getUidOfPeriod(/* periodIndex= */ 0); + AdPlaybackState adPlaybackState = + createAdPlaybackStateWithPlayedPrerollAndMidroll(/* adsId= */ new Object()); + DefaultAllocator allocator = + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024); + ServerSideAdInsertionMediaSource mediaSource = + prepareSourceWithPrerollFlagInChildStreams( + timeline, periodUid, adPlaybackState, trackGroup, allocator); + MediaSource.MediaPeriodId contentPeriodId = + new MediaSource.MediaPeriodId( + periodUid, /* windowSequenceNumber= */ 0, /* nextAdGroupIndex= */ 1); + MediaPeriod contentMediaPeriod = + mediaSource.createPeriod(contentPeriodId, allocator, /* startPositionUs= */ 0); + prepareMediaPeriod(contentMediaPeriod, /* positionUs= */ 0); + SampleStream[] unusedContentStreams = + selectSingleTrack(contentMediaPeriod, trackGroup, /* positionUs= */ 0); + // The midroll ad period continues reading the same stream right after the content period. + MediaSource.MediaPeriodId midrollAdPeriodId = + new MediaSource.MediaPeriodId( + periodUid, + /* adGroupIndex= */ 1, + /* adIndexInAdGroup= */ 0, + /* windowSequenceNumber= */ 0); + MediaPeriod adMediaPeriod = + mediaSource.createPeriod(midrollAdPeriodId, allocator, /* startPositionUs= */ 0); + prepareMediaPeriod(adMediaPeriod, /* positionUs= */ 0); + + SampleStream[] streams = selectSingleTrack(adMediaPeriod, trackGroup, /* positionUs= */ 0); + + assertThat(streams[0].getFlags()).isEqualTo(0); + } + + @Test + public void getFlags_firstStreamOfPrerollAd_keepsPrerollFlag() throws Exception { + TrackGroup trackGroup = new TrackGroup(new Format.Builder().build()); + Timeline timeline = new FakeTimeline(); + Object periodUid = timeline.getUidOfPeriod(/* periodIndex= */ 0); + AdPlaybackState adPlaybackState = + createAdPlaybackStateWithPrerollAndMidroll(/* adsId= */ new Object()); + DefaultAllocator allocator = + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024); + ServerSideAdInsertionMediaSource mediaSource = + prepareSourceWithPrerollFlagInChildStreams( + timeline, periodUid, adPlaybackState, trackGroup, allocator); + MediaSource.MediaPeriodId prerollAdPeriodId = + new MediaSource.MediaPeriodId( + periodUid, + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + /* windowSequenceNumber= */ 0); + MediaPeriod mediaPeriod = + mediaSource.createPeriod(prerollAdPeriodId, allocator, /* startPositionUs= */ 0); + prepareMediaPeriod(mediaPeriod, /* positionUs= */ 0); + + SampleStream[] streams = selectSingleTrack(mediaPeriod, trackGroup, /* positionUs= */ 0); + + assertThat(streams[0].getFlags()).isEqualTo(SampleStream.FLAG_HAS_PREROLL); + } + + @Test + public void getFlags_firstStreamStartingInsidePrerollAd_keepsPrerollFlag() throws Exception { + TrackGroup trackGroup = new TrackGroup(new Format.Builder().build()); + Timeline timeline = new FakeTimeline(); + Object periodUid = timeline.getUidOfPeriod(/* periodIndex= */ 0); + AdPlaybackState adPlaybackState = + createAdPlaybackStateWithPrerollAndMidroll(/* adsId= */ new Object()); + DefaultAllocator allocator = + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024); + ServerSideAdInsertionMediaSource mediaSource = + prepareSourceWithPrerollFlagInChildStreams( + timeline, periodUid, adPlaybackState, trackGroup, allocator); + // Playback resumes in the middle of the preroll ad, which triggers a new track selection of the + // wrapped period at that position, so any preroll it reports belongs to this stream. + MediaSource.MediaPeriodId prerollAdPeriodId = + new MediaSource.MediaPeriodId( + periodUid, + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + /* windowSequenceNumber= */ 0); + MediaPeriod mediaPeriod = + mediaSource.createPeriod(prerollAdPeriodId, allocator, /* startPositionUs= */ 100_000); + prepareMediaPeriod(mediaPeriod, /* positionUs= */ 100_000); + + SampleStream[] streams = selectSingleTrack(mediaPeriod, trackGroup, /* positionUs= */ 100_000); + + assertThat(streams[0].getFlags()).isEqualTo(SampleStream.FLAG_HAS_PREROLL); + } + + @Test + public void getFlags_streamOfContentPeriodAfterInStreamTransitionFromAd_removesPrerollFlag() + throws Exception { + TrackGroup trackGroup = new TrackGroup(new Format.Builder().build()); + Timeline timeline = new FakeTimeline(); + Object periodUid = timeline.getUidOfPeriod(/* periodIndex= */ 0); + AdPlaybackState adPlaybackState = + createAdPlaybackStateWithPlayedPrerollAndMidroll(/* adsId= */ new Object()); + DefaultAllocator allocator = + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024); + ServerSideAdInsertionMediaSource mediaSource = + prepareSourceWithPrerollFlagInChildStreams( + timeline, periodUid, adPlaybackState, trackGroup, allocator); + MediaSource.MediaPeriodId midrollAdPeriodId = + new MediaSource.MediaPeriodId( + periodUid, + /* adGroupIndex= */ 1, + /* adIndexInAdGroup= */ 0, + /* windowSequenceNumber= */ 0); + MediaPeriod adMediaPeriod = + mediaSource.createPeriod(midrollAdPeriodId, allocator, /* startPositionUs= */ 0); + prepareMediaPeriod(adMediaPeriod, /* positionUs= */ 0); + SampleStream[] unusedAdStreams = + selectSingleTrack(adMediaPeriod, trackGroup, /* positionUs= */ 0); + // The content period continues reading the same stream right after the midroll ad. The midroll + // is inserted at stream position 500_000, which is content position 300_000. + MediaSource.MediaPeriodId contentPeriodId = + new MediaSource.MediaPeriodId( + periodUid, /* windowSequenceNumber= */ 0, /* nextAdGroupIndex= */ C.INDEX_UNSET); + MediaPeriod contentMediaPeriod = + mediaSource.createPeriod(contentPeriodId, allocator, /* startPositionUs= */ 300_000); + prepareMediaPeriod(contentMediaPeriod, /* positionUs= */ 300_000); + + SampleStream[] streams = + selectSingleTrack(contentMediaPeriod, trackGroup, /* positionUs= */ 300_000); + + assertThat(streams[0].getFlags()).isEqualTo(0); + } + + /** + * Returns an {@link AdPlaybackState} with an unplayed server-side inserted preroll ad group and + * an unplayed server-side inserted midroll ad group. + */ + private static AdPlaybackState createAdPlaybackStateWithPrerollAndMidroll(Object adsId) { + AdPlaybackState adPlaybackState = + addAdGroupToAdPlaybackState( + new AdPlaybackState(adsId), + /* fromPositionUs= */ 0, + /* contentResumeOffsetUs= */ 0, + /* adDurationsUs...= */ 200_000); + return addAdGroupToAdPlaybackState( + adPlaybackState, + /* fromPositionUs= */ 500_000, + /* contentResumeOffsetUs= */ 0, + /* adDurationsUs...= */ 100_000); + } + + /** + * Returns an {@link AdPlaybackState} with an already played server-side inserted preroll ad group + * and an unplayed server-side inserted midroll ad group. + */ + private static AdPlaybackState createAdPlaybackStateWithPlayedPrerollAndMidroll(Object adsId) { + return createAdPlaybackStateWithPrerollAndMidroll(adsId) + .withPlayedAd(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0); + } + + /** + * Creates and prepares a {@link ServerSideAdInsertionMediaSource} whose wrapped media period + * creates sample streams reporting {@link SampleStream#FLAG_HAS_PREROLL}. + */ + private static ServerSideAdInsertionMediaSource prepareSourceWithPrerollFlagInChildStreams( + Timeline timeline, + Object periodUid, + AdPlaybackState adPlaybackState, + TrackGroup trackGroup, + Allocator allocator) + throws Exception { + FakeMediaPeriod childMediaPeriod = + new FakeMediaPeriod( + new TrackGroupArray(trackGroup), + allocator, + /* trackDataFactory= */ (format, mediaPeriodId) -> ImmutableList.of(), + new MediaSourceEventListener.EventDispatcher() + .withParameters( + /* windowIndex= */ 0, + new MediaSource.MediaPeriodId(/* periodUid= */ new Object())), + DrmSessionManager.DRM_UNSUPPORTED, + new DrmSessionEventListener.EventDispatcher(), + /* deferOnPrepared= */ false) { + @Override + protected FakeSampleStream createSampleStream( + Allocator allocator, + @Nullable MediaSourceEventListener.EventDispatcher mediaSourceEventDispatcher, + DrmSessionManager drmSessionManager, + DrmSessionEventListener.EventDispatcher drmEventDispatcher, + Format initialFormat, + List fakeSampleStreamItems) { + FakeSampleStream sampleStream = + new FakeSampleStream( + allocator, + mediaSourceEventDispatcher, + drmSessionManager, + drmEventDispatcher, + initialFormat, + fakeSampleStreamItems); + sampleStream.setFlags(SampleStream.FLAG_HAS_PREROLL); + return sampleStream; + } + }; + FakeMediaSource childMediaSource = + new FakeMediaSource(timeline) { + @Override + protected MediaPeriod createMediaPeriod( + MediaPeriodId id, + TrackGroupArray trackGroupArray, + Allocator allocator, + MediaSourceEventListener.EventDispatcher mediaSourceEventDispatcher, + DrmSessionManager drmSessionManager, + DrmSessionEventListener.EventDispatcher drmEventDispatcher, + @Nullable TransferListener transferListener) { + return childMediaPeriod; + } + }; + ServerSideAdInsertionMediaSource mediaSource = + new ServerSideAdInsertionMediaSource(childMediaSource, /* adPlaybackStateUpdater= */ null); + mediaSource.setAdPlaybackStates(ImmutableMap.of(periodUid, adPlaybackState), timeline); + AtomicBoolean sourcePrepared = new AtomicBoolean(); + mediaSource.prepareSource( + (source, newTimeline) -> sourcePrepared.set(true), PlayerId.UNSET, BandwidthMeter.NO_OP); + RobolectricUtil.runMainLooperUntil(sourcePrepared::get); + return mediaSource; + } + + private static void prepareMediaPeriod(MediaPeriod mediaPeriod, long positionUs) + throws Exception { + AtomicBoolean periodPrepared = new AtomicBoolean(); + mediaPeriod.prepare( + new MediaPeriod.Callback() { + @Override + public void onPrepared(MediaPeriod mediaPeriod) { + periodPrepared.set(true); + } + + @Override + public void onContinueLoadingRequested(MediaPeriod source) {} + }, + positionUs); + RobolectricUtil.runMainLooperUntil(periodPrepared::get); + } + + private static SampleStream[] selectSingleTrack( + MediaPeriod mediaPeriod, TrackGroup trackGroup, long positionUs) { + SampleStream[] streams = new SampleStream[1]; + mediaPeriod.selectTracks( + new ExoTrackSelection[] {new FixedTrackSelection(trackGroup, /* track= */ 0)}, + /* mayRetainStreamFlags= */ new boolean[] {false}, + streams, + /* streamResetFlags= */ new boolean[] {false}, + positionUs); + return streams; + } + @Test public void playlistWithTwoSsaiStreams_reportsExpectedDiscontinuities() throws Exception { Context context = ApplicationProvider.getApplicationContext(); diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/chunk/ChunkSampleStreamTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/chunk/ChunkSampleStreamTest.java index 0c0c97a37ed..84337bbc20d 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/chunk/ChunkSampleStreamTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/chunk/ChunkSampleStreamTest.java @@ -15,7 +15,12 @@ */ package androidx.media3.exoplayer.source.chunk; +import static androidx.media3.common.util.Util.createHandlerForCurrentLooper; import static com.google.common.truth.Truth.assertThat; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyLong; +import static org.mockito.Mockito.doAnswer; +import static org.mockito.Mockito.doThrow; import android.net.Uri; import androidx.annotation.Nullable; @@ -29,10 +34,16 @@ import androidx.media3.exoplayer.LoadingInfo; import androidx.media3.exoplayer.drm.DrmSessionEventListener; import androidx.media3.exoplayer.drm.DrmSessionManager; +import androidx.media3.exoplayer.source.LoadEventInfo; +import androidx.media3.exoplayer.source.MediaLoadData; +import androidx.media3.exoplayer.source.MediaSource; import androidx.media3.exoplayer.source.MediaSourceEventListener; +import androidx.media3.exoplayer.source.SampleStream; import androidx.media3.exoplayer.source.SequenceableLoader; import androidx.media3.exoplayer.upstream.DefaultAllocator; import androidx.media3.exoplayer.upstream.DefaultLoadErrorHandlingPolicy; +import androidx.media3.exoplayer.upstream.Loader; +import androidx.media3.exoplayer.upstream.Loader.LoadErrorAction; import androidx.media3.extractor.TrackOutput; import androidx.media3.test.utils.FakeAdaptiveDataSet; import androidx.media3.test.utils.FakeChunkSource; @@ -40,6 +51,7 @@ import androidx.media3.test.utils.FakeTrackSelection; import androidx.test.ext.junit.runners.AndroidJUnit4; import java.io.IOException; +import java.util.ArrayList; import java.util.List; import java.util.Random; import org.junit.Before; @@ -58,14 +70,16 @@ public final class ChunkSampleStreamTest { @Mock private ChunkSource mockChunkSource; @Mock private SequenceableLoader.Callback> mockCallback; - @Mock private DrmSessionEventListener.EventDispatcher mockDrmEventDispatcher; - @Mock private MediaSourceEventListener.EventDispatcher mockMediaSourceEventDispatcher; + private DrmSessionEventListener.EventDispatcher drmEventDispatcher; + private MediaSourceEventListener.EventDispatcher mediaSourceEventDispatcher; private ChunkSampleStream chunkSampleStream; private DefaultAllocator allocator; @Before public void setUp() { + drmEventDispatcher = new DrmSessionEventListener.EventDispatcher(); + mediaSourceEventDispatcher = new MediaSourceEventListener.EventDispatcher(); allocator = new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 65536); chunkSampleStream = @@ -78,9 +92,9 @@ public void setUp() { allocator, /* positionUs= */ 0, DrmSessionManager.DRM_UNSUPPORTED, - mockDrmEventDispatcher, + drmEventDispatcher, new DefaultLoadErrorHandlingPolicy(), - mockMediaSourceEventDispatcher, + mediaSourceEventDispatcher, /* handleInitialDiscontinuity= */ false, /* firstChunkStartTimeUs= */ C.TIME_UNSET, /* downloadExecutor= */ null); @@ -178,9 +192,9 @@ protected MediaChunk createMediaChunk( allocator, /* positionUs= */ 0, DrmSessionManager.DRM_UNSUPPORTED, - mockDrmEventDispatcher, + drmEventDispatcher, new DefaultLoadErrorHandlingPolicy(), - mockMediaSourceEventDispatcher, + mediaSourceEventDispatcher, /* handleInitialDiscontinuity= */ false, /* firstChunkStartTimeUs= */ C.TIME_UNSET, /* downloadExecutor= */ null); @@ -217,4 +231,128 @@ protected MediaChunk createMediaChunk( // Verify that the seek fell back to reset instead of succeeding inside the buffer. assertThat(chunkSampleStream.isPendingReset()).isTrue(); } + + @Test + public void getFlags_beforeChunksLoaded_returnsFlagMaybeHasPreroll() { + chunkSampleStream = createChunkSampleStreamHandlingInitialDiscontinuity(); + + @SampleStream.Flags int flags = chunkSampleStream.getFlags(); + + assertThat(flags).isEqualTo(SampleStream.FLAG_MAYBE_HAS_PREROLL); + } + + @Test + public void getFlags_loadingFinishedWithoutMediaChunks_returnsZero() { + chunkSampleStream = createChunkSampleStreamHandlingInitialDiscontinuity(); + doAnswer( + invocation -> { + ChunkHolder chunkHolder = invocation.getArgument(3); + chunkHolder.endOfStream = true; + return null; + }) + .when(mockChunkSource) + .getNextChunk(any(), anyLong(), any(), any()); + + assertThat( + chunkSampleStream.continueLoading( + new LoadingInfo.Builder() + .setPlaybackPositionUs(/* playbackPositionUs= */ 0) + .build())) + .isTrue(); + + // No media chunk will ever be loaded, so the initial discontinuity can never be evaluated and + // the stream must not stay stuck on FLAG_MAYBE_HAS_PREROLL. + assertThat(chunkSampleStream.getFlags()).isEqualTo(0); + } + + @Test + public void onLoadError_whenRetriesExhausted_returnsDontRetryFatalAndReportsCanceled() { + List wasCanceledEvents = new ArrayList<>(); + mediaSourceEventDispatcher.addEventListener( + createHandlerForCurrentLooper(), + new MediaSourceEventListener() { + @Override + public void onLoadError( + int windowIndex, + @Nullable MediaSource.MediaPeriodId mediaPeriodId, + LoadEventInfo loadEventInfo, + MediaLoadData mediaLoadData, + IOException error, + boolean wasCanceled) { + wasCanceledEvents.add(wasCanceled); + } + }); + Chunk testChunk = + new Chunk( + new FakeDataSource(), + new DataSpec(Uri.parse("http://example.com/chunk")), + C.DATA_TYPE_MEDIA, + new Format.Builder().build(), + C.SELECTION_REASON_UNKNOWN, + /* trackSelectionData= */ null, + /* startTimeUs= */ 0, + /* endTimeUs= */ 1_000_000, + /* steeredPathwayId= */ null) { + @Override + public void cancelLoad() {} + + @Override + public void load() {} + }; + + LoadErrorAction actionWithinRetryLimit = + chunkSampleStream.onLoadError( + testChunk, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + new IOException("Transient error"), + /* errorCount= */ 3); + LoadErrorAction actionRetriesExhausted = + chunkSampleStream.onLoadError( + testChunk, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + new IOException("Transient error"), + /* errorCount= */ 4); + + assertThat(actionWithinRetryLimit.isRetry()).isTrue(); + assertThat(actionRetriesExhausted).isEqualTo(Loader.DONT_RETRY_FATAL); + assertThat(wasCanceledEvents).containsExactly(false, true).inOrder(); + } + + @Test + public void mayHaveInitialDiscontinuity_whenChunkSourceThrowsError_returnsFalse() + throws Exception { + ChunkSampleStream streamWithDiscontinuity = + createChunkSampleStreamHandlingInitialDiscontinuity(); + + boolean initialDiscontinuityBeforeError = streamWithDiscontinuity.mayHaveInitialDiscontinuity(); + int flagsBeforeError = streamWithDiscontinuity.getFlags(); + doThrow(new IOException("Manifest refresh failed")).when(mockChunkSource).maybeThrowError(); + boolean initialDiscontinuityAfterError = streamWithDiscontinuity.mayHaveInitialDiscontinuity(); + int flagsAfterError = streamWithDiscontinuity.getFlags(); + + assertThat(initialDiscontinuityBeforeError).isTrue(); + assertThat(flagsBeforeError).isEqualTo(SampleStream.FLAG_MAYBE_HAS_PREROLL); + assertThat(initialDiscontinuityAfterError).isFalse(); + assertThat(flagsAfterError).isEqualTo(0); + } + + private ChunkSampleStream createChunkSampleStreamHandlingInitialDiscontinuity() { + return new ChunkSampleStream<>( + /* primaryTrackType= */ C.TRACK_TYPE_VIDEO, + /* embeddedTrackTypes= */ null, + /* embeddedTrackFormats= */ null, + mockChunkSource, + mockCallback, + allocator, + /* positionUs= */ 0, + DrmSessionManager.DRM_UNSUPPORTED, + drmEventDispatcher, + new DefaultLoadErrorHandlingPolicy(), + mediaSourceEventDispatcher, + /* handleInitialDiscontinuity= */ true, + /* firstChunkStartTimeUs= */ C.TIME_UNSET, + /* downloadExecutor= */ null); + } } diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/preload/PreCacheHelperTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/preload/PreCacheHelperTest.java index e52a893d0fc..6db83c00e85 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/preload/PreCacheHelperTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/preload/PreCacheHelperTest.java @@ -88,6 +88,7 @@ public class PreCacheHelperTest { @Rule public final InMemoryDatabaseRule cacheRule = InMemoryDatabaseRule.create(); private static final long TIMEOUT_MS = 10_000; + private static final long NEGATIVE_TIMEOUT_MS = 500; private HandlerThread preCacheThread; private Looper preCacheLooper; @Mock private PreCacheHelper.Listener mockPreCacheHelperListener; @@ -103,6 +104,7 @@ public void setUp() throws Exception { @After public void tearDown() { + shadowOf(Looper.getMainLooper()).idle(); preCacheThread.quit(); } @@ -116,7 +118,7 @@ public void preCache_succeeds() throws Exception { return null; }) .when(mockPreCacheHelperListener) - .onPreCacheProgress(any(), anyLong(), anyLong(), eq(100f)); + .onPreCacheCompleted(any()); PreCacheHelper preCacheHelper = new PreCacheHelper.Factory( ApplicationProvider.getApplicationContext(), @@ -134,7 +136,7 @@ public void preCache_succeeds() throws Exception { verify(mockPreCacheHelperListener, never()).onPrepareError(eq(mediaItem), any()); verify(mockPreCacheHelperListener, never()).onDownloadError(eq(mediaItem), any()); - preCacheHelper.release(/* removeCachedContent= */ true); + preCacheHelper.release(/* removeCachedContent= */ false); shadowOf(preCacheLooper).idle(); } @@ -191,7 +193,7 @@ public void maybeThrowSourceInfoRefreshError() throws IOException { verify(mockPreCacheHelperListener).onPrepareError(eq(testMediaItem), eq(fakeException)); verify(mockPreCacheHelperListener, never()).onDownloadError(eq(testMediaItem), any()); - preCacheHelper.release(/* removeCachedContent= */ true); + preCacheHelper.release(/* removeCachedContent= */ false); shadowOf(preCacheLooper).idle(); } @@ -243,7 +245,7 @@ public void preCache_errorThrownWhenDownloading_downloadErrorReported() throws E verify(mockPreCacheHelperListener, never()).onPrepareError(eq(testMediaItem), any()); verify(mockPreCacheHelperListener).onDownloadError(eq(testMediaItem), any()); - preCacheHelper.release(/* removeCachedContent= */ true); + preCacheHelper.release(/* removeCachedContent= */ false); shadowOf(preCacheLooper).idle(); } @@ -288,7 +290,7 @@ public void preCache_errorThrownWhenDownloading_downloadErrorReported() throws E // request was reused. assertThat(createdMediaSources).hasSize(1); - preCacheHelper.release(/* removeCachedContent= */ true); + preCacheHelper.release(/* removeCachedContent= */ false); shadowOf(preCacheLooper).idle(); } @@ -336,7 +338,7 @@ public void preCacheAgainWithSameTimeRange_whileDownloadOngoing_reuseTheOngoingP // request was reused. assertThat(createdMediaSources).hasSize(1); - preCacheHelper.release(/* removeCachedContent= */ true); + preCacheHelper.release(/* removeCachedContent= */ false); shadowOf(preCacheLooper).idle(); } @@ -395,7 +397,7 @@ public void maybeThrowSourceInfoRefreshError() throws IOException { assertThat(createMediaSourceLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); - preCacheHelper.release(/* removeCachedContent= */ true); + preCacheHelper.release(/* removeCachedContent= */ false); shadowOf(preCacheLooper).idle(); } @@ -411,6 +413,14 @@ public void preCacheAgainWithSameTimeRange_afterDownloadCompleted_startNewPreCac createMediaSourceLatch.countDown(); return fakeMediaSource; }); + AtomicBoolean preCacheTerminated = new AtomicBoolean(); + doAnswer( + invocation -> { + preCacheTerminated.set(true); + return null; + }) + .when(mockPreCacheHelperListener) + .onPreCacheCompleted(any()); CacheDataSource.Factory cacheDataSourceFactory = new CacheDataSource.Factory() .setUpstreamDataSourceFactory( @@ -435,13 +445,15 @@ public void preCacheAgainWithSameTimeRange_afterDownloadCompleted_startNewPreCac downloader.incrementBytesDownloaded(); downloader.finish(); shadowOf(preCacheLooper).idle(); + runMainLooperUntil(() -> preCacheTerminated.get()); + shadowOf(Looper.getMainLooper()).idle(); preCacheHelper.preCache(/* startPositionMs= */ 0, /* durationMs= */ 1000L); shadowOf(preCacheLooper).idle(); assertThat(createMediaSourceLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); - preCacheHelper.release(/* removeCachedContent= */ true); + preCacheHelper.release(/* removeCachedContent= */ false); shadowOf(preCacheLooper).idle(); } @@ -492,13 +504,14 @@ public void preCacheAgainWithSameTimeRange_afterDownloadFailed_startNewPreCacheR } shadowOf(preCacheLooper).idle(); runMainLooperUntil(() -> preCacheTerminated.get()); + shadowOf(Looper.getMainLooper()).idle(); preCacheHelper.preCache(/* startPositionMs= */ 0, /* durationMs= */ 1000L); shadowOf(preCacheLooper).idle(); assertThat(createMediaSourceLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); - preCacheHelper.release(/* removeCachedContent= */ true); + preCacheHelper.release(/* removeCachedContent= */ false); shadowOf(preCacheLooper).idle(); } @@ -540,7 +553,7 @@ public void preCacheAgainWithSameTimeRange_afterStopped_startNewPreCacheRequest( assertThat(createMediaSourceLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); - preCacheHelper.release(/* removeCachedContent= */ true); + preCacheHelper.release(/* removeCachedContent= */ false); shadowOf(preCacheLooper).idle(); } @@ -673,7 +686,7 @@ public void releasePeriod(MediaPeriod mediaPeriod) { // The onPreparedCalledLatch doesn't count down twice within the timeout, which means that we // got onPrepared called for 0 time or once after waiting long enough. In the later assertions // we will verify that we got onPrepared called exactly once. - assertThat(onPreparedCalledLatch.await(TIMEOUT_MS, MILLISECONDS)).isFalse(); + assertThat(onPreparedCalledLatch.await(NEGATIVE_TIMEOUT_MS, MILLISECONDS)).isFalse(); // The updated MediaItem that the PreCacheHelper.Listener received should just be the one // resulting from the second `preCache` call, and we can verify that it only contains the // StreamKey for the video track. @@ -684,8 +697,9 @@ public void releasePeriod(MediaPeriod mediaPeriod) { .containsExactly(new StreamKey(/* groupIndex= */ 0, /* streamIndex= */ 0)); FakeDownloader downloader = checkNotNull(fakeDownloaderFactory.createdDownloaders.get(0)); downloader.assertStreamKeys(new StreamKey(/* groupIndex= */ 0, /* streamIndex= */ 0)); + downloader.finish(); - preCacheHelper.release(/* removeCachedContent= */ true); + preCacheHelper.release(/* removeCachedContent= */ false); shadowOf(preCacheLooper).idle(); } @@ -740,7 +754,7 @@ public void releasePeriod(MediaPeriod mediaPeriod) { assertThat(onPreparedCalledLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); verify(mockPreCacheHelperListener, never()).onPrepareError(eq(testMediaItem), any()); - preCacheHelper.release(/* removeCachedContent= */ true); + preCacheHelper.release(/* removeCachedContent= */ false); shadowOf(preCacheLooper).idle(); } @@ -783,8 +797,9 @@ public void stop_whilePreparationOngoing_doNotNotifyOnPrepared() throws Exceptio shadowOf(preCacheLooper).idle(); verifyNoInteractions(mockPreCacheHelperListener); + assertThat(fakeDownloaderFactory.createdDownloaders).isEmpty(); - preCacheHelper.release(/* removeCachedContent= */ true); + preCacheHelper.release(/* removeCachedContent= */ false); shadowOf(preCacheLooper).idle(); } @@ -821,7 +836,7 @@ public void stop_whileDownloadOngoing_downloaderCanceled() throws Exception { downloader.assertCanceled(true); - preCacheHelper.release(/* removeCachedContent= */ true); + preCacheHelper.release(/* removeCachedContent= */ false); shadowOf(preCacheLooper).idle(); } @@ -865,8 +880,9 @@ public void releaseWithoutRemovingCachedContent_whilePreparationOngoing_doNotNot shadowOf(preCacheLooper).idle(); verifyNoInteractions(mockPreCacheHelperListener); + assertThat(fakeDownloaderFactory.createdDownloaders).isEmpty(); - preCacheHelper.release(/* removeCachedContent= */ true); + preCacheHelper.release(/* removeCachedContent= */ false); shadowOf(preCacheLooper).idle(); } @@ -950,6 +966,7 @@ public void releaseWithRemovingCachedContent_whilePreparationOngoing_doNotNotify shadowOf(preCacheLooper).idle(); verifyNoInteractions(mockPreCacheHelperListener); + assertThat(fakeDownloaderFactory.createdDownloaders).isEmpty(); } @Test @@ -984,6 +1001,7 @@ public void releaseWithRemovingCachedContent_whileDownloadOngoing_downloaderCanc downloader.assertCanceled(true); downloader.assertRemoveStarted(true); + downloader.finish(); } @Test @@ -1026,6 +1044,7 @@ public void releaseWithRemovingCachedContent_whileRemoveOngoing_doNotCancelDownl downloader.assertCanceled(false); downloader.assertRemoveStarted(false); + downloader.finish(); } private static class FakeDownloaderFactory implements DownloaderFactory { diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/preload/PreloadAndPlaybackCoordinationTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/preload/PreloadAndPlaybackCoordinationTest.java index 5a28a3252dc..ae345a23626 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/preload/PreloadAndPlaybackCoordinationTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/preload/PreloadAndPlaybackCoordinationTest.java @@ -79,7 +79,7 @@ public PreloadAndPlaybackCoordinationTest() { FakeMediaSourceFactory mediaSourceFactory = new FakeMediaSourceFactory(); LoadControl loadControl = new DefaultLoadControl(); TrackSelector trackSelector = new DefaultTrackSelector(context); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); RenderersFactory renderersFactory = (handler, videoListener, audioListener, textOutput, metadataOutput) -> new Renderer[] { diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/preload/PreloadMediaSourceTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/preload/PreloadMediaSourceTest.java index 4bed3a6b3c1..0f9870ed7d0 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/preload/PreloadMediaSourceTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/source/preload/PreloadMediaSourceTest.java @@ -89,6 +89,7 @@ import java.util.concurrent.TimeoutException; import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicReference; +import org.junit.After; import org.junit.Before; import org.junit.Rule; import org.junit.Test; @@ -102,7 +103,7 @@ public final class PreloadMediaSourceTest { @Rule public final MockitoRule mockito = MockitoJUnit.rule(); - private static final int LOADING_CHECK_INTERVAL_BYTES = 32; + private static final int LOADING_CHECK_INTERVAL_BYTES = 1024; private static final int TARGET_PRELOAD_DURATION_US = 10000; private static final int LARGE_TARGET_BUFFER_BYTES_FOR_PRELOAD = Integer.MAX_VALUE; private static final int SMALL_TARGET_BUFFER_BYTES_FOR_PRELOAD = 1024; @@ -112,6 +113,7 @@ public final class PreloadMediaSourceTest { private RenderersFactory renderersFactory; private MediaItem mediaItem; @Mock private PreloadMediaSource.PreloadControl mockPreloadControl; + @Nullable private PreloadMediaSource preloadMediaSource; @Before public void setUp() { @@ -144,6 +146,13 @@ public void setUp() { when(mockPreloadControl.onContinueLoadingRequested(any(), anyLong())).thenReturn(true); } + @After + public void tearDown() { + if (preloadMediaSource != null) { + preloadMediaSource.releasePreloadMediaSource(); + } + } + @Test public void preload_loadPeriodToTargetPreloadPosition() throws Exception { AtomicBoolean preloadTerminated = new AtomicBoolean(); @@ -163,7 +172,7 @@ public void preload_loadPeriodToTargetPreloadPosition() throws Exception { mediaSourceFactory.setContinueLoadingCheckIntervalBytes(LOADING_CHECK_INTERVAL_BYTES); TrackSelector trackSelector = new DefaultTrackSelector(ApplicationProvider.getApplicationContext()); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mediaSourceFactory, @@ -173,7 +182,7 @@ public void preload_loadPeriodToTargetPreloadPosition() throws Exception { getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); preloadMediaSource.preload(/* startPositionUs= */ 0L); runMainLooperUntil(preloadTerminated::get); @@ -203,7 +212,7 @@ public void preload_stopWhenTracksSelectedByPreloadControl() throws Exception { mediaSourceFactory.setContinueLoadingCheckIntervalBytes(LOADING_CHECK_INTERVAL_BYTES); TrackSelector trackSelector = new DefaultTrackSelector(ApplicationProvider.getApplicationContext()); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mediaSourceFactory, @@ -213,7 +222,7 @@ public void preload_stopWhenTracksSelectedByPreloadControl() throws Exception { getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); preloadMediaSource.preload(/* startPositionUs= */ 0L); runMainLooperUntil(preloadTerminated::get); @@ -242,7 +251,7 @@ public void preload_stopWhenSourcePreparedByPreloadControl() throws Exception { new DefaultDataSource.Factory(ApplicationProvider.getApplicationContext())); TrackSelector trackSelector = new DefaultTrackSelector(ApplicationProvider.getApplicationContext()); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mediaSourceFactory, @@ -252,7 +261,7 @@ public void preload_stopWhenSourcePreparedByPreloadControl() throws Exception { getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); preloadMediaSource.preload(/* startPositionUs= */ 0L); runMainLooperUntil(preloadTerminated::get); @@ -270,7 +279,7 @@ public void preload_stopWhenSourcePreparedByPreloadControl() throws Exception { @Test public void preload_whileSourceIsAccessedByExternalCaller_notProceedWithPreloading() { TrackSelector trackSelector = new FakeTrackSelector(); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( new FakeMediaSourceFactory(), @@ -280,7 +289,7 @@ public void preload_whileSourceIsAccessedByExternalCaller_notProceedWithPreloadi getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); MediaSource.MediaSourceCaller externalCaller = mock(MediaSource.MediaSourceCaller.class); preloadMediaSource.prepareSource(externalCaller, PlayerId.UNSET, bandwidthMeter); @@ -311,7 +320,7 @@ public void preload_loadToTheEndOfSource() throws Exception { mediaSourceFactory.setContinueLoadingCheckIntervalBytes(LOADING_CHECK_INTERVAL_BYTES); TrackSelector trackSelector = new DefaultTrackSelector(ApplicationProvider.getApplicationContext()); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mediaSourceFactory, @@ -321,7 +330,7 @@ public void preload_loadToTheEndOfSource() throws Exception { getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); preloadMediaSource.preload(/* startPositionUs= */ 0L); runMainLooperUntil(preloadTerminated::get); @@ -343,22 +352,23 @@ public void preload_loadToTheEndOfSource() throws Exception { } @Test - public void preload_withSmallTargetBufferBytesThreshold_onLoadingUnableToContinueCalled() { + public void preload_withSmallTargetBufferBytesThreshold_onLoadingUnableToContinueCalled() + throws Exception { AtomicBoolean preloadTerminated = new AtomicBoolean(); doAnswer( invocation -> { preloadTerminated.set(true); - return null; + return false; }) .when(mockPreloadControl) - .onLoadedToTheEndOfSource(any()); + .onLoadingUnableToContinue(any()); ProgressiveMediaSource.Factory mediaSourceFactory = new ProgressiveMediaSource.Factory( new DefaultDataSource.Factory(ApplicationProvider.getApplicationContext())); mediaSourceFactory.setContinueLoadingCheckIntervalBytes(LOADING_CHECK_INTERVAL_BYTES); TrackSelector trackSelector = new DefaultTrackSelector(ApplicationProvider.getApplicationContext()); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); LoadControl loadControl = mock( LoadControl.class, @@ -376,11 +386,11 @@ public void preload_withSmallTargetBufferBytesThreshold_onLoadingUnableToContinu getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); preloadMediaSource.preload(/* startPositionUs= */ 0L); - assertThrows(TimeoutException.class, () -> runMainLooperUntil(preloadTerminated::get)); + runMainLooperUntil(preloadTerminated::get); verify(loadControl).onPrepared(PlayerId.PRELOAD); verify(mockPreloadControl).onSourcePrepared(eq(preloadMediaSource)); verify(mockPreloadControl).onTracksSelected(eq(preloadMediaSource)); @@ -439,7 +449,7 @@ public void maybeThrowSourceInfoRefreshError() throws IOException { }; TrackSelector trackSelector = new DefaultTrackSelector(ApplicationProvider.getApplicationContext()); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mediaSourceFactory, @@ -449,7 +459,7 @@ public void maybeThrowSourceInfoRefreshError() throws IOException { getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); preloadMediaSource.preload(/* startPositionUs= */ 0L); runMainLooperUntil(() -> preloadExceptionReference.get() != null); @@ -465,6 +475,33 @@ public void maybeThrowSourceInfoRefreshError() throws IOException { verify(mockPreloadControl, never()).onLoadingUnableToContinue(eq(preloadMediaSource)); } + @Test + public void preload_calledMultipleTimes_loadControlOnPreparedCalledOnce() { + TrackSelector trackSelector = new FakeTrackSelector(); + trackSelector.init(unused -> {}, bandwidthMeter); + PreloadMediaSource.Factory preloadMediaSourceFactory = + new PreloadMediaSource.Factory( + new FakeMediaSourceFactory(), + mockPreloadControl, + trackSelector, + bandwidthMeter, + getRendererCapabilities(renderersFactory), + loadControl, + Util.getCurrentOrMainLooper()); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + + preloadMediaSource.preload(/* startPositionUs= */ 0L); + shadowOf(Looper.getMainLooper()).idle(); + preloadMediaSource.preload(/* startPositionUs= */ 1000L); + shadowOf(Looper.getMainLooper()).idle(); + preloadMediaSource.releasePreloadMediaSource(); + shadowOf(Looper.getMainLooper()).idle(); + + verify(loadControl).onPrepared(PlayerId.PRELOAD); + verify(loadControl).onReleased(PlayerId.PRELOAD); + preloadMediaSource = null; // Prevent double release in tearDown + } + @Test public void preload_periodPrepareErrorThrows_onPreloadErrorCalled() throws TimeoutException { AtomicReference preloadExceptionReference = new AtomicReference<>(); @@ -524,7 +561,7 @@ public void maybeThrowPrepareError() throws IOException { }; TrackSelector trackSelector = new DefaultTrackSelector(ApplicationProvider.getApplicationContext()); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mediaSourceFactory, @@ -534,7 +571,7 @@ public void maybeThrowPrepareError() throws IOException { getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); preloadMediaSource.preload(/* startPositionUs= */ 0L); runMainLooperUntil(() -> preloadExceptionReference.get() != null); @@ -636,7 +673,7 @@ public void maybeThrowError() throws IOException { }; TrackSelector trackSelector = new DefaultTrackSelector(ApplicationProvider.getApplicationContext()); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mediaSourceFactory, @@ -646,7 +683,7 @@ public void maybeThrowError() throws IOException { getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); preloadMediaSource.preload(/* startPositionUs= */ 0L); runMainLooperUntil(() -> preloadExceptionReference.get() != null); @@ -667,7 +704,7 @@ public void prepareSourceNotOnPreloadThread_beforePreloadCalled_throwsIllegalSta HandlerThread preloadThread = new HandlerThread("preload"); preloadThread.start(); TrackSelector trackSelector = new FakeTrackSelector(); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( new FakeMediaSourceFactory(), @@ -677,7 +714,7 @@ public void prepareSourceNotOnPreloadThread_beforePreloadCalled_throwsIllegalSta getRendererCapabilities(renderersFactory), loadControl, preloadThread.getLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); assertThrows( IllegalStateException.class, @@ -693,7 +730,7 @@ public void prepareSourceNotOnPreloadThread_afterPreloadCalled_throwsIllegalStat HandlerThread preloadThread = new HandlerThread("preload"); preloadThread.start(); TrackSelector trackSelector = new FakeTrackSelector(); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( new FakeMediaSourceFactory(), @@ -703,7 +740,7 @@ public void prepareSourceNotOnPreloadThread_afterPreloadCalled_throwsIllegalStat getRendererCapabilities(renderersFactory), loadControl, preloadThread.getLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); preloadMediaSource.preload(/* startPositionUs= */ C.TIME_UNSET); shadowOf(preloadThread.getLooper()).idle(); @@ -721,7 +758,7 @@ public void prepareSourceNotOnPreloadThread_afterPreloadCalled_throwsIllegalStat prepareSource_beforeSourceInfoRefreshedForPreloading_onlyInvokeExternalCallerOnSourceInfoRefreshed() { FakeMediaSourceFactory mediaSourceFactory = new FakeMediaSourceFactory(); TrackSelector trackSelector = new FakeTrackSelector(); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mediaSourceFactory, @@ -731,7 +768,7 @@ public void prepareSourceNotOnPreloadThread_afterPreloadCalled_throwsIllegalStat getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); FakeMediaSource wrappedMediaSource = mediaSourceFactory.getLastCreatedSource(); wrappedMediaSource.setAllowPreparation(false); preloadMediaSource.preload(/* startPositionUs= */ 0L); @@ -751,7 +788,7 @@ public void prepareSourceNotOnPreloadThread_afterPreloadCalled_throwsIllegalStat public void prepareSource_afterPreload_immediatelyInvokeExternalCallerOnSourceInfoRefreshed() { FakeMediaSourceFactory mediaSourceFactory = new FakeMediaSourceFactory(); TrackSelector trackSelector = new FakeTrackSelector(); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mediaSourceFactory, @@ -761,7 +798,7 @@ public void prepareSource_afterPreload_immediatelyInvokeExternalCallerOnSourceIn getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); preloadMediaSource.preload(/* startPositionUs= */ 0L); shadowOf(Looper.getMainLooper()).idle(); @@ -818,7 +855,7 @@ public void createPeriodWithSameMediaPeriodIdAndStartPosition_returnExistingPeri new ExoTrackSelection[0], Tracks.EMPTY, /* info= */ null)); - mockTrackSelector.init(() -> {}, bandwidthMeter); + mockTrackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mockMediaSourceFactory, @@ -828,7 +865,7 @@ public void createPeriodWithSameMediaPeriodIdAndStartPosition_returnExistingPeri getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); preloadMediaSource.preload(/* startPositionUs= */ 0L); shadowOf(Looper.getMainLooper()).idle(); @@ -895,7 +932,7 @@ public void createPeriodWithSameMediaPeriodIdAndDifferentStartPosition_returnNew new ExoTrackSelection[0], Tracks.EMPTY, /* info= */ null)); - mockTrackSelector.init(() -> {}, bandwidthMeter); + mockTrackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mockMediaSourceFactory, @@ -905,7 +942,7 @@ public void createPeriodWithSameMediaPeriodIdAndDifferentStartPosition_returnNew getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); preloadMediaSource.preload(/* startPositionUs= */ 0L); shadowOf(Looper.getMainLooper()).idle(); @@ -959,7 +996,7 @@ public void release() { }); TrackSelector trackSelector = new DefaultTrackSelector(ApplicationProvider.getApplicationContext()); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mockMediaSourceFactory, @@ -969,7 +1006,7 @@ public void release() { getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); preloadMediaSource.preload(/* startPositionUs= */ 0L); shadowOf(Looper.getMainLooper()).idle(); @@ -1010,7 +1047,7 @@ protected MediaPeriod createMediaPeriod( } }); TrackSelector trackSelector = new FakeTrackSelector(); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mockMediaSourceFactory, @@ -1020,7 +1057,7 @@ protected MediaPeriod createMediaPeriod( getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); preloadMediaSource.preload(/* startPositionUs= */ 0L); shadowOf(Looper.getMainLooper()).idle(); @@ -1067,7 +1104,7 @@ public synchronized void prepare(Callback callback, long positionUs) { } }); TrackSelector trackSelector = new FakeTrackSelector(); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mockMediaSourceFactory, @@ -1077,7 +1114,7 @@ public synchronized void prepare(Callback callback, long positionUs) { getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); preloadMediaSource.preload(/* startPositionUs= */ 0L); shadowOf(Looper.getMainLooper()).idle(); @@ -1112,7 +1149,7 @@ public void releaseSourceByAllExternalCallers_preloadNotCalledBefore_releaseInte return mockMediaSource; }); TrackSelector trackSelector = new FakeTrackSelector(); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mockMediaSourceFactory, @@ -1122,16 +1159,17 @@ public void releaseSourceByAllExternalCallers_preloadNotCalledBefore_releaseInte getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); AtomicBoolean externalCallerSourceInfoRefreshedCalled = new AtomicBoolean(); MediaSource.MediaSourceCaller externalCaller = (source, timeline) -> externalCallerSourceInfoRefreshedCalled.set(true); preloadMediaSource.prepareSource(externalCaller, PlayerId.UNSET, bandwidthMeter); - shadowOf(Looper.getMainLooper()).idle(); preloadMediaSource.releaseSource(externalCaller); + shadowOf(Looper.getMainLooper()).idle(); verify(loadControl, never()).onPrepared(PlayerId.PRELOAD); verify(loadControl, never()).onReleased(PlayerId.PRELOAD); + verify(mockPreloadControl, never()).onSourcePrepared(any()); assertThat(externalCallerSourceInfoRefreshedCalled.get()).isTrue(); MediaSource internalSource = internalSourceReference.get(); assertThat(internalSource).isNotNull(); @@ -1161,7 +1199,7 @@ public void releaseSourceByAllExternalCallers_stillPreloading_notReleaseInternal return mockMediaSource; }); TrackSelector trackSelector = new FakeTrackSelector(); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mockMediaSourceFactory, @@ -1171,7 +1209,7 @@ public void releaseSourceByAllExternalCallers_stillPreloading_notReleaseInternal getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); MediaSource.MediaSourceCaller externalCaller = mock(MediaSource.MediaSourceCaller.class); preloadMediaSource.preload(/* startPositionUs= */ 0L); shadowOf(Looper.getMainLooper()).idle(); @@ -1211,7 +1249,7 @@ public void releaseSourceByAllExternalCallers_stillPreloading_notReleaseInternal return mockMediaSource; }); TrackSelector trackSelector = new FakeTrackSelector(); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mockMediaSourceFactory, @@ -1221,7 +1259,7 @@ public void releaseSourceByAllExternalCallers_stillPreloading_notReleaseInternal getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); AtomicBoolean externalCaller1SourceInfoRefreshedCalled = new AtomicBoolean(); AtomicBoolean externalCaller2SourceInfoRefreshedCalled = new AtomicBoolean(); MediaSource.MediaSourceCaller externalCaller1 = @@ -1265,7 +1303,7 @@ public void releasePreloadMediaSource_notUsedByExternalCallers_releaseInternalSo return mockMediaSource; }); TrackSelector trackSelector = new FakeTrackSelector(); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mockMediaSourceFactory, @@ -1275,7 +1313,7 @@ public void releasePreloadMediaSource_notUsedByExternalCallers_releaseInternalSo getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); preloadMediaSource.preload(/* startPositionUs= */ 0L); shadowOf(Looper.getMainLooper()).idle(); preloadMediaSource.releasePreloadMediaSource(); @@ -1312,7 +1350,7 @@ public void releasePreloadMediaSource_stillUsedByExternalCallers_releaseInternal return mockMediaSource; }); TrackSelector trackSelector = new FakeTrackSelector(); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mockMediaSourceFactory, @@ -1322,7 +1360,7 @@ public void releasePreloadMediaSource_stillUsedByExternalCallers_releaseInternal getRendererCapabilities(renderersFactory), loadControl, Util.getCurrentOrMainLooper()); - PreloadMediaSource preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); + preloadMediaSource = preloadMediaSourceFactory.createMediaSource(mediaItem); MediaSource.MediaSourceCaller externalCaller = mock(MediaSource.MediaSourceCaller.class); preloadMediaSource.preload(/* startPositionUs= */ 0L); shadowOf(Looper.getMainLooper()).idle(); diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/text/TextRendererTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/text/TextRendererTest.java index 8fc3a8e9ad1..9eb5cd14afd 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/text/TextRendererTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/text/TextRendererTest.java @@ -17,13 +17,16 @@ import static androidx.media3.test.utils.FakeSampleStream.FakeSampleStreamItem.END_OF_STREAM_ITEM; import static androidx.media3.test.utils.FakeSampleStream.FakeSampleStreamItem.sample; +import static androidx.media3.test.utils.robolectric.RobolectricUtil.runMainLooperUntil; import static com.google.common.truth.Truth.assertThat; +import static java.nio.charset.StandardCharsets.UTF_8; import androidx.media3.common.C; import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; import androidx.media3.common.Timeline; import androidx.media3.common.text.Cue; +import androidx.media3.common.text.CueGroup; import androidx.media3.decoder.DecoderInputBuffer; import androidx.media3.exoplayer.FormatHolder; import androidx.media3.exoplayer.RendererConfiguration; @@ -36,6 +39,8 @@ import androidx.media3.test.utils.FakeTimeline; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.collect.ImmutableList; +import java.util.ArrayList; +import java.util.List; import org.junit.Test; import org.junit.runner.RunWith; @@ -62,6 +67,8 @@ public void renderText_doesNotReadAheadTooFar() throws Exception { ImmutableList.of( sample(/* timeUs= */ 100_000, C.BUFFER_FLAG_KEY_FRAME, encodedCues), sample(/* timeUs= */ 450_000, C.BUFFER_FLAG_KEY_FRAME, encodedCues), + FakeSampleStream.FakeSampleStreamItem.format( + TEXT_FORMAT.buildUpon().setId("2").build()), sample(/* timeUs= */ 2_000_000, C.BUFFER_FLAG_KEY_FRAME, encodedCues), END_OF_STREAM_ITEM)); fakeSampleStream.writeData(/* startPositionUs= */ 0); @@ -93,17 +100,19 @@ public void renderText_doesNotReadAheadTooFar() throws Exception { @Test public void renderLegacyText_doesNotReadAheadTooFar() throws Exception { - ImmutableList cues = ImmutableList.of(new Cue.Builder().setText("test").build()); - byte[] encodedCues = cueEncoder.encode(cues, /* durationUs= */ 1_000_000); - TextRenderer renderer = new TextRenderer(cueList -> {}, /* outputLooper= */ null); + byte[] webvttBytes = "WEBVTT\n\n00:00.000 --> 00:01.000\ntest\n".getBytes(UTF_8); + List outputCueGroups = new ArrayList<>(); + TextRenderer renderer = new TextRenderer(outputCueGroups::add, /* outputLooper= */ null); renderer.experimentalSetLegacyDecodingEnabled(true); FakeSampleStream fakeSampleStream = createFakeSampleStream( LEGACY_TEXT_FORMAT, ImmutableList.of( - sample(/* timeUs= */ 100_000, C.BUFFER_FLAG_KEY_FRAME, encodedCues), - sample(/* timeUs= */ 450_000, C.BUFFER_FLAG_KEY_FRAME, encodedCues), - sample(/* timeUs= */ 2_000_000, C.BUFFER_FLAG_KEY_FRAME, encodedCues), + sample(/* timeUs= */ 100_000, C.BUFFER_FLAG_KEY_FRAME, webvttBytes), + sample(/* timeUs= */ 450_000, C.BUFFER_FLAG_KEY_FRAME, webvttBytes), + FakeSampleStream.FakeSampleStreamItem.format( + LEGACY_TEXT_FORMAT.buildUpon().setId("2").build()), + sample(/* timeUs= */ 2_000_000, C.BUFFER_FLAG_KEY_FRAME, webvttBytes), END_OF_STREAM_ITEM)); fakeSampleStream.writeData(/* startPositionUs= */ 0); renderer.enable( @@ -118,10 +127,13 @@ public void renderLegacyText_doesNotReadAheadTooFar() throws Exception { new MediaSource.MediaPeriodId(new Object())); renderer.start(); - // Render a few times to render as many samples as possible. - for (int i = 0; i < 5; i++) { - renderer.render(/* positionUs= */ 123_500_000, /* elapsedRealtimeUs= */ 0); - } + // Render until the first two samples are decoded on the background thread, freeing the + // decoder's input buffers so the renderer can peek the following format and sample. + runMainLooperUntil( + () -> { + renderer.render(/* positionUs= */ 123_500_000, /* elapsedRealtimeUs= */ 0); + return outputCueGroups.size() == 3; + }); // Verify that the last sample (at 2_000_000) is NOT yet read. assertThat(renderer.getReadingPositionUs()).isEqualTo(123_450_000); diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/trackselection/AdaptiveTrackSelectionTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/trackselection/AdaptiveTrackSelectionTest.java index 90d0f58441e..8464abe9c39 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/trackselection/AdaptiveTrackSelectionTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/trackselection/AdaptiveTrackSelectionTest.java @@ -683,6 +683,185 @@ public void updateSelectedTrack_withAdaptationCheckpoints_usesOnlyAllocatedBandw assertThat(adaptiveTrackSelection.getSelectedFormat()).isEqualTo(format3); } + @Test + public void + updateSelectedTrack_withDifferentLocationSteeringPriorities_selectsHigherPriorityTrack() { + Format format1 = videoFormat(/* bitrate= */ 500, /* width= */ 320, /* height= */ 240); + Format format2 = videoFormat(/* bitrate= */ 1000, /* width= */ 640, /* height= */ 480); + TrackGroup trackGroup = new TrackGroup(format1, format2); + AdaptiveTrackSelection adaptiveTrackSelection = prepareAdaptiveTrackSelection(trackGroup); + // Set bitrate estimate to 2000kbps so that both formats are eligible for selection. + when(mockBandwidthMeter.getBitrateEstimate()).thenReturn(2000L); + MediaChunkIterator[] iterators = new MediaChunkIterator[2]; + iterators[adaptiveTrackSelection.indexOf(format1)] = + createMediaChunkIteratorWithSteeringPriorityIndex( + TEST_CHUNK_DURATION_US, /* steeringPriorityIndex= */ 0); + iterators[adaptiveTrackSelection.indexOf(format2)] = + createMediaChunkIteratorWithSteeringPriorityIndex( + TEST_CHUNK_DURATION_US, /* steeringPriorityIndex= */ 1); + adaptiveTrackSelection.enable(); + + adaptiveTrackSelection.updateSelectedTrack( + /* playbackPositionUs= */ 0, + /* bufferedDurationUs= */ 0, + /* availableDurationUs= */ C.TIME_UNSET, + /* queue= */ ImmutableList.of(), + iterators); + + // Format 1 is selected because it has higher steering priority (0 < 1), despite lower bitrate. + assertThat(adaptiveTrackSelection.getSelectedFormat()).isEqualTo(format1); + } + + @Test + public void + updateSelectedTrack_whenSwitchingToPreferredSteeringLocationWithLowerBitrate_switchesImmediately() { + Format format1 = videoFormat(/* bitrate= */ 500, /* width= */ 320, /* height= */ 240); + Format format2 = videoFormat(/* bitrate= */ 1000, /* width= */ 640, /* height= */ 480); + TrackGroup trackGroup = new TrackGroup(format1, format2); + AdaptiveTrackSelection adaptiveTrackSelection = + prepareAdaptiveTrackSelectionWithMaxDurationForQualityDecreaseMs(trackGroup, 20000); + + // Initial state: Format2 (1000kbps) on Priority 0, Format1 (500kbps) on Priority 1. + MediaChunkIterator[] initialIterators = new MediaChunkIterator[2]; + initialIterators[adaptiveTrackSelection.indexOf(format1)] = + createMediaChunkIteratorWithSteeringPriorityIndex( + TEST_CHUNK_DURATION_US, /* steeringPriorityIndex= */ 1); + initialIterators[adaptiveTrackSelection.indexOf(format2)] = + createMediaChunkIteratorWithSteeringPriorityIndex( + TEST_CHUNK_DURATION_US, /* steeringPriorityIndex= */ 0); + when(mockBandwidthMeter.getBitrateEstimate()).thenReturn(2000L); + adaptiveTrackSelection.enable(); + adaptiveTrackSelection.updateSelectedTrack( + /* playbackPositionUs= */ 0, + /* bufferedDurationUs= */ 30_000_000, + /* availableDurationUs= */ C.TIME_UNSET, + /* queue= */ ImmutableList.of(), + initialIterators); + assertThat(adaptiveTrackSelection.getSelectedFormat()).isEqualTo(format2); + + // Steering priority update: Format1 (500kbps) becomes Priority 0, Format2 (1000kbps) becomes + // Priority 1. + MediaChunkIterator[] updatedIterators = new MediaChunkIterator[2]; + updatedIterators[adaptiveTrackSelection.indexOf(format1)] = + createMediaChunkIteratorWithSteeringPriorityIndex( + TEST_CHUNK_DURATION_US, /* steeringPriorityIndex= */ 0); + updatedIterators[adaptiveTrackSelection.indexOf(format2)] = + createMediaChunkIteratorWithSteeringPriorityIndex( + TEST_CHUNK_DURATION_US, /* steeringPriorityIndex= */ 1); + FakeMediaChunk chunk = + new FakeMediaChunk(format2, /* startTimeUs= */ 0, /* endTimeUs= */ 30_000_000); + ImmutableList queue = ImmutableList.of(chunk); + + // Buffered duration (30s) is greater than maxDurationForQualityDecreaseMs (20s) though. + // The switch to format1 (lower bitrate) MUST happen immediately because format1 is on the + // preferred steering location. + adaptiveTrackSelection.updateSelectedTrack( + /* playbackPositionUs= */ 0, + /* bufferedDurationUs= */ 30_000_000, + /* availableDurationUs= */ C.TIME_UNSET, + queue, + updatedIterators); + assertThat(adaptiveTrackSelection.getSelectedFormat()).isEqualTo(format1); + } + + @Test + public void + updateSelectedTrack_whenSwitchingToLessPreferredLocationDueToBandwidthDowngrade_defersSwitchDownWhenBufferIsSufficient() { + Format format1 = videoFormat(/* bitrate= */ 500, /* width= */ 320, /* height= */ 240); + Format format2 = videoFormat(/* bitrate= */ 1000, /* width= */ 640, /* height= */ 480); + TrackGroup trackGroup = new TrackGroup(format1, format2); + AdaptiveTrackSelection adaptiveTrackSelection = + prepareAdaptiveTrackSelectionWithMaxDurationForQualityDecreaseMs(trackGroup, 20000); + + // Initial state: Format2 (1000kbps) on Priority 0, Format1 (500kbps) on Priority 1. + MediaChunkIterator[] iterators = new MediaChunkIterator[2]; + iterators[adaptiveTrackSelection.indexOf(format1)] = + createMediaChunkIteratorWithSteeringPriorityIndex( + TEST_CHUNK_DURATION_US, /* steeringPriorityIndex= */ 1); + iterators[adaptiveTrackSelection.indexOf(format2)] = + createMediaChunkIteratorWithSteeringPriorityIndex( + TEST_CHUNK_DURATION_US, /* steeringPriorityIndex= */ 0); + when(mockBandwidthMeter.getBitrateEstimate()).thenReturn(2000L); + adaptiveTrackSelection.enable(); + adaptiveTrackSelection.updateSelectedTrack( + /* playbackPositionUs= */ 0, + /* bufferedDurationUs= */ 30_000_000, + /* availableDurationUs= */ C.TIME_UNSET, + /* queue= */ ImmutableList.of(), + iterators); + assertThat(adaptiveTrackSelection.getSelectedFormat()).isEqualTo(format2); + + // Bandwidth downgrade. + when(mockBandwidthMeter.getBitrateEstimate()).thenReturn(500L); + FakeMediaChunk chunk = + new FakeMediaChunk(format2, /* startTimeUs= */ 0, /* endTimeUs= */ 30_000_000); + ImmutableList queue = ImmutableList.of(chunk); + + // Bandwidth drops to 500kbps. Format1 (Priority 1) becomes the eligible track, but because + // format1 has a lower steering priority (1 > 0) and buffer is sufficient (30s >= 20s), + // the switch down is deferred to stay on the preferred location (format2) as long as possible. + adaptiveTrackSelection.updateSelectedTrack( + /* playbackPositionUs= */ 0, + /* bufferedDurationUs= */ 30_000_000, + /* availableDurationUs= */ C.TIME_UNSET, + queue, + iterators); + assertThat(adaptiveTrackSelection.getSelectedFormat()).isEqualTo(format2); + } + + @Test + public void + updateSelectedTrack_whenSwitchingToPreferredLocationWithHigherBitrate_defersSwitchUpWhenBufferInsufficient() { + Format format1 = videoFormat(/* bitrate= */ 500, /* width= */ 320, /* height= */ 240); + Format format2 = videoFormat(/* bitrate= */ 1000, /* width= */ 640, /* height= */ 480); + TrackGroup trackGroup = new TrackGroup(format1, format2); + AdaptiveTrackSelection adaptiveTrackSelection = + prepareAdaptiveTrackSelectionWithMinDurationForQualityIncreaseMs(trackGroup, 10_000); + + // Initial state: Format1 (500kbps) on Priority 0, Format2 (1000kbps) on Priority 1. + MediaChunkIterator[] initialIterators = new MediaChunkIterator[2]; + initialIterators[adaptiveTrackSelection.indexOf(format1)] = + createMediaChunkIteratorWithSteeringPriorityIndex( + TEST_CHUNK_DURATION_US, /* steeringPriorityIndex= */ 0); + initialIterators[adaptiveTrackSelection.indexOf(format2)] = + createMediaChunkIteratorWithSteeringPriorityIndex( + TEST_CHUNK_DURATION_US, /* steeringPriorityIndex= */ 1); + when(mockBandwidthMeter.getBitrateEstimate()).thenReturn(2000L); + adaptiveTrackSelection.enable(); + adaptiveTrackSelection.updateSelectedTrack( + /* playbackPositionUs= */ 0, + /* bufferedDurationUs= */ 5_000_000, + /* availableDurationUs= */ C.TIME_UNSET, + /* queue= */ ImmutableList.of(), + initialIterators); + assertThat(adaptiveTrackSelection.getSelectedFormat()).isEqualTo(format1); + + // Steering priority update: Format2 (1000kbps) becomes Priority 0, Format1 (500kbps) becomes + // Priority 1. + MediaChunkIterator[] updatedIterators = new MediaChunkIterator[2]; + updatedIterators[adaptiveTrackSelection.indexOf(format1)] = + createMediaChunkIteratorWithSteeringPriorityIndex( + TEST_CHUNK_DURATION_US, /* steeringPriorityIndex= */ 1); + updatedIterators[adaptiveTrackSelection.indexOf(format2)] = + createMediaChunkIteratorWithSteeringPriorityIndex( + TEST_CHUNK_DURATION_US, /* steeringPriorityIndex= */ 0); + FakeMediaChunk chunk = + new FakeMediaChunk(format1, /* startTimeUs= */ 0, /* endTimeUs= */ 5_000_000); + ImmutableList queue = ImmutableList.of(chunk); + + // Buffered duration (5s) < minDurationForQualityIncreaseMs (10s). + // Switching to format2 is a quality increase (higher bitrate) and improving of steering + // priority though, + // it must defer until buffer is sufficient. + adaptiveTrackSelection.updateSelectedTrack( + /* playbackPositionUs= */ 0, + /* bufferedDurationUs= */ 5_000_000, + /* availableDurationUs= */ C.TIME_UNSET, + queue, + updatedIterators); + assertThat(adaptiveTrackSelection.getSelectedFormat()).isEqualTo(format1); + } + @Test public void builderCreateTrackSelections_withSingleAdaptiveGroup_usesCorrectAdaptationCheckpoints() { @@ -984,26 +1163,36 @@ private MediaChunkIterator[] createMediaChunkIterators( MediaChunkIterator[] iterators = new MediaChunkIterator[trackGroup.length]; for (int i = 0; i < trackGroup.length; i++) { iterators[i] = - new BaseMediaChunkIterator(/* fromIndex= */ 0, /* toIndex= */ 0) { - @Override - public DataSpec getDataSpec() { - return new DataSpec.Builder().setUri("https://test.example").build(); - } - - @Override - public long getChunkStartTimeUs() { - return 123_456_789; - } - - @Override - public long getChunkEndTimeUs() { - return 123_456_789 + chunkDurationUs; - } - }; + createMediaChunkIteratorWithSteeringPriorityIndex(chunkDurationUs, Integer.MAX_VALUE); } return iterators; } + private static MediaChunkIterator createMediaChunkIteratorWithSteeringPriorityIndex( + long chunkDurationUs, int steeringPriorityIndex) { + return new BaseMediaChunkIterator(/* fromIndex= */ 0, /* toIndex= */ 0) { + @Override + public DataSpec getDataSpec() { + return new DataSpec.Builder().setUri("https://test.example").build(); + } + + @Override + public long getChunkStartTimeUs() { + return 123_456_789; + } + + @Override + public long getChunkEndTimeUs() { + return 123_456_789 + chunkDurationUs; + } + + @Override + public int getLocationSteeringPriorityIndex() { + return steeringPriorityIndex; + } + }; + } + private int[] selectedAllTracksInGroup(TrackGroup trackGroup) { int[] listIndices = new int[trackGroup.length]; for (int i = 0; i < trackGroup.length; i++) { diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/trackselection/DefaultTrackSelectorTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/trackselection/DefaultTrackSelectorTest.java index 60bb77928df..a9cb191a6f0 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/trackselection/DefaultTrackSelectorTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/trackselection/DefaultTrackSelectorTest.java @@ -37,9 +37,10 @@ import static androidx.media3.exoplayer.audio.AudioSink.OFFLOAD_MODE_ENABLED_GAPLESS_NOT_REQUIRED; import static androidx.media3.exoplayer.audio.AudioSink.OFFLOAD_MODE_ENABLED_GAPLESS_REQUIRED; import static com.google.common.truth.Truth.assertThat; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.doAnswer; import static org.mockito.Mockito.mock; import static org.mockito.Mockito.never; -import static org.mockito.Mockito.times; import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; import static org.robolectric.Shadows.shadowOf; @@ -183,6 +184,14 @@ public void setUp() { Context context = ApplicationProvider.getApplicationContext(); defaultParameters = Parameters.DEFAULT; trackSelector = new DefaultTrackSelector(context); + doAnswer( + invocation -> { + TrackSelectionParameters parameters = invocation.getArgument(0); + trackSelector.onParametersActivated(parameters); + return null; + }) + .when(invalidationListener) + .onTrackSelectionsInvalidated(any()); trackSelector.init(invalidationListener, bandwidthMeter); } @@ -539,37 +548,38 @@ public void selectTracksWithDisabledNoSampleRenderer() throws ExoPlaybackExcepti /** * Tests that track selector will not call {@link - * InvalidationListener#onTrackSelectionsInvalidated()} when it's set with default values of - * {@link Parameters}. + * InvalidationListener#onTrackSelectionsInvalidated(TrackSelectionParameters)} when it's set with + * default values of {@link Parameters}. */ @Test public void setParameterWithDefaultParametersDoesNotNotifyInvalidationListener() { trackSelector.setParameters(defaultParameters); - verify(invalidationListener, never()).onTrackSelectionsInvalidated(); + verify(invalidationListener, never()).onTrackSelectionsInvalidated(any()); } /** - * Tests that track selector will call {@link InvalidationListener#onTrackSelectionsInvalidated()} - * when it's set with non-default values of {@link Parameters}. + * Tests that track selector will call {@link + * InvalidationListener#onTrackSelectionsInvalidated(TrackSelectionParameters)} when it's set with + * non-default values of {@link Parameters}. */ @Test public void setParameterWithNonDefaultParameterNotifyInvalidationListener() { Parameters.Builder builder = defaultParameters.buildUpon().setPreferredAudioLanguage("eng"); trackSelector.setParameters(builder); - verify(invalidationListener).onTrackSelectionsInvalidated(); + verify(invalidationListener).onTrackSelectionsInvalidated(any()); } /** * Tests that track selector will not call {@link - * InvalidationListener#onTrackSelectionsInvalidated()} again when it's set with the same values - * of {@link Parameters}. + * InvalidationListener#onTrackSelectionsInvalidated(TrackSelectionParameters)} again when it's + * set with the same values of {@link Parameters}. */ @Test public void setParameterWithSameParametersDoesNotNotifyInvalidationListenerAgain() { Parameters.Builder builder = defaultParameters.buildUpon().setPreferredAudioLanguage("eng"); trackSelector.setParameters(builder); trackSelector.setParameters(builder); - verify(invalidationListener, times(1)).onTrackSelectionsInvalidated(); + verify(invalidationListener).onTrackSelectionsInvalidated(any()); } /** @@ -3048,6 +3058,108 @@ public void selectTracks_with720pHdrAnd1080pSdr_prefersHdrOverResolution() throw assertFixedSelection(result.selections[0], trackGroups, format720pHdr); } + @Test + public void + selectTracks_with720pDolbyVisionHdrFallbackToHevcAnd1080pSdr_prefersHdrOverResolution() + throws Exception { + ColorInfo hdrColorInfo = + new ColorInfo.Builder().setColorTransfer(C.COLOR_TRANSFER_ST2084).build(); + Format format720pDvHdr = + new Format.Builder() + .setId("720p_dv_hdr") + .setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION) + .setWidth(1280) + .setHeight(720) + .setColorInfo(hdrColorInfo) + .build(); + Format format1080pSdr = + new Format.Builder() + .setId("1080p_sdr") + .setSampleMimeType(MimeTypes.VIDEO_H265) + .setWidth(1920) + .setHeight(1080) + .build(); + TrackGroupArray trackGroups = + new TrackGroupArray(new TrackGroup(format720pDvHdr), new TrackGroup(format1080pSdr)); + @Capabilities + int capabilitiesDv = + RendererCapabilities.create( + FORMAT_HANDLED, + ADAPTIVE_NOT_SEAMLESS, + TUNNELING_NOT_SUPPORTED, + HARDWARE_ACCELERATION_SUPPORTED, + DECODER_SUPPORT_FALLBACK_MIMETYPE); + @Capabilities + int capabilitiesSdr = + RendererCapabilities.create( + FORMAT_HANDLED, + ADAPTIVE_NOT_SEAMLESS, + TUNNELING_NOT_SUPPORTED, + HARDWARE_ACCELERATION_SUPPORTED, + DECODER_SUPPORT_PRIMARY); + ImmutableMap rendererCapabilitiesMap = + ImmutableMap.of("720p_dv_hdr", capabilitiesDv, "1080p_sdr", capabilitiesSdr); + RendererCapabilities rendererCapabilities = + new FakeMappedRendererCapabilities(C.TRACK_TYPE_VIDEO, rendererCapabilitiesMap); + + TrackSelectorResult result = + trackSelector.selectTracks( + new RendererCapabilities[] {rendererCapabilities}, trackGroups, periodId, TIMELINE); + + assertFixedSelection(result.selections[0], trackGroups, format720pDvHdr); + } + + @Test + public void + selectTracks_withDolbyVisionHdrFallbackToHevcAndSdrSameResolution_selectsDolbyVisionHdr() + throws Exception { + ColorInfo hdrColorInfo = + new ColorInfo.Builder().setColorTransfer(C.COLOR_TRANSFER_ST2084).build(); + Format formatDvHdr = + new Format.Builder() + .setId("dv_hdr") + .setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION) + .setWidth(1920) + .setHeight(1080) + .setColorInfo(hdrColorInfo) + .build(); + Format formatSdr = + new Format.Builder() + .setId("sdr") + .setSampleMimeType(MimeTypes.VIDEO_H265) + .setWidth(1920) + .setHeight(1080) + .build(); + TrackGroupArray trackGroups = + new TrackGroupArray(new TrackGroup(formatDvHdr), new TrackGroup(formatSdr)); + @Capabilities + int capabilitiesDv = + RendererCapabilities.create( + FORMAT_HANDLED, + ADAPTIVE_NOT_SEAMLESS, + TUNNELING_NOT_SUPPORTED, + HARDWARE_ACCELERATION_SUPPORTED, + DECODER_SUPPORT_FALLBACK_MIMETYPE); + @Capabilities + int capabilitiesSdr = + RendererCapabilities.create( + FORMAT_HANDLED, + ADAPTIVE_NOT_SEAMLESS, + TUNNELING_NOT_SUPPORTED, + HARDWARE_ACCELERATION_SUPPORTED, + DECODER_SUPPORT_PRIMARY); + ImmutableMap rendererCapabilitiesMap = + ImmutableMap.of("dv_hdr", capabilitiesDv, "sdr", capabilitiesSdr); + RendererCapabilities rendererCapabilities = + new FakeMappedRendererCapabilities(C.TRACK_TYPE_VIDEO, rendererCapabilitiesMap); + + TrackSelectorResult result = + trackSelector.selectTracks( + new RendererCapabilities[] {rendererCapabilities}, trackGroups, periodId, TIMELINE); + + assertFixedSelection(result.selections[0], trackGroups, formatDvHdr); + } + @Test public void selectTracks_withAvcAndHevcSameResolution_prefersHevcOverAvc() throws Exception { Format formatAvc = diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/upstream/DefaultLoadErrorHandlingPolicyTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/upstream/DefaultLoadErrorHandlingPolicyTest.java index f2e6ffe2f9f..ad53ea90cdc 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/upstream/DefaultLoadErrorHandlingPolicyTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/upstream/DefaultLoadErrorHandlingPolicyTest.java @@ -47,14 +47,10 @@ public final class DefaultLoadErrorHandlingPolicyTest { private static final LoadEventInfo PLACEHOLDER_LOAD_EVENT_INFO = - new LoadEventInfo( - LoadEventInfo.getNewId(), - new DataSpec(Uri.EMPTY), - Uri.EMPTY, - /* responseHeaders= */ Collections.emptyMap(), - /* elapsedRealtimeMs= */ 5000, - /* loadDurationMs= */ 1000, - /* bytesLoaded= */ 0); + new LoadEventInfo.Builder( + LoadEventInfo.getNewId(), new DataSpec(Uri.EMPTY), /* elapsedRealtimeMs= */ 5000) + .setLoadDurationMs(1000) + .build(); private static final MediaLoadData PLACEHOLDER_MEDIA_LOAD_DATA = new MediaLoadData(/* dataType= */ C.DATA_TYPE_UNKNOWN); diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/upstream/contentsteering/SteeringManifestTrackerTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/upstream/contentsteering/SteeringManifestTrackerTest.java index 4074be65309..3d1e600228b 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/upstream/contentsteering/SteeringManifestTrackerTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/upstream/contentsteering/SteeringManifestTrackerTest.java @@ -15,6 +15,7 @@ */ package androidx.media3.exoplayer.upstream.contentsteering; +import static androidx.media3.common.util.Util.createHandlerForCurrentLooper; import static androidx.media3.exoplayer.upstream.contentsteering.SteeringManifestTracker.FALLBACK_DELAY_UNTIL_NEXT_LOAD_MS; import static androidx.media3.test.utils.robolectric.RobolectricUtil.runMainLooperUntil; import static com.google.common.truth.Truth.assertThat; @@ -23,17 +24,18 @@ import static org.mockito.ArgumentMatchers.eq; import static org.mockito.Mockito.atLeastOnce; import static org.mockito.Mockito.never; -import static org.mockito.Mockito.spy; -import static org.mockito.Mockito.times; import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; import static org.robolectric.Shadows.shadowOf; import android.net.Uri; import android.os.Looper; +import androidx.annotation.Nullable; import androidx.media3.common.C; import androidx.media3.datasource.DefaultDataSource; import androidx.media3.exoplayer.source.LoadEventInfo; +import androidx.media3.exoplayer.source.MediaLoadData; +import androidx.media3.exoplayer.source.MediaSource.MediaPeriodId; import androidx.media3.exoplayer.source.MediaSourceEventListener; import androidx.media3.test.utils.FakeClock; import androidx.test.core.app.ApplicationProvider; @@ -83,6 +85,9 @@ public final class SteeringManifestTrackerTest { private FakeClock clock; private AtomicInteger loadStartedCount; private AtomicInteger loadCompletedOrErrorCount; + private List completedDataTypes; + private List errorDataTypes; + private List errorWasCanceled; private MediaSourceEventListener.EventDispatcher eventDispatcher; @Before @@ -98,36 +103,52 @@ public void setUp() { clock = new FakeClock(/* isAutoAdvancing= */ true); loadStartedCount = new AtomicInteger(); loadCompletedOrErrorCount = new AtomicInteger(); - eventDispatcher = - spy( - new MediaSourceEventListener.EventDispatcher() { - - @Override - public void loadStarted( - LoadEventInfo loadEventInfo, @C.DataType int dataType, int retryCount) { - loadStartedCount.incrementAndGet(); - } - - @Override - public void loadCompleted(LoadEventInfo loadEventInfo, @C.DataType int dataType) { - loadCompletedOrErrorCount.incrementAndGet(); - // We advance the clock for FALLBACK_DELAY_UNTIL_NEXT_LOAD_MS (5 min) to avoid - // that the test times out while waiting for the next reload. - clock.advanceTime(FALLBACK_DELAY_UNTIL_NEXT_LOAD_MS); - } - - @Override - public void loadError( - LoadEventInfo loadEventInfo, - @C.DataType int dataType, - IOException error, - boolean wasCanceled) { - loadCompletedOrErrorCount.incrementAndGet(); - // We advance the clock for FALLBACK_DELAY_UNTIL_NEXT_LOAD_MS (5 min) to avoid - // that the test times out while waiting for the next reload. - clock.advanceTime(FALLBACK_DELAY_UNTIL_NEXT_LOAD_MS); - } - }); + completedDataTypes = new ArrayList<>(); + errorDataTypes = new ArrayList<>(); + errorWasCanceled = new ArrayList<>(); + eventDispatcher = new MediaSourceEventListener.EventDispatcher(); + eventDispatcher.addEventListener( + createHandlerForCurrentLooper(), + new MediaSourceEventListener() { + @Override + public void onLoadStarted( + int windowIndex, + @Nullable MediaPeriodId mediaPeriodId, + LoadEventInfo loadEventInfo, + MediaLoadData mediaLoadData, + int retryCount) { + loadStartedCount.incrementAndGet(); + } + + @Override + public void onLoadCompleted( + int windowIndex, + @Nullable MediaPeriodId mediaPeriodId, + LoadEventInfo loadEventInfo, + MediaLoadData mediaLoadData) { + completedDataTypes.add(mediaLoadData.dataType); + loadCompletedOrErrorCount.incrementAndGet(); + // We advance the clock for FALLBACK_DELAY_UNTIL_NEXT_LOAD_MS (5 min) to avoid + // that the test times out while waiting for the next reload. + clock.advanceTime(FALLBACK_DELAY_UNTIL_NEXT_LOAD_MS); + } + + @Override + public void onLoadError( + int windowIndex, + @Nullable MediaPeriodId mediaPeriodId, + LoadEventInfo loadEventInfo, + MediaLoadData mediaLoadData, + IOException error, + boolean wasCanceled) { + errorDataTypes.add(mediaLoadData.dataType); + errorWasCanceled.add(wasCanceled); + loadCompletedOrErrorCount.incrementAndGet(); + // We advance the clock for FALLBACK_DELAY_UNTIL_NEXT_LOAD_MS (5 min) to avoid + // that the test times out while waiting for the next reload. + clock.advanceTime(FALLBACK_DELAY_UNTIL_NEXT_LOAD_MS); + } + }); } @After @@ -177,8 +198,12 @@ public void start_loadSteeringManifestSuccessfullyAndScheduleReloads() throws Ex .inOrder(); verify(mockCallback, never()) .onSteeringManifestLoadError(/* error= */ any(), /* canceled= */ anyBoolean()); - verify(eventDispatcher, times(4)) - .loadCompleted(/* loadEventInfo= */ any(), eq(C.DATA_TYPE_STEERING_MANIFEST)); + assertThat(completedDataTypes) + .containsExactly( + C.DATA_TYPE_STEERING_MANIFEST, + C.DATA_TYPE_STEERING_MANIFEST, + C.DATA_TYPE_STEERING_MANIFEST, + C.DATA_TYPE_STEERING_MANIFEST); assertThat(timeoutExceptionRef.get()).isNull(); } @@ -214,14 +239,10 @@ public void start_afterResponse410_noMoreRetryScheduled() throws Exception { .containsExactly("CDN-B", "CDN-A") .inOrder(); verify(mockCallback).onSteeringManifestLoadError(/* error= */ any(), /* canceled= */ eq(true)); - verify(eventDispatcher, times(2)) - .loadCompleted(/* loadEventInfo= */ any(), eq(C.DATA_TYPE_STEERING_MANIFEST)); - verify(eventDispatcher) - .loadError( - /* loadEventInfo= */ any(), - eq(C.DATA_TYPE_STEERING_MANIFEST), - /* error= */ any(), - /* wasCanceled= */ eq(true)); + assertThat(completedDataTypes) + .containsExactly(C.DATA_TYPE_STEERING_MANIFEST, C.DATA_TYPE_STEERING_MANIFEST); + assertThat(errorDataTypes).containsExactly(C.DATA_TYPE_STEERING_MANIFEST); + assertThat(errorWasCanceled).containsExactly(true); assertThat(timeoutExceptionRef.get()).isNotNull(); } @@ -256,14 +277,10 @@ public void start_afterResponse429WithValidRetryAfterHeader_retryScheduled() thr .containsExactly("CDN-B", "CDN-A") .inOrder(); verify(mockCallback).onSteeringManifestLoadError(/* error= */ any(), /* canceled= */ eq(false)); - verify(eventDispatcher, times(2)) - .loadCompleted(/* loadEventInfo= */ any(), eq(C.DATA_TYPE_STEERING_MANIFEST)); - verify(eventDispatcher) - .loadError( - /* loadEventInfo= */ any(), - eq(C.DATA_TYPE_STEERING_MANIFEST), - /* error= */ any(), - /* wasCanceled= */ eq(false)); + assertThat(completedDataTypes) + .containsExactly(C.DATA_TYPE_STEERING_MANIFEST, C.DATA_TYPE_STEERING_MANIFEST); + assertThat(errorDataTypes).containsExactly(C.DATA_TYPE_STEERING_MANIFEST); + assertThat(errorWasCanceled).containsExactly(false); assertThat(timeoutExceptionRef.get()).isNull(); } @@ -299,14 +316,10 @@ public void start_afterResponse429WithInvalidRetryAfterHeader_retryScheduled() t .containsExactly("CDN-B", "CDN-A") .inOrder(); verify(mockCallback).onSteeringManifestLoadError(/* error= */ any(), /* canceled= */ eq(false)); - verify(eventDispatcher, times(2)) - .loadCompleted(/* loadEventInfo= */ any(), eq(C.DATA_TYPE_STEERING_MANIFEST)); - verify(eventDispatcher) - .loadError( - /* loadEventInfo= */ any(), - eq(C.DATA_TYPE_STEERING_MANIFEST), - /* error= */ any(), - /* wasCanceled= */ eq(false)); + assertThat(completedDataTypes) + .containsExactly(C.DATA_TYPE_STEERING_MANIFEST, C.DATA_TYPE_STEERING_MANIFEST); + assertThat(errorDataTypes).containsExactly(C.DATA_TYPE_STEERING_MANIFEST); + assertThat(errorWasCanceled).containsExactly(false); assertThat(timeoutExceptionRef.get()).isNull(); } @@ -342,14 +355,10 @@ public void start_afterOtherErrorResponses_retryScheduled() throws Exception { .containsExactly("CDN-B", "CDN-A") .inOrder(); verify(mockCallback).onSteeringManifestLoadError(/* error= */ any(), /* canceled= */ eq(false)); - verify(eventDispatcher, times(2)) - .loadCompleted(/* loadEventInfo= */ any(), eq(C.DATA_TYPE_STEERING_MANIFEST)); - verify(eventDispatcher) - .loadError( - /* loadEventInfo= */ any(), - eq(C.DATA_TYPE_STEERING_MANIFEST), - /* error= */ any(), - /* wasCanceled= */ eq(false)); + assertThat(completedDataTypes) + .containsExactly(C.DATA_TYPE_STEERING_MANIFEST, C.DATA_TYPE_STEERING_MANIFEST); + assertThat(errorDataTypes).containsExactly(C.DATA_TYPE_STEERING_MANIFEST); + assertThat(errorWasCanceled).containsExactly(false); assertThat(timeoutExceptionRef.get()).isNull(); } @@ -380,14 +389,9 @@ public void start_failWhenLoadingFirstSteeringManifest_retryScheduled() throws E .containsExactly("CDN-A", "CDN-B") .inOrder(); verify(mockCallback).onSteeringManifestLoadError(/* error= */ any(), /* canceled= */ eq(false)); - verify(eventDispatcher) - .loadCompleted(/* loadEventInfo= */ any(), eq(C.DATA_TYPE_STEERING_MANIFEST)); - verify(eventDispatcher) - .loadError( - /* loadEventInfo= */ any(), - eq(C.DATA_TYPE_STEERING_MANIFEST), - /* error= */ any(), - /* wasCanceled= */ eq(false)); + assertThat(completedDataTypes).containsExactly(C.DATA_TYPE_STEERING_MANIFEST); + assertThat(errorDataTypes).containsExactly(C.DATA_TYPE_STEERING_MANIFEST); + assertThat(errorWasCanceled).containsExactly(false); assertThat(timeoutExceptionRef.get()).isNull(); } diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/util/EventLoggerTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/util/EventLoggerTest.java index e5c21e01c7a..2977d74f31a 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/util/EventLoggerTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/util/EventLoggerTest.java @@ -461,10 +461,11 @@ public void onVideoSizeChanged() { public void onLoadError() { eventLogger.onLoadError( EVENT_TIME, - new LoadEventInfo( - /* loadTaskId= */ 1, - new DataSpec(Uri.parse("http://foo.test")), - /* elapsedRealtimeMs= */ 123), + new LoadEventInfo.Builder( + /* loadTaskId= */ 1, + new DataSpec(Uri.parse("http://foo.test")), + /* elapsedRealtimeMs= */ 123) + .build(), MEDIA_LOAD_DATA, new IOException("test msg"), /* wasCanceled= */ true); diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/Av1ObuUtilTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/Av1ObuUtilTest.java index a76cb40b104..aaece4f4c51 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/Av1ObuUtilTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/Av1ObuUtilTest.java @@ -27,58 +27,66 @@ @RunWith(AndroidJUnit4.class) public final class Av1ObuUtilTest { - private static final ImmutableByteArray - TEMPORAL_DELIMITER_SEQUENCE_HEADER_AND_NON_HDR10_PLUS_METADATA = - ImmutableByteArray.ofUnsigned( - 0x12, 0x00, 0x0a, 0x0e, 0x00, 0x00, 0x00, 0x2d, 0x4c, 0xff, 0xb3, 0xcc, 0xaf, 0x95, - 0x09, 0x12, 0x09, 0x04, 0x2a, 0x34, 0x04, 0xb5, 0x00, 0x90, 0x00, 0x01, 0xe0, 0x40, - 0x59, 0xdc, 0x1b, 0x00, 0x00, 0x00, 0x28, 0x03, 0xe8, 0x04, 0xe2, 0x06, 0xd6, 0x09, - 0xc4, 0x0f, 0xa0, 0x13, 0x3e, 0x27, 0x10, 0x2a, 0x08, 0x33, 0x48, 0x3f, 0x71, 0x51, - 0x60, 0x59, 0xdc, 0x46, 0x50, 0x33, 0x93, 0x32, 0xf2, 0x36, 0x71, 0x3b, 0x19, 0x3c, - 0xcc, 0x80); + private static final ImmutableByteArray TEMPORAL_DELIMITER_OBU = + ImmutableByteArray.ofUnsigned(0x12, 0x00); - private static final ImmutableByteArray MALFORMED_METADATA = + private static final ImmutableByteArray SEQUENCE_HEADER_OBU = ImmutableByteArray.ofUnsigned( - 0x2a, 0x01, 0xb5); // Metadata OBU type, length 1, incomplete payload + 0x0a, 0x0e, 0x00, 0x00, 0x00, 0x2d, 0x4c, 0xff, 0xb3, 0xcc, 0xaf, 0x95, 0x09, 0x12, 0x09, + 0x04); + + private static final ImmutableByteArray HAGC_METADATA_OBU = + ImmutableByteArray.ofUnsigned( + 0x2a, 0x34, 0x04, 0xb5, 0x00, 0x90, 0x00, 0x01, 0xe0, 0x40, 0x59, 0xdc, 0x1b, 0x00, 0x00, + 0x00, 0x28, 0x03, 0xe8, 0x04, 0xe2, 0x06, 0xd6, 0x09, 0xc4, 0x0f, 0xa0, 0x13, 0x3e, 0x27, + 0x10, 0x2a, 0x08, 0x33, 0x48, 0x3f, 0x71, 0x51, 0x60, 0x59, 0xdc, 0x46, 0x50, 0x33, 0x93, + 0x32, 0xf2, 0x36, 0x71, 0x3b, 0x19, 0x3c, 0xcc, 0x80); + + // Same as HAGC_METADATA_OBU, but with two zero padding bytes after the trailing bits (and the + // OBU size updated accordingly from 0x34 to 0x36). + private static final ImmutableByteArray HAGC_METADATA_OBU_WITH_ZERO_PADDING = + ImmutableByteArray.ofUnsigned( + 0x2a, 0x36, 0x04, 0xb5, 0x00, 0x90, 0x00, 0x01, 0xe0, 0x40, 0x59, 0xdc, 0x1b, 0x00, 0x00, + 0x00, 0x28, 0x03, 0xe8, 0x04, 0xe2, 0x06, 0xd6, 0x09, 0xc4, 0x0f, 0xa0, 0x13, 0x3e, 0x27, + 0x10, 0x2a, 0x08, 0x33, 0x48, 0x3f, 0x71, 0x51, 0x60, 0x59, 0xdc, 0x46, 0x50, 0x33, 0x93, + 0x32, 0xf2, 0x36, 0x71, 0x3b, 0x19, 0x3c, 0xcc, 0x80, 0x00, 0x00); + + private static final ImmutableByteArray HDR10_PLUS_METADATA_OBU = + ImmutableByteArray.ofUnsigned( + 0x2a, 0x34, 0x04, 0xb5, 0x00, 0x3c, 0x00, 0x01, 0x04, 0x40, 0x59, 0xdc, 0x1b, 0x00, 0x00, + 0x00, 0x28, 0x03, 0xe8, 0x04, 0xe2, 0x06, 0xd6, 0x09, 0xc4, 0x0f, 0xa0, 0x13, 0x3e, 0x27, + 0x10, 0x2a, 0x08, 0x33, 0x48, 0x3f, 0x71, 0x51, 0x60, 0x59, 0xdc, 0x46, 0x50, 0x33, 0x93, + 0x32, 0xf2, 0x36, 0x71, 0x3b, 0x19, 0x3c, 0xcc, 0x80); private static final ImmutableByteArray NO_METADATA = + ImmutableByteArray.concat(TEMPORAL_DELIMITER_OBU, SEQUENCE_HEADER_OBU); + + private static final ImmutableByteArray MALFORMED_METADATA = ImmutableByteArray.ofUnsigned( - 0x12, 0x00, 0x0a, 0x0e, 0x00, 0x00, 0x00, 0x2d, 0x4c, 0xff, 0xb3, 0xcc, 0xaf, 0x95, 0x09); + 0x2a, 0x01, 0xb5); // Metadata OBU type, length 1, incomplete payload + + private static final ImmutableByteArray TEMPORAL_DELIMITER_SEQUENCE_HEADER_AND_HAGC_METADATA = + ImmutableByteArray.concat(TEMPORAL_DELIMITER_OBU, SEQUENCE_HEADER_OBU, HAGC_METADATA_OBU); private static final ImmutableByteArray TEMPORAL_DELIMITER_SEQUENCE_HEADER_AND_HDR10_PLUS_METADATA = - ImmutableByteArray.ofUnsigned( - 0x12, 0x00, 0x0a, 0x0e, 0x00, 0x00, 0x00, 0x2d, 0x4c, 0xff, 0xb3, 0xcc, 0xaf, 0x95, - 0x09, 0x12, 0x09, 0x04, 0x2a, 0x34, 0x04, 0xb5, 0x00, 0x3c, 0x00, 0x01, 0x04, 0x40, - 0x59, 0xdc, 0x1b, 0x00, 0x00, 0x00, 0x28, 0x03, 0xe8, 0x04, 0xe2, 0x06, 0xd6, 0x09, - 0xc4, 0x0f, 0xa0, 0x13, 0x3e, 0x27, 0x10, 0x2a, 0x08, 0x33, 0x48, 0x3f, 0x71, 0x51, - 0x60, 0x59, 0xdc, 0x46, 0x50, 0x33, 0x93, 0x32, 0xf2, 0x36, 0x71, 0x3b, 0x19, 0x3c, - 0xcc, 0x80); + ImmutableByteArray.concat( + TEMPORAL_DELIMITER_OBU, SEQUENCE_HEADER_OBU, HDR10_PLUS_METADATA_OBU); - private static final ImmutableByteArray - SEQUENCE_HEADER_NON_HDR10_PLUS_METADATA_AND_HDR10_PLUS_METADATA = - ImmutableByteArray.ofUnsigned( - 0x0a, 0x0e, 0x00, 0x00, 0x00, 0x2d, 0x4c, 0xff, 0xb3, 0xcc, 0xaf, 0x95, 0x09, 0x12, - 0x09, 0x04, 0x2a, 0x34, 0x04, 0xb5, 0x00, 0x90, 0x00, 0x01, 0xe0, 0x40, 0x59, 0xdc, - 0x1b, 0x00, 0x00, 0x00, 0x28, 0x03, 0xe8, 0x04, 0xe2, 0x06, 0xd6, 0x09, 0xc4, 0x0f, - 0xa0, 0x13, 0x3e, 0x27, 0x10, 0x2a, 0x08, 0x33, 0x48, 0x3f, 0x71, 0x51, 0x60, 0x59, - 0xdc, 0x46, 0x50, 0x33, 0x93, 0x32, 0xf2, 0x36, 0x71, 0x3b, 0x19, 0x3c, 0xcc, 0x80, - 0x2a, 0x34, 0x04, 0xb5, 0x00, 0x3c, 0x00, 0x01, 0x04, 0x40, 0x59, 0xdc, 0x1b, 0x00, - 0x00, 0x00, 0x28, 0x03, 0xe8, 0x04, 0xe2, 0x06, 0xd6, 0x09, 0xc4, 0x0f, 0xa0, 0x13, - 0x3e, 0x27, 0x10, 0x2a, 0x08, 0x33, 0x48, 0x3f, 0x71, 0x51, 0x60, 0x59, 0xdc, 0x46, - 0x50, 0x33, 0x93, 0x32, 0xf2, 0x36, 0x71, 0x3b, 0x19, 0x3c, 0xcc, 0x80); + private static final ImmutableByteArray SEQUENCE_HEADER_HAGC_METADATA_AND_HDR10_PLUS_METADATA = + ImmutableByteArray.concat(SEQUENCE_HEADER_OBU, HAGC_METADATA_OBU, HDR10_PLUS_METADATA_OBU); @Test public void stripNonHdr10PlusT35Metadata_rewritesNonHdr10PlusMetadataObu() { ByteBuffer buffer = - ByteBuffer.wrap(TEMPORAL_DELIMITER_SEQUENCE_HEADER_AND_NON_HDR10_PLUS_METADATA.toArray()); + ByteBuffer.wrap(TEMPORAL_DELIMITER_SEQUENCE_HEADER_AND_HAGC_METADATA.toArray()); int originalPosition = buffer.position(); int originalLimit = buffer.limit(); Av1ObuUtil.stripNonHdr10PlusT35Metadata(buffer); assertThat(buffer.position()).isEqualTo(originalPosition); assertThat(buffer.limit()).isEqualTo(originalLimit); ByteBuffer expectedBuffer = - ByteBuffer.wrap(TEMPORAL_DELIMITER_SEQUENCE_HEADER_AND_NON_HDR10_PLUS_METADATA.toArray()); + ByteBuffer.wrap(TEMPORAL_DELIMITER_SEQUENCE_HEADER_AND_HAGC_METADATA.toArray()); expectedBuffer.put(20, (byte) 0x1F); assertThat(buffer).isEqualTo(expectedBuffer); } @@ -86,14 +94,14 @@ public void stripNonHdr10PlusT35Metadata_rewritesNonHdr10PlusMetadataObu() { @Test public void stripNonHdr10PlusT35Metadata_rewritesNonHdr10PlusAndLeavesHdr10PlusMetadataObu() { ByteBuffer buffer = - ByteBuffer.wrap(SEQUENCE_HEADER_NON_HDR10_PLUS_METADATA_AND_HDR10_PLUS_METADATA.toArray()); + ByteBuffer.wrap(SEQUENCE_HEADER_HAGC_METADATA_AND_HDR10_PLUS_METADATA.toArray()); int originalPosition = buffer.position(); int originalLimit = buffer.limit(); Av1ObuUtil.stripNonHdr10PlusT35Metadata(buffer); assertThat(buffer.position()).isEqualTo(originalPosition); assertThat(buffer.limit()).isEqualTo(originalLimit); ByteBuffer expectedBuffer = - ByteBuffer.wrap(SEQUENCE_HEADER_NON_HDR10_PLUS_METADATA_AND_HDR10_PLUS_METADATA.toArray()); + ByteBuffer.wrap(SEQUENCE_HEADER_HAGC_METADATA_AND_HDR10_PLUS_METADATA.toArray()); expectedBuffer.put(18, (byte) 0x1F); assertThat(buffer).isEqualTo(expectedBuffer); } @@ -141,11 +149,11 @@ public void stripNonHdr10PlusT35Metadata_leavesBufferUnchanged_whenNoMetadata() @Test public void stripAllT35Metadata_rewritesNonHdr10PlusMetadataObus() { ByteBuffer buffer = - ByteBuffer.wrap(TEMPORAL_DELIMITER_SEQUENCE_HEADER_AND_NON_HDR10_PLUS_METADATA.toArray()); + ByteBuffer.wrap(TEMPORAL_DELIMITER_SEQUENCE_HEADER_AND_HAGC_METADATA.toArray()); int originalPosition = buffer.position(); int originalLimit = buffer.limit(); ByteBuffer expectedBuffer = - ByteBuffer.wrap(TEMPORAL_DELIMITER_SEQUENCE_HEADER_AND_NON_HDR10_PLUS_METADATA.toArray()); + ByteBuffer.wrap(TEMPORAL_DELIMITER_SEQUENCE_HEADER_AND_HAGC_METADATA.toArray()); expectedBuffer.put(20, (byte) 0x1F); Av1ObuUtil.stripAllT35Metadata(buffer); @@ -158,11 +166,11 @@ public void stripAllT35Metadata_rewritesNonHdr10PlusMetadataObus() { @Test public void stripAllT35Metadata_rewritesBothHdr10PlusAndNonHdr10PlusMetadataObus() { ByteBuffer buffer = - ByteBuffer.wrap(SEQUENCE_HEADER_NON_HDR10_PLUS_METADATA_AND_HDR10_PLUS_METADATA.toArray()); + ByteBuffer.wrap(SEQUENCE_HEADER_HAGC_METADATA_AND_HDR10_PLUS_METADATA.toArray()); int originalPosition = buffer.position(); int originalLimit = buffer.limit(); ByteBuffer expectedBuffer = - ByteBuffer.wrap(SEQUENCE_HEADER_NON_HDR10_PLUS_METADATA_AND_HDR10_PLUS_METADATA.toArray()); + ByteBuffer.wrap(SEQUENCE_HEADER_HAGC_METADATA_AND_HDR10_PLUS_METADATA.toArray()); expectedBuffer.put(18, (byte) 0x1F); expectedBuffer.put(72, (byte) 0x1F); @@ -185,4 +193,145 @@ public void stripNonHdr10PlusT35Metadata_doesNotCrash_whenMalformedMetadata() { assertThat(buffer.limit()).isEqualTo(originalLimit); assertThat(buffer).isEqualTo(ByteBuffer.wrap(MALFORMED_METADATA.toArray())); } + + @Test + public void + extractHagcMetadataAndStripAllT35Metadata_extractsPayloadWithoutHeaderAndTrailingBits() { + ByteBuffer buffer = + ByteBuffer.wrap(TEMPORAL_DELIMITER_SEQUENCE_HEADER_AND_HAGC_METADATA.toArray()); + int originalPosition = buffer.position(); + int originalLimit = buffer.limit(); + + byte[] hagcData = Av1ObuUtil.extractHagcMetadataAndStripAllT35Metadata(buffer); + + assertThat(buffer.position()).isEqualTo(originalPosition); + assertThat(buffer.limit()).isEqualTo(originalLimit); + ByteBuffer expectedBuffer = + ByteBuffer.wrap(TEMPORAL_DELIMITER_SEQUENCE_HEADER_AND_HAGC_METADATA.toArray()); + expectedBuffer.put(20, (byte) 0x1F); + assertThat(buffer).isEqualTo(expectedBuffer); + // The expected HAGC payload will contain only the HAGC metadata without the OBU headers (3 + // bytes) and the T.35 headers (5 bytes) in the prefix and trailing OBU byte (1 byte) in the + // suffix. + ImmutableByteArray expectedHagcPayload = + HAGC_METADATA_OBU.subArray(8, HAGC_METADATA_OBU.length() - 1); + assertThat(hagcData).isEqualTo(expectedHagcPayload.toArray()); + } + + @Test + public void + extractHagcMetadataAndStripAllT35Metadata_withZeroPadding_excludesTrailingBitsAndPadding() { + ImmutableByteArray input = + ImmutableByteArray.concat( + TEMPORAL_DELIMITER_OBU, SEQUENCE_HEADER_OBU, HAGC_METADATA_OBU_WITH_ZERO_PADDING); + ByteBuffer buffer = ByteBuffer.wrap(input.toArray()); + int originalPosition = buffer.position(); + int originalLimit = buffer.limit(); + + byte[] hagcData = Av1ObuUtil.extractHagcMetadataAndStripAllT35Metadata(buffer); + + assertThat(buffer.position()).isEqualTo(originalPosition); + assertThat(buffer.limit()).isEqualTo(originalLimit); + ByteBuffer expectedBuffer = ByteBuffer.wrap(input.toArray()); + expectedBuffer.put(20, (byte) 0x1F); + assertThat(buffer).isEqualTo(expectedBuffer); + // The extracted payload must be identical to the unpadded case: the trailing 0x80 byte and the + // zero padding bytes after it are both excluded. + ImmutableByteArray expectedHagcPayload = + HAGC_METADATA_OBU.subArray(8, HAGC_METADATA_OBU.length() - 1); + assertThat(hagcData).isEqualTo(expectedHagcPayload.toArray()); + } + + @Test + public void extractHagcMetadataAndStripAllT35Metadata_withHdr10Plus_rewritesAllT35Obus() { + ByteBuffer buffer = + ByteBuffer.wrap(SEQUENCE_HEADER_HAGC_METADATA_AND_HDR10_PLUS_METADATA.toArray()); + + byte[] hagcData = Av1ObuUtil.extractHagcMetadataAndStripAllT35Metadata(buffer); + + ByteBuffer expectedBuffer = + ByteBuffer.wrap(SEQUENCE_HEADER_HAGC_METADATA_AND_HDR10_PLUS_METADATA.toArray()); + expectedBuffer.put(18, (byte) 0x1F); + expectedBuffer.put(72, (byte) 0x1F); + assertThat(buffer).isEqualTo(expectedBuffer); + assertThat(hagcData) + .isEqualTo(HAGC_METADATA_OBU.subArray(8, HAGC_METADATA_OBU.length() - 1).toArray()); + } + + @Test + public void extractHagcMetadataAndStripAllT35Metadata_multipleHagcObus_returnsFirstPayload() { + // Same as HAGC_METADATA_OBU, but with a different first payload byte. + ImmutableByteArray secondHagcMetadataObu = + ImmutableByteArray.concat( + HAGC_METADATA_OBU.subArray(0, 8), + ImmutableByteArray.ofUnsigned(0xe1), + HAGC_METADATA_OBU.subArray(9, HAGC_METADATA_OBU.length())); + ImmutableByteArray input = + ImmutableByteArray.concat( + TEMPORAL_DELIMITER_OBU, SEQUENCE_HEADER_OBU, HAGC_METADATA_OBU, secondHagcMetadataObu); + ByteBuffer buffer = ByteBuffer.wrap(input.toArray()); + + byte[] hagcData = Av1ObuUtil.extractHagcMetadataAndStripAllT35Metadata(buffer); + + ByteBuffer expectedBuffer = ByteBuffer.wrap(input.toArray()); + expectedBuffer.put(20, (byte) 0x1F); + expectedBuffer.put(74, (byte) 0x1F); + assertThat(buffer).isEqualTo(expectedBuffer); + assertThat(hagcData) + .isEqualTo(HAGC_METADATA_OBU.subArray(8, HAGC_METADATA_OBU.length() - 1).toArray()); + } + + @Test + public void + extractHagcMetadataAndStripAllT35Metadata_missingTrailingBits_leavesBufferUnchangedAndReturnsNull() { + // Keep the OBU size unchanged and replace the trailing 0x80 byte so that the OBU is still split + // correctly. + ImmutableByteArray input = + ImmutableByteArray.concat( + TEMPORAL_DELIMITER_OBU, + SEQUENCE_HEADER_OBU, + HAGC_METADATA_OBU.subArray(0, HAGC_METADATA_OBU.length() - 1), + ImmutableByteArray.ofUnsigned(0x01)); + ByteBuffer buffer = ByteBuffer.wrap(input.toArray()); + + byte[] hagcData = Av1ObuUtil.extractHagcMetadataAndStripAllT35Metadata(buffer); + + assertThat(hagcData).isNull(); + assertThat(buffer).isEqualTo(ByteBuffer.wrap(input.toArray())); + } + + @Test + public void + extractHagcMetadataAndStripAllT35Metadata_emptyPayload_leavesBufferUnchangedAndReturnsNull() { + // Metadata OBU with size 7 containing only the metadata type, the HAGC T.35 header and the + // trailing bits. + ImmutableByteArray input = + ImmutableByteArray.concat( + TEMPORAL_DELIMITER_OBU, + SEQUENCE_HEADER_OBU, + ImmutableByteArray.ofUnsigned(0x2a, 0x07, 0x04, 0xb5, 0x00, 0x90, 0x00, 0x01, 0x80)); + ByteBuffer buffer = ByteBuffer.wrap(input.toArray()); + + byte[] hagcData = Av1ObuUtil.extractHagcMetadataAndStripAllT35Metadata(buffer); + + assertThat(hagcData).isNull(); + assertThat(buffer).isEqualTo(ByteBuffer.wrap(input.toArray())); + } + + @Test + public void extractHagcMetadataAndStripAllT35Metadata_leavesHdr10PlusUnchangedAndReturnsNull() { + ByteBuffer buffer = + ByteBuffer.wrap(TEMPORAL_DELIMITER_SEQUENCE_HEADER_AND_HDR10_PLUS_METADATA.toArray()); + int originalPosition = buffer.position(); + int originalLimit = buffer.limit(); + + byte[] hagcData = Av1ObuUtil.extractHagcMetadataAndStripAllT35Metadata(buffer); + + assertThat(hagcData).isNull(); + assertThat(buffer.position()).isEqualTo(originalPosition); + assertThat(buffer.limit()).isEqualTo(originalLimit); + assertThat(buffer) + .isEqualTo( + ByteBuffer.wrap(TEMPORAL_DELIMITER_SEQUENCE_HEADER_AND_HDR10_PLUS_METADATA.toArray())); + } } diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/DecoderVideoRendererTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/DecoderVideoRendererTest.java index f9d2aebe1c3..641e569f6ad 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/DecoderVideoRendererTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/DecoderVideoRendererTest.java @@ -94,7 +94,13 @@ public void setUp() { eventListener, /* maxDroppedFramesToNotify= */ -1) { - private final Phaser inputBuffersInCodecPhaser = new Phaser(); + private final Phaser inputBuffersInCodecPhaser = + new Phaser() { + @Override + protected boolean onAdvance(int phase, int registeredParties) { + return false; + } + }; private @C.VideoOutputMode int outputMode; @Override @@ -137,6 +143,7 @@ protected void onQueueInputBuffer(DecoderInputBuffer buffer) { // to be cleared. // 4. The tests need to call ShadowLooper.idleMainThread() to execute the wait message // sent in step (3). + ((TestDecoderInputBuffer) buffer).isQueuedInCodec = true; int currentPhase = inputBuffersInCodecPhaser.register(); new Handler().post(() -> inputBuffersInCodecPhaser.awaitAdvance(currentPhase)); super.onQueueInputBuffer(buffer); @@ -153,13 +160,7 @@ protected void onQueueInputBuffer(DecoderInputBuffer buffer) { new DecoderInputBuffer[10], new VideoDecoderOutputBuffer[10]) { @Override protected DecoderInputBuffer createInputBuffer() { - return new DecoderInputBuffer(BUFFER_REPLACEMENT_MODE_DIRECT) { - @Override - public void clear() { - super.clear(); - inputBuffersInCodecPhaser.arriveAndDeregister(); - } - }; + return new TestDecoderInputBuffer(inputBuffersInCodecPhaser); } @Override @@ -659,4 +660,23 @@ public void render_withoutStrictDuration_rendersBuffersAfterDuration() throws Ex assertThat(renderer.decoderCounters.renderedOutputBufferCount).isEqualTo(3); assertThat(renderer.decoderCounters.skippedOutputBufferCount).isEqualTo(0); } + + private static final class TestDecoderInputBuffer extends DecoderInputBuffer { + private final Phaser phaser; + private boolean isQueuedInCodec; + + private TestDecoderInputBuffer(Phaser phaser) { + super(BUFFER_REPLACEMENT_MODE_DIRECT); + this.phaser = phaser; + } + + @Override + public void clear() { + super.clear(); + if (isQueuedInCodec) { + isQueuedInCodec = false; + phaser.arriveAndDeregister(); + } + } + } } diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/HevcSeiUtilTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/HevcSeiUtilTest.java new file mode 100644 index 00000000000..22a117ecf3c --- /dev/null +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/HevcSeiUtilTest.java @@ -0,0 +1,224 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.exoplayer.video; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.media3.test.utils.ImmutableByteArray; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.nio.ByteBuffer; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Tests for {@link HevcSeiUtil}. */ +@RunWith(AndroidJUnit4.class) +public final class HevcSeiUtilTest { + + private static final ImmutableByteArray SEI_NAL_UNIT_HDR10_PLUS = + ImmutableByteArray.ofUnsigned( + 0x00, 0x00, 0x00, 0x01, // NAL start code + 0x4E, 0x01, // NAL Header (type 39) + 0x04, // payloadType: 4 (T.35) + 0x13, // payloadSize: 19 + 0xB5, 0x00, 0x3C, 0x00, 0x01, 0x04, 0x00, // HDR10+ header + 0x00, 0x00, 0x03, 0x00, // Emulation prevention byte (0x03) + 0x00, 0x00, 0x03, 0x01, // Emulation prevention byte (0x03) + 0x01, 0x02, 0x03, 0x04, 0x05, 0x06); // Some payload bytes + + private static final ImmutableByteArray SEI_NAL_UNIT_MULTIPLE_MESSAGES = + ImmutableByteArray.ofUnsigned( + 0x00, + 0x00, + 0x00, + 0x01, // NAL start code + 0x4E, + 0x01, // NAL Header (type 39) + // First message: type 5 (User Data Unregistered), size 4 + 0x05, + 0x04, + 0xAA, + 0xBB, + 0xCC, + 0xDD, + // Second message: type 4 (T.35), size 5 + 0x04, + 0x05, + 0xB5, + 0x00, + 0x3C, + 0x00, + 0x01, + // Third message: type 4 (T.35), size 5 + 0x04, + 0x05, + 0xB5, + 0x00, + 0x3C, + 0x00, + 0x02, + 0x80); // rbsp_trailing_bits + + private static final ImmutableByteArray SEI_NAL_UNIT_EMULATION_IN_SIZE = + ImmutableByteArray.ofUnsigned( + 0x00, 0x00, 0x00, 0x01, // NAL start code + 0x4E, 0x01, // NAL Header (type 39) + 0x04, // payloadType: 4 (T.35) + 0x00, 0x00, 0x03, 0xFF, // payloadSize: 255 (with emulation prevention byte) + 0x80); // rbsp_trailing_bits + + @Test + public void stripAllT35Metadata_rewritesT35PayloadTypeToUnspecified() { + ByteBuffer buffer = ByteBuffer.wrap(SEI_NAL_UNIT_HDR10_PLUS.toArray()); + int originalPosition = buffer.position(); + int originalLimit = buffer.limit(); + + ByteBuffer expectedBuffer = ByteBuffer.wrap(SEI_NAL_UNIT_HDR10_PLUS.toArray()); + expectedBuffer.put( + 6, (byte) HevcSeiUtil.SEI_PAYLOAD_TYPE_UNSPECIFIED); // Change payloadType from 4 + + HevcSeiUtil.stripAllT35Metadata(buffer); + + assertThat(buffer.position()).isEqualTo(originalPosition); + assertThat(buffer.limit()).isEqualTo(originalLimit); + assertThat(buffer.array()).isEqualTo(expectedBuffer.array()); + } + + @Test + public void stripNonHdr10PlusT35Metadata_leavesHdr10PlusMetadataUnchanged() { + ByteBuffer buffer = ByteBuffer.wrap(SEI_NAL_UNIT_HDR10_PLUS.toArray()); + int originalPosition = buffer.position(); + int originalLimit = buffer.limit(); + + ByteBuffer expectedBuffer = ByteBuffer.wrap(SEI_NAL_UNIT_HDR10_PLUS.toArray()); + + HevcSeiUtil.stripNonHdr10PlusT35Metadata(buffer); + + assertThat(buffer.position()).isEqualTo(originalPosition); + assertThat(buffer.limit()).isEqualTo(originalLimit); + assertThat(buffer.array()).isEqualTo(expectedBuffer.array()); + } + + @Test + public void stripNonHdr10PlusT35Metadata_multipleMessages_rewritesOnlyNonHdr10PlusT35() { + ImmutableByteArray seiNalUnitMultipleMessagesWithHdr10 = + ImmutableByteArray.ofUnsigned( + 0x00, + 0x00, + 0x00, + 0x01, // NAL start code + 0x4E, + 0x01, // NAL Header (type 39) + // First message: type 5 (User Data Unregistered), size 4 + 0x05, + 0x04, + 0xAA, + 0xBB, + 0xCC, + 0xDD, + // Second message: type 4 (T.35), size 6, HDR10+ + 0x04, + 0x06, + 0xB5, + 0x00, + 0x3C, + 0x00, + 0x01, + 0x04, + // Third message: type 4 (T.35), size 6, non-HDR10+ + 0x04, + 0x06, + 0xB5, + 0x00, + 0x90, + 0x00, + 0x01, + 0x00, + 0x80); // rbsp_trailing_bits + + ByteBuffer buffer = ByteBuffer.wrap(seiNalUnitMultipleMessagesWithHdr10.toArray()); + int originalPosition = buffer.position(); + int originalLimit = buffer.limit(); + + ByteBuffer expectedBuffer = ByteBuffer.wrap(seiNalUnitMultipleMessagesWithHdr10.toArray()); + // Offset 6 is first message type (5), stays 5 + // Offset 12 is second message type (4, HDR10+), stays 4 + // Offset 20 is third message type (4, non-HDR10+) + expectedBuffer.put(20, (byte) HevcSeiUtil.SEI_PAYLOAD_TYPE_UNSPECIFIED); + + HevcSeiUtil.stripNonHdr10PlusT35Metadata(buffer); + + assertThat(buffer.position()).isEqualTo(originalPosition); + assertThat(buffer.limit()).isEqualTo(originalLimit); + assertThat(buffer.array()).isEqualTo(expectedBuffer.array()); + } + + @Test + public void stripNonHdr10PlusT35Metadata_t35MessageTooShortForHdr10Plus_rewritesT35PayloadType() { + // T.35 payload size is 4, which is less than HevcSeiUtil.HDR10_PLUS_PREFIX.length (7). + ImmutableByteArray seiNalUnitShortT35 = + ImmutableByteArray.ofUnsigned( + 0x00, 0x00, 0x00, 0x01, // NAL start code + 0x4E, 0x01, // NAL Header (type 39) + 0x04, // payloadType: 4 (T.35) + 0x04, // payloadSize: 4 + 0xB5, 0x00, 0x3C, 0x00, // HDR10+ prefix bytes, but cut short + 0x80); // rbsp_trailing_bits + + ByteBuffer buffer = ByteBuffer.wrap(seiNalUnitShortT35.toArray()); + int originalPosition = buffer.position(); + int originalLimit = buffer.limit(); + + ByteBuffer expectedBuffer = ByteBuffer.wrap(seiNalUnitShortT35.toArray()); + // Offset 6 is the payload type (4), which should be rewritten because the payload is too short + // to be a valid HDR10+ message. + expectedBuffer.put(6, (byte) HevcSeiUtil.SEI_PAYLOAD_TYPE_UNSPECIFIED); + + HevcSeiUtil.stripNonHdr10PlusT35Metadata(buffer); + + assertThat(buffer.position()).isEqualTo(originalPosition); + assertThat(buffer.limit()).isEqualTo(originalLimit); + assertThat(buffer.array()).isEqualTo(expectedBuffer.array()); + } + + @Test + public void stripAllT35Metadata_multipleMessages_rewritesOnlyT35() { + ByteBuffer buffer = ByteBuffer.wrap(SEI_NAL_UNIT_MULTIPLE_MESSAGES.toArray()); + + ByteBuffer expectedBuffer = ByteBuffer.wrap(SEI_NAL_UNIT_MULTIPLE_MESSAGES.toArray()); + // Offset 6 is first message type (5), stays 5 + // Offset 12 is second message type (4) + expectedBuffer.put(12, (byte) HevcSeiUtil.SEI_PAYLOAD_TYPE_UNSPECIFIED); + // Offset 19 is third message type (4) + expectedBuffer.put(19, (byte) HevcSeiUtil.SEI_PAYLOAD_TYPE_UNSPECIFIED); + + HevcSeiUtil.stripAllT35Metadata(buffer); + + assertThat(buffer.array()).isEqualTo(expectedBuffer.array()); + } + + @Test + public void stripAllT35Metadata_handlesEmulationPreventionInSize() { + ByteBuffer buffer = ByteBuffer.wrap(SEI_NAL_UNIT_EMULATION_IN_SIZE.toArray()); + + ByteBuffer expectedBuffer = ByteBuffer.wrap(SEI_NAL_UNIT_EMULATION_IN_SIZE.toArray()); + // Offset 6 is payload type (4) + expectedBuffer.put(6, (byte) HevcSeiUtil.SEI_PAYLOAD_TYPE_UNSPECIFIED); + + HevcSeiUtil.stripAllT35Metadata(buffer); + + assertThat(buffer.array()).isEqualTo(expectedBuffer.array()); + } +} diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/MediaCodecVideoRendererTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/MediaCodecVideoRendererTest.java index 41079cc7a23..02130e0d63f 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/MediaCodecVideoRendererTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/MediaCodecVideoRendererTest.java @@ -23,6 +23,7 @@ import static android.media.MediaCodecInfo.CodecProfileLevel.AVCProfileHigh; import static android.media.MediaCodecInfo.CodecProfileLevel.DolbyVisionLevelFhd30; import static android.media.MediaCodecInfo.CodecProfileLevel.DolbyVisionProfileDvheDtr; +import static android.media.MediaCodecInfo.CodecProfileLevel.DolbyVisionProfileDvheSt; import static android.media.MediaCodecInfo.CodecProfileLevel.HEVCHighTierLevel51; import static android.media.MediaCodecInfo.CodecProfileLevel.HEVCMainTierLevel41; import static android.media.MediaCodecInfo.CodecProfileLevel.HEVCProfileMain; @@ -38,6 +39,7 @@ import static androidx.media3.test.utils.FakeSampleStream.FakeSampleStreamItem.sample; import static androidx.media3.test.utils.FakeTimeline.TimelineWindowDefinition.DEFAULT_WINDOW_OFFSET_IN_FIRST_PERIOD_US; import static androidx.media3.test.utils.TestUtil.createByteArray; +import static com.google.common.base.Preconditions.checkState; import static com.google.common.truth.Truth.assertThat; import static org.junit.Assert.assertThrows; import static org.junit.Assume.assumeTrue; @@ -53,6 +55,7 @@ import static org.robolectric.Shadows.shadowOf; import android.content.Context; +import android.graphics.Bitmap; import android.graphics.SurfaceTexture; import android.hardware.display.DisplayManager; import android.media.MediaCodec; @@ -69,11 +72,16 @@ import androidx.media3.common.C; import androidx.media3.common.ColorInfo; import androidx.media3.common.DrmInitData; +import androidx.media3.common.Effect; +import androidx.media3.common.Flags; import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; import androidx.media3.common.TrackGroup; import androidx.media3.common.VideoSize; import androidx.media3.common.util.Clock; +import androidx.media3.common.util.Size; +import androidx.media3.common.util.ThrowingRunnable; +import androidx.media3.common.util.TimestampIterator; import androidx.media3.decoder.DecoderInputBuffer; import androidx.media3.exoplayer.CodecParameters; import androidx.media3.exoplayer.DecoderCounters; @@ -109,6 +117,7 @@ import androidx.media3.test.utils.FakeMediaPeriod; import androidx.media3.test.utils.FakeSampleStream; import androidx.media3.test.utils.FakeTimeline; +import androidx.media3.test.utils.Media3FlagsRule; import androidx.media3.test.utils.robolectric.IdlingMediaCodecAdapterFactory; import androidx.media3.test.utils.robolectric.RobolectricUtil; import androidx.test.core.app.ApplicationProvider; @@ -124,7 +133,9 @@ import java.util.Collections; import java.util.List; import java.util.PriorityQueue; +import java.util.concurrent.Executor; import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicReference; import java.util.stream.Collectors; import org.junit.After; import org.junit.Before; @@ -137,6 +148,7 @@ import org.mockito.junit.MockitoRule; import org.robolectric.Shadows; import org.robolectric.annotation.Config; +import org.robolectric.shadows.MediaCodecInfoBuilder; import org.robolectric.shadows.ShadowDisplay; import org.robolectric.shadows.ShadowLooper; import org.robolectric.shadows.ShadowMediaCodec; @@ -146,6 +158,7 @@ @RunWith(AndroidJUnit4.class) public class MediaCodecVideoRendererTest { @Rule public final MockitoRule mockito = MockitoJUnit.rule(); + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); private static final Format VIDEO_H264 = new Format.Builder() @@ -228,6 +241,7 @@ public void setUp() throws Exception { /* vendor= */ false, /* forceDisableAdaptive= */ false, /* forceSecure= */ requiresSecureDecoder)); + Flags.enableFlag(Flags.FLAG_VIDEO_RENDERER_DURATION_TO_PROGRESS); mediaCodecVideoRenderer = new MediaCodecVideoRenderer( new MediaCodecVideoRenderer.Builder(ApplicationProvider.getApplicationContext()) @@ -237,8 +251,7 @@ public void setUp() throws Exception { .setEnableDecoderFallback(false) .setEventHandler(new Handler(testMainLooper)) .setEventListener(eventListener) - .setMaxDroppedFramesToNotify(1) - .setEnableDurationToProgressUs(true)) { + .setMaxDroppedFramesToNotify(1)) { @Override protected @Capabilities int supportsFormat( MediaCodecSelector mediaCodecSelector, Format format) { @@ -1897,8 +1910,7 @@ public void render_withLateAV1BufferWithoutDependencies_dropsInputBuffers() thro .setEventHandler(new Handler(testMainLooper)) .setEventListener(eventListener) .setMaxDroppedFramesToNotify(1) - .experimentalSetLateThresholdToDropDecoderInputUs(50_000) - .experimentalSetParseAv1SampleDependencies(true)) { + .experimentalSetLateThresholdToDropDecoderInputUs(50_000)) { @Override protected @Capabilities int supportsFormat( MediaCodecSelector mediaCodecSelector, Format format) { @@ -2635,6 +2647,7 @@ public void render_afterSeek_samplesWithoutDependencies_doesNotDropFrames() thro @Config(minSdk = 34) public void render_withDisabledMediaCodecBufferDecodeOnlyFlag_rendererSkipsEarlyBuffers() throws Exception { + Flags.disableFlag(Flags.FLAG_ENABLE_MEDIACODEC_BUFFER_DECODE_ONLY); MediaCodecVideoRenderer mediaCodecVideoRenderer = new MediaCodecVideoRenderer.Builder(ApplicationProvider.getApplicationContext()) .setCodecAdapterFactory(codecAdapterFactory) @@ -2644,7 +2657,6 @@ public void render_withDisabledMediaCodecBufferDecodeOnlyFlag_rendererSkipsEarly .setEventHandler(new Handler(testMainLooper)) .setEventListener(eventListener) .setMaxDroppedFramesToNotify(1) - .experimentalSetEnableMediaCodecBufferDecodeOnlyFlag(false) .build(); mediaCodecVideoRenderer.init(/* index= */ 0, PlayerId.UNSET, Clock.DEFAULT); mediaCodecVideoRenderer.handleMessage(Renderer.MSG_SET_VIDEO_OUTPUT, surface); @@ -2693,6 +2705,7 @@ public void render_withDisabledMediaCodecBufferDecodeOnlyFlag_rendererSkipsEarly @Config(minSdk = 34) public void render_withEnabledSetMediaCodecBufferDecodeOnlyFlag_decoderSkipsEarlyBuffers() throws Exception { + Flags.enableFlag(Flags.FLAG_ENABLE_MEDIACODEC_BUFFER_DECODE_ONLY); MediaCodecVideoRenderer mediaCodecVideoRenderer = new MediaCodecVideoRenderer.Builder(ApplicationProvider.getApplicationContext()) .setCodecAdapterFactory(codecAdapterFactory) @@ -2702,7 +2715,6 @@ public void render_withEnabledSetMediaCodecBufferDecodeOnlyFlag_decoderSkipsEarl .setEventHandler(new Handler(testMainLooper)) .setEventListener(eventListener) .setMaxDroppedFramesToNotify(1) - .experimentalSetEnableMediaCodecBufferDecodeOnlyFlag(true) .build(); mediaCodecVideoRenderer.init(/* index= */ 0, PlayerId.UNSET, Clock.DEFAULT); mediaCodecVideoRenderer.handleMessage(Renderer.MSG_SET_VIDEO_OUTPUT, surface); @@ -2765,6 +2777,7 @@ public void render_withEnabledSetMediaCodecBufferDecodeOnlyFlag_decoderSkipsEarl @Config(minSdk = 34) public void render_withEnabledSetMediaCodecBufferDecodeOnlyFlag_flagNotSetOnLastBuffer() throws Exception { + Flags.enableFlag(Flags.FLAG_ENABLE_MEDIACODEC_BUFFER_DECODE_ONLY); MediaCodecVideoRenderer mediaCodecVideoRenderer = new MediaCodecVideoRenderer.Builder(ApplicationProvider.getApplicationContext()) .setCodecAdapterFactory(codecAdapterFactory) @@ -2774,7 +2787,6 @@ public void render_withEnabledSetMediaCodecBufferDecodeOnlyFlag_flagNotSetOnLast .setEventHandler(new Handler(testMainLooper)) .setEventListener(eventListener) .setMaxDroppedFramesToNotify(1) - .experimentalSetEnableMediaCodecBufferDecodeOnlyFlag(true) .build(); mediaCodecVideoRenderer.init(/* index= */ 0, PlayerId.UNSET, Clock.DEFAULT); mediaCodecVideoRenderer.handleMessage(Renderer.MSG_SET_VIDEO_OUTPUT, surface); @@ -3991,6 +4003,174 @@ public void render_afterSeekToEoSWithFlushingAndDecodeOnlyFlagDisabled_rendersFr assertThat(argumentDecoderCounters.getValue().renderedOutputBufferCount).isEqualTo(6); } + @Test + @Config(minSdk = 34) + public void render_afterSeekWithFlushingDisabled_dropsStalePreSeekFrames() throws Exception { + AtomicReference adapter = new AtomicReference<>(); + MediaCodecAdapter.Factory customCodecAdapterFactory = + configuration -> { + MediaCodecAdapter mediaCodecAdapter = codecAdapterFactory.createAdapter(configuration); + StaleFrameTestingMediaCodecAdapter staleFrameTestingMediaCodecAdapter = + new StaleFrameTestingMediaCodecAdapter(mediaCodecAdapter); + adapter.set(staleFrameTestingMediaCodecAdapter); + return staleFrameTestingMediaCodecAdapter; + }; + mediaCodecVideoRenderer = + new MediaCodecVideoRenderer( + new MediaCodecVideoRenderer.Builder(ApplicationProvider.getApplicationContext()) + .setCodecAdapterFactory(customCodecAdapterFactory) + .setMediaCodecSelector(mediaCodecSelector) + .setAllowedJoiningTimeMs(0) + .setEnableDecoderFallback(false) + .setEventHandler(new Handler(testMainLooper)) + .setEventListener(eventListener) + .setMaxDroppedFramesToNotify(1)) { + @Override + protected @Capabilities int supportsFormat( + MediaCodecSelector mediaCodecSelector, Format format) { + return RendererCapabilities.create(C.FORMAT_HANDLED); + } + }; + FakeTimeline fakeTimeline = + new FakeTimeline( + new FakeTimeline.TimelineWindowDefinition.Builder().setDurationUs(1_000_000).build()); + mediaCodecVideoRenderer.init(/* index= */ 0, PlayerId.UNSET, Clock.DEFAULT); + mediaCodecVideoRenderer.handleMessage(Renderer.MSG_SET_VIDEO_OUTPUT, surface); + mediaCodecVideoRenderer.setTimeline(fakeTimeline); + FakeSampleStream fakeSampleStream1 = + createFakeSampleStream( + VIDEO_H264, + ImmutableList.of( + oneByteSample(/* timeUs= */ 10_000, C.BUFFER_FLAG_KEY_FRAME), + oneByteSample(/* timeUs= */ 20_000), + oneByteSample(/* timeUs= */ 30_000), // largestQueued = 30,000 + oneByteSample(/* timeUs= */ 40_000), + oneByteSample(/* timeUs= */ 50_000))); + mediaCodecVideoRenderer.enable( + RendererConfiguration.DEFAULT, + new Format[] {VIDEO_H264}, + fakeSampleStream1, + /* positionUs= */ 0, + /* joining= */ false, + /* mayRenderStartOfStream= */ true, + /* startPositionUs= */ 0, + /* offsetUs= */ 0, + new MediaSource.MediaPeriodId(fakeTimeline.getUidOfPeriod(0))); + shadowOf(testMainLooper).idle(); + ArgumentCaptor argumentDecoderCounters = + ArgumentCaptor.forClass(DecoderCounters.class); + verify(eventListener).onVideoEnabled(argumentDecoderCounters.capture()); + DecoderCounters decoderCounters = argumentDecoderCounters.getValue(); + mediaCodecVideoRenderer.start(); + + // Render once to populate a few input buffers. + mediaCodecVideoRenderer.render(0, SystemClock.elapsedRealtime() * 1000); + // Output the format change, followed by the first frame. + adapter.get().allowFormatChange = true; + mediaCodecVideoRenderer.render(0, SystemClock.elapsedRealtime() * 1000); + adapter.get().allowFirstFrameOutput = true; + mediaCodecVideoRenderer.render(0, SystemClock.elapsedRealtime() * 1000); + assertThat(decoderCounters.renderedOutputBufferCount).isEqualTo(1); + + ScrubbingModeParameters scrubbingModeParameters = + new ScrubbingModeParameters.Builder().setAllowSkippingMediaCodecFlush(true).build(); + mediaCodecVideoRenderer.handleMessage(Renderer.MSG_SET_SCRUBBING_MODE, scrubbingModeParameters); + seekToUs( + mediaCodecVideoRenderer, scrubbingModeParameters, fakeSampleStream1, /* positionUs= */ 0); + + FakeSampleStream fakeSampleStream2 = + createFakeSampleStream( + VIDEO_H264, + ImmutableList.of( + oneByteSample(/* timeUs= */ 12_800, C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM)); + mediaCodecVideoRenderer.replaceStream( + new Format[] {VIDEO_H264}, + fakeSampleStream2, + /* startPositionUs= */ 0, + /* offsetUs= */ 0, + new MediaSource.MediaPeriodId(fakeTimeline.getUidOfPeriod(0))); + adapter.get().availableInputBufferCount = 1; + // Render to queue the new 12,800 frame and calculate the offset. + mediaCodecVideoRenderer.render(0, SystemClock.elapsedRealtime() * 1000); + // Force the adapter to output the STALE frame. + adapter.get().allowStaleFrameOutput = true; + mediaCodecVideoRenderer.render(0, SystemClock.elapsedRealtime() * 1000); + + assertThat(decoderCounters.renderedOutputBufferCount).isEqualTo(1); + assertThat(decoderCounters.skippedOutputBufferCount).isEqualTo(1); + } + + @Test + public void + render_withVideoSinkAndTransientInvalidOutputFormat_waitsForValidFormatBeforeHandlingFrame() + throws Exception { + AtomicReference adapter = new AtomicReference<>(); + MediaCodecAdapter.Factory customCodecAdapterFactory = + configuration -> { + MediaCodecAdapter mediaCodecAdapter = codecAdapterFactory.createAdapter(configuration); + StaleFrameTestingMediaCodecAdapter customAdapter = + new StaleFrameTestingMediaCodecAdapter(mediaCodecAdapter); + customAdapter.mediaFormat.setInteger(MediaFormat.KEY_WIDTH, 0); + customAdapter.mediaFormat.setInteger(MediaFormat.KEY_HEIGHT, 0); + adapter.set(customAdapter); + return customAdapter; + }; + FakeVideoSink fakeVideoSink = new FakeVideoSink(surface); + mediaCodecVideoRenderer = + new MediaCodecVideoRenderer( + new MediaCodecVideoRenderer.Builder(ApplicationProvider.getApplicationContext()) + .setCodecAdapterFactory(customCodecAdapterFactory) + .setMediaCodecSelector(mediaCodecSelector) + .setAllowedJoiningTimeMs(0) + .setEnableDecoderFallback(false) + .setEventHandler(new Handler(testMainLooper)) + .setEventListener(eventListener) + .setVideoSink(fakeVideoSink)) { + @Override + protected @Capabilities int supportsFormat( + MediaCodecSelector mediaCodecSelector, Format format) { + return RendererCapabilities.create(C.FORMAT_HANDLED); + } + }; + FakeTimeline fakeTimeline = + new FakeTimeline( + new FakeTimeline.TimelineWindowDefinition.Builder().setDurationUs(1_000_000).build()); + mediaCodecVideoRenderer.init(/* index= */ 0, PlayerId.UNSET, Clock.DEFAULT); + mediaCodecVideoRenderer.handleMessage(Renderer.MSG_SET_VIDEO_OUTPUT, surface); + mediaCodecVideoRenderer.setTimeline(fakeTimeline); + FakeSampleStream fakeSampleStream = + createFakeSampleStream( + VIDEO_H264, + ImmutableList.of( + oneByteSample(/* timeUs= */ 10_000, C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM)); + mediaCodecVideoRenderer.enable( + RendererConfiguration.DEFAULT, + new Format[] {VIDEO_H264}, + fakeSampleStream, + /* positionUs= */ 0, + /* joining= */ false, + /* mayRenderStartOfStream= */ true, + /* startPositionUs= */ 0, + /* offsetUs= */ 0, + new MediaSource.MediaPeriodId(fakeTimeline.getUidOfPeriod(0))); + mediaCodecVideoRenderer.start(); + + mediaCodecVideoRenderer.render(/* positionUs= */ 0, SystemClock.elapsedRealtime() * 1000); + adapter.get().allowFormatChange = true; + adapter.get().allowFirstFrameOutput = true; + mediaCodecVideoRenderer.render(/* positionUs= */ 0, SystemClock.elapsedRealtime() * 1000); + boolean handledFrameWhileFormatInvalid = !fakeVideoSink.handledFrameTimestampsUs.isEmpty(); + adapter.get().mediaFormat.setInteger(MediaFormat.KEY_WIDTH, 1920); + adapter.get().mediaFormat.setInteger(MediaFormat.KEY_HEIGHT, 1080); + mediaCodecVideoRenderer.render(/* positionUs= */ 0, SystemClock.elapsedRealtime() * 1000); + + assertThat(handledFrameWhileFormatInvalid).isFalse(); + assertThat(fakeVideoSink.inputStreamFormats).hasSize(1); + assertThat(fakeVideoSink.inputStreamFormats.get(0).width).isEqualTo(1920); + assertThat(fakeVideoSink.inputStreamFormats.get(0).height).isEqualTo(1080); + assertThat(fakeVideoSink.handledFrameTimestampsUs).containsExactly(10_000L); + } + @Test public void render_afterSeekWithSkipKeyFrameResetAndDecodeOnlyFlagDisabled_rendersFramesAsExpected() @@ -4156,6 +4336,7 @@ public void render_afterSeekToEoSWithFlushingAndDecodeOnlyFlagDisabled_rendersFr @Config(minSdk = 34) public void render_afterSeekWithFlushingDisabledAndDecodeOnlyFlags_playsFrameAsExpected() throws Exception { + Flags.enableFlag(Flags.FLAG_ENABLE_MEDIACODEC_BUFFER_DECODE_ONLY); MediaCodecVideoRenderer mediaCodecVideoRenderer = new MediaCodecVideoRenderer.Builder(ApplicationProvider.getApplicationContext()) .setCodecAdapterFactory(codecAdapterFactory) @@ -4165,7 +4346,6 @@ public void render_afterSeekWithFlushingDisabledAndDecodeOnlyFlags_playsFrameAsE .setEventHandler(new Handler(testMainLooper)) .setEventListener(eventListener) .setMaxDroppedFramesToNotify(1) - .experimentalSetEnableMediaCodecBufferDecodeOnlyFlag(true) .build(); mediaCodecVideoRenderer.init(/* index= */ 0, PlayerId.UNSET, Clock.DEFAULT); mediaCodecVideoRenderer.handleMessage(Renderer.MSG_SET_VIDEO_OUTPUT, surface); @@ -4247,6 +4427,7 @@ public void render_afterSeekWithFlushingDisabledAndDecodeOnlyFlags_playsFrameAsE @Config(minSdk = 34) public void render_afterBackwardsSeekWithFlushingDisabledAndDecodeOnlyFlags_playsFrameAsExpected() throws Exception { + Flags.enableFlag(Flags.FLAG_ENABLE_MEDIACODEC_BUFFER_DECODE_ONLY); MediaCodecVideoRenderer mediaCodecVideoRenderer = new MediaCodecVideoRenderer.Builder(ApplicationProvider.getApplicationContext()) .setCodecAdapterFactory(codecAdapterFactory) @@ -4256,7 +4437,6 @@ public void render_afterBackwardsSeekWithFlushingDisabledAndDecodeOnlyFlags_play .setEventHandler(new Handler(testMainLooper)) .setEventListener(eventListener) .setMaxDroppedFramesToNotify(1) - .experimentalSetEnableMediaCodecBufferDecodeOnlyFlag(true) .build(); mediaCodecVideoRenderer.init(/* index= */ 0, PlayerId.UNSET, Clock.DEFAULT); mediaCodecVideoRenderer.handleMessage(Renderer.MSG_SET_VIDEO_OUTPUT, surface); @@ -5367,6 +5547,78 @@ public void supportsFormat_withDolbyVision_setsDecoderSupportFlagsByDisplayDolby .isEqualTo(RendererCapabilities.DECODER_SUPPORT_PRIMARY); } + @Test + @Config(minSdk = 26) + public void + supportsFormat_withMultipleDolbyVisionDecoders_returnsPrimaryDecoderForMatchingProfile() + throws Exception { + Format formatDvProfile8 = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION) + .setCodecs("dvhe.08.01") + .build(); + // Provide two Dolby Vision decoders: the first supports only Profile 4 (DvheDtr), + // the second supports Profile 8 (DvheSt). + MediaCodecSelector mediaCodecSelector = + (mimeType, requiresSecureDecoder, requiresTunnelingDecoder) -> { + if (mimeType.equals(MimeTypes.VIDEO_DOLBY_VISION)) { + CodecCapabilities capabilitiesDolbyProfile4 = + createCodecCapabilities(DolbyVisionProfileDvheDtr, DolbyVisionLevelFhd30); + CodecCapabilities capabilitiesDolbyProfile8 = + createCodecCapabilities(DolbyVisionProfileDvheSt, DolbyVisionLevelFhd30); + return ImmutableList.of( + MediaCodecInfo.newInstance( + /* name= */ "dv-p4-codec", + /* mimeType= */ mimeType, + /* codecMimeType= */ mimeType, + /* capabilities= */ capabilitiesDolbyProfile4, + /* hardwareAccelerated= */ true, + /* softwareOnly= */ false, + /* vendor= */ false, + /* forceDisableAdaptive= */ false, + /* forceSecure= */ false), + MediaCodecInfo.newInstance( + /* name= */ "dv-p8-codec", + /* mimeType= */ mimeType, + /* codecMimeType= */ mimeType, + /* capabilities= */ capabilitiesDolbyProfile8, + /* hardwareAccelerated= */ true, + /* softwareOnly= */ false, + /* vendor= */ false, + /* forceDisableAdaptive= */ false, + /* forceSecure= */ false)); + } + return ImmutableList.of(); + }; + // Set Display to have Dolby Vision support. + Context context = ApplicationProvider.getApplicationContext(); + DisplayManager displayManager = + (DisplayManager) context.getSystemService(Context.DISPLAY_SERVICE); + Display display = (displayManager != null) ? displayManager.getDisplay(DEFAULT_DISPLAY) : null; + ShadowDisplay shadowDisplay = Shadows.shadowOf(display); + int[] hdrCapabilities = + new int[] { + Display.HdrCapabilities.HDR_TYPE_HDR10, Display.HdrCapabilities.HDR_TYPE_DOLBY_VISION + }; + shadowDisplay.setDisplayHdrCapabilities( + display.getDisplayId(), + /* maxLuminance= */ 100f, + /* maxAverageLuminance= */ 100f, + /* minLuminance= */ 100f, + hdrCapabilities); + MediaCodecVideoRenderer renderer = + new MediaCodecVideoRenderer.Builder(context) + .setMediaCodecSelector(mediaCodecSelector) + .build(); + renderer.init(/* index= */ 0, PlayerId.UNSET, Clock.DEFAULT); + + @Capabilities int capabilities = renderer.supportsFormat(formatDvProfile8); + + assertThat(RendererCapabilities.getFormatSupport(capabilities)).isEqualTo(C.FORMAT_HANDLED); + assertThat(RendererCapabilities.getDecoderSupport(capabilities)) + .isEqualTo(RendererCapabilities.DECODER_SUPPORT_PRIMARY); + } + @Test public void getDecoderInfo_withNonPerformantHardwareDecoder_returnsHardwareDecoderFirst() throws Exception { @@ -5890,6 +6142,127 @@ public void getCodecMaxInputSize_withUnsupportedFormat_returnsNoValue() { .isEqualTo(Format.NO_VALUE); } + @Test + @Config(minSdk = 29) + public void + getCodecMaxValues_streamFormatWithIncompleteMatchingHdrColorInfo_includesStreamFormatMaxDimensions() { + MediaFormat adaptiveFormat = + MediaFormat.createVideoFormat(MimeTypes.VIDEO_H265, /* width= */ 3840, /* height= */ 2160); + adaptiveFormat.setFeatureEnabled(CodecCapabilities.FEATURE_AdaptivePlayback, true); + CodecCapabilities capabilities = + MediaCodecInfoBuilder.CodecCapabilitiesBuilder.newBuilder() + .setMediaFormat(adaptiveFormat) + .setColorFormats(new int[0]) + .setProfileLevels(new CodecProfileLevel[0]) + .build(); + MediaCodecInfo adaptiveCodecInfo = + MediaCodecInfo.newInstance( + /* name= */ MimeTypes.VIDEO_H265, + /* mimeType= */ MimeTypes.VIDEO_H265, + /* codecMimeType= */ MimeTypes.VIDEO_H265, + capabilities, + /* hardwareAccelerated= */ true, + /* softwareOnly= */ false, + /* vendor= */ true, + /* forceDisableAdaptive= */ false, + /* forceSecure= */ false); + Format extracted1080pFormat = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H265) + .setWidth(1920) + .setHeight(1080) + .setColorInfo( + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorTransfer(C.COLOR_TRANSFER_ST2084) + .setColorRange(C.COLOR_RANGE_LIMITED) + .setHdrStaticInfo(new byte[] {1, 2, 3}) + .setLumaBitdepth(10) + .setChromaBitdepth(10) + .build()) + .build(); + Format manifest4kStreamFormat = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H265) + .setWidth(3840) + .setHeight(2160) + .setColorInfo( + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorTransfer(C.COLOR_TRANSFER_ST2084) + .build()) + .build(); + + MediaCodecVideoRenderer.CodecMaxValues codecMaxValues = + mediaCodecVideoRenderer.getCodecMaxValues( + adaptiveCodecInfo, + extracted1080pFormat, + new Format[] {extracted1080pFormat, manifest4kStreamFormat}); + + assertThat(codecMaxValues.width).isEqualTo(3840); + assertThat(codecMaxValues.height).isEqualTo(2160); + } + + @Test + @Config(minSdk = 29) + public void + getCodecMaxValues_streamFormatWithMismatchedHdrColorTransfer_excludesStreamFormatMaxDimensions() { + MediaFormat adaptiveFormat = + MediaFormat.createVideoFormat(MimeTypes.VIDEO_H265, /* width= */ 3840, /* height= */ 2160); + adaptiveFormat.setFeatureEnabled(CodecCapabilities.FEATURE_AdaptivePlayback, true); + CodecCapabilities capabilities = + MediaCodecInfoBuilder.CodecCapabilitiesBuilder.newBuilder() + .setMediaFormat(adaptiveFormat) + .setColorFormats(new int[0]) + .setProfileLevels(new CodecProfileLevel[0]) + .build(); + MediaCodecInfo adaptiveCodecInfo = + MediaCodecInfo.newInstance( + /* name= */ MimeTypes.VIDEO_H265, + /* mimeType= */ MimeTypes.VIDEO_H265, + /* codecMimeType= */ MimeTypes.VIDEO_H265, + capabilities, + /* hardwareAccelerated= */ true, + /* softwareOnly= */ false, + /* vendor= */ true, + /* forceDisableAdaptive= */ false, + /* forceSecure= */ false); + Format extracted1080pPqFormat = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H265) + .setWidth(1920) + .setHeight(1080) + .setColorInfo( + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorTransfer(C.COLOR_TRANSFER_ST2084) + .setColorRange(C.COLOR_RANGE_LIMITED) + .setLumaBitdepth(10) + .setChromaBitdepth(10) + .build()) + .build(); + Format manifest4kHlgStreamFormat = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H265) + .setWidth(3840) + .setHeight(2160) + .setColorInfo( + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorTransfer(C.COLOR_TRANSFER_HLG) + .build()) + .build(); + + MediaCodecVideoRenderer.CodecMaxValues codecMaxValues = + mediaCodecVideoRenderer.getCodecMaxValues( + adaptiveCodecInfo, + extracted1080pPqFormat, + new Format[] {extracted1080pPqFormat, manifest4kHlgStreamFormat}); + + assertThat(codecMaxValues.width).isEqualTo(1920); + assertThat(codecMaxValues.height).isEqualTo(1080); + } + @Test public void onCodecParametersChanged_dispatchesToVideoRendererEventListener() { CodecParameters codecParameters = @@ -5947,6 +6320,7 @@ public void getDurationToProgressUs_withWaitingOutputBuffer_returnsCalculatedDur public void getDurationToProgressUs_withDurationToProgressUsDisabledAndWaitingOutputBuffer_returnsDefaultDuration() throws Exception { + Flags.disableFlag(Flags.FLAG_VIDEO_RENDERER_DURATION_TO_PROGRESS); mediaCodecVideoRenderer = new MediaCodecVideoRenderer( new MediaCodecVideoRenderer.Builder(ApplicationProvider.getApplicationContext()) @@ -5956,8 +6330,7 @@ public void getDurationToProgressUs_withWaitingOutputBuffer_returnsCalculatedDur .setEnableDecoderFallback(false) .setEventHandler(new Handler(testMainLooper)) .setEventListener(eventListener) - .setMaxDroppedFramesToNotify(1) - .setEnableDurationToProgressUs(false)) { + .setMaxDroppedFramesToNotify(1)) { @Override protected @Capabilities int supportsFormat( MediaCodecSelector mediaCodecSelector, Format format) { @@ -6070,8 +6443,7 @@ protected void onOutputFormatChanged(Format format, @Nullable MediaFormat mediaF .setEnableDecoderFallback(false) .setEventHandler(new Handler(testMainLooper)) .setEventListener(eventListener) - .setMaxDroppedFramesToNotify(1) - .setEnableDurationToProgressUs(true)) { + .setMaxDroppedFramesToNotify(1)) { @Override protected @Capabilities int supportsFormat( MediaCodecSelector mediaCodecSelector, Format format) { @@ -6289,8 +6661,7 @@ public void getDurationToProgressUs_afterRenderToEndOfStream_returnsCalculatedDu .setEnableDecoderFallback(false) .setEventHandler(new Handler(testMainLooper)) .setEventListener(eventListener) - .setMaxDroppedFramesToNotify(1) - .setEnableDurationToProgressUs(true)) { + .setMaxDroppedFramesToNotify(1)) { @Override protected @Capabilities int supportsFormat( MediaCodecSelector mediaCodecSelector, Format format) { @@ -6557,13 +6928,15 @@ public void getDurationToProgressUs_withDropToKeyFrame_returnsDefaultDuration() codecAdapterFactory.idleQueueingAndCallbackThreads(); } - // Invocation should drop to keyframe as current output buffer is very, very late. - long durationToProgressIncurringDropUs = + // Invocation should return 0 as current output buffer is very, very late and needs dropping. + long durationToProgressBeforeDropUs = mediaCodecVideoRenderer.getDurationToProgressUs( /* positionUs= */ 600_001, SystemClock.elapsedRealtime() * 1000, /* isOnBufferAvailableListenerRegistered= */ true); + mediaCodecVideoRenderer.render(600_001, SystemClock.elapsedRealtime() * 1000); + long durationToProgressPostDropUs = mediaCodecVideoRenderer.getDurationToProgressUs( /* positionUs= */ 600_001, @@ -6576,10 +6949,64 @@ public void getDurationToProgressUs_withDropToKeyFrame_returnsDefaultDuration() verify(eventListener).onVideoEnabled(argumentDecoderCounters.capture()); DecoderCounters decoderCounters = argumentDecoderCounters.getValue(); assertThat(decoderCounters.droppedBufferCount).isEqualTo(3); - assertThat(durationToProgressIncurringDropUs).isEqualTo(0); + assertThat(durationToProgressBeforeDropUs).isEqualTo(0); assertThat(durationToProgressPostDropUs).isEqualTo(10_000); } + @Test + public void getDurationToProgressUs_whenFrameBecomesSchedulable_doesNotSkipFrameOnNextRender() + throws Exception { + FakeSampleStream fakeSampleStream = + new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), + /* mediaSourceEventDispatcher= */ null, + DrmSessionManager.DRM_UNSUPPORTED, + new DrmSessionEventListener.EventDispatcher(), + /* initialFormat= */ VIDEO_H264, + ImmutableList.of( + oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME), + oneByteSample(/* timeUs= */ 75_000), + oneByteSample(/* timeUs= */ 150_000), + END_OF_STREAM_ITEM)); + fakeSampleStream.writeData(/* startPositionUs= */ 0); + mediaCodecVideoRenderer.enable( + RendererConfiguration.DEFAULT, + new Format[] {VIDEO_H264}, + fakeSampleStream, + /* positionUs= */ 0, + /* joining= */ false, + /* mayRenderStartOfStream= */ true, + /* startPositionUs= */ 0, + /* offsetUs= */ 0, + /* mediaPeriodId= */ new MediaSource.MediaPeriodId(new Object())); + mediaCodecVideoRenderer.start(); + for (int i = 0; i < 6; i++) { + // Render first frame (0 us) and process up to second frame (75_000 us), which is held with + // TRY_AGAIN_LATER because earlyUs (75_000) > 50_000. + mediaCodecVideoRenderer.render(0, SystemClock.elapsedRealtime() * 1000); + codecAdapterFactory.idleQueueingAndCallbackThreads(); + } + + // Query duration at positionUs = 30_000 so earlyUs = 45_000 <= 50_000 (FRAME_RELEASE_SCHEDULED) + long durationToProgressUs = + mediaCodecVideoRenderer.getDurationToProgressUs( + /* positionUs= */ 30_000, + SystemClock.elapsedRealtime() * 1000, + /* isOnBufferAvailableListenerRegistered= */ true); + // Subsequent render at positionUs = 30_000 should schedule the 75_000 us frame, not skip it. + mediaCodecVideoRenderer.render(30_000, SystemClock.elapsedRealtime() * 1000); + + shadowOf(testMainLooper).idle(); + ArgumentCaptor argumentDecoderCounters = + ArgumentCaptor.forClass(DecoderCounters.class); + verify(eventListener).onVideoEnabled(argumentDecoderCounters.capture()); + DecoderCounters decoderCounters = argumentDecoderCounters.getValue(); + assertThat(durationToProgressUs).isEqualTo(0); + assertThat(decoderCounters.renderedOutputBufferCount).isEqualTo(2); + assertThat(decoderCounters.skippedOutputBufferCount).isEqualTo(0); + assertThat(decoderCounters.droppedBufferCount).isEqualTo(0); + } + @Test public void getDurationToProgressUs_afterSkippedFrame_returnsDefaultDuration() throws Exception { FakeSampleStream fakeSampleStream = @@ -6732,37 +7159,9 @@ public void getDurationToProgressUs_afterDisable_returnsDefaultDuration() throws @Test public void handlesHagcSupplementalData() throws Exception { assumeTrue("Skipping HAGC test on SDK < 37", SDK_INT >= 37); - Bundle[] capturedParameters = new Bundle[1]; - MediaCodecAdapter.Factory customAdapterFactory = - configuration -> { - MediaCodecAdapter adapter = codecAdapterFactory.createAdapter(configuration); - return new ForwardingMediaCodecAdapter(adapter) { - @Override - public void setParameters(Bundle params) { - capturedParameters[0] = params; - super.setParameters(params); - } - }; - }; + List capturedParameters = new ArrayList<>(); MediaCodecVideoRenderer renderer = - new MediaCodecVideoRenderer( - new MediaCodecVideoRenderer.Builder(ApplicationProvider.getApplicationContext()) - .setCodecAdapterFactory(customAdapterFactory) - .setMediaCodecSelector(mediaCodecSelector)); - FakeClock fakeClock = new FakeClock(/* initialTimeMs= */ 0); - renderer.init(/* index= */ 0, PlayerId.UNSET, fakeClock); - Surface surface = new Surface(new SurfaceTexture(0)); - renderer.handleMessage(Renderer.MSG_SET_VIDEO_OUTPUT, surface); - FakeSampleStream fakeSampleStream = - createFakeSampleStream( - VIDEO_H264, ImmutableList.of(oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME))); - enableAndStartRenderer(renderer, VIDEO_H264, fakeSampleStream); - renderAndAdvance( - renderer, - fakeClock, - /* startPositionUs= */ 0, - /* stopCondition= */ () -> renderer.hasReadStreamToEnd(), - /* maxIterations= */ 100); + createAndStartRendererCapturingSetParameters(VIDEO_H264, capturedParameters); DecoderInputBuffer buffer = new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); byte[] hagcMagic = createByteArray(0xB5, 0x00, 0x90, 0x00, 0x01); @@ -6770,9 +7169,113 @@ public void setParameters(Bundle params) { buffer.supplementalData = ByteBuffer.wrap(Bytes.concat(hagcMagic, payload)); renderer.handleInputBufferSupplementalData(buffer); - assertThat(capturedParameters[0]).isNotNull(); - assertThat(capturedParameters[0].getByteArray("hdr-st2094-50-info")) + assertThat(capturedParameters).hasSize(1); + assertThat(capturedParameters.get(0).getByteArray("hdr-st2094-50-info")) + .isEqualTo(new byte[] {0x0A, 0x0B, 0x0C}); + } + + @Test + public void handlesAv1HighBitdepthInBandHagcMetadata() throws Exception { + assumeTrue("Skipping HAGC test on SDK < 37", SDK_INT >= 37); + List capturedParameters = new ArrayList<>(); + MediaCodecVideoRenderer renderer = + createAndStartRendererCapturingSetParameters( + createAv1Format(/* bitdepth= */ 10), capturedParameters); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + buffer.data = ByteBuffer.wrap(createAv1HagcMetadataObu()); + + renderer.onQueueInputBuffer(buffer); + + assertThat(capturedParameters).hasSize(1); + assertThat(capturedParameters.get(0).getByteArray("hdr-st2094-50-info")) .isEqualTo(new byte[] {0x0A, 0x0B, 0x0C}); + assertThat(buffer.data.get(2)).isEqualTo((byte) 0x1F); + } + + @Test + public void handlesAv1HighBitdepthInBandHagcAndHdr10PlusMetadata() throws Exception { + assumeTrue("Skipping HAGC test on SDK < 37", SDK_INT >= 37); + List capturedParameters = new ArrayList<>(); + MediaCodecVideoRenderer renderer = + createAndStartRendererCapturingSetParameters( + createAv1Format(/* bitdepth= */ 10), capturedParameters); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + byte[] av1Hdr10PlusMetadataObu = + createByteArray( + // AV1 metadata OBU (0x2A) + 0x2A, + // OBU size 10 (0x0A) + 0x0A, + // metadata_type = 4 (0x04) + 0x04, + // HDR10+ T.35 header + 0xB5, + 0x00, + 0x3C, + 0x00, + 0x01, + 0x04, + // HDR10+ payload + 0x01, + 0x02, + // OBU trailing bits (0x80) + 0x80); + // HDR10+ metadata OBU (bytes 0-11) followed by HAGC metadata OBU (bytes 12-23). + buffer.data = + ByteBuffer.wrap(Bytes.concat(av1Hdr10PlusMetadataObu, createAv1HagcMetadataObu())); + + renderer.onQueueInputBuffer(buffer); + + assertThat(capturedParameters).hasSize(1); + assertThat(capturedParameters.get(0).getByteArray("hdr-st2094-50-info")) + .isEqualTo(new byte[] {0x0A, 0x0B, 0x0C}); + // Both the HDR10+ and the HAGC metadata OBUs are rewritten as unknown metadata OBUs. + assertThat(buffer.data.get(2)).isEqualTo((byte) 0x1F); + assertThat(buffer.data.get(14)).isEqualTo((byte) 0x1F); + } + + @Test + public void av1EightBitInBandHagcMetadata_isIgnored() throws Exception { + assumeTrue("Skipping HAGC test on SDK < 37", SDK_INT >= 37); + List capturedParameters = new ArrayList<>(); + MediaCodecVideoRenderer renderer = + createAndStartRendererCapturingSetParameters( + createAv1Format(/* bitdepth= */ 8), capturedParameters); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + buffer.data = ByteBuffer.wrap(createAv1HagcMetadataObu()); + + renderer.onQueueInputBuffer(buffer); + + assertThat(capturedParameters).isEmpty(); + assertThat(buffer.data).isEqualTo(ByteBuffer.wrap(createAv1HagcMetadataObu())); + } + + @Test + public void av1HighBitdepthInBandHagcMetadata_withContainerHagc_usesContainerHagc() + throws Exception { + assumeTrue("Skipping HAGC test on SDK < 37", SDK_INT >= 37); + List capturedParameters = new ArrayList<>(); + MediaCodecVideoRenderer renderer = + createAndStartRendererCapturingSetParameters( + createAv1Format(/* bitdepth= */ 10), capturedParameters); + DecoderInputBuffer buffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_NORMAL); + byte[] hagcMagic = createByteArray(0xB5, 0x00, 0x90, 0x00, 0x01); + byte[] containerPayload = createByteArray(0x01, 0x02, 0x03); + buffer.supplementalData = ByteBuffer.wrap(Bytes.concat(hagcMagic, containerPayload)); + buffer.data = ByteBuffer.wrap(createAv1HagcMetadataObu()); + + renderer.handleInputBufferSupplementalData(buffer); + renderer.onQueueInputBuffer(buffer); + + assertThat(capturedParameters).hasSize(1); + assertThat(capturedParameters.get(0).getByteArray("hdr-st2094-50-info")) + .isEqualTo(containerPayload); + // The in-band HAGC metadata OBU is still rewritten as an unknown metadata OBU. + assertThat(buffer.data.get(2)).isEqualTo((byte) 0x1F); } private static MediaCodecInfo createMediaCodecInfo(String mimeType) { @@ -7125,7 +7628,6 @@ private TestVideoRenderer( .setEventListener(eventListener) .setMaxDroppedFramesToNotify(1) .experimentalSetLateThresholdToDropDecoderInputUs(100_000) - .setEnableDurationToProgressUs(true) .setAssumedMinimumCodecOperatingRate(assumedMinimumCodecOperatingRate)); } @@ -7558,6 +8060,79 @@ public DrmSession acquireSession( }; } + private MediaCodecVideoRenderer createAndStartRendererCapturingSetParameters( + Format format, List capturedParameters) throws Exception { + MediaCodecAdapter.Factory customAdapterFactory = + configuration -> { + MediaCodecAdapter adapter = codecAdapterFactory.createAdapter(configuration); + return new ForwardingMediaCodecAdapter(adapter) { + @Override + public void setParameters(Bundle params) { + capturedParameters.add(params); + super.setParameters(params); + } + }; + }; + MediaCodecVideoRenderer renderer = + new MediaCodecVideoRenderer( + new MediaCodecVideoRenderer.Builder(ApplicationProvider.getApplicationContext()) + .setCodecAdapterFactory(customAdapterFactory) + .setMediaCodecSelector(mediaCodecSelector)); + FakeClock fakeClock = new FakeClock(/* initialTimeMs= */ 0); + renderer.init(/* index= */ 0, PlayerId.UNSET, fakeClock); + Surface surface = new Surface(new SurfaceTexture(0)); + renderer.handleMessage(Renderer.MSG_SET_VIDEO_OUTPUT, surface); + FakeSampleStream fakeSampleStream = + createFakeSampleStream( + format, ImmutableList.of(oneByteSample(/* timeUs= */ 0, C.BUFFER_FLAG_KEY_FRAME))); + enableAndStartRenderer(renderer, format, fakeSampleStream); + renderAndAdvance( + renderer, + fakeClock, + /* startPositionUs= */ 0, + /* stopCondition= */ () -> renderer.hasReadStreamToEnd(), + /* maxIterations= */ 100); + return renderer; + } + + private static Format createAv1Format(int bitdepth) { + return new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_AV1) + .setWidth(1920) + .setHeight(1080) + .setColorInfo( + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorRange(C.COLOR_RANGE_LIMITED) + .setColorTransfer(C.COLOR_TRANSFER_ST2084) + .setLumaBitdepth(bitdepth) + .setChromaBitdepth(bitdepth) + .build()) + .build(); + } + + private static byte[] createAv1HagcMetadataObu() { + return createByteArray( + // AV1 metadata OBU (0x2A) + 0x2A, + // OBU size 10 (0x0A) + 0x0A, + // metadata_type = 4 (0x04) + 0x04, + // HAGC T.35 header + 0xB5, + 0x00, + 0x90, + 0x00, + 0x01, + // HAGC payload + 0x0A, + 0x0B, + 0x0C, + // OBU trailing bits (0x80) + 0x80); + } + private FakeSampleStream createFakeSampleStream( Format format, List samples) { FakeSampleStream fakeSampleStream = @@ -7668,4 +8243,209 @@ public void flush() { pendingBufferCount = 0; } } + + /** + * A custom MediaCodecAdapter that bypasses 1-in 1-out limitations and allows precise, manual + * dispatching of output buffers for testing stale frame boundaries. + */ + private static class StaleFrameTestingMediaCodecAdapter extends ForwardingMediaCodecAdapter { + + private boolean allowFormatChange = false; + private boolean allowFirstFrameOutput = false; + private boolean allowStaleFrameOutput = false; + private int availableInputBufferCount = 4; + private int inputBufferIndexCounter = 0; + private final MediaFormat mediaFormat = new MediaFormat(); + private final ByteBuffer buffer = ByteBuffer.allocate(1024); + + private StaleFrameTestingMediaCodecAdapter(MediaCodecAdapter delegate) { + super(delegate); + mediaFormat.setInteger(MediaFormat.KEY_WIDTH, 1920); + mediaFormat.setInteger(MediaFormat.KEY_HEIGHT, 1080); + } + + @Override + public void useBuffer(ThrowingRunnable runnable) throws E { + runnable.run(); + } + + @Override + public int dequeueInputBufferIndex() { + if (availableInputBufferCount > 0) { + availableInputBufferCount--; + return inputBufferIndexCounter++; + } + return MediaCodec.INFO_TRY_AGAIN_LATER; + } + + @Override + public int dequeueOutputBufferIndex(MediaCodec.BufferInfo bufferInfo) { + if (allowFormatChange) { + allowFormatChange = false; + return MediaCodec.INFO_OUTPUT_FORMAT_CHANGED; + } + if (allowFirstFrameOutput) { + allowFirstFrameOutput = false; + bufferInfo.set(0, 100, 10_000L, 0); + return 0; + } + if (allowStaleFrameOutput) { + allowStaleFrameOutput = false; + bufferInfo.set(0, 100, 30_000L, 0); + return 0; + } + return MediaCodec.INFO_TRY_AGAIN_LATER; + } + + @Override + public MediaFormat getOutputFormat() { + return mediaFormat; + } + + @Nullable + @Override + public ByteBuffer getInputBuffer(int index) { + return buffer; + } + + @Nullable + @Override + public ByteBuffer getOutputBuffer(int index) { + return buffer; + } + + @Override + public void queueInputBuffer( + int index, int offset, int size, long presentationTimeUs, int flags) { + // No-op: prevent Robolectric's strict state checks from crashing on fake indices + } + + @Override + public void releaseOutputBuffer(int index, boolean render) { + // No-op + } + + @Override + public void releaseOutputBuffer(int index, long renderTimeStampNs) { + // No-op + } + } + + private static final class FakeVideoSink implements VideoSink { + + private final Surface inputSurface; + private final List inputStreamFormats = new ArrayList<>(); + private final List handledFrameTimestampsUs = new ArrayList<>(); + private boolean initialized; + + private FakeVideoSink(Surface inputSurface) { + this.inputSurface = inputSurface; + } + + @Override + public void startRendering() {} + + @Override + public void stopRendering() {} + + @Override + public void setListener(Listener listener, Executor executor) {} + + @Override + public boolean initialize(Format sourceFormat) { + initialized = true; + return true; + } + + @Override + public boolean isInitialized() { + return initialized; + } + + @Override + public void redraw() {} + + @Override + public void flush(boolean resetPosition) {} + + @Override + public boolean isReady(boolean otherwiseReady) { + return otherwiseReady; + } + + @Override + public void signalEndOfCurrentInputStream() {} + + @Override + public void signalEndOfInput() {} + + @Override + public boolean isEnded() { + return false; + } + + @Override + public Surface getInputSurface() { + return inputSurface; + } + + @Override + public void setVideoFrameMetadataListener( + VideoFrameMetadataListener videoFrameMetadataListener) {} + + @Override + public void setPlaybackSpeed(float speed) {} + + @Override + public void setVideoEffects(List videoEffects) {} + + @Override + public void setBufferTimestampAdjustmentUs(long bufferTimestampAdjustmentUs) {} + + @Override + public void setOutputSurfaceInfo(Surface outputSurface, Size outputResolution) {} + + @Override + public void clearOutputSurfaceInfo() {} + + @Override + public void setChangeFrameRateStrategy(int changeFrameRateStrategy) {} + + @Override + public void onInputStreamChanged( + @InputType int inputType, + Format format, + long startPositionUs, + @FirstFrameReleaseInstruction int firstFrameReleaseInstruction, + List videoEffects) { + inputStreamFormats.add(format); + } + + @Override + public void allowReleaseFirstFrameBeforeStarted() {} + + @Override + public boolean handleInputFrame( + long bufferPresentationTimeUs, VideoFrameHandler videoFrameHandler) { + checkState(!inputStreamFormats.isEmpty()); + handledFrameTimestampsUs.add(bufferPresentationTimeUs); + videoFrameHandler.render(/* renderTimestampNs= */ 0); + return true; + } + + @Override + public boolean handleInputBitmap( + Bitmap inputBitmap, TimestampIterator bufferTimestampIterator) { + return false; + } + + @Override + public void render(long positionUs, long elapsedRealtimeUs) {} + + @Override + public void join(boolean renderNextFrameImmediately) {} + + @Override + public void release() {} + } } diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/PlaybackVideoGraphWrapperTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/PlaybackVideoGraphWrapperTest.java index 4ce7d4d4819..e29e6453e2c 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/PlaybackVideoGraphWrapperTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/PlaybackVideoGraphWrapperTest.java @@ -378,7 +378,10 @@ public boolean shouldIgnoreFrame( } }; return new VideoFrameReleaseControl( - context, frameTimingEvaluator, /* allowedJoiningTimeMs= */ 0); + context, + frameTimingEvaluator, + /* allowedJoiningTimeMs= */ 0, + /* skipBuffersWithIdenticalReleaseTime= */ true); } private static class TestVideoGraphFactory implements VideoGraph.Factory { diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/VideoFrameReleaseControlTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/VideoFrameReleaseControlTest.java index 5e0795f216b..abe5f16f4f7 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/VideoFrameReleaseControlTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/VideoFrameReleaseControlTest.java @@ -638,7 +638,8 @@ public void getFrameReleaseAction_customEarlySchedulingThreshold_returnsFrameRel /* shouldForceRelease= */ true, /* shouldDropFrame= */ false, /* shouldIgnoreFrame= */ false), - /* allowedJoiningTimeMs= */ 0); + /* allowedJoiningTimeMs= */ 0, + /* skipBuffersWithIdenticalReleaseTime= */ true); videoFrameReleaseControl.setOutputSurface(surface); videoFrameReleaseControl.setClock(clock); videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); @@ -688,7 +689,8 @@ public void getFrameReleaseAction_forceReleaseWhileNotStarted_returnsTryAgainLat /* shouldForceRelease= */ true, /* shouldDropFrame= */ false, /* shouldIgnoreFrame= */ false), - /* allowedJoiningTimeMs= */ 0); + /* allowedJoiningTimeMs= */ 0, + /* skipBuffersWithIdenticalReleaseTime= */ true); videoFrameReleaseControl.setOutputSurface(surface); videoFrameReleaseControl.setClock(clock); videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); @@ -737,7 +739,8 @@ public void getFrameReleaseAction_forceReleaseWhilePositionNotAdvancing_returnsT /* shouldForceRelease= */ true, /* shouldDropFrame= */ false, /* shouldIgnoreFrame= */ false), - /* allowedJoiningTimeMs= */ 0); + /* allowedJoiningTimeMs= */ 0, + /* skipBuffersWithIdenticalReleaseTime= */ true); videoFrameReleaseControl.setOutputSurface(surface); videoFrameReleaseControl.setClock(clock); videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); @@ -788,7 +791,8 @@ public void getFrameReleaseAction_forceReleaseWhilePositionNotAdvancing_returnsT /* shouldForceRelease= */ true, /* shouldDropFrame= */ false, /* shouldIgnoreFrame= */ false), - /* allowedJoiningTimeMs= */ 0); + /* allowedJoiningTimeMs= */ 0, + /* skipBuffersWithIdenticalReleaseTime= */ true); videoFrameReleaseControl.setOutputSurface(surface); videoFrameReleaseControl.setClock(clock); videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); @@ -839,7 +843,8 @@ public void getFrameReleaseAction_forceReleaseWhilePositionNotAdvancing_returnsT /* shouldForceRelease= */ true, /* shouldDropFrame= */ false, /* shouldIgnoreFrame= */ false), - /* allowedJoiningTimeMs= */ 0); + /* allowedJoiningTimeMs= */ 0, + /* skipBuffersWithIdenticalReleaseTime= */ true); videoFrameReleaseControl.setOutputSurface(surface); videoFrameReleaseControl.setClock(clock); videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); @@ -902,7 +907,8 @@ public void getFrameReleaseAction_frameLate_returnsDrop() throws ExoPlaybackExce /* shouldForceRelease= */ false, /* shouldDropFrame= */ true, /* shouldIgnoreFrame= */ false), - /* allowedJoiningTimeMs= */ 0); + /* allowedJoiningTimeMs= */ 0, + /* skipBuffersWithIdenticalReleaseTime= */ true); videoFrameReleaseControl.setOutputSurface(surface); videoFrameReleaseControl.setClock(clock); videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); @@ -955,7 +961,8 @@ public void getFrameReleaseAction_frameLate_returnsDrop() throws ExoPlaybackExce /* shouldForceRelease= */ false, /* shouldDropFrame= */ true, /* shouldIgnoreFrame= */ false), - /* allowedJoiningTimeMs= */ 1234); + /* allowedJoiningTimeMs= */ 1234, + /* skipBuffersWithIdenticalReleaseTime= */ true); videoFrameReleaseControl.setOutputSurface(surface); videoFrameReleaseControl.setClock(clock); videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); @@ -1007,7 +1014,8 @@ public void getFrameReleaseAction_frameLate_returnsDrop() throws ExoPlaybackExce /* shouldForceRelease= */ false, /* shouldDropFrame= */ true, /* shouldIgnoreFrame= */ false), - /* allowedJoiningTimeMs= */ 1234); + /* allowedJoiningTimeMs= */ 1234, + /* skipBuffersWithIdenticalReleaseTime= */ true); videoFrameReleaseControl.setOutputSurface(surface); videoFrameReleaseControl.setClock(clock); videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); @@ -1058,7 +1066,8 @@ public void getFrameReleaseAction_shouldIgnore() throws ExoPlaybackException { /* shouldForceRelease= */ false, /* shouldDropFrame= */ false, /* shouldIgnoreFrame= */ true), - /* allowedJoiningTimeMs= */ 0); + /* allowedJoiningTimeMs= */ 0, + /* skipBuffersWithIdenticalReleaseTime= */ true); videoFrameReleaseControl.setOutputSurface(surface); videoFrameReleaseControl.setClock(clock); videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); @@ -1201,7 +1210,8 @@ public void getFrameReleaseAction_withoutSurfaceShouldIgnore_returnsIgnore() thr /* shouldForceRelease= */ false, /* shouldDropFrame= */ false, /* shouldIgnoreFrame= */ true), - /* allowedJoiningTimeMs= */ 0); + /* allowedJoiningTimeMs= */ 0, + /* skipBuffersWithIdenticalReleaseTime= */ true); videoFrameReleaseControl.setOutputSurface(/* outputSurface= */ null); videoFrameReleaseControl.setClock(clock); videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); @@ -1274,6 +1284,629 @@ public void getFrameReleaseAction_withoutSurfaceFrameLateAndStarted_returnsSkip( .isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_SKIP); } + @Test + public void getFrameReleaseAction_duplicateReleaseTimeAndHighFps_returnsSkip() + throws ExoPlaybackException { + VideoFrameReleaseControl.FrameReleaseInfo frameReleaseInfo = + new VideoFrameReleaseControl.FrameReleaseInfo(); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ false); + VideoFrameReleaseControl videoFrameReleaseControl = createVideoFrameReleaseControl(); + videoFrameReleaseControl.setClock(clock); + videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); + videoFrameReleaseControl.onStarted(); + videoFrameReleaseControl.setVsyncData( + /* vsyncSampleTimeNs= */ 0, /* vsyncDurationNs= */ 16_666_666); + + // Frame 0 (First frame, released immediately) + assertThat( + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 0, + /* positionUs= */ 0, + /* elapsedRealtimeUs= */ 0, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, // 120fps + /* frameIndex= */ 0L, + frameReleaseInfo)) + .isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_IMMEDIATELY); + videoFrameReleaseControl.onFrameReleasedIsFirstFrame(); + + // Frame 1 (120fps, snaps to vsync 0 = -13.33ms) + clock.advanceTime(8); // ms + assertThat( + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 8_333, + /* positionUs= */ 8_333, + /* elapsedRealtimeUs= */ 8_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 1L, + frameReleaseInfo)) + .isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_SCHEDULED); + assertThat(frameReleaseInfo.getReleaseTimeNs()).isEqualTo(-13_333_332); + + // Frame 2 (120fps, snaps to vsync 1 = 3.33ms) + clock.advanceTime(8); // ms (total 16ms) + assertThat( + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 16_666, + /* positionUs= */ 16_666, + /* elapsedRealtimeUs= */ 16_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 2L, + frameReleaseInfo)) + .isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_SCHEDULED); + assertThat(frameReleaseInfo.getReleaseTimeNs()).isEqualTo(3_333_334); + + // Frame 3 (120fps, also snaps to vsync 1 = 3.33ms -> COLLISION, keeping up -> SKIP) + clock.advanceTime(8); // ms (total 24ms) + assertThat( + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 25_000, + /* positionUs= */ 25_000, + /* elapsedRealtimeUs= */ 24_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 3L, + frameReleaseInfo)) + .isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_SKIP); + } + + @Test + public void getFrameReleaseAction_duplicateReleaseTimeAndStarved_returnsDrop() + throws ExoPlaybackException { + VideoFrameReleaseControl.FrameReleaseInfo frameReleaseInfo = + new VideoFrameReleaseControl.FrameReleaseInfo(); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ false); + VideoFrameReleaseControl videoFrameReleaseControl = createVideoFrameReleaseControl(); + videoFrameReleaseControl.setClock(clock); + videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); + videoFrameReleaseControl.onStarted(); + videoFrameReleaseControl.setVsyncData( + /* vsyncSampleTimeNs= */ 0, /* vsyncDurationNs= */ 16_666_666); + + // Frame 0 (First frame, released immediately) + assertThat( + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 0, + /* positionUs= */ 0, + /* elapsedRealtimeUs= */ 0, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, // 120fps + /* frameIndex= */ 0L, + frameReleaseInfo)) + .isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_IMMEDIATELY); + videoFrameReleaseControl.onFrameReleasedIsFirstFrame(); + + // Frame 1 (120fps, snaps to vsync 0 = -13.33ms) + clock.advanceTime(8); // ms + assertThat( + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 8_333, + /* positionUs= */ 8_333, + /* elapsedRealtimeUs= */ 8_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 1L, + frameReleaseInfo)) + .isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_SCHEDULED); + assertThat(frameReleaseInfo.getReleaseTimeNs()).isEqualTo(-13_333_332); + + // Frame 2 (120fps, delayed, snaps to vsync 3 = 36.67ms) + clock.advanceTime(41); // ms (total 49ms) + assertThat( + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 16_667, + /* positionUs= */ 16_667, + /* elapsedRealtimeUs= */ 49_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 2L, + frameReleaseInfo)) + .isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_SCHEDULED); + assertThat(frameReleaseInfo.getReleaseTimeNs()).isEqualTo(36_666_666); + + // Frame 3 (120fps, wants vsync 3 = 36.67ms -> COLLISION, starved -> DROP) + // No clock advance, still 49ms + assertThat( + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 25_000, + /* positionUs= */ 16_667, // position is still at Frame 2 + /* elapsedRealtimeUs= */ 49_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 3L, + frameReleaseInfo)) + .isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_DROP); + } + + @Test + public void getFrameReleaseAction_afterPauseAndResume_returnsScheduled() + throws ExoPlaybackException { + VideoFrameReleaseControl.FrameReleaseInfo frameReleaseInfo = + new VideoFrameReleaseControl.FrameReleaseInfo(); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ false); + VideoFrameReleaseControl videoFrameReleaseControl = createVideoFrameReleaseControl(); + videoFrameReleaseControl.setClock(clock); + videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); + videoFrameReleaseControl.onStarted(); + videoFrameReleaseControl.setVsyncData( + /* vsyncSampleTimeNs= */ 0, /* vsyncDurationNs= */ 16_666_666); + + // Frame 0 (First frame, released immediately) + assertThat( + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 0, + /* positionUs= */ 0, + /* elapsedRealtimeUs= */ 0, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 16_666_666, + /* frameIndex= */ 0L, + frameReleaseInfo)) + .isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_IMMEDIATELY); + videoFrameReleaseControl.onFrameReleasedIsFirstFrame(); + + // Frame 1 (Scheduled frame at PTS 16.6ms) + clock.advanceTime(16); + assertThat( + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 16_666, + /* positionUs= */ 16_666, + /* elapsedRealtimeUs= */ 16_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 16_666_666, + /* frameIndex= */ 1L, + frameReleaseInfo)) + .isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_SCHEDULED); + + // Pause playback + videoFrameReleaseControl.onStopped(); + clock.advanceTime(100); + + // Resume playback + videoFrameReleaseControl.onStarted(); + + // Frame 2 (First frame after resume, scheduled at PTS 116.6ms) + assertThat( + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 116_666, + /* positionUs= */ 116_666, + /* elapsedRealtimeUs= */ 116_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 16_666_666, + /* frameIndex= */ 2L, + frameReleaseInfo)) + .isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_SCHEDULED); + + // Frame 3 (Second frame after resume, scheduled at PTS 133.3ms) + clock.advanceTime(17); + assertThat( + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 133_333, + /* positionUs= */ 133_333, + /* elapsedRealtimeUs= */ 133_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 16_666_666, + /* frameIndex= */ 3L, + frameReleaseInfo)) + .isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_SCHEDULED); + } + + @Test + public void + probeFrameReleaseAction_withSamePresentationTimeBeforeRelease_returnsScheduledWithoutColliding() + throws ExoPlaybackException { + VideoFrameReleaseControl.FrameReleaseInfo frameReleaseInfo = + new VideoFrameReleaseControl.FrameReleaseInfo(); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ false); + VideoFrameReleaseControl videoFrameReleaseControl = createVideoFrameReleaseControl(); + videoFrameReleaseControl.setClock(clock); + videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); + videoFrameReleaseControl.onStarted(); + videoFrameReleaseControl.setVsyncData( + /* vsyncSampleTimeNs= */ 0, /* vsyncDurationNs= */ 16_666_666); + int unused = + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 0, + /* positionUs= */ 0, + /* elapsedRealtimeUs= */ 0, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 0L, + frameReleaseInfo); + videoFrameReleaseControl.onFrameReleasedIsFirstFrame(); + clock.advanceTime(8); + + int probeAction = + videoFrameReleaseControl.probeFrameReleaseAction( + /* presentationTimeUs= */ 8_333, + /* positionUs= */ 8_333, + /* elapsedRealtimeUs= */ 8_000, + /* outputStreamStartPositionUs= */ 0, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 1L, + frameReleaseInfo); + long probedReleaseTimeNs = frameReleaseInfo.getReleaseTimeNs(); + int releaseAction = + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 8_333, + /* positionUs= */ 8_333, + /* elapsedRealtimeUs= */ 8_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 1L, + frameReleaseInfo); + long releasedReleaseTimeNs = frameReleaseInfo.getReleaseTimeNs(); + + assertThat(probeAction).isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_SCHEDULED); + assertThat(releaseAction).isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_SCHEDULED); + assertThat(releasedReleaseTimeNs).isEqualTo(probedReleaseTimeNs); + } + + @Test + public void probeFrameReleaseAction_withSamePresentationTime_isIdempotentForSkip() + throws ExoPlaybackException { + VideoFrameReleaseControl.FrameReleaseInfo frameReleaseInfo = + new VideoFrameReleaseControl.FrameReleaseInfo(); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ false); + VideoFrameReleaseControl videoFrameReleaseControl = createVideoFrameReleaseControl(); + videoFrameReleaseControl.setClock(clock); + videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); + videoFrameReleaseControl.onStarted(); + videoFrameReleaseControl.setVsyncData( + /* vsyncSampleTimeNs= */ 0, /* vsyncDurationNs= */ 16_666_666); + int unused = + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 0, + /* positionUs= */ 0, + /* elapsedRealtimeUs= */ 0, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 0L, + frameReleaseInfo); + videoFrameReleaseControl.onFrameReleasedIsFirstFrame(); + clock.advanceTime(8); + unused = + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 8_333, + /* positionUs= */ 8_333, + /* elapsedRealtimeUs= */ 8_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 1L, + frameReleaseInfo); + clock.advanceTime(8); + unused = + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 16_666, + /* positionUs= */ 16_666, + /* elapsedRealtimeUs= */ 16_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 2L, + frameReleaseInfo); + clock.advanceTime(8); + + int probedSkipAction = + videoFrameReleaseControl.probeFrameReleaseAction( + /* presentationTimeUs= */ 25_000, + /* positionUs= */ 25_000, + /* elapsedRealtimeUs= */ 24_000, + /* outputStreamStartPositionUs= */ 0, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 3L, + frameReleaseInfo); + int nonProbedSkipAction = + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 25_000, + /* positionUs= */ 25_000, + /* elapsedRealtimeUs= */ 24_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 3L, + frameReleaseInfo); + + assertThat(probedSkipAction).isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_SKIP); + assertThat(nonProbedSkipAction).isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_SKIP); + } + + @Test + public void probeFrameReleaseAction_withSamePresentationTime_isIdempotentForDrop() + throws ExoPlaybackException { + VideoFrameReleaseControl.FrameReleaseInfo frameReleaseInfo = + new VideoFrameReleaseControl.FrameReleaseInfo(); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ false); + VideoFrameReleaseControl videoFrameReleaseControl = createVideoFrameReleaseControl(); + videoFrameReleaseControl.setClock(clock); + videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); + videoFrameReleaseControl.onStarted(); + videoFrameReleaseControl.setVsyncData( + /* vsyncSampleTimeNs= */ 0, /* vsyncDurationNs= */ 16_666_666); + int unused = + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 0, + /* positionUs= */ 0, + /* elapsedRealtimeUs= */ 0, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 0L, + frameReleaseInfo); + videoFrameReleaseControl.onFrameReleasedIsFirstFrame(); + clock.advanceTime(8); + unused = + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 8_333, + /* positionUs= */ 8_333, + /* elapsedRealtimeUs= */ 8_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 1L, + frameReleaseInfo); + clock.advanceTime(41); + unused = + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 16_667, + /* positionUs= */ 16_667, + /* elapsedRealtimeUs= */ 49_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 2L, + frameReleaseInfo); + + int probedDropAction = + videoFrameReleaseControl.probeFrameReleaseAction( + /* presentationTimeUs= */ 25_000, + /* positionUs= */ 16_667, + /* elapsedRealtimeUs= */ 49_000, + /* outputStreamStartPositionUs= */ 0, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 3L, + frameReleaseInfo); + int nonProbedDropAction = + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 25_000, + /* positionUs= */ 16_667, + /* elapsedRealtimeUs= */ 49_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 3L, + frameReleaseInfo); + + assertThat(probedDropAction).isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_DROP); + assertThat(nonProbedDropAction).isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_DROP); + } + + @Test + public void + probeFrameReleaseAction_withDifferentPresentationTime_doesNotMutateStateForSubsequentFrames() + throws ExoPlaybackException { + VideoFrameReleaseControl.FrameReleaseInfo frameReleaseInfo = + new VideoFrameReleaseControl.FrameReleaseInfo(); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ false); + VideoFrameReleaseControl videoFrameReleaseControl = createVideoFrameReleaseControl(); + videoFrameReleaseControl.setClock(clock); + videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); + videoFrameReleaseControl.onStarted(); + videoFrameReleaseControl.setVsyncData( + /* vsyncSampleTimeNs= */ 0, /* vsyncDurationNs= */ 16_666_666); + int unused = + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 0, + /* positionUs= */ 0, + /* elapsedRealtimeUs= */ 0, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 0L, + frameReleaseInfo); + videoFrameReleaseControl.onFrameReleasedIsFirstFrame(); + clock.advanceTime(8); + unused = + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 8_333, + /* positionUs= */ 8_333, + /* elapsedRealtimeUs= */ 8_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 1L, + frameReleaseInfo); + clock.advanceTime(8); + + int probedAction = + videoFrameReleaseControl.probeFrameReleaseAction( + /* presentationTimeUs= */ 16_666, + /* positionUs= */ 16_666, + /* elapsedRealtimeUs= */ 16_000, + /* outputStreamStartPositionUs= */ 0, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 2L, + frameReleaseInfo); + clock.advanceTime(8); + int nextFrameAction = + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 25_000, + /* positionUs= */ 25_000, + /* elapsedRealtimeUs= */ 24_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 8_333_333, + /* frameIndex= */ 3L, + frameReleaseInfo); + + assertThat(probedAction).isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_SCHEDULED); + assertThat(nextFrameAction).isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_SCHEDULED); + assertThat(frameReleaseInfo.getReleaseTimeNs()).isEqualTo(3_333_334); + } + + @Test + public void probeFrameReleaseAction_whenShouldIgnoreFrameIsTrue_returnsDropInsteadOfIgnore() + throws ExoPlaybackException { + VideoFrameReleaseControl.FrameReleaseInfo frameReleaseInfo = + new VideoFrameReleaseControl.FrameReleaseInfo(); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ false); + VideoFrameReleaseControl videoFrameReleaseControl = + new VideoFrameReleaseControl( + ApplicationProvider.getApplicationContext(), + new TestFrameTimingEvaluator( + /* shouldForceRelease= */ false, + /* shouldDropFrame= */ true, + /* shouldIgnoreFrame= */ true), + /* allowedJoiningTimeMs= */ 0, + /* skipBuffersWithIdenticalReleaseTime= */ true); + videoFrameReleaseControl.setOutputSurface(surface); + videoFrameReleaseControl.setClock(clock); + videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); + videoFrameReleaseControl.onStarted(); + int unused = + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 0, + /* positionUs= */ 0, + /* elapsedRealtimeUs= */ 0, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 16_666_666, + /* frameIndex= */ 0L, + frameReleaseInfo); + videoFrameReleaseControl.onFrameReleasedIsFirstFrame(); + clock.advanceTime(600); + + int probedAction = + videoFrameReleaseControl.probeFrameReleaseAction( + /* presentationTimeUs= */ 16_666, + /* positionUs= */ 600_000, + /* elapsedRealtimeUs= */ 600_000, + /* outputStreamStartPositionUs= */ 0, + /* frameDurationNs= */ 16_666_666, + /* frameIndex= */ 1L, + frameReleaseInfo); + int nonProbedAction = + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 16_666, + /* positionUs= */ 600_000, + /* elapsedRealtimeUs= */ 600_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 16_666_666, + /* frameIndex= */ 1L, + frameReleaseInfo); + + assertThat(probedAction).isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_DROP); + assertThat(nonProbedAction).isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_IGNORE); + } + + @Test + public void probeFrameReleaseAction_rightAfterOnStarted_doesNotMutateInitialPositionUs() + throws ExoPlaybackException { + VideoFrameReleaseControl.FrameReleaseInfo frameReleaseInfo = + new VideoFrameReleaseControl.FrameReleaseInfo(); + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ false); + VideoFrameReleaseControl videoFrameReleaseControl = createVideoFrameReleaseControl(); + videoFrameReleaseControl.setClock(clock); + videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); + int unused = + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 0, + /* positionUs= */ 0, + /* elapsedRealtimeUs= */ 0, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 16_666_666, + /* frameIndex= */ 0L, + frameReleaseInfo); + videoFrameReleaseControl.onFrameReleasedIsFirstFrame(); + videoFrameReleaseControl.onStarted(); + + int probeAtStartAction = + videoFrameReleaseControl.probeFrameReleaseAction( + /* presentationTimeUs= */ 16_666, + /* positionUs= */ 0, + /* elapsedRealtimeUs= */ 0, + /* outputStreamStartPositionUs= */ 0, + /* frameDurationNs= */ 16_666_666, + /* frameIndex= */ 1L, + frameReleaseInfo); + int firstNonProbeAtStartAction = + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 16_666, + /* positionUs= */ 10_000, + /* elapsedRealtimeUs= */ 10_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 16_666_666, + /* frameIndex= */ 1L, + frameReleaseInfo); + int secondNonProbeAfterAdvancingAction = + videoFrameReleaseControl.getFrameReleaseAction( + /* presentationTimeUs= */ 16_666, + /* positionUs= */ 16_000, + /* elapsedRealtimeUs= */ 16_000, + /* outputStreamStartPositionUs= */ 0, + /* isDecodeOnlyFrame= */ false, + /* isLastFrame= */ false, + /* frameDurationNs= */ 16_666_666, + /* frameIndex= */ 1L, + frameReleaseInfo); + + assertThat(probeAtStartAction) + .isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_TRY_AGAIN_LATER); + assertThat(firstNonProbeAtStartAction) + .isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_TRY_AGAIN_LATER); + assertThat(secondNonProbeAfterAdvancingAction) + .isEqualTo(VideoFrameReleaseControl.FRAME_RELEASE_SCHEDULED); + } + private VideoFrameReleaseControl createVideoFrameReleaseControl() { return createVideoFrameReleaseControl(/* allowedJoiningTimeMs= */ 0); } @@ -1283,7 +1916,8 @@ private VideoFrameReleaseControl createVideoFrameReleaseControl(long allowedJoin new VideoFrameReleaseControl( ApplicationProvider.getApplicationContext(), new TestFrameTimingEvaluator(), - allowedJoiningTimeMs); + allowedJoiningTimeMs, + /* skipBuffersWithIdenticalReleaseTime= */ true); videoFrameReleaseControl.setOutputSurface(surface); frameRateEstimator = new FixedFrameRateEstimator( diff --git a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/VideoFrameRenderControlTest.java b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/VideoFrameRenderControlTest.java index 6803851b183..280cea1667c 100644 --- a/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/VideoFrameRenderControlTest.java +++ b/libraries/exoplayer/src/test/java/androidx/media3/exoplayer/video/VideoFrameRenderControlTest.java @@ -215,7 +215,7 @@ public void renderFrames_withStreamStartPositionChange_firstFrameAgain() throws videoFrameRenderControl.onStreamChanged( RELEASE_FIRST_FRAME_WHEN_PREVIOUS_STREAM_PROCESSED, /* streamStartPositionUs= */ 20_000); videoFrameRenderControl.onFrameAvailableForRendering(/* presentationTimeUs= */ 10_000); - videoFrameRenderControl.render(/* positionUs= */ 10_000, /* elapsedRealtimeUs= */ 0); + videoFrameRenderControl.render(/* positionUs= */ 10_000, /* elapsedRealtimeUs= */ 10_000); // Second frame has the second stream start position and it is also a first frame. inOrder @@ -396,7 +396,8 @@ private VideoFrameReleaseControl createVideoFrameReleaseControl( new VideoFrameReleaseControl( ApplicationProvider.getApplicationContext(), frameTimingEvaluator, - /* allowedJoiningTimeMs= */ 0); + /* allowedJoiningTimeMs= */ 0, + /* skipBuffersWithIdenticalReleaseTime= */ true); videoFrameReleaseControl.setOutputSurface(surface); return videoFrameReleaseControl; } diff --git a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/BaseUrlExclusionList.java b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/BaseUrlExclusionList.java index ebf923e23bd..16b672e3211 100644 --- a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/BaseUrlExclusionList.java +++ b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/BaseUrlExclusionList.java @@ -24,6 +24,7 @@ import androidx.annotation.VisibleForTesting; import androidx.media3.common.util.UnstableApi; import androidx.media3.exoplayer.dash.manifest.BaseUrl; +import com.google.common.collect.ImmutableList; import com.google.common.collect.Iterables; import java.util.ArrayList; import java.util.Collections; @@ -31,6 +32,7 @@ import java.util.HashSet; import java.util.List; import java.util.Map; +import java.util.Objects; import java.util.Random; import java.util.Set; @@ -46,6 +48,8 @@ public final class BaseUrlExclusionList { private final Map>, BaseUrl> selectionsTaken = new HashMap<>(); private final Random random; + @Nullable private ImmutableList serviceLocationSteeringPriority; + /** Creates an instance. */ public BaseUrlExclusionList() { this(new Random()); @@ -68,7 +72,8 @@ public BaseUrlExclusionList() { public void exclude(BaseUrl baseUrlToExclude, long exclusionDurationMs) { long excludeUntilMs = SystemClock.elapsedRealtime() + exclusionDurationMs; addExclusion(baseUrlToExclude.serviceLocation, excludeUntilMs, excludedServiceLocations); - if (baseUrlToExclude.priority != BaseUrl.PRIORITY_UNSET) { + if (serviceLocationSteeringPriority == null + && baseUrlToExclude.priority != BaseUrl.PRIORITY_UNSET) { addExclusion(baseUrlToExclude.priority, excludeUntilMs, excludedPriorities); } } @@ -91,15 +96,37 @@ public BaseUrl selectBaseUrl(List baseUrls) { } // Sort by priority and service location to make the sort order of the candidates deterministic. Collections.sort(includedBaseUrls, BaseUrlExclusionList::compareBaseUrl); + + @Nullable + ImmutableList serviceLocationSteeringPriority = this.serviceLocationSteeringPriority; + if (serviceLocationSteeringPriority != null) { + // If there is a serviceLocationSteeringPriority, select the base URL based on it. + for (int i = 0; i < serviceLocationSteeringPriority.size(); i++) { + String serviceLocation = serviceLocationSteeringPriority.get(i); + List matchingBaseUrls = findMatchingBaseUrls(serviceLocation, includedBaseUrls); + if (!matchingBaseUrls.isEmpty()) { + // Within the most preferred service location, select the base URL by priority and + // weight. + return selectBaseUrlByPriorityAndWeight(matchingBaseUrls); + } + } + } + return selectBaseUrlByPriorityAndWeight(includedBaseUrls); + } + + private BaseUrl selectBaseUrlByPriorityAndWeight(List candidates) { + if (candidates.size() == 1) { + return candidates.get(0); + } // Get candidates of the lowest priority from the head of the sorted list. List> candidateKeys = new ArrayList<>(); - int lowestPriority = includedBaseUrls.get(0).priority; - for (int i = 0; i < includedBaseUrls.size(); i++) { - BaseUrl baseUrl = includedBaseUrls.get(i); + int lowestPriority = candidates.get(0).priority; + for (int i = 0; i < candidates.size(); i++) { + BaseUrl baseUrl = candidates.get(i); if (lowestPriority != baseUrl.priority) { if (candidateKeys.size() == 1) { // Only a single candidate of lowest priority; no choice. - return includedBaseUrls.get(0); + return candidates.get(0); } break; } @@ -109,13 +136,41 @@ public BaseUrl selectBaseUrl(List baseUrls) { @Nullable BaseUrl baseUrl = selectionsTaken.get(candidateKeys); if (baseUrl == null) { // Weighted random selection from multiple candidates of the same priority. - baseUrl = selectWeighted(includedBaseUrls.subList(0, candidateKeys.size())); + baseUrl = selectWeighted(candidates.subList(0, candidateKeys.size())); // Remember the selection taken for later. selectionsTaken.put(candidateKeys, baseUrl); } return baseUrl; } + /** + * Returns the number of priority levels of the given list of base URLs. + * + * @param baseUrls The list of base URLs. + * @return The number of priority levels before exclusion. + */ + public static int getPriorityCount(List baseUrls) { + Set priorities = new HashSet<>(); + for (int i = 0; i < baseUrls.size(); i++) { + priorities.add(baseUrls.get(i).priority); + } + return priorities.size(); + } + + /** + * Returns the number of service locations for the given list of base URLs before exclusion. + * + * @param baseUrls The list of base URLs. + * @return The number of service locations before exclusion. + */ + public static int getServiceLocationCount(List baseUrls) { + Set serviceLocations = new HashSet<>(); + for (int i = 0; i < baseUrls.size(); i++) { + serviceLocations.add(baseUrls.get(i).serviceLocation); + } + return serviceLocations.size(); + } + /** * Returns the number of priority levels for the given list of base URLs after exclusion. * @@ -132,17 +187,24 @@ public int getPriorityCountAfterExclusion(List baseUrls) { } /** - * Returns the number of priority levels of the given list of base URLs. + * Returns the number of service locations for the given list of base URLs after exclusion. * * @param baseUrls The list of base URLs. - * @return The number of priority levels before exclusion. + * @return The number of service locations after exclusion. */ - public static int getPriorityCount(List baseUrls) { - Set priorities = new HashSet<>(); - for (int i = 0; i < baseUrls.size(); i++) { - priorities.add(baseUrls.get(i).priority); + public int getServiceLocationCountAfterExclusion(List baseUrls) { + Set serviceLocations = new HashSet<>(); + List includedBaseUrls = applyExclusions(baseUrls); + for (int i = 0; i < includedBaseUrls.size(); i++) { + serviceLocations.add(includedBaseUrls.get(i).serviceLocation); } - return priorities.size(); + return serviceLocations.size(); + } + + /** Updates the service location steering priority. */ + public void updateServiceLocationSteeringPriority( + @Nullable ImmutableList serviceLocationPriority) { + this.serviceLocationSteeringPriority = serviceLocationPriority; } /** Resets the state. */ @@ -150,10 +212,23 @@ public void reset() { excludedServiceLocations.clear(); excludedPriorities.clear(); selectionsTaken.clear(); + serviceLocationSteeringPriority = null; } // Internal methods. + private static List findMatchingBaseUrls( + String serviceLocation, List baseUrls) { + List matchingBaseUrls = new ArrayList<>(); + for (int i = 0; i < baseUrls.size(); i++) { + BaseUrl baseUrl = baseUrls.get(i); + if (Objects.equals(serviceLocation, baseUrl.serviceLocation)) { + matchingBaseUrls.add(baseUrl); + } + } + return matchingBaseUrls; + } + private List applyExclusions(List baseUrls) { long nowMs = SystemClock.elapsedRealtime(); removeExpiredExclusions(nowMs, excludedServiceLocations); diff --git a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashChunkSource.java b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashChunkSource.java index 52b28c3c783..ff8e363176b 100644 --- a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashChunkSource.java +++ b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashChunkSource.java @@ -25,13 +25,13 @@ import androidx.media3.datasource.TransferListener; import androidx.media3.exoplayer.analytics.PlayerId; import androidx.media3.exoplayer.dash.PlayerEmsgHandler.PlayerTrackEmsgHandler; +import androidx.media3.exoplayer.dash.manifest.BaseUrl; import androidx.media3.exoplayer.dash.manifest.DashManifest; import androidx.media3.exoplayer.source.chunk.ChunkSource; import androidx.media3.exoplayer.trackselection.ExoTrackSelection; import androidx.media3.exoplayer.upstream.CmcdConfiguration; import androidx.media3.exoplayer.upstream.LoaderErrorThrower; import androidx.media3.extractor.Extractor; -import androidx.media3.extractor.mp4.Mp4Extractor; import androidx.media3.extractor.text.SubtitleParser; import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.util.List; @@ -74,26 +74,6 @@ default Factory experimentalParseSubtitlesDuringExtraction( return this; } - /** - * Sets the set of video codecs for which within GOP sample dependency information should be - * parsed as part of extraction. Defaults to H.264 and H.265. - * - *

    Having access to additional sample dependency information can speed up seeking. See {@link - * Mp4Extractor#FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES}. - * - *

    This method is experimental and will be renamed or removed in a future release. - * - * @param codecsToParseWithinGopSampleDependencies The set of codecs for which to parse within - * GOP sample dependency information. - * @return This factory, for convenience. - */ - @CanIgnoreReturnValue - @ExperimentalApi // TODO: b/470365670 - Remove method once config is enabled by default. - default Factory experimentalSetCodecsToParseWithinGopSampleDependencies( - @C.VideoCodecFlags int codecsToParseWithinGopSampleDependencies) { - return this; - } - /** * @param manifestLoaderErrorThrower Throws errors affecting loading of manifests. * @param manifest The initial manifest. @@ -114,6 +94,8 @@ default Factory experimentalSetCodecsToParseWithinGopSampleDependencies( * May be null if no listener is available. * @param playerId The {@link PlayerId} of the player using this chunk source. * @param cmcdConfiguration The {@link CmcdConfiguration} for this chunk source. + * @param contentSteeringTracker The {@link DashContentSteeringTracker} to use for Content + * Steering, or {@code null} if Content Steering is unavailable. * @return The created {@link DashChunkSource}. */ DashChunkSource createDashChunkSource( @@ -130,7 +112,8 @@ DashChunkSource createDashChunkSource( @Nullable PlayerTrackEmsgHandler playerEmsgHandler, @Nullable TransferListener transferListener, PlayerId playerId, - @Nullable CmcdConfiguration cmcdConfiguration); + @Nullable CmcdConfiguration cmcdConfiguration, + @Nullable DashContentSteeringTracker contentSteeringTracker); /** * Returns the output {@link Format} of emitted {@linkplain C#TRACK_TYPE_TEXT text samples} @@ -167,4 +150,17 @@ default Format getOutputTextFormat(Format sourceFormat) { * @param trackSelection The new track selection instance. Must be equivalent to the previous one. */ void updateTrackSelection(ExoTrackSelection trackSelection); + + /** + * Returns the selected {@link BaseUrl}. + * + *

    The default implementation returns {@code null}. + * + * @return The selected {@link BaseUrl} of this {@link DashChunkSource}. Can be {@code null} if + * unknown or if no {@link BaseUrl} has been selected. + */ + @Nullable + default BaseUrl getSelectedBaseUrl() { + return null; + } } diff --git a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashContentSteeringTracker.java b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashContentSteeringTracker.java new file mode 100644 index 00000000000..bf2f476a312 --- /dev/null +++ b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashContentSteeringTracker.java @@ -0,0 +1,268 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.exoplayer.dash; + +import static com.google.common.base.Preconditions.checkNotNull; + +import androidx.annotation.Nullable; +import androidx.media3.common.util.UnstableApi; +import androidx.media3.datasource.DataSource; +import androidx.media3.datasource.TransferListener; +import androidx.media3.exoplayer.dash.manifest.AdaptationSet; +import androidx.media3.exoplayer.dash.manifest.BaseUrl; +import androidx.media3.exoplayer.dash.manifest.DashManifest; +import androidx.media3.exoplayer.dash.manifest.Location; +import androidx.media3.exoplayer.dash.manifest.Period; +import androidx.media3.exoplayer.dash.manifest.Representation; +import androidx.media3.exoplayer.upstream.contentsteering.BaseContentSteeringTracker; +import androidx.media3.exoplayer.upstream.contentsteering.SteeringManifest; +import androidx.media3.exoplayer.util.ReleasableExecutor; +import com.google.common.base.Supplier; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import java.util.ArrayList; +import java.util.HashMap; +import java.util.HashSet; +import java.util.Map; +import java.util.Objects; +import java.util.Set; + +/** Tracks the content steering states for a DASH stream. */ +@UnstableApi +public final class DashContentSteeringTracker extends BaseContentSteeringTracker { + + /** A callback to be notified of {@link DashContentSteeringTracker} events. */ + public interface Callback { + + /** + * Called when the service location priority is updated. + * + * @param serviceLocationPriority The updated service location priority, or {@code null} if the + * tracker becomes inactive. + */ + void onServiceLocationPriorityUpdated(@Nullable ImmutableList serviceLocationPriority); + } + + /** Provides the information for the query parameters to request the steering manifests. */ + public interface SteeringQueryParamsProvider { + + /** Returns the steered service locations observed by this provider. */ + ImmutableList getSteeredServiceLocations(); + } + + private static final String PATHWAY_PARAM = "_DASH_pathway"; + private static final String THROUGHPUT_PARAM = "_DASH_throughput"; + @Nullable private final Callback callback; + private final ServiceLocationFilteringThroughputEstimator throughputEstimator; + private final Set steeringQueryParamsProviders; + + @Nullable private DashManifest manifest; + private Map> availableManifestLocations; + private Map> availableBaseUrls; + @Nullable private ImmutableList initialPathwayIds; + + /** + * Creates a {@link DashContentSteeringTracker}. + * + * @param dataSourceFactory The {@link DataSource.Factory}. + * @param downloadExecutorSupplier A supplier for a {@link ReleasableExecutor}, or {@code null}. + * @param callback A {@link Callback}. + * @param mediaTransferListener The {@link TransferListener} to be used by the {@link DataSource} + * for media loading, which should be the {@link TransferListener} passed to the {@link + * DashMediaSource} when the source is prepared. + * @param manifest The {@link DashManifest}. + */ + public DashContentSteeringTracker( + DataSource.Factory dataSourceFactory, + @Nullable Supplier downloadExecutorSupplier, + @Nullable Callback callback, + @Nullable TransferListener mediaTransferListener, + DashManifest manifest) { + super(dataSourceFactory, downloadExecutorSupplier); + this.callback = callback; + this.throughputEstimator = + new ServiceLocationFilteringThroughputEstimator(mediaTransferListener); + this.steeringQueryParamsProviders = new HashSet<>(); + this.manifest = manifest; + availableManifestLocations = getAvailableManifestLocations(manifest); + availableBaseUrls = getAvailableBaseUrls(manifest); + } + + /** Updates the {@link DashManifest} for the stream. */ + public void updateManifest(DashManifest manifest) { + if (Objects.equals(manifest, this.manifest)) { + return; + } + this.manifest = manifest; + this.availableManifestLocations = getAvailableManifestLocations(manifest); + this.availableBaseUrls = getAvailableBaseUrls(manifest); + } + + /** Adds a {@link SteeringQueryParamsProvider} to the tracker. */ + public void addSteeringQueryParamsProvider(SteeringQueryParamsProvider provider) { + steeringQueryParamsProviders.add(provider); + } + + /** Removes a {@link SteeringQueryParamsProvider} from the tracker. */ + public void removeSteeringQueryParamsProvider(SteeringQueryParamsProvider provider) { + steeringQueryParamsProviders.remove(provider); + } + + /** + * Returns the {@link TransferListener} that should be used by the {@link DataSource} for media + * loading. + */ + public TransferListener getMediaTransferListener() { + return throughputEstimator; + } + + /** + * Returns the current service location priority, or {@code null} if the tracker is inactive or no + * priority has been established. + */ + @Nullable + public ImmutableList getCurrentServiceLocationPriority() { + if (!isActive()) { + return null; + } + @Nullable ImmutableList currentPathwayPriority = getCurrentPathwayPriority(); + if (currentPathwayPriority != null) { + return currentPathwayPriority; + } + return initialPathwayIds == null || initialPathwayIds.isEmpty() ? null : initialPathwayIds; + } + + @Override + protected void onStart(ImmutableList initialPathwayIds) { + this.initialPathwayIds = initialPathwayIds; + if (callback != null && !initialPathwayIds.isEmpty()) { + callback.onServiceLocationPriorityUpdated(initialPathwayIds); + } + } + + @Override + protected boolean isPathwayAvailable(String pathwayId) { + return availableManifestLocations.containsKey(pathwayId) + || availableBaseUrls.containsKey(pathwayId); + } + + @Override + protected ImmutableMap getSteeringQueryParameters() { + if (getCurrentPathwayPriority() == null) { + // We omit the _DASH_pathway and _DASH_throughput query parameters for the first steering + // manifest request. + return ImmutableMap.of(); + } + Map serviceLocationsToThroughputs = new HashMap<>(); + for (SteeringQueryParamsProvider provider : steeringQueryParamsProviders) { + ImmutableList steeredServiceLocations = provider.getSteeredServiceLocations(); + for (String steeredServiceLocation : steeredServiceLocations) { + if (serviceLocationsToThroughputs.containsKey(steeredServiceLocation)) { + continue; + } + long throughput = + checkNotNull(throughputEstimator).getThroughputEstimate(steeredServiceLocation); + serviceLocationsToThroughputs.put(steeredServiceLocation, throughput); + } + } + ArrayList steeredServiceLocations = new ArrayList<>(); + ArrayList throughputs = new ArrayList<>(); + boolean hasAnyThroughputEstimate = false; + for (Map.Entry entry : serviceLocationsToThroughputs.entrySet()) { + steeredServiceLocations.add(entry.getKey()); + long throughput = entry.getValue(); + if (throughput != 0L) { + hasAnyThroughputEstimate = true; + throughputs.add(String.valueOf(throughput)); + } else { + throughputs.add(""); + } + } + ImmutableMap.Builder queryParameters = ImmutableMap.builder(); + if (!steeredServiceLocations.isEmpty()) { + queryParameters.put(PATHWAY_PARAM, String.join(",", steeredServiceLocations)); + if (hasAnyThroughputEstimate) { + queryParameters.put(THROUGHPUT_PARAM, String.join(",", throughputs)); + } + } + return queryParameters.buildOrThrow(); + } + + @Override + protected void performPathwayEvaluation(ImmutableList pathwayPriority) { + if (callback != null) { + callback.onServiceLocationPriorityUpdated(pathwayPriority); + } + } + + @Override + protected void performPathwayClone(SteeringManifest.PathwayClone pathwayClone) { + // TODO: Add the support of Pathway Clone + } + + @Override + protected void onStop() { + manifest = null; + if (callback != null) { + callback.onServiceLocationPriorityUpdated(null); + } + initialPathwayIds = null; + availableManifestLocations.clear(); + availableBaseUrls.clear(); + steeringQueryParamsProviders.clear(); + throughputEstimator.stopThroughputEstimate(); + } + + private static Map> getAvailableBaseUrls(DashManifest manifest) { + Map> availableBaseUrls = new HashMap<>(); + for (int i = 0; i < manifest.getPeriodCount(); i++) { + Period period = manifest.getPeriod(i); + for (AdaptationSet adaptationSet : period.adaptationSets) { + for (Representation representation : adaptationSet.representations) { + for (BaseUrl baseUrl : representation.baseUrls) { + String serviceLocation = baseUrl.serviceLocation; + @Nullable + Set availableBaseUrlsForServiceLocation = + availableBaseUrls.get(serviceLocation); + if (availableBaseUrlsForServiceLocation == null) { + availableBaseUrlsForServiceLocation = new HashSet<>(); + availableBaseUrls.put(serviceLocation, availableBaseUrlsForServiceLocation); + } + availableBaseUrlsForServiceLocation.add(baseUrl); + } + } + } + } + return availableBaseUrls; + } + + private static Map> getAvailableManifestLocations(DashManifest manifest) { + Map> availableManifestLocations = new HashMap<>(); + for (int i = 0; i < manifest.locations.size(); i++) { + Location location = manifest.locations.get(i); + String serviceLocation = location.serviceLocation; + @Nullable + Set availableLocationsForServiceLocation = + availableManifestLocations.get(serviceLocation); + if (availableLocationsForServiceLocation == null) { + availableLocationsForServiceLocation = new HashSet<>(); + availableManifestLocations.put(serviceLocation, availableLocationsForServiceLocation); + } + availableLocationsForServiceLocation.add(location); + } + return availableManifestLocations; + } +} diff --git a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashMediaPeriod.java b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashMediaPeriod.java index 21d17a23e99..f8aad2ff878 100644 --- a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashMediaPeriod.java +++ b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashMediaPeriod.java @@ -15,6 +15,7 @@ */ package androidx.media3.exoplayer.dash; +import static com.google.common.base.Preconditions.checkNotNull; import static java.lang.Math.min; import static java.lang.annotation.ElementType.TYPE_USE; @@ -36,6 +37,7 @@ import androidx.media3.exoplayer.dash.PlayerEmsgHandler.PlayerEmsgCallback; import androidx.media3.exoplayer.dash.PlayerEmsgHandler.PlayerTrackEmsgHandler; import androidx.media3.exoplayer.dash.manifest.AdaptationSet; +import androidx.media3.exoplayer.dash.manifest.BaseUrl; import androidx.media3.exoplayer.dash.manifest.DashManifest; import androidx.media3.exoplayer.dash.manifest.Descriptor; import androidx.media3.exoplayer.dash.manifest.EventStream; @@ -48,6 +50,7 @@ import androidx.media3.exoplayer.source.MediaPeriod; import androidx.media3.exoplayer.source.MediaSourceEventListener; import androidx.media3.exoplayer.source.MediaSourceEventListener.EventDispatcher; +import androidx.media3.exoplayer.source.SampleQueue; import androidx.media3.exoplayer.source.SampleStream; import androidx.media3.exoplayer.source.SequenceableLoader; import androidx.media3.exoplayer.source.TrackGroupArray; @@ -73,6 +76,7 @@ import java.util.ArrayList; import java.util.Arrays; import java.util.HashMap; +import java.util.HashSet; import java.util.IdentityHashMap; import java.util.List; import java.util.Objects; @@ -112,8 +116,13 @@ private final DrmSessionEventListener.EventDispatcher drmEventDispatcher; private final PlayerId playerId; @Nullable private final Supplier downloadExecutorSupplier; + @Nullable private final DashContentSteeringTracker contentSteeringTracker; @Nullable private Callback callback; + + @Nullable + private DashContentSteeringTracker.SteeringQueryParamsProvider steeringQueryParamsProvider; + private ChunkSampleStream[] sampleStreams; private EventSampleStream[] eventSampleStreams; private SequenceableLoader compositeSequenceableLoader; @@ -144,7 +153,8 @@ public DashMediaPeriod( CompositeSequenceableLoaderFactory compositeSequenceableLoaderFactory, PlayerEmsgCallback playerEmsgCallback, PlayerId playerId, - @Nullable Supplier downloadExecutorSupplier) { + @Nullable Supplier downloadExecutorSupplier, + @Nullable DashContentSteeringTracker contentSteeringTracker) { this.id = id; this.manifest = manifest; this.baseUrlExclusionList = baseUrlExclusionList; @@ -162,7 +172,12 @@ public DashMediaPeriod( this.compositeSequenceableLoaderFactory = compositeSequenceableLoaderFactory; this.playerId = playerId; this.downloadExecutorSupplier = downloadExecutorSupplier; + this.contentSteeringTracker = contentSteeringTracker; this.canReportInitialDiscontinuity = true; + if (contentSteeringTracker != null && contentSteeringTracker.isActive()) { + this.steeringQueryParamsProvider = new SteeringQueryParamsProvider(); + contentSteeringTracker.addSteeringQueryParamsProvider(this.steeringQueryParamsProvider); + } playerEmsgHandler = new PlayerEmsgHandler(manifest, playerEmsgCallback, allocator); sampleStreams = newSampleStreamArray(0); eventSampleStreams = new EventSampleStream[0]; @@ -214,6 +229,11 @@ public void release() { sampleStream.release(this); } callback = null; + if (this.steeringQueryParamsProvider != null) { + checkNotNull(contentSteeringTracker) + .removeSteeringQueryParamsProvider(this.steeringQueryParamsProvider); + this.steeringQueryParamsProvider = null; + } } // ChunkSampleStream.ReleaseCallback implementation. @@ -665,8 +685,10 @@ private static int[][] getGroupedAdaptationSetIndices(List adapta trickPlayProperty = findTrickPlayProperty(adaptationSet.supplementalProperties); } if (trickPlayProperty != null) { - long mainAdaptationSetId = Long.parseLong(trickPlayProperty.value); - @Nullable Integer mainAdaptationSetIndex = adaptationSetIdToIndex.get(mainAdaptationSetId); + String mainAdaptationSetId = Util.splitAtFirst(trickPlayProperty.value, "\\s+")[0]; + @Nullable + Integer mainAdaptationSetIndex = + adaptationSetIdToIndex.get(Long.parseLong(mainAdaptationSetId)); if (mainAdaptationSetIndex != null && canMergeAdaptationSets(adaptationSet, adaptationSets.get(mainAdaptationSetIndex))) { mergedGroupIndex = mainAdaptationSetIndex; @@ -927,7 +949,7 @@ private ChunkSampleStream buildSampleStream( positionUs, manifest, periodIndex, trackGroupInfo.adaptationSetIndices); boolean handleInitialDiscontinuity = canReportInitialDiscontinuity - && !areAllSamplesSyncSamples( + && !isDiscardingAllSamplesToStartTime( manifest, periodIndex, trackGroupInfo.adaptationSetIndices, selection); DashChunkSource chunkSource = chunkSourceFactory.createDashChunkSource( @@ -944,7 +966,8 @@ private ChunkSampleStream buildSampleStream( trackPlayerEmsgHandler, transferListener, playerId, - cmcdConfiguration); + cmcdConfiguration, + contentSteeringTracker); ChunkSampleStream stream = new ChunkSampleStream<>( trackGroupInfo.trackType, @@ -970,25 +993,25 @@ private ChunkSampleStream buildSampleStream( } private static long getFirstChunkStartTimeUs( - long positionsUs, DashManifest manifest, int periodIndex, int[] adaptationSetIndices) { - // Use the first representation as all representations in one adaptation set (or in grouped - // adaptation sets) need to be segment-aligned. - Representation representation = - manifest - .getPeriod(periodIndex) - .adaptationSets - .get(adaptationSetIndices[0]) - .representations - .get(0); - @Nullable DashSegmentIndex segmentIndex = representation.getIndex(); - if (segmentIndex == null) { - return C.TIME_UNSET; - } + long positionUs, DashManifest manifest, int periodIndex, int[] adaptationSetIndices) { long periodDurationUs = manifest.getPeriodDurationUs(periodIndex); - return segmentIndex.getTimeUs(segmentIndex.getSegmentNum(positionsUs, periodDurationUs)); + List adaptationSets = manifest.getPeriod(periodIndex).adaptationSets; + // Use the first non-empty representation, as all representations in one adaptation set (or in + // grouped adaptation sets) need to be segment-aligned. + for (int i = 0; i < adaptationSetIndices.length; i++) { + int adaptationSetIndex = adaptationSetIndices[i]; + List representations = adaptationSets.get(adaptationSetIndex).representations; + for (int j = 0; j < representations.size(); j++) { + @Nullable DashSegmentIndex segmentIndex = representations.get(j).getIndex(); + if (segmentIndex != null && segmentIndex.getSegmentCount(periodDurationUs) != 0) { + return segmentIndex.getTimeUs(segmentIndex.getSegmentNum(positionUs, periodDurationUs)); + } + } + } + return C.TIME_UNSET; } - private static boolean areAllSamplesSyncSamples( + private static boolean isDiscardingAllSamplesToStartTime( DashManifest manifest, int periodIndex, int[] adaptationSetIndices, @@ -1001,8 +1024,7 @@ private static boolean areAllSamplesSyncSamples( ImmutableList representations = representationsBuilder.build(); for (int i = 0; i < trackSelection.length(); i++) { Representation representation = representations.get(trackSelection.getIndexInTrackGroup(i)); - if (!MimeTypes.allSamplesAreSyncSamples( - representation.format.sampleMimeType, representation.format.codecs)) { + if (!SampleQueue.isDiscardingAllSamplesToStartTime(representation.format)) { return false; } } @@ -1117,6 +1139,26 @@ private static ChunkSampleStream[] newSampleStreamArray(int len return new ChunkSampleStream[length]; } + private class SteeringQueryParamsProvider + implements DashContentSteeringTracker.SteeringQueryParamsProvider { + + @Override + public ImmutableList getSteeredServiceLocations() { + if (contentSteeringTracker == null || !contentSteeringTracker.isActive()) { + return ImmutableList.of(); + } + HashSet steeredServiceLocations = new HashSet<>(); + for (ChunkSampleStream sampleStream : sampleStreams) { + DashChunkSource chunkSource = sampleStream.getChunkSource(); + @Nullable BaseUrl baseUrl = chunkSource.getSelectedBaseUrl(); + if (baseUrl != null) { + steeredServiceLocations.add(baseUrl.serviceLocation); + } + } + return ImmutableList.copyOf(steeredServiceLocations); + } + } + private static final class TrackGroupInfo { @Documented diff --git a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashMediaSource.java b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashMediaSource.java index 0a4f59524dc..3f35941a54c 100644 --- a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashMediaSource.java +++ b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashMediaSource.java @@ -50,10 +50,12 @@ import androidx.media3.datasource.TransferListener; import androidx.media3.exoplayer.dash.PlayerEmsgHandler.PlayerEmsgCallback; import androidx.media3.exoplayer.dash.manifest.AdaptationSet; +import androidx.media3.exoplayer.dash.manifest.ContentSteering; import androidx.media3.exoplayer.dash.manifest.DashManifest; import androidx.media3.exoplayer.dash.manifest.DashManifestParser; import androidx.media3.exoplayer.dash.manifest.Location; import androidx.media3.exoplayer.dash.manifest.Period; +import androidx.media3.exoplayer.dash.manifest.ProducerReferenceTime; import androidx.media3.exoplayer.dash.manifest.Representation; import androidx.media3.exoplayer.dash.manifest.UtcTimingElement; import androidx.media3.exoplayer.drm.DefaultDrmSessionManagerProvider; @@ -97,6 +99,7 @@ import java.nio.charset.StandardCharsets; import java.text.ParseException; import java.text.SimpleDateFormat; +import java.util.ArrayList; import java.util.HashMap; import java.util.Iterator; import java.util.List; @@ -106,6 +109,7 @@ import java.util.TimeZone; import java.util.regex.Matcher; import java.util.regex.Pattern; +import org.checkerframework.checker.nullness.qual.EnsuresNonNullIf; /** A DASH {@link MediaSource}. */ @SuppressWarnings("nullness") // TODO: b/78934030 - Add missing nullness checks to this class. @@ -230,15 +234,6 @@ public Factory experimentalParseSubtitlesDuringExtraction( return this; } - @Override - @CanIgnoreReturnValue - public Factory experimentalSetCodecsToParseWithinGopSampleDependencies( - @C.VideoCodecFlags int codecsToParseWithinGopSampleDependencies) { - chunkSourceFactory.experimentalSetCodecsToParseWithinGopSampleDependencies( - codecsToParseWithinGopSampleDependencies); - return this; - } - /** * Sets the target {@link Player#getCurrentLiveOffset() offset for live streams} that is used if * no value is defined in the {@link MediaItem} or the manifest. @@ -462,7 +457,7 @@ public DashMediaSource createMediaSource(MediaItem mediaItem) { @Nullable private final Supplier downloadExecutorSupplier; @GuardedBy("manifestLocationsLock") - private final Map excludedServiceLocations; + private final Map excludedManifestServiceLocations; @GuardedBy("manifestLocationsLock") private ImmutableList manifestLocations; @@ -474,6 +469,13 @@ public DashMediaSource createMediaSource(MediaItem mediaItem) { @GuardedBy("manifestLocationsLock") private Uri initialManifestUri; + @GuardedBy("manifestLocationsLock") + @Nullable + private ImmutableList serviceLocationSteeringPriority; + + @Nullable private DashContentSteeringTracker contentSteeringTracker; + @Nullable private SteeringQueryParamsProvider steeringQueryParamsProvider; + private DataSource dataSource; private Loader loader; @Nullable private TransferListener mediaTransferListener; @@ -529,7 +531,7 @@ private DashMediaSource( manifestEventDispatcher = createEventDispatcher(/* mediaPeriodId= */ null); manifestLocationsLock = new Object(); manifestLocations = ImmutableList.of(new Location(initialManifestUri)); - excludedServiceLocations = new HashMap<>(); + excludedManifestServiceLocations = new HashMap<>(); periodsById = new SparseArray<>(); playerEmsgCallback = new DefaultPlayerEmsgCallback(); expiredManifestPublishTimeUs = C.TIME_UNSET; @@ -558,7 +560,7 @@ public void replaceManifestUri(Uri manifestUri) { this.initialManifestUri = manifestUri; this.currentManifestLocation = new Location(manifestUri); this.manifestLocations = ImmutableList.of(this.currentManifestLocation); - this.excludedServiceLocations.clear(); + this.excludedManifestServiceLocations.clear(); } } @@ -623,7 +625,9 @@ public MediaPeriod createPeriod(MediaPeriodId id, Allocator allocator, long star baseUrlExclusionList, periodIndex, chunkSourceFactory, - mediaTransferListener, + isContentSteeringActive() + ? contentSteeringTracker.getMediaTransferListener() + : mediaTransferListener, cmcdConfiguration, drmSessionManager, drmEventDispatcher, @@ -635,7 +639,8 @@ public MediaPeriod createPeriod(MediaPeriodId id, Allocator allocator, long star compositeSequenceableLoaderFactory, playerEmsgCallback, getPlayerId(), - downloadExecutorSupplier); + downloadExecutorSupplier, + contentSteeringTracker); periodsById.put(mediaPeriod.id, mediaPeriod); return mediaPeriod; } @@ -658,11 +663,20 @@ protected void releaseSourceInternal() { setLiveConfiguration(getMediaItem().liveConfiguration); manifestLoadStartTimestampMs = 0; manifestLoadEndTimestampMs = 0; + if (contentSteeringTracker != null) { + contentSteeringTracker.removeSteeringQueryParamsProvider( + checkNotNull(steeringQueryParamsProvider)); + contentSteeringTracker.release(); + contentSteeringTracker = null; + steeringQueryParamsProvider = null; + } synchronized (manifestLocationsLock) { manifestLocations = ImmutableList.of(new Location(initialManifestUri)); - excludedServiceLocations.clear(); + excludedManifestServiceLocations.clear(); currentManifestLocation = null; + serviceLocationSteeringPriority = null; } + mediaTransferListener = null; manifestFatalError = null; if (handler != null) { handler.removeCallbacksAndMessages(null); @@ -676,6 +690,39 @@ protected void releaseSourceInternal() { drmSessionManager.release(); } + private class ContentSteeringCallback implements DashContentSteeringTracker.Callback { + + @Override + public void onServiceLocationPriorityUpdated( + @Nullable ImmutableList serviceLocationPriority) { + synchronized (manifestLocationsLock) { + serviceLocationSteeringPriority = serviceLocationPriority; + Location unused = selectManifestLocation(); + } + baseUrlExclusionList.updateServiceLocationSteeringPriority(serviceLocationPriority); + } + } + + private class SteeringQueryParamsProvider + implements DashContentSteeringTracker.SteeringQueryParamsProvider { + + @Override + public ImmutableList getSteeredServiceLocations() { + synchronized (manifestLocationsLock) { + if (contentSteeringTracker == null || !contentSteeringTracker.isActive()) { + return ImmutableList.of(); + } + if (currentManifestLocation != null + && serviceLocationSteeringPriority != null + && serviceLocationSteeringPriority.contains(currentManifestLocation.serviceLocation)) { + String serviceLocation = currentManifestLocation.serviceLocation; + return ImmutableList.of(serviceLocation); + } + return ImmutableList.of(); + } + } + } + // PlayerEmsgCallback callbacks. /* package */ void onDashManifestRefreshRequested() { @@ -697,31 +744,27 @@ protected void releaseSourceInternal() { long elapsedRealtimeMs, long loadDurationMs, int retryCount) { - LoadEventInfo loadEventInfo = - retryCount == 0 - ? new LoadEventInfo(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) - : new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); - manifestEventDispatcher.loadStarted(loadEventInfo, loadable.type, retryCount); + LoadEventInfo.Builder loadEventInfo = + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs); + if (retryCount != 0) { + loadEventInfo + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()); + } + manifestEventDispatcher.loadStarted(loadEventInfo.build(), loadable.type, retryCount); } /* package */ void onManifestLoadCompleted( ParsingLoadable loadable, long elapsedRealtimeMs, long loadDurationMs) { LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(); loadErrorHandlingPolicy.onLoadTaskConcluded(loadable.loadTaskId); manifestEventDispatcher.loadCompleted(loadEventInfo, loadable.type); DashManifest newManifest = loadable.getResult(); @@ -804,7 +847,7 @@ protected void releaseSourceInternal() { } } } - + maybeInitializeContentSteeringTracker(manifest); if (manifest.dynamic && elapsedRealtimeOffsetMs == C.TIME_UNSET) { // Determine elapsedRealtimeOffsetMs before processing the manifest further. if (manifest.utcTiming != null) { @@ -824,14 +867,12 @@ protected void releaseSourceInternal() { IOException error, int errorCount) { LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(); MediaLoadData mediaLoadData = new MediaLoadData(loadable.type); LoadErrorInfo loadErrorInfo = new LoadErrorInfo(loadEventInfo, mediaLoadData, error, errorCount); @@ -843,7 +884,7 @@ protected void releaseSourceInternal() { numberOfLocations = manifestLocations.size(); numberOfExcludedLocations = 0; for (Location location : manifestLocations) { - Long exclusionEndMs = excludedServiceLocations.get(location.serviceLocation); + Long exclusionEndMs = excludedManifestServiceLocations.get(location.serviceLocation); if (exclusionEndMs != null && exclusionEndMs > elapsedRealtimeMs) { numberOfExcludedLocations++; } @@ -855,7 +896,7 @@ protected void releaseSourceInternal() { numberOfExcludedLocations, /* numberOfTracks= */ 1, /* numberOfExcludedTracks= */ 0, - /* locationSteeringActive= */ false); + /* locationSteeringActive= */ isContentSteeringActive()); @Nullable LoadErrorHandlingPolicy.FallbackSelection fallbackSelection = loadErrorHandlingPolicy.getFallbackSelectionFor(fallbackOptions, loadErrorInfo); @@ -878,7 +919,7 @@ protected void releaseSourceInternal() { } if (failingLocation != null) { long excludeUntilMs = elapsedRealtimeMs + fallbackSelection.exclusionDurationMs; - excludedServiceLocations.put(failingLocation.serviceLocation, excludeUntilMs); + excludedManifestServiceLocations.put(failingLocation.serviceLocation, excludeUntilMs); } } if (failingLocation != null) { @@ -891,7 +932,10 @@ protected void releaseSourceInternal() { } // If fallback is not available or rejected, fall back to retry delay or fatal error. - long retryDelayMs = loadErrorHandlingPolicy.getRetryDelayMsFor(loadErrorInfo); + long retryDelayMs = + errorCount > loadErrorHandlingPolicy.getMinimumLoadableRetryCount(loadable.type) + ? C.TIME_UNSET + : loadErrorHandlingPolicy.getRetryDelayMsFor(loadErrorInfo); LoadErrorAction loadErrorAction = retryDelayMs == C.TIME_UNSET ? Loader.DONT_RETRY_FATAL @@ -907,14 +951,12 @@ protected void releaseSourceInternal() { /* package */ void onUtcTimestampLoadCompleted( ParsingLoadable loadable, long elapsedRealtimeMs, long loadDurationMs) { LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(); loadErrorHandlingPolicy.onLoadTaskConcluded(loadable.loadTaskId); manifestEventDispatcher.loadCompleted(loadEventInfo, loadable.type); onUtcTimestampResolved(loadable.getResult() - elapsedRealtimeMs); @@ -926,14 +968,12 @@ protected void releaseSourceInternal() { long loadDurationMs, IOException error) { manifestEventDispatcher.loadError( - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()), + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(), loadable.type, error, /* wasCanceled= */ true); @@ -945,14 +985,12 @@ protected void releaseSourceInternal() { /* package */ void onLoadCanceled( ParsingLoadable loadable, long elapsedRealtimeMs, long loadDurationMs) { LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(); loadErrorHandlingPolicy.onLoadTaskConcluded(loadable.loadTaskId); manifestEventDispatcher.loadCanceled(loadEventInfo, loadable.type); } @@ -991,8 +1029,12 @@ private void resolveUtcTimingElementDirect(UtcTimingElement timingElement) { private void resolveUtcTimingElementHttp( UtcTimingElement timingElement, ParsingLoadable.Parser parser) { startLoading( - new ParsingLoadable<>( - dataSource, Uri.parse(timingElement.value), C.DATA_TYPE_TIME_SYNCHRONIZATION, parser), + new ParsingLoadable.Builder<>( + dataSource, + Uri.parse(timingElement.value), + C.DATA_TYPE_TIME_SYNCHRONIZATION, + parser) + .build(), new UtcTimestampCallback(), 1); } @@ -1055,7 +1097,14 @@ private void processManifest(boolean scheduleRefresh) { max(windowStartTimeInManifestUs, timeShiftBufferStartTimeInManifestUs); } long windowDurationUs = windowEndTimeInManifestUs - windowStartTimeInManifestUs; - long windowStartUnixTimeMs = C.TIME_UNSET; + long presentationStartTimeMs = manifest.availabilityStartTimeMs; + if (!manifest.dynamic && presentationStartTimeMs == C.TIME_UNSET) { + presentationStartTimeMs = getPresentationStartTimeMsFromProducerReferenceTime(manifest); + } + long windowStartUnixTimeMs = + presentationStartTimeMs != C.TIME_UNSET + ? presentationStartTimeMs + Util.usToMs(windowStartTimeInManifestUs) + : C.TIME_UNSET; long windowDefaultPositionUs = 0; if (manifest.dynamic) { checkState(manifest.availabilityStartTimeMs != C.TIME_UNSET); @@ -1064,8 +1113,6 @@ private void processManifest(boolean scheduleRefresh) { - Util.msToUs(manifest.availabilityStartTimeMs) - windowStartTimeInManifestUs; updateLiveConfiguration(nowInWindowUs, windowDurationUs); - windowStartUnixTimeMs = - manifest.availabilityStartTimeMs + Util.usToMs(windowStartTimeInManifestUs); windowDefaultPositionUs = nowInWindowUs - Util.msToUs(getLiveConfiguration().targetOffsetMs); long minimumWindowDefaultPositionUs = min(minLiveStartPositionUs, windowDurationUs / 2); if (windowDefaultPositionUs < minimumWindowDefaultPositionUs) { @@ -1078,7 +1125,7 @@ private void processManifest(boolean scheduleRefresh) { long offsetInFirstPeriodUs = windowStartTimeInManifestUs - Util.msToUs(firstPeriod.startMs); DashTimeline timeline = new DashTimeline( - manifest.availabilityStartTimeMs, + presentationStartTimeMs, windowStartUnixTimeMs, elapsedRealtimeOffsetMs, firstPeriodId, @@ -1089,7 +1136,9 @@ private void processManifest(boolean scheduleRefresh) { getMediaItem(), manifest.dynamic ? getLiveConfiguration() : null); refreshSourceInfo(timeline); - + if (isContentSteeringActive()) { + contentSteeringTracker.updateManifest(manifest); + } if (!sideloadedManifest) { // Remove any pending simulated refresh. handler.removeCallbacks(simulateManifestRefreshRunnable); @@ -1121,6 +1170,26 @@ private void processManifest(boolean scheduleRefresh) { } } + private void maybeInitializeContentSteeringTracker(DashManifest manifest) { + if (contentSteeringTracker != null || manifest.contentSteering == null) { + return; + } + contentSteeringTracker = + new DashContentSteeringTracker( + manifestDataSourceFactory, + downloadExecutorSupplier, + new ContentSteeringCallback(), + mediaTransferListener, + manifest); + ContentSteering contentSteering = manifest.contentSteering; + steeringQueryParamsProvider = new SteeringQueryParamsProvider(); + contentSteeringTracker.addSteeringQueryParamsProvider(steeringQueryParamsProvider); + contentSteeringTracker.start( + contentSteering.steeringServerUri, + contentSteering.defaultServiceLocation, + manifestEventDispatcher); + } + private void updateLiveConfiguration(long nowInWindowUs, long windowDurationUs) { MediaItem.LiveConfiguration mediaItemLiveConfiguration = getMediaItem().liveConfiguration; // Default maximum offset: start of window. @@ -1235,7 +1304,8 @@ private void scheduleManifestRefresh(long delayUntilNextLoadMs) { private Location selectManifestLocation() { long nowMs = SystemClock.elapsedRealtime(); // Clear expired exclusions - Iterator> iterator = excludedServiceLocations.entrySet().iterator(); + Iterator> iterator = + excludedManifestServiceLocations.entrySet().iterator(); while (iterator.hasNext()) { Map.Entry entry = iterator.next(); if (entry.getValue() <= nowMs) { @@ -1243,27 +1313,62 @@ private Location selectManifestLocation() { } } + Location selectedLocation = + serviceLocationSteeringPriority != null + ? selectManifestLocationWithSteeringPriority(serviceLocationSteeringPriority) + : selectManifestLocationDefault(); + currentManifestLocation = selectedLocation; + return selectedLocation; + } + + @GuardedBy("manifestLocationsLock") + private Location selectManifestLocationWithSteeringPriority( + ImmutableList steeringPriority) { + List candidates = new ArrayList<>(); + for (int i = 0; i < steeringPriority.size(); i++) { + // Iterate through the steering priority list from high to low. + String serviceLocation = steeringPriority.get(i); + List matchingLocations = findMatchingManifestLocations(serviceLocation); + candidates.addAll(matchingLocations); + } + if (!candidates.isEmpty()) { + // The candidates list is ordered by the serviceLocation's priority, so we should be either + // select the non-excluded location that has the highest priority, or the one has the + // earliest exclusion timeout, if all candidates are currently excluded. + return selectFirstNonExcludedOrEarliestExclusionTimeout(candidates); + } + // If the candidates list is empty, it is possible that the steering priority list is empty (the + // content steering tracker just started without default service location), or no available + // manifest locations matching any service locations on the list, then we fallback to select + // with the default logic. + return selectManifestLocationDefault(); + } + + @GuardedBy("manifestLocationsLock") + private Location selectManifestLocationDefault() { // If the current location's serviceLocation matches one of the manifest locations and is not // excluded, stay on it. if (currentManifestLocation != null - && !excludedServiceLocations.containsKey(currentManifestLocation.serviceLocation)) { + && !excludedManifestServiceLocations.containsKey(currentManifestLocation.serviceLocation)) { for (int i = 0; i < manifestLocations.size(); i++) { Location location = manifestLocations.get(i); if (location.serviceLocation.equals(currentManifestLocation.serviceLocation)) { - currentManifestLocation = location; return location; } } } + return selectFirstNonExcludedOrEarliestExclusionTimeout(manifestLocations); + } - // Select the first non-excluded location, or track the one with the earliest exclusion timeout. + @GuardedBy("manifestLocationsLock") + private Location selectFirstNonExcludedOrEarliestExclusionTimeout(List candidates) { + checkState(!candidates.isEmpty()); long earliestTimeoutMs = Long.MAX_VALUE; - Location locationWithEarliestExclusionTimeout = checkNotNull(manifestLocations.get(0)); - for (int i = 0; i < manifestLocations.size(); i++) { - Location location = manifestLocations.get(i); - @Nullable Long timeoutMs = excludedServiceLocations.get(location.serviceLocation); + Location locationWithEarliestExclusionTimeout = candidates.get(0); + for (int i = 0; i < candidates.size(); i++) { + Location location = candidates.get(i); + @Nullable Long timeoutMs = excludedManifestServiceLocations.get(location.serviceLocation); if (timeoutMs == null) { - currentManifestLocation = location; return location; } if (timeoutMs < earliestTimeoutMs) { @@ -1271,10 +1376,21 @@ private Location selectManifestLocation() { locationWithEarliestExclusionTimeout = location; } } - currentManifestLocation = locationWithEarliestExclusionTimeout; return locationWithEarliestExclusionTimeout; } + @GuardedBy("manifestLocationsLock") + private List findMatchingManifestLocations(String serviceLocation) { + List matchingLocations = new ArrayList<>(); + for (int i = 0; i < manifestLocations.size(); i++) { + Location location = manifestLocations.get(i); + if (Objects.equals(location.serviceLocation, serviceLocation)) { + matchingLocations.add(location); + } + } + return matchingLocations; + } + private void startLoadingManifest() { handler.removeCallbacks(refreshManifestRunnable); if (loader.hasFatalError()) { @@ -1302,7 +1418,9 @@ private void startLoadingManifest() { dataSpec = cmcdDataFactory.createCmcdData().addToDataSpec(dataSpec); } startLoading( - new ParsingLoadable<>(dataSource, dataSpec, C.DATA_TYPE_MANIFEST, manifestParser), + new ParsingLoadable.Builder( + dataSource, dataSpec, C.DATA_TYPE_MANIFEST, manifestParser) + .build(), manifestCallback, loadErrorHandlingPolicy.getMinimumLoadableRetryCount(C.DATA_TYPE_MANIFEST)); } @@ -1376,33 +1494,58 @@ private static long getAvailableStartTimeInManifestUs( boolean haveAudioVideoAdaptationSets = hasVideoOrAudioAdaptationSets(period); for (int i = 0; i < period.adaptationSets.size(); i++) { AdaptationSet adaptationSet = period.adaptationSets.get(i); - List representations = adaptationSet.representations; // Exclude other adaptation sets from duration calculations, if we have at least one audio or // video adaptation set. See: https://github.com/google/ExoPlayer/issues/4029. boolean adaptationSetIsNotAudioVideo = adaptationSet.type != C.TRACK_TYPE_AUDIO && adaptationSet.type != C.TRACK_TYPE_VIDEO; if ((haveAudioVideoAdaptationSets && adaptationSetIsNotAudioVideo) - || representations.isEmpty()) { + || adaptationSet.representations.isEmpty()) { continue; } - @Nullable DashSegmentIndex index = representations.get(0).getIndex(); - if (index == null) { - return periodStartTimeInManifestUs; - } - long availableSegmentCount = index.getAvailableSegmentCount(periodDurationUs, nowUnixTimeUs); - if (availableSegmentCount == 0) { - return periodStartTimeInManifestUs; - } - long firstAvailableSegmentNum = - index.getFirstAvailableSegmentNum(periodDurationUs, nowUnixTimeUs); - long adaptationSetAvailableStartTimeInManifestUs = - periodStartTimeInManifestUs + index.getTimeUs(firstAvailableSegmentNum); + long adaptationSetStartTimeUs = + getAvailableStartTimeInAdaptationSetUs( + adaptationSet, periodStartTimeInManifestUs, periodDurationUs, nowUnixTimeUs); availableStartTimeInManifestUs = - max(availableStartTimeInManifestUs, adaptationSetAvailableStartTimeInManifestUs); + max(availableStartTimeInManifestUs, adaptationSetStartTimeUs); } return availableStartTimeInManifestUs; } + private static long getAvailableStartTimeInAdaptationSetUs( + AdaptationSet adaptationSet, + long periodStartTimeInManifestUs, + long periodDurationUs, + long nowUnixTimeUs) { + long availableStartTimeInAdaptationSetUs = periodStartTimeInManifestUs; + List representations = adaptationSet.representations; + for (int i = 0; i < representations.size(); i++) { + long representationStartTimeUs = + getAvailableStartTimeInRepresentationUs( + representations.get(i), periodStartTimeInManifestUs, periodDurationUs, nowUnixTimeUs); + availableStartTimeInAdaptationSetUs = + max(availableStartTimeInAdaptationSetUs, representationStartTimeUs); + } + return availableStartTimeInAdaptationSetUs; + } + + private static long getAvailableStartTimeInRepresentationUs( + Representation representation, + long periodStartTimeInManifestUs, + long periodDurationUs, + long nowUnixTimeUs) { + @Nullable DashSegmentIndex index = representation.getIndex(); + if (index == null) { + return periodStartTimeInManifestUs; + } + long availableSegmentCount = index.getAvailableSegmentCount(periodDurationUs, nowUnixTimeUs); + if (availableSegmentCount == 0) { + return periodStartTimeInManifestUs; + } + long firstAvailableSegmentNum = + index.getFirstAvailableSegmentNum(periodDurationUs, nowUnixTimeUs); + return periodStartTimeInManifestUs + index.getTimeUs(firstAvailableSegmentNum); + } + private static long getAvailableEndTimeInManifestUs( Period period, long periodDurationUs, long nowUnixTimeUs) { long periodStartTimeInManifestUs = Util.msToUs(period.startMs); @@ -1410,36 +1553,60 @@ private static long getAvailableEndTimeInManifestUs( boolean haveAudioVideoAdaptationSets = hasVideoOrAudioAdaptationSets(period); for (int i = 0; i < period.adaptationSets.size(); i++) { AdaptationSet adaptationSet = period.adaptationSets.get(i); - List representations = adaptationSet.representations; // Exclude other adaptation sets from duration calculations, if we have at least one audio or // video adaptation set. See: https://github.com/google/ExoPlayer/issues/4029 boolean adaptationSetIsNotAudioVideo = adaptationSet.type != C.TRACK_TYPE_AUDIO && adaptationSet.type != C.TRACK_TYPE_VIDEO; if ((haveAudioVideoAdaptationSets && adaptationSetIsNotAudioVideo) - || representations.isEmpty()) { + || adaptationSet.representations.isEmpty()) { continue; } - @Nullable DashSegmentIndex index = representations.get(0).getIndex(); - if (index == null) { - return periodStartTimeInManifestUs + periodDurationUs; - } - long availableSegmentCount = index.getAvailableSegmentCount(periodDurationUs, nowUnixTimeUs); - if (availableSegmentCount == 0) { - return periodStartTimeInManifestUs; - } - long firstAvailableSegmentNum = - index.getFirstAvailableSegmentNum(periodDurationUs, nowUnixTimeUs); - long lastAvailableSegmentNum = firstAvailableSegmentNum + availableSegmentCount - 1; - long adaptationSetAvailableEndTimeInManifestUs = - periodStartTimeInManifestUs - + index.getTimeUs(lastAvailableSegmentNum) - + index.getDurationUs(lastAvailableSegmentNum, periodDurationUs); - availableEndTimeInManifestUs = - min(availableEndTimeInManifestUs, adaptationSetAvailableEndTimeInManifestUs); + long adaptationSetEndTimeUs = + getAvailableEndTimeInAdaptationSetUs( + adaptationSet, periodStartTimeInManifestUs, periodDurationUs, nowUnixTimeUs); + availableEndTimeInManifestUs = min(availableEndTimeInManifestUs, adaptationSetEndTimeUs); } return availableEndTimeInManifestUs; } + private static long getAvailableEndTimeInAdaptationSetUs( + AdaptationSet adaptationSet, + long periodStartTimeInManifestUs, + long periodDurationUs, + long nowUnixTimeUs) { + long availableEndTimeInAdaptationSetUs = Long.MAX_VALUE; + List representations = adaptationSet.representations; + for (int i = 0; i < representations.size(); i++) { + long representationEndTimeUs = + getAvailableEndTimeInRepresentationUs( + representations.get(i), periodStartTimeInManifestUs, periodDurationUs, nowUnixTimeUs); + availableEndTimeInAdaptationSetUs = + min(availableEndTimeInAdaptationSetUs, representationEndTimeUs); + } + return availableEndTimeInAdaptationSetUs; + } + + private static long getAvailableEndTimeInRepresentationUs( + Representation representation, + long periodStartTimeInManifestUs, + long periodDurationUs, + long nowUnixTimeUs) { + @Nullable DashSegmentIndex index = representation.getIndex(); + if (index == null) { + return periodStartTimeInManifestUs + periodDurationUs; + } + long availableSegmentCount = index.getAvailableSegmentCount(periodDurationUs, nowUnixTimeUs); + if (availableSegmentCount == 0) { + return periodStartTimeInManifestUs; + } + long firstAvailableSegmentNum = + index.getFirstAvailableSegmentNum(periodDurationUs, nowUnixTimeUs); + long lastAvailableSegmentNum = firstAvailableSegmentNum + availableSegmentCount - 1; + return periodStartTimeInManifestUs + + index.getTimeUs(lastAvailableSegmentNum) + + index.getDurationUs(lastAvailableSegmentNum, periodDurationUs); + } + private static boolean isIndexExplicit(Period period) { for (int i = 0; i < period.adaptationSets.size(); i++) { @Nullable @@ -1451,6 +1618,44 @@ private static boolean isIndexExplicit(Period period) { return false; } + private static long getPresentationStartTimeMsFromProducerReferenceTime(DashManifest manifest) { + for (int periodIndex = 0; periodIndex < manifest.getPeriodCount(); periodIndex++) { + Period period = manifest.getPeriod(periodIndex); + if (period.startMs == C.TIME_UNSET) { + continue; + } + for (int i = 0; i < period.adaptationSets.size(); i++) { + AdaptationSet adaptationSet = period.adaptationSets.get(i); + if ((adaptationSet.type != C.TRACK_TYPE_AUDIO && adaptationSet.type != C.TRACK_TYPE_VIDEO) + || adaptationSet.representations.isEmpty() + || adaptationSet.producerReferenceTimes.isEmpty()) { + continue; + } + for (int j = 0; j < adaptationSet.producerReferenceTimes.size(); j++) { + ProducerReferenceTime producerReferenceTime = adaptationSet.producerReferenceTimes.get(j); + if (producerReferenceTime.wallClockTimeMs == C.TIME_UNSET) { + continue; + } + long presentationTimeInPeriodUs = 0; + if (producerReferenceTime.presentationTime != C.TIME_UNSET) { + Representation firstRepresentation = adaptationSet.representations.get(0); + long presentationTimeUs = + Util.scaleLargeTimestamp( + producerReferenceTime.presentationTime, + C.MICROS_PER_SECOND, + firstRepresentation.timescale); + presentationTimeInPeriodUs = + presentationTimeUs - firstRepresentation.presentationTimeOffsetUs; + } + return producerReferenceTime.wallClockTimeMs + - Util.usToMs(presentationTimeInPeriodUs) + - period.startMs; + } + } + } + return C.TIME_UNSET; + } + private static boolean hasVideoOrAudioAdaptationSets(Period period) { for (int i = 0; i < period.adaptationSets.size(); i++) { int type = period.adaptationSets.get(i).type; @@ -1461,6 +1666,11 @@ private static boolean hasVideoOrAudioAdaptationSets(Period period) { return false; } + @EnsuresNonNullIf(result = true, expression = "contentSteeringTracker") + private boolean isContentSteeringActive() { + return contentSteeringTracker != null && contentSteeringTracker.isActive(); + } + private static final class DashTimeline extends Timeline { private final long presentationStartTimeMs; @@ -1505,7 +1715,7 @@ public int getPeriodCount() { } @Override - public Period getPeriod(int periodIndex, Period period, boolean setIds) { + public Timeline.Period getPeriod(int periodIndex, Timeline.Period period, boolean setIds) { checkElementIndex(periodIndex, getPeriodCount()); Object id = setIds ? manifest.getPeriod(periodIndex).id : null; Object uid = setIds ? (firstPeriodId + periodIndex) : null; diff --git a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashUtil.java b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashUtil.java index a1e7cad2e02..a9c41f44c8d 100644 --- a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashUtil.java +++ b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DashUtil.java @@ -25,6 +25,7 @@ import androidx.media3.common.util.UnstableApi; import androidx.media3.datasource.DataSource; import androidx.media3.datasource.DataSpec; +import androidx.media3.exoplayer.dash.manifest.BaseUrl; import androidx.media3.exoplayer.dash.manifest.DashManifest; import androidx.media3.exoplayer.dash.manifest.DashManifestParser; import androidx.media3.exoplayer.dash.manifest.Period; @@ -49,10 +50,11 @@ public final class DashUtil { /** - * Builds a {@link DataSpec} for a given {@link RangedUri} belonging to {@link Representation}. + * Builds a {@link DataSpec} for a given {@link RangedUri} belonging to {@link Representation} and + * {@link BaseUrl}. * * @param representation The {@link Representation} to which the request belongs. - * @param baseUrl The base url with which to resolve the request URI. + * @param baseUrl The {@link BaseUrl} to which the request belongs. * @param requestUri The {@link RangedUri} of the data to request. * @param flags Flags to be set on the returned {@link DataSpec}. See {@link * DataSpec.Builder#setFlags(int)}. @@ -61,12 +63,13 @@ public final class DashUtil { */ public static DataSpec buildDataSpec( Representation representation, - String baseUrl, + BaseUrl baseUrl, RangedUri requestUri, int flags, Map httpRequestHeaders) { return new DataSpec.Builder() - .setUri(requestUri.resolveUri(baseUrl)) + .setUri(requestUri.resolveUri(baseUrl.url)) + .setLocation(baseUrl.serviceLocation) .setPosition(requestUri.start) .setLength(requestUri.length) .setKey(resolveCacheKey(representation, requestUri)) @@ -76,27 +79,23 @@ public static DataSpec buildDataSpec( } /** - * @deprecated Use {@link #buildDataSpec(Representation, String, RangedUri, int, Map)} instead. + * @deprecated Use {@link #buildDataSpec(Representation, BaseUrl, RangedUri, int, Map)} instead. */ @Deprecated public static DataSpec buildDataSpec( - Representation representation, String baseUrl, RangedUri requestUri, int flags) { - return buildDataSpec( - representation, baseUrl, requestUri, flags, /* httpRequestHeaders= */ ImmutableMap.of()); - } - - /** - * @deprecated Use {@link #buildDataSpec(Representation, String, RangedUri, int, Map)} instead. - */ - @Deprecated - public static DataSpec buildDataSpec( - Representation representation, RangedUri requestUri, int flags) { - return buildDataSpec( - representation, - representation.baseUrls.get(0).url, - requestUri, - flags, - /* httpRequestHeaders= */ ImmutableMap.of()); + Representation representation, + String baseUrl, + RangedUri requestUri, + int flags, + Map httpRequestHeaders) { + return new DataSpec.Builder() + .setUri(requestUri.resolveUri(baseUrl)) + .setPosition(requestUri.start) + .setLength(requestUri.length) + .setKey(resolveCacheKey(representation, requestUri)) + .setFlags(flags) + .setHttpRequestHeaders(httpRequestHeaders) + .build(); } /** @@ -308,7 +307,7 @@ private static void loadInitializationData( DataSpec dataSpec = DashUtil.buildDataSpec( representation, - representation.baseUrls.get(baseUrlIndex).url, + representation.baseUrls.get(baseUrlIndex), requestUri, /* flags= */ 0, /* httpRequestHeaders= */ ImmutableMap.of()); diff --git a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DefaultDashChunkSource.java b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DefaultDashChunkSource.java index 46cea070e54..26b646d6492 100644 --- a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DefaultDashChunkSource.java +++ b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/DefaultDashChunkSource.java @@ -61,6 +61,7 @@ import androidx.media3.exoplayer.upstream.LoaderErrorThrower; import androidx.media3.extractor.ChunkIndex; import androidx.media3.extractor.text.SubtitleParser; +import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableMap; import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.io.IOException; @@ -129,15 +130,6 @@ public Factory experimentalParseSubtitlesDuringExtraction( return this; } - @CanIgnoreReturnValue - @Override - public Factory experimentalSetCodecsToParseWithinGopSampleDependencies( - @C.VideoCodecFlags int codecsToParseWithinGopSampleDependencies) { - chunkExtractorFactory.experimentalSetCodecsToParseWithinGopSampleDependencies( - codecsToParseWithinGopSampleDependencies); - return this; - } - @Override public DashChunkSource createDashChunkSource( LoaderErrorThrower manifestLoaderErrorThrower, @@ -153,7 +145,8 @@ public DashChunkSource createDashChunkSource( @Nullable PlayerTrackEmsgHandler playerEmsgHandler, @Nullable TransferListener transferListener, PlayerId playerId, - @Nullable CmcdConfiguration cmcdConfiguration) { + @Nullable CmcdConfiguration cmcdConfiguration, + @Nullable DashContentSteeringTracker contentSteeringTracker) { DataSource dataSource = dataSourceFactory.createDataSource(); if (transferListener != null) { dataSource.addTransferListener(transferListener); @@ -174,7 +167,8 @@ public DashChunkSource createDashChunkSource( closedCaptionFormats, playerEmsgHandler, playerId, - cmcdConfiguration); + cmcdConfiguration, + contentSteeringTracker); } /** @@ -198,6 +192,7 @@ public Format getOutputTextFormat(Format sourceFormat) { private final int maxSegmentsPerLoad; @Nullable private final PlayerTrackEmsgHandler playerTrackEmsgHandler; @Nullable private final CmcdConfiguration cmcdConfiguration; + @Nullable private final DashContentSteeringTracker contentSteeringTracker; protected final RepresentationHolder[] representationHolders; @@ -206,6 +201,7 @@ public Format getOutputTextFormat(Format sourceFormat) { private int periodIndex; @Nullable private IOException fatalError; private boolean missingLastSegment; + @Nullable private BaseUrl selectedBaseUrl; /** * The time at which the last {@link #getNextChunk(LoadingInfo, long, List, ChunkHolder)} method @@ -236,6 +232,8 @@ public Format getOutputTextFormat(Format sourceFormat) { * messages targeting the player. Maybe null if this is not necessary. * @param playerId The {@link PlayerId} of the player using this chunk source. * @param cmcdConfiguration The {@link CmcdConfiguration} for this chunk source. + * @param contentSteeringTracker The {@link DashContentSteeringTracker} to use for Content + * Steering, or {@code null} if Content Steering is unavailable. */ public DefaultDashChunkSource( ChunkExtractor.Factory chunkExtractorFactory, @@ -253,7 +251,8 @@ public DefaultDashChunkSource( List closedCaptionFormats, @Nullable PlayerTrackEmsgHandler playerTrackEmsgHandler, PlayerId playerId, - @Nullable CmcdConfiguration cmcdConfiguration) { + @Nullable CmcdConfiguration cmcdConfiguration, + @Nullable DashContentSteeringTracker contentSteeringTracker) { this.manifestLoaderErrorThrower = manifestLoaderErrorThrower; this.manifest = manifest; this.baseUrlExclusionList = baseUrlExclusionList; @@ -266,6 +265,7 @@ public DefaultDashChunkSource( this.maxSegmentsPerLoad = maxSegmentsPerLoad; this.playerTrackEmsgHandler = playerTrackEmsgHandler; this.cmcdConfiguration = cmcdConfiguration; + this.contentSteeringTracker = contentSteeringTracker; this.lastChunkRequestRealtimeMs = C.TIME_UNSET; long periodDurationUs = manifest.getPeriodDurationUs(periodIndex); @@ -340,6 +340,12 @@ public void updateTrackSelection(ExoTrackSelection trackSelection) { this.trackSelection = trackSelection; } + @Override + @Nullable + public BaseUrl getSelectedBaseUrl() { + return selectedBaseUrl; + } + @Override public void maybeThrowError() throws IOException { if (fatalError != null) { @@ -415,7 +421,11 @@ public void getNextChunk( representationHolder = updateSelectedBaseUrl(/* trackIndex= */ i); chunkIterators[i] = new RepresentationSegmentIterator( - representationHolder, segmentNum, lastAvailableSegmentNum, nowPeriodTimeUs); + representationHolder, + segmentNum, + lastAvailableSegmentNum, + nowPeriodTimeUs, + contentSteeringTracker); } } } @@ -607,7 +617,7 @@ public boolean onChunkLoadError( } LoadErrorHandlingPolicy.FallbackOptions fallbackOptions = - createFallbackOptions(trackSelection, representationHolder.representation.baseUrls); + createFallbackOptions(trackSelection, representationHolder); if (!fallbackOptions.isFallbackAvailable(LoadErrorHandlingPolicy.FALLBACK_TYPE_TRACK) && !fallbackOptions.isFallbackAvailable(LoadErrorHandlingPolicy.FALLBACK_TYPE_LOCATION)) { return false; @@ -646,22 +656,64 @@ public void release() { // Internal methods. private LoadErrorHandlingPolicy.FallbackOptions createFallbackOptions( - ExoTrackSelection trackSelection, List baseUrls) { + ExoTrackSelection trackSelection, RepresentationHolder representationHolder) { long nowMs = SystemClock.elapsedRealtime(); - int numberOfTracks = trackSelection.length(); - int numberOfExcludedTracks = 0; - for (int i = 0; i < numberOfTracks; i++) { - if (trackSelection.isTrackExcluded(i, nowMs)) { - numberOfExcludedTracks++; + ImmutableList baseUrls = representationHolder.representation.baseUrls; + + if (contentSteeringTracker != null && contentSteeringTracker.isActive()) { + int numberOfTracks = 0; + int numberOfExcludedTracks = 0; + String serviceLocation = representationHolder.selectedBaseUrl.serviceLocation; + for (int i = 0; i < trackSelection.length(); i++) { + if (!hasBaseUrlForServiceLocation( + representationHolders[i].representation.baseUrls, serviceLocation)) { + // Track i does not have a base URL for the current service location. In the Content + // Steering case, we do not count this track as an option for track fallback, as we should + // stay on the current service location. + continue; + } + numberOfTracks++; + if (trackSelection.isTrackExcluded(i, nowMs)) { + numberOfExcludedTracks++; + } + } + int numberOfLocations = BaseUrlExclusionList.getServiceLocationCount(baseUrls); + int numberOfExcludedLocations = + numberOfLocations - baseUrlExclusionList.getServiceLocationCountAfterExclusion(baseUrls); + return new LoadErrorHandlingPolicy.FallbackOptions( + numberOfLocations, + numberOfExcludedLocations, + numberOfTracks, + numberOfExcludedTracks, + /* locationSteeringActive= */ true); + } else { + int numberOfTracks = trackSelection.length(); + int numberOfExcludedTracks = 0; + for (int i = 0; i < numberOfTracks; i++) { + if (trackSelection.isTrackExcluded(i, nowMs)) { + numberOfExcludedTracks++; + } + } + int numberOfLocations = BaseUrlExclusionList.getPriorityCount(baseUrls); + int numberOfExcludedLocations = + numberOfLocations - baseUrlExclusionList.getPriorityCountAfterExclusion(baseUrls); + return new LoadErrorHandlingPolicy.FallbackOptions( + numberOfLocations, + numberOfExcludedLocations, + numberOfTracks, + numberOfExcludedTracks, + /* locationSteeringActive= */ false); + } + } + + private static boolean hasBaseUrlForServiceLocation( + ImmutableList baseUrls, String serviceLocation) { + for (int i = 0; i < baseUrls.size(); i++) { + if (serviceLocation.equals(baseUrls.get(i).serviceLocation)) { + return true; } } - int priorityCount = BaseUrlExclusionList.getPriorityCount(baseUrls); - return new LoadErrorHandlingPolicy.FallbackOptions( - /* numberOfLocations= */ priorityCount, - /* numberOfExcludedLocations= */ priorityCount - - baseUrlExclusionList.getPriorityCountAfterExclusion(baseUrls), - numberOfTracks, - numberOfExcludedTracks); + return false; } private long getSegmentNum( @@ -748,7 +800,7 @@ protected Chunk newInitializationChunk( DataSpec dataSpec = DashUtil.buildDataSpec( representation, - representationHolder.selectedBaseUrl.url, + representationHolder.selectedBaseUrl, requestUri, /* flags= */ 0, /* httpRequestHeaders= */ ImmutableMap.of()); @@ -795,7 +847,7 @@ protected Chunk newMediaChunk( DataSpec dataSpec = DashUtil.buildDataSpec( representation, - representationHolder.selectedBaseUrl.url, + representationHolder.selectedBaseUrl, segmentUri, flags, /* httpRequestHeaders= */ ImmutableMap.of()); @@ -852,7 +904,7 @@ protected Chunk newMediaChunk( DataSpec dataSpec = DashUtil.buildDataSpec( representation, - representationHolder.selectedBaseUrl.url, + representationHolder.selectedBaseUrl, segmentUri, flags, /* httpRequestHeaders= */ ImmutableMap.of()); @@ -869,10 +921,10 @@ protected Chunk newMediaChunk( CmcdData cmcdData = cmcdDataFactory.createCmcdData(); dataSpec = cmcdData.addToDataSpec(dataSpec); } - long sampleOffsetUs = -representation.presentationTimeOffsetUs; - if (MimeTypes.isImage(trackFormat.sampleMimeType)) { - sampleOffsetUs += startTimeUs; - } + long sampleOffsetUs = + MimeTypes.isImage(trackFormat.sampleMimeType) + ? startTimeUs + : -representation.presentationTimeOffsetUs; return new ContainerMediaChunk( dataSource, dataSpec, @@ -913,9 +965,12 @@ private RepresentationHolder updateSelectedBaseUrl(int trackIndex) { @Nullable BaseUrl selectedBaseUrl = baseUrlExclusionList.selectBaseUrl(representationHolder.representation.baseUrls); - if (selectedBaseUrl != null && !selectedBaseUrl.equals(representationHolder.selectedBaseUrl)) { - representationHolder = representationHolder.copyWithNewSelectedBaseUrl(selectedBaseUrl); - representationHolders[trackIndex] = representationHolder; + if (selectedBaseUrl != null) { + if (!selectedBaseUrl.equals(representationHolder.selectedBaseUrl)) { + representationHolder = representationHolder.copyWithNewSelectedBaseUrl(selectedBaseUrl); + representationHolders[trackIndex] = representationHolder; + } + this.selectedBaseUrl = selectedBaseUrl; } return representationHolder; } @@ -927,6 +982,7 @@ protected static final class RepresentationSegmentIterator extends BaseMediaChun private final RepresentationHolder representationHolder; private final long nowPeriodTimeUs; + @Nullable private final DashContentSteeringTracker contentSteeringTracker; /** * Creates iterator. @@ -936,15 +992,33 @@ protected static final class RepresentationSegmentIterator extends BaseMediaChun * @param lastAvailableSegmentNum The number of the last available segment. * @param nowPeriodTimeUs The current time in microseconds since the start of the period used * for calculating if segments are available at full network speed. + * @param contentSteeringTracker The {@link DashContentSteeringTracker}, or {@code null}. */ public RepresentationSegmentIterator( RepresentationHolder representation, long firstAvailableSegmentNum, long lastAvailableSegmentNum, - long nowPeriodTimeUs) { + long nowPeriodTimeUs, + @Nullable DashContentSteeringTracker contentSteeringTracker) { super(/* fromIndex= */ firstAvailableSegmentNum, /* toIndex= */ lastAvailableSegmentNum); this.representationHolder = representation; this.nowPeriodTimeUs = nowPeriodTimeUs; + this.contentSteeringTracker = contentSteeringTracker; + } + + @Override + public int getLocationSteeringPriorityIndex() { + if (contentSteeringTracker == null) { + return Integer.MAX_VALUE; + } + @Nullable + ImmutableList priorityList = + contentSteeringTracker.getCurrentServiceLocationPriority(); + if (priorityList == null) { + return Integer.MAX_VALUE; + } + int index = priorityList.indexOf(representationHolder.selectedBaseUrl.serviceLocation); + return index != -1 ? index : Integer.MAX_VALUE; } @Override @@ -958,7 +1032,7 @@ public DataSpec getDataSpec() { : DataSpec.FLAG_MIGHT_NOT_USE_FULL_NETWORK_SPEED; return DashUtil.buildDataSpec( representationHolder.representation, - representationHolder.selectedBaseUrl.url, + representationHolder.selectedBaseUrl, segmentUri, flags, /* httpRequestHeaders= */ ImmutableMap.of()); @@ -1011,19 +1085,12 @@ protected static final class RepresentationHolder { @Nullable DashSegmentIndex oldIndex = representation.getIndex(); @Nullable DashSegmentIndex newIndex = newRepresentation.getIndex(); - if (oldIndex == null) { - // Segment numbers cannot shift if the index isn't defined by the manifest. - return new RepresentationHolder( - newPeriodDurationUs, - newRepresentation, - selectedBaseUrl, - chunkExtractor, - segmentNumShift, - oldIndex); - } - - if (!oldIndex.isExplicit()) { - // Segment numbers cannot shift if the index isn't explicit. + if (oldIndex == null + || newIndex == null + || !oldIndex.isExplicit() + || oldIndex.getSegmentCount(newPeriodDurationUs) == 0 + || newIndex.getSegmentCount(newPeriodDurationUs) == 0) { + // Segment numbers cannot shift if an index is missing, non-explicit, or empty. return new RepresentationHolder( newPeriodDurationUs, newRepresentation, @@ -1034,19 +1101,6 @@ protected static final class RepresentationHolder { } long oldIndexSegmentCount = oldIndex.getSegmentCount(newPeriodDurationUs); - if (oldIndexSegmentCount == 0) { - // Segment numbers cannot shift if the old index was empty. - return new RepresentationHolder( - newPeriodDurationUs, - newRepresentation, - selectedBaseUrl, - chunkExtractor, - segmentNumShift, - newIndex); - } - - checkNotNull(newIndex); - long oldIndexFirstSegmentNum = oldIndex.getFirstSegmentNum(); long oldIndexStartTimeUs = oldIndex.getTimeUs(oldIndexFirstSegmentNum); long oldIndexLastSegmentNum = oldIndexFirstSegmentNum + oldIndexSegmentCount - 1; diff --git a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/ServiceLocationFilteringThroughputEstimator.java b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/ServiceLocationFilteringThroughputEstimator.java new file mode 100644 index 00000000000..30dcfdacca6 --- /dev/null +++ b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/ServiceLocationFilteringThroughputEstimator.java @@ -0,0 +1,155 @@ +/* + * Copyright (C) 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.exoplayer.dash; + +import androidx.annotation.Nullable; +import androidx.media3.common.util.UnstableApi; +import androidx.media3.datasource.DataSource; +import androidx.media3.datasource.DataSpec; +import androidx.media3.datasource.TransferListener; +import androidx.media3.exoplayer.upstream.experimental.BandwidthEstimator; +import androidx.media3.exoplayer.upstream.experimental.SplitParallelSampleBandwidthEstimator; +import java.util.concurrent.ConcurrentHashMap; + +/** + * An estimator that estimates media download throughput from network per service location. + * + *

    The throughput estimates for each service location are calculated based on a sliding window of + * average of the throughput samples. + */ +@UnstableApi +/* package */ final class ServiceLocationFilteringThroughputEstimator implements TransferListener { + + @Nullable private final TransferListener delegateTransferListener; + private final ConcurrentHashMap estimatorsByServiceLocation; + + private volatile boolean throughputEstimateStopped; + + /** + * Creates an instance. + * + * @param delegateTransferListener A delegate {@link TransferListener} to forward transfer events + * to, or {@code null}. + */ + public ServiceLocationFilteringThroughputEstimator( + @Nullable TransferListener delegateTransferListener) { + this.delegateTransferListener = delegateTransferListener; + this.estimatorsByServiceLocation = new ConcurrentHashMap<>(); + } + + /** + * Returns the estimated throughput in bits per second for the given service location, or {@code + * 0} if no estimate is available. + * + * @param serviceLocation The ID of service location. + * @return The throughput estimate in bits per second. + */ + public long getThroughputEstimate(String serviceLocation) { + @Nullable BandwidthEstimator estimator = estimatorsByServiceLocation.get(serviceLocation); + if (estimator == null) { + return 0L; + } + synchronized (estimator) { + long estimate = estimator.getBandwidthEstimate(); + return estimate == BandwidthEstimator.ESTIMATE_NOT_AVAILABLE ? 0L : estimate; + } + } + + /** + * Stops throughput estimate. + * + *

    After this method is called, {@link #getThroughputEstimate(String)} will return {@code 0} + * for all service locations. The estimator will still forward transfer events to the delegate + * transfer listener. + */ + public void stopThroughputEstimate() { + throughputEstimateStopped = true; + estimatorsByServiceLocation.clear(); + } + + // TransferListener implementations + + @Override + public void onTransferInitializing(DataSource source, DataSpec dataSpec, boolean isNetwork) { + if (delegateTransferListener != null) { + delegateTransferListener.onTransferInitializing(source, dataSpec, isNetwork); + } + } + + @Override + public void onTransferStart(DataSource source, DataSpec dataSpec, boolean isNetwork) { + if (delegateTransferListener != null) { + delegateTransferListener.onTransferStart(source, dataSpec, isNetwork); + } + if (throughputEstimateStopped || !isTransferAtFullNetworkSpeed(dataSpec, isNetwork)) { + return; + } + @Nullable String serviceLocation = dataSpec.location; + if (serviceLocation != null) { + estimatorsByServiceLocation.putIfAbsent( + serviceLocation, new SplitParallelSampleBandwidthEstimator.Builder().build()); + @Nullable BandwidthEstimator estimator = estimatorsByServiceLocation.get(serviceLocation); + if (estimator != null) { + synchronized (estimator) { + estimator.onTransferStart(source); + } + } + } + } + + @Override + public void onBytesTransferred( + DataSource source, DataSpec dataSpec, boolean isNetwork, int bytesTransferred) { + if (delegateTransferListener != null) { + delegateTransferListener.onBytesTransferred(source, dataSpec, isNetwork, bytesTransferred); + } + if (throughputEstimateStopped || !isTransferAtFullNetworkSpeed(dataSpec, isNetwork)) { + return; + } + @Nullable String serviceLocation = dataSpec.location; + if (serviceLocation != null) { + @Nullable BandwidthEstimator estimator = estimatorsByServiceLocation.get(serviceLocation); + if (estimator != null) { + synchronized (estimator) { + estimator.onBytesTransferred(source, bytesTransferred); + } + } + } + } + + @Override + public void onTransferEnd(DataSource source, DataSpec dataSpec, boolean isNetwork) { + if (delegateTransferListener != null) { + delegateTransferListener.onTransferEnd(source, dataSpec, isNetwork); + } + if (throughputEstimateStopped || !isTransferAtFullNetworkSpeed(dataSpec, isNetwork)) { + return; + } + @Nullable String serviceLocation = dataSpec.location; + if (serviceLocation != null) { + @Nullable BandwidthEstimator estimator = estimatorsByServiceLocation.get(serviceLocation); + if (estimator != null) { + synchronized (estimator) { + estimator.onTransferEnd(source); + } + } + } + } + + private static boolean isTransferAtFullNetworkSpeed(DataSpec dataSpec, boolean isNetwork) { + return isNetwork && !dataSpec.isFlagSet(DataSpec.FLAG_MIGHT_NOT_USE_FULL_NETWORK_SPEED); + } +} diff --git a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/manifest/AdaptationSet.java b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/manifest/AdaptationSet.java index 6522052503b..4a97475e96c 100644 --- a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/manifest/AdaptationSet.java +++ b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/manifest/AdaptationSet.java @@ -17,6 +17,7 @@ import androidx.media3.common.C; import androidx.media3.common.util.UnstableApi; +import com.google.common.collect.ImmutableList; import java.util.Collections; import java.util.List; @@ -48,6 +49,9 @@ public class AdaptationSet { /** Supplemental properties in the adaptation set. */ public final List supplementalProperties; + /** {@link ProducerReferenceTime} instances in the adaptation set. */ + public final ImmutableList producerReferenceTimes; + /** * @param id A non-negative identifier for the adaptation set that's unique in the scope of its * containing period, or {@link #ID_UNSET} if not specified. @@ -56,6 +60,7 @@ public class AdaptationSet { * @param accessibilityDescriptors Accessibility descriptors in the adaptation set. * @param essentialProperties Essential properties in the adaptation set. * @param supplementalProperties Supplemental properties in the adaptation set. + * @param producerReferenceTimes {@link ProducerReferenceTime} instances in the adaptation set. */ public AdaptationSet( long id, @@ -63,12 +68,14 @@ public AdaptationSet( List representations, List accessibilityDescriptors, List essentialProperties, - List supplementalProperties) { + List supplementalProperties, + ImmutableList producerReferenceTimes) { this.id = id; this.type = type; this.representations = Collections.unmodifiableList(representations); this.accessibilityDescriptors = Collections.unmodifiableList(accessibilityDescriptors); this.essentialProperties = Collections.unmodifiableList(essentialProperties); this.supplementalProperties = Collections.unmodifiableList(supplementalProperties); + this.producerReferenceTimes = producerReferenceTimes; } } diff --git a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/manifest/ContentSteering.java b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/manifest/ContentSteering.java new file mode 100644 index 00000000000..bd0be01f263 --- /dev/null +++ b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/manifest/ContentSteering.java @@ -0,0 +1,62 @@ +/* + * Copyright (C) 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.exoplayer.dash.manifest; + +import android.net.Uri; +import androidx.annotation.Nullable; +import androidx.media3.common.util.UnstableApi; +import com.google.common.collect.ImmutableList; +import java.util.Objects; + +/** A parsed Content Steering element. */ +@UnstableApi +public final class ContentSteering { + /** The steering server URI. */ + public final Uri steeringServerUri; + + /** The default service location. */ + public final ImmutableList defaultServiceLocation; + + /** Whether to resolve the response from the steering server prior to starting playback. */ + public final boolean queryBeforeStart; + + /** Creates an instance. */ + public ContentSteering( + Uri steeringServerUri, String[] defaultServiceLocation, boolean queryBeforeStart) { + this.steeringServerUri = steeringServerUri; + this.defaultServiceLocation = ImmutableList.copyOf(defaultServiceLocation); + this.queryBeforeStart = queryBeforeStart; + } + + @Override + public boolean equals(@Nullable Object o) { + if (this == o) { + return true; + } + if (!(o instanceof ContentSteering)) { + return false; + } + ContentSteering other = (ContentSteering) o; + return Objects.equals(steeringServerUri, other.steeringServerUri) + && Objects.equals(defaultServiceLocation, other.defaultServiceLocation) + && queryBeforeStart == other.queryBeforeStart; + } + + @Override + public int hashCode() { + return Objects.hash(steeringServerUri, defaultServiceLocation, queryBeforeStart); + } +} diff --git a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/manifest/DashManifest.java b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/manifest/DashManifest.java index d18af4e579d..11fc0aad5ef 100644 --- a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/manifest/DashManifest.java +++ b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/manifest/DashManifest.java @@ -100,9 +100,13 @@ public class DashManifest implements FilterableManifest { /** The locations to request future updates of the DASH media presentation description (mpd). */ public final ImmutableList locations; + /** The {@link ContentSteering}, or {@code null} if not present. */ + @Nullable public final ContentSteering contentSteering; + /** * @deprecated Use {@link #DashManifest(long, long, long, boolean, long, long, long, long, - * ProgramInformation, UtcTimingElement, ServiceDescriptionElement, List, List)} instead. + * ProgramInformation, UtcTimingElement, ServiceDescriptionElement, List, List, + * ContentSteering)} instead. */ @Deprecated public DashManifest( @@ -132,9 +136,16 @@ public DashManifest( utcTiming, serviceDescription, periods, - location == null ? ImmutableList.of() : ImmutableList.of(new Location(location))); + location == null ? ImmutableList.of() : ImmutableList.of(new Location(location)), + /* contentSteering= */ null); } + /** + * @deprecated Use {@link #DashManifest(long, long, long, boolean, long, long, long, long, + * ProgramInformation, UtcTimingElement, ServiceDescriptionElement, List, List, + * ContentSteering)} instead. + */ + @Deprecated public DashManifest( long availabilityStartTimeMs, long durationMs, @@ -149,6 +160,38 @@ public DashManifest( @Nullable ServiceDescriptionElement serviceDescription, List periods, List locations) { + this( + availabilityStartTimeMs, + durationMs, + minBufferTimeMs, + dynamic, + minUpdatePeriodMs, + timeShiftBufferDepthMs, + suggestedPresentationDelayMs, + publishTimeMs, + programInformation, + utcTiming, + serviceDescription, + periods, + locations, + /* contentSteering= */ null); + } + + public DashManifest( + long availabilityStartTimeMs, + long durationMs, + long minBufferTimeMs, + boolean dynamic, + long minUpdatePeriodMs, + long timeShiftBufferDepthMs, + long suggestedPresentationDelayMs, + long publishTimeMs, + @Nullable ProgramInformation programInformation, + @Nullable UtcTimingElement utcTiming, + @Nullable ServiceDescriptionElement serviceDescription, + List periods, + List locations, + @Nullable ContentSteering contentSteering) { this.availabilityStartTimeMs = availabilityStartTimeMs; this.durationMs = durationMs; this.minBufferTimeMs = minBufferTimeMs; @@ -163,6 +206,7 @@ public DashManifest( this.periods = periods == null ? Collections.emptyList() : periods; this.locations = locations == null ? ImmutableList.of() : ImmutableList.copyOf(locations); this.location = locations == null || locations.isEmpty() ? null : getLast(locations).url; + this.contentSteering = contentSteering; } public final int getPeriodCount() { @@ -204,7 +248,11 @@ public final DashManifest copy(List streamKeys) { copyAdaptationSets(period.adaptationSets, keys); Period copiedPeriod = new Period( - period.id, period.startMs - shiftMs, copyAdaptationSets, period.eventStreams); + period.id, + period.startMs - shiftMs, + copyAdaptationSets, + period.eventStreams, + period.assetIdentifier); copyPeriods.add(copiedPeriod); } } @@ -222,7 +270,8 @@ public final DashManifest copy(List streamKeys) { utcTiming, serviceDescription, copyPeriods, - locations); + locations, + contentSteering); } private static ArrayList copyAdaptationSets( @@ -249,7 +298,8 @@ private static ArrayList copyAdaptationSets( copyRepresentations, adaptationSet.accessibilityDescriptors, adaptationSet.essentialProperties, - adaptationSet.supplementalProperties)); + adaptationSet.supplementalProperties, + adaptationSet.producerReferenceTimes)); } while (key.periodIndex == periodIndex); // Add back the last key which doesn't belong to the period being processed keys.addFirst(key); diff --git a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/manifest/DashManifestParser.java b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/manifest/DashManifestParser.java index ff1aa226f11..e895f9cf859 100644 --- a/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/manifest/DashManifestParser.java +++ b/libraries/exoplayer_dash/src/main/java/androidx/media3/exoplayer/dash/manifest/DashManifestParser.java @@ -144,7 +144,7 @@ protected DashManifest parseMediaPresentationDescription(XmlPullParser xpp, Uri long availabilityStartTime = parseDateTime(xpp, "availabilityStartTime", C.TIME_UNSET); long durationMs = parseDuration(xpp, "mediaPresentationDuration", C.TIME_UNSET); long minBufferTimeMs = parseDuration(xpp, "minBufferTime", C.TIME_UNSET); - String typeString = xpp.getAttributeValue(null, "type"); + @Nullable String typeString = XmlPullParserUtil.getAttributeValue(xpp, "type"); boolean dynamic = "dynamic".equals(typeString); long minUpdateTimeMs = dynamic ? parseDuration(xpp, "minimumUpdatePeriod", C.TIME_UNSET) : C.TIME_UNSET; @@ -153,10 +153,11 @@ protected DashManifest parseMediaPresentationDescription(XmlPullParser xpp, Uri long suggestedPresentationDelayMs = dynamic ? parseDuration(xpp, "suggestedPresentationDelay", C.TIME_UNSET) : C.TIME_UNSET; long publishTimeMs = parseDateTime(xpp, "publishTime", C.TIME_UNSET); - ProgramInformation programInformation = null; - UtcTimingElement utcTiming = null; + @Nullable ProgramInformation programInformation = null; + @Nullable UtcTimingElement utcTiming = null; List locations = new ArrayList<>(); - ServiceDescriptionElement serviceDescription = null; + @Nullable ServiceDescriptionElement serviceDescription = null; + @Nullable ContentSteering contentSteering = null; long baseUrlAvailabilityTimeOffsetUs = dynamic ? 0 : C.TIME_UNSET; String documentBaseUriString = documentBaseUri.toString(); BaseUrl documentBaseUrl = @@ -186,8 +187,9 @@ protected DashManifest parseMediaPresentationDescription(XmlPullParser xpp, Uri } else if (XmlPullParserUtil.isStartTag(xpp, "UTCTiming")) { utcTiming = parseUtcTiming(xpp); } else if (XmlPullParserUtil.isStartTag(xpp, "Location")) { - String serviceLocation = xpp.getAttributeValue(null, "serviceLocation"); - String locationUrlString = xpp.nextText(); + @Nullable + String serviceLocation = XmlPullParserUtil.getAttributeValue(xpp, "serviceLocation"); + String locationUrlString = parseText(xpp, "Location").trim(); Uri resolvedUrl = Uri.parse(UriUtil.resolve(documentBaseUriString, locationUrlString)); locations.add( serviceLocation != null @@ -195,6 +197,8 @@ protected DashManifest parseMediaPresentationDescription(XmlPullParser xpp, Uri : new Location(resolvedUrl)); } else if (XmlPullParserUtil.isStartTag(xpp, "ServiceDescription")) { serviceDescription = parseServiceDescription(xpp); + } else if (XmlPullParserUtil.isStartTag(xpp, "ContentSteering")) { + contentSteering = parseContentSteering(xpp, documentBaseUri); } else if (XmlPullParserUtil.isStartTag(xpp, "Period") && !seenEarlyAccessPeriod) { Pair periodWithDurationMs = parsePeriod( @@ -253,13 +257,14 @@ protected DashManifest parseMediaPresentationDescription(XmlPullParser xpp, Uri utcTiming, serviceDescription, periods, - locations); + locations, + contentSteering); } /** * @deprecated Use {@link #buildMediaPresentationDescription(long, long, long, boolean, long, * long, long, long, ProgramInformation, UtcTimingElement, ServiceDescriptionElement, List, - * List)} instead. + * List, ContentSteering)} instead. */ @Deprecated protected DashManifest buildMediaPresentationDescription( @@ -289,9 +294,16 @@ protected DashManifest buildMediaPresentationDescription( utcTiming, serviceDescription, periods, - location == null ? ImmutableList.of() : ImmutableList.of(new Location(location))); + location == null ? ImmutableList.of() : ImmutableList.of(new Location(location)), + /* contentSteering= */ null); } + /** + * @deprecated Use {@link #buildMediaPresentationDescription(long, long, long, boolean, long, + * long, long, long, ProgramInformation, UtcTimingElement, ServiceDescriptionElement, List, + * List, ContentSteering)} instead. + */ + @Deprecated protected DashManifest buildMediaPresentationDescription( long availabilityStartTime, long durationMs, @@ -319,13 +331,57 @@ protected DashManifest buildMediaPresentationDescription( utcTiming, serviceDescription, periods, - locations); + locations, + /* contentSteering= */ null); + } + + protected DashManifest buildMediaPresentationDescription( + long availabilityStartTime, + long durationMs, + long minBufferTimeMs, + boolean dynamic, + long minUpdateTimeMs, + long timeShiftBufferDepthMs, + long suggestedPresentationDelayMs, + long publishTimeMs, + @Nullable ProgramInformation programInformation, + @Nullable UtcTimingElement utcTiming, + @Nullable ServiceDescriptionElement serviceDescription, + List periods, + List locations, + @Nullable ContentSteering contentSteering) { + return new DashManifest( + availabilityStartTime, + durationMs, + minBufferTimeMs, + dynamic, + minUpdateTimeMs, + timeShiftBufferDepthMs, + suggestedPresentationDelayMs, + publishTimeMs, + programInformation, + utcTiming, + serviceDescription, + periods, + locations, + contentSteering); } + protected ContentSteering parseContentSteering(XmlPullParser xpp, Uri documentBaseUri) + throws XmlPullParserException, IOException { + String[] defaultServiceLocation = + parseCommaSeparatedList(xpp, "defaultServiceLocation", new String[0]); + boolean queryBeforeStart = parseBoolean(xpp, "queryBeforeStart", /* defaultValue= */ false); + String serverUriString = parseText(xpp, "ContentSteering").trim(); + Uri serverUri = UriUtil.resolveToUri(documentBaseUri.toString(), serverUriString); + return new ContentSteering(serverUri, defaultServiceLocation, queryBeforeStart); + } + + @Nullable protected UtcTimingElement parseUtcTiming(XmlPullParser xpp) { - String schemeIdUri = xpp.getAttributeValue(null, "schemeIdUri"); - String value = xpp.getAttributeValue(null, "value"); - return buildUtcTimingElement(schemeIdUri, value); + @Nullable String schemeIdUri = XmlPullParserUtil.getAttributeValue(xpp, "schemeIdUri"); + @Nullable String value = XmlPullParserUtil.getAttributeValue(xpp, "value"); + return schemeIdUri != null && value != null ? buildUtcTimingElement(schemeIdUri, value) : null; } protected UtcTimingElement buildUtcTimingElement(String schemeIdUri, String value) { @@ -363,7 +419,7 @@ protected Pair parsePeriod( long timeShiftBufferDepthMs, boolean dvbProfileDeclared) throws XmlPullParserException, IOException { - @Nullable String id = xpp.getAttributeValue(null, "id"); + @Nullable String id = XmlPullParserUtil.getAttributeValue(xpp, "id"); long startMs = parseDuration(xpp, "start", defaultStartMs); long periodStartUnixTimeMs = availabilityStartTimeMs != C.TIME_UNSET ? availabilityStartTimeMs + startMs : C.TIME_UNSET; @@ -461,25 +517,30 @@ protected AdaptationSet parseAdaptationSet( long id = parseLong(xpp, "id", AdaptationSet.ID_UNSET); @C.TrackType int contentType = parseContentType(xpp); - String mimeType = xpp.getAttributeValue(null, "mimeType"); - String codecs = xpp.getAttributeValue(null, "codecs"); - String supplementalCodecs = xpp.getAttributeValue(null, "scte214:supplementalCodecs"); - String supplementalProfiles = xpp.getAttributeValue(null, "scte214:supplementalProfiles"); + @Nullable String mimeType = XmlPullParserUtil.getAttributeValue(xpp, "mimeType"); + @Nullable String codecs = XmlPullParserUtil.getAttributeValue(xpp, "codecs"); + @Nullable + String supplementalCodecs = + XmlPullParserUtil.getAttributeValue(xpp, "scte214:supplementalCodecs"); + @Nullable + String supplementalProfiles = + XmlPullParserUtil.getAttributeValue(xpp, "scte214:supplementalProfiles"); int width = parseInt(xpp, "width", Format.NO_VALUE); int height = parseInt(xpp, "height", Format.NO_VALUE); float frameRate = parseFrameRate(xpp, Format.NO_VALUE); int audioChannels = Format.NO_VALUE; int audioSamplingRate = parseInt(xpp, "audioSamplingRate", Format.NO_VALUE); - String language = xpp.getAttributeValue(null, "lang"); - String label = xpp.getAttributeValue(null, "label"); + @Nullable String language = XmlPullParserUtil.getAttributeValue(xpp, "lang"); + @Nullable String label = XmlPullParserUtil.getAttributeValue(xpp, "label"); List

    The lists {@code newPlaylistUrls} and {@code basePlaylistUrls} have the equal size, and - * each URI in the {@code newPlaylistUrls} is cloned from the base URI in the {@code - * basePlaylistUrls} of the same index. + *

    The lists {@code newUris} and {@code baseUris} have the equal size, and each URI in the + * {@code newUris} is cloned from the base URI in the {@code baseUris} of the same index. * * @param newPathwayId The new pathway ID. * @param basePathwayId The base pathway ID. - * @param newPlaylistUrls The list of new playlist URLs. - * @param basePlaylistUrls The list of base playlist URLs. + * @param newUris The list of new cloned URIs. + * @param baseUris The list of base URIs. */ void onNewPathwayAvailable( String newPathwayId, String basePathwayId, - ImmutableList newPlaylistUrls, - ImmutableList basePlaylistUrls); + ImmutableList newUris, + ImmutableList baseUris); } private static final String PATHWAY_PARAM = "_HLS_pathway"; private static final String THROUGHPUT_PARAM = "_HLS_throughput"; - private final SteeringManifestTracker steeringManifestTracker; - @Nullable private final Callback callback; private final BandwidthMeter bandwidthMeter; + @Nullable private final Callback callback; + private final Set availablePathwayIds; + private final Set excludedPathwayIds; + private final HandlerWrapper handler; private final List variantRedundantGroups; private final List videoRenditionRedundantGroups; private final List audioRenditionRedundantGroups; private final List subtitleRenditionRedundantGroups; - private final Set availablePathwayIds; - private final Set excludedPathwayIds; - private final HandlerWrapper handler; - private boolean isActive; - private String currentPathwayId; - @Nullable private ImmutableList currentPathwayPriority; + + @Nullable private String currentPathwayId; /** * Creates an {@link HlsContentSteeringTracker}. * - * @param dataSourceFactory The {@link HlsDataSourceFactory} to create data sources for loading - * steering manifests. - * @param downloadExecutorSupplier A supplier to obtain a {@link ReleasableExecutor} for - * downloading steering manifests, or {@code null}. + * @param dataSourceFactory The {@link HlsDataSourceFactory}. + * @param downloadExecutorSupplier A supplier to obtain a {@link ReleasableExecutor}, or {@code + * null}. * @param playlistTracker The {@link HlsPlaylistTracker}. - * @param callback A {@link Callback} to receive events, or {@code null}. - * @param bandwidthMeter The {@link BandwidthMeter} to obtain throughput estimates. - * @param clock The {@link Clock} to schedule handler messages. + * @param callback A {@link HlsContentSteeringTracker.Callback} to receive events, or {@code + * null}. + * @param bandwidthMeter The {@link BandwidthMeter}. + * @param clock The {@link Clock}. */ public HlsContentSteeringTracker( HlsDataSourceFactory dataSourceFactory, @Nullable Supplier downloadExecutorSupplier, HlsPlaylistTracker playlistTracker, - @Nullable Callback callback, + @Nullable HlsContentSteeringTracker.Callback callback, BandwidthMeter bandwidthMeter, Clock clock) { - this.steeringManifestTracker = - new SteeringManifestTracker( - () -> dataSourceFactory.createDataSource(DATA_TYPE_STEERING_MANIFEST), - downloadExecutorSupplier); + super( + () -> dataSourceFactory.createDataSource(DATA_TYPE_STEERING_MANIFEST), + downloadExecutorSupplier); this.callback = callback; this.bandwidthMeter = bandwidthMeter; + this.availablePathwayIds = + new HashSet<>(getFirstVariantRedundantGroup(playlistTracker).getAllPathwayIds()); + this.excludedPathwayIds = new HashSet<>(); this.variantRedundantGroups = checkNotNull(playlistTracker.getRedundantGroups(HlsRedundantGroup.VARIANT)); this.videoRenditionRedundantGroups = @@ -136,43 +131,22 @@ public HlsContentSteeringTracker( checkNotNull(playlistTracker.getRedundantGroups(HlsRedundantGroup.AUDIO_RENDITION)); this.subtitleRenditionRedundantGroups = checkNotNull(playlistTracker.getRedundantGroups(HlsRedundantGroup.SUBTITLE_RENDITION)); - checkState(!variantRedundantGroups.isEmpty()); - availablePathwayIds = - new HashSet<>(checkNotNull(variantRedundantGroups.get(0)).getAllPathwayIds()); - excludedPathwayIds = new HashSet<>(); - handler = clock.createHandler(Util.getCurrentOrMainLooper(), /* callback= */ null); - currentPathwayId = variantRedundantGroups.get(0).getCurrentPathwayId(); + this.handler = clock.createHandler(Util.getCurrentOrMainLooper(), /* callback= */ null); } /** - * {@inheritDoc} + * Excludes the current pathway for the given duration. * - * @throws IllegalStateException If the {@code initialPathwayId} is not declared in the redundant - * group. + * @param excludeDurationMs The duration in milliseconds to exclude the current pathway. + * @return Whether the current pathway was excluded. */ - @Override - public void start( - Uri initialSteeringManifestUri, - @Nullable String initialPathwayId, - MediaSourceEventListener.EventDispatcher eventDispatcher) { - if (initialPathwayId != null) { - checkState(availablePathwayIds.contains(initialPathwayId)); - currentPathwayId = initialPathwayId; - } - isActive = true; - notifyOnCurrentPathwayUpdated( - /* previousPathwayId= */ null, /* previousPathwayExcludeDurationMs= */ C.TIME_UNSET); - steeringManifestTracker.start( - initialSteeringManifestUri, new SteeringManifestTrackerCallback(), eventDispatcher); - } - - @Override public boolean excludeCurrentPathway(long excludeDurationMs) { - if (isActive && currentPathwayPriority != null) { - String previousPathwayId = currentPathwayId; + @Nullable ImmutableList currentPathwayPriority = getCurrentPathwayPriority(); + if (isActive() && currentPathwayPriority != null) { + String previousPathwayId = checkNotNull(currentPathwayId); performPathwayEvaluationAndUpdate( - /* previousPathwayIdExcludeDurationMs= */ excludeDurationMs); - if (!currentPathwayId.equals(previousPathwayId)) { + currentPathwayPriority, /* previousPathwayIdExcludeDurationMs= */ excludeDurationMs); + if (!Objects.equals(currentPathwayId, previousPathwayId)) { excludedPathwayIds.add(previousPathwayId); handler.postDelayed(() -> expireExclusion(previousPathwayId), excludeDurationMs); return true; @@ -181,34 +155,51 @@ public boolean excludeCurrentPathway(long excludeDurationMs) { return false; } + private void expireExclusion(String pathwayId) { + checkState(isActive()); + excludedPathwayIds.remove(pathwayId); + @Nullable ImmutableList currentPathwayPriority = getCurrentPathwayPriority(); + if (currentPathwayPriority != null) { + performPathwayEvaluationAndUpdate(currentPathwayPriority, C.TIME_UNSET); + } + } + @Override - public boolean isActive() { - return isActive; + protected void onStart(ImmutableList initialPathwayIds) { + checkState(initialPathwayIds.size() <= 1); + currentPathwayId = + !initialPathwayIds.isEmpty() + ? Iterables.getOnlyElement(initialPathwayIds) + : variantRedundantGroups.get(0).getCurrentPathwayId(); + notifyOnCurrentPathwayUpdated( + currentPathwayId, + /* previousPathwayId= */ null, + /* previousPathwayExcludeDurationMs= */ C.TIME_UNSET); } @Override - public void stop() { - stopInternal(); + protected boolean isPathwayAvailable(String pathwayId) { + return availablePathwayIds.contains(pathwayId); } - private void expireExclusion(String pathwayId) { - checkState(isActive); - excludedPathwayIds.remove(pathwayId); - if (currentPathwayPriority != null) { - performPathwayEvaluationAndUpdate(/* previousPathwayIdExcludeDurationMs= */ C.TIME_UNSET); - } + @Override + protected ImmutableMap getSteeringQueryParameters() { + return ImmutableMap.of( + PATHWAY_PARAM, + checkNotNull(currentPathwayId), + THROUGHPUT_PARAM, + String.valueOf(bandwidthMeter.getBitrateEstimate())); } - private void stopInternal() { - steeringManifestTracker.stop(); - handler.removeCallbacksAndMessages(null); - isActive = false; + @Override + protected void performPathwayEvaluation(ImmutableList pathwayPriority) { + performPathwayEvaluationAndUpdate(pathwayPriority, C.TIME_UNSET); } - @RequiresNonNull("currentPathwayPriority") - private void performPathwayEvaluationAndUpdate(long previousPathwayIdExcludeDurationMs) { + private void performPathwayEvaluationAndUpdate( + ImmutableList pathwayPriority, long previousPathwayIdExcludeDurationMs) { String previousPathwayId = currentPathwayId; - for (String pathwayId : currentPathwayPriority) { + for (String pathwayId : pathwayPriority) { if (previousPathwayIdExcludeDurationMs != C.TIME_UNSET && pathwayId.equals(previousPathwayId)) { continue; @@ -218,41 +209,39 @@ private void performPathwayEvaluationAndUpdate(long previousPathwayIdExcludeDura break; } } - if (!currentPathwayId.equals(previousPathwayId)) { - notifyOnCurrentPathwayUpdated(previousPathwayId, previousPathwayIdExcludeDurationMs); + if (!Objects.equals(currentPathwayId, previousPathwayId)) { + notifyOnCurrentPathwayUpdated( + checkNotNull(currentPathwayId), previousPathwayId, previousPathwayIdExcludeDurationMs); } } - private void performPathwayClones(ImmutableList pathwayClones) { - for (SteeringManifest.PathwayClone pathwayClone : pathwayClones) { - if (!availablePathwayIds.contains(pathwayClone.baseId) - || availablePathwayIds.contains(pathwayClone.id)) { - // We should ignore a PathwayClone if its baseId is unknown to the tracker, or the new - // pathwayId of the clone matches any existing pathwayId. - continue; - } - ImmutableList.Builder newPlaylistUrls = new ImmutableList.Builder<>(); - ImmutableList.Builder basePlaylistUrls = new ImmutableList.Builder<>(); - performPathwayClonesForVariants( - pathwayClone, variantRedundantGroups, newPlaylistUrls, basePlaylistUrls); - performPathwayClonesForRenditions( - pathwayClone, videoRenditionRedundantGroups, newPlaylistUrls, basePlaylistUrls); - performPathwayClonesForRenditions( - pathwayClone, audioRenditionRedundantGroups, newPlaylistUrls, basePlaylistUrls); - performPathwayClonesForRenditions( - pathwayClone, subtitleRenditionRedundantGroups, newPlaylistUrls, basePlaylistUrls); - availablePathwayIds.add(pathwayClone.id); - if (callback != null) { - callback.onNewPathwayAvailable( - pathwayClone.id, - pathwayClone.baseId, - newPlaylistUrls.build(), - basePlaylistUrls.build()); - } + @Override + protected void performPathwayClone(SteeringManifest.PathwayClone pathwayClone) { + availablePathwayIds.add(pathwayClone.id); + ImmutableList.Builder newPlaylistUrls = new ImmutableList.Builder<>(); + ImmutableList.Builder basePlaylistUrls = new ImmutableList.Builder<>(); + performPathwayCloneForVariants( + pathwayClone, variantRedundantGroups, newPlaylistUrls, basePlaylistUrls); + performPathwayCloneForRenditions( + pathwayClone, videoRenditionRedundantGroups, newPlaylistUrls, basePlaylistUrls); + performPathwayCloneForRenditions( + pathwayClone, audioRenditionRedundantGroups, newPlaylistUrls, basePlaylistUrls); + performPathwayCloneForRenditions( + pathwayClone, subtitleRenditionRedundantGroups, newPlaylistUrls, basePlaylistUrls); + if (callback != null) { + callback.onNewPathwayAvailable( + pathwayClone.id, pathwayClone.baseId, newPlaylistUrls.build(), basePlaylistUrls.build()); } } - private void performPathwayClonesForVariants( + @Override + protected void onStop() { + currentPathwayId = null; + handler.removeCallbacksAndMessages(null); + excludedPathwayIds.clear(); + } + + private void performPathwayCloneForVariants( SteeringManifest.PathwayClone pathwayClone, List variantRedundantGroups, ImmutableList.Builder newPlaylistUrls, @@ -261,19 +250,16 @@ private void performPathwayClonesForVariants( for (HlsRedundantGroup variantRedundantGroup : variantRedundantGroups) { String basePathwayId = pathwayClone.baseId; Uri basePlaylistUrl = checkNotNull(variantRedundantGroup.getPlaylistUrl(basePathwayId)); - @Nullable Uri newPlaylistUrl = perVariantUris.containsKey(variantRedundantGroup.groupKey.stableId) ? checkNotNull(perVariantUris.get(variantRedundantGroup.groupKey.stableId)) - : getUriBuilder(basePlaylistUrl, pathwayClone.uriReplacement).build(); - if (newPlaylistUrl != null) { - newPlaylistUrls.add(newPlaylistUrl); - basePlaylistUrls.add(basePlaylistUrl); - } + : applyUriReplacement(basePlaylistUrl, pathwayClone.uriReplacement); + newPlaylistUrls.add(newPlaylistUrl); + basePlaylistUrls.add(basePlaylistUrl); } } - private void performPathwayClonesForRenditions( + private void performPathwayCloneForRenditions( SteeringManifest.PathwayClone pathwayClone, List renditionRedundantGroups, ImmutableList.Builder newPlaylistUrls, @@ -282,72 +268,30 @@ private void performPathwayClonesForRenditions( for (HlsRedundantGroup renditionRedundantGroup : renditionRedundantGroups) { String basePathwayId = pathwayClone.baseId; Uri basePlaylistUrl = checkNotNull(renditionRedundantGroup.getPlaylistUrl(basePathwayId)); - @Nullable Uri newPlaylistUrl = perRenditionUris.containsKey(renditionRedundantGroup.groupKey.stableId) ? checkNotNull(perRenditionUris.get(renditionRedundantGroup.groupKey.stableId)) - : getUriBuilder(basePlaylistUrl, pathwayClone.uriReplacement).build(); - if (newPlaylistUrl != null) { - newPlaylistUrls.add(newPlaylistUrl); - basePlaylistUrls.add(basePlaylistUrl); - } + : applyUriReplacement(basePlaylistUrl, pathwayClone.uriReplacement); + newPlaylistUrls.add(newPlaylistUrl); + basePlaylistUrls.add(basePlaylistUrl); } } - private Uri.Builder getUriBuilder(Uri url, SteeringManifest.UriReplacement uriReplacement) { - Uri.Builder newUrlBuilder = url.buildUpon().clearQuery(); - if (uriReplacement.host != null) { - newUrlBuilder.authority(uriReplacement.host); - } - // Combine existing and new query parameters, giving precedence to new ones. - Map combinedParams = new TreeMap<>(); - // Add existing parameters. - for (String existingParamName : url.getQueryParameterNames()) { - combinedParams.put(existingParamName, checkNotNull(url.getQueryParameter(existingParamName))); - } - // Add new parameters, overwriting existing ones if keys clash. - combinedParams.putAll(uriReplacement.params); - // Append all parameters from the sorted map. - for (Map.Entry param : combinedParams.entrySet()) { - newUrlBuilder.appendQueryParameter(param.getKey(), param.getValue()); - } - return newUrlBuilder; + private static HlsRedundantGroup getFirstVariantRedundantGroup( + HlsPlaylistTracker playlistTracker) { + List variantRedundantGroups = + checkNotNull(playlistTracker.getRedundantGroups(HlsRedundantGroup.VARIANT)); + checkState(!variantRedundantGroups.isEmpty()); + return variantRedundantGroups.get(0); } private void notifyOnCurrentPathwayUpdated( - @Nullable String previousPathwayId, long previousPathwayExcludeDurationMs) { + String currentPathwayId, + @Nullable String previousPathwayId, + long previousPathwayExcludeDurationMs) { if (callback != null) { callback.onCurrentPathwayUpdated( currentPathwayId, previousPathwayId, previousPathwayExcludeDurationMs); } } - - private class SteeringManifestTrackerCallback implements SteeringManifestTracker.Callback { - - @Override - public ImmutableMap getSteeringQueryParameters() { - checkState(isActive); - return ImmutableMap.of( - PATHWAY_PARAM, - currentPathwayId, - THROUGHPUT_PARAM, - String.valueOf(bandwidthMeter.getBitrateEstimate())); - } - - @Override - public void onSteeringManifestUpdated(SteeringManifest steeringManifest) { - checkState(isActive); - performPathwayClones(steeringManifest.pathwayClones); - currentPathwayPriority = steeringManifest.pathwayPriority; - performPathwayEvaluationAndUpdate(/* previousPathwayIdExcludeDurationMs= */ C.TIME_UNSET); - } - - @Override - public void onSteeringManifestLoadError(IOException error, boolean canceled) { - checkState(isActive); - if (canceled && currentPathwayPriority == null) { - stopInternal(); - } - } - } } diff --git a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsExtractorFactory.java b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsExtractorFactory.java index 193d44d843e..6a77f4f8bee 100644 --- a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsExtractorFactory.java +++ b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsExtractorFactory.java @@ -27,7 +27,6 @@ import androidx.media3.extractor.Extractor; import androidx.media3.extractor.ExtractorInput; import androidx.media3.extractor.PositionHolder; -import androidx.media3.extractor.mp4.FragmentedMp4Extractor; import androidx.media3.extractor.text.SubtitleParser; import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.io.IOException; @@ -104,26 +103,6 @@ default HlsExtractorFactory experimentalParseSubtitlesDuringExtraction( return this; } - /** - * Sets the set of video codecs for which within GOP sample dependency information should be - * parsed as part of extraction. Defaults to H.264 and H.265. - * - *

    Having access to additional sample dependency information can speed up seeking. See {@link - * FragmentedMp4Extractor#FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES}. - * - *

    This method is experimental and will be renamed or removed in a future release. - * - * @param codecsToParseWithinGopSampleDependencies The set of codecs for which to parse within GOP - * sample dependency information. - * @return This factory, for convenience. - */ - @CanIgnoreReturnValue - @ExperimentalApi // TODO: b/470365670 - Remove method once config is enabled by default. - default HlsExtractorFactory experimentalSetCodecsToParseWithinGopSampleDependencies( - @C.VideoCodecFlags int codecsToParseWithinGopSampleDependencies) { - return this; - } - /** * Returns the output {@link Format} of emitted {@linkplain C#TRACK_TYPE_TEXT text samples} which * were originally in {@code sourceFormat}. diff --git a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsInterstitialsAdsLoader.java b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsInterstitialsAdsLoader.java index 7377dd2d9b9..291c3c24b35 100644 --- a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsInterstitialsAdsLoader.java +++ b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsInterstitialsAdsLoader.java @@ -53,7 +53,6 @@ import android.util.Pair; import androidx.annotation.Nullable; import androidx.annotation.VisibleForTesting; -import androidx.concurrent.futures.CallbackToFutureAdapter; import androidx.media3.common.AdPlaybackState; import androidx.media3.common.AdPlaybackState.AdGroup; import androidx.media3.common.AdPlaybackState.SkipInfo; @@ -88,17 +87,20 @@ import androidx.media3.exoplayer.upstream.LoadErrorHandlingPolicy; import androidx.media3.exoplayer.upstream.Loader; import androidx.media3.exoplayer.upstream.ParsingLoadable; -import com.google.common.base.Preconditions; +import com.google.common.base.Supplier; import com.google.common.collect.ImmutableList; import com.google.common.util.concurrent.ListenableFuture; +import com.google.common.util.concurrent.SettableFuture; import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.io.IOException; import java.util.ArrayList; import java.util.Collection; import java.util.HashMap; import java.util.HashSet; +import java.util.Iterator; import java.util.List; import java.util.Map; +import java.util.NavigableMap; import java.util.Objects; import java.util.Set; import java.util.TreeMap; @@ -954,6 +956,7 @@ private boolean setWithAssetListReset( /* adIndexInAdGroup= */ 0, adGroup.timeUs, checkNotNull(lastProcessedPlaylist).targetDurationUs); + session.removeAssetListDurations(interstitial.id); putAndNotifyAdPlaybackStateUpdate(session.adsId, adPlaybackState); AdsConfiguration currentAdsConfiguration = @@ -1035,29 +1038,7 @@ public boolean setWithAssetListReset(Object adsId, String interstitialsId) { */ @CanIgnoreReturnValue public ListenableFuture setWithAssetListResetAsync(Object adsId, int adGroupIndex) { - return CallbackToFutureAdapter.getFuture( - completer -> { - // Create the task to be posted to the player looper - Runnable resetTask = - () -> { - try { - // Execute the existing synchronous reset logic - boolean result = setWithAssetListReset(adsId, adGroupIndex); - completer.set(result); - } catch (RuntimeException e) { - completer.setException(e); - } - }; - - Handler handler = this.handler; - handler.post(resetTask); - - // Provide a cancellation listener to remove the task if the future is cancelled - completer.addCancellationListener( - () -> handler.removeCallbacks(resetTask), directExecutor()); - - return "setWithAssetListReset for adsId=" + adsId + " at index=" + adGroupIndex; - }); + return postResetTask(() -> setWithAssetListReset(adsId, adGroupIndex)); } /** @@ -1085,32 +1066,31 @@ public ListenableFuture setWithAssetListResetAsync(Object adsId, int ad @CanIgnoreReturnValue public ListenableFuture setWithAssetListResetAsync( Object adsId, String interstitialsId) { - return CallbackToFutureAdapter.getFuture( - completer -> { - // Create the task to be posted to the player looper - Runnable resetTask = - () -> { - try { - // Execute the existing synchronous reset logic - boolean result = setWithAssetListReset(adsId, interstitialsId); - completer.set(result); - } catch (RuntimeException e) { - completer.setException(e); - } - }; + return postResetTask(() -> setWithAssetListReset(adsId, interstitialsId)); + } - Handler handler = this.handler; - handler.post(resetTask); + private ListenableFuture postResetTask(Supplier resetAction) { + SettableFuture future = SettableFuture.create(); + Runnable resetTask = + () -> { + try { + future.set(resetAction.get()); + } catch (RuntimeException e) { + future.setException(e); + } + }; - // Provide a cancellation listener to remove the task if the future is cancelled - completer.addCancellationListener( - () -> handler.removeCallbacks(resetTask), directExecutor()); + handler.post(resetTask); - return "setWithAssetListReset for adsId=" - + adsId - + " with interstitialsId=" - + interstitialsId; - }); + future.addListener( + () -> { + if (future.isCancelled()) { + handler.removeCallbacks(resetTask); + } + }, + directExecutor()); + + return future; } private boolean validateAdGroupForReset(Interstitial interstitial, AdGroup adGroup) { @@ -1238,18 +1218,17 @@ public boolean handleContentTimelineChanged(AdsMediaSource adsMediaSource, Timel Window window = timeline.getWindow(0, new Window()); if (window.manifest instanceof HlsManifest) { HlsMediaPlaylist mediaPlaylist = ((HlsManifest) window.manifest).mediaPlaylist; + session.lastProcessedPlaylist = mediaPlaylist; int assetListCount = session.unresolvedAssetLists.size(); adPlaybackState = window.isLive() ? mapInterstitialsForLive( - window.mediaItem, mediaPlaylist, adPlaybackState, session, window.positionInFirstPeriodUs, window.defaultPositionUs) : mapInterstitialsForVod( - window.mediaItem, mediaPlaylist, adPlaybackState, session, @@ -1284,7 +1263,6 @@ public boolean handleContentTimelineChanged(AdsMediaSource adsMediaSource, Timel maybeExecuteOrSetNextAssetListResolutionMessage( adsId, timeline, /* windowIndex= */ 0, publicPositionInFirstPeriod, contentPositionUs); } - session.lastProcessedPlaylist = mediaPlaylist; } boolean adPlaybackStateUpdated = putAndNotifyAdPlaybackStateUpdate(adsId, adPlaybackState); if (!adsMediaSourceSessionManager.isUnsupportedContentMediaSource(adsId)) { @@ -1395,11 +1373,12 @@ private void startLoadingAssetList(AssetListData assetListData) { cancelPendingAssetListResolutionMessage(); getLoader() .startLoading( - new ParsingLoadable<>( - dataSourceFactory.createDataSource(), - checkNotNull(assetListData.interstitial.assetListUri), - C.DATA_TYPE_AD, - new AssetListParser()), + new ParsingLoadable.Builder<>( + dataSourceFactory.createDataSource(), + checkNotNull(assetListData.interstitial.assetListUri), + C.DATA_TYPE_AD, + new AssetListParser()) + .build(), new LoaderCallback(assetListData), /* defaultMinRetryCount= */ 1); notifyListeners( @@ -1576,7 +1555,6 @@ private static boolean isHlsMediaItem(MediaItem mediaItem) { } private AdPlaybackState mapInterstitialsForLive( - MediaItem mediaItem, HlsMediaPlaylist mediaPlaylist, AdPlaybackState adPlaybackState, HlsAdSession session, @@ -1636,7 +1614,6 @@ private AdPlaybackState mapInterstitialsForLive( adPlaybackState = insertOrUpdateInterstitialInAdGroup( mediaPlaylist, - mediaItem, interstitial, adPlaybackState, /* adGroupIndex= */ insertionIndex, @@ -1651,35 +1628,43 @@ private AdPlaybackState mapInterstitialsForLive( private AdPlaybackState maybeResolvePendingSnapInResolutions( AdPlaybackState adPlaybackState, HlsMediaPlaylist mediaPlaylist, HlsAdSession session) { long endOfPlaylistUs = mediaPlaylist.startTimeUs + mediaPlaylist.durationUs; - List pendingSnapInResolutions = session.pendingSnapInResolutions; - int resolvedIndex = C.INDEX_UNSET; - for (int i = 0; i < pendingSnapInResolutions.size(); i++) { - PendingSnapInResolution pendingSnapInResolution = pendingSnapInResolutions.get(i); - if (pendingSnapInResolution.resumeTimeUs > endOfPlaylistUs) { - break; - } - Interstitial interstitial = pendingSnapInResolution.interstitial; - // Resolve the resume offset according to the snap position of the segement start - long resolvedResumeOffsetUs = resolveInterstitialResumeOffsetUs(interstitial, mediaPlaylist); - AdGroup adGroup = adPlaybackState.getAdGroup(pendingSnapInResolution.adGroupIndex); - long interstitialDurationUs = - resolveInterstitialDurationUs(interstitial, /* defaultDurationUs= */ C.TIME_UNSET); - // The content resume offset that was used before resolving the actuals offset. - long oldResumeOffsetIncrementUs = - interstitial.resumeOffsetUs != C.TIME_UNSET - ? interstitial.resumeOffsetUs - : (interstitialDurationUs != C.TIME_UNSET ? interstitialDurationUs : 0L); - // Recalculate the resume offset of the group in case the interstitial offset has changed. - long correctedAdGroupContentResumeOffsetUs = - adGroup.contentResumeOffsetUs - oldResumeOffsetIncrementUs + resolvedResumeOffsetUs; - adPlaybackState = - adPlaybackState.withContentResumeOffsetUs( - pendingSnapInResolution.adGroupIndex, correctedAdGroupContentResumeOffsetUs); - resolvedIndex++; + NavigableMap headMap = + session.pendingSnapInResolutions.headMap(endOfPlaylistUs, /* inclusive= */ true); + if (headMap.isEmpty()) { + return adPlaybackState; } - if (resolvedIndex != C.INDEX_UNSET) { - // Remove resolved interstitials from the list of pending resolutions. - session.removePendingSnapInResolutionUntilIndexInclusive(resolvedIndex); + Iterator> iterator = headMap.entrySet().iterator(); + while (iterator.hasNext()) { + Map.Entry entry = iterator.next(); + PendingSnapInResolution pendingSnapInResolution = entry.getValue(); + // The head map is expected to have very few entries only. No need to care about the + // overhead here. + Interstitial interstitial = + getInterstitialById(mediaPlaylist, pendingSnapInResolution.interstitialId); + if (interstitial != null) { + // Once the asset list durations are set, we use these instead of what's in the playlist. + long adDurationsUs = session.getSumOfAssetListDurationsUs(interstitial.id); + if (adDurationsUs == C.TIME_UNSET) { + adDurationsUs = resolveInterstitialDurationUs(interstitial, /* defaultDurationUs= */ 0); + } + // Resolve the resume offset according to the snap position of the segment start. + long resolvedResumeOffsetUs = + resolveInterstitialResumeOffsetUs( + interstitial, /* defaultDurationUs= */ adDurationsUs, mediaPlaylist); + AdGroup adGroup = adPlaybackState.getAdGroup(pendingSnapInResolution.adGroupIndex); + // The content resume offset that was used before resolving the actual offset. + long oldResumeOffsetIncrementUs = + interstitial.resumeOffsetUs != C.TIME_UNSET + ? interstitial.resumeOffsetUs + : resolveInterstitialFallbackResumeOffsetUs(interstitial, adDurationsUs); + // Recalculate the resume offset of the group in case the interstitial offset has changed. + long correctedAdGroupContentResumeOffsetUs = + adGroup.contentResumeOffsetUs - oldResumeOffsetIncrementUs + resolvedResumeOffsetUs; + adPlaybackState = + adPlaybackState.withContentResumeOffsetUs( + pendingSnapInResolution.adGroupIndex, correctedAdGroupContentResumeOffsetUs); + } + iterator.remove(); } return adPlaybackState; } @@ -1710,7 +1695,6 @@ private LongSparseArray> filterAndSortWithResolvedStartPositi } private AdPlaybackState mapInterstitialsForVod( - MediaItem mediaItem, HlsMediaPlaylist mediaPlaylist, AdPlaybackState adPlaybackState, HlsAdSession session, @@ -1767,7 +1751,6 @@ private AdPlaybackState mapInterstitialsForVod( adPlaybackState = insertOrUpdateInterstitialInAdGroup( mediaPlaylist, - mediaItem, interstitial, adPlaybackState, adGroupIndex, @@ -1781,7 +1764,6 @@ private AdPlaybackState mapInterstitialsForVod( private AdPlaybackState insertOrUpdateInterstitialInAdGroup( HlsMediaPlaylist mediaPlaylist, - MediaItem mediaItem, Interstitial interstitial, AdPlaybackState adPlaybackState, int adGroupIndex, @@ -1809,20 +1791,9 @@ private AdPlaybackState insertOrUpdateInterstitialInAdGroup( } adDurations[adDurations.length - 1] = interstitial.playoutLimitUs; long resumeOffsetIncrementUs = - interstitial.resumeOffsetUs != C.TIME_UNSET - ? interstitial.resumeOffsetUs - : (interstitialDurationUs != C.TIME_UNSET ? interstitialDurationUs : 0L); - if (interstitial.snapTypes.contains(SNAP_TYPE_IN)) { - long resumeTimeUs = interstitial.startDateUnixUs + resumeOffsetIncrementUs; - if (resumeTimeUs < mediaPlaylist.startTimeUs + mediaPlaylist.durationUs) { - resumeOffsetIncrementUs = resolveInterstitialResumeOffsetUs(interstitial, mediaPlaylist); - } else { - // The segment at which to resume is not yet in the playlist. Deferring offset calculation. - session.pendingSnapInResolutions.add( - new PendingSnapInResolution(resumeTimeUs, adGroupIndex, interstitial)); - } - } - long resumeOffsetUs = adGroup.contentResumeOffsetUs + resumeOffsetIncrementUs; + getResumeOffsetUsOrDeferSnapInResolution( + interstitial, interstitialDurationUs, adGroupIndex, mediaPlaylist, session); + long resumeOffsetUs = max(adGroup.contentResumeOffsetUs, 0) + resumeOffsetIncrementUs; adPlaybackState = adPlaybackState .withAdCount(adGroupIndex, adIndexInAdGroup + 1) @@ -1860,6 +1831,45 @@ private AdPlaybackState insertOrUpdateInterstitialInAdGroup( return adPlaybackState; } + /** + * Returns the content resume offset in microseconds for a given interstitial or defers the + * calculation if the interstitial needs to snap into a segment that is not yet in the playlist. + * + *

    The {@link #maybeResolvePendingSnapInResolutions} method will handle deferred resolutions + * when the timeline updates with a playlist containing the snap-in segment. + * + * @param interstitial The {@link Interstitial}. + * @param interstitialDurationUs The resolved duration of the interstitial or {@link C#TIME_UNSET} + * if unknown. + * @param adGroupIndex The index of the ad group. + * @param mediaPlaylist The current {@link HlsMediaPlaylist}. + * @param session The {@link HlsAdSession}. + * @return The content resume offset in microseconds. + */ + private long getResumeOffsetUsOrDeferSnapInResolution( + Interstitial interstitial, + long interstitialDurationUs, + int adGroupIndex, + HlsMediaPlaylist mediaPlaylist, + HlsAdSession session) { + if (interstitial.resumeOffsetUs != C.TIME_UNSET) { + return interstitial.resumeOffsetUs; + } + long resumeOffsetUs = + resolveInterstitialFallbackResumeOffsetUs(interstitial, interstitialDurationUs); + if (interstitial.snapTypes.contains(SNAP_TYPE_IN)) { + long resumeTimeUs = interstitial.startDateUnixUs + resumeOffsetUs; + if (resumeTimeUs < mediaPlaylist.startTimeUs + mediaPlaylist.durationUs) { + resumeOffsetUs = + resolveInterstitialResumeOffsetUs(interstitial, interstitialDurationUs, mediaPlaylist); + } else { + // The segment at which to resume is not yet in the playlist. Deferring offset calculation. + session.addPendingSnapInResolution(resumeTimeUs, adGroupIndex, interstitial); + } + } + return resumeOffsetUs; + } + private static int getLowestValidAdGroupInsertionIndex(AdPlaybackState adPlaybackState) { for (int adGroupIndex = adPlaybackState.adGroupCount - 1; adGroupIndex >= adPlaybackState.removedAdGroupCount; @@ -1876,9 +1886,7 @@ private static int getLowestValidAdGroupInsertionIndex(AdPlaybackState adPlaybac private static long resolveInterstitialDurationUs( Interstitial interstitial, long defaultDurationUs) { - if (interstitial.playoutLimitUs != C.TIME_UNSET) { - return interstitial.playoutLimitUs; - } else if (interstitial.durationUs != C.TIME_UNSET) { + if (interstitial.durationUs != C.TIME_UNSET) { return interstitial.durationUs; } else if (interstitial.endDateUnixUs != C.TIME_UNSET) { return interstitial.endDateUnixUs - interstitial.startDateUnixUs; @@ -1888,6 +1896,27 @@ private static long resolveInterstitialDurationUs( return defaultDurationUs; } + /** + * Resolves the fallback content resume offset in microseconds when {@code X-RESUME-OFFSET} is not + * present on the {@link Interstitial} (see RFC 8216bis-22, Appendix D.2). + */ + private static long resolveInterstitialFallbackResumeOffsetUs( + Interstitial interstitial, long interstitialDurationUs) { + if (interstitialDurationUs == C.TIME_UNSET) { + // Total duration of the interstitial is unknown; X-PLAYOUT-LIMIT is an upper bound on playout + // time rather than the interstitial duration, so default to 0L. + return 0L; + } + if (interstitial.playoutLimitUs != C.TIME_UNSET) { + // RFC 8216bis-22, Appendix D.2: If X-RESUME-OFFSET is not present and X-PLAYOUT-LIMIT is less + // than the total duration of the interstitial, the resumption offset is the playout limit. + return min(interstitialDurationUs, interstitial.playoutLimitUs); + } + // RFC 8216bis-22, Appendix D.2: If X-RESUME-OFFSET is not present, its value is the duration + // of the interstitial. + return interstitialDurationUs; + } + private static long resolveInterstitialStartTimeUs( Interstitial interstitial, HlsMediaPlaylist mediaPlaylist, long defaultPositionUs) { if (interstitial.cue.contains(CUE_TRIGGER_PRE)) { @@ -1902,21 +1931,25 @@ private static long resolveInterstitialStartTimeUs( } private static long resolveInterstitialResumeOffsetUs( - Interstitial interstitial, HlsMediaPlaylist mediaPlaylist) { + Interstitial interstitial, long defaultDurationUs, HlsMediaPlaylist mediaPlaylist) { + long fallbackResumeOffsetUs = + resolveInterstitialFallbackResumeOffsetUs(interstitial, defaultDurationUs); if (interstitial.snapTypes.contains(SNAP_TYPE_IN)) { long resumeOffsetUs = interstitial.resumeOffsetUs != C.TIME_UNSET ? interstitial.resumeOffsetUs - : resolveInterstitialDurationUs(interstitial, /* defaultDurationUs= */ 0L); + : fallbackResumeOffsetUs; long startTimeUs = interstitial.snapTypes.contains(SNAP_TYPE_OUT) ? getClosestSegmentBoundaryUs(interstitial.startDateUnixUs, mediaPlaylist) : interstitial.startDateUnixUs; - return getClosestSegmentBoundaryUs(startTimeUs + resumeOffsetUs, mediaPlaylist) - startTimeUs; + return max( + 0L, + getClosestSegmentBoundaryUs(startTimeUs + resumeOffsetUs, mediaPlaylist) - startTimeUs); } else { return interstitial.resumeOffsetUs != C.TIME_UNSET ? interstitial.resumeOffsetUs - : resolveInterstitialDurationUs(interstitial, C.TIME_UNSET); + : fallbackResumeOffsetUs; } } @@ -1928,10 +1961,17 @@ private static long resolveInterstitialResumeOffsetUs( if (lastProcessedPlaylist == null) { return null; } - for (int i = 0; i < lastProcessedPlaylist.interstitials.size(); i++) { - if (id.equals(lastProcessedPlaylist.interstitials.get(i).id)) { - Interstitial interstitial = lastProcessedPlaylist.interstitials.get(i); - return interstitial.assetListUri != null ? interstitial : null; + Interstitial interstitial = getInterstitialById(lastProcessedPlaylist, id); + return interstitial != null && interstitial.assetListUri != null ? interstitial : null; + } + + @Nullable + private static Interstitial getInterstitialById( + HlsMediaPlaylist mediaPlaylist, String interstitialId) { + for (int i = 0; i < mediaPlaylist.interstitials.size(); i++) { + Interstitial interstitial = mediaPlaylist.interstitials.get(i); + if (interstitial.id.equals(interstitialId)) { + return interstitial; } } return null; @@ -2157,13 +2197,16 @@ private void markAdAsPlayedAndNotifyListeners( /** A session for each media source that was started to play ads with this ads loader. */ /* package */ static final class HlsAdSession { + @VisibleForTesting /* package */ final TreeMap unresolvedAssetLists; + + @VisibleForTesting + /* package */ final TreeMap pendingSnapInResolutions; + private final Object adsId; private final Set insertedInterstitialIds; private final MediaItem contentMediaItem; @Nullable private final EventListener eventListener; - - @VisibleForTesting /* package */ final TreeMap unresolvedAssetLists; - @VisibleForTesting /* package */ final List pendingSnapInResolutions; + private final Map assetListDurations = new HashMap<>(); private AdPlaybackState adPlaybackState; @Nullable private HlsMediaPlaylist lastProcessedPlaylist; @@ -2176,7 +2219,7 @@ private void markAdAsPlayedAndNotifyListeners( this.eventListener = eventListener; insertedInterstitialIds = new HashSet<>(); unresolvedAssetLists = new TreeMap<>(); - pendingSnapInResolutions = new ArrayList<>(); + pendingSnapInResolutions = new TreeMap<>(); adPlaybackState = AdPlaybackState.NONE; awaitingFirstAdToStart = true; } @@ -2187,12 +2230,6 @@ private void markAdAsPlayedAndNotifyListeners( return entry != null ? entry.getValue() : null; } - /* package */ void removePendingSnapInResolutionUntilIndexInclusive(int resolvedIndex) { - Preconditions.checkArgument(resolvedIndex >= 0); - resolvedIndex = min(resolvedIndex, pendingSnapInResolutions.size() - 1); - pendingSnapInResolutions.subList(0, resolvedIndex + 1).clear(); - } - /* package */ void addUnresolvedAssetList( Interstitial interstitial, int adGroupIndex, @@ -2220,6 +2257,91 @@ private void markAdAsPlayedAndNotifyListeners( adGroupTimeUs == C.TIME_END_OF_SOURCE ? Long.MAX_VALUE : adGroupTimeUs; return unresolvedAssetLists.remove(adGroupInsertionTimeUs); } + + /** + * Sets the durations that resulted from fetching the asset list. + * + *

    Available asset list durations take precedence over the durations that were reported by + * the HLS playlist. + * + * @param interstitialId The {@linkplain Interstitial#id ID of the interstitital}. + * @param durations The durations of the interstitial as declared by the asset list. + */ + private void setAssetListDurations(String interstitialId, long[] durations) { + assetListDurations.put(interstitialId, durations); + } + + /** + * Removes the durations set for a given interstitial ID. + * + *

    This is called if an asset list of a given interstitial is {@link + * #setWithAssetListReset(Object, String) reset by the user}. + * + * @param interstitialId The {@linkplain Interstitial#id ID of the interstitital}. + */ + private void removeAssetListDurations(String interstitialId) { + assetListDurations.remove(interstitialId); + } + + /** + * Returns the sum of the durations that were declared in the asset list after resolving the + * asset, or {link C#TIME_UNSET} if no durations are available. If available, asset list + * durations take precedence over the durations that were reported in the HLS playlist. + * + * @param interstitialId The ID of the interstitial. + * @return The sum of the durations or {@link C#TIME_UNSET} if the asset list has not yet been + * resolved, or the given interstitial is not an asset list interstitial. + */ + private long getSumOfAssetListDurationsUs(String interstitialId) { + long totalDurationUs = 0; + long[] durations = assetListDurations.get(interstitialId); + if (durations == null) { + return C.TIME_UNSET; + } + for (long durationUs : durations) { + totalDurationUs += durationUs; + } + return totalDurationUs; + } + + /** + * Adds a pending snap-in resolution by making sure that there are no duplicates for the same + * interstitial. + * + *

    Note that the same interstitial may have a different {@code resumeTimeUs} when the + * duration is different. This can be the case for the same interstitial when calculating the + * resume time and the duration in the asset list is different than the duration in the + * playlist. + * + * @param resumeTimeUs The estimated resumption time of the interstitials that is used to snap + * to the corresponding nearest segment boundary. + * @param adGroupIndex The index of the ad group. + * @param interstitial The interstitial. + */ + @VisibleForTesting + /* package */ void addPendingSnapInResolution( + long resumeTimeUs, int adGroupIndex, Interstitial interstitial) { + removePendingSnapInResolution(adGroupIndex, interstitial); + pendingSnapInResolutions.put( + resumeTimeUs, new PendingSnapInResolution(adGroupIndex, interstitial.id)); + } + + @CanIgnoreReturnValue + private boolean removePendingSnapInResolution(int adGroupIndex, Interstitial interstitial) { + PendingSnapInResolution pendingSnapInResolution = + new PendingSnapInResolution(adGroupIndex, interstitial.id); + // Linear iteration is fine to search for duplicates as we expect very few items in the map. + Iterator> iterator = + pendingSnapInResolutions.entrySet().iterator(); + while (iterator.hasNext()) { + Map.Entry entry = iterator.next(); + if (pendingSnapInResolution.equals(entry.getValue())) { + iterator.remove(); + return true; + } + } + return false; + } } /** A manager that manages active sessions of media sources that play ads. */ @@ -2312,14 +2434,16 @@ public void onLoadCompleted( Pair result = checkNotNull(loadable.getResult()); @Nullable AssetList assetList = result.first; HlsAdSession session = adsMediaSourceSessionManager.getSession(assetListData.adsId); - AdPlaybackState adPlaybackState = session != null ? session.adPlaybackState : null; + if (session == null) { + // Stale asset list load finished after session was removed. + return; + } + AdPlaybackState adPlaybackState = session.adPlaybackState; // Get the state of the ad to validate there was no manual change since we started // loading. int assetListAdState = - adPlaybackState != null - ? adPlaybackState.getAdGroup(assetListData.adGroupIndex) - .states[assetListData.adIndexInAdGroup] - : AD_STATE_ERROR; + adPlaybackState.getAdGroup(assetListData.adGroupIndex) + .states[assetListData.adIndexInAdGroup]; if (assetListAdState != AD_STATE_UNAVAILABLE) { // The ad was manipulated manually since the asset loading was started. Ignore asset // list and make sure the next asset list is scheduled for loading (if any). @@ -2353,13 +2477,23 @@ public void onLoadCompleted( adGroup = adPlaybackState.getAdGroup(assetListData.adGroupIndex); } int adIndex = assetListData.adIndexInAdGroup; + long[] assetDurations = new long[assetList.assets.size()]; long[] newDurationsUs = adGroup.durationsUs.clone(); + long remainingPlayoutLimitUs = assetListData.interstitial.playoutLimitUs; for (int i = 0; i < assetList.assets.size(); i++) { Asset asset = assetList.assets.get(i); if (i > 0) { + // TODO: b/570090695 - Support expanding an asset list when other interstitials are + // already in the same AdGroup without interleaving ads. adIndex = oldAdCount + i - 1; } - newDurationsUs[adIndex] = asset.durationUs; + long adDurationUs = asset.durationUs; + if (remainingPlayoutLimitUs != C.TIME_UNSET) { + adDurationUs = min(adDurationUs, remainingPlayoutLimitUs); + remainingPlayoutLimitUs = max(0L, remainingPlayoutLimitUs - adDurationUs); + } + newDurationsUs[adIndex] = adDurationUs; + assetDurations[i] = asset.durationUs; sumOfAssetListAdDurationUs += asset.durationUs; MediaItem mediaItem = new MediaItem.Builder() @@ -2371,27 +2505,53 @@ public void onLoadCompleted( adPlaybackState .withAvailableAdMediaItem(assetListData.adGroupIndex, adIndex, mediaItem) .withAdId(assetListData.adGroupIndex, adIndex, assetListData.interstitial.id); + if (adDurationUs == 0L) { + adPlaybackState = + adPlaybackState.withSkippedAd(assetListData.adGroupIndex, adIndex); + } if (assetList.skipInfo != null) { adPlaybackState = adPlaybackState.withAdSkipInfo( assetListData.adGroupIndex, adIndex, assetList.skipInfo); } } + session.setAssetListDurations(assetListData.interstitial.id, assetDurations); adPlaybackState = adPlaybackState.withAdDurationsUs(assetListData.adGroupIndex, newDurationsUs); - if (assetListData.interstitial.resumeOffsetUs == C.TIME_UNSET) { - adGroup = adPlaybackState.getAdGroup(assetListData.adGroupIndex); - long oldAdContentResumeOffset = - resolveInterstitialDurationUs( - assetListData.interstitial, /* defaultDurationUs= */ 0); - long newContentResumeOffsetUs = - adGroup.contentResumeOffsetUs - - oldAdContentResumeOffset - + sumOfAssetListAdDurationUs; - adPlaybackState = - adPlaybackState.withContentResumeOffsetUs( - assetListData.adGroupIndex, newContentResumeOffsetUs); + HlsMediaPlaylist lastProcessedPlaylist = checkNotNull(session.lastProcessedPlaylist); + long unresolvedInterstitialDurationUs = + resolveInterstitialDurationUs( + assetListData.interstitial, /* defaultDurationUs= */ C.TIME_UNSET); + boolean hadPendingSnapInResolution = + session.removePendingSnapInResolution( + assetListData.adGroupIndex, assetListData.interstitial); + long oldResumeOffsetIncrementUs; + if (assetListData.interstitial.resumeOffsetUs != C.TIME_UNSET) { + oldResumeOffsetIncrementUs = assetListData.interstitial.resumeOffsetUs; + } else if (hadPendingSnapInResolution) { + oldResumeOffsetIncrementUs = + resolveInterstitialFallbackResumeOffsetUs( + assetListData.interstitial, unresolvedInterstitialDurationUs); + } else { + oldResumeOffsetIncrementUs = + resolveInterstitialResumeOffsetUs( + assetListData.interstitial, + unresolvedInterstitialDurationUs, + lastProcessedPlaylist); } + long resolvedResumeOffsetUs = + getResumeOffsetUsOrDeferSnapInResolution( + assetListData.interstitial, + sumOfAssetListAdDurationUs, + assetListData.adGroupIndex, + lastProcessedPlaylist, + session); + long correctedAdGroupContentResumeOffsetUs = + max(0L, adGroup.contentResumeOffsetUs - oldResumeOffsetIncrementUs) + + resolvedResumeOffsetUs; + adPlaybackState = + adPlaybackState.withContentResumeOffsetUs( + assetListData.adGroupIndex, correctedAdGroupContentResumeOffsetUs); putAndNotifyAdPlaybackStateUpdate(assetListData.adsId, adPlaybackState); notifyListeners( listener -> @@ -2538,15 +2698,32 @@ public int hashCode() { @VisibleForTesting /* package */ static class PendingSnapInResolution { - private final long resumeTimeUs; private final int adGroupIndex; - private final Interstitial interstitial; + private final String interstitialId; - /* package */ PendingSnapInResolution( - long resumeTimeUs, int adGroupIndex, Interstitial interstitial) { - this.resumeTimeUs = resumeTimeUs; + /* package */ PendingSnapInResolution(int adGroupIndex, String interstitialId) { this.adGroupIndex = adGroupIndex; - this.interstitial = interstitial; + this.interstitialId = interstitialId; + } + + @Override + public boolean equals(@Nullable Object o) { + if (this == o) { + return true; + } + if (!(o instanceof PendingSnapInResolution)) { + return false; + } + PendingSnapInResolution that = (PendingSnapInResolution) o; + return adGroupIndex == that.adGroupIndex + && Objects.equals(interstitialId, that.interstitialId); + } + + @Override + public int hashCode() { + int result = interstitialId.hashCode(); + result = 31 * result + adGroupIndex; + return result; } } } diff --git a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsMediaChunk.java b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsMediaChunk.java index 188b9de4eca..f6543df84ac 100644 --- a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsMediaChunk.java +++ b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsMediaChunk.java @@ -25,6 +25,7 @@ import androidx.media3.common.DrmInitData; import androidx.media3.common.Format; import androidx.media3.common.Metadata; +import androidx.media3.common.ParserException; import androidx.media3.common.util.ParsableByteArray; import androidx.media3.common.util.TimestampAdjuster; import androidx.media3.common.util.UriUtil; @@ -58,30 +59,31 @@ /** * Creates a new instance. * - * @param extractorFactory A {@link HlsExtractorFactory} from which the {@link - * HlsMediaChunkExtractor} is obtained. - * @param dataSource The source from which the data should be loaded. - * @param format The chunk format. - * @param startOfPlaylistInPeriodUs The position of the playlist in the period in microseconds. - * @param mediaPlaylist The media playlist from which this chunk was obtained. + * @param extractorFactory An {@link HlsExtractorFactory} for creating {@link + * HlsMediaChunkExtractor}s. + * @param dataSource The {@link DataSource} for loading the data. + * @param format The chunk {@link Format}. + * @param startOfPlaylistInPeriodUs The start time of the playlist in the period, in microseconds. + * @param mediaPlaylist The {@link HlsMediaPlaylist} from which this chunk was obtained. * @param segmentBaseHolder The segment holder. * @param playlistUrl The url of the playlist from which this chunk was obtained. + * @param steeredPathwayId The ID of the steered pathway from which data is being loaded, or + * {@code null} if not applicable. * @param muxedCaptionFormats List of muxed caption {@link Format}s. Null if no closed caption * information is available in the multivariant playlist. * @param trackSelectionReason See {@link #trackSelectionReason}. * @param trackSelectionData See {@link #trackSelectionData}. - * @param isPrimaryTimestampSource True if the chunk can initialize the timestamp adjuster. - * @param timestampAdjusterProvider The provider from which to obtain the {@link - * TimestampAdjuster}. - * @param timestampAdjusterInitializationTimeoutMs The timeout for the loading thread to wait for - * the timestamp adjuster to initialize, in milliseconds. A timeout of zero is interpreted as - * an infinite timeout. - * @param previousChunk The {@link HlsMediaChunk} that preceded this one. May be null. + * @param isPrimaryTimestampSource Whether this chunk is providing the timestamp source. + * @param timestampAdjusterProvider A provider of {@link TimestampAdjuster}s. + * @param timestampAdjusterInitializationTimeoutMs The timeout for waiting for the timestamp + * adjuster to be initialized, in milliseconds. + * @param previousChunk The previous chunk in the output, or null. * @param mediaSegmentKey The media segment decryption key, if fully encrypted. Null otherwise. * @param initSegmentKey The initialization segment decryption key, if fully encrypted. Null * otherwise. * @param shouldSpliceIn Whether samples for this chunk should be spliced into existing samples. * @param isIndependent Whether the chunk starts with a keyframe. + * @param playerId The {@link PlayerId} of the player. * @param cmcdDataFactory The {@link CmcdData.Factory} for generating {@link CmcdData}. */ public static HlsMediaChunk createInstance( @@ -92,6 +94,7 @@ public static HlsMediaChunk createInstance( HlsMediaPlaylist mediaPlaylist, HlsChunkSource.SegmentBaseHolder segmentBaseHolder, Uri playlistUrl, + @Nullable String steeredPathwayId, @Nullable List muxedCaptionFormats, @C.SelectionReason int trackSelectionReason, @Nullable Object trackSelectionData, @@ -215,7 +218,8 @@ public static HlsMediaChunk createInstance( scratchId3Data, shouldSpliceIn, isIndependent, - playerId); + playerId, + steeredPathwayId); } /** @@ -334,7 +338,8 @@ private HlsMediaChunk( ParsableByteArray scratchId3Data, boolean shouldSpliceIn, boolean isIndependent, - PlayerId playerId) { + PlayerId playerId, + @Nullable String steeredPathwayId) { super( mediaDataSource, dataSpec, @@ -343,7 +348,8 @@ private HlsMediaChunk( trackSelectionData, startTimeUs, endTimeUs, - chunkMediaSequence); + chunkMediaSequence, + steeredPathwayId); this.mediaSegmentEncrypted = mediaSegmentEncrypted; this.partIndex = partIndex; this.publishedDurationUs = isPublished ? endTimeUs - startTimeUs : C.TIME_UNSET; @@ -539,6 +545,11 @@ private void feedDataToExtractor( // See onTruncatedSegmentParsed's javadoc for more info on why we are swallowing the EOF // exception for trick play tracks. extractor.onTruncatedSegmentParsed(); + } else if (dataSpec.length != C.LENGTH_UNSET + && input.getPosition() >= dataSpec.position + dataSpec.length) { + // The chunk is fully loaded, but an EOFException was thrown. This can happen if the + // chunk is corrupted. + throw ParserException.createForMalformedContainer(/* message= */ null, e); } else { throw e; } diff --git a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsMediaPeriod.java b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsMediaPeriod.java index 501a5f7af29..02c0d38323c 100644 --- a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsMediaPeriod.java +++ b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsMediaPeriod.java @@ -57,7 +57,6 @@ import com.google.common.base.Supplier; import com.google.common.collect.ImmutableList; import com.google.common.collect.Lists; -import com.google.common.primitives.Ints; import java.io.IOException; import java.util.ArrayList; import java.util.Collections; @@ -100,9 +99,6 @@ private @MonotonicNonNull TrackGroupArray trackGroups; private HlsSampleStreamWrapper[] sampleStreamWrappers; private HlsSampleStreamWrapper[] enabledSampleStreamWrappers; - // Maps sample stream wrappers to variant/rendition redundant group index by matching array - // positions. - private int[][] redundantGroupIndicesPerWrapper; private int audioVideoSampleStreamWrapperCount; private SequenceableLoader compositeSequenceableLoader; private long endPositionUs; @@ -177,7 +173,6 @@ public HlsMediaPeriod( timestampAdjusterProvider = new TimestampAdjusterProvider(); sampleStreamWrappers = new HlsSampleStreamWrapper[0]; enabledSampleStreamWrappers = new HlsSampleStreamWrapper[0]; - redundantGroupIndicesPerWrapper = new int[0][]; endPositionUs = C.TIME_END_OF_SOURCE; } @@ -217,27 +212,17 @@ public List getStreamKeys(List trackSelections) { // See HlsMultivariantPlaylist.copy for interpretation of StreamKeys. HlsMultivariantPlaylist multivariantPlaylist = checkNotNull(playlistTracker.getMultivariantPlaylist()); - List variantRedundantGroups = - checkNotNull(playlistTracker.getRedundantGroups(HlsRedundantGroup.VARIANT)); - List audioRenditionRedundantGroups = - checkNotNull(playlistTracker.getRedundantGroups(HlsRedundantGroup.AUDIO_RENDITION)); - List subtitleRenditionRedundantGroups = - checkNotNull(playlistTracker.getRedundantGroups(HlsRedundantGroup.SUBTITLE_RENDITION)); boolean hasVariants = !multivariantPlaylist.variants.isEmpty(); int audioWrapperOffset = hasVariants ? 1 : 0; - TrackGroupArray mainWrapperTrackGroups; - int mainWrapperPrimaryTrackGroupIndex; - int[] mainWrapperRedundantGroupIndices; + @Nullable HlsSampleStreamWrapper mainWrapper; + @Nullable TrackGroupArray mainWrapperTrackGroups; if (hasVariants) { - HlsSampleStreamWrapper mainWrapper = sampleStreamWrappers[0]; - mainWrapperRedundantGroupIndices = redundantGroupIndicesPerWrapper[0]; + mainWrapper = sampleStreamWrappers[0]; mainWrapperTrackGroups = mainWrapper.getTrackGroups(); - mainWrapperPrimaryTrackGroupIndex = mainWrapper.getPrimaryTrackGroupIndex(); } else { - mainWrapperRedundantGroupIndices = new int[0]; - mainWrapperTrackGroups = TrackGroupArray.EMPTY; - mainWrapperPrimaryTrackGroupIndex = 0; + mainWrapper = null; + mainWrapperTrackGroups = null; } List streamKeys = new ArrayList<>(); @@ -245,18 +230,21 @@ public List getStreamKeys(List trackSelections) { boolean hasPrimaryTrackGroupSelection = false; for (ExoTrackSelection trackSelection : trackSelections) { TrackGroup trackSelectionGroup = trackSelection.getTrackGroup(); - int mainWrapperTrackGroupIndex = mainWrapperTrackGroups.indexOf(trackSelectionGroup); + int mainWrapperTrackGroupIndex = + mainWrapperTrackGroups != null + ? mainWrapperTrackGroups.indexOf(trackSelectionGroup) + : C.INDEX_UNSET; if (mainWrapperTrackGroupIndex != C.INDEX_UNSET) { - if (mainWrapperTrackGroupIndex == mainWrapperPrimaryTrackGroupIndex) { + if (mainWrapperTrackGroupIndex == mainWrapper.getPrimaryTrackGroupIndex()) { // Primary group in main wrapper. hasPrimaryTrackGroupSelection = true; + ImmutableList mainWrapperRedundantGroups = + mainWrapper.getRedundantGroups(); for (int i = 0; i < trackSelection.length(); i++) { - int variantRedundantGroupIndex = - mainWrapperRedundantGroupIndices[trackSelection.getIndexInTrackGroup(i)]; - HlsRedundantGroup variantRedundantGroup = - variantRedundantGroups.get(variantRedundantGroupIndex); + HlsRedundantGroup mainWrapperSelectedRedundantGroup = + mainWrapperRedundantGroups.get(trackSelection.getIndexInTrackGroup(i)); for (int variantUrlIndexInMultivariantPlaylist : - variantRedundantGroup.getIndicesInMultivariantPlaylist()) { + mainWrapperSelectedRedundantGroup.getIndicesInMultivariantPlaylist()) { streamKeys.add( new StreamKey( HlsMultivariantPlaylist.GROUP_INDEX_VARIANT, @@ -270,28 +258,26 @@ public List getStreamKeys(List trackSelections) { } else { // Audio or subtitle group. for (int i = audioWrapperOffset; i < sampleStreamWrappers.length; i++) { + HlsSampleStreamWrapper wrapper = sampleStreamWrappers[i]; TrackGroupArray wrapperTrackGroups = sampleStreamWrappers[i].getTrackGroups(); int selectedTrackGroupIndex = wrapperTrackGroups.indexOf(trackSelectionGroup); if (selectedTrackGroupIndex != C.INDEX_UNSET) { - int groupIndexType = - wrapperTrackGroups.get(selectedTrackGroupIndex).type == C.TRACK_TYPE_AUDIO - ? HlsMultivariantPlaylist.GROUP_INDEX_AUDIO - : HlsMultivariantPlaylist.GROUP_INDEX_SUBTITLE; - List renditionRedundantGroups = - (groupIndexType == HlsMultivariantPlaylist.GROUP_INDEX_AUDIO) - ? audioRenditionRedundantGroups - : subtitleRenditionRedundantGroups; - int[] selectedWrapperRedundantGroupIndices = redundantGroupIndicesPerWrapper[i]; - for (int trackIndex = 0; trackIndex < trackSelection.length(); trackIndex++) { - int renditionRedundantGroupIndex = - selectedWrapperRedundantGroupIndices[ - trackSelection.getIndexInTrackGroup(trackIndex)]; - HlsRedundantGroup renditionRedundantGroup = - renditionRedundantGroups.get(renditionRedundantGroupIndex); - for (int renditionUrlIndexInMultivariantPlaylist : - renditionRedundantGroup.getIndicesInMultivariantPlaylist()) { - streamKeys.add( - new StreamKey(groupIndexType, renditionUrlIndexInMultivariantPlaylist)); + if (selectedTrackGroupIndex == wrapper.getPrimaryTrackGroupIndex()) { + // Primary group in the rendition wrapper. + int groupIndexType = + wrapperTrackGroups.get(selectedTrackGroupIndex).type == C.TRACK_TYPE_AUDIO + ? HlsMultivariantPlaylist.GROUP_INDEX_AUDIO + : HlsMultivariantPlaylist.GROUP_INDEX_SUBTITLE; + ImmutableList wrapperRedundantGroups = + wrapper.getRedundantGroups(); + for (int trackIndex = 0; trackIndex < trackSelection.length(); trackIndex++) { + HlsRedundantGroup wrapperSelectedRedundantGroup = + wrapperRedundantGroups.get(trackSelection.getIndexInTrackGroup(trackIndex)); + for (int renditionUrlIndexInMultivariantPlaylist : + wrapperSelectedRedundantGroup.getIndicesInMultivariantPlaylist()) { + streamKeys.add( + new StreamKey(groupIndexType, renditionUrlIndexInMultivariantPlaylist)); + } } } break; @@ -299,24 +285,25 @@ public List getStreamKeys(List trackSelections) { } } } - if (needsPrimaryTrackGroupSelection && !hasPrimaryTrackGroupSelection) { + if (hasVariants && needsPrimaryTrackGroupSelection && !hasPrimaryTrackGroupSelection) { // A track selection includes a variant-embedded track, but no variant is added yet. We use // the valid variant with the lowest bitrate to reduce overhead. - int lowestBitrateRedundantGroupIndex = mainWrapperRedundantGroupIndices[0]; + ImmutableList mainWrapperRedundantGroups = + mainWrapper.getRedundantGroups(); + int lowestBitrateRedundantGroupIndex = 0; int lowestBitrate = - variantRedundantGroups.get(mainWrapperRedundantGroupIndices[0]).groupKey.format.bitrate; - for (int i = 1; i < mainWrapperRedundantGroupIndices.length; i++) { - int variantBitrate = - variantRedundantGroups.get(mainWrapperRedundantGroupIndices[i]).groupKey.format.bitrate; + mainWrapperRedundantGroups.get(lowestBitrateRedundantGroupIndex).groupKey.format.bitrate; + for (int i = 1; i < mainWrapperRedundantGroups.size(); i++) { + int variantBitrate = mainWrapperRedundantGroups.get(i).groupKey.format.bitrate; if (variantBitrate < lowestBitrate) { lowestBitrate = variantBitrate; - lowestBitrateRedundantGroupIndex = mainWrapperRedundantGroupIndices[i]; + lowestBitrateRedundantGroupIndex = i; } } - HlsRedundantGroup variantRedundantGroupWithLowestBitrate = - variantRedundantGroups.get(lowestBitrateRedundantGroupIndex); + HlsRedundantGroup mainWrapperRedundantGroupWithLowestBitrate = + mainWrapperRedundantGroups.get(lowestBitrateRedundantGroupIndex); for (int variantUrlIndexInMultivariantPlaylist : - variantRedundantGroupWithLowestBitrate.getIndicesInMultivariantPlaylist()) { + mainWrapperRedundantGroupWithLowestBitrate.getIndicesInMultivariantPlaylist()) { streamKeys.add( new StreamKey( HlsMultivariantPlaylist.GROUP_INDEX_VARIANT, @@ -552,7 +539,6 @@ private void buildAndPrepareSampleStreamWrappers(long positionUs) { pendingPrepareCount = 0; ArrayList sampleStreamWrappers = new ArrayList<>(); - ArrayList redundantGroupIndicesPerWrapper = new ArrayList<>(); if (hasVariants) { buildAndPrepareMainSampleStreamWrapper( @@ -562,30 +548,20 @@ private void buildAndPrepareSampleStreamWrappers(long positionUs) { multivariantPlaylist.muxedCaptionFormats, positionUs, sampleStreamWrappers, - redundantGroupIndicesPerWrapper, overridingDrmInitData); } // TODO: Build video stream wrappers here. buildAndPrepareAudioSampleStreamWrappers( - positionUs, - audioRedundantGroups, - sampleStreamWrappers, - redundantGroupIndicesPerWrapper, - overridingDrmInitData); + positionUs, audioRedundantGroups, sampleStreamWrappers, overridingDrmInitData); audioVideoSampleStreamWrapperCount = sampleStreamWrappers.size(); buildAndPrepareSubtitleSampleStreamWrappers( - positionUs, - subtitleRedundantGroups, - sampleStreamWrappers, - redundantGroupIndicesPerWrapper, - overridingDrmInitData); + positionUs, subtitleRedundantGroups, sampleStreamWrappers, overridingDrmInitData); this.sampleStreamWrappers = sampleStreamWrappers.toArray(new HlsSampleStreamWrapper[0]); - this.redundantGroupIndicesPerWrapper = redundantGroupIndicesPerWrapper.toArray(new int[0][]); pendingPrepareCount = this.sampleStreamWrappers.length; // Set primary timestamp source and trigger preparation (if not already prepared) for (int i = 0; i < audioVideoSampleStreamWrapperCount; i++) { @@ -630,8 +606,6 @@ private void buildAndPrepareSampleStreamWrappers(long positionUs) { * @param positionUs If preparation requires any chunk downloads, the position in microseconds at * which downloading should start. Ignored otherwise. * @param sampleStreamWrappers List to which the built main sample stream wrapper should be added. - * @param redundantGroupIndicesPerWrapper List to which the selected variant redundant group - * indices should be added. * @param overridingDrmInitData Overriding {@link DrmInitData}, keyed by protection scheme type * (i.e. {@link DrmInitData#schemeType}). */ @@ -642,7 +616,6 @@ private void buildAndPrepareMainSampleStreamWrapper( @Nullable List muxedCaptionFormats, long positionUs, List sampleStreamWrappers, - List redundantGroupIndicesPerWrapper, Map overridingDrmInitData) { int[] variantTypes = new int[variantRedundantGroups.size()]; int videoVariantCount = 0; @@ -676,7 +649,6 @@ private void buildAndPrepareMainSampleStreamWrapper( } HlsRedundantGroup[] selectedRedundantGroups = new HlsRedundantGroup[selectedVariantsCount]; Format[] selectedRedundantGroupFormats = new Format[selectedVariantsCount]; - int[] selectedVariantRedundantGroupIndices = new int[selectedVariantsCount]; int outIndex = 0; for (int i = 0; i < variantRedundantGroups.size(); i++) { if ((!useVideoVariantsOnly || variantTypes[i] == C.TRACK_TYPE_VIDEO) @@ -684,7 +656,7 @@ private void buildAndPrepareMainSampleStreamWrapper( HlsRedundantGroup variantRedundantGroup = variantRedundantGroups.get(i); selectedRedundantGroups[outIndex] = variantRedundantGroup; selectedRedundantGroupFormats[outIndex] = variantRedundantGroup.groupKey.format; - selectedVariantRedundantGroupIndices[outIndex++] = i; + outIndex++; } } String codecs = selectedRedundantGroupFormats[0].codecs; @@ -711,7 +683,6 @@ private void buildAndPrepareMainSampleStreamWrapper( overridingDrmInitData, positionUs); sampleStreamWrappers.add(sampleStreamWrapper); - redundantGroupIndicesPerWrapper.add(selectedVariantRedundantGroupIndices); if (allowChunklessPreparation && codecsStringAllowsChunklessPreparation) { List muxedTrackGroups = new ArrayList<>(); if (numberOfVideoCodecs > 0) { @@ -781,14 +752,11 @@ private void buildAndPrepareAudioSampleStreamWrappers( long positionUs, List audioRedundantGroups, List sampleStreamWrappers, - List redundantGroupIndicesPerWrapper, Map overridingDrmInitData) { ArrayList scratchRedundantGroups = new ArrayList<>(/* initialCapacity= */ audioRedundantGroups.size()); ArrayList scratchRedundantGroupFormats = new ArrayList<>(/* initialCapacity= */ audioRedundantGroups.size()); - ArrayList scratchIndicesList = - new ArrayList<>(/* initialCapacity= */ audioRedundantGroups.size()); HashSet alreadyGroupedNames = new HashSet<>(); for (int renditionByNameIndex = 0; renditionByNameIndex < audioRedundantGroups.size(); @@ -802,7 +770,6 @@ private void buildAndPrepareAudioSampleStreamWrappers( boolean codecStringsAllowChunklessPreparation = true; scratchRedundantGroups.clear(); scratchRedundantGroupFormats.clear(); - scratchIndicesList.clear(); // Group all redundantGroups with matching name. for (int redundantGroupIndex = 0; redundantGroupIndex < audioRedundantGroups.size(); @@ -810,7 +777,6 @@ private void buildAndPrepareAudioSampleStreamWrappers( if (Objects.equals(name, audioRedundantGroups.get(redundantGroupIndex).groupKey.name)) { HlsRedundantGroup audioRedundantGroup = audioRedundantGroups.get(redundantGroupIndex); Format format = audioRedundantGroup.groupKey.format; - scratchIndicesList.add(redundantGroupIndex); scratchRedundantGroups.add(audioRedundantGroup); scratchRedundantGroupFormats.add(format); codecStringsAllowChunklessPreparation &= @@ -829,7 +795,6 @@ private void buildAndPrepareAudioSampleStreamWrappers( /* muxedCaptionFormats= */ Collections.emptyList(), overridingDrmInitData, positionUs); - redundantGroupIndicesPerWrapper.add(Ints.toArray(scratchIndicesList)); sampleStreamWrappers.add(sampleStreamWrapper); if (allowChunklessPreparation && codecStringsAllowChunklessPreparation) { @@ -856,14 +821,11 @@ private void buildAndPrepareSubtitleSampleStreamWrappers( long positionUs, List subtitleRedundantGroups, List sampleStreamWrappers, - List redundantGroupIndicesPerWrapper, Map overridingDrmInitData) { ArrayList scratchRedundantGroups = new ArrayList<>(/* initialCapacity= */ subtitleRedundantGroups.size()); ArrayList scratchRedundantGroupFormats = new ArrayList<>(/* initialCapacity= */ subtitleRedundantGroups.size()); - ArrayList scratchIndicesList = - new ArrayList<>(/* initialCapacity= */ subtitleRedundantGroups.size()); HashSet alreadyGroupedNames = new HashSet<>(); for (int renditionByNameIndex = 0; renditionByNameIndex < subtitleRedundantGroups.size(); @@ -876,7 +838,6 @@ private void buildAndPrepareSubtitleSampleStreamWrappers( scratchRedundantGroups.clear(); scratchRedundantGroupFormats.clear(); - scratchIndicesList.clear(); // Group all redundantGroups with matching name. for (int redundantGroupIndex = 0; redundantGroupIndex < subtitleRedundantGroups.size(); @@ -884,7 +845,6 @@ private void buildAndPrepareSubtitleSampleStreamWrappers( if (Objects.equals(name, subtitleRedundantGroups.get(redundantGroupIndex).groupKey.name)) { HlsRedundantGroup subtitleRedundantGroup = subtitleRedundantGroups.get(redundantGroupIndex); - scratchIndicesList.add(redundantGroupIndex); scratchRedundantGroups.add(subtitleRedundantGroup); scratchRedundantGroupFormats.add(subtitleRedundantGroup.groupKey.format); } @@ -902,7 +862,6 @@ private void buildAndPrepareSubtitleSampleStreamWrappers( /* muxedCaptionFormats= */ ImmutableList.of(), overridingDrmInitData, positionUs); - redundantGroupIndicesPerWrapper.add(Ints.toArray(scratchIndicesList)); sampleStreamWrappers.add(sampleStreamWrapper); Format[] outputTextFormats = new Format[originalSubtitleFormats.length]; diff --git a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsMediaSource.java b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsMediaSource.java index 90a989d867e..d55adb2d407 100644 --- a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsMediaSource.java +++ b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsMediaSource.java @@ -110,7 +110,6 @@ public static final class Factory implements MediaSourceFactory { @Nullable private HlsExtractorFactory extractorFactory; @Nullable private SubtitleParser.Factory subtitleParserFactoryOverride; private boolean parseSubtitlesDuringExtraction; - private @C.VideoCodecFlags int codecsToParseWithinGopSampleDependencies; private HlsPlaylistParserFactory playlistParserFactory; private HlsPlaylistTracker.Factory playlistTrackerFactory; private CompositeSequenceableLoaderFactory compositeSequenceableLoaderFactory; @@ -175,7 +174,6 @@ public Factory(HlsDataSourceFactory hlsDataSourceFactory) { metadataType = METADATA_TYPE_ID3; elapsedRealTimeOffsetMs = C.TIME_UNSET; allowChunklessPreparation = true; - codecsToParseWithinGopSampleDependencies = C.VIDEO_CODEC_FLAG_H264 | C.VIDEO_CODEC_FLAG_H265; experimentalParseSubtitlesDuringExtraction(true); } @@ -225,14 +223,6 @@ public Factory experimentalParseSubtitlesDuringExtraction( return this; } - @Override - @CanIgnoreReturnValue - public Factory experimentalSetCodecsToParseWithinGopSampleDependencies( - @C.VideoCodecFlags int codecsToParseWithinGopSampleDependencies) { - this.codecsToParseWithinGopSampleDependencies = codecsToParseWithinGopSampleDependencies; - return this; - } - /** * Sets the factory from which playlist parsers will be obtained. * @@ -416,8 +406,6 @@ public HlsMediaSource createMediaSource(MediaItem mediaItem) { extractorFactory.setSubtitleParserFactory(subtitleParserFactoryOverride); } extractorFactory.experimentalParseSubtitlesDuringExtraction(parseSubtitlesDuringExtraction); - extractorFactory.experimentalSetCodecsToParseWithinGopSampleDependencies( - codecsToParseWithinGopSampleDependencies); HlsExtractorFactory extractorFactory = this.extractorFactory; HlsPlaylistParserFactory playlistParserFactory = this.playlistParserFactory; List streamKeys = mediaItem.localConfiguration.streamKeys; diff --git a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsSampleStream.java b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsSampleStream.java index d7bb951a8db..614d978c9f1 100644 --- a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsSampleStream.java +++ b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsSampleStream.java @@ -87,6 +87,11 @@ public int skipData(long positionUs) { : 0; } + @Override + public @Flags int getFlags() { + return sampleStreamWrapper.getStreamFlags(trackGroupIndex); + } + // Internal methods. private boolean hasValidSampleQueueIndex() { diff --git a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsSampleStreamWrapper.java b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsSampleStreamWrapper.java index 5d58cb51213..35fa63b7527 100644 --- a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsSampleStreamWrapper.java +++ b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/HlsSampleStreamWrapper.java @@ -49,6 +49,7 @@ import androidx.media3.exoplayer.drm.DrmSession; import androidx.media3.exoplayer.drm.DrmSessionEventListener; import androidx.media3.exoplayer.drm.DrmSessionManager; +import androidx.media3.exoplayer.hls.playlist.HlsRedundantGroup; import androidx.media3.exoplayer.source.LoadEventInfo; import androidx.media3.exoplayer.source.MediaLoadData; import androidx.media3.exoplayer.source.MediaSourceEventListener; @@ -735,6 +736,40 @@ public int skipData(int sampleQueueIndex, long positionUs) { return skipCount; } + /** + * Returns the stream flags for the given track group index. + * + * @param trackGroupIndex The index of the track group. + * @return The {@link SampleStream.Flags} for the stream. + */ + public @SampleStream.Flags int getStreamFlags(int trackGroupIndex) { + assertIsPrepared(); + Format playlistFormat = trackGroups.get(trackGroupIndex).getFormat(0); + if (SampleQueue.isDiscardingAllSamplesToStartTime(playlistFormat)) { + return 0; + } + if (mediaChunks.isEmpty()) { + return loadingFinished || loader.hasFatalError() ? 0 : SampleStream.FLAG_MAYBE_HAS_PREROLL; + } + long resetPositionUs = isPendingReset() ? pendingResetPositionUs : lastSeekPositionUs; + if (resetPositionUs == C.TIME_UNSET || mediaChunks.get(0).startTimeUs >= resetPositionUs) { + return 0; + } + if (trackGroupToSampleQueueIndex != null) { + int sampleQueueIndex = trackGroupToSampleQueueIndex[trackGroupIndex]; + if (sampleQueueIndex == C.INDEX_UNSET) { + return 0; + } + @Nullable Format upstreamFormat = sampleQueues[sampleQueueIndex].getUpstreamFormat(); + if (upstreamFormat != null) { + return SampleQueue.isDiscardingAllSamplesToStartTime(upstreamFormat) + ? 0 + : SampleStream.FLAG_HAS_PREROLL; + } + } + return loadingFinished || loader.hasFatalError() ? 0 : SampleStream.FLAG_MAYBE_HAS_PREROLL; + } + /** * Sets the end position at which the period stops loading and providing samples. * @@ -961,19 +996,18 @@ public void reevaluateBuffer(long positionUs) { @Override public void onLoadStarted( Chunk loadable, long elapsedRealtimeMs, long loadDurationMs, int retryCount) { - LoadEventInfo loadEventInfo = - retryCount == 0 - ? new LoadEventInfo(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) - : new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + LoadEventInfo.Builder loadEventInfo = + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setSteeredPathwayId(loadable.steeredPathwayId); + if (retryCount != 0) { + loadEventInfo + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()); + } mediaSourceEventDispatcher.loadStarted( - loadEventInfo, + loadEventInfo.build(), loadable.type, trackType, loadable.trackFormat, @@ -989,14 +1023,13 @@ public void onLoadCompleted(Chunk loadable, long elapsedRealtimeMs, long loadDur loadingChunk = null; chunkSource.onChunkLoadCompleted(loadable); LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setSteeredPathwayId(loadable.steeredPathwayId) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(); loadErrorHandlingPolicy.onLoadTaskConcluded(loadable.loadTaskId); mediaSourceEventDispatcher.loadCompleted( loadEventInfo, @@ -1019,14 +1052,13 @@ public void onLoadCanceled( Chunk loadable, long elapsedRealtimeMs, long loadDurationMs, boolean released) { loadingChunk = null; LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setSteeredPathwayId(loadable.steeredPathwayId) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(); loadErrorHandlingPolicy.onLoadTaskConcluded(loadable.loadTaskId); mediaSourceEventDispatcher.loadCanceled( loadEventInfo, @@ -1057,7 +1089,8 @@ public LoadErrorAction onLoadError( boolean isMediaChunk = isMediaChunk(loadable); if (isMediaChunk && !((HlsMediaChunk) loadable).isPublished() - && error instanceof HttpDataSource.InvalidResponseCodeException) { + && error instanceof HttpDataSource.InvalidResponseCodeException + && errorCount <= loadErrorHandlingPolicy.getMinimumLoadableRetryCount(loadable.type)) { int responseCode = ((HttpDataSource.InvalidResponseCodeException) error).responseCode; if (responseCode == 410 || responseCode == 404) { // According to RFC 8216, Section 6.2.6 a server should respond with an HTTP 404 (Not found) @@ -1068,14 +1101,13 @@ public LoadErrorAction onLoadError( } long bytesLoaded = loadable.bytesLoaded(); LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - bytesLoaded); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setSteeredPathwayId(loadable.steeredPathwayId) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(bytesLoaded) + .build(); MediaLoadData mediaLoadData = new MediaLoadData( loadable.type, @@ -1107,7 +1139,10 @@ public LoadErrorAction onLoadError( } loadErrorAction = Loader.DONT_RETRY; } else /* did not exclude */ { - long retryDelayMs = loadErrorHandlingPolicy.getRetryDelayMsFor(loadErrorInfo); + long retryDelayMs = + errorCount > loadErrorHandlingPolicy.getMinimumLoadableRetryCount(loadable.type) + ? C.TIME_UNSET + : loadErrorHandlingPolicy.getRetryDelayMsFor(loadErrorInfo); loadErrorAction = retryDelayMs != C.TIME_UNSET ? Loader.createRetryAction(/* resetErrorCount= */ false, retryDelayMs) @@ -1184,7 +1219,7 @@ private void discardUpstream(int preferredQueueSize) { long endTimeUs = getLastMediaChunk().endTimeUs; HlsMediaChunk firstRemovedChunk = discardUpstreamMediaChunksFromIndex(newQueueSize); if (mediaChunks.isEmpty()) { - pendingResetPositionUs = lastSeekPositionUs; + pendingResetPositionUs = max(lastSeekPositionUs, firstRemovedChunk.startTimeUs); } else { getLast(mediaChunks).invalidateExtractor(); } @@ -1366,6 +1401,11 @@ public void setDrmInitData(@Nullable DrmInitData drmInitData) { } } + /** Returns the redundant groups for this sample stream wrapper. */ + public ImmutableList getRedundantGroups() { + return chunkSource.getRedundantGroups(); + } + // Internal methods. private void updateSampleStreams(@NullableType SampleStream[] streams) { diff --git a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/offline/HlsDownloader.java b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/offline/HlsDownloader.java index f2ab72c551e..facffbd7ed3 100644 --- a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/offline/HlsDownloader.java +++ b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/offline/HlsDownloader.java @@ -157,6 +157,8 @@ public HlsDownloader create(MediaItem mediaItem) { } } + private final Parser manifestParser; + /** * @deprecated Use {@link HlsDownloader.Factory#create(MediaItem)} instead. */ @@ -214,27 +216,36 @@ private HlsDownloader( maxMergedSegmentStartTimeDiffMs, startPositionUs, durationUs); + this.manifestParser = manifestParser; } @Override protected List getSegments(DataSource dataSource, HlsPlaylist manifest, boolean removing) throws IOException, InterruptedException { ArrayList mediaPlaylistDataSpecs = new ArrayList<>(); + HlsMultivariantPlaylist multivariantPlaylist = HlsMultivariantPlaylist.EMPTY; if (manifest instanceof HlsMultivariantPlaylist) { - HlsMultivariantPlaylist multivariantPlaylist = (HlsMultivariantPlaylist) manifest; + multivariantPlaylist = (HlsMultivariantPlaylist) manifest; addMediaPlaylistDataSpecs(multivariantPlaylist.mediaPlaylistUrls, mediaPlaylistDataSpecs); } else { mediaPlaylistDataSpecs.add( SegmentDownloader.getCompressibleDataSpec(Uri.parse(manifest.baseUri))); } + Parser mediaPlaylistParser = + !multivariantPlaylist.variableDefinitions.isEmpty() + ? new HlsPlaylistParser(multivariantPlaylist, /* previousMediaPlaylist= */ null) + : manifestParser; + ArrayList segments = new ArrayList<>(); HashSet seenEncryptionKeyUris = new HashSet<>(); for (DataSpec mediaPlaylistDataSpec : mediaPlaylistDataSpecs) { segments.add(new Segment(/* startTimeUs= */ 0, mediaPlaylistDataSpec)); HlsMediaPlaylist mediaPlaylist; try { - mediaPlaylist = (HlsMediaPlaylist) getManifest(dataSource, mediaPlaylistDataSpec, removing); + mediaPlaylist = + (HlsMediaPlaylist) + getManifest(dataSource, mediaPlaylistParser, mediaPlaylistDataSpec, removing); } catch (IOException e) { if (!removing) { throw e; diff --git a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/playlist/DefaultHlsPlaylistTracker.java b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/playlist/DefaultHlsPlaylistTracker.java index de4c14f1f0f..9fd1d8c0f56 100644 --- a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/playlist/DefaultHlsPlaylistTracker.java +++ b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/playlist/DefaultHlsPlaylistTracker.java @@ -186,11 +186,12 @@ public void start( dataSpec = cmcdData.addToDataSpec(dataSpec); } ParsingLoadable multivariantPlaylistLoadable = - new ParsingLoadable<>( - dataSourceFactory.createDataSource(C.DATA_TYPE_MANIFEST), - dataSpec, - C.DATA_TYPE_MANIFEST, - playlistParserFactory.createPlaylistParser()); + new ParsingLoadable.Builder<>( + dataSourceFactory.createDataSource(C.DATA_TYPE_MANIFEST), + dataSpec, + C.DATA_TYPE_MANIFEST, + playlistParserFactory.createPlaylistParser()) + .build(); checkState(initialPlaylistLoader == null); initialPlaylistLoader = downloadExecutorSupplier != null @@ -208,7 +209,7 @@ public void stop() { primaryMediaPlaylistSnapshot = null; multivariantPlaylist = null; if (contentSteeringTracker != null) { - contentSteeringTracker.stop(); + contentSteeringTracker.release(); } variantRedundantGroups = null; videoRedundantGroups = null; @@ -385,18 +386,16 @@ public void onLoadStarted( long elapsedRealtimeMs, long loadDurationMs, int retryCount) { - LoadEventInfo loadEventInfo = - retryCount == 0 - ? new LoadEventInfo(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) - : new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); - eventDispatcher.loadStarted(loadEventInfo, loadable.type, retryCount); + LoadEventInfo.Builder loadEventInfo = + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs); + if (retryCount != 0) { + loadEventInfo + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()); + } + eventDispatcher.loadStarted(loadEventInfo.build(), loadable.type, retryCount); } @Override @@ -439,8 +438,12 @@ public void onLoadCompleted( new ContentSteeringCallback(), checkNotNull(bandwidthMeter), SystemClock.DEFAULT); + @Nullable String initialPathwayId = contentSteeringInfo.pathwayId; + + ImmutableList initialPathwayIds = + initialPathwayId != null ? ImmutableList.of(initialPathwayId) : ImmutableList.of(); contentSteeringTracker.start( - contentSteeringInfo.serverUri, contentSteeringInfo.pathwayId, eventDispatcher); + contentSteeringInfo.serverUri, initialPathwayIds, eventDispatcher); } HlsRedundantGroup primaryRedundantGroup = variantRedundantGroups.get(0); primaryMediaPlaylistUrl = primaryRedundantGroup.getCurrentPlaylistUrl(); @@ -448,14 +451,12 @@ public void onLoadCompleted( listeners.add(new FirstPrimaryMediaPlaylistListener()); createBundles(); LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(); RedundantGroupBundle primaryRedundantGroupBundle = checkNotNull(redundantGroupBundles.get(primaryMediaPlaylistUrl)); if (isMediaPlaylist) { @@ -477,14 +478,12 @@ public void onLoadCanceled( long loadDurationMs, boolean released) { LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(); loadErrorHandlingPolicy.onLoadTaskConcluded(loadable.loadTaskId); eventDispatcher.loadCanceled(loadEventInfo, C.DATA_TYPE_MANIFEST); } @@ -497,18 +496,18 @@ public LoadErrorAction onLoadError( IOException error, int errorCount) { LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(); MediaLoadData mediaLoadData = new MediaLoadData(loadable.type); long retryDelayMs = - loadErrorHandlingPolicy.getRetryDelayMsFor( - new LoadErrorInfo(loadEventInfo, mediaLoadData, error, errorCount)); + errorCount > loadErrorHandlingPolicy.getMinimumLoadableRetryCount(loadable.type) + ? C.TIME_UNSET + : loadErrorHandlingPolicy.getRetryDelayMsFor( + new LoadErrorInfo(loadEventInfo, mediaLoadData, error, errorCount)); boolean isFatal = retryDelayMs == C.TIME_UNSET; eventDispatcher.loadError(loadEventInfo, loadable.type, error, isFatal); if (isFatal) { @@ -997,18 +996,17 @@ public void onLoadStarted( long elapsedRealtimeMs, long loadDurationMs, int retryCount) { - LoadEventInfo loadEventInfo = - retryCount == 0 - ? new LoadEventInfo(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) - : new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); - eventDispatcher.loadStarted(loadEventInfo, loadable.type, retryCount); + LoadEventInfo.Builder loadEventInfo = + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setSteeredPathwayId(loadable.steeredPathwayId); + if (retryCount != 0) { + loadEventInfo + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()); + } + eventDispatcher.loadStarted(loadEventInfo.build(), loadable.type, retryCount); } @Override @@ -1016,14 +1014,13 @@ public void onLoadCompleted( ParsingLoadable loadable, long elapsedRealtimeMs, long loadDurationMs) { @Nullable HlsPlaylist result = loadable.getResult(); LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setSteeredPathwayId(loadable.steeredPathwayId) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(); if (result instanceof HlsMediaPlaylist) { processLoadedPlaylist((HlsMediaPlaylist) result, loadEventInfo); eventDispatcher.loadCompleted(loadEventInfo, C.DATA_TYPE_MANIFEST); @@ -1044,14 +1041,13 @@ public void onLoadCanceled( long loadDurationMs, boolean released) { LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setSteeredPathwayId(loadable.steeredPathwayId) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(); loadErrorHandlingPolicy.onLoadTaskConcluded(loadable.loadTaskId); eventDispatcher.loadCanceled(loadEventInfo, C.DATA_TYPE_MANIFEST); } @@ -1064,14 +1060,13 @@ public LoadErrorAction onLoadError( IOException error, int errorCount) { LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setSteeredPathwayId(loadable.steeredPathwayId) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(); boolean isBlockingRequest = loadable.getUri().getQueryParameter(BLOCK_MSN_PARAM) != null; boolean deltaUpdateFailed = error instanceof HlsPlaylistParser.DeltaUpdateException; if (isBlockingRequest || deltaUpdateFailed) { @@ -1097,7 +1092,10 @@ public LoadErrorAction onLoadError( notifyPlaylistError(playlistUrl, loadErrorInfo, /* forceRetry= */ false); LoadErrorAction loadErrorAction; if (exclusionFailed) { - long retryDelay = loadErrorHandlingPolicy.getRetryDelayMsFor(loadErrorInfo); + long retryDelay = + errorCount > loadErrorHandlingPolicy.getMinimumLoadableRetryCount(loadable.type) + ? C.TIME_UNSET + : loadErrorHandlingPolicy.getRetryDelayMsFor(loadErrorInfo); loadErrorAction = retryDelay != C.TIME_UNSET ? Loader.createRetryAction(false, retryDelay) @@ -1128,7 +1126,9 @@ private void loadPlaylistInternal(Uri playlistRequestUri) { playlistRefreshHandler.postDelayed( () -> { loadPending = false; - loadPlaylistImmediately(playlistRequestUri); + if (playlistUrl.equals(primaryMediaPlaylistUrl) || isActiveForPlayback()) { + loadPlaylistImmediately(playlistRequestUri); + } }, earliestNextLoadTimeMs - currentTimeMs); } else { @@ -1154,8 +1154,10 @@ private void loadPlaylistImmediately(Uri playlistRequestUri) { dataSpec = cmcdDataFactory.createCmcdData().addToDataSpec(dataSpec); } ParsingLoadable mediaPlaylistLoadable = - new ParsingLoadable<>( - mediaPlaylistDataSource, dataSpec, C.DATA_TYPE_MANIFEST, mediaPlaylistParser); + new ParsingLoadable.Builder( + mediaPlaylistDataSource, dataSpec, C.DATA_TYPE_MANIFEST, mediaPlaylistParser) + .setSteeredPathwayId(getReportedPathwayId()) + .build(); mediaPlaylistLoader.startLoading( mediaPlaylistLoadable, /* callback= */ this, @@ -1265,6 +1267,24 @@ private Uri getMediaPlaylistUriForReload() { private void excludePlaylist(long exclusionDurationMs) { excludeUntilMs = SystemClock.DEFAULT.elapsedRealtime() + exclusionDurationMs; } + + @Nullable + private String getReportedPathwayId() { + if (contentSteeringTracker == null || !contentSteeringTracker.isActive()) { + return null; + } + @Nullable RedundantGroupBundle bundle = redundantGroupBundles.get(playlistUrl); + if (bundle != null) { + String currentPathwayId = bundle.redundantGroup.getCurrentPathwayId(); + if (pathwayIds.contains(currentPathwayId)) { + return currentPathwayId; + } + } + // If the current pathway ID is not associated with this URL, we fall back to reporting the + // first available pathway ID. This is a heuristic for rare cases where multiple pathways + // might map to the same URL. + return Iterables.getFirst(pathwayIds, null); + } } /** @@ -1352,16 +1372,19 @@ private class ContentSteeringCallback implements HlsContentSteeringTracker.Callb @Override public void onCurrentPathwayUpdated( - String currentPathwayId, String oldPathwayId, long oldPathwayExcludeDurationMs) { + String currentPathwayId, + @Nullable String previousPathwayId, + long previousPathwayExcludeDurationMs) { // Defer updating primary url on pathway change if we haven't loaded the first primary // playlist yet, so that the playlist of the old pathway can still be propagated to the // client, who doesn't have to wait extra time for the playlist of the new pathway. boolean shouldDeferUpdatingPrimaryUrl = - (oldPathwayExcludeDurationMs == C.TIME_UNSET && primaryMediaPlaylistSnapshot == null); + (previousPathwayExcludeDurationMs == C.TIME_UNSET + && primaryMediaPlaylistSnapshot == null); for (RedundantGroupBundle redundantGroupBundle : redundantGroupBundles.values()) { @Nullable Uri currentPlaylistUrlToExclude = - oldPathwayExcludeDurationMs != C.TIME_UNSET + previousPathwayExcludeDurationMs != C.TIME_UNSET ? redundantGroupBundle.redundantGroup.getCurrentPlaylistUrl() : null; if (currentPlaylistUrlToExclude == null) { @@ -1370,7 +1393,7 @@ public void onCurrentPathwayUpdated( } else { boolean unusedExcludeResult = redundantGroupBundle.excludePlaylist( - currentPlaylistUrlToExclude, oldPathwayExcludeDurationMs, currentPathwayId); + currentPlaylistUrlToExclude, previousPathwayExcludeDurationMs, currentPathwayId); } } } @@ -1379,13 +1402,13 @@ public void onCurrentPathwayUpdated( public void onNewPathwayAvailable( String newPathwayId, String basePathwayId, - ImmutableList newPlaylistUrls, - ImmutableList basePlaylistUrls) { - checkState(newPlaylistUrls.size() == basePlaylistUrls.size()); - for (int i = 0; i < newPlaylistUrls.size(); i++) { + ImmutableList newUris, + ImmutableList baseUris) { + checkState(newUris.size() == baseUris.size()); + for (int i = 0; i < newUris.size(); i++) { RedundantGroupBundle redundantGroupBundle = - checkNotNull(redundantGroupBundles.get(basePlaylistUrls.get(i))); - Uri newPlaylistUrl = newPlaylistUrls.get(i); + checkNotNull(redundantGroupBundles.get(baseUris.get(i))); + Uri newPlaylistUrl = newUris.get(i); redundantGroupBundle.addPlaylistUrl(newPathwayId, newPlaylistUrl); redundantGroupBundles.put(newPlaylistUrl, redundantGroupBundle); } diff --git a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/playlist/HlsPlaylistTracker.java b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/playlist/HlsPlaylistTracker.java index c8c3786c7f5..82aa967b2a3 100644 --- a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/playlist/HlsPlaylistTracker.java +++ b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/playlist/HlsPlaylistTracker.java @@ -20,12 +20,12 @@ import androidx.media3.common.C; import androidx.media3.common.util.SystemClock; import androidx.media3.common.util.UnstableApi; +import androidx.media3.exoplayer.hls.HlsContentSteeringTracker; import androidx.media3.exoplayer.hls.HlsDataSourceFactory; import androidx.media3.exoplayer.source.MediaSourceEventListener.EventDispatcher; import androidx.media3.exoplayer.upstream.BandwidthMeter; import androidx.media3.exoplayer.upstream.CmcdConfiguration; import androidx.media3.exoplayer.upstream.LoadErrorHandlingPolicy; -import androidx.media3.exoplayer.upstream.contentsteering.ContentSteeringTracker; import androidx.media3.exoplayer.util.ReleasableExecutor; import com.google.common.base.Supplier; import java.io.IOException; @@ -179,13 +179,13 @@ void start( HlsMultivariantPlaylist getMultivariantPlaylist(); /** - * Returns the {@linkplain ContentSteeringTracker content steering tracker}. + * Returns the {@link HlsContentSteeringTracker content steering tracker}. * - * @return The content steering tracker. Null if the initial playlist has yet to be loaded, or no - * information for content steering is declared in the initial playlist. + * @return The {@link HlsContentSteeringTracker}. Null if the initial playlist has yet to be + * loaded, or no information for content steering is declared in the initial playlist. */ @Nullable - ContentSteeringTracker getContentSteeringTracker(); + HlsContentSteeringTracker getContentSteeringTracker(); /** * Returns the {@link HlsRedundantGroup} list corresponding to the {@code type}. diff --git a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/playlist/HlsRedundantGroup.java b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/playlist/HlsRedundantGroup.java index 5bbf03b1f1a..340dae4ef54 100644 --- a/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/playlist/HlsRedundantGroup.java +++ b/libraries/exoplayer_hls/src/main/java/androidx/media3/exoplayer/hls/playlist/HlsRedundantGroup.java @@ -401,11 +401,13 @@ private void validateRenditionRedundantGroupList( public static class GroupKey { /** - * The {@link Format} shared by the {@linkplain Variant grouped variants} or the {@linkplain - * Rendition grouped renditions}. + * The representative {@link Format} for the {@linkplain Variant grouped variants} or the + * {@linkplain Rendition grouped renditions}. */ public final Format format; + private final Format normalizedFormat; + /** * The stable identifier shared by variants or renditions, or {@code null}. * @@ -440,8 +442,9 @@ public GroupKey(Format format, @Nullable String stableId) { * @param name See {@link #name}. */ public GroupKey(Format format, @Nullable String stableId, @Nullable String name) { + this.format = format; // Normalize the format to ensure only fields affecting identity are part of the key. - this.format = format.buildUpon().setId(null).setMetadata(null).build(); + this.normalizedFormat = format.buildUpon().setId(null).setMetadata(null).build(); this.stableId = stableId; this.name = name; } @@ -455,14 +458,14 @@ public boolean equals(@Nullable Object other) { return false; } GroupKey groupKey = (GroupKey) other; - return Objects.equals(format, groupKey.format) + return Objects.equals(normalizedFormat, groupKey.normalizedFormat) && Objects.equals(stableId, groupKey.stableId) && Objects.equals(name, groupKey.name); } @Override public int hashCode() { - return Objects.hash(format, stableId, name); + return Objects.hash(normalizedFormat, stableId, name); } } diff --git a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsChunkSourceTest.java b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsChunkSourceTest.java index ef1230aefa2..0c4c652ea20 100644 --- a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsChunkSourceTest.java +++ b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsChunkSourceTest.java @@ -17,15 +17,19 @@ import static androidx.media3.exoplayer.upstream.LoadErrorHandlingPolicy.FALLBACK_TYPE_LOCATION; import static androidx.media3.exoplayer.upstream.LoadErrorHandlingPolicy.FALLBACK_TYPE_TRACK; +import static androidx.media3.test.utils.robolectric.RobolectricUtil.runMainLooperUntil; import static com.google.common.truth.Truth.assertThat; import static org.junit.Assert.assertThrows; import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.anyBoolean; import static org.mockito.ArgumentMatchers.anyLong; import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.clearInvocations; +import static org.mockito.Mockito.doAnswer; import static org.mockito.Mockito.doThrow; import static org.mockito.Mockito.mock; import static org.mockito.Mockito.never; +import static org.mockito.Mockito.times; import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; @@ -37,6 +41,7 @@ import androidx.media3.common.MediaItem; import androidx.media3.common.MimeTypes; import androidx.media3.common.TrackGroup; +import androidx.media3.common.util.Clock; import androidx.media3.common.util.TimestampAdjuster; import androidx.media3.exoplayer.LoadingInfo; import androidx.media3.exoplayer.SeekParameters; @@ -51,11 +56,11 @@ import androidx.media3.exoplayer.source.MediaSourceEventListener; import androidx.media3.exoplayer.source.chunk.Chunk; import androidx.media3.exoplayer.upstream.Allocator; +import androidx.media3.exoplayer.upstream.BandwidthMeter; import androidx.media3.exoplayer.upstream.CmcdConfiguration; import androidx.media3.exoplayer.upstream.LoadErrorHandlingPolicy; import androidx.media3.exoplayer.upstream.LoadErrorHandlingPolicy.FallbackOptions; import androidx.media3.exoplayer.upstream.LoadErrorHandlingPolicy.FallbackSelection; -import androidx.media3.exoplayer.upstream.contentsteering.ContentSteeringTracker; import androidx.media3.extractor.ExtractorInput; import androidx.media3.extractor.ExtractorOutput; import androidx.media3.test.utils.ExoPlayerTestRunner; @@ -70,10 +75,15 @@ import com.google.common.collect.ImmutableMap; import java.io.IOException; import java.io.InputStream; +import java.nio.charset.Charset; import java.time.Duration; import java.util.Arrays; import java.util.List; import java.util.Map; +import java.util.Objects; +import java.util.concurrent.TimeoutException; +import java.util.concurrent.atomic.AtomicBoolean; +import org.junit.After; import org.junit.Before; import org.junit.Rule; import org.junit.Test; @@ -116,13 +126,19 @@ public class HlsChunkSourceTest { private static final String DEFAULT_PATHWAY_ID = "."; @Mock private HlsPlaylistTracker mockPlaylistTracker; - @Mock private ContentSteeringTracker mockContentSteeringTracker; + private HlsContentSteeringTracker.Callback mockCallback; + private HlsContentSteeringTracker contentSteeringTracker; @Before - public void setUp() { + public void setUp() throws Exception { when(mockPlaylistTracker.excludeMediaPlaylist(any(), anyLong())).thenReturn(true); - when(mockContentSteeringTracker.excludeCurrentPathway(anyLong())).thenReturn(true); - when(mockContentSteeringTracker.isActive()).thenReturn(true); + mockCallback = mock(HlsContentSteeringTracker.Callback.class); + contentSteeringTracker = createFakeContentSteeringTracker(mockPlaylistTracker, mockCallback); + } + + @After + public void tearDown() { + contentSteeringTracker.release(); } @Test @@ -725,6 +741,73 @@ public void getNextChunk_reloadingCurrentPreloadPartAfterPublication_returnsShou assertThat(((HlsMediaChunk) output.chunk).shouldSpliceIn()).isTrue(); } + @Test + public void getNextChunk_withContentSteeringActive_chunkHasSteeredPathwayId() throws Exception { + HlsRedundantGroup[] redundantGroups = createSampleRedundantGroups(); + when(mockPlaylistTracker.getContentSteeringTracker()).thenReturn(contentSteeringTracker); + HlsChunkSource testChunkSource = createHlsChunkSource(redundantGroups, mockPlaylistTracker); + HlsChunkSource.HlsChunkHolder output = new HlsChunkSource.HlsChunkHolder(); + + testChunkSource.getNextChunk( + new LoadingInfo.Builder().setPlaybackPositionUs(0).setPlaybackSpeed(1.0f).build(), + /* loadPositionUs= */ 0, + /* largestReadPositionUs= */ 0, + /* queue= */ ImmutableList.of(), + /* allowEndOfStream= */ true, + output); + + assertThat(output.chunk.steeredPathwayId).isEqualTo("CDN-A"); + + // Switch/select CDN-B pathway ID for all redundant groups + for (HlsRedundantGroup group : redundantGroups) { + group.setCurrentPathwayId("CDN-B"); + } + + testChunkSource.getNextChunk( + new LoadingInfo.Builder().setPlaybackPositionUs(4_000_000).setPlaybackSpeed(1.0f).build(), + /* loadPositionUs= */ 4_000_000, + /* largestReadPositionUs= */ 0, + /* queue= */ ImmutableList.of((HlsMediaChunk) output.chunk), + /* allowEndOfStream= */ true, + output); + + assertThat(output.chunk.steeredPathwayId).isEqualTo("CDN-B"); + } + + @Test + public void getNextChunk_withContentSteeringInactive_chunkSteeredPathwayIdIsNull() + throws IOException { + HlsRedundantGroup[] redundantGroups = createSampleRedundantGroups(); + when(mockPlaylistTracker.getContentSteeringTracker()).thenReturn(null); + HlsChunkSource testChunkSource = createHlsChunkSource(redundantGroups, mockPlaylistTracker); + HlsChunkSource.HlsChunkHolder output = new HlsChunkSource.HlsChunkHolder(); + + testChunkSource.getNextChunk( + new LoadingInfo.Builder().setPlaybackPositionUs(0).setPlaybackSpeed(1.0f).build(), + /* loadPositionUs= */ 0, + /* largestReadPositionUs= */ 0, + /* queue= */ ImmutableList.of(), + /* allowEndOfStream= */ true, + output); + + assertThat(output.chunk.steeredPathwayId).isNull(); + + // Switch/select CDN-B pathway ID for all redundant groups + for (HlsRedundantGroup group : redundantGroups) { + group.setCurrentPathwayId("CDN-B"); + } + + testChunkSource.getNextChunk( + new LoadingInfo.Builder().setPlaybackPositionUs(4_000_000).setPlaybackSpeed(1.0f).build(), + /* loadPositionUs= */ 4_000_000, + /* largestReadPositionUs= */ 0, + /* queue= */ ImmutableList.of((HlsMediaChunk) output.chunk), + /* allowEndOfStream= */ true, + output); + + assertThat(output.chunk.steeredPathwayId).isNull(); + } + @Test public void getChunkPublicationState_withCmcdQueryParameters_returnsPublished() throws Exception { CmcdConfiguration.Factory cmcdConfigurationFactory = @@ -1154,9 +1237,9 @@ public void createFallbackOptionsForChunkError_withAllTracksSelected_returnsCorr @Test public void createFallbackOptionsForPlaylistError_withContentSteeringActive_returnsCorrectResult() - throws IOException { + throws Exception { HlsRedundantGroup[] redundantGroups = createSampleRedundantGroups(); - when(mockPlaylistTracker.getContentSteeringTracker()).thenReturn(mockContentSteeringTracker); + when(mockPlaylistTracker.getContentSteeringTracker()).thenReturn(contentSteeringTracker); HlsChunkSource testChunkSource = createHlsChunkSource(redundantGroups, mockPlaylistTracker); Uri failingPlaylistUrl = Uri.parse("https://test/media-a/playlist0.m3u8"); // There is a playlist url from the same pathway but of a different track has been excluded. @@ -1179,9 +1262,9 @@ public void createFallbackOptionsForPlaylistError_withContentSteeringActive_retu @Test public void createFallbackOptionsForChunkError_withContentSteeringActive_returnsCorrectResult() - throws IOException { + throws Exception { HlsRedundantGroup[] redundantGroups = createSampleRedundantGroups(); - when(mockPlaylistTracker.getContentSteeringTracker()).thenReturn(mockContentSteeringTracker); + when(mockPlaylistTracker.getContentSteeringTracker()).thenReturn(contentSteeringTracker); HlsChunkSource testChunkSource = createHlsChunkSource(redundantGroups, mockPlaylistTracker); HlsChunkSource.HlsChunkHolder output = new HlsChunkSource.HlsChunkHolder(); testChunkSource.getNextChunk( @@ -1268,9 +1351,9 @@ public void onPlaylistError_fallbackSelectionIsLocationType_excludesPlaylistOnly @Test public void onPlaylistErrorWithContentSteeringActive_fallbackSelectionIsNull_returnsFalse() - throws IOException { + throws Exception { HlsRedundantGroup[] redundantGroups = createSampleRedundantGroups(); - when(mockPlaylistTracker.getContentSteeringTracker()).thenReturn(mockContentSteeringTracker); + when(mockPlaylistTracker.getContentSteeringTracker()).thenReturn(contentSteeringTracker); HlsChunkSource testChunkSource = createHlsChunkSource(redundantGroups, mockPlaylistTracker); TrackGroup trackGroup = testChunkSource.getTrackGroup(); TestTrackSelection trackSelection = new TestTrackSelection(trackGroup, /* selectedIndex= */ 0); @@ -1285,9 +1368,9 @@ public void onPlaylistErrorWithContentSteeringActive_fallbackSelectionIsNull_ret @Test public void onPlaylistErrorWithContentSteeringActive_fallbackSelectionIsTrackType_excludesTrackAndPlaylist() - throws IOException { + throws Exception { HlsRedundantGroup[] redundantGroups = createSampleRedundantGroups(); - when(mockPlaylistTracker.getContentSteeringTracker()).thenReturn(mockContentSteeringTracker); + when(mockPlaylistTracker.getContentSteeringTracker()).thenReturn(contentSteeringTracker); HlsChunkSource testChunkSource = createHlsChunkSource(redundantGroups, mockPlaylistTracker); TrackGroup trackGroup = testChunkSource.getTrackGroup(); TestTrackSelection trackSelection = new TestTrackSelection(trackGroup, /* selectedIndex= */ 0); @@ -1304,15 +1387,15 @@ public void onPlaylistErrorWithContentSteeringActive_fallbackSelectionIsNull_ret assertThat(trackSelection.isTrackExcluded(/* index= */ 0, SystemClock.elapsedRealtime())) .isTrue(); verify(mockPlaylistTracker).excludeMediaPlaylist(playlistUrl, 10_000); - verify(mockContentSteeringTracker, never()).excludeCurrentPathway(anyLong()); + verify(mockCallback, never()).onCurrentPathwayUpdated(any(), any(), anyLong()); } @Test public void onPlaylistErrorWithContentSteeringActive_fallbackSelectionIsLocationType_excludesCurrentPathway() - throws IOException { + throws Exception { HlsRedundantGroup[] redundantGroups = createSampleRedundantGroups(); - when(mockPlaylistTracker.getContentSteeringTracker()).thenReturn(mockContentSteeringTracker); + when(mockPlaylistTracker.getContentSteeringTracker()).thenReturn(contentSteeringTracker); HlsChunkSource testChunkSource = createHlsChunkSource(redundantGroups, mockPlaylistTracker); TrackGroup trackGroup = testChunkSource.getTrackGroup(); TestTrackSelection trackSelection = new TestTrackSelection(trackGroup, /* selectedIndex= */ 0); @@ -1330,7 +1413,11 @@ public void onPlaylistErrorWithContentSteeringActive_fallbackSelectionIsNull_ret playlistUrl, new FallbackSelection(FALLBACK_TYPE_LOCATION, 10_000)); assertThat(exclusionResult).isTrue(); - verify(mockContentSteeringTracker).excludeCurrentPathway(10_000); + verify(mockCallback) + .onCurrentPathwayUpdated( + /* currentPathwayId= */ "CDN-B", + /* previousPathwayId= */ "CDN-A", + /* previousPathwayExcludeDurationMs= */ 10_000L); assertThat(trackSelection.isTrackExcluded(/* index= */ 1, SystemClock.elapsedRealtime())) .isFalse(); } @@ -1411,9 +1498,9 @@ public void onChunkError_fallbackSelectionIsLocationType_excludesPlaylistOnly() @Test public void onChunkErrorWithContentSteeringActive_fallbackSelectionIsNull_returnsFalse() - throws IOException { + throws Exception { HlsRedundantGroup[] redundantGroups = createSampleRedundantGroups(); - when(mockPlaylistTracker.getContentSteeringTracker()).thenReturn(mockContentSteeringTracker); + when(mockPlaylistTracker.getContentSteeringTracker()).thenReturn(contentSteeringTracker); HlsChunkSource testChunkSource = createHlsChunkSource(redundantGroups, mockPlaylistTracker); TrackGroup trackGroup = testChunkSource.getTrackGroup(); TestTrackSelection trackSelection = new TestTrackSelection(trackGroup, /* selectedIndex= */ 0); @@ -1434,9 +1521,9 @@ public void onChunkErrorWithContentSteeringActive_fallbackSelectionIsNull_return @Test public void onChunkErrorWithContentSteeringActive_fallbackSelectionIsTrackType_excludesTrackOnly() - throws IOException { + throws Exception { HlsRedundantGroup[] redundantGroups = createSampleRedundantGroups(); - when(mockPlaylistTracker.getContentSteeringTracker()).thenReturn(mockContentSteeringTracker); + when(mockPlaylistTracker.getContentSteeringTracker()).thenReturn(contentSteeringTracker); HlsChunkSource testChunkSource = createHlsChunkSource(redundantGroups, mockPlaylistTracker); TrackGroup trackGroup = testChunkSource.getTrackGroup(); TestTrackSelection trackSelection = new TestTrackSelection(trackGroup, /* selectedIndex= */ 0); @@ -1458,15 +1545,15 @@ public void onChunkErrorWithContentSteeringActive_fallbackSelectionIsTrackType_e assertThat(exclusionResult).isTrue(); assertThat(trackSelection.isTrackExcluded(0, SystemClock.elapsedRealtime())).isTrue(); verify(mockPlaylistTracker, never()).excludeMediaPlaylist(any(), anyLong()); - verify(mockContentSteeringTracker, never()).excludeCurrentPathway(anyLong()); + verify(mockCallback, never()).onCurrentPathwayUpdated(any(), any(), anyLong()); } @Test public void onChunkErrorWithContentSteeringActive_fallbackSelectionIsLocationType_excludesCurrentPathway() - throws IOException { + throws Exception { HlsRedundantGroup[] redundantGroups = createSampleRedundantGroups(); - when(mockPlaylistTracker.getContentSteeringTracker()).thenReturn(mockContentSteeringTracker); + when(mockPlaylistTracker.getContentSteeringTracker()).thenReturn(contentSteeringTracker); HlsChunkSource testChunkSource = createHlsChunkSource(redundantGroups, mockPlaylistTracker); TrackGroup trackGroup = testChunkSource.getTrackGroup(); TestTrackSelection trackSelection = new TestTrackSelection(trackGroup, /* selectedIndex= */ 0); @@ -1491,7 +1578,11 @@ public void onChunkErrorWithContentSteeringActive_fallbackSelectionIsTrackType_e output.chunk, new FallbackSelection(FALLBACK_TYPE_LOCATION, 10_000)); assertThat(exclusionResult).isTrue(); - verify(mockContentSteeringTracker).excludeCurrentPathway(10_000); + verify(mockCallback) + .onCurrentPathwayUpdated( + /* currentPathwayId= */ "CDN-B", + /* previousPathwayId= */ "CDN-A", + /* previousPathwayExcludeDurationMs= */ 10_000L); assertThat(trackSelection.isTrackExcluded(/* index= */ 1, SystemClock.elapsedRealtime())) .isFalse(); } @@ -1721,9 +1812,9 @@ private static void loadMediaChunk(HlsMediaChunk mediaChunk, HlsChunkSource chun /* positionUs= */ 0, /* muxedAudioFormat= */ null, mock(DrmSessionManager.class), - mock(DrmSessionEventListener.EventDispatcher.class), + new DrmSessionEventListener.EventDispatcher(), mock(LoadErrorHandlingPolicy.class), - mock(MediaSourceEventListener.EventDispatcher.class), + new MediaSourceEventListener.EventDispatcher(), /* metadataType= */ HlsMediaSource.METADATA_TYPE_ID3, /* downloadExecutor= */ null); mediaChunk.init(sampleStreamWrapper, ImmutableList.of()); @@ -1763,4 +1854,67 @@ public void clearTrackExclusions() { Arrays.fill(excludeUntilMs, 0); } } + + private static HlsContentSteeringTracker createFakeContentSteeringTracker( + HlsPlaylistTracker playlistTracker, HlsContentSteeringTracker.Callback callback) + throws TimeoutException { + String initialPathwayId = "CDN-B"; + String steeringManifest = + "{\"VERSION\": 1, \"PATHWAY-PRIORITY\": [\"CDN-A\", \"CDN-B\", \"CDN-C\"]}"; + FakeDataSource fakeDataSource = + new FakeDataSource( + new FakeDataSet() + .newDefaultData() + .appendReadData(steeringManifest.getBytes(Charset.defaultCharset())) + .endData()); + + // Mock redundant groups to define available pathways + HlsRedundantGroup redundantGroup1 = + new HlsRedundantGroup( + new HlsRedundantGroup.GroupKey(new Format.Builder().build(), /* stableId= */ "id1"), + "CDN-A", + Uri.parse("https://url-a")); + redundantGroup1.put("CDN-B", Uri.parse("https://url-b")); + redundantGroup1.put("CDN-C", Uri.parse("https://url-c")); + + when(playlistTracker.getRedundantGroups(HlsRedundantGroup.VARIANT)) + .thenReturn(ImmutableList.of(redundantGroup1)); + when(playlistTracker.getRedundantGroups(HlsRedundantGroup.VIDEO_RENDITION)) + .thenReturn(ImmutableList.of()); + when(playlistTracker.getRedundantGroups(HlsRedundantGroup.AUDIO_RENDITION)) + .thenReturn(ImmutableList.of()); + when(playlistTracker.getRedundantGroups(HlsRedundantGroup.SUBTITLE_RENDITION)) + .thenReturn(ImmutableList.of()); + + AtomicBoolean manifestLoaded = new AtomicBoolean(); + doAnswer( + invocation -> { + String currentPathwayId = invocation.getArgument(0); + String previousPathwayId = invocation.getArgument(1); + if (Objects.equals("CDN-A", currentPathwayId) + && Objects.equals("CDN-B", previousPathwayId)) { + manifestLoaded.set(true); + } + return null; + }) + .when(callback) + .onCurrentPathwayUpdated(any(), any(), anyLong()); + HlsContentSteeringTracker tracker = + new HlsContentSteeringTracker( + dataType -> fakeDataSource, + /* downloadExecutorSupplier= */ null, + playlistTracker, + callback, + BandwidthMeter.NO_OP, + Clock.DEFAULT); + tracker.start( + Uri.parse("https://steering"), + ImmutableList.of(initialPathwayId), + new MediaSourceEventListener.EventDispatcher()); + runMainLooperUntil(manifestLoaded::get); + verify(callback, times(2)).onCurrentPathwayUpdated(any(), any(), anyLong()); + clearInvocations(callback); + + return tracker; + } } diff --git a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsContentSteeringTrackerTest.java b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsContentSteeringTrackerTest.java index 6fc7b20160b..1bdf5bc48ff 100644 --- a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsContentSteeringTrackerTest.java +++ b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsContentSteeringTrackerTest.java @@ -19,12 +19,10 @@ import static com.google.common.truth.Truth.assertThat; import static org.junit.Assert.assertThrows; import static org.mockito.ArgumentMatchers.any; -import static org.mockito.ArgumentMatchers.anyInt; import static org.mockito.ArgumentMatchers.anyLong; import static org.mockito.ArgumentMatchers.eq; import static org.mockito.Mockito.clearInvocations; import static org.mockito.Mockito.doAnswer; -import static org.mockito.Mockito.mock; import static org.mockito.Mockito.never; import static org.mockito.Mockito.verify; import static org.mockito.Mockito.verifyNoInteractions; @@ -33,6 +31,7 @@ import android.net.Uri; import android.os.Looper; +import androidx.annotation.Nullable; import androidx.media3.common.C; import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; @@ -54,6 +53,7 @@ import java.util.concurrent.TimeoutException; import java.util.concurrent.atomic.AtomicInteger; import java.util.concurrent.atomic.AtomicReference; +import org.junit.After; import org.junit.Before; import org.junit.Rule; import org.junit.Test; @@ -73,6 +73,7 @@ public final class HlsContentSteeringTrackerTest { @Mock private HlsPlaylistTracker mockPlaylistTracker; @Mock private HlsContentSteeringTracker.Callback mockContentSteeringTrackerCallback; private AtomicInteger currentPathwayUpdateCount; + @Nullable private HlsContentSteeringTracker contentSteeringTracker; @Before public void setUp() { @@ -84,10 +85,17 @@ public void setUp() { .onCurrentPathwayUpdated(any(), any(), anyLong()); } + @After + public void tearDown() { + if (contentSteeringTracker != null) { + contentSteeringTracker.release(); + } + } + @Test public void start_withValidInitialPathwayId_picksTheInitialPathwayAndNotifyCallback() { AtomicReference timeoutExceptionRef = new AtomicReference<>(); - HlsContentSteeringTracker contentSteeringTracker = + contentSteeringTracker = runContentSteeringTracker( new FakeDataSource(), /* initialPathwayId= */ "CDN-B", @@ -105,28 +113,6 @@ public void start_withValidInitialPathwayId_picksTheInitialPathwayAndNotifyCallb assertThat(contentSteeringTracker.isActive()).isTrue(); } - @Test - public void - start_withoutInitialPathwayId_picksTheCurrentPathwayOfVariantRedundantGroupAndNotifyCallback() { - AtomicReference timeoutExceptionRef = new AtomicReference<>(); - HlsContentSteeringTracker contentSteeringTracker = - runContentSteeringTracker( - new FakeDataSource(), - /* initialPathwayId= */ null, - mockContentSteeringTrackerCallback, - SystemClock.DEFAULT, - /* awaitedCurrentPathwayUpdateCount= */ 1, - timeoutExceptionRef); - - assertThat(timeoutExceptionRef.get()).isNull(); - verify(mockContentSteeringTrackerCallback) - .onCurrentPathwayUpdated( - /* currentPathwayId= */ eq("CDN-A"), - /* previousPathwayId= */ eq(null), - /* previousPathwayExcludeDurationMs= */ eq(C.TIME_UNSET)); - assertThat(contentSteeringTracker.isActive()).isTrue(); - } - @Test public void start_withInvalidInitialPathwayId_throwsIllegalStateException() { assertThrows( @@ -153,7 +139,7 @@ public void start_onSteeringManifestUpdated_evaluatesAndUpdatesPathway() { .endData()); AtomicReference timeoutExceptionRef = new AtomicReference<>(); - HlsContentSteeringTracker contentSteeringTracker = + contentSteeringTracker = runContentSteeringTracker( fakeDataSource, /* initialPathwayId= */ "CDN-A", @@ -210,7 +196,7 @@ public void start_onSteeringManifestUpdatedWithPathwayClones_notifiesCallback() .endData()); AtomicReference timeoutExceptionRef = new AtomicReference<>(); - HlsContentSteeringTracker contentSteeringTracker = + contentSteeringTracker = runContentSteeringTracker( fakeDataSource, /* initialPathwayId= */ "CDN-A", @@ -245,8 +231,8 @@ public void start_onSteeringManifestUpdatedWithPathwayClones_notifiesCallback() .onNewPathwayAvailable( /* newPathwayId= */ eq("CDN-A-CLONE"), /* basePathwayId= */ eq("CDN-A"), - /* newPlaylistUrls= */ eq(expectedNewPlaylistUrls.build()), - /* basePlaylistUrls= */ eq(expectedBasePlaylistUrls.build())); + /* newUris= */ eq(expectedNewPlaylistUrls.build()), + /* baseUris= */ eq(expectedBasePlaylistUrls.build())); verify(mockContentSteeringTrackerCallback) .onCurrentPathwayUpdated( @@ -282,7 +268,7 @@ public void start_onSteeringManifestUpdatedWithPerVariantUrisPathwayCloneOnly_no .endData()); AtomicReference timeoutExceptionRef = new AtomicReference<>(); - HlsContentSteeringTracker contentSteeringTracker = + contentSteeringTracker = runContentSteeringTracker( fakeDataSource, /* initialPathwayId= */ "CDN-A", @@ -316,8 +302,8 @@ public void start_onSteeringManifestUpdatedWithPerVariantUrisPathwayCloneOnly_no .onNewPathwayAvailable( /* newPathwayId= */ eq("CDN-A-CLONE"), /* basePathwayId= */ eq("CDN-A"), - /* newPlaylistUrls= */ eq(expectedNewPlaylistUrls.build()), - /* basePlaylistUrls= */ eq(expectedBasePlaylistUrls.build())); + /* newUris= */ eq(expectedNewPlaylistUrls.build()), + /* baseUris= */ eq(expectedBasePlaylistUrls.build())); verify(mockContentSteeringTrackerCallback) .onCurrentPathwayUpdated( @@ -353,7 +339,7 @@ public void start_onSteeringManifestUpdatedWithPathwayCloneWithUnknownBaseId_ski .endData()); AtomicReference timeoutExceptionRef = new AtomicReference<>(); - HlsContentSteeringTracker contentSteeringTracker = + contentSteeringTracker = runContentSteeringTracker( fakeDataSource, /* initialPathwayId= */ "CDN-A", @@ -399,7 +385,7 @@ public void start_onSteeringManifestUpdatedWithPathwayCloneWithExistingId_skipsC .endData()); AtomicReference timeoutExceptionRef = new AtomicReference<>(); - HlsContentSteeringTracker contentSteeringTracker = + contentSteeringTracker = runContentSteeringTracker( fakeDataSource, /* initialPathwayId= */ "CDN-A", @@ -437,7 +423,7 @@ public void start_onSteeringManifestUpdatedWithPathwayCloneWithExistingId_skipsC .endData()); AtomicReference timeoutExceptionRef = new AtomicReference<>(); - HlsContentSteeringTracker contentSteeringTracker = + contentSteeringTracker = runContentSteeringTracker( fakeDataSource, /* initialPathwayId= */ "CDN-A", @@ -478,7 +464,7 @@ public void start_steeringManifestLoadCanceledWhenNoPriorityListAvailable_stopsT FakeDataSource fakeDataSource = new FakeDataSource(fakeDataSet); AtomicReference timeoutExceptionRef = new AtomicReference<>(); - HlsContentSteeringTracker contentSteeringTracker = + contentSteeringTracker = runContentSteeringTracker( fakeDataSource, /* initialPathwayId= */ "CDN-A", @@ -521,7 +507,7 @@ public void start_steeringManifestLoadCanceledWhenPriorityListAvailable_doesNotS FakeDataSource fakeDataSource = new FakeDataSource(fakeDataSet); AtomicReference timeoutExceptionRef = new AtomicReference<>(); - HlsContentSteeringTracker contentSteeringTracker = + contentSteeringTracker = runContentSteeringTracker( fakeDataSource, /* initialPathwayId= */ "CDN-A", @@ -563,7 +549,7 @@ public void start_steeringManifestLoadErrorButNotCanceled_doesNotStopTracker() { FakeDataSource fakeDataSource = new FakeDataSource(fakeDataSet); AtomicReference timeoutExceptionRef = new AtomicReference<>(); - HlsContentSteeringTracker contentSteeringTracker = + contentSteeringTracker = runContentSteeringTracker( fakeDataSource, /* initialPathwayId= */ "CDN-A", @@ -596,7 +582,7 @@ public void start_steeringManifestLoadErrorButNotCanceled_doesNotStopTracker() { .appendReadData(getBytes(steeringManifest)) .endData()); AtomicReference timeoutExceptionRef = new AtomicReference<>(); - HlsContentSteeringTracker contentSteeringTracker = + contentSteeringTracker = runContentSteeringTracker( fakeDataSource, /* initialPathwayId= */ "CDN-A", @@ -647,7 +633,7 @@ public void start_steeringManifestLoadErrorButNotCanceled_doesNotStopTracker() { .appendReadData(getBytes(steeringManifest)) .endData()); AtomicReference timeoutExceptionRef = new AtomicReference<>(); - HlsContentSteeringTracker contentSteeringTracker = + contentSteeringTracker = runContentSteeringTracker( fakeDataSource, /* initialPathwayId= */ "CDN-A", @@ -669,7 +655,7 @@ public void start_steeringManifestLoadErrorButNotCanceled_doesNotStopTracker() { public void excludeCurrentPathway_beforeFirstPathwayPriorityListAvailable_excludesUnsuccessfullyAndDoesNotUpdatePathway() { AtomicReference timeoutExceptionRef = new AtomicReference<>(); - HlsContentSteeringTracker contentSteeringTracker = + contentSteeringTracker = runContentSteeringTracker( new FakeDataSource(), /* initialPathwayId= */ "CDN-A", @@ -737,11 +723,9 @@ private HlsContentSteeringTracker runContentSteeringTracker( Clock clock, int awaitedCurrentPathwayUpdateCount, AtomicReference timeoutExceptionRef) { - HlsDataSourceFactory mockHlsDataSourceFactory = mock(HlsDataSourceFactory.class); - when(mockHlsDataSourceFactory.createDataSource(anyInt())).thenReturn(fakeDataSource); - HlsContentSteeringTracker contentSteeringTracker = + contentSteeringTracker = new HlsContentSteeringTracker( - mockHlsDataSourceFactory, + dataType -> fakeDataSource, /* downloadExecutorSupplier= */ null, mockPlaylistTracker, callback, @@ -750,7 +734,7 @@ private HlsContentSteeringTracker runContentSteeringTracker( contentSteeringTracker.start( Uri.parse(TEST_INITIAL_STEERING_URI_STRING), - initialPathwayId, + ImmutableList.of(initialPathwayId), new MediaSourceEventListener.EventDispatcher()); try { diff --git a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsInterstitialsAdsLoaderTest.java b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsInterstitialsAdsLoaderTest.java index f301d8a8f5d..6b1393f63a2 100644 --- a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsInterstitialsAdsLoaderTest.java +++ b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsInterstitialsAdsLoaderTest.java @@ -16,6 +16,7 @@ package androidx.media3.exoplayer.hls; import static androidx.media3.common.AdPlaybackState.AD_STATE_AVAILABLE; +import static androidx.media3.common.AdPlaybackState.AD_STATE_SKIPPED; import static androidx.media3.common.AdPlaybackState.AD_STATE_UNAVAILABLE; import static androidx.media3.common.Player.DISCONTINUITY_REASON_AUTO_TRANSITION; import static androidx.media3.common.Player.DISCONTINUITY_REASON_SKIP; @@ -92,7 +93,10 @@ import java.nio.charset.Charset; import java.util.ArrayList; import java.util.Arrays; +import java.util.Iterator; import java.util.List; +import java.util.Map; +import java.util.NavigableMap; import java.util.concurrent.TimeoutException; import java.util.concurrent.atomic.AtomicInteger; import org.json.JSONObject; @@ -967,9 +971,9 @@ public void handleContentTimelineChanged_midRollAndPostRollNotInOrder_insertedCo .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) .withAdCount(/* adGroupIndex= */ 1, /* adCount= */ 1) .withAdCount(/* adGroupIndex= */ 2, /* adCount= */ 1) - .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 1_000_000L) - .withContentResumeOffsetUs(/* adGroupIndex= */ 1, 2_000_000L) - .withContentResumeOffsetUs(/* adGroupIndex= */ 2, 3_000_000L) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 0L) + .withContentResumeOffsetUs(/* adGroupIndex= */ 1, 0L) + .withContentResumeOffsetUs(/* adGroupIndex= */ 2, 0L) .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, "ad0-0") .withAdId(/* adGroupIndex= */ 1, /* adIndexInAdGroup= */ 0, "ad1-0") .withAdId(/* adGroupIndex= */ 2, /* adIndexInAdGroup= */ 0, "ad2-0") @@ -1011,6 +1015,7 @@ public void handleContentTimelineChanged_resumeOffsetSetToZero_contentResumeOffs + "ID=\"ad0-0\"," + "CLASS=\"com.apple.hls.interstitial\"," + "START-DATE=\"2020-01-02T21:55:41.123Z\"," + + "DURATION=2.0," + "X-PLAYOUT-LIMIT=1.0," + "CUE=\"PRE\"," + "X-RESUME-OFFSET=0.0," @@ -1036,7 +1041,7 @@ public void handleContentTimelineChanged_resumeOffsetSetToZero_contentResumeOffs new AdPlaybackState("adsId", /* adGroupTimesUs...= */ 0L) .withAdDurationsUs(/* adGroupIndex= */ 0, 1_000_000L, 1_000_000L) .withAdCount(/* adGroupIndex= */ 0, 2) - .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 1_000_000L) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 0L) .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, "ad0-0") .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 1, "ad0-1") .withAvailableAdMediaItem( @@ -1139,13 +1144,13 @@ public void handleContentTimelineChanged_durationPlayoutLimitAndContentOffset_co + "DURATION=1.0," // added to resume offset + "X-ASSET-URI=\"http://example.com/media-0-1.m3u8\"" + "\n" - // duration set, playout limit set, no resume offset + // duration set, playout limit > duration, no resume offset + "#EXT-X-DATERANGE:" + "ID=\"ad0-2\"," + "CLASS=\"com.apple.hls.interstitial\"," + "START-DATE=\"2020-01-02T21:55:44.000Z\"," - + "DURATION=1.0," - + "X-PLAYOUT-LIMIT=2.1," // added to resume offset + + "DURATION=1.0," // added to resume offset + + "X-PLAYOUT-LIMIT=2.1," + "X-ASSET-URI=\"http://example.com/media-0-2.m3u8\"" + "\n" // duration set, playout limit set, resume offset set @@ -1166,6 +1171,15 @@ public void handleContentTimelineChanged_durationPlayoutLimitAndContentOffset_co + "DURATION=1.0," + "X-RESUME-OFFSET=3.2," // added to resume offset + "X-ASSET-URI=\"http://example.com/media-0-4.m3u8\"" + + "\n" + // duration set, playout limit < duration, no resume offset + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-5\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T21:55:44.000Z\"," + + "DURATION=3.0," + + "X-PLAYOUT-LIMIT=2.1," // added to resume offset + + "X-ASSET-URI=\"http://example.com/media-0-5.m3u8\"" + "\n"; assertThat( @@ -1183,14 +1197,16 @@ public void handleContentTimelineChanged_durationPlayoutLimitAndContentOffset_co C.TIME_UNSET, 2_100_000L, 2_100_000L, - C.TIME_UNSET) - .withAdCount(/* adGroupIndex= */ 0, 5) - .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 9_500_000L) + C.TIME_UNSET, + 2_100_000L) + .withAdCount(/* adGroupIndex= */ 0, 6) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 10_500_000L) .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, "ad0-0") .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 1, "ad0-1") .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 2, "ad0-2") .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 3, "ad0-3") .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 4, "ad0-4") + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 5, "ad0-5") .withAvailableAdMediaItem( /* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, @@ -1225,11 +1241,18 @@ public void handleContentTimelineChanged_durationPlayoutLimitAndContentOffset_co new MediaItem.Builder() .setUri("http://example.com/media-0-4.m3u8") .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 5, + new MediaItem.Builder() + .setUri("http://example.com/media-0-5.m3u8") + .setMimeType(MimeTypes.APPLICATION_M3U8) .build())); } @Test - public void handleContentTimelineChanged_playoutLimitSet_playoutLimitUsedAsDuration() + public void handleContentTimelineChanged_playoutLimitSet_playoutLimitUsedAsAdDuration() throws IOException { String playlistString = "#EXTM3U\n" @@ -1261,7 +1284,49 @@ public void handleContentTimelineChanged_playoutLimitSet_playoutLimitUsedAsDurat new AdPlaybackState("adsId", /* adGroupTimesUs...= */ 1_123_000L) .withAdDurationsUs(/* adGroupIndex= */ 0, 4_000_000L) .withAdCount(/* adGroupIndex= */ 0, 1) - .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 4_000_000L) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 2_000_000L) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + new MediaItem.Builder() + .setUri("http://example.com/media-0-0.m3u8") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build())); + } + + @Test + public void + handleContentTimelineChanged_playoutLimitOnlyWithoutDuration_setsAdDurationAndZeroContentResumeOffset() + throws IOException { + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:55:40.000Z\n" + + "#EXTINF:6,\n" + + "main1.0.ts\n" + + "#EXT-X-ENDLIST" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T21:55:41.123Z\"," + + "X-PLAYOUT-LIMIT=4.0," + + "X-ASSET-URI=\"http://example.com/media-0-0.m3u8\"" + + "\n"; + + assertThat( + callHandleContentTimelineChangedAndCaptureAdPlaybackState( + playlistString, + adsLoader, + /* windowIndex= */ 0, + /* windowPositionInPeriodUs= */ 0, + /* windowEndPositionInPeriodUs= */ C.TIME_END_OF_SOURCE)) + .isEqualTo( + new AdPlaybackState("adsId", /* adGroupTimesUs...= */ 1_123_000L) + .withAdDurationsUs(/* adGroupIndex= */ 0, 4_000_000L) + .withAdCount(/* adGroupIndex= */ 0, 1) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 0L) .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, "ad0-0") .withAvailableAdMediaItem( /* adGroupIndex= */ 0, @@ -1511,7 +1576,7 @@ public void handleContentTimelineChanged_snapIn_snapToClosestSegmentBoundaryOfRe + "CLASS=\"com.apple.hls.interstitial\"," + "START-DATE=\"2020-01-02T21:55:40.000\"," // + "X-SNAP=\"IN\"," - + "X-RESUME-OFFSET=3.222," // Snaps to start of next content segment. + + "DURATION=3.222," + "X-PLAYOUT-LIMIT=3.222," + "X-ASSET-URI=\"http://example.com/media-0-0.m3u8\"" + "\n" @@ -1520,7 +1585,7 @@ public void handleContentTimelineChanged_snapIn_snapToClosestSegmentBoundaryOfRe + "CLASS=\"com.apple.hls.interstitial\"," + "START-DATE=\"2020-01-02T21:55:52.222\"," + "X-SNAP=\"IN\"," - + "X-RESUME-OFFSET=4.333," // Snaps to start of next content segment. + + "DURATION=4.333," + "X-PLAYOUT-LIMIT=4.333," + "X-ASSET-URI=\"http://example.com/media-1-0.m3u8\"" + "\n", @@ -1554,6 +1619,67 @@ public void handleContentTimelineChanged_snapIn_snapToClosestSegmentBoundaryOfRe .build())); } + @Test + public void handleContentTimelineChanged_snapInWithExplicitResumeOffset_usesExplicitResumeOffset() + throws IOException { + assertThat( + callHandleContentTimelineChangedAndCaptureAdPlaybackState( + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:55:40.000Z\n" + + "#EXTINF:6.111,\nmain1.ts\n" + + "#EXTINF:6.111,\nmain2.ts\n" // segment start at 6.111 + + "#EXTINF:6.111,\nmain3.ts\n" // end of window at 18.333 + + "#EXT-X-ENDLIST" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T21:55:40.000\"," // + + "X-SNAP=\"IN\"," + + "X-RESUME-OFFSET=1.234," + + "X-PLAYOUT-LIMIT=3.222," + + "X-ASSET-URI=\"http://example.com/media-0-0.m3u8\"" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad1-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T21:55:52.222\"," + + "X-SNAP=\"IN\"," + + "X-RESUME-OFFSET=2.345," + + "X-PLAYOUT-LIMIT=4.333," + + "X-ASSET-URI=\"http://example.com/media-1-0.m3u8\"" + + "\n", + adsLoader, + /* windowIndex= */ 0, + /* windowPositionInPeriodUs= */ 0, + /* windowEndPositionInPeriodUs= */ C.TIME_END_OF_SOURCE)) + .isEqualTo( + new AdPlaybackState("adsId", /* adGroupTimesUs...= */ 0L, 12_222_000) + .withAdCount(/* adGroupIndex= */ 0, 1) + .withAdCount(/* adGroupIndex= */ 1, 1) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 1_234_000L) + .withContentResumeOffsetUs(/* adGroupIndex= */ 1, 2_345_000L) + .withAdDurationsUs(/* adGroupIndex= */ 0, 3_222_000L) + .withAdDurationsUs(/* adGroupIndex= */ 1, 4_333_000L) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, "ad0-0") + .withAdId(/* adGroupIndex= */ 1, /* adIndexInAdGroup= */ 0, "ad1-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + new MediaItem.Builder() + .setUri("http://example.com/media-0-0.m3u8") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withAvailableAdMediaItem( + /* adGroupIndex= */ 1, + /* adIndexInAdGroup= */ 0, + new MediaItem.Builder() + .setUri("http://example.com/media-1-0.m3u8") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build())); + } + @Test public void handleContentTimelineChanged_snapInLive_snapToClosestSegmentBoundaryOfResumptionPosition() @@ -1570,8 +1696,8 @@ public void handleContentTimelineChanged_snapIn_snapToClosestSegmentBoundaryOfRe + "ID=\"ad0-0\"," + "CLASS=\"com.apple.hls.interstitial\"," + "START-DATE=\"2020-01-02T21:55:40.111\"," + + "PLANNED-DURATION=4.333," + "X-SNAP=\"IN\"," - + "X-RESUME-OFFSET=3.222," // snap to end of first segment: 6.000 + "X-ASSET-URI=\"http://example.com/media-0-0.m3u8\"" + "\n" + "#EXT-X-DATERANGE:" @@ -1580,7 +1706,6 @@ public void handleContentTimelineChanged_snapIn_snapToClosestSegmentBoundaryOfRe + "START-DATE=\"2020-01-02T21:55:52.332\"," + "X-SNAP=\"IN\"," + "DURATION=4.333," - + "X-RESUME-OFFSET=4.333," // snap to end of first segment: 6.001 + "X-ASSET-URI=\"http://example.com/media-1-0.m3u8\"" + "\n" + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:55:40.000Z\n" @@ -1711,7 +1836,7 @@ public void handleContentTimelineChanged_snapInLiveBeforeOrAfterWindow_ignored() + "ID=\"ad0-0\"," // pre roll + "CLASS=\"com.apple.hls.interstitial\"," + "START-DATE=\"2020-01-02T21:55:48.111Z\"," // snap OUT to start of segment - + "X-RESUME-OFFSET=10.000," // snap IN to end of playlist: 12_222 + + "DURATION=10.000," // snap IN to end of playlist: 12_222 + "X-SNAP=\"OUT,IN\"," + "X-ASSET-URI=\"http://example.com/media-0-0.m3u8\"" + "\n" @@ -1726,7 +1851,7 @@ public void handleContentTimelineChanged_snapInLiveBeforeOrAfterWindow_ignored() + "ID=\"ad0-0\"," // pre roll + "CLASS=\"com.apple.hls.interstitial\"," + "START-DATE=\"2020-01-02T21:55:46.111Z\"," - + "X-RESUME-OFFSET=10.000," + + "DURATION=10.000," + "X-SNAP=\"IN,OUT\"," + "X-ASSET-URI=\"http://example.com/media-0-0.m3u8\"" + "\n" @@ -1740,6 +1865,148 @@ public void handleContentTimelineChanged_snapInLiveBeforeOrAfterWindow_ignored() expectedAdPlaybackState.withContentResumeOffsetUs(/* adGroupIndex= */ 0, 12_222_000L)); } + @Test + public void + handleContentTimelineChanged_snapInLiveWithPlayoutLimitLessThanDuration_deferredResumeOffsetUsesPlayoutLimit() + throws IOException { + AdPlaybackState expectedAdPlaybackState = + new AdPlaybackState("adsId", 6_111_000L) + .withLivePostrollPlaceholderAppended(/* isServerSideInserted= */ false) + .withAdCount(/* adGroupIndex= */ 0, 1) + .withAdDurationsUs(/* adGroupIndex= */ 0, 8_000_000L) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 8_000_000L) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + new MediaItem.Builder() + .setUri("http://example.com/media-0-0.m3u8") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()); + + assertThat( + callHandleContentTimelineChangedForLiveAndCaptureAdPlaybackStates( + adsLoader, + /* startAdsLoader= */ true, + /* windowOffsetInFirstPeriodUs= */ 0, + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:0\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T21:55:46.111Z\"," + + "DURATION=15.000," + + "X-PLAYOUT-LIMIT=8.000," + + "X-SNAP=\"OUT,IN\"," + + "X-ASSET-URI=\"http://example.com/media-0-0.m3u8\"" + + "\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:55:40.000Z\n" + + "#EXTINF:6.111,\nmain1.0.ts\n" + + "#EXTINF:6.111,\nmain2.0.ts\n" // end of window: 12:222 - 21:55:52.222 + + "\n", + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:0\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T21:55:46.111Z\"," + + "DURATION=15.000," + + "X-PLAYOUT-LIMIT=8.000," + + "X-SNAP=\"IN,OUT\"," + + "X-ASSET-URI=\"http://example.com/media-0-0.m3u8\"" + + "\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:55:40.000Z\n" + + "#EXTINF:6.111,\nmain1.0.ts\n" + + "#EXTINF:6.111,\nmain2.0.ts\n" // end of window: 12:222 - 21:55:52.222 + + "#EXTINF:6.111,\nmain3.0.ts\n" // new segment triggers deferred resolution + + "\n")) + .containsExactly( + expectedAdPlaybackState, + expectedAdPlaybackState.withContentResumeOffsetUs(/* adGroupIndex= */ 0, 6_111_000L)); + } + + @Test + public void handleContentTimelineChanged_snapInLiveMultipleDeferred_resolvesInOrder() + throws IOException { + AdPlaybackState firstAdPlaybackState = + new AdPlaybackState("adsId", /* adGroupTimesUs...= */ 0L, 6_111_000L) + .withLivePostrollPlaceholderAppended(/* isServerSideInserted= */ false) + .withAdCount(/* adGroupIndex= */ 0, 1) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 12_222_000L) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + new MediaItem.Builder() + .setUri("http://example.com/media-0-0.m3u8") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withAdCount(/* adGroupIndex= */ 1, 1) + .withContentResumeOffsetUs(/* adGroupIndex= */ 1, 10_000_000L) + .withAdId(/* adGroupIndex= */ 1, /* adIndexInAdGroup= */ 0, "ad1-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 1, + /* adIndexInAdGroup= */ 0, + new MediaItem.Builder() + .setUri("http://example.com/media-1-0.m3u8") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()); + + assertThat( + callHandleContentTimelineChangedForLiveAndCaptureAdPlaybackStates( + adsLoader, + /* startAdsLoader= */ true, + /* windowOffsetInFirstPeriodUs= */ 0, + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:0\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T21:55:40.000Z\"," + + "DURATION=10.000," + + "X-SNAP=\"IN,OUT\"," + + "X-ASSET-URI=\"http://example.com/media-0-0.m3u8\"\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad1-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T21:55:46.111Z\"," + + "DURATION=10.000," + + "X-SNAP=\"IN,OUT\"," + + "X-ASSET-URI=\"http://example.com/media-1-0.m3u8\"\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:55:40.000Z\n" + + "#EXTINF:6.111,\nmain1.0.ts\n" + + "#EXTINF:6.111,\nmain2.0.ts\n" + + "\n", + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-MEDIA-SEQUENCE:0\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T21:55:40.000Z\"," + + "DURATION=10.000," + + "X-SNAP=\"IN,OUT\"," + + "X-ASSET-URI=\"http://example.com/media-0-0.m3u8\"\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad1-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T21:55:46.111Z\"," + + "DURATION=10.000," + + "X-SNAP=\"IN,OUT\"," + + "X-ASSET-URI=\"http://example.com/media-1-0.m3u8\"\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:55:40.000Z\n" + + "#EXTINF:6.111,\nmain1.0.ts\n" + + "#EXTINF:6.111,\nmain2.0.ts\n" + + "#EXTINF:6.111,\nmain3.0.ts\n" + + "\n")) + .containsExactly( + firstAdPlaybackState, + firstAdPlaybackState.withContentResumeOffsetUs(/* adGroupIndex= */ 1, 12_222_000L)); + } + @Test public void handleContentTimelineChanged_snapInOut_snapToSameSegmentBoundaryMergedIntoSameAdGroup() @@ -1766,7 +2033,7 @@ public void handleContentTimelineChanged_snapInLiveBeforeOrAfterWindow_ignored() + "CLASS=\"com.apple.hls.interstitial\"," + "START-DATE=\"2020-01-02T21:55:47.111\"," // snaps OUT to 6_111_000 + "X-SNAP=\"OUT,IN\"," - + "X-RESUME-OFFSET=3.111," // Changed by snap IN to 6.111 + + "DURATION=3.111," // Changed by snap IN to 6.111 + "X-ASSET-URI=\"http://example.com/media-0-1.m3u8\"" + "\n", adsLoader, @@ -1812,7 +2079,7 @@ public void handleContentTimelineChanged_snapInLiveBeforeOrAfterWindow_ignored() + "CLASS=\"com.apple.hls.interstitial\"," + "START-DATE=\"2020-01-02T21:55:45.000\"," // snaps OUT to 6_111_000L + "X-SNAP=\"OUT,IN\"," - + "X-RESUME-OFFSET=4.222," // snaps IN to 6.111 + + "DURATION=4.222," // snaps IN to 6.111 + "X-ASSET-URI=\"http://example.com/media-0-0.m3u8\"" + "\n" + "#EXT-X-DATERANGE:" @@ -2535,43 +2802,36 @@ public void handleContentTimelineChanged_preRollWithAssetList_resolveAssetListIm } @Test - public void - handleContentTimelineChanged_preRollWithAssetList_resolvesAndSchedulesNextPlayerMessage() - throws IOException, TimeoutException { + public void handleContentTimelineChanged_assetListResolvedTwice_correctContentResumeOffsetUs() + throws IOException, TimeoutException { String playlistString = "#EXTM3U\n" - + "#EXT-X-TARGETDURATION:9\n" - + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:00.000Z\n" - + "#EXTINF:9,\n" - + "main0.ts\n" - + "#EXTINF:81,\n" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-PROGRAM-DATE-TIME:2026-07-14T08:56:00.000Z\n" + + "#EXTINF:6.037333,\n" + "main0.ts\n" + + "#EXTINF:6.037333,\n" + + "main1.ts\n" + + "#EXTINF:6.037333,\n" + + "main2.ts\n" + + "#EXTINF:6.037333,\n" + + "main3.ts\n" + + "#EXTINF:6.037333,\n" + + "main4.ts\n" + + "#EXTINF:6.037333,\n" + + "main5.ts\n" + "#EXT-X-ENDLIST" + "\n" + "#EXT-X-DATERANGE:" - + "ID=\"ad0-0\"," - + "CLASS=\"com.apple.hls.interstitial\"," - + "START-DATE=\"2020-01-02T22:00:00.000Z\"," - + "CUE=\"PRE\"," - + "X-ASSET-LIST=\"http://example.com/assetlist-0-0.json\"" - + "\n" - + "#EXT-X-DATERANGE:" - + "ID=\"ad1-0\"," + + "ID=\"ad-break-12338\"," + "CLASS=\"com.apple.hls.interstitial\"," - + "START-DATE=\"2020-01-02T21:01:21.000Z\"," - + "X-ASSET-LIST=\"http://example.com/assetlist-1-0.json\"" + + "START-DATE=\"2026-07-14T08:56:17.860Z\"," + + "PLANNED-DURATION=12.074666," + + "X-PLAYOUT-LIMIT=12.074666," + + "X-ASSET-LIST=\"asset-list-two-ads\"," + + "X-SNAP=\"OUT,IN\"" + "\n"; when(mockPlayer.getContentPosition()).thenReturn(0L); - PlayerMessage midRollPlayerMessage = - new PlayerMessage( - mock(PlayerMessage.Sender.class), - mock(PlayerMessage.Target.class), - Timeline.EMPTY, - /* defaultMediaItemIndex= */ 0, - /* Clock ignored */ null, - /* Looper ignored */ null); - when(mockPlayer.createMessage(any())).thenReturn(midRollPlayerMessage); - callHandleContentTimelineChangedAndCaptureAdPlaybackState( playlistString, adsLoader, @@ -2580,263 +2840,1292 @@ public void handleContentTimelineChanged_preRollWithAssetList_resolveAssetListIm /* windowEndPositionInPeriodUs= */ C.TIME_END_OF_SOURCE); runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); - verify(mockAdsLoaderListener) - .onAssetListLoadCompleted(eq(contentMediaItem), eq("adsId"), eq(0), eq(0), any(), any()); - assertThat(midRollPlayerMessage.getPositionMs()).isEqualTo(54_000L); - assertThat(midRollPlayerMessage.getPayload()).isEqualTo(contentMediaItem); - assertThat(midRollPlayerMessage.getLooper()).isEqualTo(Looper.myLooper()); - InOrder inOrder = inOrder(mockPlayer); - inOrder.verify(mockPlayer).addListener(any()); - verifyTimelineUpdate( - inOrder, - mockPlayer, - /* verifyMessageScheduled= */ false, - /* currentTimelineRequestCount= */ 1); - verifyAssetListLoadCompleted(inOrder, mockPlayer, /* verifyMessageScheduled= */ true); - inOrder.verifyNoMoreInteractions(); + + assetListLoadingListener.reset(); + adsLoader.setWithSkippedAdGroup(/* adGroupIndex= */ 0); + assertThat(adsLoader.setWithAssetListReset("adsId", "ad-break-12338")).isTrue(); + runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); + + ArgumentCaptor adPlaybackStateCaptor = + ArgumentCaptor.forClass(AdPlaybackState.class); + verify(mockEventListener, times(5)).onAdPlaybackState(adPlaybackStateCaptor.capture()); + List capturedStates = adPlaybackStateCaptor.getAllValues(); + AdPlaybackState initialState = capturedStates.get(0); + AdPlaybackState firstResolvedState = capturedStates.get(1); + AdPlaybackState skippedState = capturedStates.get(2); + AdPlaybackState resetState = capturedStates.get(3); + AdPlaybackState secondResolvedState = capturedStates.get(4); + assertThat(initialState.adGroupCount).isEqualTo(1); + assertThat(initialState.getAdGroup(0).contentResumeOffsetUs).isEqualTo(12_074_666L); + assertThat(firstResolvedState.adGroupCount).isEqualTo(1); + assertThat(firstResolvedState.getAdGroup(/* adGroupIndex= */ 0).count).isEqualTo(2); + assertThat(firstResolvedState.getAdGroup(/* adGroupIndex= */ 0).contentResumeOffsetUs) + .isEqualTo(12_074_666L); + assertThat(skippedState.getAdGroup(/* adGroupIndex= */ 0).count).isEqualTo(2); + assertThat(skippedState.getAdGroup(/* adGroupIndex= */ 0).states) + .asList() + .containsExactly(AD_STATE_SKIPPED, AD_STATE_SKIPPED); + assertThat(resetState.getAdGroup(/* adGroupIndex= */ 0).count).isEqualTo(1); + assertThat(resetState.getAdGroup(/* adGroupIndex= */ 0).states) + .asList() + .containsExactly(AD_STATE_UNAVAILABLE); + assertThat(secondResolvedState.getAdGroup(/* adGroupIndex= */ 0).count).isEqualTo(2); + assertThat(secondResolvedState.getAdGroup(/* adGroupIndex= */ 0).states) + .asList() + .containsExactly(AD_STATE_AVAILABLE, AD_STATE_AVAILABLE); + assertThat(secondResolvedState.getAdGroup(/* adGroupIndex= */ 0).contentResumeOffsetUs) + .isEqualTo(12_074_666L); } @Test - public void - handleContentTimelineChanged_midRollCloserToPreviousAdThan3TimesTargetDuration_schedulesNextPlayerMessageAtEndOfPreviousAdGroup() - throws IOException, TimeoutException { + public void handleContentTimelineChanged_assetListResolved_calculatesContentResumeOffsetUs() + throws Exception { String playlistString = "#EXTM3U\n" - + "#EXT-X-TARGETDURATION:9\n" - + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:00.000Z\n" - + "#EXTINF:9,\n" - + "main0.ts\n" - + "#EXTINF:81,\n" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-PROGRAM-DATE-TIME:2026-07-14T08:56:00.000Z\n" + + "#EXTINF:6.037333,\n" + "main0.ts\n" - + "#EXT-X-ENDLIST" + + "#EXTINF:6.037333,\n" + + "main1.ts" + "\n" + "#EXT-X-DATERANGE:" - // The pre roll ad has a content resume offset equal to its duration of 10.123s. - // The duration is set by the asset list JSON loaded during the test. - + "ID=\"ad0-0\"," + + "ID=\"ad-break-12338\"," + "CLASS=\"com.apple.hls.interstitial\"," - + "START-DATE=\"2020-01-02T22:00:00.000Z\"," - + "CUE=\"PRE\"," - + "X-ASSET-LIST=\"http://example.com/assetlist-0-0.json\"" + + "START-DATE=\"2026-07-14T08:56:17.860Z\"," + + "PLANNED-DURATION=12.074665," + + "X-PLAYOUT-LIMIT=12.074665," + + "X-ASSET-LIST=\"asset-list-two-ads\"," + + "X-SNAP=\"OUT,IN\"," + + "\n" + + "#EXTINF:6.037333,\n" + + "main2.ts" + + "\n" + + "#EXTINF:6.037333,\n" + + "main3.ts" + + "\n"; + String playlistString2 = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-PROGRAM-DATE-TIME:2026-07-14T08:56:00.000Z\n" + + "#EXTINF:6.037333,\n" + + "main0.ts\n" + + "#EXTINF:6.037333,\n" + + "main1.ts" + "\n" + "#EXT-X-DATERANGE:" - + "ID=\"ad1-0\"," + + "ID=\"ad-break-12338\"," + "CLASS=\"com.apple.hls.interstitial\"," - // trigger at 00:02.000 (29 - (3 * 9)) is within content resume offset - + "START-DATE=\"2020-01-02T21:00:29.000Z\"," - + "X-ASSET-LIST=\"http://example.com/assetlist-1-0.json\"" + + "START-DATE=\"2026-07-14T08:56:17.860Z\"," + + "PLANNED-DURATION=12.074665," + + "X-PLAYOUT-LIMIT=12.074665," + + "X-ASSET-LIST=\"asset-list-two-ads\"," + + "X-SNAP=\"OUT,IN\"," + + "\n" + + "#EXTINF:6.037333,\n" + + "main2.ts" + + "\n" + + "#EXTINF:6.037333,\n" + + "main3.ts" + + "\n" + + "#EXTINF:6.037333,\n" + + "main4.ts" + "\n"; when(mockPlayer.getContentPosition()).thenReturn(0L); - PlayerMessage midRollPlayerMessage = - new PlayerMessage( - mock(PlayerMessage.Sender.class), - mock(PlayerMessage.Target.class), - Timeline.EMPTY, - /* defaultMediaItemIndex= */ 0, - /* Clock ignored */ null, - /* Looper ignored */ null); - when(mockPlayer.createMessage(any())).thenReturn(midRollPlayerMessage); + when(mockPlayer.getCurrentMediaItem()).thenReturn(contentMediaItem); + when(mockPlayer.getCurrentTimeline()).thenReturn(new FakeTimeline(contentWindowDefinition)); - callHandleContentTimelineChangedAndCaptureAdPlaybackState( - playlistString, + callHandleContentTimelineChangedForLiveAndCaptureAdPlaybackStates( adsLoader, - /* windowIndex= */ 0, - /* windowPositionInPeriodUs= */ 0, - /* windowEndPositionInPeriodUs= */ C.TIME_END_OF_SOURCE); + /* startAdsLoader= */ true, + /* windowOffsetInFirstPeriodUs= */ 0, + playlistString, + playlistString2); runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); - // Assert the message is scheduled at the position right after the pre roll's resume offset. - assertThat(midRollPlayerMessage.getPositionMs()).isEqualTo(10_123L); + ArgumentCaptor adPlaybackStateCaptor = + ArgumentCaptor.forClass(AdPlaybackState.class); + verify(mockEventListener, times(3)).onAdPlaybackState(adPlaybackStateCaptor.capture()); + List capturedStates = adPlaybackStateCaptor.getAllValues(); + AdPlaybackState initialState = capturedStates.get(0); + AdPlaybackState firstResolvedState = capturedStates.get(1); + AdPlaybackState secondResolvedState = capturedStates.get(2); + assertThat(initialState.adGroupCount).isEqualTo(2); + assertThat(initialState.getAdGroup(/* adGroupIndex= */ 0).count).isEqualTo(1); + assertThat(initialState.getAdGroup(/* adGroupIndex= */ 0).contentResumeOffsetUs) + .isEqualTo(12_074_665L); + assertThat(initialState.getAdGroup(/* adGroupIndex= */ 0).states) + .asList() + .containsExactly(AD_STATE_UNAVAILABLE); + assertThat(firstResolvedState.adGroupCount).isEqualTo(2); + assertThat(firstResolvedState.getAdGroup(/* adGroupIndex= */ 0).count).isEqualTo(1); + assertThat(firstResolvedState.getAdGroup(/* adGroupIndex= */ 0).contentResumeOffsetUs) + .isEqualTo(12_074_666L); + assertThat(firstResolvedState.getAdGroup(/* adGroupIndex= */ 0).states) + .asList() + .containsExactly(AD_STATE_UNAVAILABLE); + assertThat(secondResolvedState.getAdGroup(/* adGroupIndex= */ 0).count).isEqualTo(2); + assertThat(secondResolvedState.getAdGroup(/* adGroupIndex= */ 0).states) + .asList() + .containsExactly(AD_STATE_AVAILABLE, AD_STATE_AVAILABLE); + assertThat(secondResolvedState.getAdGroup(/* adGroupIndex= */ 0).contentResumeOffsetUs) + .isEqualTo(12_074_666L); } @Test public void - handleContentTimelineChanged_liveMidRollCloserToPreviousAdThan3TimesTargetDuration_schedulesNextPlayerMessageAtEndOfPreviousAdGroup() + handleContentTimelineChanged_assetListResolvedWithExplicitResumeOffset_usesExplicitResumeOffsetUs() throws IOException, TimeoutException { - when(mockPlayer.getContentPosition()).thenReturn(0L); - PlayerMessage midRollPlayerMessage = - new PlayerMessage( - mock(PlayerMessage.Sender.class), - mock(PlayerMessage.Target.class), - Timeline.EMPTY, - /* defaultMediaItemIndex= */ 0, - /* Clock ignored */ null, - /* Looper ignored */ null); - when(mockPlayer.createMessage(any())).thenReturn(midRollPlayerMessage); - when(mockPlayer.getCurrentMediaItem()).thenReturn(contentMediaItem); - - callHandleContentTimelineChangedForLiveAndCaptureAdPlaybackStates( - adsLoader, - /* startAdsLoader= */ true, - /* windowOffsetInFirstPeriodUs= */ 0L, - "#EXTM3U\n" - + "#EXT-X-TARGETDURATION:9\n" - // window.positionInFirstPeriod = 0 - + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:00.000Z\n" - + "#EXTINF:9,\n" - + "main0.ts\n" - + "#EXTINF:9,\n" - + "main0.ts\n" - + "#EXTINF:9,\n" - + "main0.ts\n" - + "\n", + String playlistString = "#EXTM3U\n" - + "#EXT-X-TARGETDURATION:9\n" - // window.positionInFirstPeriod = 9_000_000 - + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:09.000Z\n" - + "#EXTINF:9,\n" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-PROGRAM-DATE-TIME:2026-07-14T08:56:00.000Z\n" + + "#EXTINF:6.037333,\n" + "main0.ts\n" - + "#EXTINF:9,\n" - + "main0.ts" + + "#EXTINF:6.037333,\n" + + "main1.ts" + "\n" - // adGroup.timeUs = 18_000_900 (window positionMs: 9_000; resumes 29_123) + "#EXT-X-DATERANGE:" - + "ID=\"ad0-0\"," + + "ID=\"ad-break-12338\"," + "CLASS=\"com.apple.hls.interstitial\"," - + "START-DATE=\"2020-01-02T21:00:18.000Z\"," - + "DURATION=20.123," - + "X-ASSET-URI=\"http://example.com/media-0-0.ts\"" - + "\n", + + "START-DATE=\"2026-07-14T08:56:17.860Z\"," + + "PLANNED-DURATION=12.074665," + + "X-PLAYOUT-LIMIT=12.074665," + + "X-RESUME-OFFSET=1.2345," + + "X-ASSET-LIST=\"asset-list-two-ads\"," + + "X-SNAP=\"OUT,IN\"," + + "\n" + + "#EXTINF:6.037333,\n" + + "main2.ts" + + "\n" + + "#EXTINF:6.037333,\n" + + "main3.ts" + + "\n"; + String playlistString2 = "#EXTM3U\n" - + "#EXT-X-TARGETDURATION:9\n" - // window.positionInFirstPeriod = 18_000_000 - + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:18.000Z\n" - + "#EXTINF:9,\n" - + "main0.ts" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-PROGRAM-DATE-TIME:2026-07-14T08:56:00.000Z\n" + + "#EXTINF:6.037333,\n" + + "main0.ts\n" + + "#EXTINF:6.037333,\n" + + "main1.ts" + "\n" + "#EXT-X-DATERANGE:" - + "ID=\"ad0-0\"," + + "ID=\"ad-break-12338\"," + "CLASS=\"com.apple.hls.interstitial\"," - // adGroup.timeUs = 18_000_000 (window positionMs: 0; resumes 20_123) - + "START-DATE=\"2020-01-02T21:00:18.000Z\"," - + "DURATION=20.123," - + "X-ASSET-URI=\"http://example.com/media-0-0.ts\"" + + "START-DATE=\"2026-07-14T08:56:17.860Z\"," + + "PLANNED-DURATION=12.074665," + + "X-PLAYOUT-LIMIT=12.074665," + + "X-RESUME-OFFSET=1.2345," + + "X-ASSET-LIST=\"asset-list-two-ads\"," + + "X-SNAP=\"OUT,IN\"," + "\n" - + "#EXTINF:81,\n" - + "main0.ts" + + "#EXTINF:6.037333,\n" + + "main2.ts" + "\n" - + "#EXT-X-DATERANGE:" - + "ID=\"ad1-0\"," - + "CLASS=\"com.apple.hls.interstitial\"," - // adGroup.timeUs = 58_000_000 (window positionMs: 40_000) - // Message trigger at window positionMs: 40_000 - 27_000 = 13_000 which is in the resume - // offset of the previous ad. - + "START-DATE=\"2020-01-02T21:00:58.000Z\"," - + "X-ASSET-LIST=\"http://example.com/assetlist-1-0.json\"" - + "\n"); + + "#EXTINF:6.037333,\n" + + "main3.ts" + + "\n" + + "#EXTINF:6.037333,\n" + + "main4.ts" + + "\n"; + when(mockPlayer.getContentPosition()).thenReturn(0L); + when(mockPlayer.getCurrentMediaItem()).thenReturn(contentMediaItem); + when(mockPlayer.getCurrentTimeline()).thenReturn(new FakeTimeline(contentWindowDefinition)); - runMainLooperUntil( - () -> midRollPlayerMessage.getPositionMs() != C.TIME_UNSET, TIMEOUT_MS, Clock.DEFAULT); - // Assert that the message is scheduled at the position right after the mid roll's resume - // offset. At the moment of scheduling, the most recent timeline/playlist has not yet been - // published. Hence the message must be scheduled at the window position of the last published - // timeline, which was created with the previous playlist. - assertThat(midRollPlayerMessage.getPositionMs()).isEqualTo(29_123L); + callHandleContentTimelineChangedForLiveAndCaptureAdPlaybackStates( + adsLoader, + /* startAdsLoader= */ true, + /* windowOffsetInFirstPeriodUs= */ 0, + playlistString, + playlistString2); + + runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); + ArgumentCaptor adPlaybackStateCaptor = + ArgumentCaptor.forClass(AdPlaybackState.class); + verify(mockEventListener, times(2)).onAdPlaybackState(adPlaybackStateCaptor.capture()); + List capturedStates = adPlaybackStateCaptor.getAllValues(); + AdPlaybackState initialState = capturedStates.get(0); + AdPlaybackState firstResolvedState = capturedStates.get(1); + assertThat(initialState.adGroupCount).isEqualTo(2); + assertThat(initialState.getAdGroup(/* adGroupIndex= */ 0).count).isEqualTo(1); + assertThat(initialState.getAdGroup(/* adGroupIndex= */ 0).contentResumeOffsetUs) + .isEqualTo(1_234_500L); + assertThat(initialState.getAdGroup(/* adGroupIndex= */ 0).states) + .asList() + .containsExactly(AD_STATE_UNAVAILABLE); + assertThat(firstResolvedState.adGroupCount).isEqualTo(2); + assertThat(firstResolvedState.getAdGroup(/* adGroupIndex= */ 0).count).isEqualTo(2); + assertThat(firstResolvedState.getAdGroup(/* adGroupIndex= */ 0).contentResumeOffsetUs) + .isEqualTo(1_234_500L); + assertThat(firstResolvedState.getAdGroup(/* adGroupIndex= */ 0).states) + .asList() + .containsExactly(AD_STATE_AVAILABLE, AD_STATE_AVAILABLE); } @Test public void - handleContentTimelineChanged_livePlayingOnLastSegmentWhenAdInserted_triggersAssetLoading() + handleContentTimelineChanged_durationChangedByAssetList_deferredCalculationSnapsToSegmentEnd() throws Exception { - AssetListLoadingListener assetListLoadingListener = new AssetListLoadingListener(); - adsLoader.addListener(assetListLoadingListener); + when(mockPlayer.getContentPosition()).thenReturn(0L); when(mockPlayer.getCurrentMediaItem()).thenReturn(contentMediaItem); - when(mockPlayer.getContentPosition()).thenReturn(27_000L); // content pos in first window + when(mockPlayer.getCurrentTimeline()).thenReturn(new FakeTimeline(contentWindowDefinition)); callHandleContentTimelineChangedForLiveAndCaptureAdPlaybackStates( adsLoader, /* startAdsLoader= */ true, - /* windowOffsetInFirstPeriodUs= */ 0L, - "#EXTM3U\n" - + "#EXT-X-TARGETDURATION:9\n" - + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:00.000Z\n" - + "#EXTINF:9,\n" - + "main0.ts\n" - + "#EXTINF:9,\n" - + "main0.ts\n" - + "#EXTINF:9,\n" // ends at pos 26_999 - + "main0.ts" - + "\n", + /* windowOffsetInFirstPeriodUs= */ 0, "#EXTM3U\n" - + "#EXT-X-TARGETDURATION:9\n" - + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:09.000Z\n" - + "#EXTINF:9,\n" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-PROGRAM-DATE-TIME:2026-07-14T08:56:00.000Z\n" + + "#EXTINF:6.037333,\n" + "main0.ts\n" - + "#EXTINF:9,\n" - + "main0.ts" + + "#EXTINF:6.037333,\n" + + "main1.ts" + "\n" + "#EXT-X-DATERANGE:" - + "ID=\"ad1-0\"," + + "ID=\"ad-break-12338\"," + "CLASS=\"com.apple.hls.interstitial\"," - + "START-DATE=\"2020-01-02T21:00:27.000Z\"," - + "DURATION=20.123," - + "X-ASSET-LIST=\"http://example.com/media-1-0.ts\"" + + "START-DATE=\"2026-07-14T08:56:17.860Z\"," + + "PLANNED-DURATION=12.074665," // Initial duration defined by playlist. + + "X-ASSET-LIST=\"asset-list-two-ads\"," + + "X-SNAP=\"OUT,IN\"," + "\n" - + "#EXTINF:9,\n" // Starts at pos 18 in second window; at pos 27 in first window. - + "main0.ts" + + "#EXTINF:6.037333,\n" + + "main2.ts" + + "\n" + + "#EXTINF:6.037333,\n" + + "main3.ts" + "\n"); runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); - ArgumentCaptor captor = ArgumentCaptor.forClass(AdPlaybackState.class); - verify(mockEventListener, times(3)).onAdPlaybackState(captor.capture()); - assertThat(captor.getAllValues().get(2).getAdGroup(0).mediaItems[0]) - .isEqualTo( - new MediaItem.Builder() - .setUri("http://0") - .setMimeType(MimeTypes.APPLICATION_M3U8) - .build()); - } + ArgumentCaptor adPlaybackStateCaptor = + ArgumentCaptor.forClass(AdPlaybackState.class); + verify(mockEventListener, times(2)).onAdPlaybackState(adPlaybackStateCaptor.capture()); + List capturedStates = adPlaybackStateCaptor.getAllValues(); + AdPlaybackState initialState = capturedStates.get(0); + AdPlaybackState firstResolvedState = capturedStates.get(1); + assertThat(initialState.adGroupCount).isEqualTo(2); + assertThat(initialState.getAdGroup(/* adGroupIndex= */ 0).count).isEqualTo(1); + assertThat(initialState.getAdGroup(/* adGroupIndex= */ 0).contentResumeOffsetUs) + .isEqualTo(12_074_665L); // Interstitial duration in the playlist. + assertThat(initialState.getAdGroup(/* adGroupIndex= */ 0).states) + .asList() + .containsExactly(AD_STATE_UNAVAILABLE); + assertThat(firstResolvedState.adGroupCount).isEqualTo(2); + assertThat(firstResolvedState.getAdGroup(/* adGroupIndex= */ 0).count).isEqualTo(2); + assertThat(firstResolvedState.getAdGroup(/* adGroupIndex= */ 0).contentResumeOffsetUs) + .isEqualTo(12_074_664L); // Snap segment not available: durations from asset list. + assertThat(firstResolvedState.getAdGroup(/* adGroupIndex= */ 0).states) + .asList() + .containsExactly(AD_STATE_AVAILABLE, AD_STATE_AVAILABLE); - @Test - public void - handleContentTimelineChanged_startPositionAfterMidRollTimeUs_resolvesAndSchedulesMidRoll() - throws IOException, TimeoutException { - String playlistString = + reset(mockEventListener); + callHandleContentTimelineChangedForLiveAndCaptureAdPlaybackStates( + adsLoader, + /* startAdsLoader= */ false, + /* windowOffsetInFirstPeriodUs= */ 0, "#EXTM3U\n" - + "#EXT-X-TARGETDURATION:30\n" - + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:00.000Z\n" - + "#EXTINF:30,\n" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-PROGRAM-DATE-TIME:2026-07-14T08:56:00.000Z\n" + + "#EXTINF:6.037333,\n" + "main0.ts\n" - + "#EXTINF:30,\n" - + "main1.ts\n" - + "#EXTINF:30,\n" - + "main2.ts\n" - + "#EXTINF:30,\n" - + "main3.ts\n" - + "#EXT-X-ENDLIST" + + "#EXTINF:6.037333,\n" + + "main1.ts" + "\n" + "#EXT-X-DATERANGE:" - + "ID=\"ad0-0\"," + + "ID=\"ad-break-12338\"," + "CLASS=\"com.apple.hls.interstitial\"," - + "START-DATE=\"2020-01-02T21:00:00.000Z\"," - + "CUE=\"PRE\"," - + "X-ASSET-LIST=\"http://example.com/assetlist-0-0.json\"" + + "START-DATE=\"2026-07-14T08:56:17.860Z\"," + + "PLANNED-DURATION=12.074665," // Initial duration defined by playlist. + + "X-ASSET-LIST=\"asset-list-two-ads\"," + + "X-SNAP=\"OUT,IN\"," + + "\n" + + "#EXTINF:6.037333,\n" + + "main2.ts" + + "\n" + + "#EXTINF:6.037333,\n" + + "main3.ts" + + "\n" + + "#EXTINF:6.037333,\n" + + "main4.ts" + + "\n"); + + verify(mockEventListener).onAdPlaybackState(adPlaybackStateCaptor.capture()); + AdPlaybackState secondResolvedState = adPlaybackStateCaptor.getValue(); + assertThat(secondResolvedState.getAdGroup(/* adGroupIndex= */ 0).count).isEqualTo(2); + assertThat(secondResolvedState.getAdGroup(/* adGroupIndex= */ 0).states) + .asList() + .containsExactly(AD_STATE_AVAILABLE, AD_STATE_AVAILABLE); + assertThat(secondResolvedState.getAdGroup(/* adGroupIndex= */ 0).contentResumeOffsetUs) + .isEqualTo(12_074_666L); // The offset snapped to the end of the segment. + } + + @Test + public void + handleContentTimelineChanged_assetListShorterThanPlannedDurationSnapsImmediately_removesPendingSnapInResolution() + throws Exception { + when(mockPlayer.getContentPosition()).thenReturn(0L); + when(mockPlayer.getCurrentMediaItem()).thenReturn(contentMediaItem); + when(mockPlayer.getCurrentTimeline()).thenReturn(new FakeTimeline(contentWindowDefinition)); + callHandleContentTimelineChangedForLiveAndCaptureAdPlaybackStates( + adsLoader, + /* startAdsLoader= */ true, + /* windowOffsetInFirstPeriodUs= */ 0, + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-PROGRAM-DATE-TIME:2026-07-14T08:56:00.000Z\n" + + "#EXTINF:6.037333,\n" + + "main0.ts\n" + + "#EXTINF:6.037333,\n" + + "main1.ts" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad-break-12338\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2026-07-14T08:56:05.800Z\"," + + "PLANNED-DURATION=20.0," // Exceeds initial playlist end, so snap-in is deferred. + + "X-ASSET-LIST=\"asset-list-two-ads\"," + + "X-SNAP=\"OUT,IN\"," + + "\n" + + "#EXTINF:6.037333,\n" + + "main2.ts" + + "\n" + + "#EXTINF:6.037333,\n" + + "main3.ts" + + "\n"); + runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); + ArgumentCaptor adPlaybackStateCaptor = + ArgumentCaptor.forClass(AdPlaybackState.class); + verify(mockEventListener, times(2)).onAdPlaybackState(adPlaybackStateCaptor.capture()); + List capturedStates = adPlaybackStateCaptor.getAllValues(); + AdPlaybackState initialState = capturedStates.get(0); + AdPlaybackState resolvedState = capturedStates.get(1); + assertThat(initialState.getAdGroup(/* adGroupIndex= */ 0).contentResumeOffsetUs) + .isEqualTo(20_000_000L); + assertThat(resolvedState.getAdGroup(/* adGroupIndex= */ 0).contentResumeOffsetUs) + .isEqualTo(12_074_666L); + reset(mockEventListener); + + callHandleContentTimelineChangedForLiveAndCaptureAdPlaybackStates( + adsLoader, + /* startAdsLoader= */ false, + /* windowOffsetInFirstPeriodUs= */ 6_037_333L, + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-PROGRAM-DATE-TIME:2026-07-14T08:56:00.000Z\n" + + "#EXTINF:6.037333,\n" + + "main0.ts\n" + + "#EXTINF:6.037333,\n" + + "main1.ts" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad-break-12338\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2026-07-14T08:56:05.800Z\"," + + "PLANNED-DURATION=20.0," + + "X-ASSET-LIST=\"asset-list-two-ads\"," + + "X-SNAP=\"OUT,IN\"," + + "\n" + + "#EXTINF:6.037333,\n" + + "main2.ts" + + "\n" + + "#EXTINF:6.037333,\n" + + "main3.ts" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad-break-12339\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2026-07-14T08:56:24.149Z\"," + + "X-ASSET-URI=\"http://example.com/ad2.m3u8\"" + + "\n" + + "#EXTINF:6.037333,\n" + + "main4.ts" + + "\n"); + + verify(mockEventListener).onAdPlaybackState(adPlaybackStateCaptor.capture()); + AdPlaybackState stateAfterPlaylistAdvance = adPlaybackStateCaptor.getValue(); + assertThat(stateAfterPlaylistAdvance.getAdGroup(/* adGroupIndex= */ 0).contentResumeOffsetUs) + .isEqualTo(12_074_666L); + } + + @Test + public void + handleContentTimelineChanged_undefinedDurationChangedByAssetList_deferredCalculationSnapsToSegmentEnd() + throws Exception { + when(mockPlayer.getContentPosition()).thenReturn(0L); + when(mockPlayer.getCurrentMediaItem()).thenReturn(contentMediaItem); + when(mockPlayer.getCurrentTimeline()).thenReturn(new FakeTimeline(contentWindowDefinition)); + + callHandleContentTimelineChangedForLiveAndCaptureAdPlaybackStates( + adsLoader, + /* startAdsLoader= */ true, + /* windowOffsetInFirstPeriodUs= */ 0, + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-PROGRAM-DATE-TIME:2026-07-14T08:56:00.000Z\n" + + "#EXTINF:6.037333,\n" + + "main0.ts\n" + + "#EXTINF:6.037333,\n" + + "main1.ts" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad-break-12338\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2026-07-14T08:56:17.860Z\"," + + "X-ASSET-LIST=\"asset-list-two-ads\"," + + "X-SNAP=\"OUT,IN\"," // Snap requested with undefined duration (defaults to 0). + + "\n" + + "#EXTINF:6.037333,\n" + + "main2.ts" + + "\n" + + "#EXTINF:6.037333,\n" + + "main3.ts" + + "\n"); + + runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); + ArgumentCaptor adPlaybackStateCaptor = + ArgumentCaptor.forClass(AdPlaybackState.class); + verify(mockEventListener, times(2)).onAdPlaybackState(adPlaybackStateCaptor.capture()); + List capturedStates = adPlaybackStateCaptor.getAllValues(); + AdPlaybackState initialState = capturedStates.get(0); + AdPlaybackState firstResolvedState = capturedStates.get(1); + assertThat(initialState.adGroupCount).isEqualTo(2); + assertThat(initialState.getAdGroup(/* adGroupIndex= */ 0).count).isEqualTo(1); + assertThat(initialState.getAdGroup(/* adGroupIndex= */ 0).contentResumeOffsetUs) + .isEqualTo(0); // Undefined duration in the playlist defaults to 0. + assertThat(initialState.getAdGroup(/* adGroupIndex= */ 0).states) + .asList() + .containsExactly(AD_STATE_UNAVAILABLE); + assertThat(firstResolvedState.adGroupCount).isEqualTo(2); + assertThat(firstResolvedState.getAdGroup(/* adGroupIndex= */ 0).count).isEqualTo(2); + assertThat(firstResolvedState.getAdGroup(/* adGroupIndex= */ 0).contentResumeOffsetUs) + .isEqualTo(12_074_664L); // Snap segment not available: durations from asset list. + assertThat(firstResolvedState.getAdGroup(/* adGroupIndex= */ 0).states) + .asList() + .containsExactly(AD_STATE_AVAILABLE, AD_STATE_AVAILABLE); + + reset(mockEventListener); + callHandleContentTimelineChangedForLiveAndCaptureAdPlaybackStates( + adsLoader, + /* startAdsLoader= */ false, + /* windowOffsetInFirstPeriodUs= */ 0, + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-PROGRAM-DATE-TIME:2026-07-14T08:56:00.000Z\n" + + "#EXTINF:6.037333,\n" + + "main0.ts\n" + + "#EXTINF:6.037333,\n" + + "main1.ts" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad-break-12338\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2026-07-14T08:56:17.860Z\"," + + "PLANNED-DURATION=12.074665," + + "X-PLAYOUT-LIMIT=12.074665," + + "X-ASSET-LIST=\"asset-list-two-ads\"," + + "X-SNAP=\"OUT,IN\"," // Snap requested with duration available from asset list. + + "\n" + + "#EXTINF:6.037333,\n" + + "main2.ts" + + "\n" + + "#EXTINF:6.037333,\n" + + "main3.ts" + + "\n" + + "#EXTINF:6.037333,\n" + + "main4.ts" + + "\n"); + + verify(mockEventListener).onAdPlaybackState(adPlaybackStateCaptor.capture()); + AdPlaybackState secondResolvedState = adPlaybackStateCaptor.getValue(); + assertThat(secondResolvedState.getAdGroup(/* adGroupIndex= */ 0).count).isEqualTo(2); + assertThat(secondResolvedState.getAdGroup(/* adGroupIndex= */ 0).states) + .asList() + .containsExactly(AD_STATE_AVAILABLE, AD_STATE_AVAILABLE); + assertThat(secondResolvedState.getAdGroup(/* adGroupIndex= */ 0).contentResumeOffsetUs) + .isEqualTo(12_074_666L); // The offset snapped to the end of the segment. + // Reset the listener to be able to wait for another asset list resolution to happen. + assetListLoadingListener.reset(); + // Skip ads to allow resetting the asset list. + adsLoader.setWithSkippedAdGroup(/* adGroupIndex= */ 0); + + // Reset the asset list which removes the known durations and recalculates resume offset again. + assertThat(adsLoader.setWithAssetListReset("adsId", /* adGroupIndex= */ 0)).isTrue(); + + verify(mockEventListener, times(3)).onAdPlaybackState(adPlaybackStateCaptor.capture()); + AdPlaybackState adPlaybackStateAfterAssetReset = adPlaybackStateCaptor.getValue(); + assertThat(adPlaybackStateAfterAssetReset.getAdGroup(/* adGroupIndex= */ 0).count).isEqualTo(1); + assertThat(adPlaybackStateAfterAssetReset.getAdGroup(/* adGroupIndex= */ 0).states) + .asList() + .containsExactly(AD_STATE_UNAVAILABLE); + assertThat( + adPlaybackStateAfterAssetReset.getAdGroup(/* adGroupIndex= */ 0).contentResumeOffsetUs) + .isEqualTo(0L); // The offset collapsed because the interstitial duration is unset. + + // Wait for the async asset list resolution to complete... + runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); + // and verify the updated ad playback state that now should have a resume offset calculated with + // the durations declared by the asset list that was fetched again. + verify(mockEventListener, times(4)).onAdPlaybackState(adPlaybackStateCaptor.capture()); + AdPlaybackState adPlaybackStateAfterAssetResolution = adPlaybackStateCaptor.getValue(); + assertThat(adPlaybackStateAfterAssetResolution.getAdGroup(/* adGroupIndex= */ 0).count) + .isEqualTo(2); + assertThat(adPlaybackStateAfterAssetResolution.getAdGroup(/* adGroupIndex= */ 0).states) + .asList() + .containsExactly(AD_STATE_AVAILABLE, AD_STATE_AVAILABLE); + assertThat( + adPlaybackStateAfterAssetResolution.getAdGroup(/* adGroupIndex= */ 0) + .contentResumeOffsetUs) + .isEqualTo(12_074_666L); // The offset snapped to the end of the segment again. + } + + @Test + public void + handleContentTimelineChanged_preRollWithAssetList_resolvesAndSchedulesNextPlayerMessage() + throws IOException, TimeoutException { + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:9\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:00.000Z\n" + + "#EXTINF:9,\n" + + "main0.ts\n" + + "#EXTINF:81,\n" + + "main0.ts\n" + + "#EXT-X-ENDLIST" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T22:00:00.000Z\"," + + "CUE=\"PRE\"," + + "X-ASSET-LIST=\"http://example.com/assetlist-0-0.json\"" + "\n" + "#EXT-X-DATERANGE:" + "ID=\"ad1-0\"," + "CLASS=\"com.apple.hls.interstitial\"," - + "START-DATE=\"2020-01-02T21:00:15.000Z\"," + + "START-DATE=\"2020-01-02T21:01:21.000Z\"," + "X-ASSET-LIST=\"http://example.com/assetlist-1-0.json\"" + "\n"; - when(mockPlayer.getContentPosition()).thenReturn(52_000L); + when(mockPlayer.getContentPosition()).thenReturn(0L); + PlayerMessage midRollPlayerMessage = + new PlayerMessage( + mock(PlayerMessage.Sender.class), + mock(PlayerMessage.Target.class), + Timeline.EMPTY, + /* defaultMediaItemIndex= */ 0, + /* Clock ignored */ null, + /* Looper ignored */ null); + when(mockPlayer.createMessage(any())).thenReturn(midRollPlayerMessage); callHandleContentTimelineChangedAndCaptureAdPlaybackState( playlistString, adsLoader, /* windowIndex= */ 0, - /* windowPositionInPeriodUs= */ 0L, + /* windowPositionInPeriodUs= */ 0, /* windowEndPositionInPeriodUs= */ C.TIME_END_OF_SOURCE); runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); verify(mockAdsLoaderListener) - .onAssetListLoadStarted(eq(contentMediaItem), eq("adsId"), eq(1), eq(0)); - verify(mockAdsLoaderListener) - .onAssetListLoadCompleted(eq(contentMediaItem), eq("adsId"), eq(1), eq(0), any(), any()); + .onAssetListLoadCompleted(eq(contentMediaItem), eq("adsId"), eq(0), eq(0), any(), any()); + assertThat(midRollPlayerMessage.getPositionMs()).isEqualTo(54_000L); + assertThat(midRollPlayerMessage.getPayload()).isEqualTo(contentMediaItem); + assertThat(midRollPlayerMessage.getLooper()).isEqualTo(Looper.myLooper()); + InOrder inOrder = inOrder(mockPlayer); + inOrder.verify(mockPlayer).addListener(any()); + verifyTimelineUpdate( + inOrder, + mockPlayer, + /* verifyMessageScheduled= */ false, + /* currentTimelineRequestCount= */ 1); + verifyAssetListLoadCompleted(inOrder, mockPlayer, /* verifyMessageScheduled= */ true); + inOrder.verifyNoMoreInteractions(); + } + + @Test + public void + handleContentTimelineChanged_midRollCloserToPreviousAdThan3TimesTargetDuration_schedulesNextPlayerMessageAtEndOfPreviousAdGroup() + throws IOException, TimeoutException { + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:9\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:00.000Z\n" + + "#EXTINF:9,\n" + + "main0.ts\n" + + "#EXTINF:81,\n" + + "main0.ts\n" + + "#EXT-X-ENDLIST" + + "\n" + + "#EXT-X-DATERANGE:" + // The pre roll ad has a content resume offset equal to its duration of 10.123s. + // The duration is set by the asset list JSON loaded during the test. + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T22:00:00.000Z\"," + + "CUE=\"PRE\"," + + "X-ASSET-LIST=\"http://example.com/assetlist-0-0.json\"" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad1-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + // trigger at 00:02.000 (29 - (3 * 9)) is within content resume offset + + "START-DATE=\"2020-01-02T21:00:29.000Z\"," + + "X-ASSET-LIST=\"http://example.com/assetlist-1-0.json\"" + + "\n"; + when(mockPlayer.getContentPosition()).thenReturn(0L); + PlayerMessage midRollPlayerMessage = + new PlayerMessage( + mock(PlayerMessage.Sender.class), + mock(PlayerMessage.Target.class), + Timeline.EMPTY, + /* defaultMediaItemIndex= */ 0, + /* Clock ignored */ null, + /* Looper ignored */ null); + when(mockPlayer.createMessage(any())).thenReturn(midRollPlayerMessage); + + callHandleContentTimelineChangedAndCaptureAdPlaybackState( + playlistString, + adsLoader, + /* windowIndex= */ 0, + /* windowPositionInPeriodUs= */ 0, + /* windowEndPositionInPeriodUs= */ C.TIME_END_OF_SOURCE); + + runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); + // Assert the message is scheduled at the position right after the pre roll's resume offset. + assertThat(midRollPlayerMessage.getPositionMs()).isEqualTo(10_123L); + } + + @Test + public void + handleContentTimelineChanged_liveMidRollCloserToPreviousAdThan3TimesTargetDuration_schedulesNextPlayerMessageAtEndOfPreviousAdGroup() + throws IOException, TimeoutException { + when(mockPlayer.getContentPosition()).thenReturn(0L); + PlayerMessage midRollPlayerMessage = + new PlayerMessage( + mock(PlayerMessage.Sender.class), + mock(PlayerMessage.Target.class), + Timeline.EMPTY, + /* defaultMediaItemIndex= */ 0, + /* Clock ignored */ null, + /* Looper ignored */ null); + when(mockPlayer.createMessage(any())).thenReturn(midRollPlayerMessage); + when(mockPlayer.getCurrentMediaItem()).thenReturn(contentMediaItem); + + callHandleContentTimelineChangedForLiveAndCaptureAdPlaybackStates( + adsLoader, + /* startAdsLoader= */ true, + /* windowOffsetInFirstPeriodUs= */ 0L, + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:9\n" + // window.positionInFirstPeriod = 0 + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:00.000Z\n" + + "#EXTINF:9,\n" + + "main0.ts\n" + + "#EXTINF:9,\n" + + "main0.ts\n" + + "#EXTINF:9,\n" + + "main0.ts\n" + + "\n", + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:9\n" + // window.positionInFirstPeriod = 9_000_000 + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:09.000Z\n" + + "#EXTINF:9,\n" + + "main0.ts\n" + + "#EXTINF:9,\n" + + "main0.ts" + + "\n" + // adGroup.timeUs = 18_000_900 (window positionMs: 9_000; resumes 29_123) + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T21:00:18.000Z\"," + + "DURATION=20.123," + + "X-ASSET-URI=\"http://example.com/media-0-0.ts\"" + + "\n", + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:9\n" + // window.positionInFirstPeriod = 18_000_000 + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:18.000Z\n" + + "#EXTINF:9,\n" + + "main0.ts" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + // adGroup.timeUs = 18_000_000 (window positionMs: 0; resumes 20_123) + + "START-DATE=\"2020-01-02T21:00:18.000Z\"," + + "DURATION=20.123," + + "X-ASSET-URI=\"http://example.com/media-0-0.ts\"" + + "\n" + + "#EXTINF:81,\n" + + "main0.ts" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad1-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + // adGroup.timeUs = 58_000_000 (window positionMs: 40_000) + // Message trigger at window positionMs: 40_000 - 27_000 = 13_000 which is in the resume + // offset of the previous ad. + + "START-DATE=\"2020-01-02T21:00:58.000Z\"," + + "X-ASSET-LIST=\"http://example.com/assetlist-1-0.json\"" + + "\n"); + + runMainLooperUntil( + () -> midRollPlayerMessage.getPositionMs() != C.TIME_UNSET, TIMEOUT_MS, Clock.DEFAULT); + // Assert that the message is scheduled at the position right after the mid roll's resume + // offset. At the moment of scheduling, the most recent timeline/playlist has not yet been + // published. Hence the message must be scheduled at the window position of the last published + // timeline, which was created with the previous playlist. + assertThat(midRollPlayerMessage.getPositionMs()).isEqualTo(29_123L); + } + + @Test + public void + handleContentTimelineChanged_livePlayingOnLastSegmentWhenAdInserted_triggersAssetLoading() + throws Exception { + AssetListLoadingListener assetListLoadingListener = new AssetListLoadingListener(); + adsLoader.addListener(assetListLoadingListener); + when(mockPlayer.getCurrentMediaItem()).thenReturn(contentMediaItem); + when(mockPlayer.getContentPosition()).thenReturn(27_000L); // content pos in first window + + callHandleContentTimelineChangedForLiveAndCaptureAdPlaybackStates( + adsLoader, + /* startAdsLoader= */ true, + /* windowOffsetInFirstPeriodUs= */ 0L, + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:9\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:00.000Z\n" + + "#EXTINF:9,\n" + + "main0.ts\n" + + "#EXTINF:9,\n" + + "main0.ts\n" + + "#EXTINF:9,\n" // ends at pos 26_999 + + "main0.ts" + + "\n", + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:9\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:09.000Z\n" + + "#EXTINF:9,\n" + + "main0.ts\n" + + "#EXTINF:9,\n" + + "main0.ts" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad1-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T21:00:27.000Z\"," + + "DURATION=20.123," + + "X-ASSET-LIST=\"http://example.com/media-1-0.ts\"" + + "\n" + + "#EXTINF:9,\n" // Starts at pos 18 in second window; at pos 27 in first window. + + "main0.ts" + + "\n"); + + runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); + ArgumentCaptor captor = ArgumentCaptor.forClass(AdPlaybackState.class); + verify(mockEventListener, times(3)).onAdPlaybackState(captor.capture()); + assertThat(captor.getAllValues().get(2).getAdGroup(0).mediaItems[0]) + .isEqualTo( + new MediaItem.Builder() + .setUri("http://0") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()); + } + + @Test + public void + handleContentTimelineChanged_startPositionAfterMidRollTimeUs_resolvesAndSchedulesMidRoll() + throws IOException, TimeoutException { + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:30\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:00.000Z\n" + + "#EXTINF:30,\n" + + "main0.ts\n" + + "#EXTINF:30,\n" + + "main1.ts\n" + + "#EXTINF:30,\n" + + "main2.ts\n" + + "#EXTINF:30,\n" + + "main3.ts\n" + + "#EXT-X-ENDLIST" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T21:00:00.000Z\"," + + "CUE=\"PRE\"," + + "X-ASSET-LIST=\"http://example.com/assetlist-0-0.json\"" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad1-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T21:00:15.000Z\"," + + "X-ASSET-LIST=\"http://example.com/assetlist-1-0.json\"" + + "\n"; + when(mockPlayer.getContentPosition()).thenReturn(52_000L); + + callHandleContentTimelineChangedAndCaptureAdPlaybackState( + playlistString, + adsLoader, + /* windowIndex= */ 0, + /* windowPositionInPeriodUs= */ 0L, + /* windowEndPositionInPeriodUs= */ C.TIME_END_OF_SOURCE); + + runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); + verify(mockAdsLoaderListener) + .onAssetListLoadStarted(eq(contentMediaItem), eq("adsId"), eq(1), eq(0)); + verify(mockAdsLoaderListener) + .onAssetListLoadCompleted(eq(contentMediaItem), eq("adsId"), eq(1), eq(0), any(), any()); + } + + @Test + public void handleContentTimelineChanged_assetListWithMultipleAssets_resolvesAndExpandsAdGroup() + throws IOException, TimeoutException { + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:9\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:00.000Z\n" + + "#EXTINF:9,\n" + + "main0.ts\n" + + "#EXT-X-ENDLIST" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T22:00:00.000Z\"," + + "CUE=\"PRE\"," + + "X-ASSET-LIST=\"http://three-assets\"" + + "\n"; + + when(mockPlayer.getContentPosition()).thenReturn(0L); + AdPlaybackState expectedAdPlaybackStateAtTimelineChange = + new AdPlaybackState("adsId", 0L) + .withAdCount(/* adGroupIndex= */ 0, 1) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, "ad0-0"); + + assertThat( + callHandleContentTimelineChangedAndCaptureAdPlaybackState( + playlistString, + adsLoader, + /* windowIndex= */ 0, + /* windowPositionInPeriodUs= */ 0, + /* windowEndPositionInPeriodUs= */ C.TIME_END_OF_SOURCE)) + .isEqualTo(expectedAdPlaybackStateAtTimelineChange); + + runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); + ArgumentCaptor adPlaybackStateCaptor = + ArgumentCaptor.forClass(AdPlaybackState.class); + verify(mockEventListener, times(2)).onAdPlaybackState(adPlaybackStateCaptor.capture()); + assertThat(adPlaybackStateCaptor.getAllValues()) + .containsExactly( + expectedAdPlaybackStateAtTimelineChange, + expectedAdPlaybackStateAtTimelineChange + .withAdCount(/* adGroupIndex= */ 0, 3) + .withAdDurationsUs(/* adGroupIndex= */ 0, 10_123_000L, 11_123_000L, 12_123_000L) + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + new MediaItem.Builder() + .setUri("http://0") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 1, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 1, + new MediaItem.Builder() + .setUri("http://1") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 2, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 2, + new MediaItem.Builder() + .setUri("http://2") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withContentResumeOffsetUs( + /* adGroupIndex= */ 0, 10_123_000L + 11_123_000L + 12_123_000L)) + .inOrder(); + InOrder inOrder = inOrder(mockPlayer); + inOrder.verify(mockPlayer).addListener(any()); + verifyTimelineUpdate( + inOrder, + mockPlayer, + /* verifyMessageScheduled= */ false, + /* currentTimelineRequestCount= */ 1); + verifyAssetListLoadCompleted(inOrder, mockPlayer, /* verifyMessageScheduled= */ false); + inOrder.verifyNoMoreInteractions(); + } + + @Test + public void + handleContentTimelineChanged_assetListWithPlayoutLimitClippingLastAd_capsAdDurationAndResumeOffset() + throws IOException, TimeoutException { + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:9\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:00.000Z\n" + + "#EXTINF:9,\n" + + "main0.ts\n" + + "#EXT-X-ENDLIST" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T22:00:00.000Z\"," + + "CUE=\"PRE\"," + + "X-PLAYOUT-LIMIT=25.0," + + "X-ASSET-LIST=\"http://three-assets\"" + + "\n"; + when(mockPlayer.getContentPosition()).thenReturn(0L); + AdPlaybackState expectedAdPlaybackStateAtTimelineChange = + new AdPlaybackState("adsId", 0L) + .withAdCount(/* adGroupIndex= */ 0, 1) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, "ad0-0") + .withAdDurationsUs(/* adGroupIndex= */ 0, 25_000_000L); + + callHandleContentTimelineChangedAndCaptureAdPlaybackState( + playlistString, + adsLoader, + /* windowIndex= */ 0, + /* windowPositionInPeriodUs= */ 0, + /* windowEndPositionInPeriodUs= */ C.TIME_END_OF_SOURCE); + runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); + + ArgumentCaptor adPlaybackStateCaptor = + ArgumentCaptor.forClass(AdPlaybackState.class); + verify(mockEventListener, times(2)).onAdPlaybackState(adPlaybackStateCaptor.capture()); + assertThat(adPlaybackStateCaptor.getAllValues()) + .containsExactly( + expectedAdPlaybackStateAtTimelineChange, + expectedAdPlaybackStateAtTimelineChange + .withAdCount(/* adGroupIndex= */ 0, 3) + .withAdDurationsUs(/* adGroupIndex= */ 0, 10_123_000L, 11_123_000L, 3_754_000L) + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + new MediaItem.Builder() + .setUri("http://0") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 1, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 1, + new MediaItem.Builder() + .setUri("http://1") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 2, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 2, + new MediaItem.Builder() + .setUri("http://2") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 25_000_000L)) + .inOrder(); + } + + @Test + public void + handleContentTimelineChanged_assetListWithPlayoutLimitExceedingTotalDuration_usesAssetDurations() + throws IOException, TimeoutException { + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:9\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:00.000Z\n" + + "#EXTINF:9,\n" + + "main0.ts\n" + + "#EXT-X-ENDLIST" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T22:00:00.000Z\"," + + "CUE=\"PRE\"," + + "X-PLAYOUT-LIMIT=50.0," + + "X-ASSET-LIST=\"http://three-assets\"" + + "\n"; + when(mockPlayer.getContentPosition()).thenReturn(0L); + AdPlaybackState expectedAdPlaybackStateAtTimelineChange = + new AdPlaybackState("adsId", 0L) + .withAdCount(/* adGroupIndex= */ 0, 1) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, "ad0-0") + .withAdDurationsUs(/* adGroupIndex= */ 0, 50_000_000L); + + callHandleContentTimelineChangedAndCaptureAdPlaybackState( + playlistString, + adsLoader, + /* windowIndex= */ 0, + /* windowPositionInPeriodUs= */ 0, + /* windowEndPositionInPeriodUs= */ C.TIME_END_OF_SOURCE); + runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); + + ArgumentCaptor adPlaybackStateCaptor = + ArgumentCaptor.forClass(AdPlaybackState.class); + verify(mockEventListener, times(2)).onAdPlaybackState(adPlaybackStateCaptor.capture()); + assertThat(adPlaybackStateCaptor.getAllValues()) + .containsExactly( + expectedAdPlaybackStateAtTimelineChange, + expectedAdPlaybackStateAtTimelineChange + .withAdCount(/* adGroupIndex= */ 0, 3) + .withAdDurationsUs(/* adGroupIndex= */ 0, 10_123_000L, 11_123_000L, 12_123_000L) + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + new MediaItem.Builder() + .setUri("http://0") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 1, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 1, + new MediaItem.Builder() + .setUri("http://1") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 2, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 2, + new MediaItem.Builder() + .setUri("http://2") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withContentResumeOffsetUs( + /* adGroupIndex= */ 0, 10_123_000L + 11_123_000L + 12_123_000L)) + .inOrder(); + } + + @Test + public void + handleContentTimelineChanged_assetListWithPlayoutLimitAndResumeOffset_usesExplicitResumeOffsetAndCapsAdDurations() + throws IOException, TimeoutException { + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:9\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:00.000Z\n" + + "#EXTINF:9,\n" + + "main0.ts\n" + + "#EXT-X-ENDLIST" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T22:00:00.000Z\"," + + "CUE=\"PRE\"," + + "X-PLAYOUT-LIMIT=15.0," + + "X-RESUME-OFFSET=5.0," + + "X-ASSET-LIST=\"http://three-assets\"" + + "\n"; + when(mockPlayer.getContentPosition()).thenReturn(0L); + AdPlaybackState expectedAdPlaybackStateAtTimelineChange = + new AdPlaybackState("adsId", 0L) + .withAdCount(/* adGroupIndex= */ 0, 1) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, "ad0-0") + .withAdDurationsUs(/* adGroupIndex= */ 0, 15_000_000L) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 5_000_000L); + + callHandleContentTimelineChangedAndCaptureAdPlaybackState( + playlistString, + adsLoader, + /* windowIndex= */ 0, + /* windowPositionInPeriodUs= */ 0, + /* windowEndPositionInPeriodUs= */ C.TIME_END_OF_SOURCE); + runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); + + ArgumentCaptor adPlaybackStateCaptor = + ArgumentCaptor.forClass(AdPlaybackState.class); + verify(mockEventListener, times(2)).onAdPlaybackState(adPlaybackStateCaptor.capture()); + assertThat(adPlaybackStateCaptor.getAllValues()) + .containsExactly( + expectedAdPlaybackStateAtTimelineChange, + expectedAdPlaybackStateAtTimelineChange + .withAdCount(/* adGroupIndex= */ 0, 3) + .withAdDurationsUs(/* adGroupIndex= */ 0, 10_123_000L, 4_877_000L, 0L) + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + new MediaItem.Builder() + .setUri("http://0") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 1, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 1, + new MediaItem.Builder() + .setUri("http://1") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 2, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 2, + new MediaItem.Builder() + .setUri("http://2") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withSkippedAd(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 2) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 5_000_000L)) + .inOrder(); + } + + @Test + public void + handleContentTimelineChanged_assetListWithPlayoutLimitEqualToFirstAssetDuration_skipsRemainingAds() + throws IOException, TimeoutException { + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:9\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:00.000Z\n" + + "#EXTINF:9,\n" + + "main0.ts\n" + + "#EXT-X-ENDLIST" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T22:00:00.000Z\"," + + "CUE=\"PRE\"," + + "X-PLAYOUT-LIMIT=10.123," + + "X-ASSET-LIST=\"http://three-assets\"" + + "\n"; + when(mockPlayer.getContentPosition()).thenReturn(0L); + AdPlaybackState expectedAdPlaybackStateAtTimelineChange = + new AdPlaybackState("adsId", 0L) + .withAdCount(/* adGroupIndex= */ 0, 1) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, "ad0-0") + .withAdDurationsUs(/* adGroupIndex= */ 0, 10_123_000L); + + callHandleContentTimelineChangedAndCaptureAdPlaybackState( + playlistString, + adsLoader, + /* windowIndex= */ 0, + /* windowPositionInPeriodUs= */ 0, + /* windowEndPositionInPeriodUs= */ C.TIME_END_OF_SOURCE); + runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); + + ArgumentCaptor adPlaybackStateCaptor = + ArgumentCaptor.forClass(AdPlaybackState.class); + verify(mockEventListener, times(2)).onAdPlaybackState(adPlaybackStateCaptor.capture()); + assertThat(adPlaybackStateCaptor.getAllValues()) + .containsExactly( + expectedAdPlaybackStateAtTimelineChange, + expectedAdPlaybackStateAtTimelineChange + .withAdCount(/* adGroupIndex= */ 0, 3) + .withAdDurationsUs(/* adGroupIndex= */ 0, 10_123_000L, 0L, 0L) + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + new MediaItem.Builder() + .setUri("http://0") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 1, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 1, + new MediaItem.Builder() + .setUri("http://1") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withSkippedAd(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 1) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 2, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 2, + new MediaItem.Builder() + .setUri("http://2") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withSkippedAd(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 2) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 10_123_000L)) + .inOrder(); + } + + @Test + public void + handleContentTimelineChanged_assetListWithPlayoutLimitEqualToTotalAssetDuration_doesNotSkipAds() + throws IOException, TimeoutException { + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:9\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:00.000Z\n" + + "#EXTINF:9,\n" + + "main0.ts\n" + + "#EXT-X-ENDLIST" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T22:00:00.000Z\"," + + "CUE=\"PRE\"," + + "X-PLAYOUT-LIMIT=33.369," + + "X-ASSET-LIST=\"http://three-assets\"" + + "\n"; + when(mockPlayer.getContentPosition()).thenReturn(0L); + AdPlaybackState expectedAdPlaybackStateAtTimelineChange = + new AdPlaybackState("adsId", 0L) + .withAdCount(/* adGroupIndex= */ 0, 1) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, "ad0-0") + .withAdDurationsUs(/* adGroupIndex= */ 0, 33_369_000L); + + callHandleContentTimelineChangedAndCaptureAdPlaybackState( + playlistString, + adsLoader, + /* windowIndex= */ 0, + /* windowPositionInPeriodUs= */ 0, + /* windowEndPositionInPeriodUs= */ C.TIME_END_OF_SOURCE); + runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); + + ArgumentCaptor adPlaybackStateCaptor = + ArgumentCaptor.forClass(AdPlaybackState.class); + verify(mockEventListener, times(2)).onAdPlaybackState(adPlaybackStateCaptor.capture()); + assertThat(adPlaybackStateCaptor.getAllValues()) + .containsExactly( + expectedAdPlaybackStateAtTimelineChange, + expectedAdPlaybackStateAtTimelineChange + .withAdCount(/* adGroupIndex= */ 0, 3) + .withAdDurationsUs(/* adGroupIndex= */ 0, 10_123_000L, 11_123_000L, 12_123_000L) + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + new MediaItem.Builder() + .setUri("http://0") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 1, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 1, + new MediaItem.Builder() + .setUri("http://1") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 2, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 2, + new MediaItem.Builder() + .setUri("http://2") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 33_369_000L)) + .inOrder(); } @Test - public void handleContentTimelineChanged_assetListWithMultipleAssets_resolvesAndExpandsAdGroup() - throws IOException, TimeoutException { + public void + handleContentTimelineChanged_assetListWithZeroPlayoutLimit_skipsAllAdsWithZeroDurationAndResumeOffset() + throws IOException, TimeoutException { String playlistString = "#EXTM3U\n" + "#EXT-X-TARGETDURATION:9\n" @@ -2850,25 +4139,24 @@ public void handleContentTimelineChanged_assetListWithMultipleAssets_resolvesAnd + "CLASS=\"com.apple.hls.interstitial\"," + "START-DATE=\"2020-01-02T22:00:00.000Z\"," + "CUE=\"PRE\"," + + "X-PLAYOUT-LIMIT=0.0," + "X-ASSET-LIST=\"http://three-assets\"" + "\n"; - when(mockPlayer.getContentPosition()).thenReturn(0L); AdPlaybackState expectedAdPlaybackStateAtTimelineChange = new AdPlaybackState("adsId", 0L) .withAdCount(/* adGroupIndex= */ 0, 1) - .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, "ad0-0"); - - assertThat( - callHandleContentTimelineChangedAndCaptureAdPlaybackState( - playlistString, - adsLoader, - /* windowIndex= */ 0, - /* windowPositionInPeriodUs= */ 0, - /* windowEndPositionInPeriodUs= */ C.TIME_END_OF_SOURCE)) - .isEqualTo(expectedAdPlaybackStateAtTimelineChange); + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, "ad0-0") + .withAdDurationsUs(/* adGroupIndex= */ 0, 0L); + callHandleContentTimelineChangedAndCaptureAdPlaybackState( + playlistString, + adsLoader, + /* windowIndex= */ 0, + /* windowPositionInPeriodUs= */ 0, + /* windowEndPositionInPeriodUs= */ C.TIME_END_OF_SOURCE); runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); + ArgumentCaptor adPlaybackStateCaptor = ArgumentCaptor.forClass(AdPlaybackState.class); verify(mockEventListener, times(2)).onAdPlaybackState(adPlaybackStateCaptor.capture()); @@ -2877,7 +4165,7 @@ public void handleContentTimelineChanged_assetListWithMultipleAssets_resolvesAnd expectedAdPlaybackStateAtTimelineChange, expectedAdPlaybackStateAtTimelineChange .withAdCount(/* adGroupIndex= */ 0, 3) - .withAdDurationsUs(/* adGroupIndex= */ 0, 10_123_000L, 11_123_000L, 12_123_000L) + .withAdDurationsUs(/* adGroupIndex= */ 0, 0L, 0L, 0L) .withAvailableAdMediaItem( /* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, @@ -2885,6 +4173,7 @@ public void handleContentTimelineChanged_assetListWithMultipleAssets_resolvesAnd .setUri("http://0") .setMimeType(MimeTypes.APPLICATION_M3U8) .build()) + .withSkippedAd(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0) .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 1, "ad0-0") .withAvailableAdMediaItem( /* adGroupIndex= */ 0, @@ -2893,6 +4182,7 @@ public void handleContentTimelineChanged_assetListWithMultipleAssets_resolvesAnd .setUri("http://1") .setMimeType(MimeTypes.APPLICATION_M3U8) .build()) + .withSkippedAd(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 1) .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 2, "ad0-0") .withAvailableAdMediaItem( /* adGroupIndex= */ 0, @@ -2901,18 +4191,9 @@ public void handleContentTimelineChanged_assetListWithMultipleAssets_resolvesAnd .setUri("http://2") .setMimeType(MimeTypes.APPLICATION_M3U8) .build()) - .withContentResumeOffsetUs( - /* adGroupIndex= */ 0, 10_123_000L + 11_123_000L + 12_123_000L)) + .withSkippedAd(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 2) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 0L)) .inOrder(); - InOrder inOrder = inOrder(mockPlayer); - inOrder.verify(mockPlayer).addListener(any()); - verifyTimelineUpdate( - inOrder, - mockPlayer, - /* verifyMessageScheduled= */ false, - /* currentTimelineRequestCount= */ 1); - verifyAssetListLoadCompleted(inOrder, mockPlayer, /* verifyMessageScheduled= */ false); - inOrder.verifyNoMoreInteractions(); } @Test @@ -3207,6 +4488,7 @@ public void handleContentTimelineChanged_emptyAssetList_marksAdAsFailedAndSchedu + "#EXT-X-DATERANGE:" + "ID=\"ad1-0\"," + "CLASS=\"com.apple.hls.interstitial\"," + + "DURATION=3.246," + "X-PLAYOUT-LIMIT=3.246," + "START-DATE=\"2020-01-02T22:00:00.000Z\"," + "CUE=\"PRE\"," @@ -3252,6 +4534,8 @@ public void handleContentTimelineChanged_emptyAssetList_marksAdAsFailedAndSchedu .containsExactly( expectedAdPlaybackStateAtTimelineChange, expectedAdPlaybackStateAtTimelineChange + // TODO: b/570090695 - Expanding ad0-0 at index 0 should shift ad1-0 rather than + // placing ad0-0's remaining assets at indices 2 and 3 after ad1-0 at index 1. .withAdCount(/* adGroupIndex= */ 0, 4) .withAdDurationsUs( /* adGroupIndex= */ 0, 10_123_000L, 3_246_000L, 11_123_000L, 12_123_000L) @@ -3278,8 +4562,192 @@ public void handleContentTimelineChanged_emptyAssetList_marksAdAsFailedAndSchedu .setUri("http://2") .setMimeType(MimeTypes.APPLICATION_M3U8) .build()) - .withContentResumeOffsetUs( - /* adGroupIndex= */ 0, 3_246_000L + 10_123_000L + 11_123_000L + 12_123_000L)) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 3_246_000L + 33_369_000L)) + .inOrder(); + } + + @Test + public void + handleContentTimelineChanged_assetListWithPlannedDurationAndAssetUriInSameAdGroup_replacesOnlyAssetListResumeOffsetShare() + throws IOException, TimeoutException { + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:9\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:00.000Z\n" + + "#EXTINF:9,\n" + + "main0.0.ts\n" + + "#EXT-X-ENDLIST" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T22:00:00.000Z\"," + + "CUE=\"PRE\"," + + "PLANNED-DURATION=20.0," + + "X-ASSET-LIST=\"http://three-assets\"" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad1-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "DURATION=3.246," + + "START-DATE=\"2020-01-02T22:00:00.000Z\"," + + "CUE=\"PRE\"," + + "X-ASSET-URI=\"http://example.com/media-1-0.m3u8\"" + + "\n"; + when(mockPlayer.getContentPosition()).thenReturn(0L); + AdPlaybackState expectedAdPlaybackStateAtTimelineChange = + new AdPlaybackState("adsId", 0L) + .withAdCount(/* adGroupIndex= */ 0, 2) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, "ad0-0") + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 1, "ad1-0") + .withAdDurationsUs(/* adGroupIndex= */ 0, C.TIME_UNSET, C.TIME_UNSET) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 20_000_000L + 3_246_000L) + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 1, + new MediaItem.Builder() + .setUri("http://example.com/media-1-0.m3u8") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()); + + callHandleContentTimelineChangedAndCaptureAdPlaybackState( + playlistString, + adsLoader, + /* windowIndex= */ 0, + /* windowPositionInPeriodUs= */ 0, + /* windowEndPositionInPeriodUs= */ C.TIME_END_OF_SOURCE); + runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); + + ArgumentCaptor adPlaybackStateCaptor = + ArgumentCaptor.forClass(AdPlaybackState.class); + verify(mockEventListener, times(2)).onAdPlaybackState(adPlaybackStateCaptor.capture()); + assertThat(adPlaybackStateCaptor.getAllValues()) + .containsExactly( + expectedAdPlaybackStateAtTimelineChange, + expectedAdPlaybackStateAtTimelineChange + .withAdCount(/* adGroupIndex= */ 0, 4) + .withAdDurationsUs( + /* adGroupIndex= */ 0, 10_123_000L, C.TIME_UNSET, 11_123_000L, 12_123_000L) + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + new MediaItem.Builder() + .setUri("http://0") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 2, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 2, + new MediaItem.Builder() + .setUri("http://1") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 3, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 3, + new MediaItem.Builder() + .setUri("http://2") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 3_246_000L + 33_369_000L)) + .inOrder(); + } + + @Test + public void + handleContentTimelineChanged_assetListWithSnapInAndAssetUriInSameAdGroup_replacesSnappedAssetListResumeOffsetShare() + throws IOException, TimeoutException { + String playlistString = + "#EXTM3U\n" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-PROGRAM-DATE-TIME:2020-01-02T21:00:00.000Z\n" + + "#EXTINF:6,\nmain0.ts\n" + + "#EXTINF:6,\nmain1.ts\n" + + "#EXTINF:6,\nmain2.ts\n" + + "#EXTINF:6,\nmain3.ts\n" + + "#EXTINF:6,\nmain4.ts\n" + + "#EXTINF:6,\nmain5.ts\n" + + "#EXTINF:6,\nmain6.ts\n" + + "#EXTINF:6,\nmain7.ts\n" + + "#EXT-X-ENDLIST" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad0-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "START-DATE=\"2020-01-02T21:00:00.000Z\"," + + "CUE=\"PRE\"," + + "PLANNED-DURATION=10.0," + + "X-SNAP=\"IN\"," + + "X-ASSET-LIST=\"http://three-assets\"" + + "\n" + + "#EXT-X-DATERANGE:" + + "ID=\"ad1-0\"," + + "CLASS=\"com.apple.hls.interstitial\"," + + "DURATION=3.246," + + "START-DATE=\"2020-01-02T21:00:00.000Z\"," + + "CUE=\"PRE\"," + + "X-ASSET-URI=\"http://example.com/media-1-0.m3u8\"" + + "\n"; + when(mockPlayer.getContentPosition()).thenReturn(0L); + AdPlaybackState expectedAdPlaybackStateAtTimelineChange = + new AdPlaybackState("adsId", 0L) + .withAdCount(/* adGroupIndex= */ 0, 2) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0, "ad0-0") + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 1, "ad1-0") + .withAdDurationsUs(/* adGroupIndex= */ 0, C.TIME_UNSET, C.TIME_UNSET) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 12_000_000L + 3_246_000L) + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 1, + new MediaItem.Builder() + .setUri("http://example.com/media-1-0.m3u8") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()); + + callHandleContentTimelineChangedAndCaptureAdPlaybackState( + playlistString, + adsLoader, + /* windowIndex= */ 0, + /* windowPositionInPeriodUs= */ 0, + /* windowEndPositionInPeriodUs= */ C.TIME_END_OF_SOURCE); + runMainLooperUntil(assetListLoadingListener::completed, TIMEOUT_MS, Clock.DEFAULT); + + ArgumentCaptor adPlaybackStateCaptor = + ArgumentCaptor.forClass(AdPlaybackState.class); + verify(mockEventListener, times(2)).onAdPlaybackState(adPlaybackStateCaptor.capture()); + assertThat(adPlaybackStateCaptor.getAllValues()) + .containsExactly( + expectedAdPlaybackStateAtTimelineChange, + expectedAdPlaybackStateAtTimelineChange + .withAdCount(/* adGroupIndex= */ 0, 4) + .withAdDurationsUs( + /* adGroupIndex= */ 0, 10_123_000L, C.TIME_UNSET, 11_123_000L, 12_123_000L) + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 0, + new MediaItem.Builder() + .setUri("http://0") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 2, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 2, + new MediaItem.Builder() + .setUri("http://1") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withAdId(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 3, "ad0-0") + .withAvailableAdMediaItem( + /* adGroupIndex= */ 0, + /* adIndexInAdGroup= */ 3, + new MediaItem.Builder() + .setUri("http://2") + .setMimeType(MimeTypes.APPLICATION_M3U8) + .build()) + .withContentResumeOffsetUs(/* adGroupIndex= */ 0, 3_246_000L + 36_000_000L)) .inOrder(); } @@ -6486,7 +7954,7 @@ public void getClosestSegmentBoundaryUs_positionBeforeAndAfterPlaylist_getsSanit } @Test - public void removePendingSnapInResolutionUntilIncludingIndex_correctOperations() { + public void addPendingSnapInResolution_withOutOfOrderInserts_maintainsSortedOrder() { Interstitial interstitial = new Interstitial( /* id= */ "ad1", @@ -6511,53 +7979,73 @@ public void removePendingSnapInResolutionUntilIncludingIndex_correctOperations() /* skipControlLabelId= */ null); AdsMediaSourceSessionManager holder = new AdsMediaSourceSessionManager(); holder.startContentSource("adsId", MediaItem.fromUri("http://1"), mockEventListener); - holder.startContentSource("adsId_other", MediaItem.fromUri("http://1"), mockEventListener); - for (int i = 0; i < 10; i++) { + long[] resumeTimesUs = new long[] {25_001L, 20_000L, 10_000L}; + // Add pending snaps out of order. + for (int i = 0; i < resumeTimesUs.length; i++) { holder .getSession("adsId") - .pendingSnapInResolutions - .add( - new PendingSnapInResolution( - /* resumeTimeUs= */ i * 10_000L, - /* adGroupIndex= */ i, - /* interstitial= */ interstitial)); - holder - .getSession("adsId_other") - .pendingSnapInResolutions - .add( - new PendingSnapInResolution( - /* resumeTimeUs= */ i * 10_000L, - /* adGroupIndex= */ i, - /* interstitial= */ interstitial)); + .addPendingSnapInResolution( + /* resumeTimeUs= */ resumeTimesUs[i], + /* adGroupIndex= */ i, + /* interstitial= */ interstitial); } - assertThat(holder.getSession("adsId").pendingSnapInResolutions).hasSize(10); - assertThat(holder.getSession("adsId_other").pendingSnapInResolutions).hasSize(10); + assertThat(holder.getSession("adsId").pendingSnapInResolutions).hasSize(3); + assertThat(holder.getSession("adsId").pendingSnapInResolutions.keySet()) + .containsExactly(10_000L, 20_000L, 25_001L) + .inOrder(); - holder - .getSession("adsId") - .removePendingSnapInResolutionUntilIndexInclusive(/* resolvedIndex= */ 4); - holder - .getSession("adsId_other") - .removePendingSnapInResolutionUntilIndexInclusive(/* resolvedIndex= */ 4); + // Get all snaps before and including a given position (like the end of the playlist). + NavigableMap headMap = + holder.getSession("adsId").pendingSnapInResolutions.headMap(25_000L, /* inclusive= */ true); - assertThat(holder.getSession("adsId").pendingSnapInResolutions).hasSize(5); - assertThat(holder.getSession("adsId_other").pendingSnapInResolutions).hasSize(5); + Iterator> iterator = headMap.entrySet().iterator(); + while (iterator.hasNext()) { + iterator.next(); + iterator.remove(); + } + assertThat(holder.getSession("adsId").pendingSnapInResolutions).hasSize(1); + assertThat(holder.getSession("adsId").pendingSnapInResolutions.keySet()) + .containsExactly(25_001L); + } + @Test + public void addPendingSnapInResolution_forSameAdGroupAndInterstitial_replacesExistingEntry() { + Interstitial interstitial = + new Interstitial( + /* id= */ "ad1", + /* assetUri= */ Uri.parse("http://example.com/ad1.m3u8"), + /* assetListUri= */ null, + /* startDateUnixUs= */ 0L, + /* endDateUnixUs= */ C.TIME_UNSET, + /* durationUs= */ C.TIME_UNSET, + /* plannedDurationUs= */ C.TIME_UNSET, + /* cue= */ ImmutableList.of(), + /* endOnNext= */ false, + /* resumeOffsetUs= */ 0, + /* playoutLimitUs= */ C.TIME_UNSET, + ImmutableList.of(SNAP_TYPE_IN, SNAP_TYPE_OUT), + ImmutableList.of(), + /* clientDefinedAttributes= */ ImmutableList.of(), + /* contentMayVary= */ false, + /* timelineOccupies= */ Interstitial.TIMELINE_OCCUPIES_POINT, + /* timelineStyle= */ Interstitial.TIMELINE_STYLE_PRIMARY, + /* skipControlOffsetUs= */ C.TIME_UNSET, + /* skipControlDurationUs= */ C.TIME_UNSET, + /* skipControlLabelId= */ null); + AdsMediaSourceSessionManager holder = new AdsMediaSourceSessionManager(); + holder.startContentSource("adsId", MediaItem.fromUri("http://1"), mockEventListener); holder .getSession("adsId") - .removePendingSnapInResolutionUntilIndexInclusive(/* resolvedIndex= */ 4); + .addPendingSnapInResolution( + /* resumeTimeUs= */ 10_000L, /* adGroupIndex= */ 0, interstitial); + holder - .getSession("adsId_other") - .removePendingSnapInResolutionUntilIndexInclusive(/* resolvedIndex= */ 1234); + .getSession("adsId") + .addPendingSnapInResolution( + /* resumeTimeUs= */ 20_000L, /* adGroupIndex= */ 0, interstitial); - assertThat(holder.getSession("adsId").pendingSnapInResolutions).isEmpty(); - assertThat(holder.getSession("adsId_other").pendingSnapInResolutions).isEmpty(); - assertThrows( - IllegalArgumentException.class, - () -> - holder - .getSession("adsId") - .removePendingSnapInResolutionUntilIndexInclusive(/* resolvedIndex= */ -1)); + assertThat(holder.getSession("adsId").pendingSnapInResolutions.keySet()) + .containsExactly(20_000L); } @Test @@ -8068,6 +9556,20 @@ public byte[] resolve(Uri uri) { /* skipInfoLabelId= */ "skip_label_from_json"); case "http://slow_loading": return getJsonAssetList(/* assetCount= */ 1, /* delayMs= */ 150); + case "asset-list-two-ads": + return ("{\n" + + " \"ASSETS\": [\n" + + " {\n" + + " \"URI\": \"http://example.com/ad/1\",\n" + + " \"DURATION\": 6.037332\n" + + " },\n" + + " {\n" + + " \"URI\": \"http://example.com/ad/2\",\n" + + " \"DURATION\": 6.037332\n" + + " }\n" + + " ]\n" + + "}\n") + .getBytes(Charset.defaultCharset()); default: return getJsonAssetList(/* assetCount= */ 1, /* delayMs= */ 0); } diff --git a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsMediaChunkTest.java b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsMediaChunkTest.java new file mode 100644 index 00000000000..ffeb8531353 --- /dev/null +++ b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsMediaChunkTest.java @@ -0,0 +1,207 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.exoplayer.hls; + +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +import android.net.Uri; +import androidx.media3.common.C; +import androidx.media3.common.Format; +import androidx.media3.common.ParserException; +import androidx.media3.exoplayer.analytics.PlayerId; +import androidx.media3.exoplayer.drm.DrmSessionEventListener; +import androidx.media3.exoplayer.drm.DrmSessionManager; +import androidx.media3.exoplayer.hls.playlist.HlsMediaPlaylist; +import androidx.media3.exoplayer.source.MediaSourceEventListener; +import androidx.media3.exoplayer.upstream.Allocator; +import androidx.media3.exoplayer.upstream.LoadErrorHandlingPolicy; +import androidx.media3.extractor.ExtractorInput; +import androidx.media3.test.utils.FakeDataSource; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import java.io.EOFException; +import java.io.IOException; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.Mock; +import org.mockito.junit.MockitoJUnit; +import org.mockito.junit.MockitoRule; + +/** Unit tests for {@link HlsMediaChunk}. */ +@RunWith(AndroidJUnit4.class) +public final class HlsMediaChunkTest { + + @Rule public final MockitoRule mockito = MockitoJUnit.rule(); + + @Mock private HlsMediaChunkExtractor mockExtractor; + @Mock private HlsExtractorFactory extractorFactory; + + @Before + public void setUp() throws Exception { + when(mockExtractor.isReusable()).thenReturn(true); + when(extractorFactory.createExtractor(any(), any(), any(), any(), any(), any(), any())) + .thenReturn(mockExtractor); + } + + @Test + public void load_extractorThrowsEOFExceptionAfterFullyLoadedChunk_throwsParserException() + throws Exception { + when(mockExtractor.read(any(ExtractorInput.class))) + .thenAnswer( + invocation -> { + ExtractorInput input = invocation.getArgument(0); + input.skipFully(100); + throw new EOFException(); + }); + HlsMediaPlaylist.Segment segment = + new HlsMediaPlaylist.Segment( + "https://example.com/segment.ts", + /* byteRangeOffset= */ 0, + /* byteRangeLength= */ 100, // Explicit finite length + /* fullSegmentEncryptionKeyUri= */ null, + /* encryptionIV= */ null); + HlsMediaChunk mediaChunk = createChunkWithSegment(extractorFactory, segment); + + ParserException exception = assertThrows(ParserException.class, mediaChunk::load); + assertThat(exception).hasCauseThat().isInstanceOf(EOFException.class); + } + + @Test + public void load_extractorThrowsEOFExceptionBeforeFullyLoadedChunk_propagatesEOFException() + throws Exception { + when(mockExtractor.read(any(ExtractorInput.class))) + .thenAnswer( + invocation -> { + ExtractorInput input = invocation.getArgument(0); + input.skipFully(50); // Premature read (less than segment's 100 length) + throw new EOFException(); + }); + HlsMediaPlaylist.Segment segment = + new HlsMediaPlaylist.Segment( + "https://example.com/segment.ts", + /* byteRangeOffset= */ 0, + /* byteRangeLength= */ 100, + /* fullSegmentEncryptionKeyUri= */ null, + /* encryptionIV= */ null); + HlsMediaChunk mediaChunk = createChunkWithSegment(extractorFactory, segment); + + assertThrows(EOFException.class, mediaChunk::load); + } + + @Test + public void load_extractorThrowsEOFExceptionForUnsetLengthChunk_propagatesEOFException() + throws Exception { + when(mockExtractor.read(any(ExtractorInput.class))) + .thenAnswer( + invocation -> { + throw new EOFException(); + }); + HlsMediaPlaylist.Segment segment = + new HlsMediaPlaylist.Segment( + "https://example.com/segment.ts", + /* byteRangeOffset= */ 0, + /* byteRangeLength= */ C.LENGTH_UNSET, // Unset length + /* fullSegmentEncryptionKeyUri= */ null, + /* encryptionIV= */ null); + HlsMediaChunk mediaChunk = createChunkWithSegment(extractorFactory, segment); + + assertThrows(EOFException.class, mediaChunk::load); + } + + private static HlsMediaChunk createChunkWithSegment( + HlsExtractorFactory extractorFactory, HlsMediaPlaylist.Segment segment) throws IOException { + HlsMediaPlaylist mediaPlaylist = + new HlsMediaPlaylist( + HlsMediaPlaylist.PLAYLIST_TYPE_UNKNOWN, + /* baseUri= */ "http://example.com/", + /* tags= */ ImmutableList.of(), + /* startOffsetUs= */ C.TIME_UNSET, + /* preciseStart= */ false, + /* startTimeUs= */ 0L, + /* hasDiscontinuitySequence= */ false, + /* discontinuitySequence= */ 0, + /* mediaSequence= */ 0L, + /* version= */ 7, + /* targetDurationUs= */ 4_000_000L, + /* partTargetDurationUs= */ C.TIME_UNSET, + /* hasIndependentSegments= */ true, + /* hasEndTag= */ false, + /* hasProgramDateTime= */ false, + /* protectionSchemes= */ null, + /* segments= */ ImmutableList.of(segment), + /* trailingParts= */ ImmutableList.of(), + new HlsMediaPlaylist.ServerControl( + /* skipUntilUs= */ C.TIME_UNSET, + /* canSkipDateRanges= */ false, + /* holdBackUs= */ C.TIME_UNSET, + /* partHoldBackUs= */ C.TIME_UNSET, + /* canBlockReload= */ false), + /* renditionReports= */ ImmutableMap.of(), + /* interstitials= */ ImmutableList.of(), + /* lastSeenInitSegment= */ null); + FakeDataSource fakeDataSource = new FakeDataSource(); + fakeDataSource.getDataSet().newDefaultData().appendReadData(100).endData(); + HlsMediaChunk mediaChunk = + HlsMediaChunk.createInstance( + extractorFactory, + fakeDataSource, + new Format.Builder().build(), + /* startOfPlaylistInPeriodUs= */ 0, + mediaPlaylist, + new HlsChunkSource.SegmentBaseHolder( + segment, /* mediaSequence= */ 1, /* partIndex= */ 0), + Uri.parse("https://playlist.uri/"), + /* steeredPathwayId= */ null, + /* muxedCaptionFormats= */ null, + C.SELECTION_REASON_UNKNOWN, + /* trackSelectionData= */ null, + /* isPrimaryTimestampSource= */ true, + new TimestampAdjusterProvider(), + /* timestampAdjusterInitializationTimeoutMs= */ 0, + /* previousChunk= */ null, + /* mediaSegmentKey= */ null, + /* initSegmentKey= */ null, + /* shouldSpliceIn= */ false, + /* isIndependent= */ true, + PlayerId.UNSET, + /* cmcdDataFactory= */ null); + HlsSampleStreamWrapper sampleStreamWrapper = + new HlsSampleStreamWrapper( + /* uid= */ "", + C.TRACK_TYPE_VIDEO, + mock(HlsSampleStreamWrapper.Callback.class), + mock(HlsChunkSource.class), + /* overridingDrmInitData= */ ImmutableMap.of(), + mock(Allocator.class), + /* positionUs= */ 0, + /* muxedAudioFormat= */ null, + mock(DrmSessionManager.class), + new DrmSessionEventListener.EventDispatcher(), + mock(LoadErrorHandlingPolicy.class), + new MediaSourceEventListener.EventDispatcher(), + /* metadataType= */ HlsMediaSource.METADATA_TYPE_ID3, + /* downloadExecutor= */ null); + mediaChunk.init(sampleStreamWrapper, ImmutableList.of()); + return mediaChunk; + } +} diff --git a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsMediaPeriodTest.java b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsMediaPeriodTest.java index 1ea8a2b4a1a..2413ef8bdba 100644 --- a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsMediaPeriodTest.java +++ b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsMediaPeriodTest.java @@ -27,8 +27,10 @@ import android.net.Uri; import android.os.SystemClock; +import androidx.annotation.Nullable; import androidx.media3.common.C; import androidx.media3.common.Format; +import androidx.media3.common.Metadata; import androidx.media3.common.MimeTypes; import androidx.media3.common.TrackGroup; import androidx.media3.datasource.DataSource; @@ -70,33 +72,6 @@ public final class HlsMediaPeriodTest { @Test public void getSteamKeys_isCompatibleWithHlsMultivariantPlaylistFilter() { - HlsMultivariantPlaylist testMultivariantPlaylist = - createMultivariantPlaylist( - /* variants= */ Arrays.asList( - createAudioOnlyVariant(Uri.parse("https://variant1"), /* peakBitrate= */ 10000), - createMuxedVideoAudioVariant( - Uri.parse("https://variant2"), /* peakBitrate= */ 200000), - createMuxedVideoAudioVariant( - Uri.parse("https://backup/variant2"), /* peakBitrate= */ 200000), - createAudioOnlyVariant(Uri.parse("https://variant3"), /* peakBitrate= */ 300000), - createMuxedVideoAudioVariant( - Uri.parse("https://variant4"), /* peakBitrate= */ 400000), - createMuxedVideoAudioVariant( - Uri.parse("https://variant5"), /* peakBitrate= */ 600000)), - /* audios= */ Arrays.asList( - createAudioRendition(Uri.parse("https://audio1"), /* language= */ "spa"), - createAudioRendition(Uri.parse("https://backup/audio1"), /* language= */ "spa"), - createAudioRendition(Uri.parse("https://audio2"), /* language= */ "ger"), - createAudioRendition(Uri.parse("https://audio3"), /* language= */ "tur")), - /* subtitles= */ Arrays.asList( - createSubtitleRendition(Uri.parse("https://subtitle1"), /* language= */ "spa"), - createSubtitleRendition( - Uri.parse("https://backup/subtitle1"), /* language= */ "spa"), - createSubtitleRendition(Uri.parse("https://subtitle2"), /* language= */ "ger"), - createSubtitleRendition(Uri.parse("https://subtitle3"), /* language= */ "tur")), - /* muxedAudioFormat= */ createAudioFormat("eng"), - /* muxedCaptionFormats= */ Arrays.asList( - createSubtitleFormat("eng"), createSubtitleFormat("gsw"))); FilterableManifestMediaPeriodFactory mediaPeriodFactory = (playlist, periodIndex) -> { HlsExtractorFactory mockHlsExtractorFactory = mock(HlsExtractorFactory.class); @@ -129,11 +104,60 @@ public void getSteamKeys_isCompatibleWithHlsMultivariantPlaylistFilter() { /* downloadExecutorSupplier= */ null); }; + HlsMultivariantPlaylist testMultivariantPlaylist = + createMultivariantPlaylist( + /* variants= */ Arrays.asList( + createAudioOnlyVariant(Uri.parse("https://variant1"), /* peakBitrate= */ 10000), + createMuxedVideoAudioVariant( + Uri.parse("https://variant2"), /* peakBitrate= */ 200000), + createMuxedVideoAudioVariant( + Uri.parse("https://backup/variant2"), /* peakBitrate= */ 200000), + createAudioOnlyVariant(Uri.parse("https://variant3"), /* peakBitrate= */ 300000), + createMuxedVideoAudioVariant( + Uri.parse("https://variant4"), /* peakBitrate= */ 400000), + createMuxedVideoAudioVariant( + Uri.parse("https://variant5"), /* peakBitrate= */ 600000)), + /* audios= */ Arrays.asList( + createAudioRendition(Uri.parse("https://audio1"), /* language= */ "spa"), + createAudioRendition(Uri.parse("https://backup/audio1"), /* language= */ "spa"), + createAudioRendition(Uri.parse("https://audio2"), /* language= */ "ger"), + createAudioRendition(Uri.parse("https://audio3"), /* language= */ "tur")), + /* subtitles= */ Arrays.asList( + createSubtitleRendition(Uri.parse("https://subtitle1"), /* language= */ "spa"), + createSubtitleRendition( + Uri.parse("https://backup/subtitle1"), /* language= */ "spa"), + createSubtitleRendition(Uri.parse("https://subtitle2"), /* language= */ "ger"), + createSubtitleRendition(Uri.parse("https://subtitle3"), /* language= */ "tur")), + /* muxedAudioFormat= */ createAudioFormat("eng"), + /* muxedCaptionFormats= */ Arrays.asList( + createSubtitleFormat("eng"), createSubtitleFormat("gsw"))); MediaPeriodAsserts.assertGetStreamKeysAndManifestFilterIntegration( mediaPeriodFactory, testMultivariantPlaylist, /* periodIndex= */ 0, /* ignoredMimeType= */ APPLICATION_ID3); + + HlsMultivariantPlaylist testMultivariantPlaylistWithoutSubtitles = + createMultivariantPlaylist( + /* variants= */ Arrays.asList( + createAudioOnlyVariant(Uri.parse("https://variant1"), /* peakBitrate= */ 10000), + createMuxedVideoAudioVariant( + Uri.parse("https://variant2"), /* peakBitrate= */ 200000), + createAudioOnlyVariant(Uri.parse("https://variant3"), /* peakBitrate= */ 300000), + createMuxedVideoAudioVariant( + Uri.parse("https://variant4"), /* peakBitrate= */ 400000)), + /* audios= */ Arrays.asList( + createAudioRendition(Uri.parse("https://audio1"), /* language= */ "spa"), + createAudioRendition(Uri.parse("https://audio2"), /* language= */ "ger")), + /* subtitles= */ ImmutableList.of(), + /* muxedAudioFormat= */ createAudioFormat("eng"), + /* muxedCaptionFormats= */ Arrays.asList( + createSubtitleFormat("eng"), createSubtitleFormat("gsw"))); + MediaPeriodAsserts.assertGetStreamKeysAndManifestFilterIntegration( + mediaPeriodFactory, + testMultivariantPlaylistWithoutSubtitles, + /* periodIndex= */ 0, + /* ignoredMimeType= */ APPLICATION_ID3); } @Test @@ -246,6 +270,136 @@ public void onPlaylistError_returnsCorrectResult() { .isFalse(); } + @Test + public void prepare_withVariantsAndRenditions_preservesFormatIdAndMetadata() { + Metadata variantMetadata1 = + new Metadata( + new HlsTrackMetadataEntry( + /* groupId= */ null, /* name= */ null, /* variantInfos= */ ImmutableList.of())); + Metadata variantMetadata2 = + new Metadata( + new HlsTrackMetadataEntry( + /* groupId= */ "backup", /* name= */ null, /* variantInfos= */ ImmutableList.of())); + Metadata audioMetadata = + new Metadata( + new HlsTrackMetadataEntry( + /* groupId= */ "audio", + /* name= */ "English", + /* variantInfos= */ ImmutableList.of())); + Metadata subtitleMetadata = + new Metadata( + new HlsTrackMetadataEntry( + /* groupId= */ "sub", + /* name= */ "English", + /* variantInfos= */ ImmutableList.of())); + HlsMultivariantPlaylist multivariantPlaylist = + createMultivariantPlaylist( + /* variants= */ ImmutableList.of( + createMuxedVideoAudioVariant( + Uri.parse("https://variant1"), + /* peakBitrate= */ 400000, + /* id= */ "0", + variantMetadata1), + createMuxedVideoAudioVariant( + Uri.parse("https://backup/variant1"), + /* peakBitrate= */ 400000, + /* id= */ "1", + variantMetadata2)), + /* audios= */ ImmutableList.of( + createAudioRendition( + Uri.parse("https://audio1"), + /* language= */ "en", + /* id= */ "audio:English", + audioMetadata, + /* groupId= */ "audio", + /* name= */ "English")), + /* subtitles= */ ImmutableList.of( + createSubtitleRendition( + Uri.parse("https://subtitle1"), + /* language= */ "en", + /* id= */ "sub:English", + subtitleMetadata, + /* groupId= */ "sub", + /* name= */ "English")), + /* muxedAudioFormat= */ null, + /* muxedCaptionFormats= */ ImmutableList.of()); + + HlsMediaPeriod mediaPeriod = createMediaPeriod(multivariantPlaylist); + + TrackGroupArray expectedGroups = + new TrackGroupArray( + new TrackGroup( + "main", + new Format.Builder() + .setId("0") + .setContainerMimeType(MimeTypes.APPLICATION_M3U8) + .setCodecs("avc1.100.41") + .setSampleMimeType(VIDEO_H264) + .setPeakBitrate(400000) + .setMetadata(variantMetadata1) + .build()), + new TrackGroup( + "main:id3", + new Format.Builder() + .setId("ID3") + .setSampleMimeType(APPLICATION_ID3) + .setPrimaryTrackGroupId("main") + .build()), + new TrackGroup( + "audio:English", + createAudioFormat("en") + .buildUpon() + .setId("audio:English") + .setMetadata(audioMetadata) + .build()), + new TrackGroup( + "audio:English:id3", + new Format.Builder() + .setId("ID3") + .setSampleMimeType(APPLICATION_ID3) + .setPrimaryTrackGroupId("audio:English") + .build()), + new TrackGroup( + "subtitle:English", + createSubtitleFormat("en") + .buildUpon() + .setId("sub:English") + .setMetadata(subtitleMetadata) + .build())); + MediaPeriodAsserts.assertTrackGroups(mediaPeriod, expectedGroups); + } + + private static HlsMediaPeriod createMediaPeriod(HlsMultivariantPlaylist multivariantPlaylist) { + HlsExtractorFactory mockHlsExtractorFactory = mock(HlsExtractorFactory.class); + when(mockHlsExtractorFactory.getOutputTextFormat(any())) + .then(invocation -> invocation.getArguments()[0]); + HlsDataSourceFactory mockDataSourceFactory = mock(HlsDataSourceFactory.class); + when(mockDataSourceFactory.createDataSource(anyInt())).thenReturn(mock(DataSource.class)); + HlsPlaylistTracker mockPlaylistTracker = mock(HlsPlaylistTracker.class); + setupPlaylistTracker(mockPlaylistTracker, multivariantPlaylist); + MediaPeriodId mediaPeriodId = new MediaPeriodId(/* periodUid= */ new Object()); + return new HlsMediaPeriod( + mockHlsExtractorFactory, + mockPlaylistTracker, + mockDataSourceFactory, + mock(TransferListener.class), + /* cmcdConfiguration= */ null, + mock(DrmSessionManager.class), + new DrmSessionEventListener.EventDispatcher() + .withParameters(/* windowIndex= */ 0, mediaPeriodId), + mock(LoadErrorHandlingPolicy.class), + new MediaSourceEventListener.EventDispatcher() + .withParameters(/* windowIndex= */ 0, mediaPeriodId), + mock(Allocator.class), + mock(CompositeSequenceableLoaderFactory.class), + /* allowChunklessPreparation= */ true, + HlsMediaSource.METADATA_TYPE_ID3, + /* useSessionKeys= */ false, + PlayerId.UNSET, + /* timestampAdjusterInitializationTimeoutMs= */ 0, + /* downloadExecutorSupplier= */ null); + } + private static HlsMultivariantPlaylist createMultivariantPlaylist( List variants, List audios, @@ -269,12 +423,19 @@ private static HlsMultivariantPlaylist createMultivariantPlaylist( } private static Variant createMuxedVideoAudioVariant(Uri url, int peakBitrate) { + return createMuxedVideoAudioVariant(url, peakBitrate, /* id= */ null, /* metadata= */ null); + } + + private static Variant createMuxedVideoAudioVariant( + Uri url, int peakBitrate, @Nullable String id, @Nullable Metadata metadata) { return createVariant( url, new Format.Builder() + .setId(id) .setContainerMimeType(MimeTypes.APPLICATION_M3U8) .setCodecs("avc1.100.41,mp4a.40.2") .setPeakBitrate(peakBitrate) + .setMetadata(metadata) .build()); } @@ -289,11 +450,41 @@ private static Variant createAudioOnlyVariant(Uri url, int peakBitrate) { } private static Rendition createAudioRendition(Uri url, String language) { - return createRendition(url, createAudioFormat(language), "", ""); + return createAudioRendition( + url, language, /* id= */ null, /* metadata= */ null, /* groupId= */ "", /* name= */ ""); + } + + private static Rendition createAudioRendition( + Uri url, + String language, + @Nullable String id, + @Nullable Metadata metadata, + String groupId, + String name) { + return createRendition( + url, + createAudioFormat(language).buildUpon().setId(id).setMetadata(metadata).build(), + groupId, + name); } private static Rendition createSubtitleRendition(Uri url, String language) { - return createRendition(url, createSubtitleFormat(language), "", ""); + return createSubtitleRendition( + url, language, /* id= */ null, /* metadata= */ null, /* groupId= */ "", /* name= */ ""); + } + + private static Rendition createSubtitleRendition( + Uri url, + String language, + @Nullable String id, + @Nullable Metadata metadata, + String groupId, + String name) { + return createRendition( + url, + createSubtitleFormat(language).buildUpon().setId(id).setMetadata(metadata).build(), + groupId, + name); } private static Variant createVariant(Uri url, Format format) { @@ -332,7 +523,8 @@ private static Format createSubtitleFormat(String language) { private LoadErrorHandlingPolicy.LoadErrorInfo createFakeLoadErrorInfo( DataSpec dataSpec, int httpResponseCode, int errorCount) { LoadEventInfo loadEventInfo = - new LoadEventInfo(/* loadTaskId= */ 0, dataSpec, SystemClock.elapsedRealtime()); + new LoadEventInfo.Builder(/* loadTaskId= */ 0, dataSpec, SystemClock.elapsedRealtime()) + .build(); MediaLoadData mediaLoadData = new MediaLoadData(C.DATA_TYPE_MEDIA); HttpDataSource.InvalidResponseCodeException invalidResponseCodeException = new HttpDataSource.InvalidResponseCodeException( diff --git a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsPreCacheHelperTest.java b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsPreCacheHelperTest.java index 77d4277c042..94f5abfa0c3 100644 --- a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsPreCacheHelperTest.java +++ b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsPreCacheHelperTest.java @@ -17,8 +17,6 @@ import static androidx.media3.test.utils.robolectric.RobolectricUtil.runMainLooperUntil; import static org.mockito.ArgumentMatchers.any; -import static org.mockito.ArgumentMatchers.anyLong; -import static org.mockito.ArgumentMatchers.eq; import static org.mockito.Mockito.doAnswer; import static org.mockito.Mockito.mock; import static org.mockito.Mockito.never; @@ -72,7 +70,7 @@ public void preCache_succeeds() throws Exception { return null; }) .when(preCacheHelperListener) - .onPreCacheProgress(any(), anyLong(), anyLong(), eq(100f)); + .onPreCacheCompleted(any()); PreCacheHelper preCacheHelper = new PreCacheHelper.Factory( ApplicationProvider.getApplicationContext(), @@ -90,7 +88,7 @@ public void preCache_succeeds() throws Exception { verify(preCacheHelperListener, never()).onPrepareError(any(), any()); verify(preCacheHelperListener, never()).onDownloadError(any(), any()); - preCacheHelper.release(/* removeCachedContent= */ true); + preCacheHelper.release(/* removeCachedContent= */ false); shadowOf(preCacheLooper).idle(); } } diff --git a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsPreloadMediaSourceTest.java b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsPreloadMediaSourceTest.java index f993cabf81a..2dba372a3bc 100644 --- a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsPreloadMediaSourceTest.java +++ b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsPreloadMediaSourceTest.java @@ -128,7 +128,7 @@ public void preload_loadPeriodToTargetPreloadPosition() throws Exception { }); TrackSelector trackSelector = new DefaultTrackSelector(ApplicationProvider.getApplicationContext()); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mediaSourceFactory, @@ -164,7 +164,7 @@ public void preload_stopWhenTracksSelectedByPreloadControl() throws Exception { }); TrackSelector trackSelector = new DefaultTrackSelector(ApplicationProvider.getApplicationContext()); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mediaSourceFactory, @@ -200,7 +200,7 @@ public void preload_stopWhenSourcePreparedByPreloadControl() throws Exception { }); TrackSelector trackSelector = new DefaultTrackSelector(ApplicationProvider.getApplicationContext()); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mediaSourceFactory, @@ -237,7 +237,7 @@ public void preload_loadToTheEndOfSource() throws Exception { .onLoadedToTheEndOfSource(any()); TrackSelector trackSelector = new DefaultTrackSelector(ApplicationProvider.getApplicationContext()); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); PreloadMediaSource.Factory preloadMediaSourceFactory = new PreloadMediaSource.Factory( mediaSourceFactory, @@ -277,7 +277,7 @@ public void preload_withSmallTargetBufferBytesThreshold_onLoadingUnableToContinu .onLoadedToTheEndOfSource(any()); TrackSelector trackSelector = new DefaultTrackSelector(ApplicationProvider.getApplicationContext()); - trackSelector.init(() -> {}, bandwidthMeter); + trackSelector.init(unused -> {}, bandwidthMeter); LoadControl loadControl = mock( LoadControl.class, diff --git a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsSampleStreamWrapperTest.java b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsSampleStreamWrapperTest.java new file mode 100644 index 00000000000..e94eaef1030 --- /dev/null +++ b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/HlsSampleStreamWrapperTest.java @@ -0,0 +1,879 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.exoplayer.hls; + +import static androidx.media3.common.util.Util.createHandlerForCurrentLooper; +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.util.concurrent.MoreExecutors.directExecutor; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyBoolean; +import static org.mockito.ArgumentMatchers.anyLong; +import static org.mockito.Mockito.doAnswer; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; +import static org.robolectric.Shadows.shadowOf; + +import android.net.Uri; +import android.os.Looper; +import androidx.annotation.Nullable; +import androidx.media3.common.C; +import androidx.media3.common.Format; +import androidx.media3.common.MimeTypes; +import androidx.media3.common.TrackGroup; +import androidx.media3.datasource.DataSpec; +import androidx.media3.datasource.HttpDataSource; +import androidx.media3.exoplayer.LoadingInfo; +import androidx.media3.exoplayer.analytics.PlayerId; +import androidx.media3.exoplayer.drm.DrmSessionEventListener; +import androidx.media3.exoplayer.drm.DrmSessionManager; +import androidx.media3.exoplayer.hls.playlist.HlsMediaPlaylist; +import androidx.media3.exoplayer.source.LoadEventInfo; +import androidx.media3.exoplayer.source.MediaLoadData; +import androidx.media3.exoplayer.source.MediaSource; +import androidx.media3.exoplayer.source.MediaSourceEventListener; +import androidx.media3.exoplayer.source.SampleStream; +import androidx.media3.exoplayer.source.chunk.Chunk; +import androidx.media3.exoplayer.upstream.Allocator; +import androidx.media3.exoplayer.upstream.DefaultLoadErrorHandlingPolicy; +import androidx.media3.exoplayer.upstream.LoadErrorHandlingPolicy; +import androidx.media3.exoplayer.upstream.Loader; +import androidx.media3.exoplayer.util.ReleasableExecutor; +import androidx.media3.extractor.ExtractorOutput; +import androidx.media3.extractor.TrackOutput; +import androidx.media3.test.utils.FakeDataSource; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.atomic.AtomicReference; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.mockito.Mock; +import org.mockito.junit.MockitoJUnit; +import org.mockito.junit.MockitoRule; + +/** Unit tests for {@link HlsSampleStreamWrapper}. */ +@RunWith(AndroidJUnit4.class) +public final class HlsSampleStreamWrapperTest { + + @Rule public final MockitoRule mockito = MockitoJUnit.rule(); + + @Mock private HlsExtractorFactory mockExtractorFactory; + @Mock private HlsChunkSource mockChunkSource; + @Mock private HlsMediaChunkExtractor mockExtractor; + + @Before + public void setUp() throws Exception { + when(mockExtractorFactory.createExtractor(any(), any(), any(), any(), any(), any(), any())) + .thenReturn(mockExtractor); + // Extractor returns false for read to signal mock end-of-stream and complete load successfully + when(mockExtractor.read(any())).thenReturn(false); + } + + @Test + public void discardUpstream_allChunksDiscarded_keepLoadingFromStartPositionOfFirstDroppedChunk() { + HlsSampleStreamWrapper sampleStreamWrapper = + new HlsSampleStreamWrapper( + /* uid= */ "uid", + C.TRACK_TYPE_VIDEO, + mock(HlsSampleStreamWrapper.Callback.class), + mockChunkSource, + /* overridingDrmInitData= */ ImmutableMap.of(), + mock(Allocator.class), + /* positionUs= */ 0L, + /* muxedAudioFormat= */ null, + mock(DrmSessionManager.class), + new DrmSessionEventListener.EventDispatcher(), + mock(LoadErrorHandlingPolicy.class), + new MediaSourceEventListener.EventDispatcher() + .withParameters(/* windowIndex= */ 0, new MediaSource.MediaPeriodId(new Object())), + /* metadataType= */ HlsMediaSource.METADATA_TYPE_ID3, + /* downloadExecutor= */ ReleasableExecutor.from(directExecutor(), e -> {})); + sampleStreamWrapper.prepareWithMultivariantPlaylistInfo( + new TrackGroup[] {new TrackGroup(new Format.Builder().build())}, + /* primaryTrackGroupIndex= */ 0); + // Create a Part with start time at 1_000_000L and duration of 1_000_000L. + HlsMediaPlaylist.Part part = + new HlsMediaPlaylist.Part( + /* url= */ "part1.ts", + /* initializationSegment= */ null, + /* durationUs= */ 1_000_000L, + /* relativeDiscontinuitySequence= */ 0, + /* relativeStartTimeUs= */ 1_000_000L, + /* drmInitData= */ null, + /* fullSegmentEncryptionKeyUri= */ null, + /* encryptionIV= */ null, + /* byteRangeOffset= */ 0, + /* byteRangeLength= */ C.LENGTH_UNSET, + /* hasGapTag= */ false, + /* isIndependent= */ true, + /* isPreload= */ false); + HlsMediaPlaylist playlist = + new HlsMediaPlaylist( + HlsMediaPlaylist.PLAYLIST_TYPE_UNKNOWN, + /* baseUri= */ "http://example.com/", + /* tags= */ ImmutableList.of(), + /* startOffsetUs= */ C.TIME_UNSET, + /* preciseStart= */ false, + /* startTimeUs= */ 0L, + /* hasDiscontinuitySequence= */ false, + /* discontinuitySequence= */ 0, + /* mediaSequence= */ 0L, + /* version= */ 7, + /* targetDurationUs= */ 4_000_000L, + /* partTargetDurationUs= */ C.TIME_UNSET, + /* hasIndependentSegments= */ true, + /* hasEndTag= */ false, + /* hasProgramDateTime= */ false, + /* protectionSchemes= */ null, + /* segments= */ ImmutableList.of(), + /* trailingParts= */ ImmutableList.of(part), + new HlsMediaPlaylist.ServerControl( + /* skipUntilUs= */ C.TIME_UNSET, + /* canSkipDateRanges= */ false, + /* holdBackUs= */ C.TIME_UNSET, + /* partHoldBackUs= */ C.TIME_UNSET, + /* canBlockReload= */ false), + /* renditionReports= */ ImmutableMap.of(), + /* interstitials= */ ImmutableList.of(), + /* lastSeenInitSegment= */ null); + FakeDataSource fakeDataSource = new FakeDataSource(); + fakeDataSource.getDataSet().newDefaultData().appendReadData(10).endData(); + HlsMediaChunk chunk = + HlsMediaChunk.createInstance( + mockExtractorFactory, + fakeDataSource, + new Format.Builder().build(), + /* startOfPlaylistInPeriodUs= */ 0L, + playlist, + new HlsChunkSource.SegmentBaseHolder( + playlist.trailingParts.get(0), /* mediaSequence= */ 0, /* partIndex= */ 0), + Uri.parse("http://example.com/part1.ts"), + /* steeredPathwayId= */ null, + /* muxedCaptionFormats= */ null, + C.SELECTION_REASON_INITIAL, + /* trackSelectionData= */ null, + /* isPrimaryTimestampSource= */ true, + new TimestampAdjusterProvider(), + /* timestampAdjusterInitializationTimeoutMs= */ 0, + /* previousChunk= */ null, + /* mediaSegmentKey= */ null, + /* initSegmentKey= */ null, + /* shouldSpliceIn= */ false, + /* isIndependent= */ true, + PlayerId.UNSET, + /* cmcdDataFactory= */ null); + doAnswer( + invocation -> { + HlsChunkSource.HlsChunkHolder chunkHolder = invocation.getArgument(5); + chunkHolder.chunk = chunk; + return null; + }) + .when(mockChunkSource) + .getNextChunk(any(), anyLong(), anyLong(), any(), anyBoolean(), any()); + // Call continueLoading to add the chunk to mediaChunks queue + sampleStreamWrapper.continueLoading(new LoadingInfo.Builder().setPlaybackPositionUs(0).build()); + shadowOf(Looper.getMainLooper()).idle(); + // Next load position should be 2_000_000L + assertThat(sampleStreamWrapper.getNextLoadPositionUs()).isEqualTo(2_000_000L); + // Stub mockChunkSource to mark the chunk as removed during reevaluation + when(mockChunkSource.shouldCancelLoad(anyLong(), any(), any())).thenReturn(false); + when(mockChunkSource.getChunkPublicationState(chunk)) + .thenReturn(HlsChunkSource.CHUNK_PUBLICATION_STATE_REMOVED); + + // Call reevaluateBuffer to trigger discardUpstream, assuming the playback position is 500ms. + sampleStreamWrapper.reevaluateBuffer(/* positionUs= */ 500_000L); + + // Since the chunk was discarded and the chunk queue is empty, the next load position should + // be reset to the start position of the first chunk dropped. + assertThat(sampleStreamWrapper.getNextLoadPositionUs()).isEqualTo(1_000_000L); + } + + @Test + public void getStreamFlags_beforeChunksLoaded_returnsFlagMaybeHasPreroll() { + HlsSampleStreamWrapper sampleStreamWrapper = + createSampleStreamWrapper(/* positionUs= */ 1_500_000L); + assertThat(sampleStreamWrapper.getStreamFlags(/* trackGroupIndex= */ 0)) + .isEqualTo(SampleStream.FLAG_MAYBE_HAS_PREROLL); + } + + @Test + public void getStreamFlags_beforeChunksLoaded_syncAudioTrack_returnsZero() { + HlsSampleStreamWrapper sampleStreamWrapper = + createSampleStreamWrapper( + /* positionUs= */ 1_500_000L, + new Format.Builder() + .setSampleMimeType(MimeTypes.AUDIO_AAC) + .setCodecs("mp4a.40.2") + .build()); + assertThat(sampleStreamWrapper.getStreamFlags(/* trackGroupIndex= */ 0)).isEqualTo(0); + } + + @Test + public void getStreamFlags_segmentAlignedResumption_returnsZero() { + HlsSampleStreamWrapper sampleStreamWrapper = + createSampleStreamWrapper(/* positionUs= */ 1_000_000L); + HlsMediaChunk chunk = + createMediaChunk(/* startTimeUs= */ 1_000_000L, /* durationUs= */ 1_000_000L); + doAnswer( + invocation -> { + HlsChunkSource.HlsChunkHolder chunkHolder = invocation.getArgument(5); + chunkHolder.chunk = chunk; + return null; + }) + .when(mockChunkSource) + .getNextChunk(any(), anyLong(), anyLong(), any(), anyBoolean(), any()); + + sampleStreamWrapper.continueLoading( + new LoadingInfo.Builder().setPlaybackPositionUs(1_000_000L).build()); + shadowOf(Looper.getMainLooper()).idle(); + + assertThat(sampleStreamWrapper.getStreamFlags(/* trackGroupIndex= */ 0)).isEqualTo(0); + } + + @Test + public void getStreamFlags_midSegmentResumption_returnsFlagHasPreroll() { + HlsSampleStreamWrapper sampleStreamWrapper = + createSampleStreamWrapper( + /* positionUs= */ 1_500_000L, + new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build()); + Format sampleFormat = new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build(); + doAnswer( + invocation -> { + ExtractorOutput extractorOutput = invocation.getArgument(0); + extractorOutput.track(/* id= */ 0, C.TRACK_TYPE_VIDEO).format(sampleFormat); + extractorOutput.endTracks(); + return null; + }) + .when(mockExtractor) + .init(any()); + HlsMediaChunk chunk = + createMediaChunk(/* startTimeUs= */ 1_000_000L, /* durationUs= */ 1_000_000L); + doAnswer( + invocation -> { + HlsChunkSource.HlsChunkHolder chunkHolder = invocation.getArgument(5); + chunkHolder.chunk = chunk; + return null; + }) + .when(mockChunkSource) + .getNextChunk(any(), anyLong(), anyLong(), any(), anyBoolean(), any()); + + sampleStreamWrapper.continueLoading( + new LoadingInfo.Builder().setPlaybackPositionUs(1_500_000L).build()); + shadowOf(Looper.getMainLooper()).idle(); + + assertThat(sampleStreamWrapper.getStreamFlags(/* trackGroupIndex= */ 0)) + .isEqualTo(SampleStream.FLAG_HAS_PREROLL); + } + + @Test + public void getStreamFlags_midSegmentResumption_upstreamFormatNull_returnsFlagMaybeHasPreroll() { + HlsSampleStreamWrapper sampleStreamWrapper = + createSampleStreamWrapper( + /* positionUs= */ 1_500_000L, + new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build()); + AtomicReference trackOutput = new AtomicReference<>(); + doAnswer( + invocation -> { + ExtractorOutput extractorOutput = invocation.getArgument(0); + trackOutput.set(extractorOutput.track(/* id= */ 0, C.TRACK_TYPE_VIDEO)); + extractorOutput.endTracks(); + return null; + }) + .when(mockExtractor) + .init(any()); + HlsMediaChunk chunk = + createMediaChunk(/* startTimeUs= */ 1_000_000L, /* durationUs= */ 1_000_000L); + doAnswer( + invocation -> { + HlsChunkSource.HlsChunkHolder chunkHolder = invocation.getArgument(5); + chunkHolder.chunk = chunk; + return null; + }) + .when(mockChunkSource) + .getNextChunk(any(), anyLong(), anyLong(), any(), anyBoolean(), any()); + + sampleStreamWrapper.continueLoading( + new LoadingInfo.Builder().setPlaybackPositionUs(1_500_000L).build()); + shadowOf(Looper.getMainLooper()).idle(); + + // The chunk is loaded, but headers have not been parsed yet (upstreamFormat == null). + assertThat(sampleStreamWrapper.getStreamFlags(/* trackGroupIndex= */ 0)) + .isEqualTo(SampleStream.FLAG_MAYBE_HAS_PREROLL); + + // Headers are parsed and upstreamFormat is populated. + trackOutput.get().format(new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build()); + shadowOf(Looper.getMainLooper()).idle(); + + // Video tracks require preroll frames to be decoded; preroll flag must be reported. + assertThat(sampleStreamWrapper.getStreamFlags(/* trackGroupIndex= */ 0)) + .isEqualTo(SampleStream.FLAG_HAS_PREROLL); + } + + @Test + public void getStreamFlags_syncAudioTrack_reportsZeroForMidSegmentResumption() { + HlsSampleStreamWrapper sampleStreamWrapper = + createSampleStreamWrapper( + /* positionUs= */ 1_500_000L, + new Format.Builder() + .setSampleMimeType(MimeTypes.AUDIO_AAC) + .setCodecs("mp4a.40.2") + .build()); + HlsMediaChunk chunk = + createMediaChunk(/* startTimeUs= */ 1_000_000L, /* durationUs= */ 1_000_000L); + doAnswer( + invocation -> { + HlsChunkSource.HlsChunkHolder chunkHolder = invocation.getArgument(5); + chunkHolder.chunk = chunk; + return null; + }) + .when(mockChunkSource) + .getNextChunk(any(), anyLong(), anyLong(), any(), anyBoolean(), any()); + + sampleStreamWrapper.continueLoading( + new LoadingInfo.Builder().setPlaybackPositionUs(1_500_000L).build()); + shadowOf(Looper.getMainLooper()).idle(); + + // Sync audio discards samples up to start time; no preroll should be reported. + assertThat(sampleStreamWrapper.getStreamFlags(/* trackGroupIndex= */ 0)).isEqualTo(0); + } + + @Test + public void getStreamFlags_beforeChunksLoaded_videoApv_returnsMaybeHasPreroll() { + HlsSampleStreamWrapper sampleStreamWrapper = + createSampleStreamWrapper( + /* positionUs= */ 1_500_000L, + new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_APV).build()); + assertThat(sampleStreamWrapper.getStreamFlags(/* trackGroupIndex= */ 0)) + .isEqualTo(SampleStream.FLAG_MAYBE_HAS_PREROLL); + } + + @Test + public void getStreamFlags_midSegmentResumption_videoApv_returnsFlagHasPreroll() { + HlsSampleStreamWrapper sampleStreamWrapper = + createSampleStreamWrapper( + /* positionUs= */ 1_500_000L, + new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_APV).build()); + Format sampleFormat = new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_APV).build(); + doAnswer( + invocation -> { + ExtractorOutput extractorOutput = invocation.getArgument(0); + extractorOutput.track(/* id= */ 0, C.TRACK_TYPE_VIDEO).format(sampleFormat); + extractorOutput.endTracks(); + return null; + }) + .when(mockExtractor) + .init(any()); + HlsMediaChunk chunk = + createMediaChunk(/* startTimeUs= */ 1_000_000L, /* durationUs= */ 1_000_000L); + doAnswer( + invocation -> { + HlsChunkSource.HlsChunkHolder chunkHolder = invocation.getArgument(5); + chunkHolder.chunk = chunk; + return null; + }) + .when(mockChunkSource) + .getNextChunk(any(), anyLong(), anyLong(), any(), anyBoolean(), any()); + + sampleStreamWrapper.continueLoading( + new LoadingInfo.Builder().setPlaybackPositionUs(1_500_000L).build()); + shadowOf(Looper.getMainLooper()).idle(); + + // Even though APV only contains sync frames, it is a video format and SampleQueue does not + // discard samples to start time; preroll flag must be reported. + assertThat(sampleStreamWrapper.getStreamFlags(/* trackGroupIndex= */ 0)) + .isEqualTo(SampleStream.FLAG_HAS_PREROLL); + } + + @Test + public void getStreamFlags_midSegmentResumption_nonSyncAudio_returnsFlagHasPreroll() { + HlsSampleStreamWrapper sampleStreamWrapper = + createSampleStreamWrapper( + /* positionUs= */ 1_500_000L, + new Format.Builder().setSampleMimeType(MimeTypes.AUDIO_VORBIS).build()); + Format sampleFormat = new Format.Builder().setSampleMimeType(MimeTypes.AUDIO_VORBIS).build(); + doAnswer( + invocation -> { + ExtractorOutput extractorOutput = invocation.getArgument(0); + extractorOutput.track(/* id= */ 0, C.TRACK_TYPE_AUDIO).format(sampleFormat); + extractorOutput.endTracks(); + return null; + }) + .when(mockExtractor) + .init(any()); + HlsMediaChunk chunk = + createMediaChunk(/* startTimeUs= */ 1_000_000L, /* durationUs= */ 1_000_000L); + doAnswer( + invocation -> { + HlsChunkSource.HlsChunkHolder chunkHolder = invocation.getArgument(5); + chunkHolder.chunk = chunk; + return null; + }) + .when(mockChunkSource) + .getNextChunk(any(), anyLong(), anyLong(), any(), anyBoolean(), any()); + + sampleStreamWrapper.continueLoading( + new LoadingInfo.Builder().setPlaybackPositionUs(1_500_000L).build()); + shadowOf(Looper.getMainLooper()).idle(); + + // Non-sync audio does not discard samples up to start time; preroll flag must be reported. + assertThat(sampleStreamWrapper.getStreamFlags(/* trackGroupIndex= */ 0)) + .isEqualTo(SampleStream.FLAG_HAS_PREROLL); + } + + @Test + public void getStreamFlags_loadingFinishedWithoutMediaChunks_returnsZero() { + HlsSampleStreamWrapper sampleStreamWrapper = + createSampleStreamWrapper(/* positionUs= */ 1_500_000L); + doAnswer( + invocation -> { + HlsChunkSource.HlsChunkHolder chunkHolder = invocation.getArgument(5); + chunkHolder.endOfStream = true; + return null; + }) + .when(mockChunkSource) + .getNextChunk(any(), anyLong(), anyLong(), any(), anyBoolean(), any()); + + sampleStreamWrapper.continueLoading( + new LoadingInfo.Builder().setPlaybackPositionUs(1_500_000L).build()); + shadowOf(Looper.getMainLooper()).idle(); + + // No chunk will ever be loaded, so the stream must not stay stuck on FLAG_MAYBE_HAS_PREROLL. + assertThat(sampleStreamWrapper.getStreamFlags(/* trackGroupIndex= */ 0)).isEqualTo(0); + } + + @Test + public void getStreamFlags_unmappedTrackGroup_returnsZero() { + Format videoFormat = new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build(); + Format optionalFormat = + new Format.Builder().setSampleMimeType(MimeTypes.APPLICATION_SUBRIP).build(); + HlsSampleStreamWrapper sampleStreamWrapper = + new HlsSampleStreamWrapper( + /* uid= */ "uid", + C.TRACK_TYPE_VIDEO, + mock(HlsSampleStreamWrapper.Callback.class), + mockChunkSource, + /* overridingDrmInitData= */ ImmutableMap.of(), + mock(Allocator.class), + /* positionUs= */ 1_500_000L, + /* muxedAudioFormat= */ null, + mock(DrmSessionManager.class), + new DrmSessionEventListener.EventDispatcher(), + mock(LoadErrorHandlingPolicy.class), + new MediaSourceEventListener.EventDispatcher(), + /* metadataType= */ HlsMediaSource.METADATA_TYPE_ID3, + /* downloadExecutor= */ ReleasableExecutor.from(directExecutor(), e -> {})); + sampleStreamWrapper.prepareWithMultivariantPlaylistInfo( + new TrackGroup[] {new TrackGroup(videoFormat), new TrackGroup(optionalFormat)}, + /* primaryTrackGroupIndex= */ 0, + /* optionalTrackGroupsIndices...= */ 1); + doAnswer( + invocation -> { + ExtractorOutput extractorOutput = invocation.getArgument(0); + extractorOutput.track(/* id= */ 0, C.TRACK_TYPE_VIDEO).format(videoFormat); + extractorOutput.endTracks(); + return null; + }) + .when(mockExtractor) + .init(any()); + HlsMediaChunk chunk = + createMediaChunk(/* startTimeUs= */ 1_000_000L, /* durationUs= */ 1_000_000L); + doAnswer( + invocation -> { + HlsChunkSource.HlsChunkHolder chunkHolder = invocation.getArgument(5); + chunkHolder.chunk = chunk; + return null; + }) + .when(mockChunkSource) + .getNextChunk(any(), anyLong(), anyLong(), any(), anyBoolean(), any()); + + sampleStreamWrapper.continueLoading( + new LoadingInfo.Builder().setPlaybackPositionUs(1_500_000L).build()); + shadowOf(Looper.getMainLooper()).idle(); + + // Primary track has preroll. + assertThat(sampleStreamWrapper.getStreamFlags(/* trackGroupIndex= */ 0)) + .isEqualTo(SampleStream.FLAG_HAS_PREROLL); + // Unmapped optional track returns 0. + assertThat(sampleStreamWrapper.getStreamFlags(/* trackGroupIndex= */ 1)).isEqualTo(0); + } + + @Test + public void getStreamFlags_syncAudioWithoutCodecsInMultivariantPlaylist_returnsZero() { + // The multivariant playlist rendition has no CODECS attribute, but the extractor reports the + // codecs of the actual samples. + HlsSampleStreamWrapper sampleStreamWrapper = + createSampleStreamWrapper( + /* positionUs= */ 1_500_000L, + new Format.Builder().setSampleMimeType(MimeTypes.AUDIO_AAC).build()); + Format sampleFormat = + new Format.Builder().setSampleMimeType(MimeTypes.AUDIO_AAC).setCodecs("mp4a.40.2").build(); + AtomicReference trackOutput = new AtomicReference<>(); + doAnswer( + invocation -> { + ExtractorOutput extractorOutput = invocation.getArgument(0); + trackOutput.set(extractorOutput.track(/* id= */ 0, C.TRACK_TYPE_AUDIO)); + extractorOutput.endTracks(); + return null; + }) + .when(mockExtractor) + .init(any()); + HlsMediaChunk chunk = + createMediaChunk(/* startTimeUs= */ 1_000_000L, /* durationUs= */ 1_000_000L); + doAnswer( + invocation -> { + HlsChunkSource.HlsChunkHolder chunkHolder = invocation.getArgument(5); + chunkHolder.chunk = chunk; + return null; + }) + .when(mockChunkSource) + .getNextChunk(any(), anyLong(), anyLong(), any(), anyBoolean(), any()); + + sampleStreamWrapper.continueLoading( + new LoadingInfo.Builder().setPlaybackPositionUs(1_500_000L).build()); + shadowOf(Looper.getMainLooper()).idle(); + + // The chunk is loaded, but headers have not been parsed yet (upstreamFormat == null). + assertThat(sampleStreamWrapper.getStreamFlags(/* trackGroupIndex= */ 0)) + .isEqualTo(SampleStream.FLAG_MAYBE_HAS_PREROLL); + + // Headers are parsed and upstreamFormat is populated. + trackOutput.get().format(sampleFormat); + shadowOf(Looper.getMainLooper()).idle(); + + // Sync audio discards all samples before the start time, so there is no preroll even though + // the resumption is mid-segment. + assertThat(sampleStreamWrapper.getStreamFlags(/* trackGroupIndex= */ 0)).isEqualTo(0); + } + + private HlsSampleStreamWrapper createSampleStreamWrapper(long positionUs) { + return createSampleStreamWrapper(positionUs, new Format.Builder().build()); + } + + private HlsSampleStreamWrapper createSampleStreamWrapper(long positionUs, Format format) { + HlsSampleStreamWrapper sampleStreamWrapper = + new HlsSampleStreamWrapper( + /* uid= */ "uid", + C.TRACK_TYPE_VIDEO, + mock(HlsSampleStreamWrapper.Callback.class), + mockChunkSource, + /* overridingDrmInitData= */ ImmutableMap.of(), + mock(Allocator.class), + positionUs, + /* muxedAudioFormat= */ null, + mock(DrmSessionManager.class), + new DrmSessionEventListener.EventDispatcher(), + mock(LoadErrorHandlingPolicy.class), + new MediaSourceEventListener.EventDispatcher(), + /* metadataType= */ HlsMediaSource.METADATA_TYPE_ID3, + /* downloadExecutor= */ ReleasableExecutor.from(directExecutor(), e -> {})); + sampleStreamWrapper.prepareWithMultivariantPlaylistInfo( + new TrackGroup[] {new TrackGroup(format)}, /* primaryTrackGroupIndex= */ 0); + return sampleStreamWrapper; + } + + private HlsMediaChunk createMediaChunk(long startTimeUs, long durationUs) { + HlsMediaPlaylist.Part part = + new HlsMediaPlaylist.Part( + /* url= */ "part1.ts", + /* initializationSegment= */ null, + durationUs, + /* relativeDiscontinuitySequence= */ 0, + startTimeUs, + /* drmInitData= */ null, + /* fullSegmentEncryptionKeyUri= */ null, + /* encryptionIV= */ null, + /* byteRangeOffset= */ 0, + /* byteRangeLength= */ C.LENGTH_UNSET, + /* hasGapTag= */ false, + /* isIndependent= */ true, + /* isPreload= */ false); + HlsMediaPlaylist playlist = + new HlsMediaPlaylist( + HlsMediaPlaylist.PLAYLIST_TYPE_UNKNOWN, + /* baseUri= */ "http://example.com/", + /* tags= */ ImmutableList.of(), + /* startOffsetUs= */ C.TIME_UNSET, + /* preciseStart= */ false, + /* startTimeUs= */ 0L, + /* hasDiscontinuitySequence= */ false, + /* discontinuitySequence= */ 0, + /* mediaSequence= */ 0L, + /* version= */ 7, + /* targetDurationUs= */ 4_000_000L, + /* partTargetDurationUs= */ C.TIME_UNSET, + /* hasIndependentSegments= */ true, + /* hasEndTag= */ false, + /* hasProgramDateTime= */ false, + /* protectionSchemes= */ null, + /* segments= */ ImmutableList.of(), + /* trailingParts= */ ImmutableList.of(part), + new HlsMediaPlaylist.ServerControl( + /* skipUntilUs= */ C.TIME_UNSET, + /* canSkipDateRanges= */ false, + /* holdBackUs= */ C.TIME_UNSET, + /* partHoldBackUs= */ C.TIME_UNSET, + /* canBlockReload= */ false), + /* renditionReports= */ ImmutableMap.of(), + /* interstitials= */ ImmutableList.of(), + /* lastSeenInitSegment= */ null); + FakeDataSource fakeDataSource = new FakeDataSource(); + fakeDataSource.getDataSet().newDefaultData().appendReadData(10).endData(); + return HlsMediaChunk.createInstance( + mockExtractorFactory, + fakeDataSource, + new Format.Builder().build(), + /* startOfPlaylistInPeriodUs= */ 0L, + playlist, + new HlsChunkSource.SegmentBaseHolder( + playlist.trailingParts.get(0), /* mediaSequence= */ 0, /* partIndex= */ 0), + Uri.parse("http://example.com/part1.ts"), + /* steeredPathwayId= */ null, + /* muxedCaptionFormats= */ null, + C.SELECTION_REASON_INITIAL, + /* trackSelectionData= */ null, + /* isPrimaryTimestampSource= */ true, + new TimestampAdjusterProvider(), + /* timestampAdjusterInitializationTimeoutMs= */ 0, + /* previousChunk= */ null, + /* mediaSegmentKey= */ null, + /* initSegmentKey= */ null, + /* shouldSpliceIn= */ false, + /* isIndependent= */ true, + PlayerId.UNSET, + /* cmcdDataFactory= */ null); + } + + @Test + public void onLoadError_whenRetriesExhausted_returnsDontRetryFatalAndReportsCanceled() { + MediaSourceEventListener.EventDispatcher eventDispatcher = + new MediaSourceEventListener.EventDispatcher(); + List wasCanceledEvents = new ArrayList<>(); + eventDispatcher.addEventListener( + createHandlerForCurrentLooper(), + new MediaSourceEventListener() { + @Override + public void onLoadError( + int windowIndex, + @Nullable MediaSource.MediaPeriodId mediaPeriodId, + LoadEventInfo loadEventInfo, + MediaLoadData mediaLoadData, + IOException error, + boolean wasCanceled) { + wasCanceledEvents.add(wasCanceled); + } + }); + HlsSampleStreamWrapper sampleStreamWrapper = + new HlsSampleStreamWrapper( + /* uid= */ "uid", + C.TRACK_TYPE_VIDEO, + mock(HlsSampleStreamWrapper.Callback.class), + mockChunkSource, + /* overridingDrmInitData= */ ImmutableMap.of(), + mock(Allocator.class), + /* positionUs= */ 0L, + /* muxedAudioFormat= */ null, + mock(DrmSessionManager.class), + new DrmSessionEventListener.EventDispatcher(), + new DefaultLoadErrorHandlingPolicy(), + eventDispatcher, + /* metadataType= */ HlsMediaSource.METADATA_TYPE_ID3, + /* downloadExecutor= */ ReleasableExecutor.from(directExecutor(), e -> {})); + Chunk chunk = + new Chunk( + new FakeDataSource(), + new DataSpec(Uri.parse("http://example.com/chunk.ts")), + C.DATA_TYPE_MEDIA, + new Format.Builder().build(), + C.SELECTION_REASON_UNKNOWN, + /* trackSelectionData= */ null, + /* startTimeUs= */ 0, + /* endTimeUs= */ 1_000_000, + /* steeredPathwayId= */ null) { + @Override + public void cancelLoad() {} + + @Override + public void load() {} + }; + when(mockChunkSource.onChunkError(any(), any())).thenReturn(false); + + Loader.LoadErrorAction actionWithinLimit = + sampleStreamWrapper.onLoadError( + chunk, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + new IOException("Transient error"), + /* errorCount= */ 3); + Loader.LoadErrorAction actionExhausted = + sampleStreamWrapper.onLoadError( + chunk, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + new IOException("Transient error"), + /* errorCount= */ 4); + + assertThat(actionWithinLimit.isRetry()).isTrue(); + assertThat(actionExhausted).isEqualTo(Loader.DONT_RETRY_FATAL); + assertThat(wasCanceledEvents).containsExactly(false, true).inOrder(); + } + + @Test + public void onLoadError_unpublishedPreloadPartHttp404_stopsRetryingWhenRetriesExhausted() { + MediaSourceEventListener.EventDispatcher eventDispatcher = + new MediaSourceEventListener.EventDispatcher(); + List wasCanceledEvents = new ArrayList<>(); + eventDispatcher.addEventListener( + createHandlerForCurrentLooper(), + new MediaSourceEventListener() { + @Override + public void onLoadError( + int windowIndex, + @Nullable MediaSource.MediaPeriodId mediaPeriodId, + LoadEventInfo loadEventInfo, + MediaLoadData mediaLoadData, + IOException error, + boolean wasCanceled) { + wasCanceledEvents.add(wasCanceled); + } + }); + HlsSampleStreamWrapper sampleStreamWrapper = + new HlsSampleStreamWrapper( + /* uid= */ "uid", + C.TRACK_TYPE_VIDEO, + mock(HlsSampleStreamWrapper.Callback.class), + mockChunkSource, + /* overridingDrmInitData= */ ImmutableMap.of(), + mock(Allocator.class), + /* positionUs= */ 0L, + /* muxedAudioFormat= */ null, + mock(DrmSessionManager.class), + new DrmSessionEventListener.EventDispatcher(), + new DefaultLoadErrorHandlingPolicy(), + eventDispatcher, + /* metadataType= */ HlsMediaSource.METADATA_TYPE_ID3, + /* downloadExecutor= */ ReleasableExecutor.from(directExecutor(), e -> {})); + HlsMediaPlaylist.Part preloadPart = + new HlsMediaPlaylist.Part( + /* url= */ "preload.ts", + /* initializationSegment= */ null, + /* durationUs= */ 1_000_000L, + /* relativeDiscontinuitySequence= */ 0, + /* relativeStartTimeUs= */ 0L, + /* drmInitData= */ null, + /* fullSegmentEncryptionKeyUri= */ null, + /* encryptionIV= */ null, + /* byteRangeOffset= */ 0, + /* byteRangeLength= */ C.LENGTH_UNSET, + /* hasGapTag= */ false, + /* isIndependent= */ true, + /* isPreload= */ true); + HlsMediaPlaylist playlist = + new HlsMediaPlaylist( + HlsMediaPlaylist.PLAYLIST_TYPE_UNKNOWN, + /* baseUri= */ "http://example.com/", + /* tags= */ ImmutableList.of(), + /* startOffsetUs= */ C.TIME_UNSET, + /* preciseStart= */ false, + /* startTimeUs= */ 0L, + /* hasDiscontinuitySequence= */ false, + /* discontinuitySequence= */ 0, + /* mediaSequence= */ 0L, + /* version= */ 7, + /* targetDurationUs= */ 4_000_000L, + /* partTargetDurationUs= */ C.TIME_UNSET, + /* hasIndependentSegments= */ true, + /* hasEndTag= */ false, + /* hasProgramDateTime= */ false, + /* protectionSchemes= */ null, + /* segments= */ ImmutableList.of(), + /* trailingParts= */ ImmutableList.of(preloadPart), + new HlsMediaPlaylist.ServerControl( + /* skipUntilUs= */ C.TIME_UNSET, + /* canSkipDateRanges= */ false, + /* holdBackUs= */ C.TIME_UNSET, + /* partHoldBackUs= */ C.TIME_UNSET, + /* canBlockReload= */ false), + /* renditionReports= */ ImmutableMap.of(), + /* interstitials= */ ImmutableList.of(), + /* lastSeenInitSegment= */ null); + HlsMediaChunk preloadChunk = + HlsMediaChunk.createInstance( + mockExtractorFactory, + new FakeDataSource(), + new Format.Builder().build(), + /* startOfPlaylistInPeriodUs= */ 0L, + playlist, + new HlsChunkSource.SegmentBaseHolder( + playlist.trailingParts.get(0), /* mediaSequence= */ 0, /* partIndex= */ 0), + Uri.parse("http://example.com/preload.ts"), + /* steeredPathwayId= */ null, + /* muxedCaptionFormats= */ null, + C.SELECTION_REASON_INITIAL, + /* trackSelectionData= */ null, + /* isPrimaryTimestampSource= */ true, + new TimestampAdjusterProvider(), + /* timestampAdjusterInitializationTimeoutMs= */ 0, + /* previousChunk= */ null, + /* mediaSegmentKey= */ null, + /* initSegmentKey= */ null, + /* shouldSpliceIn= */ false, + /* isIndependent= */ true, + PlayerId.UNSET, + /* cmcdDataFactory= */ null); + when(mockChunkSource.onChunkError(any(), any())).thenReturn(false); + when(mockChunkSource.createFallbackOptions(any(Chunk.class))) + .thenReturn( + new LoadErrorHandlingPolicy.FallbackOptions( + /* numberOfLocations= */ 1, + /* numberOfExcludedLocations= */ 0, + /* numberOfTracks= */ 1, + /* numberOfExcludedTracks= */ 0, + /* locationSteeringActive= */ false)); + HttpDataSource.InvalidResponseCodeException http404Exception = + new HttpDataSource.InvalidResponseCodeException( + /* responseCode= */ 404, + /* responseMessage= */ "Not Found", + /* cause= */ null, + ImmutableMap.of(), + new DataSpec(Uri.parse("http://example.com/preload.ts")), + new byte[0]); + + Loader.LoadErrorAction actionWithinLimit = + sampleStreamWrapper.onLoadError( + preloadChunk, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + http404Exception, + /* errorCount= */ 3); + Loader.LoadErrorAction actionExhausted = + sampleStreamWrapper.onLoadError( + preloadChunk, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + http404Exception, + /* errorCount= */ 4); + + assertThat(preloadChunk.isPublished()).isFalse(); + assertThat(actionWithinLimit).isEqualTo(Loader.RETRY); + assertThat(actionExhausted).isEqualTo(Loader.DONT_RETRY_FATAL); + assertThat(wasCanceledEvents).containsExactly(true); + } +} diff --git a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/e2etest/ClippingHlsPlaybackTest.java b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/e2etest/ClippingHlsPlaybackTest.java index 996028cc0bb..f410e15134f 100644 --- a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/e2etest/ClippingHlsPlaybackTest.java +++ b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/e2etest/ClippingHlsPlaybackTest.java @@ -27,6 +27,7 @@ import android.view.Surface; import androidx.annotation.Nullable; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.MediaItem; import androidx.media3.common.Player; import androidx.media3.datasource.DataSource; @@ -41,9 +42,11 @@ import androidx.media3.exoplayer.source.MediaLoadData; import androidx.media3.exoplayer.source.MediaSource; import androidx.media3.exoplayer.source.MediaSourceEventListener; +import androidx.media3.test.utils.BindFlag; import androidx.media3.test.utils.DumpFileAsserts; import androidx.media3.test.utils.Dumper; import androidx.media3.test.utils.FakeClock; +import androidx.media3.test.utils.Media3FlagsRule; import androidx.media3.test.utils.robolectric.CapturingRenderersFactory; import androidx.media3.test.utils.robolectric.PlaybackOutput; import androidx.media3.test.utils.robolectric.ShadowMediaCodecConfig; @@ -66,7 +69,11 @@ public final class ClippingHlsPlaybackTest { private static final String TEST_HLS_URI = "asset:///media/cmaf/multi-segment/playlist.m3u8"; - @TestParameter private boolean enableMediaPeriodClipping; + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); + + @TestParameter + @BindFlag(Flags.FLAG_ENABLE_CLIPPING_IN_MEDIA_PERIOD) + private boolean enableMediaPeriodClipping; @Rule public ShadowMediaCodecConfig mediaCodecConfig = @@ -572,7 +579,6 @@ public MediaSource createMediaSource(MediaItem mediaItem) { return new ClippingMediaSource.Builder(hlsSource) .setStartPositionUs(mediaItem.clippingConfiguration.startPositionUs) .setEndPositionUs(mediaItem.clippingConfiguration.endPositionUs) - .setEnableClippingInMediaPeriod(enableMediaPeriodClipping) .build(); } }) diff --git a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/e2etest/HlsPlaybackTest.java b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/e2etest/HlsPlaybackTest.java index 732b070f9e8..dfc08604268 100644 --- a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/e2etest/HlsPlaybackTest.java +++ b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/e2etest/HlsPlaybackTest.java @@ -31,7 +31,6 @@ import android.net.Uri; import android.view.Surface; import androidx.annotation.Nullable; -import androidx.media3.common.C; import androidx.media3.common.MediaItem; import androidx.media3.common.Player; import androidx.media3.datasource.DefaultDataSource; @@ -45,7 +44,6 @@ import androidx.media3.exoplayer.source.LoadEventInfo; import androidx.media3.exoplayer.source.MediaLoadData; import androidx.media3.exoplayer.upstream.CmcdConfiguration; -import androidx.media3.extractor.DefaultExtractorsFactory; import androidx.media3.test.utils.DumpFileAsserts; import androidx.media3.test.utils.FakeClock; import androidx.media3.test.utils.ThrowingSubtitleParserFactory; @@ -485,13 +483,8 @@ public void playVideo_usingWithinGopSampleDependencies_withSeek() throws Excepti FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); CapturingRenderersFactory capturingRenderersFactory = new CapturingRenderersFactory(applicationContext, clock); - DefaultMediaSourceFactory defaultMediaSourceFactory = - new DefaultMediaSourceFactory(applicationContext, new DefaultExtractorsFactory()); - defaultMediaSourceFactory.experimentalSetCodecsToParseWithinGopSampleDependencies( - C.VIDEO_CODEC_FLAG_H264); ExoPlayer player = - new ExoPlayer.Builder( - applicationContext, capturingRenderersFactory, defaultMediaSourceFactory) + new ExoPlayer.Builder(applicationContext, capturingRenderersFactory) .setClock(clock) .build(); Surface surface = new Surface(new SurfaceTexture(/* texName= */ 1)); diff --git a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/offline/HlsDownloaderTest.java b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/offline/HlsDownloaderTest.java index 207b50ee80f..f59da5233d5 100644 --- a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/offline/HlsDownloaderTest.java +++ b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/offline/HlsDownloaderTest.java @@ -33,6 +33,7 @@ import static androidx.media3.test.utils.CacheAsserts.assertCacheEmpty; import static androidx.media3.test.utils.CacheAsserts.assertCachedData; import static com.google.common.truth.Truth.assertThat; +import static java.nio.charset.StandardCharsets.UTF_8; import android.net.Uri; import androidx.media3.common.C; @@ -290,6 +291,162 @@ public void downloadEncMediaPlaylist() throws Exception { assertCachedData(cache, fakeDataSet); } + @Test + public void download_withVariableSubstitutionInSegmentUrls_resolvesVariables() throws Exception { + // Multivariant playlist defines a variable via EXT-X-DEFINE NAME/VALUE + byte[] multivariantPlaylistData = + ("#EXTM3U\n" + + "#EXT-X-DEFINE:NAME=\"cdnPrefix\",VALUE=\"https://cdn.example.com/\"\n" + + "#EXT-X-STREAM-INF:BANDWIDTH=232370,CODECS=\"mp4a.40.2, avc1.4d4015\"\n" + + "media_with_vars.m3u8\n") + .getBytes(UTF_8); + // Media playlist uses the variable in segment URLs with IMPORT + byte[] mediaPlaylistData = + ("#EXTM3U\n" + + "#EXT-X-TARGETDURATION:10\n" + + "#EXT-X-VERSION:8\n" + + "#EXT-X-MEDIA-SEQUENCE:0\n" + + "#EXT-X-PLAYLIST-TYPE:VOD\n" + + "#EXT-X-DEFINE:IMPORT=\"cdnPrefix\"\n" + + "#EXTINF:9.97667,\n" + + "{$cdnPrefix}segment0.ts\n" + + "#EXTINF:9.97667,\n" + + "{$cdnPrefix}segment1.ts\n" + + "#EXT-X-ENDLIST") + .getBytes(UTF_8); + fakeDataSet = + new FakeDataSet() + .setData("master_vars.m3u8", multivariantPlaylistData) + .setData("media_with_vars.m3u8", mediaPlaylistData) + .setRandomData("https://cdn.example.com/segment0.ts", 10) + .setRandomData("https://cdn.example.com/segment1.ts", 11); + // Create a downloader with CacheDataSource for validating downloaded data. + HlsDownloader downloader = + getHlsDownloader(/* mediaPlaylistUri= */ "master_vars.m3u8", getKeys(0)); + + downloader.download(progressListener); + + // Verify segments were downloaded with resolved URLs + assertCachedData(cache, fakeDataSet); + } + + @Test + public void download_withMultipleVariablesInSegmentUrls_resolvesAllVariables() throws Exception { + // Multivariant playlist defines multiple variables. + byte[] multivariantPlaylistData = + ("#EXTM3U\n" + + "#EXT-X-DEFINE:NAME=\"contentPrefix\",VALUE=\"https://media.example.com/\"\n" + + "#EXT-X-DEFINE:NAME=\"sessionId\",VALUE=\"abc123\"\n" + + "#EXT-X-STREAM-INF:BANDWIDTH=500000,CODECS=\"avc1.4d401f,mp4a.40.2\"\n" + + "{$contentPrefix}video/720p.m3u8?sid={$sessionId}\n") + .getBytes(UTF_8); + // Media playlist imports and uses both variables. + byte[] mediaPlaylistData = + ("#EXTM3U\n" + + "#EXT-X-TARGETDURATION:6\n" + + "#EXT-X-VERSION:8\n" + + "#EXT-X-MEDIA-SEQUENCE:0\n" + + "#EXT-X-PLAYLIST-TYPE:VOD\n" + + "#EXT-X-DEFINE:IMPORT=\"contentPrefix\"\n" + + "#EXT-X-DEFINE:IMPORT=\"sessionId\"\n" + + "#EXTINF:6.0,\n" + + "{$contentPrefix}seg/s0.ts?sid={$sessionId}\n" + + "#EXTINF:6.0,\n" + + "{$contentPrefix}seg/s1.ts?sid={$sessionId}\n" + + "#EXT-X-ENDLIST") + .getBytes(UTF_8); + fakeDataSet = + new FakeDataSet() + .setData("master_multi_vars.m3u8", multivariantPlaylistData) + .setData("https://media.example.com/video/720p.m3u8?sid=abc123", mediaPlaylistData) + .setRandomData("https://media.example.com/seg/s0.ts?sid=abc123", 10) + .setRandomData("https://media.example.com/seg/s1.ts?sid=abc123", 11); + // Create a downloader with CacheDataSource for validating downloaded data. + HlsDownloader downloader = + getHlsDownloader(/* mediaPlaylistUri= */ "master_multi_vars.m3u8", getKeys(0)); + + downloader.download(progressListener); + + // Verify variables have been inserted into URIs correctly. + assertCachedData(cache, fakeDataSet); + } + + @Test + public void download_withVariableInChildManifestUri_resolvesVariables() throws Exception { + // Multivariant playlist uses variable in the child manifest URI itself + byte[] multivariantPlaylistData = + ("#EXTM3U\n" + + "#EXT-X-DEFINE:NAME=\"manifestBase\",VALUE=\"https://manifest.example.com/\"\n" + + "#EXT-X-DEFINE:NAME=\"segmentBase\",VALUE=\"https://segments.example.com/\"\n" + + "#EXT-X-STREAM-INF:BANDWIDTH=128000,CODECS=\"mp4a.40.2\"\n" + + "{$manifestBase}audio/playlist.m3u8\n") + .getBytes(UTF_8); + // Media playlist uses a different variable for segment URLs + byte[] mediaPlaylistData = + ("#EXTM3U\n" + + "#EXT-X-TARGETDURATION:10\n" + + "#EXT-X-VERSION:8\n" + + "#EXT-X-MEDIA-SEQUENCE:0\n" + + "#EXT-X-PLAYLIST-TYPE:VOD\n" + + "#EXT-X-DEFINE:IMPORT=\"segmentBase\"\n" + + "#EXTINF:10.0,\n" + + "{$segmentBase}audio/chunk_0.ts\n" + + "#EXTINF:10.0,\n" + + "{$segmentBase}audio/chunk_1.ts\n" + + "#EXT-X-ENDLIST") + .getBytes(UTF_8); + fakeDataSet = + new FakeDataSet() + .setData("master_child_var.m3u8", multivariantPlaylistData) + .setData("https://manifest.example.com/audio/playlist.m3u8", mediaPlaylistData) + .setRandomData("https://segments.example.com/audio/chunk_0.ts", 10) + .setRandomData("https://segments.example.com/audio/chunk_1.ts", 11); + // Create a downloader with CacheDataSource for validating downloaded data. + HlsDownloader downloader = + getHlsDownloader(/* mediaPlaylistUri= */ "master_child_var.m3u8", getKeys(0)); + + downloader.download(progressListener); + + assertCachedData(cache, fakeDataSet); + } + + @Test + public void download_withVariableInEncryptionKeyUri_resolvesVariables() throws Exception { + // Multivariant playlist defines a variable for the key base URL + byte[] multivariantPlaylistData = + ("#EXTM3U\n" + + "#EXT-X-DEFINE:NAME=\"keyBase\",VALUE=\"https://keys.example.com/\"\n" + + "#EXT-X-STREAM-INF:BANDWIDTH=232370,CODECS=\"mp4a.40.2, avc1.4d4015\"\n" + + "media_with_key_var.m3u8\n") + .getBytes(UTF_8); + // Media playlist imports the variable and uses it in the EXT-X-KEY URI + byte[] mediaPlaylistData = + ("#EXTM3U\n" + + "#EXT-X-TARGETDURATION:10\n" + + "#EXT-X-VERSION:8\n" + + "#EXT-X-MEDIA-SEQUENCE:0\n" + + "#EXT-X-PLAYLIST-TYPE:VOD\n" + + "#EXT-X-DEFINE:IMPORT=\"keyBase\"\n" + + "#EXT-X-KEY:METHOD=AES-128,URI=\"{$keyBase}enc.key\"\n" + + "#EXTINF:9.97667,\n" + + "segment0.ts\n" + + "#EXT-X-ENDLIST") + .getBytes(UTF_8); + fakeDataSet = + new FakeDataSet() + .setData("master_key_var.m3u8", multivariantPlaylistData) + .setData("media_with_key_var.m3u8", mediaPlaylistData) + .setRandomData("https://keys.example.com/enc.key", 8) + .setRandomData("segment0.ts", 10); + // Create a downloader with CacheDataSource for validating downloaded data. + HlsDownloader downloader = + getHlsDownloader(/* mediaPlaylistUri= */ "master_key_var.m3u8", getKeys(0)); + + downloader.download(progressListener); + + assertCachedData(cache, fakeDataSet); + } + private HlsDownloader getHlsDownloader(String mediaPlaylistUri, List keys) { CacheDataSource.Factory cacheDataSourceFactory = new CacheDataSource.Factory() diff --git a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/playlist/DefaultHlsPlaylistTrackerTest.java b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/playlist/DefaultHlsPlaylistTrackerTest.java index 804fe55c348..a6628288eca 100644 --- a/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/playlist/DefaultHlsPlaylistTrackerTest.java +++ b/libraries/exoplayer_hls/src/test/java/androidx/media3/exoplayer/hls/playlist/DefaultHlsPlaylistTrackerTest.java @@ -16,22 +16,38 @@ package androidx.media3.exoplayer.hls.playlist; import static com.google.common.truth.Truth.assertThat; +import static com.google.common.truth.Truth.assertWithMessage; import static com.google.common.util.concurrent.MoreExecutors.directExecutor; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyBoolean; +import static org.mockito.ArgumentMatchers.anyInt; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.times; +import static org.mockito.Mockito.verify; import android.net.Uri; import androidx.annotation.Nullable; +import androidx.media3.common.C; +import androidx.media3.common.util.Util; import androidx.media3.datasource.DataSource; import androidx.media3.datasource.DefaultHttpDataSource; +import androidx.media3.exoplayer.source.LoadEventInfo; +import androidx.media3.exoplayer.source.MediaLoadData; +import androidx.media3.exoplayer.source.MediaSource; import androidx.media3.exoplayer.source.MediaSourceEventListener; import androidx.media3.exoplayer.upstream.BandwidthMeter; import androidx.media3.exoplayer.upstream.DefaultLoadErrorHandlingPolicy; import androidx.media3.exoplayer.upstream.LoadErrorHandlingPolicy; +import androidx.media3.exoplayer.upstream.Loader; +import androidx.media3.exoplayer.upstream.ParsingLoadable; import androidx.media3.exoplayer.util.ReleasableExecutor; import androidx.media3.test.utils.TestUtil; import androidx.media3.test.utils.robolectric.RobolectricUtil; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.base.Supplier; +import com.google.common.collect.ImmutableSet; import java.io.IOException; import java.util.ArrayList; import java.util.List; @@ -46,6 +62,7 @@ import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; +import org.mockito.ArgumentCaptor; /** Unit test for {@link DefaultHlsPlaylistTracker}. */ @RunWith(AndroidJUnit4.class) @@ -584,6 +601,75 @@ public boolean onPlaylistError( assertThat(mediaPlaylists.get(1).trailingParts).hasSize(2); } + @Test + public void start_refreshScheduledButNotExecutedForNonPlayingPlaylist() throws Exception { + List httpUrls = + enqueueWebServerResponses( + new String[] { + "/multivariant.m3u8", + "/media0/playlist.m3u8", + "/media1/playlist.m3u8", + "/media1/playlist.m3u8", + }, + getMockResponse(SAMPLE_M3U8_LIVE_MULTIVARIANT), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_NOT_SKIP), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_NOT_SKIP), + getMockResponse(SAMPLE_M3U8_LIVE_MEDIA_CAN_NOT_SKIP_NEXT)); + DefaultHlsPlaylistTracker defaultHlsPlaylistTracker = + new DefaultHlsPlaylistTracker( + dataType -> new DefaultHttpDataSource.Factory().createDataSource(), + new DefaultLoadErrorHandlingPolicy(), + new DefaultHlsPlaylistParserFactory(), + /* cmcdConfiguration= */ null, + /* downloadExecutorSupplier= */ null); + AtomicInteger playlistChangedCounter = new AtomicInteger(); + defaultHlsPlaylistTracker.addListener( + new HlsPlaylistTracker.PlaylistEventListener() { + @Override + public void onPlaylistChanged() { + playlistChangedCounter.addAndGet(1); + } + + @Override + public boolean onPlaylistError( + Uri url, LoadErrorHandlingPolicy.LoadErrorInfo loadErrorInfo, boolean forceRetry) { + return false; + } + }); + + defaultHlsPlaylistTracker.start( + Uri.parse(mockWebServer.url("/multivariant.m3u8").toString()), + new MediaSourceEventListener.EventDispatcher(), + mediaPlaylist -> {}, + BandwidthMeter.NO_OP); + // Wait for playlist A (media0) to load. Since A is the initial primary playlist, it will + // schedule a refresh with a 4-second delay (based on target duration) once it finishes loading. + Uri playlistUrlA = Uri.parse(mockWebServer.url("/media0/playlist.m3u8").toString()); + RobolectricUtil.runMainLooperUntil( + () -> defaultHlsPlaylistTracker.isSnapshotValid(playlistUrlA)); + // Explicitly activate playlist A for playback. + defaultHlsPlaylistTracker.getPlaylistSnapshot(playlistUrlA, /* isForPlayback= */ true); + // Refresh B (media1) and wait for it to load. + Uri playlistUrlB = Uri.parse(mockWebServer.url("/media1/playlist.m3u8").toString()); + defaultHlsPlaylistTracker.refreshPlaylist(playlistUrlB); + RobolectricUtil.runMainLooperUntil( + () -> defaultHlsPlaylistTracker.isSnapshotValid(playlistUrlB)); + // Make playlist B primary and active. + defaultHlsPlaylistTracker.getPlaylistSnapshot(playlistUrlB, /* isForPlayback= */ true); + // Explicitly deactivate playlist A for playback, simulating the track change. The Playlist A + // is now non-primary and inactive, but its refresh task is still pending in the looper queue + // with the original 4-second delay. + defaultHlsPlaylistTracker.deactivatePlaylistForPlayback(playlistUrlA); + // Keep running the looper until the scheduled refresh tasks got run. Both A and B's refresh + // tasks will run by the looper. But A is no longer primary or active, so its task should skip + // the actual loading. + RobolectricUtil.runMainLooperUntil( + /* maxTimeDiffMs= */ 10_000, () -> playlistChangedCounter.get() >= 3); + defaultHlsPlaylistTracker.stop(); + + assertRequestUrlsCalled(httpUrls); + } + @Test public void start_refreshPlaylistWithAllowingDeliveryDirectives_requestWithCorrectDeliveryDirectives() @@ -802,18 +888,54 @@ public void start_withContentSteering_switchesToMostPrioritizedPathway() throws new MockResponse().setResponseCode(200).setBody(steeringManifest), new MockResponse().setResponseCode(200).setBody(CDN_A_PLAYLIST), new MockResponse().setResponseCode(200).setBody(CDN_B_PLAYLIST)); + MediaSourceEventListener mockListener = mock(MediaSourceEventListener.class); + MediaSourceEventListener.EventDispatcher eventDispatcher = + new MediaSourceEventListener.EventDispatcher(); + eventDispatcher.addEventListener(Util.createHandlerForCurrentLooper(), mockListener); // Use the directExecutor() to ensure the order of the playlist arrivals. List mediaPlaylists = runPlaylistTrackerAndCollectMediaPlaylists( /* dataSourceFactory= */ new DefaultHttpDataSource.Factory(), () -> ReleasableExecutor.from(directExecutor(), e -> {}), + eventDispatcher, Uri.parse(mockWebServer.url("/multivariant.m3u8").toString()), /* awaitedMediaPlaylistCount= */ 2); assertRequestUrlsCalled(httpUrls); assertThat(mediaPlaylists.get(0).segments.get(0).url).endsWith("a-segment1.ts"); assertThat(mediaPlaylists.get(1).segments.get(0).url).endsWith("b-segment1.ts"); + ArgumentCaptor loadStartedCaptor = ArgumentCaptor.forClass(LoadEventInfo.class); + ArgumentCaptor loadStartedMediaLoadDataCaptor = + ArgumentCaptor.forClass(MediaLoadData.class); + ArgumentCaptor loadCompletedCaptor = + ArgumentCaptor.forClass(LoadEventInfo.class); + ArgumentCaptor loadCompletedMediaLoadDataCaptor = + ArgumentCaptor.forClass(MediaLoadData.class); + verify(mockListener, times(4)) + .onLoadStarted( + anyInt(), + any(), + loadStartedCaptor.capture(), + loadStartedMediaLoadDataCaptor.capture(), + anyInt()); + verify(mockListener, times(4)) + .onLoadCompleted( + anyInt(), + any(), + loadCompletedCaptor.capture(), + loadCompletedMediaLoadDataCaptor.capture()); + verify(mockListener, never()).onLoadError(anyInt(), any(), any(), any(), any(), anyBoolean()); + verifySteeredPathwayIds( + loadStartedCaptor.getAllValues(), + loadStartedMediaLoadDataCaptor.getAllValues(), + "CDN-A", + "CDN-B"); + verifySteeredPathwayIds( + loadCompletedCaptor.getAllValues(), + loadCompletedMediaLoadDataCaptor.getAllValues(), + "CDN-A", + "CDN-B"); } @Test @@ -836,17 +958,64 @@ public void start_withContentSteering_switchesToMostPrioritizedPathway() throws new MockResponse().setResponseCode(404), new MockResponse().setResponseCode(404), new MockResponse().setResponseCode(200).setBody(CDN_B_PLAYLIST)); + MediaSourceEventListener mockListener = mock(MediaSourceEventListener.class); + MediaSourceEventListener.EventDispatcher eventDispatcher = + new MediaSourceEventListener.EventDispatcher(); + eventDispatcher.addEventListener(Util.createHandlerForCurrentLooper(), mockListener); // Use the directExecutor() to ensure the order of the playlist arrivals. List mediaPlaylists = runPlaylistTrackerAndCollectMediaPlaylists( /* dataSourceFactory= */ new DefaultHttpDataSource.Factory(), () -> ReleasableExecutor.from(directExecutor(), e -> {}), + eventDispatcher, Uri.parse(mockWebServer.url("/multivariant.m3u8").toString()), /* awaitedMediaPlaylistCount= */ 1); assertRequestUrlsCalled(httpUrls); assertThat(mediaPlaylists.get(0).segments.get(0).url).isEqualTo("b-segment1.ts"); + ArgumentCaptor loadStartedCaptor = ArgumentCaptor.forClass(LoadEventInfo.class); + ArgumentCaptor loadStartedMediaLoadDataCaptor = + ArgumentCaptor.forClass(MediaLoadData.class); + ArgumentCaptor loadCompletedCaptor = + ArgumentCaptor.forClass(LoadEventInfo.class); + ArgumentCaptor loadCompletedMediaLoadDataCaptor = + ArgumentCaptor.forClass(MediaLoadData.class); + ArgumentCaptor loadErrorCaptor = ArgumentCaptor.forClass(LoadEventInfo.class); + ArgumentCaptor loadErrorMediaLoadDataCaptor = + ArgumentCaptor.forClass(MediaLoadData.class); + verify(mockListener, times(5)) + .onLoadStarted( + anyInt(), + any(), + loadStartedCaptor.capture(), + loadStartedMediaLoadDataCaptor.capture(), + anyInt()); + verify(mockListener, times(3)) + .onLoadCompleted( + anyInt(), + any(), + loadCompletedCaptor.capture(), + loadCompletedMediaLoadDataCaptor.capture()); + verify(mockListener, times(2)) + .onLoadError( + anyInt(), + any(), + loadErrorCaptor.capture(), + loadErrorMediaLoadDataCaptor.capture(), + any(), + anyBoolean()); + verifySteeredPathwayIds( + loadStartedCaptor.getAllValues(), + loadStartedMediaLoadDataCaptor.getAllValues(), + "CDN-A", + "CDN-B"); + verifySteeredPathwayIds( + loadCompletedCaptor.getAllValues(), + loadCompletedMediaLoadDataCaptor.getAllValues(), + "CDN-B"); + verifySteeredPathwayIds( + loadErrorCaptor.getAllValues(), loadErrorMediaLoadDataCaptor.getAllValues(), "CDN-A"); } @Test @@ -880,18 +1049,131 @@ public void start_withContentSteeringAndPathwayClones_switchesToClonedPathway() new MockResponse().setResponseCode(200).setBody(steeringManifest), new MockResponse().setResponseCode(200).setBody(CDN_A_PLAYLIST), new MockResponse().setResponseCode(200).setBody(CDN_A_CLONE_PLAYLIST)); + MediaSourceEventListener mockListener = mock(MediaSourceEventListener.class); + MediaSourceEventListener.EventDispatcher eventDispatcher = + new MediaSourceEventListener.EventDispatcher(); + eventDispatcher.addEventListener(Util.createHandlerForCurrentLooper(), mockListener); // Use the directExecutor() to ensure the order of the playlist arrivals. List mediaPlaylists = runPlaylistTrackerAndCollectMediaPlaylists( /* dataSourceFactory= */ new DefaultHttpDataSource.Factory(), () -> ReleasableExecutor.from(directExecutor(), e -> {}), + eventDispatcher, Uri.parse(mockWebServer.url("/multivariant.m3u8").toString()), /* awaitedMediaPlaylistCount= */ 2); assertRequestUrlsCalled(httpUrls); assertThat(mediaPlaylists.get(0).segments.get(0).url).endsWith("a-segment1.ts"); assertThat(mediaPlaylists.get(1).segments.get(0).url).endsWith("a-clone-segment1.ts"); + ArgumentCaptor loadStartedCaptor = ArgumentCaptor.forClass(LoadEventInfo.class); + ArgumentCaptor loadStartedMediaLoadDataCaptor = + ArgumentCaptor.forClass(MediaLoadData.class); + ArgumentCaptor loadCompletedCaptor = + ArgumentCaptor.forClass(LoadEventInfo.class); + ArgumentCaptor loadCompletedMediaLoadDataCaptor = + ArgumentCaptor.forClass(MediaLoadData.class); + verify(mockListener, times(4)) + .onLoadStarted( + anyInt(), + any(), + loadStartedCaptor.capture(), + loadStartedMediaLoadDataCaptor.capture(), + anyInt()); + verify(mockListener, times(4)) + .onLoadCompleted( + anyInt(), + any(), + loadCompletedCaptor.capture(), + loadCompletedMediaLoadDataCaptor.capture()); + verify(mockListener, never()).onLoadError(anyInt(), any(), any(), any(), any(), anyBoolean()); + verifySteeredPathwayIds( + loadStartedCaptor.getAllValues(), + loadStartedMediaLoadDataCaptor.getAllValues(), + "CDN-A", + "CDN-A-CLONE"); + verifySteeredPathwayIds( + loadCompletedCaptor.getAllValues(), + loadCompletedMediaLoadDataCaptor.getAllValues(), + "CDN-A", + "CDN-A-CLONE"); + } + + @Test + public void onLoadError_whenRetriesExhausted_reportsCanceledTrue() throws Exception { + mockWebServer.enqueue( + new MockResponse() + .setResponseCode(200) + .setBody( + new Buffer() + .writeUtf8("#EXTM3U\n#EXT-X-STREAM-INF:BANDWIDTH=1280000\nmedia.m3u8\n"))); + mockWebServer.enqueue(new MockResponse().setResponseCode(500)); + mockWebServer.enqueue(new MockResponse().setResponseCode(500)); + mockWebServer.enqueue(new MockResponse().setResponseCode(500)); + mockWebServer.enqueue(new MockResponse().setResponseCode(500)); + DefaultHlsPlaylistTracker defaultHlsPlaylistTracker = + new DefaultHlsPlaylistTracker( + dataType -> new DefaultHttpDataSource.Factory().createDataSource(), + new DefaultLoadErrorHandlingPolicy(/* minimumLoadableRetryCount= */ 3) { + @Override + public long getRetryDelayMsFor(LoadErrorInfo loadErrorInfo) { + return 0; + } + }, + new DefaultHlsPlaylistParserFactory(), + /* cmcdConfiguration= */ null, + /* downloadExecutorSupplier= */ () -> + ReleasableExecutor.from(directExecutor(), e -> {})); + MediaSourceEventListener.EventDispatcher eventDispatcher = + new MediaSourceEventListener.EventDispatcher(); + List wasCanceledEvents = new ArrayList<>(); + eventDispatcher.addEventListener( + Util.createHandlerForCurrentLooper(), + new MediaSourceEventListener() { + @Override + public void onLoadError( + int windowIndex, + @Nullable MediaSource.MediaPeriodId mediaPeriodId, + LoadEventInfo loadEventInfo, + MediaLoadData mediaLoadData, + IOException error, + boolean wasCanceled) { + wasCanceledEvents.add(wasCanceled); + } + }); + ParsingLoadable multivariantLoadable = + new ParsingLoadable.Builder<>( + new DefaultHttpDataSource.Factory().createDataSource(), + Uri.parse("http://example.com/multivariant.m3u8"), + C.DATA_TYPE_MANIFEST, + new HlsPlaylistParser()) + .build(); + + defaultHlsPlaylistTracker.start( + Uri.parse(mockWebServer.url("/multivariant.m3u8").toString()), + eventDispatcher, + mediaPlaylist -> {}, + BandwidthMeter.NO_OP); + RobolectricUtil.runMainLooperUntil(() -> wasCanceledEvents.contains(true)); + Loader.LoadErrorAction actionWithinLimit = + defaultHlsPlaylistTracker.onLoadError( + multivariantLoadable, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + new IOException("Transient error"), + /* errorCount= */ 3); + Loader.LoadErrorAction actionExhausted = + defaultHlsPlaylistTracker.onLoadError( + multivariantLoadable, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + new IOException("Transient error"), + /* errorCount= */ 4); + defaultHlsPlaylistTracker.stop(); + + assertThat(actionWithinLimit.isRetry()).isTrue(); + assertThat(actionExhausted).isEqualTo(Loader.DONT_RETRY_FATAL); + assertThat(wasCanceledEvents).containsExactly(false, false, false, true, false, true).inOrder(); } private List enqueueWebServerResponses(String[] paths, MockResponse... mockResponses) { @@ -920,6 +1202,21 @@ private static List runPlaylistTrackerAndCollectMediaPlaylists Uri multivariantPlaylistUri, int awaitedMediaPlaylistCount) throws TimeoutException { + return runPlaylistTrackerAndCollectMediaPlaylists( + dataSourceFactory, + downloadExecutorSupplier, + new MediaSourceEventListener.EventDispatcher(), + multivariantPlaylistUri, + awaitedMediaPlaylistCount); + } + + private static List runPlaylistTrackerAndCollectMediaPlaylists( + DataSource.Factory dataSourceFactory, + @Nullable Supplier downloadExecutorSupplier, + MediaSourceEventListener.EventDispatcher eventDispatcher, + Uri multivariantPlaylistUri, + int awaitedMediaPlaylistCount) + throws TimeoutException { DefaultHlsPlaylistTracker defaultHlsPlaylistTracker = new DefaultHlsPlaylistTracker( @@ -933,7 +1230,7 @@ private static List runPlaylistTrackerAndCollectMediaPlaylists AtomicInteger playlistCounter = new AtomicInteger(); defaultHlsPlaylistTracker.start( multivariantPlaylistUri, - new MediaSourceEventListener.EventDispatcher(), + eventDispatcher, mediaPlaylist -> { mediaPlaylists.add(mediaPlaylist); playlistCounter.addAndGet(1); @@ -948,6 +1245,40 @@ private static List runPlaylistTrackerAndCollectMediaPlaylists return mediaPlaylists; } + private static void verifySteeredPathwayIds( + List loadEventInfos, + List mediaLoadDatas, + String... expectedPathwayIds) { + ImmutableSet.Builder mediaPlaylistLoadingSteeredPathwayIds = ImmutableSet.builder(); + for (int i = 0; i < loadEventInfos.size(); i++) { + LoadEventInfo eventInfo = loadEventInfos.get(i); + MediaLoadData mediaLoadData = mediaLoadDatas.get(i); + if (eventInfo.uri.getPath() != null + && eventInfo.uri.getPath().endsWith("multivariant.m3u8")) { + // The multivariant playlist load event is initiated before the content steering manifest + // is parsed, so its steered pathway ID must be null. + assertWithMessage("loadEventInfos[%s]: %s", i, loadEventInfoString(eventInfo)) + .that(eventInfo.steeredPathwayId) + .isNull(); + } else if (mediaLoadData.dataType == C.DATA_TYPE_STEERING_MANIFEST) { + // Content steering manifest load events are decoupled from any specific steered pathway, + // so their steered pathway ID must be null. + assertWithMessage("loadEventInfos[%s]: %s", i, loadEventInfoString(eventInfo)) + .that(eventInfo.steeredPathwayId) + .isNull(); + } else if (eventInfo.steeredPathwayId != null) { + // Collect the steered pathway IDs from the media playlist requests. + mediaPlaylistLoadingSteeredPathwayIds.add(eventInfo.steeredPathwayId); + } + } + assertThat(mediaPlaylistLoadingSteeredPathwayIds.build()) + .containsExactlyElementsIn(expectedPathwayIds); + } + + private static String loadEventInfoString(LoadEventInfo loadEventInfo) { + return "LoadEventInfo{uri=" + loadEventInfo.uri + "}"; + } + private static MockResponse getMockResponse(String assetFile) throws IOException { return new MockResponse().setResponseCode(200).setBody(new Buffer().write(getBytes(assetFile))); } diff --git a/libraries/exoplayer_ima/src/main/java/androidx/media3/exoplayer/ima/AdTagLoader.java b/libraries/exoplayer_ima/src/main/java/androidx/media3/exoplayer/ima/AdTagLoader.java index 427b8e960f0..f828c83e15e 100644 --- a/libraries/exoplayer_ima/src/main/java/androidx/media3/exoplayer/ima/AdTagLoader.java +++ b/libraries/exoplayer_ima/src/main/java/androidx/media3/exoplayer/ima/AdTagLoader.java @@ -119,13 +119,13 @@ private static final int IMA_AD_STATE_NONE = 0; /** - * The ad playback state when IMA has called {@link ComponentListener#playAd(AdMediaInfo)} and not - * {@link ComponentListener##pauseAd(AdMediaInfo)}. + * The ad playback state when IMA has called {@link VideoAdPlayer#playAd(AdMediaInfo)} and not + * {@link VideoAdPlayer#pauseAd(AdMediaInfo)}. */ private static final int IMA_AD_STATE_PLAYING = 1; /** - * The ad playback state when IMA has called {@link ComponentListener#pauseAd(AdMediaInfo)} while + * The ad playback state when IMA has called {@link VideoAdPlayer#pauseAd(AdMediaInfo)} while * playing an ad. */ private static final int IMA_AD_STATE_PAUSED = 2; @@ -201,9 +201,9 @@ /** * If a content period has finished but IMA has not yet called {@link - * ComponentListener#playAd(AdMediaInfo)}, stores the value of {@link - * SystemClock#elapsedRealtime()} when the content stopped playing. This can be used to determine - * a fake, increasing content position. {@link C#TIME_UNSET} otherwise. + * VideoAdPlayer#playAd(AdMediaInfo)}, stores the value of {@link SystemClock#elapsedRealtime()} + * when the content stopped playing. This can be used to determine a fake, increasing content + * position. {@link C#TIME_UNSET} otherwise. */ private long fakeContentProgressElapsedRealtimeMs; @@ -217,7 +217,7 @@ private long pendingContentPositionMs; /** - * Whether {@link ComponentListener#getContentProgress()} has sent {@link + * Whether {@link ContentProgressProvider#getContentProgress()} has sent {@link * #pendingContentPositionMs} to IMA. */ private boolean sentPendingContentPositionMs; @@ -498,7 +498,6 @@ public void onTimelineChanged(Timeline timeline, @Player.TimelineChangeReason in window.firstPeriodIndex == window.lastPeriodIndex, "Only single-period Timelines are supported."); long contentDurationUs = timeline.getPeriod(window.firstPeriodIndex, period).durationUs; - contentDurationMs = Util.usToMs(contentDurationUs); if (contentDurationUs != adPlaybackState.contentDurationUs) { adPlaybackState = adPlaybackState.withContentDurationUs(contentDurationUs); diff --git a/libraries/exoplayer_ima/src/main/java/androidx/media3/exoplayer/ima/ImaServerSideAdInsertionMediaSource.java b/libraries/exoplayer_ima/src/main/java/androidx/media3/exoplayer/ima/ImaServerSideAdInsertionMediaSource.java index e121aacb0b8..34ce28a4e08 100644 --- a/libraries/exoplayer_ima/src/main/java/androidx/media3/exoplayer/ima/ImaServerSideAdInsertionMediaSource.java +++ b/libraries/exoplayer_ima/src/main/java/androidx/media3/exoplayer/ima/ImaServerSideAdInsertionMediaSource.java @@ -1091,6 +1091,18 @@ && isCurrentlyPlayingMediaPeriodFromThisSource(player, getMediaItem(), adsId)) { } } + @Override + public void onIsPlayingChanged(boolean isPlaying) { + if (!isCurrentlyPlayingMediaPeriodFromThisSource(player, getMediaItem(), adsId)) { + return; + } + if (isPlaying) { + streamPlayer.onContentResumed(); + } else { + streamPlayer.onContentPaused(); + } + } + @Override public void onVolumeChanged(float volume) { if (!isCurrentlyPlayingMediaPeriodFromThisSource(player, getMediaItem(), adsId)) { @@ -1330,6 +1342,20 @@ public void onContentCompleted() { } } + /** Called when the content has paused playback. */ + void onContentPaused() { + for (VideoStreamPlayer.VideoStreamPlayerCallback callback : callbacks) { + callback.onPause(); + } + } + + /** Called when the content has resumed playback. */ + void onContentResumed() { + for (VideoStreamPlayer.VideoStreamPlayerCallback callback : callbacks) { + callback.onResume(); + } + } + /** Called when the content player changed the volume. */ public void onContentVolumeChanged(int volumePct) { for (VideoStreamPlayer.VideoStreamPlayerCallback callback : callbacks) { @@ -1393,7 +1419,7 @@ public VideoProgressUpdate getContentProgress() { usToMs( ServerSideAdInsertionUtil.getStreamPositionUs( player, checkNotNull(adPlaybackState.adsId))); - if (window.windowStartTimeMs != C.TIME_UNSET) { + if (window.isLive() && window.windowStartTimeMs != C.TIME_UNSET) { // Add the time since epoch at start of the window for live streams. streamPositionMs += window.windowStartTimeMs + period.getPositionInWindowMs(); } else if (currentPeriodIndex > window.firstPeriodIndex) { diff --git a/libraries/exoplayer_ima/src/main/java/androidx/media3/exoplayer/ima/ImaUtil.java b/libraries/exoplayer_ima/src/main/java/androidx/media3/exoplayer/ima/ImaUtil.java index 81bb206cd4d..7b4eb045b6c 100644 --- a/libraries/exoplayer_ima/src/main/java/androidx/media3/exoplayer/ima/ImaUtil.java +++ b/libraries/exoplayer_ima/src/main/java/androidx/media3/exoplayer/ima/ImaUtil.java @@ -40,11 +40,13 @@ import androidx.media3.common.AdPlaybackState.AdGroup; import androidx.media3.common.AdViewProvider; import androidx.media3.common.C; +import androidx.media3.common.Player; import androidx.media3.common.Timeline; import androidx.media3.common.util.Util; import androidx.media3.datasource.DataSchemeDataSource; import androidx.media3.datasource.DataSourceUtil; import androidx.media3.datasource.DataSpec; +import com.google.ads.interactivemedia.v3.api.Ad; import com.google.ads.interactivemedia.v3.api.AdDisplayContainer; import com.google.ads.interactivemedia.v3.api.AdError; import com.google.ads.interactivemedia.v3.api.AdErrorEvent; @@ -52,6 +54,7 @@ import com.google.ads.interactivemedia.v3.api.AdPodInfo; import com.google.ads.interactivemedia.v3.api.AdSlot; import com.google.ads.interactivemedia.v3.api.AdsLoader; +import com.google.ads.interactivemedia.v3.api.AdsManager; import com.google.ads.interactivemedia.v3.api.AdsRenderingSettings; import com.google.ads.interactivemedia.v3.api.AdsRequest; import com.google.ads.interactivemedia.v3.api.CompanionAdSlot; diff --git a/libraries/exoplayer_ima/src/test/java/androidx/media3/exoplayer/ima/ImaServerSideAdInsertionMediaSourceTest.java b/libraries/exoplayer_ima/src/test/java/androidx/media3/exoplayer/ima/ImaServerSideAdInsertionMediaSourceTest.java index 7820fec8288..cfb3153b3e8 100644 --- a/libraries/exoplayer_ima/src/test/java/androidx/media3/exoplayer/ima/ImaServerSideAdInsertionMediaSourceTest.java +++ b/libraries/exoplayer_ima/src/test/java/androidx/media3/exoplayer/ima/ImaServerSideAdInsertionMediaSourceTest.java @@ -15,6 +15,7 @@ */ package androidx.media3.exoplayer.ima; +import static androidx.media3.test.utils.robolectric.RobolectricUtil.runMainLooperUntil; import static androidx.media3.test.utils.robolectric.TestPlayerRunHelper.advance; import static com.google.common.truth.Truth.assertThat; import static org.mockito.ArgumentMatchers.any; @@ -27,11 +28,19 @@ import androidx.media3.common.AdPlaybackState; import androidx.media3.common.C; import androidx.media3.common.MediaItem; +import androidx.media3.common.Player; +import androidx.media3.common.Timeline; import androidx.media3.exoplayer.ExoPlayer; +import androidx.media3.exoplayer.drm.DrmSessionManagerProvider; import androidx.media3.exoplayer.ima.ImaServerSideAdInsertionMediaSource.AdsLoader.State; -import androidx.media3.exoplayer.source.DefaultMediaSourceFactory; +import androidx.media3.exoplayer.source.ForwardingTimeline; import androidx.media3.exoplayer.source.MediaSource; import androidx.media3.exoplayer.source.ads.ServerSideAdInsertionUtil; +import androidx.media3.exoplayer.upstream.LoadErrorHandlingPolicy; +import androidx.media3.test.utils.FakeMediaSource; +import androidx.media3.test.utils.FakeMediaSourceFactory; +import androidx.media3.test.utils.FakeTimeline; +import androidx.media3.test.utils.FakeTimeline.TimelineWindowDefinition; import androidx.media3.test.utils.TestExoPlayerBuilder; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; @@ -42,9 +51,14 @@ import com.google.ads.interactivemedia.v3.api.ImaSdkFactory; import com.google.ads.interactivemedia.v3.api.StreamDisplayContainer; import com.google.ads.interactivemedia.v3.api.StreamManager; +import com.google.ads.interactivemedia.v3.api.player.VideoProgressUpdate; +import com.google.ads.interactivemedia.v3.api.player.VideoStreamPlayer; +import com.google.ads.interactivemedia.v3.api.player.VideoStreamPlayer.VideoStreamPlayerCallback; +import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableMap; import java.util.ArrayList; import java.util.List; +import java.util.concurrent.atomic.AtomicReference; import org.junit.After; import org.junit.Before; import org.junit.Rule; @@ -67,6 +81,7 @@ public class ImaServerSideAdInsertionMediaSourceTest { @Mock private AdsRenderingSettings mockAdsRenderingSettings; @Mock private AdsManagerLoadedEvent mockAdsManagerLoadedEvent; @Mock private StreamManager mockStreamManager; + @Mock private VideoStreamPlayerCallback mockStreamPlayerCallback; private ExoPlayer player; private ImaServerSideAdInsertionMediaSource.AdsLoader adsLoader; @@ -83,8 +98,7 @@ public void setUp() { .build(); adsLoader.setPlayer(player); factory = - new ImaServerSideAdInsertionMediaSource.Factory( - adsLoader, new DefaultMediaSourceFactory(context)); + new ImaServerSideAdInsertionMediaSource.Factory(adsLoader, new FakeMediaSourceFactory()); factory.setImaSdkFactory(mockImaFactory); mediaSource = factory.createMediaSource( @@ -175,8 +189,7 @@ public void prepare_withEnableCustomTabsInAdsLoader_enablesCustomTabsInAdsRender .build(); adsLoader.setPlayer(player); ImaServerSideAdInsertionMediaSource.Factory factory = - new ImaServerSideAdInsertionMediaSource.Factory( - adsLoader, new DefaultMediaSourceFactory(context)); + new ImaServerSideAdInsertionMediaSource.Factory(adsLoader, new FakeMediaSourceFactory()); factory.setImaSdkFactory(mockImaFactory); MediaSource mediaSource = factory.createMediaSource( @@ -190,9 +203,141 @@ public void prepare_withEnableCustomTabsInAdsLoader_enablesCustomTabsInAdsRender verify(mockAdsRenderingSettings).setEnableCustomTabs(true); } + @Test + public void + getContentProgress_multiPeriodVodWithWindowStartTime_returnsPositionWithoutWindowStartTime() + throws Exception { + AtomicReference videoStreamPlayerRef = new AtomicReference<>(); + when(mockImaFactory.createAdsLoader(any(), any(), any(StreamDisplayContainer.class))) + .thenAnswer( + invocation -> { + StreamDisplayContainer container = invocation.getArgument(2); + videoStreamPlayerRef.set(container.getVideoStreamPlayer()); + return mockAdsLoader; + }); + when(mockAdsLoader.requestStream(any())) + .thenAnswer( + invocation -> { + videoStreamPlayerRef + .get() + .loadUrl("https://example.com/manifest.mpd", ImmutableList.of()); + return null; + }); + AdPlaybackState adPlaybackState = + ServerSideAdInsertionUtil.addAdGroupToAdPlaybackState( + new AdPlaybackState("2"), + /* fromPositionUs= */ 0, + /* contentResumeOffsetUs= */ 0, + /* adDurationsUs...= */ 10_000_000L) + .withPlayedAd(/* adGroupIndex= */ 0, /* adIndexInAdGroup= */ 0); + ImaServerSideAdInsertionMediaSource.AdsLoader adsLoader = + new ImaServerSideAdInsertionMediaSource.AdsLoader.Builder( + context, /* adViewProvider= */ () -> new LinearLayout(context)) + .setAdsLoaderState(new State(ImmutableMap.of("2", adPlaybackState))) + .build(); + adsLoader.setPlayer(player); + Timeline contentTimeline = + new ForwardingTimeline( + new FakeTimeline( + new TimelineWindowDefinition.Builder() + .setPeriodCount(2) + .setDurationUs(20_000_000L) + .setWindowPositionInFirstPeriodUs(0) + .build())) { + @Override + public Window getWindow( + int windowIndex, Window window, long defaultPositionProjectionUs) { + super.getWindow(windowIndex, window, defaultPositionProjectionUs); + window.presentationStartTimeMs = 1_000_000L; + window.windowStartTimeMs = 1_000_000L; + return window; + } + }; + MediaSource.Factory contentMediaSourceFactory = + new MediaSource.Factory() { + @Override + public MediaSource.Factory setDrmSessionManagerProvider( + DrmSessionManagerProvider drmSessionManagerProvider) { + return this; + } + + @Override + public MediaSource.Factory setLoadErrorHandlingPolicy( + LoadErrorHandlingPolicy loadErrorHandlingPolicy) { + return this; + } + + @Override + public @C.ContentType int[] getSupportedTypes() { + return new int[] {C.CONTENT_TYPE_DASH}; + } + + @Override + public MediaSource createMediaSource(MediaItem mediaItem) { + return new FakeMediaSource(contentTimeline); + } + }; + ImaServerSideAdInsertionMediaSource.Factory factory = + new ImaServerSideAdInsertionMediaSource.Factory(adsLoader, contentMediaSourceFactory); + factory.setImaSdkFactory(mockImaFactory); + MediaSource mediaSource = + factory.createMediaSource( + MediaItem.fromUri( + new ImaServerSideAdInsertionUriBuilder() + .setContentSourceId("123") + .setVideoId("456") + .setFormat(C.CONTENT_TYPE_DASH) + .setAdsId("2") + .build())); + player.setMediaSource(mediaSource); + player.seekTo(/* positionMs= */ 2_000); + player.prepare(); + runMainLooperUntil(() -> !player.getCurrentTimeline().isEmpty()); + advance(player).untilState(Player.STATE_READY); + + VideoProgressUpdate contentProgress = videoStreamPlayerRef.get().getContentProgress(); + + assertThat(contentProgress).isEqualTo(new VideoProgressUpdate(12_000, 20_000)); + } + + @Test + public void play_withMediaSource_notifiesStreamPlayerCallbackOnResume() throws Exception { + player.setMediaSource(mediaSource); + player.prepare(); + advance(player).untilState(Player.STATE_READY); + + player.play(); + advance(player).untilPendingCommandsAreFullyHandled(); + + verify(mockStreamPlayerCallback).onResume(); + } + + @Test + public void pause_whilePlaying_notifiesStreamPlayerCallbackOnPause() throws Exception { + player.setMediaSource(mediaSource); + player.prepare(); + advance(player).untilState(Player.STATE_READY); + player.play(); + advance(player).untilPendingCommandsAreFullyHandled(); + + player.pause(); + advance(player).untilPendingCommandsAreFullyHandled(); + + verify(mockStreamPlayerCallback).onPause(); + } + + private static final String SAMPLE_URI = "http://google.com/video.mpd"; + private void setupMocks() { + AtomicReference displayContainer = new AtomicReference<>(); when(mockImaFactory.createAdsLoader(any(), any(), any(StreamDisplayContainer.class))) - .thenReturn(mockAdsLoader); + .thenAnswer( + invocation -> { + StreamDisplayContainer container = invocation.getArgument(2); + displayContainer.set(container); + container.getVideoStreamPlayer().addCallback(mockStreamPlayerCallback); + return mockAdsLoader; + }); List adsLoadedListeners = new ArrayList<>(); doAnswer( invocation -> { @@ -208,6 +353,12 @@ private void setupMocks() { for (AdsLoadedListener listener : adsLoadedListeners) { listener.onAdsManagerLoaded(mockAdsManagerLoadedEvent); } + if (displayContainer.get() != null) { + displayContainer + .get() + .getVideoStreamPlayer() + .loadUrl(SAMPLE_URI, new ArrayList<>()); + } return null; }); when(mockAdsManagerLoadedEvent.getStreamManager()).thenReturn(mockStreamManager); diff --git a/libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/RtspClient.java b/libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/RtspClient.java index e2d412012e2..ed2534940c1 100644 --- a/libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/RtspClient.java +++ b/libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/RtspClient.java @@ -38,6 +38,7 @@ import android.net.Uri; import android.os.Handler; +import android.os.Looper; import android.util.SparseArray; import androidx.annotation.IntDef; import androidx.annotation.Nullable; @@ -46,7 +47,6 @@ import androidx.media3.common.ParserException; import androidx.media3.common.util.Log; import androidx.media3.common.util.Util; -import androidx.media3.exoplayer.rtsp.RtspMediaPeriod.RtpLoadInfo; import androidx.media3.exoplayer.rtsp.RtspMediaSource.RtspPlaybackException; import androidx.media3.exoplayer.rtsp.RtspMediaSource.RtspUdpUnsupportedTransportException; import androidx.media3.exoplayer.rtsp.RtspMessageChannel.InterleavedBinaryDataListener; @@ -133,12 +133,27 @@ public interface PlaybackEventListener { void onPlaybackError(RtspPlaybackException error); } + /** Groups track URI and transport string for RTSP SETUP requests. */ + public static final class TrackSetupInfo { + /** The {@link Uri} of the loading RTSP track. */ + public final Uri trackUri; + + /** The transport string for RTP loading. */ + public final String transport; + + /** Creates a new instance. */ + public TrackSetupInfo(Uri trackUri, String transport) { + this.trackUri = trackUri; + this.transport = transport; + } + } + private final SessionInfoListener sessionInfoListener; private final PlaybackEventListener playbackEventListener; private final String userAgent; private final SocketFactory socketFactory; private final boolean debugLoggingEnabled; - private final ArrayDeque pendingSetupRtpLoadInfos; + private final ArrayDeque pendingSetupTrackInfos; // TODO(b/172331505) Add a timeout monitor for pending requests. private final SparseArray pendingRequests; private final MessageSender messageSender; @@ -186,7 +201,7 @@ public RtspClient( this.userAgent = userAgent; this.socketFactory = socketFactory; this.debugLoggingEnabled = debugLoggingEnabled; - this.pendingSetupRtpLoadInfos = new ArrayDeque<>(); + this.pendingSetupTrackInfos = new ArrayDeque<>(); this.pendingRequests = new SparseArray<>(); this.messageSender = new MessageSender(); this.uri = RtspMessageUtil.removeUserInfo(uri); @@ -223,12 +238,12 @@ public void start() throws IOException { /** * Triggers RTSP SETUP requests after track selection. * - *

    All selected tracks (represented by {@link RtpLoadInfo}) must have valid transport. + *

    All selected tracks (represented by {@link TrackSetupInfo}) must have valid transport. * - * @param loadInfos A list of selected tracks represented by {@link RtpLoadInfo}. + * @param trackSetupInfos A list of selected tracks represented by {@link TrackSetupInfo}. */ - public void setupSelectedTracks(List loadInfos) { - pendingSetupRtpLoadInfos.addAll(loadInfos); + public void setupSelectedTracks(List trackSetupInfos) { + pendingSetupTrackInfos.addAll(trackSetupInfos); continueSetupRtspTrack(); } @@ -265,12 +280,16 @@ public void seekToUs(long positionUs) { @Override public void close() throws IOException { if (keepAliveMonitor != null) { - // Playback has started. We have to stop the periodic keep alive and send a TEARDOWN so that - // the RTSP server stops sending RTP packets and frees up resources. keepAliveMonitor.close(); keepAliveMonitor = null; - messageSender.sendTeardownRequest(uri, checkNotNull(sessionId)); } + if (sessionId != null) { + messageSender.sendTeardownRequest(uri, sessionId); + } + rtspState = RTSP_STATE_UNINITIALIZED; + sessionId = null; + pendingSetupTrackInfos.clear(); + pendingRequests.clear(); messageChannel.close(); } @@ -284,7 +303,6 @@ public void retryWithRtpTcp() { close(); messageChannel = new RtspMessageChannel(new MessageListener()); messageChannel.open(getSocket(uri)); - sessionId = null; receivedAuthorizationRequest = false; rtspAuthenticationInfo = null; } catch (IOException e) { @@ -299,12 +317,12 @@ public void registerInterleavedDataChannel( } private void continueSetupRtspTrack() { - @Nullable RtpLoadInfo loadInfo = pendingSetupRtpLoadInfos.pollFirst(); - if (loadInfo == null) { + @Nullable TrackSetupInfo trackInfo = pendingSetupTrackInfos.pollFirst(); + if (trackInfo == null) { playbackEventListener.onRtspSetupCompleted(); return; } - messageSender.sendSetupRequest(loadInfo.getTrackUri(), loadInfo.getTransport(), sessionId); + messageSender.sendSetupRequest(trackInfo.trackUri, trackInfo.transport, sessionId); } private void maybeLogMessage(List message) { @@ -392,6 +410,7 @@ public void sendDescribeRequest(Uri uri, @Nullable String sessionId) { public void sendSetupRequest(Uri trackUri, String transport, @Nullable String sessionId) { rtspState = RTSP_STATE_INIT; + pendingRequests.clear(); sendRequest( getRequestWithCommonHeaders( METHOD_SETUP, @@ -419,6 +438,7 @@ public void sendTeardownRequest(Uri uri, String sessionId) { } rtspState = RTSP_STATE_INIT; + pendingRequests.clear(); sendRequest( getRequestWithCommonHeaders( METHOD_TEARDOWN, sessionId, /* additionalHeaders= */ ImmutableMap.of(), uri)); @@ -565,6 +585,7 @@ private void handleRtspResponse(List message) { if (rtspState != RTSP_STATE_UNINITIALIZED) { rtspState = RTSP_STATE_INIT; } + pendingRequests.clear(); @Nullable String redirectionUriString = response.headers.get(RtspHeaders.LOCATION); if (redirectionUriString == null) { sessionInfoListener.onSessionTimelineRequestFailed( diff --git a/libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/RtspMediaPeriod.java b/libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/RtspMediaPeriod.java index 587b0054087..070e094d4f3 100644 --- a/libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/RtspMediaPeriod.java +++ b/libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/RtspMediaPeriod.java @@ -279,39 +279,45 @@ public long seekToUs(long positionUs) { // used in the following PLAY request. // TODO(internal: b/213153670) Handle dropped seek position. - if (getBufferedPositionUs() == 0 && !isUsingRtpTcp) { + if (getBufferedPositionUs() == 0 && !prepared && !isUsingRtpTcp) { // Stores the seek position for later, if no RTP packet is received when using UDP. pendingSeekPositionUsForTcpRetry = positionUs; return positionUs; } discardBuffer(positionUs, /* toKeyframe= */ false); - requestedSeekPositionUs = positionUs; if (isSeekPending()) { + requestedSeekPositionUs = positionUs; switch (rtspClient.getState()) { case RtspClient.RTSP_STATE_READY: // PLAY request is sent, yet to receive the response. requestedSeekPositionUs stores the // new position to do another seek upon receiving the PLAY response. - return positionUs; + break; case RtspClient.RTSP_STATE_PLAYING: - // Pending PAUSE response, updates client with the newest seek position for the following - // PLAY request. - pendingSeekPositionUs = positionUs; - rtspClient.seekToUs(pendingSeekPositionUs); - return positionUs; + // Pending PAUSE response, updates client with the newest seek position for the following + // PLAY request. case RtspClient.RTSP_STATE_UNINITIALIZED: case RtspClient.RTSP_STATE_INIT: + // Pending SETUP response, updates client with the newest seek position for the PLAY + // request. + pendingSeekPositionUs = positionUs; + rtspClient.seekToUs(pendingSeekPositionUs); + break; default: - // Never happens. throw new IllegalStateException(); } + for (int i = 0; i < rtspLoaderWrappers.size(); i++) { + rtspLoaderWrappers.get(i).seekTo(positionUs); + } + return positionUs; } if (seekInsideBufferUs(positionUs)) { return positionUs; } + requestedSeekPositionUs = positionUs; pendingSeekPositionUs = positionUs; if (loadingFinished) { @@ -475,7 +481,13 @@ private void maybeSetupTracks() { } if (transportReady && trackSelected) { - rtspClient.setupSelectedTracks(selectedLoadInfos); + ImmutableList.Builder trackSetupInfos = ImmutableList.builder(); + for (int i = 0; i < selectedLoadInfos.size(); i++) { + RtpLoadInfo loadInfo = selectedLoadInfos.get(i); + trackSetupInfos.add( + new RtspClient.TrackSetupInfo(loadInfo.getTrackUri(), loadInfo.getTransport())); + } + rtspClient.setupSelectedTracks(trackSetupInfos.build()); } } @@ -535,7 +547,7 @@ private final class InternalListener @Override public void onLoadCompleted( RtpDataLoadable loadable, long elapsedRealtimeMs, long loadDurationMs) { - if (getBufferedPositionUs() == 0) { + if (getBufferedPositionUs() == 0 && !prepared) { if (!isUsingRtpTcp) { // Retry playback with TCP if no sample has been received so far, and we are not already // using TCP. Retrying will setup new loadables, so will not retry with the current @@ -657,8 +669,10 @@ public void onPlaybackStarted( requestedSeekPositionUs = C.TIME_UNSET; } else { // Resets pendingSeekPositionUs to perform a fresh RTSP seek. + long seekPositionUs = requestedSeekPositionUs; pendingSeekPositionUs = C.TIME_UNSET; - seekToUs(requestedSeekPositionUs); + requestedSeekPositionUs = C.TIME_UNSET; + seekToUs(seekPositionUs); } } else if (pendingSeekPositionUsForTcpRetry != C.TIME_UNSET && isUsingRtpTcp) { seekToUs(pendingSeekPositionUsForTcpRetry); diff --git a/libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/reader/RtpH264Reader.java b/libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/reader/RtpH264Reader.java index 3a792d374d1..a50e5625a30 100644 --- a/libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/reader/RtpH264Reader.java +++ b/libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/reader/RtpH264Reader.java @@ -200,7 +200,7 @@ private void processSingleNalUnitPacket(ParsableByteArray data) { fragmentedSampleSizeBytes += numBytesInData; int nalHeaderType = data.getData()[0] & 0x1F; - bufferFlags = getBufferFlagsFromNalType(nalHeaderType); + bufferFlags |= getBufferFlagsFromNalType(nalHeaderType); } /** @@ -238,13 +238,11 @@ private void processSingleTimeAggregationPacket(ParsableByteArray data) { int nalUnitLength; while (data.bytesLeft() > 4) { nalUnitLength = data.readUnsignedShort(); + bufferFlags |= getBufferFlagsFromNalType(data.peekUnsignedByte() & 0x1F); fragmentedSampleSizeBytes += writeStartCode(); trackOutput.sampleData(data, nalUnitLength); fragmentedSampleSizeBytes += nalUnitLength; } - - // Treat Aggregated NAL units as non key frames. - bufferFlags = 0; } /** @@ -289,6 +287,7 @@ private void processFragmentationUnitPacket(ParsableByteArray data, int packetSe // Interruption: A new FU started before the previous one finished. isCurrentAccessUnitCorrupted = true; fragmentedSampleSizeBytes = 0; + return; } isProcessingFragmentationUnit = true; // Prepends starter code. @@ -300,7 +299,8 @@ private void processFragmentationUnitPacket(ParsableByteArray data, int packetSe fuScratchBuffer.reset(data.getData()); fuScratchBuffer.setPosition(1); } else { - if (isCurrentAccessUnitCorrupted) { + if (isCurrentAccessUnitCorrupted || !isProcessingFragmentationUnit) { + isCurrentAccessUnitCorrupted = true; return; } // Check that this packet is in the sequence of the previous packet. @@ -327,7 +327,7 @@ private void processFragmentationUnitPacket(ParsableByteArray data, int packetSe if (isLastFuPacket) { isProcessingFragmentationUnit = false; - bufferFlags = getBufferFlagsFromNalType(nalHeader & 0x1F); + bufferFlags |= getBufferFlagsFromNalType(nalHeader & 0x1F); } } diff --git a/libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/reader/RtpH265Reader.java b/libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/reader/RtpH265Reader.java index bef5e5cb0ef..2bad3f06c0c 100644 --- a/libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/reader/RtpH265Reader.java +++ b/libraries/exoplayer_rtsp/src/main/java/androidx/media3/exoplayer/rtsp/reader/RtpH265Reader.java @@ -199,7 +199,7 @@ private void processSingleNalUnitPacket(ParsableByteArray data) { fragmentedSampleSizeBytes += numBytesInData; int nalHeaderType = (data.getData()[0] >> 1) & 0x3F; - bufferFlags = getBufferFlagsFromNalType(nalHeaderType); + bufferFlags |= getBufferFlagsFromNalType(nalHeaderType); } /** @@ -317,6 +317,7 @@ private void processFragmentationUnitPacket(ParsableByteArray data, int packetSe // Interruption: A new FU started before the previous one finished. isCurrentAccessUnitCorrupted = true; fragmentedSampleSizeBytes = 0; + return; } isProcessingFragmentationUnit = true; // Prepends starter code. @@ -332,7 +333,8 @@ private void processFragmentationUnitPacket(ParsableByteArray data, int packetSe fuScratchBuffer.getData()[2] = (byte) tid; fuScratchBuffer.setPosition(1); } else { - if (isCurrentAccessUnitCorrupted) { + if (isCurrentAccessUnitCorrupted || !isProcessingFragmentationUnit) { + isCurrentAccessUnitCorrupted = true; return; } // Check that this packet is in the sequence of the previous packet. @@ -359,7 +361,7 @@ private void processFragmentationUnitPacket(ParsableByteArray data, int packetSe if (isLastFuPacket) { isProcessingFragmentationUnit = false; - bufferFlags = getBufferFlagsFromNalType(nalUnitType); + bufferFlags |= getBufferFlagsFromNalType(nalUnitType); } } diff --git a/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/RtspClientTest.java b/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/RtspClientTest.java index ef260f542d1..4737e6006b1 100644 --- a/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/RtspClientTest.java +++ b/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/RtspClientTest.java @@ -38,6 +38,7 @@ import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; +import org.robolectric.shadows.ShadowLooper; /** Tests the {@link RtspClient} using the {@link RtspServer}. */ @RunWith(AndroidJUnit4.class) @@ -530,4 +531,152 @@ public void onSessionTimelineRequestFailed( RobolectricUtil.runMainLooperUntil(timelineRequestFailed::get); assertThat(rtspClient.getState()).isEqualTo(RtspClient.RTSP_STATE_UNINITIALIZED); } + + @Test + public void close_serverWithoutDescribeSupport_clearsPendingRequestAndPreventsError() + throws Exception { + AtomicBoolean optionsResponseSent = new AtomicBoolean(); + AtomicBoolean onSessionTimelineRequestFailed = new AtomicBoolean(); + rtspServer = + new RtspServer( + () -> { + optionsResponseSent.set(true); + // Return an OPTIONS response that does NOT include DESCRIBE. + // If this is incorrectly processed after close(), it will trigger a failure. + return new RtspResponse( + /* status= */ 200, + new RtspHeaders.Builder().add(RtspHeaders.PUBLIC, "OPTIONS").build()); + }); + rtspClient = + new RtspClient( + new SessionInfoListener() { + @Override + public void onSessionTimelineUpdated( + RtspSessionTiming timing, ImmutableList tracks) {} + + @Override + public void onSessionTimelineRequestFailed( + String message, @Nullable Throwable cause) { + onSessionTimelineRequestFailed.set(true); + } + }, + EMPTY_PLAYBACK_LISTENER, + /* userAgent= */ "ExoPlayer:RtspClientTest", + RtspTestUtils.getTestUri(rtspServer.startAndGetPortNumber()), + SocketFactory.getDefault(), + /* debugLoggingEnabled= */ false); + rtspClient.start(); + + RobolectricUtil.runMainLooperUntil(optionsResponseSent::get); + int maxWaitLoops = 5000; + while (ShadowLooper.shadowMainLooper().isIdle() && maxWaitLoops-- > 0) { + Thread.sleep(1); + } + rtspClient.close(); + ShadowLooper.idleMainLooper(); + + assertThat(onSessionTimelineRequestFailed.get()).isFalse(); + } + + @Test + public void setupSelectedTracks_withDelayedPlayResponse_clearsPendingRequestAndPreventsError() + throws Exception { + AtomicBoolean playResponseSent = new AtomicBoolean(); + rtspServer = + new RtspServer( + new RtspServer.ResponseProvider() { + @Override + public RtspResponse getOptionsResponse() { + return new RtspResponse( + 200, + new RtspHeaders.Builder() + .add(RtspHeaders.PUBLIC, "OPTIONS, DESCRIBE, SETUP, PLAY") + .build()); + } + + @Override + public RtspResponse getDescribeResponse(Uri requestedUri, RtspHeaders headers) { + return RtspTestUtils.newDescribeResponseWithSdpMessage( + SESSION_DESCRIPTION, rtpPacketStreamDumps, requestedUri); + } + + @Override + public RtspResponse getSetupResponse(Uri requestedUri, RtspHeaders headers) { + return new RtspResponse( + 200, + new RtspHeaders.Builder() + .add(RtspHeaders.SESSION, "12345678") + .add(RtspHeaders.TRANSPORT, "RTP/AVP;unicast;client_port=65536-65537") + .build()); + } + + @Override + public RtspResponse getPlayResponse() { + playResponseSent.set(true); + return new RtspResponse( + 200, + new RtspHeaders.Builder() + .add( + RtspHeaders.RTP_INFO, + RtspTestUtils.getRtpInfoForDumps(rtpPacketStreamDumps)) + .build()); + } + }); + Uri testUri = RtspTestUtils.getTestUri(rtspServer.startAndGetPortNumber()); + AtomicBoolean onSessionTimelineRequestFailed = new AtomicBoolean(); + AtomicReference playbackError = new AtomicReference<>(); + rtspClient = + new RtspClient( + new SessionInfoListener() { + @Override + public void onSessionTimelineUpdated( + RtspSessionTiming timing, ImmutableList tracks) { + rtspClient.setupSelectedTracks( + ImmutableList.of( + new RtspClient.TrackSetupInfo( + tracks.get(0).uri, "RTP/AVP;unicast;client_port=65536-65537"))); + } + + @Override + public void onSessionTimelineRequestFailed( + String message, @Nullable Throwable cause) { + onSessionTimelineRequestFailed.set(true); + } + }, + new PlaybackEventListener() { + @Override + public void onRtspSetupCompleted() { + rtspClient.startPlayback(0); + } + + @Override + public void onPlaybackStarted( + long startPositionUs, ImmutableList trackTimingList) {} + + @Override + public void onPlaybackError(RtspPlaybackException error) { + playbackError.set(error); + } + }, + "ExoPlayer:RtspClientTest", + testUri, + SocketFactory.getDefault(), + false); + rtspClient.start(); + + RobolectricUtil.runMainLooperUntil(playResponseSent::get); + rtspClient.setupSelectedTracks( + ImmutableList.of( + new RtspClient.TrackSetupInfo( + testUri.buildUpon().appendEncodedPath("track1").build(), + "RTP/AVP;unicast;client_port=65536-65537"))); + RobolectricUtil.runMainLooperUntil( + () -> + rtspClient.getState() == RtspClient.RTSP_STATE_READY + || playbackError.get() != null + || onSessionTimelineRequestFailed.get()); + + assertThat(onSessionTimelineRequestFailed.get()).isFalse(); + assertThat(playbackError.get()).isNull(); + } } diff --git a/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/RtspMediaPeriodTest.java b/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/RtspMediaPeriodTest.java index 2c54d3abd39..d4a4cd096c4 100644 --- a/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/RtspMediaPeriodTest.java +++ b/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/RtspMediaPeriodTest.java @@ -16,22 +16,29 @@ package androidx.media3.exoplayer.rtsp; import static com.google.common.truth.Truth.assertThat; +import static java.lang.Math.min; import android.net.Uri; import androidx.annotation.Nullable; import androidx.media3.common.C; import androidx.media3.common.TrackGroup; import androidx.media3.common.util.Util; +import androidx.media3.datasource.DataSpec; +import androidx.media3.datasource.TransferListener; import androidx.media3.exoplayer.LoadingInfo; import androidx.media3.exoplayer.source.MediaPeriod; import androidx.media3.exoplayer.source.SampleStream; import androidx.media3.exoplayer.trackselection.ExoTrackSelection; import androidx.media3.exoplayer.trackselection.FixedTrackSelection; import androidx.media3.exoplayer.upstream.DefaultAllocator; +import androidx.media3.test.utils.TestUtil; import androidx.media3.test.utils.robolectric.RobolectricUtil; +import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.collect.ImmutableList; import java.io.IOException; +import java.util.List; +import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicLong; import java.util.concurrent.atomic.AtomicReference; @@ -208,6 +215,275 @@ public void isLoading_afterSelectTracksAndSendPlayRequest_returnsTrue() throws E assertThat(mediaPeriod.isLoading()).isTrue(); } + @Test + public void seekToUs_withRapidScrubbingAfterSuccessfulPlayResponse_doesNotTriggerTcpFallback() + throws Exception { + AtomicBoolean getPlayResponseReference = new AtomicBoolean(); + rtspServer = + new RtspServer( + new TestResponseProvider( + rtpPacketStreamDump, + /* getPlayResponseReference= */ getPlayResponseReference, + /* isWwwAuthenticationMode= */ false)); + mediaPeriod = + new RtspMediaPeriod( + new DefaultAllocator(/* trimOnReset= */ true, C.DEFAULT_BUFFER_SEGMENT_SIZE), + new FakeRtpDataChannelFactory(), + RtspTestUtils.getTestUri(rtspServer.startAndGetPortNumber()), + /* listener= */ timing -> {}, + /* userAgent= */ "ExoPlayer:RtspPeriodTest", + /* socketFactory= */ SocketFactory.getDefault(), + /* debugLoggingEnabled= */ false); + mediaPeriod.prepare(mediaPeriodCallback, /* positionUs= */ 0); + RobolectricUtil.runMainLooperUntil(() -> trackGroupAtomicReference.get() != null); + SampleStream[] sampleStreams = new SampleStream[1]; + mediaPeriod.selectTracks( + new ExoTrackSelection[] { + new FixedTrackSelection(trackGroupAtomicReference.get(), /* track= */ 0) + }, + /* mayRetainStreamFlags= */ new boolean[] {false}, + sampleStreams, + /* streamResetFlags= */ new boolean[] {true}, + /* positionUs= */ 0); + RobolectricUtil.runMainLooperUntil(getPlayResponseReference::get); + long seekPositionUs = 5000000; + + mediaPeriod.seekToUs(0); + mediaPeriod.seekToUs(seekPositionUs); + + assertThat(mediaPeriod.getBufferedPositionUs()).isEqualTo(seekPositionUs); + } + + @Test + public void seekToUs_duringInitialization_succeedsWithoutException() throws Exception { + rtspServer = + new RtspServer( + new TestResponseProvider( + rtpPacketStreamDump, + /* getPlayResponseReference= */ null, + /* isWwwAuthenticationMode= */ false)); + mediaPeriod = + new RtspMediaPeriod( + new DefaultAllocator(/* trimOnReset= */ true, C.DEFAULT_BUFFER_SEGMENT_SIZE), + new FakeRtpDataChannelFactory(), + RtspTestUtils.getTestUri(rtspServer.startAndGetPortNumber()), + /* listener= */ timing -> {}, + /* userAgent= */ "ExoPlayer:RtspPeriodTest", + /* socketFactory= */ SocketFactory.getDefault(), + /* debugLoggingEnabled= */ false); + mediaPeriod.prepare(mediaPeriodCallback, /* positionUs= */ 0); + RobolectricUtil.runMainLooperUntil(() -> trackGroupAtomicReference.get() != null); + SampleStream[] sampleStreams = new SampleStream[1]; + mediaPeriod.selectTracks( + new ExoTrackSelection[] { + new FixedTrackSelection(trackGroupAtomicReference.get(), /* track= */ 0) + }, + /* mayRetainStreamFlags= */ new boolean[] {false}, + sampleStreams, + /* streamResetFlags= */ new boolean[] {true}, + /* positionUs= */ 0); + + // Perform multiple seeks before PLAY response is received (during RTSP_STATE_INIT). + mediaPeriod.seekToUs(1000000); + mediaPeriod.seekToUs(5000000); + + assertThat(mediaPeriod.getBufferedPositionUs()).isEqualTo(5000000); + } + + @Test + public void seekToUs_toEarlierPositionWhileSeekPending_buffersSamplesFromNewPosition() + throws Exception { + FakeRtpDataChannelFactory rtpDataChannelFactory = new FakeRtpDataChannelFactory(); + RtpPacketStreamDump aacLcStreamDump = + RtpPacketStreamDump.parse( + TestUtil.getString( + ApplicationProvider.getApplicationContext(), "media/rtsp/aac-dump.json") + .replace("profile-level-id=1", "profile-level-id=2")); + rtspServer = + new RtspServer( + new TestResponseProvider( + aacLcStreamDump, + /* getPlayResponseReference= */ null, + /* isWwwAuthenticationMode= */ false)); + mediaPeriod = + new RtspMediaPeriod( + new DefaultAllocator(/* trimOnReset= */ true, C.DEFAULT_BUFFER_SEGMENT_SIZE), + rtpDataChannelFactory, + RtspTestUtils.getTestUri(rtspServer.startAndGetPortNumber()), + /* listener= */ timing -> {}, + /* userAgent= */ "ExoPlayer:RtspPeriodTest", + /* socketFactory= */ SocketFactory.getDefault(), + /* debugLoggingEnabled= */ false); + mediaPeriod.prepare(mediaPeriodCallback, /* positionUs= */ 0); + RobolectricUtil.runMainLooperUntil(() -> trackGroupAtomicReference.get() != null); + SampleStream[] sampleStreams = new SampleStream[1]; + mediaPeriod.selectTracks( + new ExoTrackSelection[] { + new FixedTrackSelection(trackGroupAtomicReference.get(), /* track= */ 0) + }, + /* mayRetainStreamFlags= */ new boolean[] {false}, + sampleStreams, + /* streamResetFlags= */ new boolean[] {true}, + /* positionUs= */ 0); + + mediaPeriod.seekToUs(10_000_000); + mediaPeriod.seekToUs(500_000); + RobolectricUtil.runMainLooperUntil(() -> mediaPeriod.getBufferedPositionUs() == 0); + rtpDataChannelFactory.getDataChannel().enqueuePackets(aacLcStreamDump.packets); + RobolectricUtil.runMainLooperUntil( + rtpDataChannelFactory.getDataChannel()::areAllPacketsProcessed); + + assertThat(mediaPeriod.getBufferedPositionUs()).isEqualTo(640_000); + } + + @Test + public void + seekToUs_toEarlierPositionDuringPlayingStateWhilePausePending_buffersSamplesFromNewPosition() + throws Exception { + AtomicBoolean getPlayResponseReference = new AtomicBoolean(); + FakeRtpDataChannelFactory rtpDataChannelFactory = new FakeRtpDataChannelFactory(); + RtpPacketStreamDump aacLcStreamDump = + RtpPacketStreamDump.parse( + TestUtil.getString( + ApplicationProvider.getApplicationContext(), "media/rtsp/aac-dump.json") + .replace("profile-level-id=1", "profile-level-id=2")); + rtspServer = + new RtspServer( + new TestResponseProvider( + aacLcStreamDump, + /* getPlayResponseReference= */ getPlayResponseReference, + /* isWwwAuthenticationMode= */ false)); + mediaPeriod = + new RtspMediaPeriod( + new DefaultAllocator(/* trimOnReset= */ true, C.DEFAULT_BUFFER_SEGMENT_SIZE), + rtpDataChannelFactory, + RtspTestUtils.getTestUri(rtspServer.startAndGetPortNumber()), + /* listener= */ timing -> {}, + /* userAgent= */ "ExoPlayer:RtspPeriodTest", + /* socketFactory= */ SocketFactory.getDefault(), + /* debugLoggingEnabled= */ false); + mediaPeriod.prepare(mediaPeriodCallback, /* positionUs= */ 0); + RobolectricUtil.runMainLooperUntil(() -> trackGroupAtomicReference.get() != null); + SampleStream[] sampleStreams = new SampleStream[1]; + mediaPeriod.selectTracks( + new ExoTrackSelection[] { + new FixedTrackSelection(trackGroupAtomicReference.get(), /* track= */ 0) + }, + /* mayRetainStreamFlags= */ new boolean[] {false}, + sampleStreams, + /* streamResetFlags= */ new boolean[] {true}, + /* positionUs= */ 0); + RobolectricUtil.runMainLooperUntil(getPlayResponseReference::get); + + // Seek forward then backward while in RTSP_STATE_PLAYING waiting for PAUSE response. + mediaPeriod.seekToUs(10_000_000); + mediaPeriod.seekToUs(500_000); + RobolectricUtil.runMainLooperUntil(() -> mediaPeriod.getBufferedPositionUs() == 0); + rtpDataChannelFactory.getDataChannel().enqueuePackets(aacLcStreamDump.packets); + RobolectricUtil.runMainLooperUntil( + rtpDataChannelFactory.getDataChannel()::areAllPacketsProcessed); + + assertThat(mediaPeriod.getBufferedPositionUs()).isEqualTo(640_000); + } + + @Test + public void seekToUs_withinBufferedRange_retainsBufferedPosition() throws Exception { + AtomicBoolean getPlayResponseReference = new AtomicBoolean(); + FakeRtpDataChannelFactory rtpDataChannelFactory = new FakeRtpDataChannelFactory(); + RtpPacketStreamDump aacLcStreamDump = + RtpPacketStreamDump.parse( + TestUtil.getString( + ApplicationProvider.getApplicationContext(), "media/rtsp/aac-dump.json") + .replace("profile-level-id=1", "profile-level-id=2")); + rtspServer = + new RtspServer( + new TestResponseProvider( + aacLcStreamDump, + /* getPlayResponseReference= */ getPlayResponseReference, + /* isWwwAuthenticationMode= */ false)); + mediaPeriod = + new RtspMediaPeriod( + new DefaultAllocator(/* trimOnReset= */ true, C.DEFAULT_BUFFER_SEGMENT_SIZE), + rtpDataChannelFactory, + RtspTestUtils.getTestUri(rtspServer.startAndGetPortNumber()), + /* listener= */ timing -> {}, + /* userAgent= */ "ExoPlayer:RtspPeriodTest", + /* socketFactory= */ SocketFactory.getDefault(), + /* debugLoggingEnabled= */ false); + mediaPeriod.prepare(mediaPeriodCallback, /* positionUs= */ 0); + RobolectricUtil.runMainLooperUntil(() -> trackGroupAtomicReference.get() != null); + SampleStream[] sampleStreams = new SampleStream[1]; + mediaPeriod.selectTracks( + new ExoTrackSelection[] { + new FixedTrackSelection(trackGroupAtomicReference.get(), /* track= */ 0) + }, + /* mayRetainStreamFlags= */ new boolean[] {false}, + sampleStreams, + /* streamResetFlags= */ new boolean[] {true}, + /* positionUs= */ 0); + RobolectricUtil.runMainLooperUntil(getPlayResponseReference::get); + rtpDataChannelFactory.getDataChannel().enqueuePackets(aacLcStreamDump.packets); + RobolectricUtil.runMainLooperUntil( + rtpDataChannelFactory.getDataChannel()::areAllPacketsProcessed); + assertThat(mediaPeriod.getBufferedPositionUs()).isEqualTo(640_000); + + mediaPeriod.seekToUs(200_000); + + assertThat(mediaPeriod.getBufferedPositionUs()).isEqualTo(640_000); + } + + @Test + public void seekToUs_duringReadyStateWhilePlayResponseInFlight_requestsNewPlayAtLatestPosition() + throws Exception { + AtomicBoolean getPlayResponseReference = new AtomicBoolean(); + FakeRtpDataChannelFactory rtpDataChannelFactory = new FakeRtpDataChannelFactory(); + RtpPacketStreamDump aacLcStreamDump = + RtpPacketStreamDump.parse( + TestUtil.getString( + ApplicationProvider.getApplicationContext(), "media/rtsp/aac-dump.json") + .replace("profile-level-id=1", "profile-level-id=2")); + rtspServer = + new RtspServer( + new TestResponseProvider( + aacLcStreamDump, + /* getPlayResponseReference= */ getPlayResponseReference, + /* isWwwAuthenticationMode= */ false)); + mediaPeriod = + new RtspMediaPeriod( + new DefaultAllocator(/* trimOnReset= */ true, C.DEFAULT_BUFFER_SEGMENT_SIZE), + rtpDataChannelFactory, + RtspTestUtils.getTestUri(rtspServer.startAndGetPortNumber()), + /* listener= */ timing -> {}, + /* userAgent= */ "ExoPlayer:RtspPeriodTest", + /* socketFactory= */ SocketFactory.getDefault(), + /* debugLoggingEnabled= */ false); + mediaPeriod.prepare(mediaPeriodCallback, /* positionUs= */ 0); + RobolectricUtil.runMainLooperUntil(() -> trackGroupAtomicReference.get() != null); + SampleStream[] sampleStreams = new SampleStream[1]; + mediaPeriod.selectTracks( + new ExoTrackSelection[] { + new FixedTrackSelection(trackGroupAtomicReference.get(), /* track= */ 0) + }, + /* mayRetainStreamFlags= */ new boolean[] {false}, + sampleStreams, + /* streamResetFlags= */ new boolean[] {true}, + /* positionUs= */ 10_000_000); + RobolectricUtil.runMainLooperUntil(getPlayResponseReference::get); + + // Seek to 500_000 us while in RTSP_STATE_READY before the client processes the first PLAY + // response on the main looper. + getPlayResponseReference.set(false); + mediaPeriod.seekToUs(500_000); + RobolectricUtil.runMainLooperUntil(getPlayResponseReference::get); + RobolectricUtil.runMainLooperUntil(() -> mediaPeriod.getBufferedPositionUs() == 0); + + rtpDataChannelFactory.getDataChannel().enqueuePackets(aacLcStreamDump.packets); + RobolectricUtil.runMainLooperUntil( + rtpDataChannelFactory.getDataChannel()::areAllPacketsProcessed); + + assertThat(mediaPeriod.getBufferedPositionUs()).isEqualTo(640_000); + } + private static class TestResponseProvider implements RtspServer.ResponseProvider { private static final String SESSION_ID = "00000000"; @@ -229,7 +505,7 @@ public RtspResponse getOptionsResponse() { return new RtspResponse( /* status= */ 200, new RtspHeaders.Builder() - .add(RtspHeaders.PUBLIC, "OPTIONS, DESCRIBE, SETUP, PLAY") + .add(RtspHeaders.PUBLIC, "OPTIONS, DESCRIBE, SETUP, PLAY, PAUSE") .build()); } @@ -290,4 +566,100 @@ public RtspResponse getPlayResponse() { .build()); } } + + private static final class FakeRtpDataChannel implements RtpDataChannel { + private final LinkedBlockingQueue packetQueue = new LinkedBlockingQueue<>(); + private final AtomicBoolean allPacketsProcessed = new AtomicBoolean(); + + private void enqueuePackets(List hexPackets) { + allPacketsProcessed.set(false); + for (String hexPacket : hexPackets) { + packetQueue.add(Util.getBytesFromHexString(hexPacket)); + } + // Enqueue an empty sentinel packet so read() knows when all preceding packets have been + // extracted and processed by the background loader thread. + packetQueue.add(new byte[0]); + } + + private boolean areAllPacketsProcessed() { + return allPacketsProcessed.get(); + } + + @Override + public String getTransport() { + return "RTP/AVP;unicast;client_port=1234-1235"; + } + + @Override + public int getLocalPort() { + return 1234; + } + + @Override + public boolean needsClosingOnLoadCompletion() { + return false; + } + + @Nullable + @Override + public RtspMessageChannel.InterleavedBinaryDataListener getInterleavedBinaryDataListener() { + return null; + } + + @Override + public void addTransferListener(TransferListener transferListener) {} + + @Override + public long open(DataSpec dataSpec) { + return C.LENGTH_UNSET; + } + + @Nullable + @Override + public Uri getUri() { + return null; + } + + @Override + public void close() {} + + @Override + public int read(byte[] buffer, int offset, int length) { + try { + byte[] packet = packetQueue.take(); + if (packet.length == 0) { + if (packetQueue.isEmpty()) { + allPacketsProcessed.set(true); + } + // Block the read thread until a packet is available or interrupted, simulating a UDP + // socket. + packet = packetQueue.take(); + } + int bytesToRead = min(length, packet.length); + System.arraycopy(packet, /* srcPos= */ 0, buffer, offset, bytesToRead); + return bytesToRead; + } catch (InterruptedException e) { + Thread.currentThread().interrupt(); + return C.RESULT_END_OF_INPUT; + } + } + } + + private static final class FakeRtpDataChannelFactory implements RtpDataChannel.Factory { + private final FakeRtpDataChannel dataChannel = new FakeRtpDataChannel(); + + @Override + public RtpDataChannel createAndOpenDataChannel(int trackId) { + return dataChannel; + } + + private FakeRtpDataChannel getDataChannel() { + return dataChannel; + } + + @Override + public RtpDataChannel.Factory createFallbackDataChannelFactory() { + return new TransferRtpDataChannelFactory(DEFAULT_TIMEOUT_MS); + } + } } diff --git a/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/RtspServer.java b/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/RtspServer.java index 85b247a8b54..4c8058b4bf0 100644 --- a/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/RtspServer.java +++ b/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/RtspServer.java @@ -17,6 +17,7 @@ import static androidx.media3.exoplayer.rtsp.RtspRequest.METHOD_DESCRIBE; import static androidx.media3.exoplayer.rtsp.RtspRequest.METHOD_OPTIONS; +import static androidx.media3.exoplayer.rtsp.RtspRequest.METHOD_PAUSE; import static androidx.media3.exoplayer.rtsp.RtspRequest.METHOD_PLAY; import static androidx.media3.exoplayer.rtsp.RtspRequest.METHOD_SETUP; import static androidx.media3.exoplayer.rtsp.RtspRequest.METHOD_TEARDOWN; @@ -59,6 +60,11 @@ default RtspResponse getPlayResponse() { return RtspTestUtils.RTSP_ERROR_METHOD_NOT_ALLOWED; } + /** Returns an RTSP PAUSE {@link RtspResponse response}. */ + default RtspResponse getPauseResponse() { + return new RtspResponse(/* status= */ 200, RtspHeaders.EMPTY); + } + /** Returns an RTSP TEARDOWN {@link RtspResponse response}. */ default RtspResponse getTearDownResponse() { return new RtspResponse(/* status= */ 200, RtspHeaders.EMPTY); @@ -155,6 +161,10 @@ private void handleRtspMessage(List message) { sendResponse(responseProvider.getPlayResponse(), cSeq); break; + case METHOD_PAUSE: + sendResponse(responseProvider.getPauseResponse(), cSeq); + break; + case METHOD_TEARDOWN: sendResponse(responseProvider.getTearDownResponse(), cSeq); break; diff --git a/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/RtspTestUtils.java b/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/RtspTestUtils.java index b3afff0a2a9..87c8a4de878 100644 --- a/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/RtspTestUtils.java +++ b/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/RtspTestUtils.java @@ -29,7 +29,7 @@ private static final String TEST_BASE_URI = "rtsp://localhost:%d/test"; private static final String TEST_BASE_URI_WITH_USER_INFO = "rtsp://%s:%s@localhost:%d/test"; - private static final String RTP_TIME_FORMAT = "url=rtsp://localhost/test/%s;seq=%d;rtptime=%d"; + private static final String RTP_TIME_FORMAT = "url=%s;seq=%d;rtptime=%d"; /** RTSP error Method Not Allowed (RFC2326 Section 7.1.1). */ public static final RtspResponse RTSP_ERROR_METHOD_NOT_ALLOWED = diff --git a/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/reader/RtpH264ReaderTest.java b/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/reader/RtpH264ReaderTest.java index 1ac2f6093a6..d8903d120d6 100644 --- a/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/reader/RtpH264ReaderTest.java +++ b/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/reader/RtpH264ReaderTest.java @@ -18,6 +18,7 @@ import static com.google.common.truth.Truth.assertThat; import static org.junit.Assert.assertThrows; +import androidx.media3.common.C; import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; import androidx.media3.common.ParserException; @@ -122,6 +123,7 @@ public void consume_validPackets() throws ParserException { .isEqualTo( ImmutableByteArray.concat(NALU_1_START_DELIMITED, NALU_2_START_DELIMITED).toArray()); assertThat(trackOutput.getSampleData(0).length).isEqualTo(32); + assertThat(trackOutput.getSampleFlags(0)).isEqualTo(C.BUFFER_FLAG_KEY_FRAME); } @Test @@ -182,6 +184,7 @@ public void consume_multipleSingleNalUnitsInSingleAccessUnit_reportsCorrectCombi assertThat(trackOutput.getSampleCount()).isEqualTo(1); int expectedSize = NALU_1_START_DELIMITED.length() + NALU_2_START_DELIMITED.length(); assertThat(trackOutput.getSampleData(0).length).isEqualTo(expectedSize); + assertThat(trackOutput.getSampleFlags(0)).isEqualTo(C.BUFFER_FLAG_KEY_FRAME); } @Test @@ -214,6 +217,56 @@ public void consume_interruptedFragmentationUnitByNewFragmentationUnit_discardsS assertThat(trackOutput.getSampleCount()).isEqualTo(0); } + @Test + public void consume_corruptedIdrAccessUnitWithMarker_resetsKeyFrameFlagForNextAccessUnit() + throws ParserException { + long corruptedAuTimestamp = 9_000_000; + long nextAuTimestamp = 9_003_000; + // Packet 1: Valid IDR Single NALU before the interrupted FU-A. + RtpPacket idrSingleNalu = + new RtpPacket.Builder() + .setTimestamp(corruptedAuTimestamp) + .setSequenceNumber(99) + .setMarker(false) + .setPayloadData(ImmutableByteArray.concatToArray(NALU_2_HEADER, NALU_2_PAYLOAD)) + .build(); + // Packet 2: FU-A Start for NALU A. + RtpPacket fuStartA = + createFragmentedPacket( + /* sequenceNumber= */ 100, + corruptedAuTimestamp, + /* marker= */ false, + ImmutableByteArray.ofHexString("85"), // S=1, E=0, NALU type 5 + ImmutableByteArray.ofHexString("0102")); + // Packet 3: Another FU-A Start for NALU B (Interruption, marker = true). + RtpPacket fuStartB = + createFragmentedPacket( + /* sequenceNumber= */ 101, + corruptedAuTimestamp, + /* marker= */ true, + ImmutableByteArray.ofHexString("85"), + ImmutableByteArray.ofHexString("0304")); + // Next Access Unit: Valid non-IDR Single NALU. + RtpPacket nextAuNonIdr = + new RtpPacket.Builder() + .setTimestamp(nextAuTimestamp) + .setSequenceNumber(102) + .setMarker(true) + .setPayloadData(ImmutableByteArray.concatToArray(NALU_1_HEADER, NALU_1_PAYLOAD)) + .build(); + rtpH264Reader.createTracks(extractorOutput, /* trackId= */ 0); + + consume(rtpH264Reader, idrSingleNalu); + consume(rtpH264Reader, fuStartA); + consume(rtpH264Reader, fuStartB); + consume(rtpH264Reader, nextAuNonIdr); + + FakeTrackOutput trackOutput = extractorOutput.trackOutputs.get(0); + assertThat(trackOutput.getSampleCount()).isEqualTo(1); + assertThat(trackOutput.getSampleData(0)).isEqualTo(NALU_1_START_DELIMITED.toArray()); + assertThat(trackOutput.getSampleFlags(0)).isEqualTo(0); + } + @Test public void consume_newAccessUnit_resetsStateCorrectly() throws ParserException { RtpPacket packet1 = @@ -228,7 +281,14 @@ public void consume_newAccessUnit_resetsStateCorrectly() throws ParserException .setTimestamp(9000) .setSequenceNumber(PACKET_SEQUENCE_NUMBER + 1) .setMarker(true) - .setPayloadData(ImmutableByteArray.concatToArray(NALU_1_HEADER, NALU_1_PAYLOAD)) + .setPayloadData(ImmutableByteArray.concatToArray(NALU_2_HEADER, NALU_2_PAYLOAD)) + .build(); + RtpPacket incompleteIdrNalu = + new RtpPacket.Builder() + .setTimestamp(18000) + .setSequenceNumber(PACKET_SEQUENCE_NUMBER + 2) + .setMarker(false) + .setPayloadData(ImmutableByteArray.concatToArray(NALU_2_HEADER, NALU_2_PAYLOAD)) .build(); RtpPacket fuContinuation = createFragmentedPacket( @@ -246,26 +306,37 @@ public void consume_newAccessUnit_resetsStateCorrectly() throws ParserException .build(); rtpH264Reader.createTracks(extractorOutput, /* trackId= */ 0); - // Access Unit 1: Two packets. + // Access Unit 1: Two packets (second is IDR). consume(rtpH264Reader, packet1); consume(rtpH264Reader, packet2); - // Access Unit 2: Missing packets. + // Access Unit 2: IDR packet followed by FU continuation without FU start (corrupted, no + // marker). + consume(rtpH264Reader, incompleteIdrNalu); consume(rtpH264Reader, fuContinuation); - // Access Unit 3: One packet with different timestamp. + // Access Unit 3: One non-IDR packet with different timestamp. consume(rtpH264Reader, packet3); FakeTrackOutput trackOutput = extractorOutput.trackOutputs.get(0); assertThat(trackOutput.getSampleCount()).isEqualTo(2); - // Verify sizes are independent (not accumulated) + assertThat(trackOutput.getSampleFlags(0)).isEqualTo(C.BUFFER_FLAG_KEY_FRAME); + // Verify sizes and flags are independent (not accumulated across access units). assertThat(trackOutput.getSampleData(1).length).isEqualTo(NALU_1_START_DELIMITED.length()); + assertThat(trackOutput.getSampleFlags(1)).isEqualTo(0); } @Test public void seek_resetsStateCorrectly() throws ParserException { long seekTimeUs = 1_000_000; + RtpPacket incompleteIdrNalu = + new RtpPacket.Builder() + .setTimestamp(9000) + .setSequenceNumber(PACKET_SEQUENCE_NUMBER) + .setMarker(false) + .setPayloadData(ImmutableByteArray.concatToArray(NALU_2_HEADER, NALU_2_PAYLOAD)) + .build(); RtpPacket fuContinuation = createFragmentedPacket( - PACKET_SEQUENCE_NUMBER, + PACKET_SEQUENCE_NUMBER + 1, 9000, /* marker= */ false, ImmutableByteArray.ofHexString("05"), // S=0, E=0 @@ -273,11 +344,12 @@ public void seek_resetsStateCorrectly() throws ParserException { RtpPacket singleNalu = new RtpPacket.Builder() .setTimestamp(18000) - .setSequenceNumber(PACKET_SEQUENCE_NUMBER + 1) + .setSequenceNumber(PACKET_SEQUENCE_NUMBER + 2) .setMarker(true) .setPayloadData(ImmutableByteArray.concatToArray(NALU_1_HEADER, NALU_1_PAYLOAD)) .build(); rtpH264Reader.createTracks(extractorOutput, /* trackId= */ 0); + consume(rtpH264Reader, incompleteIdrNalu); consume(rtpH264Reader, fuContinuation); rtpH264Reader.seek(/* nextRtpTimestamp= */ 18000, /* timeUs= */ seekTimeUs); @@ -287,19 +359,244 @@ public void seek_resetsStateCorrectly() throws ParserException { assertThat(trackOutput.getSampleCount()).isEqualTo(1); assertThat(trackOutput.getSampleData(0).length).isEqualTo(NALU_1_START_DELIMITED.length()); assertThat(trackOutput.getSampleTimeUs(0)).isEqualTo(seekTimeUs); + assertThat(trackOutput.getSampleFlags(0)).isEqualTo(0); + } + + @Test + public void consume_idrFragmentationUnitFollowedByNonIdrStapA_setsKeyFrameFlag() + throws ParserException { + long idrAuTimestamp = 9_000_000; + long nonIdrAuTimestamp = 9_003_000; + ImmutableByteArray fuPayloadPart1 = ImmutableByteArray.ofHexString("010203"); + ImmutableByteArray fuPayloadPart2 = ImmutableByteArray.ofHexString("040506"); + // FU-A indicator 0x7c (F=0, NRI=3) combined with type 5 (IDR) -> 0x65; with type 1 -> 0x61. + ImmutableByteArray reconstructedFuIdrNalu = + ImmutableByteArray.concat( + NALU_START_CODE, ImmutableByteArray.ofHexString("65"), fuPayloadPart1, fuPayloadPart2); + ImmutableByteArray reconstructedFuNonIdrNalu = + ImmutableByteArray.concat( + NALU_START_CODE, ImmutableByteArray.ofHexString("61"), fuPayloadPart1, fuPayloadPart2); + // Access Unit 1: Leading non-IDR STAP-A -> IDR in FU-A (type 5) -> trailing non-IDR STAP-A + // (marker = true). + RtpPacket stapANonIdrStart = + createAggregationPacket( + /* sequenceNumber= */ 100, + idrAuTimestamp, + /* marker= */ false, + NALU_1_LENGTH, + NALU_1_HEADER, + NALU_1_PAYLOAD, + NALU_1_LENGTH, + NALU_1_HEADER, + NALU_1_PAYLOAD); + RtpPacket fuIdrStart = + createFragmentedPacket( + /* sequenceNumber= */ 101, + idrAuTimestamp, + /* marker= */ false, + ImmutableByteArray.ofHexString("85"), // S=1, E=0, NALU type 5 (IDR) + fuPayloadPart1); + RtpPacket fuIdrEnd = + createFragmentedPacket( + /* sequenceNumber= */ 102, + idrAuTimestamp, + /* marker= */ false, + ImmutableByteArray.ofHexString("45"), // S=0, E=1, NALU type 5 (IDR) + fuPayloadPart2); + RtpPacket stapANonIdrEnd = + createAggregationPacket( + /* sequenceNumber= */ 103, + idrAuTimestamp, + /* marker= */ true, + NALU_1_LENGTH, + NALU_1_HEADER, + NALU_1_PAYLOAD, + NALU_1_LENGTH, + NALU_1_HEADER, + NALU_1_PAYLOAD); + // Access Unit 2: Multi-packet non-IDR Access Unit combining STAP-A, FU-A, and Single NALU to + // verify bufferFlags resets after marker and stays 0 across all non-IDR packetization modes. + RtpPacket nextAuStapANonIdr = + createAggregationPacket( + /* sequenceNumber= */ 104, + nonIdrAuTimestamp, + /* marker= */ false, + NALU_1_LENGTH, + NALU_1_HEADER, + NALU_1_PAYLOAD, + NALU_1_LENGTH, + NALU_1_HEADER, + NALU_1_PAYLOAD); + RtpPacket nextAuFuNonIdrStart = + createFragmentedPacket( + /* sequenceNumber= */ 105, + nonIdrAuTimestamp, + /* marker= */ false, + ImmutableByteArray.ofHexString("81"), + fuPayloadPart1); + RtpPacket nextAuFuNonIdrEnd = + createFragmentedPacket( + /* sequenceNumber= */ 106, + nonIdrAuTimestamp, + /* marker= */ false, + ImmutableByteArray.ofHexString("41"), + fuPayloadPart2); + RtpPacket nextAuNonIdrSingleNalu = + new RtpPacket.Builder() + .setTimestamp(nonIdrAuTimestamp) + .setSequenceNumber(107) + .setMarker(true) + .setPayloadData(ImmutableByteArray.concatToArray(NALU_1_HEADER, NALU_1_PAYLOAD)) + .build(); + rtpH264Reader.createTracks(extractorOutput, /* trackId= */ 0); + + consume(rtpH264Reader, stapANonIdrStart); + consume(rtpH264Reader, fuIdrStart); + consume(rtpH264Reader, fuIdrEnd); + consume(rtpH264Reader, stapANonIdrEnd); + consume(rtpH264Reader, nextAuStapANonIdr); + consume(rtpH264Reader, nextAuFuNonIdrStart); + consume(rtpH264Reader, nextAuFuNonIdrEnd); + consume(rtpH264Reader, nextAuNonIdrSingleNalu); + + FakeTrackOutput trackOutput = extractorOutput.trackOutputs.get(0); + assertThat(trackOutput.getSampleCount()).isEqualTo(2); + assertThat(trackOutput.getSampleFlags(0)).isEqualTo(C.BUFFER_FLAG_KEY_FRAME); + assertThat(trackOutput.getSampleData(0)) + .isEqualTo( + ImmutableByteArray.concat( + NALU_1_START_DELIMITED, + NALU_1_START_DELIMITED, + reconstructedFuIdrNalu, + NALU_1_START_DELIMITED, + NALU_1_START_DELIMITED) + .toArray()); + assertThat(trackOutput.getSampleFlags(1)).isEqualTo(0); + assertThat(trackOutput.getSampleData(1)) + .isEqualTo( + ImmutableByteArray.concat( + NALU_1_START_DELIMITED, + NALU_1_START_DELIMITED, + reconstructedFuNonIdrNalu, + NALU_1_START_DELIMITED) + .toArray()); + } + + @Test + public void consume_stapAWithIdrAndTrailingNonIdrNalus_setsKeyFrameFlag() throws ParserException { + long auTimestamp = 9_000_000; + // IDR in middle of 3-NALU STAP-A (NALU_1 non-IDR -> NALU_2 IDR -> NALU_1 non-IDR), followed by + // a trailing non-IDR Single NALU (marker = true). + RtpPacket stapANonIdrThenIdrThenNonIdr = + createAggregationPacket( + /* sequenceNumber= */ 100, + auTimestamp, + /* marker= */ false, + NALU_1_LENGTH, + NALU_1_HEADER, + NALU_1_PAYLOAD, + NALU_2_LENGTH, + NALU_2_HEADER, + NALU_2_PAYLOAD, + NALU_1_LENGTH, + NALU_1_HEADER, + NALU_1_PAYLOAD); + RtpPacket nonIdrSingleNaluEnd = + new RtpPacket.Builder() + .setTimestamp(auTimestamp) + .setSequenceNumber(101) + .setMarker(true) + .setPayloadData(ImmutableByteArray.concatToArray(NALU_1_HEADER, NALU_1_PAYLOAD)) + .build(); + rtpH264Reader.createTracks(extractorOutput, /* trackId= */ 0); + + consume(rtpH264Reader, stapANonIdrThenIdrThenNonIdr); + consume(rtpH264Reader, nonIdrSingleNaluEnd); + + FakeTrackOutput trackOutput = extractorOutput.trackOutputs.get(0); + assertThat(trackOutput.getSampleCount()).isEqualTo(1); + assertThat(trackOutput.getSampleFlags(0)).isEqualTo(C.BUFFER_FLAG_KEY_FRAME); + assertThat(trackOutput.getSampleData(0)) + .isEqualTo( + ImmutableByteArray.concat( + NALU_1_START_DELIMITED, + NALU_2_START_DELIMITED, + NALU_1_START_DELIMITED, + NALU_1_START_DELIMITED) + .toArray()); + } + + @Test + public void consume_idrSingleNaluFollowedByNonIdrFragmentationUnit_setsKeyFrameFlag() + throws ParserException { + long auTimestamp = 9_000_000; + ImmutableByteArray fuPayloadPart1 = ImmutableByteArray.ofHexString("010203"); + ImmutableByteArray fuPayloadPart2 = ImmutableByteArray.ofHexString("040506"); + ImmutableByteArray reconstructedFuNonIdrNalu = + ImmutableByteArray.concat( + NALU_START_CODE, ImmutableByteArray.ofHexString("61"), fuPayloadPart1, fuPayloadPart2); + // Leading non-IDR Single NALU -> IDR in Single NALU (NALU_2 type 5) -> trailing non-IDR FU-A + // (marker = true). + RtpPacket nonIdrSingleNaluStart = + new RtpPacket.Builder() + .setTimestamp(auTimestamp) + .setSequenceNumber(100) + .setMarker(false) + .setPayloadData(ImmutableByteArray.concatToArray(NALU_1_HEADER, NALU_1_PAYLOAD)) + .build(); + RtpPacket idrSingleNalu = + new RtpPacket.Builder() + .setTimestamp(auTimestamp) + .setSequenceNumber(101) + .setMarker(false) + .setPayloadData(ImmutableByteArray.concatToArray(NALU_2_HEADER, NALU_2_PAYLOAD)) + .build(); + RtpPacket fuNonIdrStart = + createFragmentedPacket( + /* sequenceNumber= */ 102, + auTimestamp, + /* marker= */ false, + ImmutableByteArray.ofHexString("81"), // S=1, E=0, NALU type 1 (non-IDR) + fuPayloadPart1); + RtpPacket fuNonIdrEnd = + createFragmentedPacket( + /* sequenceNumber= */ 103, + auTimestamp, + /* marker= */ true, + ImmutableByteArray.ofHexString("41"), // S=0, E=1, NALU type 1 (non-IDR) + fuPayloadPart2); + rtpH264Reader.createTracks(extractorOutput, /* trackId= */ 0); + + consume(rtpH264Reader, nonIdrSingleNaluStart); + consume(rtpH264Reader, idrSingleNalu); + consume(rtpH264Reader, fuNonIdrStart); + consume(rtpH264Reader, fuNonIdrEnd); + + FakeTrackOutput trackOutput = extractorOutput.trackOutputs.get(0); + assertThat(trackOutput.getSampleCount()).isEqualTo(1); + assertThat(trackOutput.getSampleFlags(0)).isEqualTo(C.BUFFER_FLAG_KEY_FRAME); + assertThat(trackOutput.getSampleData(0)) + .isEqualTo( + ImmutableByteArray.concat( + NALU_1_START_DELIMITED, NALU_2_START_DELIMITED, reconstructedFuNonIdrNalu) + .toArray()); } private static RtpPacket createAggregationPacket( int sequenceNumber, long timeStamp, ImmutableByteArray... nalUnits) { + return createAggregationPacket(sequenceNumber, timeStamp, /* marker= */ true, nalUnits); + } + + private static RtpPacket createAggregationPacket( + int sequenceNumber, long timeStamp, boolean marker, ImmutableByteArray... nalUnits) { ImmutableByteArray.Builder payloadData = new ImmutableByteArray.Builder().addAll(STAP_A_HEADER); - for (int i = 0; i < nalUnits.length; i += 2) { - payloadData.addAll(nalUnits[i]); // Length - payloadData.addAll(nalUnits[i + 1]); // Header + Payload + for (ImmutableByteArray nalUnit : nalUnits) { + payloadData.addAll(nalUnit); } return new RtpPacket.Builder() .setTimestamp(timeStamp) .setSequenceNumber(sequenceNumber) - .setMarker(true) + .setMarker(marker) .setPayloadData(payloadData.build().toArray()) .build(); } diff --git a/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/reader/RtpH265ReaderTest.java b/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/reader/RtpH265ReaderTest.java index d7d3fe8227a..e6039aed169 100644 --- a/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/reader/RtpH265ReaderTest.java +++ b/libraries/exoplayer_rtsp/src/test/java/androidx/media3/exoplayer/rtsp/reader/RtpH265ReaderTest.java @@ -517,6 +517,60 @@ public void consume_multipleMixedNalUnitsInSingleAccessUnit_aggregatesCorrectly( assertThat(trackOutput.getSampleCount()).isEqualTo(0); } + @Test + public void consume_corruptedIdrAccessUnitWithMarker_resetsKeyFrameFlagForNextAccessUnit() + throws ParserException { + long corruptedAuTimestamp = 9_000_000; + long nextAuTimestamp = 9_003_000; + // Packet 1: Valid IDR Single NALU (type 20, NAL_IDR_N_LP -> 0x2801) before the corrupted FU. + RtpPacket idrSingleNalu = + new RtpPacket.Builder() + .setTimestamp(corruptedAuTimestamp) + .setSequenceNumber(PACKET_SEQUENCE_NUMBER - 1) + .setMarker(false) + .setPayloadData( + ImmutableByteArray.concatToArray( + ImmutableByteArray.ofHexString("2801"), NALU_1_PAYLOAD)) + .build(); + // Packet 2: FU Start. + RtpPacket fuStart = + createFragmentedPacket( + PACKET_SEQUENCE_NUMBER, + corruptedAuTimestamp, + /* marker= */ false, + FU_1_PACKET_1_FU_HEADER, + FU_1_PACKET_1_NALU_PAYLOAD); + // Packet 3: MISSING (Simulated loss) + // Packet 4: Valid Single NALU in same AU (Marker True). + RtpPacket singleNalu = + new RtpPacket.Builder() + .setTimestamp(corruptedAuTimestamp) + .setSequenceNumber(PACKET_SEQUENCE_NUMBER + 2) // Gap of 1 + .setMarker(true) + .setPayloadData(ImmutableByteArray.concatToArray(NALU_1_HEADER, NALU_1_PAYLOAD)) + .build(); + // Next Access Unit: Valid non-IDR Single NALU. + RtpPacket nextAuNonIdr = + new RtpPacket.Builder() + .setTimestamp(nextAuTimestamp) + .setSequenceNumber(PACKET_SEQUENCE_NUMBER + 3) + .setMarker(true) + .setPayloadData(ImmutableByteArray.concatToArray(NALU_1_HEADER, NALU_1_PAYLOAD)) + .build(); + rtpH265Reader.createTracks(extractorOutput, /* trackId= */ 0); + rtpH265Reader.onReceivingFirstPacket(idrSingleNalu.timestamp, idrSingleNalu.sequenceNumber); + + consume(rtpH265Reader, idrSingleNalu); + consume(rtpH265Reader, fuStart); + consume(rtpH265Reader, singleNalu); + consume(rtpH265Reader, nextAuNonIdr); + + FakeTrackOutput trackOutput = extractorOutput.trackOutputs.get(0); + assertThat(trackOutput.getSampleCount()).isEqualTo(1); + assertThat(trackOutput.getSampleData(0)).isEqualTo(NALU_1_START_DELIMITED.toArray()); + assertThat(trackOutput.getSampleFlags(0)).isEqualTo(0); + } + @Test public void consume_fragmentationUnitContinuationAtAccessUnitStart_discardsWholeSample() throws ParserException { @@ -576,12 +630,81 @@ public void consume_timestampRollover_resetsCorrectly() throws ParserException { assertThat(trackOutput.getSampleTimeUs(1)).isGreaterThan(trackOutput.getSampleTimeUs(0)); } + @Test + public void consume_newAccessUnit_resetsStateCorrectly() throws ParserException { + // IDR NALU header: type 20 (NAL_IDR_N_LP) -> (20 << 1) = 0x28, TID = 1 -> 0x2801. + ImmutableByteArray idrNLpHeader = ImmutableByteArray.ofHexString("2801"); + RtpPacket packet1 = + new RtpPacket.Builder() + .setTimestamp(9000) + .setSequenceNumber(PACKET_SEQUENCE_NUMBER) + .setMarker(false) + .setPayloadData(ImmutableByteArray.concatToArray(NALU_1_HEADER, NALU_1_PAYLOAD)) + .build(); + RtpPacket packet2 = + new RtpPacket.Builder() + .setTimestamp(9000) + .setSequenceNumber(PACKET_SEQUENCE_NUMBER + 1) + .setMarker(true) + .setPayloadData(ImmutableByteArray.concatToArray(idrNLpHeader, NALU_1_PAYLOAD)) + .build(); + RtpPacket incompleteIdrNalu = + new RtpPacket.Builder() + .setTimestamp(18000) + .setSequenceNumber(PACKET_SEQUENCE_NUMBER + 2) + .setMarker(false) + .setPayloadData(ImmutableByteArray.concatToArray(idrNLpHeader, NALU_1_PAYLOAD)) + .build(); + RtpPacket fuContinuation = + createFragmentedPacket( + PACKET_SEQUENCE_NUMBER + 3, + 18000, + /* marker= */ false, + FU_1_PACKET_2_FU_HEADER, // S=0, E=0 + FU_1_PACKET_2_NALU_PAYLOAD); + RtpPacket packet3 = + new RtpPacket.Builder() + .setTimestamp(27000) + .setSequenceNumber(PACKET_SEQUENCE_NUMBER + 4) + .setMarker(true) + .setPayloadData(ImmutableByteArray.concatToArray(NALU_1_HEADER, NALU_1_PAYLOAD)) + .build(); + rtpH265Reader.createTracks(extractorOutput, /* trackId= */ 0); + + // Access Unit 1: Two packets (second is IDR). + consume(rtpH265Reader, packet1); + consume(rtpH265Reader, packet2); + // Access Unit 2: IDR packet followed by FU continuation without FU start (corrupted, no + // marker). + consume(rtpH265Reader, incompleteIdrNalu); + consume(rtpH265Reader, fuContinuation); + // Access Unit 3: One non-IDR packet with different timestamp. + consume(rtpH265Reader, packet3); + + FakeTrackOutput trackOutput = extractorOutput.trackOutputs.get(0); + assertThat(trackOutput.getSampleCount()).isEqualTo(2); + assertThat(trackOutput.getSampleFlags(0)).isEqualTo(C.BUFFER_FLAG_KEY_FRAME); + // Verify sizes and flags are independent (not accumulated across access units). + assertThat(trackOutput.getSampleData(1).length).isEqualTo(NALU_1_START_DELIMITED.length()); + assertThat(trackOutput.getSampleFlags(1)).isEqualTo(0); + } + @Test public void seek_resetsStateCorrectly() throws ParserException { long seekTimeUs = 1_000_000; + // IDR NALU header: type 20 (NAL_IDR_N_LP) -> (20 << 1) = 0x28, TID = 1 -> 0x2801. + RtpPacket incompleteIdrNalu = + new RtpPacket.Builder() + .setTimestamp(9000) + .setSequenceNumber(PACKET_SEQUENCE_NUMBER) + .setMarker(false) + .setPayloadData( + ImmutableByteArray.concatToArray( + ImmutableByteArray.ofHexString("2801"), NALU_1_PAYLOAD)) + .build(); RtpPacket fuContinuation = createFragmentedPacket( - PACKET_SEQUENCE_NUMBER, + PACKET_SEQUENCE_NUMBER + 1, 9000, /* marker= */ false, FU_1_PACKET_2_FU_HEADER, // S=0, E=0 @@ -589,11 +712,12 @@ public void seek_resetsStateCorrectly() throws ParserException { RtpPacket singleNalu = new RtpPacket.Builder() .setTimestamp(18000) - .setSequenceNumber(PACKET_SEQUENCE_NUMBER + 1) + .setSequenceNumber(PACKET_SEQUENCE_NUMBER + 2) .setMarker(true) .setPayloadData(ImmutableByteArray.concatToArray(NALU_1_HEADER, NALU_1_PAYLOAD)) .build(); rtpH265Reader.createTracks(extractorOutput, /* trackId= */ 0); + consume(rtpH265Reader, incompleteIdrNalu); consume(rtpH265Reader, fuContinuation); rtpH265Reader.seek(/* nextRtpTimestamp= */ 18000, /* timeUs= */ seekTimeUs); @@ -603,10 +727,247 @@ public void seek_resetsStateCorrectly() throws ParserException { assertThat(trackOutput.getSampleCount()).isEqualTo(1); assertThat(trackOutput.getSampleData(0).length).isEqualTo(NALU_1_START_DELIMITED.length()); assertThat(trackOutput.getSampleTimeUs(0)).isEqualTo(seekTimeUs); + assertThat(trackOutput.getSampleFlags(0)).isEqualTo(0); + } + + @Test + public void consume_idrFragmentationUnitFollowedByNonIdrAggregationPacket_setsKeyFrameFlag() + throws ParserException { + long idrAuTimestamp = 9_000_000; + long nonIdrAuTimestamp = 9_003_000; + // Type 20 (NAL_IDR_N_LP) -> (20 << 1) = 0x28, TID = 1 -> 0x2801. + ImmutableByteArray idrNLpHeader = ImmutableByteArray.ofHexString("2801"); + ImmutableByteArray reconstructedFuIdrNalu = + ImmutableByteArray.concat( + NALU_START_CODE, idrNLpHeader, FU_1_PACKET_1_NALU_PAYLOAD, FU_1_PACKET_3_NALU_PAYLOAD); + // Access Unit 1: Leading non-IDR AP -> IDR in FU (type 20, NAL_IDR_N_LP) -> trailing non-IDR + // AP (marker = true). + RtpPacket apNonIdrStart = + createAggregationPacket( + /* sequenceNumber= */ 100, + idrAuTimestamp, + /* marker= */ false, + NALU_1_LENGTH, + NALU_1_HEADER, + NALU_1_PAYLOAD, + NALU_2_LENGTH, + NALU_2_HEADER, + NALU_2_PAYLOAD); + RtpPacket fuIdrStart = + createFragmentedPacket( + /* sequenceNumber= */ 101, + idrAuTimestamp, + /* marker= */ false, + ImmutableByteArray.ofHexString("94"), // S=1, E=0, NALU type 20 (NAL_IDR_N_LP) + FU_1_PACKET_1_NALU_PAYLOAD); + RtpPacket fuIdrEnd = + createFragmentedPacket( + /* sequenceNumber= */ 102, + idrAuTimestamp, + /* marker= */ false, + ImmutableByteArray.ofHexString("54"), // S=0, E=1, NALU type 20 (NAL_IDR_N_LP) + FU_1_PACKET_3_NALU_PAYLOAD); + RtpPacket apNonIdrEnd = + createAggregationPacket( + /* sequenceNumber= */ 103, + idrAuTimestamp, + /* marker= */ true, + NALU_1_LENGTH, + NALU_1_HEADER, + NALU_1_PAYLOAD, + NALU_2_LENGTH, + NALU_2_HEADER, + NALU_2_PAYLOAD); + // Access Unit 2: Multi-packet non-IDR Access Unit combining AP, FU, and Single NALU to verify + // bufferFlags resets after marker and stays 0 across all non-IDR packetization modes. + RtpPacket nextAuApNonIdr = + createAggregationPacket( + /* sequenceNumber= */ 104, + nonIdrAuTimestamp, + /* marker= */ false, + NALU_1_LENGTH, + NALU_1_HEADER, + NALU_1_PAYLOAD, + NALU_2_LENGTH, + NALU_2_HEADER, + NALU_2_PAYLOAD); + RtpPacket nextAuFuNonIdrStart = + createFragmentedPacket( + /* sequenceNumber= */ 105, + nonIdrAuTimestamp, + /* marker= */ false, + FU_1_PACKET_1_FU_HEADER, + FU_1_PACKET_1_NALU_PAYLOAD); + RtpPacket nextAuFuNonIdrMiddle = + createFragmentedPacket( + /* sequenceNumber= */ 106, + nonIdrAuTimestamp, + /* marker= */ false, + FU_1_PACKET_2_FU_HEADER, + FU_1_PACKET_2_NALU_PAYLOAD); + RtpPacket nextAuFuNonIdrEnd = + createFragmentedPacket( + /* sequenceNumber= */ 107, + nonIdrAuTimestamp, + /* marker= */ false, + FU_1_PACKET_3_FU_HEADER, + FU_1_PACKET_3_NALU_PAYLOAD); + RtpPacket nextAuNonIdrSingleNalu = + new RtpPacket.Builder() + .setTimestamp(nonIdrAuTimestamp) + .setSequenceNumber(108) + .setMarker(true) + .setPayloadData(ImmutableByteArray.concatToArray(NALU_1_HEADER, NALU_1_PAYLOAD)) + .build(); + rtpH265Reader.createTracks(extractorOutput, /* trackId= */ 0); + + consume(rtpH265Reader, apNonIdrStart); + consume(rtpH265Reader, fuIdrStart); + consume(rtpH265Reader, fuIdrEnd); + consume(rtpH265Reader, apNonIdrEnd); + consume(rtpH265Reader, nextAuApNonIdr); + consume(rtpH265Reader, nextAuFuNonIdrStart); + consume(rtpH265Reader, nextAuFuNonIdrMiddle); + consume(rtpH265Reader, nextAuFuNonIdrEnd); + consume(rtpH265Reader, nextAuNonIdrSingleNalu); + + FakeTrackOutput trackOutput = extractorOutput.trackOutputs.get(0); + assertThat(trackOutput.getSampleCount()).isEqualTo(2); + assertThat(trackOutput.getSampleFlags(0)).isEqualTo(C.BUFFER_FLAG_KEY_FRAME); + assertThat(trackOutput.getSampleData(0)) + .isEqualTo( + ImmutableByteArray.concat( + NALU_1_AND_2_START_DELIMITED, + reconstructedFuIdrNalu, + NALU_1_AND_2_START_DELIMITED) + .toArray()); + assertThat(trackOutput.getSampleFlags(1)).isEqualTo(0); + assertThat(trackOutput.getSampleData(1)) + .isEqualTo( + ImmutableByteArray.concat( + NALU_1_AND_2_START_DELIMITED, NALU_1_START_DELIMITED, NALU_1_START_DELIMITED) + .toArray()); + } + + @Test + public void consume_aggregationPacketWithIdrAndTrailingNonIdrNalus_setsKeyFrameFlag() + throws ParserException { + long auTimestamp = 9_000_000; + // Type 19 (NAL_IDR_W_RADL) -> (19 << 1) = 0x26, TID = 1 -> 0x2601. + ImmutableByteArray idrWRadlHeader = ImmutableByteArray.ofHexString("2601"); + ImmutableByteArray idrWRadlStartDelimited = + ImmutableByteArray.concat(NALU_START_CODE, idrWRadlHeader, NALU_1_PAYLOAD); + // IDR in middle of 3-NALU AP (non-IDR NALU_1 -> type 19 NAL_IDR_W_RADL -> non-IDR NALU_1), + // followed by a trailing non-IDR Single NALU (marker = true). + RtpPacket apNonIdrThenIdrThenNonIdr = + createAggregationPacket( + /* sequenceNumber= */ 100, + auTimestamp, + /* marker= */ false, + NALU_1_LENGTH, + NALU_1_HEADER, + NALU_1_PAYLOAD, + NALU_1_LENGTH, + idrWRadlHeader, + NALU_1_PAYLOAD, + NALU_1_LENGTH, + NALU_1_HEADER, + NALU_1_PAYLOAD); + RtpPacket nonIdrSingleNaluEnd = + new RtpPacket.Builder() + .setTimestamp(auTimestamp) + .setSequenceNumber(101) + .setMarker(true) + .setPayloadData(ImmutableByteArray.concatToArray(NALU_1_HEADER, NALU_1_PAYLOAD)) + .build(); + rtpH265Reader.createTracks(extractorOutput, /* trackId= */ 0); + + consume(rtpH265Reader, apNonIdrThenIdrThenNonIdr); + consume(rtpH265Reader, nonIdrSingleNaluEnd); + + FakeTrackOutput trackOutput = extractorOutput.trackOutputs.get(0); + assertThat(trackOutput.getSampleCount()).isEqualTo(1); + assertThat(trackOutput.getSampleFlags(0)).isEqualTo(C.BUFFER_FLAG_KEY_FRAME); + assertThat(trackOutput.getSampleData(0)) + .isEqualTo( + ImmutableByteArray.concat( + NALU_1_START_DELIMITED, + idrWRadlStartDelimited, + NALU_1_START_DELIMITED, + NALU_1_START_DELIMITED) + .toArray()); + } + + @Test + public void consume_idrSingleNaluFollowedByNonIdrFragmentationUnit_setsKeyFrameFlag() + throws ParserException { + long auTimestamp = 9_000_000; + // Type 20 (NAL_IDR_N_LP) -> (20 << 1) = 0x28, TID = 1 -> 0x2801. + ImmutableByteArray idrNLpHeader = ImmutableByteArray.ofHexString("2801"); + ImmutableByteArray idrNLpStartDelimited = + ImmutableByteArray.concat(NALU_START_CODE, idrNLpHeader, NALU_1_PAYLOAD); + // Leading non-IDR Single NALU -> IDR in Single NALU (type 20, NAL_IDR_N_LP) -> trailing non-IDR + // FU (marker = true). + RtpPacket nonIdrSingleNaluStart = + new RtpPacket.Builder() + .setTimestamp(auTimestamp) + .setSequenceNumber(100) + .setMarker(false) + .setPayloadData(ImmutableByteArray.concatToArray(NALU_1_HEADER, NALU_1_PAYLOAD)) + .build(); + RtpPacket idrSingleNalu = + new RtpPacket.Builder() + .setTimestamp(auTimestamp) + .setSequenceNumber(101) + .setMarker(false) + .setPayloadData(ImmutableByteArray.concatToArray(idrNLpHeader, NALU_1_PAYLOAD)) + .build(); + RtpPacket fuNonIdrStart = + createFragmentedPacket( + /* sequenceNumber= */ 102, + auTimestamp, + /* marker= */ false, + FU_1_PACKET_1_FU_HEADER, + FU_1_PACKET_1_NALU_PAYLOAD); + RtpPacket fuNonIdrMiddle = + createFragmentedPacket( + /* sequenceNumber= */ 103, + auTimestamp, + /* marker= */ false, + FU_1_PACKET_2_FU_HEADER, + FU_1_PACKET_2_NALU_PAYLOAD); + RtpPacket fuNonIdrEnd = + createFragmentedPacket( + /* sequenceNumber= */ 104, + auTimestamp, + /* marker= */ true, + FU_1_PACKET_3_FU_HEADER, + FU_1_PACKET_3_NALU_PAYLOAD); + rtpH265Reader.createTracks(extractorOutput, /* trackId= */ 0); + + consume(rtpH265Reader, nonIdrSingleNaluStart); + consume(rtpH265Reader, idrSingleNalu); + consume(rtpH265Reader, fuNonIdrStart); + consume(rtpH265Reader, fuNonIdrMiddle); + consume(rtpH265Reader, fuNonIdrEnd); + + FakeTrackOutput trackOutput = extractorOutput.trackOutputs.get(0); + assertThat(trackOutput.getSampleCount()).isEqualTo(1); + assertThat(trackOutput.getSampleFlags(0)).isEqualTo(C.BUFFER_FLAG_KEY_FRAME); + assertThat(trackOutput.getSampleData(0)) + .isEqualTo( + ImmutableByteArray.concat( + NALU_1_START_DELIMITED, idrNLpStartDelimited, NALU_1_START_DELIMITED) + .toArray()); } private static RtpPacket createAggregationPacket( int sequenceNumber, long timeStamp, ImmutableByteArray... nalUnits) { + return createAggregationPacket(sequenceNumber, timeStamp, /* marker= */ true, nalUnits); + } + + private static RtpPacket createAggregationPacket( + int sequenceNumber, long timeStamp, boolean marker, ImmutableByteArray... nalUnits) { ImmutableByteArray.Builder payloadData = new ImmutableByteArray.Builder().addAll(AP_NALU_HEADER); for (ImmutableByteArray nalUnit : nalUnits) { @@ -615,7 +976,7 @@ private static RtpPacket createAggregationPacket( return new RtpPacket.Builder() .setTimestamp(timeStamp) .setSequenceNumber(sequenceNumber) - .setMarker(true) + .setMarker(marker) .setPayloadData(payloadData.build().toArray()) .build(); } diff --git a/libraries/exoplayer_smoothstreaming/src/main/java/androidx/media3/exoplayer/smoothstreaming/DefaultSsChunkSource.java b/libraries/exoplayer_smoothstreaming/src/main/java/androidx/media3/exoplayer/smoothstreaming/DefaultSsChunkSource.java index 93e13d7c4c4..1d2c6d001da 100644 --- a/libraries/exoplayer_smoothstreaming/src/main/java/androidx/media3/exoplayer/smoothstreaming/DefaultSsChunkSource.java +++ b/libraries/exoplayer_smoothstreaming/src/main/java/androidx/media3/exoplayer/smoothstreaming/DefaultSsChunkSource.java @@ -412,7 +412,8 @@ public boolean onChunkLoadError( /* numberOfLocations= */ 1, /* numberOfExcludedLocations= */ 0, numberOfTracks, - numberOfExcludedTracks); + numberOfExcludedTracks, + /* locationSteeringActive= */ false); @Nullable FallbackSelection fallbackSelection = loadErrorHandlingPolicy.getFallbackSelectionFor(fallbackOptions, loadErrorInfo); diff --git a/libraries/exoplayer_smoothstreaming/src/main/java/androidx/media3/exoplayer/smoothstreaming/SsMediaSource.java b/libraries/exoplayer_smoothstreaming/src/main/java/androidx/media3/exoplayer/smoothstreaming/SsMediaSource.java index bbfbe2448d7..9e1eb6f4703 100644 --- a/libraries/exoplayer_smoothstreaming/src/main/java/androidx/media3/exoplayer/smoothstreaming/SsMediaSource.java +++ b/libraries/exoplayer_smoothstreaming/src/main/java/androidx/media3/exoplayer/smoothstreaming/SsMediaSource.java @@ -529,32 +529,28 @@ public void onLoadStarted( long elapsedRealtimeMs, long loadDurationMs, int retryCount) { - LoadEventInfo loadEventInfo = - retryCount == 0 - ? new LoadEventInfo(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) - : new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); - manifestEventDispatcher.loadStarted(loadEventInfo, loadable.type, retryCount); + LoadEventInfo.Builder loadEventInfo = + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs); + if (retryCount != 0) { + loadEventInfo + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()); + } + manifestEventDispatcher.loadStarted(loadEventInfo.build(), loadable.type, retryCount); } @Override public void onLoadCompleted( ParsingLoadable loadable, long elapsedRealtimeMs, long loadDurationMs) { LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(); loadErrorHandlingPolicy.onLoadTaskConcluded(loadable.loadTaskId); manifestEventDispatcher.loadCompleted(loadEventInfo, loadable.type); manifest = loadable.getResult(); @@ -570,14 +566,12 @@ public void onLoadCanceled( long loadDurationMs, boolean released) { LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(); loadErrorHandlingPolicy.onLoadTaskConcluded(loadable.loadTaskId); manifestEventDispatcher.loadCanceled(loadEventInfo, loadable.type); } @@ -590,18 +584,18 @@ public LoadErrorAction onLoadError( IOException error, int errorCount) { LoadEventInfo loadEventInfo = - new LoadEventInfo( - loadable.loadTaskId, - loadable.dataSpec, - loadable.getUri(), - loadable.getResponseHeaders(), - elapsedRealtimeMs, - loadDurationMs, - loadable.bytesLoaded()); + new LoadEventInfo.Builder(loadable.loadTaskId, loadable.dataSpec, elapsedRealtimeMs) + .setUri(loadable.getUri()) + .setResponseHeaders(loadable.getResponseHeaders()) + .setLoadDurationMs(loadDurationMs) + .setBytesLoaded(loadable.bytesLoaded()) + .build(); MediaLoadData mediaLoadData = new MediaLoadData(loadable.type); long retryDelayMs = - loadErrorHandlingPolicy.getRetryDelayMsFor( - new LoadErrorInfo(loadEventInfo, mediaLoadData, error, errorCount)); + errorCount > loadErrorHandlingPolicy.getMinimumLoadableRetryCount(loadable.type) + ? C.TIME_UNSET + : loadErrorHandlingPolicy.getRetryDelayMsFor( + new LoadErrorInfo(loadEventInfo, mediaLoadData, error, errorCount)); LoadErrorAction loadErrorAction = retryDelayMs == C.TIME_UNSET ? Loader.DONT_RETRY_FATAL @@ -714,7 +708,9 @@ private void startLoadingManifest() { dataSpec = cmcdDataFactory.createCmcdData().addToDataSpec(dataSpec); } ParsingLoadable loadable = - new ParsingLoadable<>(manifestDataSource, dataSpec, C.DATA_TYPE_MANIFEST, manifestParser); + new ParsingLoadable.Builder( + manifestDataSource, dataSpec, C.DATA_TYPE_MANIFEST, manifestParser) + .build(); manifestLoader.startLoading( loadable, this, loadErrorHandlingPolicy.getMinimumLoadableRetryCount(loadable.type)); } diff --git a/libraries/exoplayer_smoothstreaming/src/test/java/androidx/media3/exoplayer/smoothstreaming/SsMediaSourceTest.java b/libraries/exoplayer_smoothstreaming/src/test/java/androidx/media3/exoplayer/smoothstreaming/SsMediaSourceTest.java index f8f1a068251..246d69952d8 100644 --- a/libraries/exoplayer_smoothstreaming/src/test/java/androidx/media3/exoplayer/smoothstreaming/SsMediaSourceTest.java +++ b/libraries/exoplayer_smoothstreaming/src/test/java/androidx/media3/exoplayer/smoothstreaming/SsMediaSourceTest.java @@ -15,9 +15,18 @@ */ package androidx.media3.exoplayer.smoothstreaming; +import static androidx.media3.common.util.Util.createHandlerForCurrentLooper; import static com.google.common.truth.Truth.assertThat; import static org.junit.Assert.fail; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyInt; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; +import static org.robolectric.Shadows.shadowOf; +import android.net.Uri; +import android.os.Looper; import androidx.annotation.Nullable; import androidx.media3.common.C; import androidx.media3.common.MediaItem; @@ -25,14 +34,20 @@ import androidx.media3.common.Timeline; import androidx.media3.datasource.ByteArrayDataSource; import androidx.media3.datasource.DataSource; +import androidx.media3.datasource.DefaultHttpDataSource; import androidx.media3.datasource.TransferListener; import androidx.media3.exoplayer.analytics.PlayerId; import androidx.media3.exoplayer.smoothstreaming.manifest.SsManifest; +import androidx.media3.exoplayer.smoothstreaming.manifest.SsManifestParser; import androidx.media3.exoplayer.source.MediaSource; +import androidx.media3.exoplayer.source.MediaSourceEventListener; import androidx.media3.exoplayer.trackselection.ExoTrackSelection; import androidx.media3.exoplayer.upstream.BandwidthMeter; import androidx.media3.exoplayer.upstream.CmcdConfiguration; +import androidx.media3.exoplayer.upstream.DefaultLoadErrorHandlingPolicy; +import androidx.media3.exoplayer.upstream.Loader; import androidx.media3.exoplayer.upstream.LoaderErrorThrower; +import androidx.media3.exoplayer.upstream.ParsingLoadable; import androidx.media3.extractor.text.SubtitleParser; import androidx.media3.test.utils.FakeDataSource; import androidx.media3.test.utils.TestUtil; @@ -184,6 +199,47 @@ public void updateMediaItem_createsTimelineWithUpdatedItem() throws Exception { assertThat(window.mediaItem).isEqualTo(updatedMediaItem); } + @Test + public void onLoadError_whenRetriesExhausted_returnsDontRetryFatalAndReportsCanceledTrue() { + SsMediaSource mediaSource = + new SsMediaSource.Factory(new DefaultHttpDataSource.Factory()) + .setLoadErrorHandlingPolicy( + new DefaultLoadErrorHandlingPolicy(/* minimumLoadableRetryCount= */ 3)) + .createMediaSource(MediaItem.fromUri("http://example.com/manifest.ismc")); + MediaSourceEventListener mockListener = mock(MediaSourceEventListener.class); + mediaSource.addEventListener(createHandlerForCurrentLooper(), mockListener); + ParsingLoadable loadable = + new ParsingLoadable.Builder<>( + new DefaultHttpDataSource.Factory().createDataSource(), + Uri.parse("http://example.com/manifest.ismc"), + C.DATA_TYPE_MANIFEST, + new SsManifestParser()) + .build(); + + Loader.LoadErrorAction actionWithinLimit = + mediaSource.onLoadError( + loadable, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + new IOException("Transient error"), + /* errorCount= */ 3); + Loader.LoadErrorAction actionExhausted = + mediaSource.onLoadError( + loadable, + /* elapsedRealtimeMs= */ 0, + /* loadDurationMs= */ 0, + new IOException("Transient error"), + /* errorCount= */ 4); + shadowOf(Looper.getMainLooper()).idle(); + + assertThat(actionWithinLimit.isRetry()).isTrue(); + verify(mockListener) + .onLoadError(anyInt(), any(), any(), any(), any(IOException.class), eq(false)); + assertThat(actionExhausted).isEqualTo(Loader.DONT_RETRY_FATAL); + verify(mockListener) + .onLoadError(anyInt(), any(), any(), any(), any(IOException.class), eq(true)); + } + private static Timeline.Window prepareAndWaitForTimelineRefresh(MediaSource mediaSource) throws Exception { AtomicReference windowReference = new AtomicReference<>(); diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/DefaultExtractorsFactory.java b/libraries/extractor/src/main/java/androidx/media3/extractor/DefaultExtractorsFactory.java index 03f062508b1..8552c7373f3 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/DefaultExtractorsFactory.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/DefaultExtractorsFactory.java @@ -21,7 +21,6 @@ import android.net.Uri; import androidx.annotation.GuardedBy; import androidx.annotation.Nullable; -import androidx.media3.common.C; import androidx.media3.common.FileTypes; import androidx.media3.common.Format; import androidx.media3.common.PlaybackException; @@ -103,8 +102,8 @@ public final class DefaultExtractorsFactory implements ExtractorsFactory { // Extractors order is optimized according to // https://docs.google.com/document/d/1w2mKaWMxfz2Ei8-LdxqbPs1VLe_oudB-eryXXw9OvQQ. - // The JPEG extractor appears after audio/video extractors because we expect audio/video input to - // be more common. + // Image extractors appear after audio/video extractors because we expect audio/video input to + // be more common. MP3 is an exception and is sniffed last (see below). private static final int[] DEFAULT_EXTRACTOR_ORDER = new int[] { FileTypes.FLV, @@ -119,7 +118,6 @@ public final class DefaultExtractorsFactory implements ExtractorsFactory { FileTypes.ADTS, FileTypes.AC3, FileTypes.AC4, - FileTypes.MP3, // The following extractors are not part of the optimized ordering, and were appended // without further analysis. FileTypes.AVI, @@ -129,7 +127,11 @@ public final class DefaultExtractorsFactory implements ExtractorsFactory { FileTypes.WEBP, FileTypes.BMP, FileTypes.HEIF, - FileTypes.AVIF + FileTypes.AVIF, + // MP3 is last (as in the optimized ordering doc) because Mp3Extractor.sniff() scans up to + // 128 KB for frame headers and can falsely match other formats (e.g. JPEG). New extractors + // should be added before MP3. + FileTypes.MP3 }; private static final ExtensionLoader FLAC_EXTENSION_LOADER = @@ -154,16 +156,16 @@ public final class DefaultExtractorsFactory implements ExtractorsFactory { private int tsTimestampSearchBytes; private boolean textTrackTranscodingEnabled; private SubtitleParser.Factory subtitleParserFactory; - private @C.VideoCodecFlags int codecsToParseWithinGopSampleDependencies; private @JpegExtractor.Flags int jpegFlags; private @HeifExtractor.Flags int heifFlags; + private boolean parseHagcMetadata; public DefaultExtractorsFactory() { tsMode = TsExtractor.MODE_SINGLE_PMT; tsTimestampSearchBytes = TsExtractor.DEFAULT_TIMESTAMP_SEARCH_BYTES; subtitleParserFactory = new DefaultSubtitleParserFactory(); textTrackTranscodingEnabled = true; - codecsToParseWithinGopSampleDependencies = C.VIDEO_CODEC_FLAG_H264 | C.VIDEO_CODEC_FLAG_H265; + parseHagcMetadata = true; } /** @@ -406,15 +408,6 @@ public synchronized DefaultExtractorsFactory setSubtitleParserFactory( return this; } - @CanIgnoreReturnValue - @Override - public synchronized DefaultExtractorsFactory - experimentalSetCodecsToParseWithinGopSampleDependencies( - @C.VideoCodecFlags int codecsToParseWithinGopSampleDependencies) { - this.codecsToParseWithinGopSampleDependencies = codecsToParseWithinGopSampleDependencies; - return this; - } - /** * Sets flags for {@link JpegExtractor} instances created by the factory. * @@ -443,6 +436,13 @@ public synchronized DefaultExtractorsFactory setHeifExtractorFlags( return this; } + @CanIgnoreReturnValue + @Override + public synchronized DefaultExtractorsFactory setParseHagcMetadata(boolean parseHagcMetadata) { + this.parseHagcMetadata = parseHagcMetadata; + return this; + } + @Override public synchronized Extractor[] createExtractors() { return createExtractors(Uri.EMPTY, new HashMap<>()); @@ -524,7 +524,8 @@ private void addExtractorsForFileType(@FileTypes.Type int fileType, ListPCM encodings are not supported. + * + * @param encoding A {@link C.Encoding}. + * @param buffer The {@link ByteBuffer} containing the encoded sample. + * @return The number of audio frames per sample for the given encoding. + * @throws IllegalArgumentException If the sample data is invalid. + * @throws IllegalStateException If the encoding is not an encoded audio format, such as a PCM + * encoding. + */ + public static int getFramesPerEncodedSample(@C.Encoding int encoding, ByteBuffer buffer) { + switch (encoding) { + case C.ENCODING_MP3: + int headerDataInBigEndian = Util.getBigEndianInt(buffer, buffer.position()); + int frameCount = MpegAudioUtil.parseMpegAudioFrameSampleCount(headerDataInBigEndian); + if (frameCount == C.LENGTH_UNSET) { + throw new IllegalArgumentException(); + } + return frameCount; + case C.ENCODING_AAC_LC: + return AacUtil.AAC_LC_AUDIO_SAMPLE_COUNT; + case C.ENCODING_AAC_HE_V1: + case C.ENCODING_AAC_HE_V2: + return AacUtil.AAC_HE_AUDIO_SAMPLE_COUNT; + case C.ENCODING_AAC_XHE: + return AacUtil.AAC_XHE_AUDIO_SAMPLE_COUNT; + case C.ENCODING_AAC_ELD: + return AacUtil.AAC_LD_AUDIO_SAMPLE_COUNT; + case C.ENCODING_DTS: + case C.ENCODING_DTS_HD: + case C.ENCODING_DTS_UHD_P2: + return DtsUtil.parseDtsAudioSampleCount(buffer); + case C.ENCODING_AC3: + case C.ENCODING_E_AC3: + case C.ENCODING_E_AC3_JOC: + return Ac3Util.parseAc3SyncframeAudioSampleCount(buffer); + case C.ENCODING_AC4: + return Ac4Util.parseAc4SyncframeAudioSampleCount(buffer); + case C.ENCODING_DOLBY_TRUEHD: + int syncframeOffset = Ac3Util.findTrueHdSyncframeOffset(buffer); + return syncframeOffset == C.INDEX_UNSET + ? 0 + : (Ac3Util.parseTrueHdSyncframeAudioSampleCount(buffer, syncframeOffset) + * Ac3Util.TRUEHD_RECHUNK_SAMPLE_COUNT); + case C.ENCODING_OPUS: + return OpusUtil.parseOggPacketAudioSampleCount(buffer); + case C.ENCODING_PCM_16BIT: + case C.ENCODING_PCM_16BIT_BIG_ENDIAN: + case C.ENCODING_PCM_24BIT: + case C.ENCODING_PCM_24BIT_BIG_ENDIAN: + case C.ENCODING_PCM_32BIT: + case C.ENCODING_PCM_32BIT_BIG_ENDIAN: + case C.ENCODING_PCM_8BIT: + case C.ENCODING_PCM_FLOAT: + case C.ENCODING_PCM_FLOAT_BIG_ENDIAN: + case C.ENCODING_PCM_DOUBLE: + case C.ENCODING_PCM_DOUBLE_BIG_ENDIAN: + case C.ENCODING_AAC_ER_BSAC: + case C.ENCODING_DSD: + case C.ENCODING_INVALID: + case Format.NO_VALUE: + default: + throw new IllegalStateException("Unexpected audio encoding: " + encoding); + } + } + private ExtractorUtil() {} } diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/ExtractorsFactory.java b/libraries/extractor/src/main/java/androidx/media3/extractor/ExtractorsFactory.java index f165979009d..4e631fad71c 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/ExtractorsFactory.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/ExtractorsFactory.java @@ -16,11 +16,9 @@ package androidx.media3.extractor; import android.net.Uri; -import androidx.media3.common.C; import androidx.media3.common.MimeTypes; import androidx.media3.common.util.ExperimentalApi; import androidx.media3.common.util.UnstableApi; -import androidx.media3.extractor.mp4.Mp4Extractor; import androidx.media3.extractor.text.SubtitleParser; import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.util.List; @@ -71,22 +69,13 @@ default ExtractorsFactory setSubtitleParserFactory(SubtitleParser.Factory subtit } /** - * Sets the set of video codecs for which within GOP sample dependency information should be - * parsed as part of extraction. Defaults to H.264 and H.265. + * Sets whether to parse HAGC (ST 2094-50) metadata. * - *

    Having access to additional sample dependency information can speed up seeking. See {@link - * Mp4Extractor#FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES}. - * - *

    This method is experimental and will be renamed or removed in a future release. - * - * @param codecsToParseWithinGopSampleDependencies The set of codecs for which to parse within GOP - * sample dependency information. + * @param parseHagcMetadata Whether HAGC metadata parsing is enabled. * @return This factory, for convenience. */ @CanIgnoreReturnValue - @ExperimentalApi // TODO: b/470365670 - Remove method once config is enabled by default. - default ExtractorsFactory experimentalSetCodecsToParseWithinGopSampleDependencies( - @C.VideoCodecFlags int codecsToParseWithinGopSampleDependencies) { + default ExtractorsFactory setParseHagcMetadata(boolean parseHagcMetadata) { return this; } diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/ForwardingExtractorsFactory.java b/libraries/extractor/src/main/java/androidx/media3/extractor/ForwardingExtractorsFactory.java index ac53cfa1b51..e4cfde4bf27 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/ForwardingExtractorsFactory.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/ForwardingExtractorsFactory.java @@ -16,7 +16,6 @@ package androidx.media3.extractor; import android.net.Uri; -import androidx.media3.common.C; import androidx.media3.common.util.UnstableApi; import androidx.media3.extractor.text.SubtitleParser; import java.util.List; @@ -53,10 +52,8 @@ public ExtractorsFactory setSubtitleParserFactory(SubtitleParser.Factory subtitl } @Override - public ExtractorsFactory experimentalSetCodecsToParseWithinGopSampleDependencies( - @C.VideoCodecFlags int codecsToParseWithinGopSampleDependencies) { - return factory.experimentalSetCodecsToParseWithinGopSampleDependencies( - codecsToParseWithinGopSampleDependencies); + public ExtractorsFactory setParseHagcMetadata(boolean parseHagcMetadata) { + return factory.setParseHagcMetadata(parseHagcMetadata); } @Override diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/Id3Peeker.java b/libraries/extractor/src/main/java/androidx/media3/extractor/Id3Peeker.java index 0e1c699145a..1ebeafc6d85 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/Id3Peeker.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/Id3Peeker.java @@ -21,6 +21,7 @@ import androidx.annotation.Nullable; import androidx.media3.common.C; import androidx.media3.common.Metadata; +import androidx.media3.common.util.Log; import androidx.media3.common.util.ParsableByteArray; import androidx.media3.common.util.UnstableApi; import androidx.media3.extractor.metadata.id3.Id3Decoder; @@ -35,6 +36,8 @@ @UnstableApi public final class Id3Peeker { + private static final String TAG = "Id3Peeker"; + private final ParsableByteArray scratch; public Id3Peeker() { @@ -78,6 +81,16 @@ public Metadata peekId3Data( scratch.skipBytes(6); // Skip header, major version, minor version and flags. int framesLength = scratch.readSynchSafeInt(); int tagLength = ID3_HEADER_LENGTH + framesLength; + if (input.getLength() != C.LENGTH_UNSET + && input.getPeekPosition() + framesLength > input.getLength()) { + Log.w( + TAG, + "Skipping ID3 tag with tagLength=" + + tagLength + + " greater than input length=" + + input.getLength()); + break; + } if (metadata == null) { byte[] id3Data = new byte[tagLength]; @@ -129,7 +142,12 @@ private boolean peekId3HeaderIntoScratch(ExtractorInput input, int maxTagPeekByt scratch.setPosition(headerStartIndexInScratch); scratch.setLimit(headerEndIndexInScratch); if (scratch.peekUnsignedInt24() == Id3Decoder.ID3_TAG) { - return true; + scratch.skipBytes(3); + int majorVersion = scratch.readUnsignedByte(); + if (Id3Decoder.isSupportedMajorVersion(majorVersion)) { + scratch.setPosition(headerStartIndexInScratch); + return true; + } } else if (MpegAudioUtil.getFrameSize(scratch.peekInt()) != C.LENGTH_UNSET) { // We've found something that looks like an MP3 header, stop searching for ID3 headers. return false; diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/jpeg/MotionPhotoDescription.java b/libraries/extractor/src/main/java/androidx/media3/extractor/MotionPhotoDescription.java similarity index 95% rename from libraries/extractor/src/main/java/androidx/media3/extractor/jpeg/MotionPhotoDescription.java rename to libraries/extractor/src/main/java/androidx/media3/extractor/MotionPhotoDescription.java index 74ea06650ac..66af551efea 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/jpeg/MotionPhotoDescription.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/MotionPhotoDescription.java @@ -13,16 +13,20 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package androidx.media3.extractor.jpeg; +package androidx.media3.extractor; + +import static androidx.annotation.RestrictTo.Scope.LIBRARY; import androidx.annotation.Nullable; +import androidx.annotation.RestrictTo; import androidx.media3.common.C; import androidx.media3.common.MimeTypes; import androidx.media3.extractor.metadata.MotionPhotoMetadata; import java.util.List; /** Describes the layout and metadata of a motion photo file. */ -/* package */ final class MotionPhotoDescription { +@RestrictTo(LIBRARY) +public final class MotionPhotoDescription { /** Describes a media item in the motion photo. */ public static final class ContainerItem { diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/jpeg/XmpMotionPhotoDescriptionParser.java b/libraries/extractor/src/main/java/androidx/media3/extractor/XmpMotionPhotoDescriptionParser.java similarity index 97% rename from libraries/extractor/src/main/java/androidx/media3/extractor/jpeg/XmpMotionPhotoDescriptionParser.java rename to libraries/extractor/src/main/java/androidx/media3/extractor/XmpMotionPhotoDescriptionParser.java index 6d1dbc3ce73..0b1168a2f12 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/jpeg/XmpMotionPhotoDescriptionParser.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/XmpMotionPhotoDescriptionParser.java @@ -13,9 +13,12 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package androidx.media3.extractor.jpeg; +package androidx.media3.extractor; + +import static androidx.annotation.RestrictTo.Scope.LIBRARY; import androidx.annotation.Nullable; +import androidx.annotation.RestrictTo; import androidx.media3.common.C; import androidx.media3.common.MimeTypes; import androidx.media3.common.ParserException; @@ -33,7 +36,8 @@ * Parser for motion photo metadata, handling XMP following the Motion Photo V1 and Micro Video V1b * specifications. */ -/* package */ final class XmpMotionPhotoDescriptionParser { +@RestrictTo(LIBRARY) +public final class XmpMotionPhotoDescriptionParser { /** * Attempts to parse the specified XMP data describing the motion photo, returning the resulting diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/heif/HeicMotionPhotoExtractor.java b/libraries/extractor/src/main/java/androidx/media3/extractor/heif/HeicMotionPhotoExtractor.java index f241ec387a1..2d31f383b61 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/heif/HeicMotionPhotoExtractor.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/heif/HeicMotionPhotoExtractor.java @@ -31,12 +31,15 @@ import androidx.media3.extractor.Extractor; import androidx.media3.extractor.ExtractorInput; import androidx.media3.extractor.ExtractorOutput; +import androidx.media3.extractor.MotionPhotoDescription; import androidx.media3.extractor.PositionHolder; import androidx.media3.extractor.SeekMap; import androidx.media3.extractor.StartOffsetExtractorInput; import androidx.media3.extractor.StartOffsetExtractorOutput; import androidx.media3.extractor.TrackOutput; +import androidx.media3.extractor.XmpMotionPhotoDescriptionParser; import androidx.media3.extractor.metadata.MotionPhotoMetadata; +import androidx.media3.extractor.mp4.BoxParser; import androidx.media3.extractor.mp4.Mp4Extractor; import androidx.media3.extractor.text.SubtitleParser; import java.io.IOException; @@ -63,6 +66,7 @@ STATE_SNIFFING_MOTION_PHOTO_VIDEO, STATE_READING_MOTION_PHOTO_VIDEO, STATE_ENDED, + STATE_READING_XMP, }) private @interface State {} @@ -70,7 +74,8 @@ private static final int STATE_READING_ATOM_PAYLOAD = 1; private static final int STATE_SNIFFING_MOTION_PHOTO_VIDEO = 2; private static final int STATE_READING_MOTION_PHOTO_VIDEO = 3; - private static final int STATE_ENDED = 4; + private static final int STATE_READING_XMP = 4; + private static final int STATE_ENDED = 5; private final ParsableByteArray scratch; @@ -85,11 +90,18 @@ private long atomSize; private int atomHeaderBytesRead; private long mp4StartPosition; + private int mpvdHeaderSize; + + private long xmpOffset; + private int xmpLength; + @Nullable private ParsableByteArray iinfPayload; + @Nullable private ParsableByteArray ilocPayload; /** Creates an instance. */ public HeicMotionPhotoExtractor() { scratch = new ParsableByteArray(Mp4Box.LONG_HEADER_SIZE); mp4StartPosition = C.INDEX_UNSET; + xmpOffset = C.INDEX_UNSET; state = STATE_READING_ATOM_HEADER; } @@ -116,6 +128,15 @@ public int read(ExtractorInput input, PositionHolder seekPosition) throws IOExce case STATE_READING_ATOM_PAYLOAD: readAtomPayload(input); break; + case STATE_READING_XMP: + if (input.getPosition() != xmpOffset) { + seekPosition.position = xmpOffset; + return RESULT_SEEK; + } + readXmpAndSetupMetadata(input); + seekPosition.position = mp4StartPosition; + state = STATE_SNIFFING_MOTION_PHOTO_VIDEO; + return RESULT_SEEK; case STATE_SNIFFING_MOTION_PHOTO_VIDEO: sniffMotionPhotoVideo(input); break; @@ -135,6 +156,10 @@ public void seek(long position, long timeUs) { state = STATE_READING_ATOM_HEADER; atomHeaderBytesRead = 0; mp4StartPosition = C.INDEX_UNSET; + xmpOffset = C.INDEX_UNSET; + xmpLength = 0; + iinfPayload = null; + ilocPayload = null; if (mp4Extractor != null) { mp4Extractor.release(); mp4Extractor = null; @@ -173,16 +198,31 @@ private boolean readAtomHeader(ExtractorInput input) throws IOException { if (atomType == Mp4Box.TYPE_mpvd) { mp4StartPosition = input.getPosition(); - long boxStartPosition = mp4StartPosition - atomHeaderBytesRead; - motionPhotoMetadata = - new MotionPhotoMetadata( - /* photoStartPosition= */ 0, - /* photoSize= */ boxStartPosition, - /* photoPresentationTimestampUs= */ C.TIME_UNSET, - /* videoStartPosition= */ mp4StartPosition, - /* videoSize= */ atomSize - atomHeaderBytesRead); - outputImageTrack(motionPhotoMetadata); - state = STATE_SNIFFING_MOTION_PHOTO_VIDEO; + mpvdHeaderSize = atomHeaderBytesRead; + atomHeaderBytesRead = 0; + processIinfAndIloc(); + if (xmpOffset != C.INDEX_UNSET) { + state = STATE_READING_XMP; + } else { + setupMotionPhotoMetadata(C.TIME_UNSET); + state = STATE_SNIFFING_MOTION_PHOTO_VIDEO; + } + } else if (atomType == Mp4Box.TYPE_meta) { + input.skipFully(4); // Skip version and flags + atomHeaderBytesRead = 0; + state = STATE_READING_ATOM_HEADER; + } else if (atomType == Mp4Box.TYPE_iinf) { + int payloadSize = (int) (atomSize - atomHeaderBytesRead); + iinfPayload = new ParsableByteArray(payloadSize); + input.readFully(iinfPayload.getData(), 0, payloadSize); + atomHeaderBytesRead = 0; + state = STATE_READING_ATOM_HEADER; + } else if (atomType == Mp4Box.TYPE_iloc) { + int payloadSize = (int) (atomSize - atomHeaderBytesRead); + ilocPayload = new ParsableByteArray(payloadSize); + input.readFully(ilocPayload.getData(), 0, payloadSize); + atomHeaderBytesRead = 0; + state = STATE_READING_ATOM_HEADER; } else { state = STATE_READING_ATOM_PAYLOAD; } @@ -228,6 +268,143 @@ private int readMotionPhotoVideo(ExtractorInput input, PositionHolder seekPositi return readResult; } + private void setupMotionPhotoMetadata(long photoPresentationTimestampUs) { + long boxStartPosition = mp4StartPosition - mpvdHeaderSize; + long mpvdPayloadSize = atomSize - mpvdHeaderSize; + motionPhotoMetadata = + new MotionPhotoMetadata( + /* photoStartPosition= */ 0, + /* photoSize= */ boxStartPosition, + photoPresentationTimestampUs, + /* videoStartPosition= */ mp4StartPosition, + /* videoSize= */ mpvdPayloadSize); + outputImageTrack(motionPhotoMetadata); + } + + private void readXmpAndSetupMetadata(ExtractorInput input) throws IOException { + ParsableByteArray xmpData = new ParsableByteArray(xmpLength); + input.readFully(xmpData.getData(), 0, xmpLength); + String xmpString = xmpData.readString(xmpLength); + long timestampUs = C.TIME_UNSET; + try { + MotionPhotoDescription desc = XmpMotionPhotoDescriptionParser.parse(xmpString); + if (desc != null) { + timestampUs = desc.photoPresentationTimestampUs; + } + } catch (IOException e) { + // Ignore parser exception + } + setupMotionPhotoMetadata(timestampUs); + } + + private void processIinfAndIloc() { + ParsableByteArray iinfPayload = this.iinfPayload; + ParsableByteArray ilocPayload = this.ilocPayload; + if (iinfPayload == null || ilocPayload == null) { + return; + } + int xmpItemId = -1; + iinfPayload.setPosition(0); + int version = BoxParser.parseFullBoxVersion(iinfPayload.readInt()); + int entryCount = + version == 0 + ? iinfPayload.readUnsignedShort() + : iinfPayload.readUnsignedIntToInt(); // entry_count + for (int i = 0; i < entryCount; i++) { + if (iinfPayload.bytesLeft() < 8) { + break; + } + int start = iinfPayload.getPosition(); + int size = iinfPayload.readInt(); + if (size < Mp4Box.HEADER_SIZE) { + break; + } + int type = iinfPayload.readInt(); + if (type == Mp4Box.TYPE_infe) { + // ISO/IEC 14496-12:2022 8.11.6 Item Info Entry + int infeVersion = BoxParser.parseFullBoxVersion(iinfPayload.readInt()); + int itemId; // item_ID + if (infeVersion < 3) { + itemId = iinfPayload.readUnsignedShort(); + } else { + itemId = iinfPayload.readInt(); + } + if (infeVersion >= 2) { + iinfPayload.skipBytes(2); // item_protection_index + int itemType = iinfPayload.readInt(); // item_type + iinfPayload.readNullTerminatedString(); // item_name + if (itemType == 0x6d696d65) { // 'mime' + String contentType = iinfPayload.readNullTerminatedString(); // content_type + if (contentType != null && contentType.equals(MimeTypes.APPLICATION_RDF_XML)) { + xmpItemId = itemId; + break; + } + } + } + } + iinfPayload.setPosition(start + size); + } + + if (xmpItemId == -1) { + return; + } + + // ISO/IEC 14496-12:2022 8.11.3 Item Location Box + ilocPayload.setPosition(0); + int ilocVersion = BoxParser.parseFullBoxVersion(ilocPayload.readInt()); + int offsetAndLengthSize = ilocPayload.readUnsignedByte(); // offset_size (4) and length_size (4) + int offsetSize = offsetAndLengthSize >> 4; + int lengthSize = offsetAndLengthSize & 0x0F; + int baseOffsetSizeAndIndexSize = + ilocPayload.readUnsignedByte(); // base_offset_size (4) and index_size (4) + int baseOffsetSize = baseOffsetSizeAndIndexSize >> 4; + int indexSize = baseOffsetSizeAndIndexSize & 0x0F; + int itemCount = + ilocVersion < 2 ? ilocPayload.readUnsignedShort() : ilocPayload.readInt(); // item_count + for (int i = 0; i < itemCount; i++) { + int itemId = + ilocVersion < 2 ? ilocPayload.readUnsignedShort() : ilocPayload.readInt(); // item_ID + if (ilocVersion == 1 || ilocVersion == 2) { + ilocPayload.skipBytes(2); // reserved (12) and construction_method (4) + } + ilocPayload.skipBytes(2); // data_reference_index + long baseOffset = 0; // base_offset + if (baseOffsetSize == 4) { + baseOffset = ilocPayload.readUnsignedInt(); + } else if (baseOffsetSize == 8) { + baseOffset = ilocPayload.readUnsignedLongToLong(); + } + int extentCount = ilocPayload.readUnsignedShort(); // extent_count + for (int j = 0; j < extentCount; j++) { + if (ilocVersion == 1 || ilocVersion == 2) { + if (indexSize > 0) { + ilocPayload.skipBytes(indexSize); // item_reference_index + } + } + long extentOffset = 0; // extent_offset + if (offsetSize == 4) { + extentOffset = ilocPayload.readUnsignedInt(); + } else if (offsetSize == 8) { + extentOffset = ilocPayload.readUnsignedLongToLong(); + } + long extentLength = 0; // extent_length + if (lengthSize == 4) { + extentLength = ilocPayload.readUnsignedInt(); + } else if (lengthSize == 8) { + extentLength = ilocPayload.readUnsignedLongToLong(); + } + if (itemId == xmpItemId) { + xmpOffset = baseOffset + extentOffset; + xmpLength = (int) extentLength; + return; + } + } + if (itemId == xmpItemId) { + break; + } + } + } + private void outputImageTrack(MotionPhotoMetadata motionPhotoMetadata) { TrackOutput imageTrackOutput = checkNotNull(extractorOutput).track(IMAGE_TRACK_ID, C.TRACK_TYPE_IMAGE); diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/jpeg/JpegMotionPhotoExtractor.java b/libraries/extractor/src/main/java/androidx/media3/extractor/jpeg/JpegMotionPhotoExtractor.java index b32ae0b892e..274778dfd15 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/jpeg/JpegMotionPhotoExtractor.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/jpeg/JpegMotionPhotoExtractor.java @@ -30,11 +30,13 @@ import androidx.media3.extractor.Extractor; import androidx.media3.extractor.ExtractorInput; import androidx.media3.extractor.ExtractorOutput; +import androidx.media3.extractor.MotionPhotoDescription; import androidx.media3.extractor.PositionHolder; import androidx.media3.extractor.SeekMap; import androidx.media3.extractor.StartOffsetExtractorInput; import androidx.media3.extractor.StartOffsetExtractorOutput; import androidx.media3.extractor.TrackOutput; +import androidx.media3.extractor.XmpMotionPhotoDescriptionParser; import androidx.media3.extractor.metadata.MotionPhotoMetadata; import androidx.media3.extractor.mp4.Mp4Extractor; import androidx.media3.extractor.text.SubtitleParser; diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/metadata/id3/Id3Decoder.java b/libraries/extractor/src/main/java/androidx/media3/extractor/metadata/id3/Id3Decoder.java index 55912eb131e..d8fba7652f5 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/metadata/id3/Id3Decoder.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/metadata/id3/Id3Decoder.java @@ -105,6 +105,16 @@ public Id3Decoder(@Nullable FramePredicate framePredicate) { this.framePredicate = framePredicate; } + /** + * Returns whether the given major version number is supported by {@link Id3Decoder}. + * + * @param majorVersion The ID3v2 major version number. + * @return Whether the major version is supported (ID3v2.2, ID3v2.3, or ID3v2.4). + */ + public static boolean isSupportedMajorVersion(int majorVersion) { + return majorVersion >= 2 && majorVersion <= 4; + } + @Override @Nullable @SuppressWarnings("ByteBufferBackingArray") // Buffer validated by SimpleMetadataDecoder.decode @@ -183,6 +193,10 @@ private static Id3Header decodeHeader(ParsableByteArray data) { } int majorVersion = data.readUnsignedByte(); + if (!isSupportedMajorVersion(majorVersion)) { + Log.w(TAG, "Skipped ID3 tag with unsupported majorVersion=" + majorVersion); + return null; + } data.skipBytes(1); // Skip minor version. int flags = data.readUnsignedByte(); int framesSize = data.readSynchSafeInt(); @@ -211,9 +225,6 @@ private static Id3Header decodeHeader(ParsableByteArray data) { if (hasFooter) { framesSize -= 10; } - } else { - Log.w(TAG, "Skipped ID3 tag with unsupported majorVersion=" + majorVersion); - return null; } // isUnsynchronized is advisory only in version 4. Frame level flags are used instead. diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/mkv/MatroskaExtractor.java b/libraries/extractor/src/main/java/androidx/media3/extractor/mkv/MatroskaExtractor.java index 6801755c364..4fc7cf7aa8c 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/mkv/MatroskaExtractor.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/mkv/MatroskaExtractor.java @@ -37,6 +37,7 @@ import androidx.media3.common.Metadata; import androidx.media3.common.MimeTypes; import androidx.media3.common.ParserException; +import androidx.media3.common.util.CodecSpecificDataUtil; import androidx.media3.common.util.Log; import androidx.media3.common.util.NullableType; import androidx.media3.common.util.ParsableByteArray; @@ -75,6 +76,7 @@ import java.lang.annotation.Target; import java.nio.ByteBuffer; import java.nio.ByteOrder; +import java.util.ArrayDeque; import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; @@ -109,7 +111,7 @@ public static ExtractorsFactory newFactory(SubtitleParser.Factory subtitleParser @Target(TYPE_USE) @IntDef( flag = true, - value = {FLAG_DISABLE_SEEK_FOR_CUES, FLAG_EMIT_RAW_SUBTITLE_DATA}) + value = {FLAG_DISABLE_SEEK_FOR_CUES, FLAG_EMIT_RAW_SUBTITLE_DATA, FLAG_DISABLE_HAGC_METADATA}) public @interface Flags {} /** @@ -128,6 +130,9 @@ public static ExtractorsFactory newFactory(SubtitleParser.Factory subtitleParser */ public static final int FLAG_EMIT_RAW_SUBTITLE_DATA = 1 << 1; // 2 + /** Flag to disable parsing of HAGC (ST 2094-50) metadata. */ + public static final int FLAG_DISABLE_HAGC_METADATA = 1 << 2; // 4 + /** * @deprecated Use {@link #newFactory(SubtitleParser.Factory)} instead. */ @@ -453,6 +458,7 @@ public static ExtractorsFactory newFactory(SubtitleParser.Factory subtitleParser private final SparseArray tracks; private final LongSparseArray chapters; private final boolean seekForCuesEnabled; + private final boolean parseHagcMetadata; private final boolean parseSubtitlesDuringExtraction; private final SubtitleParser.Factory subtitleParserFactory; @@ -477,8 +483,8 @@ public static ExtractorsFactory newFactory(SubtitleParser.Factory subtitleParser private boolean isWebm; private boolean pendingEndTracks; - // The chapter corresponding to the current EditionEntry element, or null. - @Nullable private ChapterEntry currentChapter; + // Stack of ChapterAtom elements currently being parsed (to support nested ChapterAtom elements). + private final ArrayDeque currentChapters; // The track corresponding to the current TrackEntry element, or null. @Nullable private Track currentTrack; @@ -586,8 +592,10 @@ public MatroskaExtractor(SubtitleParser.Factory subtitleParserFactory, @Flags in this.perTrackCues = new SparseArray<>(); seekForCuesEnabled = (flags & FLAG_DISABLE_SEEK_FOR_CUES) == 0; parseSubtitlesDuringExtraction = (flags & FLAG_EMIT_RAW_SUBTITLE_DATA) == 0; + parseHagcMetadata = (flags & FLAG_DISABLE_HAGC_METADATA) == 0; varintReader = new VarintReader(); chapters = new LongSparseArray<>(); + currentChapters = new ArrayDeque<>(); tracks = new SparseArray<>(); scratch = new ParsableByteArray(4); vorbisNumPageSamples = new ParsableByteArray(ByteBuffer.allocate(4).putInt(-1).array()); @@ -624,6 +632,7 @@ public void seek(long position, long timeUs) { reader.reset(); varintReader.reset(); resetWriteSampleData(); + currentChapters.clear(); inCuesElement = false; currentCueTimeUs = C.TIME_UNSET; currentCueTrackNumber = C.INDEX_UNSET; @@ -852,18 +861,8 @@ protected void startMasterElement(int id, long contentPosition, long contentSize case ID_CLUSTER: if (tracksContentPosition != C.INDEX_UNSET && !readTracks) { seekForTracks = true; - } - if (!sentSeekMap) { - // We need to build cues before parsing the cluster. - if (seekForCuesEnabled && cuesContentPosition != C.INDEX_UNSET) { - // We know where the Cues element is located. Seek to request it. - seekForCues = true; - } else { - // We don't know where the Cues element is located. It's most likely omitted. Allow - // playback, but disable seeking. - extractorOutput.seekMap(new SeekMap.Unseekable(durationUs)); - sentSeekMap = true; - } + } else { + maybePrepareSeekMap(); } break; case ID_BLOCK_GROUP: @@ -877,7 +876,7 @@ protected void startMasterElement(int id, long contentPosition, long contentSize getCurrentTrack(id).hasContentEncryption = true; break; case ID_CHAPTER_ATOM: - currentChapter = new ChapterEntry(); + currentChapters.push(new ChapterEntry()); break; case ID_CHAPTER_DISPLAY: getCurrentChapter(id).currentDisplayString = null; @@ -979,14 +978,13 @@ protected void endMasterElement(int id) throws ParserException { } break; case ID_CHAPTER_ATOM: - ChapterEntry chapter = checkNotNull(currentChapter); + ChapterEntry chapter = checkNotNull(currentChapters.poll()); if (chapter.uid != 0) { chapters.put(chapter.uid, chapter); } - currentChapter = null; break; case ID_CHAPTER_DISPLAY: - ChapterEntry chapterEntry = checkNotNull(currentChapter); + ChapterEntry chapterEntry = getCurrentChapter(id); if (chapterEntry.chapString == null && chapterEntry.currentDisplayString != null) { chapterEntry.chapString = chapterEntry.currentDisplayString; if (chapterEntry.currentDisplayLanguage != null) { @@ -1127,6 +1125,9 @@ protected void endMasterElement(int id) throws ParserException { } readTracks = true; + if (seekPositionAfterReadingTracks != C.INDEX_UNSET) { + maybePrepareSeekMap(); + } if (maybeSendFormatsEarly) { maybeEndTracks(); } @@ -1719,20 +1720,18 @@ protected void handleBlockAdditionalData( && CODEC_ID_VP9.equals(track.codecId)) { supplementalData.reset(contentSize); input.readFully(supplementalData.getData(), 0, contentSize); + if (!parseHagcMetadata) { + byte[] data = supplementalData.getData(); + if (CodecSpecificDataUtil.isHagcMetadata(data, contentSize)) { + supplementalData.reset(0); + } + } } else { // Unhandled block additional data. input.skipFully(contentSize); } } - @EnsuresNonNull("currentChapter") - private void assertInEditionEntry(int id) throws ParserException { - if (currentChapter == null) { - throw ParserException.createForMalformedContainer( - "Element " + id + " must be in an EditionEntry", /* cause= */ null); - } - } - @EnsuresNonNull("currentTrack") private void assertInTrackEntry(int id) throws ParserException { if (currentTrack == null) { @@ -1749,12 +1748,16 @@ private void assertInCues(int id) throws ParserException { } /** - * Returns the chapter corresponding to the current EditionEntry element. + * Returns the chapter corresponding to the current ChapterAtom element. * - * @throws ParserException if the element id is not in an EditionEntry. + * @throws ParserException if the element id is not in a ChapterAtom. */ protected ChapterEntry getCurrentChapter(int currentElementId) throws ParserException { - assertInEditionEntry(currentElementId); + @Nullable ChapterEntry currentChapter = currentChapters.peek(); + if (currentChapter == null) { + throw ParserException.createForMalformedContainer( + "Element " + currentElementId + " must be in a ChapterAtom", /* cause= */ null); + } return currentChapter; } @@ -2289,6 +2292,22 @@ private void assertInitialized() { checkNotNull(extractorOutput); } + private void maybePrepareSeekMap() { + if (sentSeekMap) { + return; // Already evaluated + } + // We need to build cues before parsing the cluster. + if (seekForCuesEnabled && cuesContentPosition != C.INDEX_UNSET) { + // We know where the Cues element is located. Seek to request it. + seekForCues = true; + } else { + // We don't know where the Cues element is located. It's most likely omitted. Allow + // playback, but disable seeking. + checkNotNull(extractorOutput).seekMap(new SeekMap.Unseekable(durationUs)); + sentSeekMap = true; + } + } + private void maybeEndTracks() { if (!pendingEndTracks) { return; @@ -2700,13 +2719,10 @@ public void initializeFormat(int trackId) throws ParserException { } if (dolbyVisionConfigBytes != null) { - @Nullable DolbyVisionConfig dolbyVisionConfig = DolbyVisionConfig.parse(new ParsableByteArray(this.dolbyVisionConfigBytes)); - if (dolbyVisionConfig != null) { - codecs = dolbyVisionConfig.codecs; - mimeType = MimeTypes.VIDEO_DOLBY_VISION; - } + codecs = dolbyVisionConfig.codecs; + mimeType = MimeTypes.VIDEO_DOLBY_VISION; } @C.SelectionFlags int selectionFlags = 0; diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/mp3/XingFrame.java b/libraries/extractor/src/main/java/androidx/media3/extractor/mp3/XingFrame.java index ed6565c0ac1..4fbaf984a47 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/mp3/XingFrame.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/mp3/XingFrame.java @@ -25,13 +25,6 @@ /** Representation of a LAME Xing or Info frame. */ /* package */ final class XingFrame { - /** - * Offset between LAME/Xing delay/padding fields and decoded PCM trim samples. FFmpeg's MP3 muxer - * subtracts this offset when writing LAME metadata, and its demuxer adds it back when exposing - * decoded PCM skip/discard samples. - */ - private static final int LAME_TO_DECODED_PCM_TRIM_OFFSET_SAMPLES = 528 + 1; - /** The header of the Xing or Info frame. */ public final MpegAudioUtil.Header header; @@ -114,14 +107,12 @@ public static XingFrame parse(MpegAudioUtil.Header mpegAudioHeader, ParsableByte @Nullable Mp3InfoReplayGain replayGain; int encoderDelay; int encoderPadding; - // Skip: revision & VBR method (1), lowpass filter (1). - int bytesToSkipBeforeReplayGain = 1 + 1; + // Skip: version string (9), revision & VBR method (1), lowpass filter (1). + int bytesToSkipBeforeReplayGain = 9 + 1 + 1; // Skip: encoding flags & ATH type (1), bitrate (1). int bytesToSkipAfterReplayGain = 1 + 1; - // And account for values we parse: version string (9), ReplayGain (8) and encoder delay & - // padding (3). - if (frame.bytesLeft() >= 9 + bytesToSkipBeforeReplayGain + 8 + bytesToSkipAfterReplayGain + 3) { - String encoderIdentifier = frame.readString(9); + // And account for values we parse, ReplayGain (8) and encoder delay & padding (3). + if (frame.bytesLeft() >= bytesToSkipBeforeReplayGain + 8 + bytesToSkipAfterReplayGain + 3) { frame.skipBytes(bytesToSkipBeforeReplayGain); float peak = frame.readFloat(); int field1 = frame.readUnsignedShort(); @@ -132,10 +123,6 @@ public static XingFrame parse(MpegAudioUtil.Header mpegAudioHeader, ParsableByte int encoderDelayAndPadding = frame.readUnsignedInt24(); encoderDelay = (encoderDelayAndPadding & 0xFFF000) >> 12; encoderPadding = (encoderDelayAndPadding & 0xFFF); - if (usesLameGaplessEncoding(encoderIdentifier)) { - encoderDelay += LAME_TO_DECODED_PCM_TRIM_OFFSET_SAMPLES; - encoderPadding = Math.max(0, encoderPadding - LAME_TO_DECODED_PCM_TRIM_OFFSET_SAMPLES); - } } else { replayGain = null; encoderDelay = C.LENGTH_UNSET; @@ -174,12 +161,6 @@ public long computeDurationUs() { return Util.sampleCountToDurationUs(sampleCount - 1, header.sampleRate); } - private static boolean usesLameGaplessEncoding(String encoderIdentifier) { - return encoderIdentifier.startsWith("LAME") - || encoderIdentifier.startsWith("Lavf") - || encoderIdentifier.startsWith("Lavc"); - } - /** Provide the metadata derived from this Xing frame, such as ReplayGain data. */ @Nullable public Metadata getMetadata() { diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/mp4/BoxParser.java b/libraries/extractor/src/main/java/androidx/media3/extractor/mp4/BoxParser.java index c994e35a441..ca8cc757ec1 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/mp4/BoxParser.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/mp4/BoxParser.java @@ -25,6 +25,7 @@ import android.util.Pair; import androidx.annotation.Nullable; +import androidx.annotation.VisibleForTesting; import androidx.media3.common.C; import androidx.media3.common.ColorInfo; import androidx.media3.common.DrmInitData; @@ -72,6 +73,8 @@ import java.util.Arrays; import java.util.List; import java.util.Objects; +import java.util.regex.Matcher; +import java.util.regex.Pattern; /** Utility methods for parsing MP4 format box payloads according to ISO/IEC 14496-12. */ @SuppressWarnings("ConstantField") @@ -758,18 +761,15 @@ public static TrackSampleTable parseStbl( if (omitTrackSampleTable) { long editedDurationUs; - ImmutableLongArray editListMediaTimes = checkNotNull(track.editListMediaTimes); if (track.editListDurations.length() == 1 && track.editListDurations.get(0) == 0) { - long editStartTime = editListMediaTimes.get(0); + long editStartTime = checkNotNull(track.editListMediaTimes).get(0); editedDurationUs = Util.scaleLargeTimestamp( duration - editStartTime, C.MICROS_PER_SECOND, track.timescale); } else { long pts = 0; for (int i = 0; i < track.editListDurations.length(); i++) { - if (editListMediaTimes.get(i) != -1) { - pts += track.editListDurations.get(i); - } + pts += track.editListDurations.get(i); } editedDurationUs = Util.scaleLargeTimestamp(pts, C.MICROS_PER_SECOND, track.movieTimescale); } @@ -953,7 +953,6 @@ public static TrackSampleTable parseStbl( long[] editedTimestamps = new long[editedSampleCount]; long pts = 0; int sampleIndex = 0; - boolean hasPrerollSamples = false; for (int i = 0; i < track.editListDurations.length(); i++) { long editMediaTime = track.editListMediaTimes.get(i); int startIndex = startIndices[i]; @@ -969,9 +968,6 @@ public static TrackSampleTable parseStbl( long timeInSegmentUs = Util.scaleLargeTimestamp( timestamps[j] - editMediaTime, C.MICROS_PER_SECOND, track.timescale); - if (timeInSegmentUs < 0) { - hasPrerollSamples = true; - } editedTimestamps[sampleIndex] = ptsUs + timeInSegmentUs; if (copyMetadata && editedSizes[sampleIndex] > editedMaximumSize) { editedMaximumSize = sizes[j]; @@ -987,10 +983,6 @@ public static TrackSampleTable parseStbl( } long editedDurationUs = Util.scaleLargeTimestamp(pts, C.MICROS_PER_SECOND, track.movieTimescale); - if (hasPrerollSamples) { - Format format = track.format.buildUpon().setHasPrerollSamples(true).build(); - track = track.buildUpon().setFormat(format).build(); - } return new TrackSampleTable( track, editedOffsets, @@ -1034,23 +1026,34 @@ private static Metadata parseIlst(ParsableByteArray ilst, int limit, boolean ign return entries.isEmpty() ? null : new Metadata(entries); } + private static final Pattern XYZ_PATTERN = + Pattern.compile("([+-][0-9.]+)([+-][0-9.]+)(?:([+-][0-9.]+))?(?:CRS[^/]*)?/.*"); + /** Parses the location metadata from the xyz atom. */ + @VisibleForTesting @Nullable - private static Metadata parseXyz(ParsableByteArray xyzBox) { + /* package */ static Metadata parseXyz(ParsableByteArray xyzBox) { int length = xyzBox.readShort(); xyzBox.skipBytes(2); // language code. String location = xyzBox.readString(length); - // The location string looks like "+35.1345-15.1020/". - int plusSignIndex = location.lastIndexOf('+'); - int minusSignIndex = location.lastIndexOf('-'); - int latitudeEndIndex = max(plusSignIndex, minusSignIndex); + // The location string looks like "+35.1345-15.1020/" or "+37.7749-122.4194+15.0000/". + Matcher matcher = XYZ_PATTERN.matcher(location); + if (!matcher.matches()) { + return null; + } + String latitudeString = castNonNull(matcher.group(1)); + String longitudeString = castNonNull(matcher.group(2)); try { - float latitude = Float.parseFloat(location.substring(0, latitudeEndIndex)); - float longitude = - Float.parseFloat(location.substring(latitudeEndIndex, location.length() - 1)); + float latitude = Float.parseFloat(latitudeString); + float longitude = Float.parseFloat(longitudeString); + @Nullable String altitudeString = matcher.group(3); + if (altitudeString != null) { + float altitude = Float.parseFloat(altitudeString); + return new Metadata(new Mp4LocationData(latitude, longitude, altitude)); + } return new Metadata(new Mp4LocationData(latitude, longitude)); - } catch (IndexOutOfBoundsException | NumberFormatException exception) { - // Invalid input. + } catch (IllegalArgumentException exception) { + // Invalid input (e.g. NumberFormatException or invalid latitude/longitude range). return null; } } @@ -1675,8 +1678,8 @@ private static void parseVideoSampleEntry( mimeType = (atomType == Mp4Box.TYPE_vp08) ? MimeTypes.VIDEO_VP8 : MimeTypes.VIDEO_VP9; parent.setPosition(childStartPosition + Mp4Box.FULL_HEADER_SIZE); // See vpcC atom syntax: https://www.webmproject.org/vp9/mp4/#syntax_1 - byte profile = (byte) parent.readUnsignedByte(); - byte level = (byte) parent.readUnsignedByte(); + byte profile = parent.readByte(); + byte level = parent.readByte(); int byte3 = parent.readUnsignedByte(); bitdepthLuma = byte3 >> 4; bitdepthChroma = bitdepthLuma; @@ -2251,9 +2254,15 @@ private static void parseAudioSampleEntry( out.format = Ac3Util.parseAc3AnnexFFormat(parent, Integer.toString(trackId), language, drmInitData); } else if (childAtomType == Mp4Box.TYPE_dec3) { - parent.setPosition(Mp4Box.HEADER_SIZE + childPosition); - out.format = + int childAtomBodySize = childAtomSize - Mp4Box.HEADER_SIZE; + byte[] dec3Payload = new byte[childAtomBodySize]; + parent.setPosition(childPosition + Mp4Box.HEADER_SIZE); + parent.readBytes(dec3Payload, 0, childAtomBodySize); + parent.setPosition(childPosition + Mp4Box.HEADER_SIZE); + Format parsedFormat = Ac3Util.parseEAc3AnnexFFormat(parent, Integer.toString(trackId), language, drmInitData); + out.format = + parsedFormat.buildUpon().setInitializationData(ImmutableList.of(dec3Payload)).build(); } else if (childAtomType == Mp4Box.TYPE_dac4) { parent.setPosition(Mp4Box.HEADER_SIZE + childPosition); out.format = @@ -2280,13 +2289,27 @@ private static void parseAudioSampleEntry( .setLanguage(language) .build(); } else if (childAtomType == Mp4Box.TYPE_dOps) { - // Build an Opus Identification Header (defined in RFC-7845) by concatenating the Opus Magic - // Signature and the body of the dOps atom. + // Build an Opus Identification Header (defined in RFC-7845 Section 5.1) from the dOps atom + // (defined in https://opus-codec.org/docs/opus_in_isobmff.html Section 4.3.2). int childAtomBodySize = childAtomSize - Mp4Box.HEADER_SIZE; - byte[] headerBytes = Arrays.copyOf(opusMagic, opusMagic.length + childAtomBodySize); - parent.setPosition(childPosition + Mp4Box.HEADER_SIZE); - parent.readBytes(headerBytes, opusMagic.length, childAtomBodySize); - initializationData = OpusUtil.buildInitializationData(headerBytes); + ExtractorUtil.checkContainerInput( + childAtomBodySize >= 11, "dOps box body is too short: " + childAtomBodySize); + // RFC-7845 OpusHead uses little-endian byte order, whereas the MP4 dOps atom uses + // big-endian. + ByteBuffer headerBytes = + ByteBuffer.allocate(opusMagic.length + childAtomBodySize).order(LITTLE_ENDIAN); + headerBytes.put(opusMagic); + headerBytes.put((byte) 1); // RFC-7845 OpusHead Version is 1. + // Skip dOps Version (1 byte) to read OutputChannelCount, PreSkip, InputSampleRate, and + // OutputGain. + parent.setPosition(childPosition + Mp4Box.HEADER_SIZE + 1); + headerBytes.put(parent.readByte()); // OutputChannelCount + headerBytes.putShort(parent.readShort()); // PreSkip + headerBytes.putInt(parent.readInt()); // InputSampleRate + headerBytes.putShort(parent.readShort()); // OutputGain + // Copy remaining bytes (ChannelMappingFamily and optional ChannelMappingTable). + parent.readBytes(headerBytes, childPosition + childAtomSize - parent.getPosition()); + initializationData = OpusUtil.buildInitializationData(headerBytes.array()); } else if (childAtomType == Mp4Box.TYPE_dfLa) { int childAtomBodySize = childAtomSize - Mp4Box.FULL_HEADER_SIZE; byte[] initializationDataBytes = new byte[4 + childAtomBodySize]; diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/mp4/FixedSampleSizeRechunker.java b/libraries/extractor/src/main/java/androidx/media3/extractor/mp4/FixedSampleSizeRechunker.java index 6ee36efbcdf..0079e103b43 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/mp4/FixedSampleSizeRechunker.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/mp4/FixedSampleSizeRechunker.java @@ -71,7 +71,9 @@ public static Results rechunk( long[] chunkOffsets, int[] chunkSampleCounts, long timestampDeltaInTimeUnits) { - int maxSampleCount = MAX_SAMPLE_SIZE / fixedSampleSize; + // Ensure maxSampleCount is at least 1 when fixedSampleSize exceeds MAX_SAMPLE_SIZE to prevent + // division by zero in rechunking. + int maxSampleCount = max(1, MAX_SAMPLE_SIZE / fixedSampleSize); // Count the number of new, rechunked buffers. int rechunkedSampleCount = 0; diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/mp4/FragmentedMp4Extractor.java b/libraries/extractor/src/main/java/androidx/media3/extractor/mp4/FragmentedMp4Extractor.java index 5dcc32c7392..342cb94e7c3 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/mp4/FragmentedMp4Extractor.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/mp4/FragmentedMp4Extractor.java @@ -35,6 +35,7 @@ import androidx.media3.common.Metadata; import androidx.media3.common.MimeTypes; import androidx.media3.common.ParserException; +import androidx.media3.common.util.CodecSpecificDataUtil; import androidx.media3.common.util.Log; import androidx.media3.common.util.ParsableByteArray; import androidx.media3.common.util.TimestampAdjuster; @@ -96,8 +97,7 @@ public static ExtractorsFactory newFactory(SubtitleParser.Factory subtitleParser * Flags controlling the behavior of the extractor. Possible flag values are {@link * #FLAG_WORKAROUND_EVERY_VIDEO_FRAME_IS_SYNC_FRAME}, {@link #FLAG_WORKAROUND_IGNORE_TFDT_BOX}, * {@link #FLAG_ENABLE_EMSG_TRACK}, {@link #FLAG_WORKAROUND_IGNORE_EDIT_LISTS}, {@link - * #FLAG_EMIT_RAW_SUBTITLE_DATA}, {@link #FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES}, {@link - * #FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES_H265}, {@link #FLAG_MERGE_FRAGMENTED_SIDX} and {@link + * #FLAG_EMIT_RAW_SUBTITLE_DATA}, {@link #FLAG_MERGE_FRAGMENTED_SIDX} and {@link * #FLAG_READ_MFRA_FOR_SEEK_MAP}. */ @Documented @@ -111,11 +111,10 @@ public static ExtractorsFactory newFactory(SubtitleParser.Factory subtitleParser FLAG_ENABLE_EMSG_TRACK, FLAG_WORKAROUND_IGNORE_EDIT_LISTS, FLAG_EMIT_RAW_SUBTITLE_DATA, - FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES, - FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES_H265, FLAG_MERGE_FRAGMENTED_SIDX, FLAG_READ_MFRA_FOR_SEEK_MAP, - FLAG_DISABLE_ARTWORK_METADATA + FLAG_DISABLE_ARTWORK_METADATA, + FLAG_DISABLE_HAGC_METADATA }) public @interface Flags {} @@ -146,28 +145,6 @@ public static ExtractorsFactory newFactory(SubtitleParser.Factory subtitleParser */ public static final int FLAG_EMIT_RAW_SUBTITLE_DATA = 1 << 5; // 32 - /** - * Flag to extract additional sample dependency information, and mark output buffers with {@link - * C#BUFFER_FLAG_NOT_DEPENDED_ON} for {@linkplain MimeTypes#VIDEO_H264 H.264} video. - * - *

    This class always marks the samples at the start of each group of picture (GOP) with {@link - * C#BUFFER_FLAG_KEY_FRAME}. Usually, key frames can be decoded independently, without depending - * on other samples. - * - *

    Setting this flag enables elementary stream parsing to identify disposable samples that are - * not depended on by other samples. Any disposable sample can be safely omitted, and the rest of - * the track will remain valid. - */ - public static final int FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES = 1 << 6; // 64 - - /** - * Flag to extract additional sample dependency information, and mark output buffers with {@link - * C#BUFFER_FLAG_NOT_DEPENDED_ON} for {@linkplain MimeTypes#VIDEO_H265 H.265} video. - * - *

    See {@link #FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES}. - */ - public static final int FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES_H265 = 1 << 7; - /** Flag to enable reading and merging of all sidx boxes before continuing extraction. */ public static final int FLAG_MERGE_FRAGMENTED_SIDX = 1 << 8; @@ -177,6 +154,9 @@ public static ExtractorsFactory newFactory(SubtitleParser.Factory subtitleParser /** Flag to disable parsing of artwork metadata. */ public static final int FLAG_DISABLE_ARTWORK_METADATA = 1 << 10; + /** Flag to disable parsing of HAGC (ST 2094-50) metadata. */ + public static final int FLAG_DISABLE_HAGC_METADATA = 1 << 11; + /** * @deprecated Use {@link #newFactory(SubtitleParser.Factory)} instead. */ @@ -464,23 +444,6 @@ public FragmentedMp4Extractor( seekPositionBeforeMfraProcessing = C.INDEX_UNSET; } - /** - * Returns {@link Flags} denoting if an extractor should parse within GOP sample dependencies. - * - * @param videoCodecFlags The set of codecs for which to parse within GOP sample dependencies. - */ - public static @Flags int codecsToParseWithinGopSampleDependenciesAsFlags( - @C.VideoCodecFlags int videoCodecFlags) { - @Flags int flags = 0; - if ((videoCodecFlags & C.VIDEO_CODEC_FLAG_H264) != 0) { - flags |= FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES; - } - if ((videoCodecFlags & C.VIDEO_CODEC_FLAG_H265) != 0) { - flags |= FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES_H265; - } - return flags; - } - @Override public boolean sniff(ExtractorInput input) throws IOException { @Nullable SniffFailure sniffFailure = Sniffer.sniffFragmented(input); @@ -853,7 +816,8 @@ private void onMoovContainerAtomRead(ContainerBox moov) throws ParserException { continue; } TrackOutput output = extractorOutput.track(i, track.type); - output.durationUs(track.durationUs); + long trackDurationUs = getTrackDurationUs(track, /* mehdDuration= */ duration); + output.durationUs(trackDurationUs); Format.Builder formatBuilder = track.format.buildUpon(); formatBuilder.setContainerMimeType(containerMimeType); MetadataUtil.setFormatGaplessInfo(track.type, gaplessInfoHolder, formatBuilder); @@ -871,7 +835,7 @@ private void onMoovContainerAtomRead(ContainerBox moov) throws ParserException { getDefaultSampleValues(defaultSampleValuesArray, track.id), formatBuilder.build()); trackBundles.put(track.id, trackBundle); - durationUs = max(durationUs, track.durationUs); + durationUs = max(durationUs, trackDurationUs); } extractorOutput.endTracks(); } else { @@ -895,8 +859,32 @@ private void onMoovContainerAtomRead(ContainerBox moov) throws ParserException { } } + /** + * Returns the duration of the track in microseconds. + * + *

    If {@code mehd} duration is unset and no edit list is applied to the samples, the {@code + * mdhd} duration is preferred because it is expressed in the track's own timescale, which is + * typically finer than the movie timescale used for the {@code tkhd} duration. Otherwise, {@link + * Track#durationUs} (from {@code mehd} or {@code tkhd}) is used. + */ + private static long getTrackDurationUs(Track track, long mehdDuration) { + if (mehdDuration == C.TIME_UNSET + && track.editListDurations == null + && track.mediaDurationUs != C.TIME_UNSET) { + return track.mediaDurationUs; + } + return track.durationUs; + } + @Nullable protected Track modifyTrack(@Nullable Track track) { + if (track == null) { + return null; + } + if ((flags & FLAG_DISABLE_HAGC_METADATA) != 0 + && CodecSpecificDataUtil.isHagcTrack(track.format)) { + return null; + } return track; } @@ -1318,7 +1306,12 @@ private static long parseTfdt(ParsableByteArray tfdt) { tfdt.setPosition(Mp4Box.HEADER_SIZE); int fullAtom = tfdt.readInt(); int version = BoxParser.parseFullBoxVersion(fullAtom); - return version == 1 ? tfdt.readUnsignedLongToLong() : tfdt.readUnsignedInt(); + // The BMFF spec (ISO/IEC 14496-12) states that baseMediaDecodeTime should be an + // unsigned integer, however some streams violate the spec and use signed integers in + // version 1 tfdt boxes. It's safe to always decode version 1 values as signed integers + // here, because unsigned integers will still be parsed correctly (unless their top bit + // is set, which is never true in practice). + return version == 1 ? tfdt.readLong() : tfdt.readUnsignedInt(); } private static boolean isEdtsListDurationForEntireMediaTimeline(Track track) { @@ -1899,14 +1892,9 @@ private boolean readSample(ExtractorInput input) throws IOException { * Returns whether reading within GOP sample dependencies is enabled for the sample {@link * Format}. */ - private boolean canReadWithinGopSampleDependencies(Format format) { - if (Objects.equals(format.sampleMimeType, MimeTypes.VIDEO_H264)) { - return (flags & FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES) != 0; - } - if (Objects.equals(format.sampleMimeType, MimeTypes.VIDEO_H265)) { - return (flags & FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES_H265) != 0; - } - return false; + private static boolean canReadWithinGopSampleDependencies(Format format) { + return Objects.equals(format.sampleMimeType, MimeTypes.VIDEO_H264) + || Objects.equals(format.sampleMimeType, MimeTypes.VIDEO_H265); } /** diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/mp4/Mp4Extractor.java b/libraries/extractor/src/main/java/androidx/media3/extractor/mp4/Mp4Extractor.java index e4f1ca810e9..55f75ba3c1d 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/mp4/Mp4Extractor.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/mp4/Mp4Extractor.java @@ -37,6 +37,7 @@ import androidx.media3.common.Metadata; import androidx.media3.common.MimeTypes; import androidx.media3.common.ParserException; +import androidx.media3.common.util.CodecSpecificDataUtil; import androidx.media3.common.util.ParsableByteArray; import androidx.media3.common.util.UnstableApi; import androidx.media3.common.util.Util; @@ -44,6 +45,7 @@ import androidx.media3.container.Mp4Box; import androidx.media3.container.Mp4Box.ContainerBox; import androidx.media3.container.NalUnitUtil; +import androidx.media3.container.XmpData; import androidx.media3.extractor.Ac3Util; import androidx.media3.extractor.Ac4Util; import androidx.media3.extractor.DtsUtil; @@ -91,8 +93,7 @@ public static ExtractorsFactory newFactory(SubtitleParser.Factory subtitleParser /** * Flags controlling the behavior of the extractor. Possible flag values are {@link * #FLAG_WORKAROUND_IGNORE_EDIT_LISTS}, {@link #FLAG_READ_SEF_DATA}, {@link - * #FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES}, {@link #FLAG_READ_AUXILIARY_TRACKS}, {@link - * #FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES_H265} and {@link #FLAG_OMIT_TRACK_SAMPLE_TABLE}. + * #FLAG_READ_AUXILIARY_TRACKS}, and {@link #FLAG_OMIT_TRACK_SAMPLE_TABLE}. */ @Documented @Retention(RetentionPolicy.SOURCE) @@ -104,11 +105,11 @@ public static ExtractorsFactory newFactory(SubtitleParser.Factory subtitleParser FLAG_READ_SEF_DATA, FLAG_MARK_FIRST_VIDEO_TRACK_WITH_MAIN_ROLE, FLAG_EMIT_RAW_SUBTITLE_DATA, - FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES, FLAG_READ_AUXILIARY_TRACKS, - FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES_H265, FLAG_OMIT_TRACK_SAMPLE_TABLE, - FLAG_DISABLE_ARTWORK_METADATA + FLAG_DISABLE_ARTWORK_METADATA, + FLAG_DISABLE_HAGC_METADATA, + FLAG_READ_XMP_METADATA }) public @interface Flags {} @@ -129,20 +130,6 @@ public static ExtractorsFactory newFactory(SubtitleParser.Factory subtitleParser public static final int FLAG_EMIT_RAW_SUBTITLE_DATA = 1 << 4; - /** - * Flag to extract additional sample dependency information, and mark output buffers with {@link - * C#BUFFER_FLAG_NOT_DEPENDED_ON} for {@linkplain MimeTypes#VIDEO_H264 H.264} video. - * - *

    This class always marks the samples at the start of each group of picture (GOP) with {@link - * C#BUFFER_FLAG_KEY_FRAME}. Usually, key frames can be decoded independently, without depending - * on other samples. - * - *

    Setting this flag enables elementary stream parsing to identify disposable samples that are - * not depended on by other samples. Any disposable sample can be safely omitted, and the rest of - * the track will remain valid. - */ - public static final int FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES = 1 << 5; - /** * Flag to extract the auxiliary tracks from the MP4 With Auxiliary Tracks Extension (MP4-AT) file * format. @@ -157,14 +144,6 @@ public static ExtractorsFactory newFactory(SubtitleParser.Factory subtitleParser */ public static final int FLAG_READ_AUXILIARY_TRACKS = 1 << 6; - /** - * Flag to extract additional sample dependency information, and mark output buffers with {@link - * C#BUFFER_FLAG_NOT_DEPENDED_ON} for {@linkplain MimeTypes#VIDEO_H265 H.265} video. - * - *

    See {@link #FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES}. - */ - public static final int FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES_H265 = 1 << 7; - /** * Flag to omit allocating and populating the large per-sample arrays (offsets, sizes, timestamps, * flags) within {@link TrackSampleTable}. This is used to reduce memory consumption in metadata @@ -175,6 +154,17 @@ public static ExtractorsFactory newFactory(SubtitleParser.Factory subtitleParser /** Flag to disable parsing of artwork metadata. */ public static final int FLAG_DISABLE_ARTWORK_METADATA = 1 << 9; + /** Flag to disable parsing of HAGC (ST 2094-50) metadata. */ + public static final int FLAG_DISABLE_HAGC_METADATA = 1 << 10; + + /** + * Flag to extract Adobe XMP metadata. + * + *

    If set, the extractor will peek for a top-level uuid box containing XMP metadata immediately + * following the moov box, and expose it inside {@link Format#metadata} for all tracks. + */ + public static final int FLAG_READ_XMP_METADATA = 1 << 11; + /** The maximum number of sync samples to scan when searching for a thumbnail. */ private static final int MAX_SYNC_SAMPLES_TO_SCAN_FOR_THUMBNAIL = 20; @@ -254,6 +244,7 @@ public static ExtractorsFactory newFactory(SubtitleParser.Factory subtitleParser private long atomSize; private int atomHeaderBytesRead; @Nullable private ParsableByteArray atomData; + @Nullable private XmpData xmpData; private int sampleTrackIndex; private int sampleBytesRead; @@ -333,23 +324,6 @@ public Mp4Extractor(SubtitleParser.Factory subtitleParserFactory, @Flags int fla quickTimeChapters = new ArrayList<>(); } - /** - * Returns {@link Flags} denoting if an extractor should parse within GOP sample dependencies. - * - * @param videoCodecFlags The set of codecs for which to parse within GOP sample dependencies. - */ - public static @Flags int codecsToParseWithinGopSampleDependenciesAsFlags( - @C.VideoCodecFlags int videoCodecFlags) { - @Flags int flags = 0; - if ((videoCodecFlags & C.VIDEO_CODEC_FLAG_H264) != 0) { - flags |= FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES; - } - if ((videoCodecFlags & C.VIDEO_CODEC_FLAG_H265) != 0) { - flags |= FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES_H265; - } - return flags; - } - @Override public boolean sniff(ExtractorInput input) throws IOException { @Nullable SniffFailure sniffFailure = Sniffer.sniffUnfragmented(input); @@ -380,6 +354,7 @@ public void seek(long position, long timeUs) { sampleCurrentNalBytesRemaining = 0; isSampleDependedOn = false; moovAtomProcessed = false; + xmpData = null; chapterTrackIndex = 0; chapterSampleIndex = 0; chapterSampleTables.clear(); @@ -508,7 +483,7 @@ private boolean readAtomHeader(ExtractorInput input) throws IOException { } containerAtoms.push(new ContainerBox(atomType, endPosition)); if (atomSize == atomHeaderBytesRead) { - processAtomEnded(endPosition); + processAtomEnded(input, endPosition); } else { // Start reading the first child atom. enterReadingAtomHeaderState(); @@ -564,7 +539,7 @@ private boolean readAtomPayload(ExtractorInput input, PositionHolder positionHol seekRequired = true; } } - processAtomEnded(atomEndPosition); + processAtomEnded(input, atomEndPosition); if (seekToAxteAtom) { readingAuxiliaryTracks = true; positionHolder.position = axteAtomOffset; @@ -583,10 +558,51 @@ private boolean readAtomPayload(ExtractorInput input, PositionHolder positionHol return result; } - private void processAtomEnded(long atomEndPosition) throws ParserException { + private void maybePeekUuidXmpMetadata(ExtractorInput input, long moovEndPosition) { + if ((flags & FLAG_READ_XMP_METADATA) == 0 || input.getPosition() != moovEndPosition) { + return; + } + // We expect uuid immediately after moov. + try { + scratch.reset(/* limit= */ Mp4Box.HEADER_SIZE); + input.peekFully(scratch.getData(), /* offset= */ 0, /* length= */ Mp4Box.HEADER_SIZE); + long size = scratch.readUnsignedInt(); + int type = scratch.readInt(); + if (type == Mp4Box.TYPE_uuid) { + int headerSize = Mp4Box.HEADER_SIZE; + if (size == Mp4Box.DEFINES_LARGE_SIZE) { + scratch.reset(/* limit= */ 8); + input.peekFully(scratch.getData(), /* offset= */ 0, /* length= */ 8); + size = scratch.readUnsignedLongToLong(); + headerSize = Mp4Box.LONG_HEADER_SIZE; + } else if (size == Mp4Box.EXTENDS_TO_END_SIZE) { + return; + } + + long payloadSize = size - headerSize; + if (payloadSize > 16 && payloadSize <= 2 * 1024 * 1024) { // Only read up to 2MB. + scratch.reset(/* limit= */ 16); + input.peekFully(scratch.getData(), /* offset= */ 0, /* length= */ 16); + if (scratch.readLong() == XmpData.XMP_UUID.getMostSignificantBits() + && scratch.readLong() == XmpData.XMP_UUID.getLeastSignificantBits()) { + byte[] xmpDataBytes = new byte[(int) (payloadSize - 16)]; + input.peekFully(xmpDataBytes, /* offset= */ 0, /* length= */ xmpDataBytes.length); + xmpData = new XmpData(xmpDataBytes); + } + } + } + } catch (IOException | RuntimeException e) { + // Ignore peek failures, we shouldn't fail moov parsing! + } finally { + input.resetPeekPosition(); + } + } + + private void processAtomEnded(ExtractorInput input, long atomEndPosition) throws ParserException { while (!containerAtoms.isEmpty() && containerAtoms.peek().endPosition == atomEndPosition) { ContainerBox containerAtom = containerAtoms.pop(); if (containerAtom.type == Mp4Box.TYPE_moov) { + maybePeekUuidXmpMetadata(input, containerAtom.endPosition); // We've reached the end of the moov atom. Process it and prepare to read samples. processMoovAtom(containerAtom); containerAtoms.clear(); @@ -657,7 +673,16 @@ private void processMoovAtom(ContainerBox moov) throws ParserException { /* drmInitData= */ null, ignoreEditLists, isQuickTime, - /* modifyTrackFunction= */ track -> track, + /* modifyTrackFunction= */ track -> { + if (track == null) { + return null; + } + if ((flags & FLAG_DISABLE_HAGC_METADATA) != 0 + && CodecSpecificDataUtil.isHagcTrack(track.format)) { + return null; + } + return track; + }, omitTrackSampleTable); if (readingAuxiliaryTracks) { @@ -699,8 +724,7 @@ private void processMoovAtom(ContainerBox moov) throws ParserException { } Mp4Track mp4Track = new Mp4Track(track, trackSampleTable, extractorOutput.track(trackIndex++, track.type)); - long trackDurationUs = - track.durationUs != C.TIME_UNSET ? track.durationUs : trackSampleTable.durationUs; + long trackDurationUs = getTrackDurationUs(track, trackSampleTable); mp4Track.trackOutput.durationUs(trackDurationUs); durationUs = max(durationUs, trackDurationUs); @@ -745,7 +769,8 @@ private void processMoovAtom(ContainerBox moov) throws ParserException { slowMotionMetadataEntries.isEmpty() ? null : new Metadata(slowMotionMetadataEntries), udtaMetadata, mvhdMetadata, - thumbnailMetadata); + thumbnailMetadata, + xmpData != null ? new Metadata(xmpData) : null); formatBuilder.setContainerMimeType(containerMimeType); Format format = formatBuilder.build(); // The moov and esds boxes don't contain enough information to distinguish between MPEG @@ -786,6 +811,21 @@ private void processMoovAtom(ContainerBox moov) throws ParserException { extractorOutput.seekMap(new Mp4SeekMap(durationUs, this.tracks, firstVideoTrackIndex)); } + /** + * Returns the duration of the track in microseconds. + * + *

    If no edit list is applied to the samples, the {@code mdhd} duration is preferred because it + * is expressed in the track's own timescale, which is typically finer than the movie timescale + * used for the {@code tkhd} duration. If an edit list is applied, the {@code tkhd} duration + * reflects the edited presentation duration and is used instead. + */ + private static long getTrackDurationUs(Track track, TrackSampleTable trackSampleTable) { + if (track.editListDurations == null && track.mediaDurationUs != C.TIME_UNSET) { + return track.mediaDurationUs; + } + return track.durationUs != C.TIME_UNSET ? track.durationUs : trackSampleTable.durationUs; + } + private static long findBestThumbnailPresentationTimeUs( TrackSampleTable sampleTable, long durationUs) { if (!MimeTypes.isVideo(sampleTable.track.format.sampleMimeType) @@ -1153,6 +1193,10 @@ private int readQuickTimeChapters(ExtractorInput input, PositionHolder seekPosit * with the next logical sample (based on sample time) exceeds {@link * #MAXIMUM_READ_AHEAD_BYTES_STREAM}. If this is the case, we continue with this sample even * though it may require a source reload. + * + *

    As an exception, dependent metadata tracks are prioritized over the aforementioned logic if + * their timestamp is behind or equal to the primary track (e.g., video). This ensures they can be + * extracted before the frames they apply to. */ private int getTrackIndexOfNextReadSample(long inputPosition) { long preferredSkipAmount = Long.MAX_VALUE; @@ -1162,12 +1206,27 @@ private int getTrackIndexOfNextReadSample(long inputPosition) { long minAccumulatedBytes = Long.MAX_VALUE; boolean minAccumulatedBytesRequiresReload = true; int minAccumulatedBytesTrackIndex = C.INDEX_UNSET; + + long minVideoTimestampUs = Long.MAX_VALUE; + long minIt35TimestampUs = Long.MAX_VALUE; + int minIt35TrackIndex = C.INDEX_UNSET; + for (int trackIndex = 0; trackIndex < tracks.length; trackIndex++) { Mp4Track track = tracks[trackIndex]; int sampleIndex = track.sampleIndex; if (sampleIndex == track.sampleTable.sampleCount) { continue; } + + long sampleTimestampUs = track.sampleTable.timestampsUs[sampleIndex]; + if (track.isVideo) { + minVideoTimestampUs = Math.min(minVideoTimestampUs, sampleTimestampUs); + } else if (track.isItutT35) { + if (sampleTimestampUs < minIt35TimestampUs) { + minIt35TimestampUs = sampleTimestampUs; + minIt35TrackIndex = trackIndex; + } + } long sampleOffset = track.sampleTable.offsets[sampleIndex]; long sampleAccumulatedBytes = checkNotNull(accumulatedSampleSizes)[trackIndex][sampleIndex]; long skipAmount = sampleOffset - inputPosition; @@ -1185,6 +1244,12 @@ private int getTrackIndexOfNextReadSample(long inputPosition) { minAccumulatedBytesTrackIndex = trackIndex; } } + if (minVideoTimestampUs != Long.MAX_VALUE + && minIt35TrackIndex != C.INDEX_UNSET + && minIt35TimestampUs <= minVideoTimestampUs) { + return minIt35TrackIndex; + } + return minAccumulatedBytes == Long.MAX_VALUE || !minAccumulatedBytesRequiresReload || preferredAccumulatedBytes < minAccumulatedBytes + MAXIMUM_READ_AHEAD_BYTES_STREAM @@ -1215,16 +1280,10 @@ private void maybeSkipRemainingMetaAtomHeaderBytes(ExtractorInput input) throws * Returns whether reading within GOP sample dependencies is enabled for the sample {@link * Format}. */ - private boolean canReadWithinGopSampleDependencies(Format format) { - if (Objects.equals(format.sampleMimeType, MimeTypes.VIDEO_H264)) { - return (flags & FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES) != 0; - } - if (Objects.equals(format.sampleMimeType, MimeTypes.VIDEO_H265)) { - return (flags & FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES_H265) != 0; - } - // Do not flag gate APV sample dependency parsing - prior experiments with H.264 and H.265 - // were positive, and the defaults have been updated. - return Objects.equals(format.sampleMimeType, MimeTypes.VIDEO_APV); + private static boolean canReadWithinGopSampleDependencies(Format format) { + return Objects.equals(format.sampleMimeType, MimeTypes.VIDEO_H264) + || Objects.equals(format.sampleMimeType, MimeTypes.VIDEO_H265) + || Objects.equals(format.sampleMimeType, MimeTypes.VIDEO_APV); } /** @@ -1379,6 +1438,8 @@ private static final class Mp4Track { public final TrackSampleTable sampleTable; public final TrackOutput trackOutput; @Nullable public final TrueHdSampleRechunker trueHdSampleRechunker; + private final boolean isVideo; + private final boolean isItutT35; public int sampleIndex; @@ -1392,6 +1453,8 @@ public Mp4Track(Track track, TrackSampleTable sampleTable, TrackOutput trackOutp this.track = track; this.sampleTable = sampleTable; this.trackOutput = trackOutput; + this.isVideo = track.type == C.TRACK_TYPE_VIDEO; + this.isItutT35 = Objects.equals(track.format.sampleMimeType, MimeTypes.APPLICATION_ITUT_T35); trueHdSampleRechunker = MimeTypes.AUDIO_TRUEHD.equals(track.format.sampleMimeType) ? new TrueHdSampleRechunker() diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/text/CharsetDetector.java b/libraries/extractor/src/main/java/androidx/media3/extractor/text/CharsetDetector.java new file mode 100644 index 00000000000..0d6fb82d701 --- /dev/null +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/text/CharsetDetector.java @@ -0,0 +1,38 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.extractor.text; + +import androidx.annotation.Nullable; +import androidx.media3.common.util.UnstableApi; +import java.nio.charset.Charset; + +/** Detects the character encoding of subtitle byte data. */ +@UnstableApi +public interface CharsetDetector { + + /** + * Detects the character encoding of the requested range of {@code data}. + * + *

    The requested range is not guaranteed to contain a complete subtitle file. + * + * @param data The subtitle byte data. + * @param offset The start offset in {@code data}. + * @param length The number of bytes to inspect. + * @return The detected character encoding, or {@code null} if it could not be determined. + */ + @Nullable + Charset detect(byte[] data, int offset, int length); +} diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/text/DefaultSubtitleParserFactory.java b/libraries/extractor/src/main/java/androidx/media3/extractor/text/DefaultSubtitleParserFactory.java index b2413e3be55..60ac5a90bf8 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/text/DefaultSubtitleParserFactory.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/text/DefaultSubtitleParserFactory.java @@ -50,6 +50,22 @@ @UnstableApi public final class DefaultSubtitleParserFactory implements SubtitleParser.Factory { + @Nullable private final CharsetDetector charsetDetector; + + /** Creates an instance. */ + public DefaultSubtitleParserFactory() { + this(/* charsetDetector= */ null); + } + + /** + * Creates an instance that passes {@code charsetDetector} to delegate factories (where relevant). + * + * @param charsetDetector The detector to use, or {@code null}. + */ + public DefaultSubtitleParserFactory(@Nullable CharsetDetector charsetDetector) { + this.charsetDetector = charsetDetector; + } + @Override public boolean supportsFormat(Format format) { @Nullable String mimeType = format.sampleMimeType; @@ -106,7 +122,7 @@ public SubtitleParser create(Format format) { case MimeTypes.APPLICATION_MP4VTT: return new Mp4WebvttParser(); case MimeTypes.APPLICATION_SUBRIP: - return new SubripParser(); + return new SubripParser(charsetDetector, format); case MimeTypes.APPLICATION_TX3G: return new Tx3gParser(format.initializationData); case MimeTypes.APPLICATION_PGS: diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/text/cea/Cea708Decoder.java b/libraries/extractor/src/main/java/androidx/media3/extractor/text/cea/Cea708Decoder.java index 8b9b306f271..a6d9f906b4d 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/text/cea/Cea708Decoder.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/text/cea/Cea708Decoder.java @@ -229,8 +229,8 @@ protected void decode(SubtitleInputBuffer inputBuffer) { int ccType = ccTypeAndValid & (DTVCC_PACKET_DATA | DTVCC_PACKET_START); boolean ccValid = (ccTypeAndValid & CC_VALID_FLAG) == CC_VALID_FLAG; - byte ccData1 = (byte) ccData.readUnsignedByte(); - byte ccData2 = (byte) ccData.readUnsignedByte(); + byte ccData1 = ccData.readByte(); + byte ccData2 = ccData.readByte(); // Ignore any non-CEA-708 data if (ccType != DTVCC_PACKET_DATA && ccType != DTVCC_PACKET_START) { diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/text/subrip/SubripParser.java b/libraries/extractor/src/main/java/androidx/media3/extractor/text/subrip/SubripParser.java index 8f607ed6357..77be5ce7679 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/text/subrip/SubripParser.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/text/subrip/SubripParser.java @@ -26,11 +26,13 @@ import androidx.media3.common.C; import androidx.media3.common.Format; import androidx.media3.common.Format.CueReplacementBehavior; +import androidx.media3.common.MimeTypes; import androidx.media3.common.text.Cue; import androidx.media3.common.util.Consumer; import androidx.media3.common.util.Log; import androidx.media3.common.util.ParsableByteArray; import androidx.media3.common.util.UnstableApi; +import androidx.media3.extractor.text.CharsetDetector; import androidx.media3.extractor.text.CuesWithTiming; import androidx.media3.extractor.text.SubtitleParser; import com.google.common.collect.ImmutableList; @@ -82,11 +84,28 @@ public final class SubripParser implements SubtitleParser { private final StringBuilder textBuilder; private final ArrayList tags; private final ParsableByteArray parsableByteArray; + @Nullable private final CharsetDetector charsetDetector; + private final Format format; + /** Creates an instance. */ public SubripParser() { + this(/* charsetDetector= */ null, new Format.Builder().build()); + } + + /** + * Creates an instance that uses {@code charsetDetector} when the charset of the input is + * ambiguous. + * + * @param charsetDetector The detector to use, or {@code null} to default to UTF-8 when the + * charset of the input can't be determined by other means. + * @param format The format of the subrip track. + */ + public SubripParser(@Nullable CharsetDetector charsetDetector, Format format) { textBuilder = new StringBuilder(); tags = new ArrayList<>(); parsableByteArray = new ParsableByteArray(); + this.charsetDetector = charsetDetector; + this.format = format; } @Override @@ -103,7 +122,19 @@ public void parse( Consumer output) { parsableByteArray.reset(data, /* limit= */ offset + length); parsableByteArray.setPosition(offset); - Charset charset = detectUtfCharset(parsableByteArray); + Charset charset = detectCharset(); + if (!ParsableByteArray.isCharsetSupported(charset)) { + // ParsableByteArray doesn't support directly reading the detected charset, so we transcode + // the data to UTF-8 (which is supported). + parsableByteArray.reset( + new String( + parsableByteArray.getData(), + parsableByteArray.getPosition(), + parsableByteArray.bytesLeft(), + charset) + .getBytes(StandardCharsets.UTF_8)); + charset = StandardCharsets.UTF_8; + } @Nullable List cuesWithTimingBeforeRequestedStartTimeUs = @@ -188,11 +219,35 @@ public void parse( } /** - * Determine UTF encoding of the byte array from a byte order mark (BOM), defaulting to UTF-8 if - * no BOM is found. + * Returns the charset to use for line parsing of {@link #parsableByteArray}. + * + *

    The result is derived from, in order: + * + *

      + *
    1. Content from a Matroska container is assumed to always be UTF-8 (per spec). + *
    2. A byte order mark in the data, if present. + *
    3. The result of calling {@link #charsetDetector}, if non-null. + *
    4. Fallback to assuming UTF-8. + *
    */ - private Charset detectUtfCharset(ParsableByteArray data) { - @Nullable Charset charset = data.readUtfCharsetFromBom(); + private Charset detectCharset() { + if (MimeTypes.isMatroska(format.containerMimeType)) { + // Subrip muxed into a Matroska container is always UTF-8. + return StandardCharsets.UTF_8; + } + @Nullable Charset utfCharset = parsableByteArray.readUtfCharsetFromBom(); + if (utfCharset != null) { + return utfCharset; + } + if (charsetDetector == null) { + return StandardCharsets.UTF_8; + } + @Nullable + Charset charset = + charsetDetector.detect( + parsableByteArray.getData(), + parsableByteArray.getPosition(), + parsableByteArray.bytesLeft()); return charset != null ? charset : StandardCharsets.UTF_8; } diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/text/ttml/TtmlParser.java b/libraries/extractor/src/main/java/androidx/media3/extractor/text/ttml/TtmlParser.java index fcf0f9b32e0..94c9e37f6aa 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/text/ttml/TtmlParser.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/text/ttml/TtmlParser.java @@ -101,12 +101,12 @@ public final class TtmlParser implements SubtitleParser { + "(?:(\\.[0-9]+)|:([0-9][0-9])(?:\\.([0-9]+))?)?$"); private static final Pattern OFFSET_TIME = Pattern.compile("^([0-9]+(?:\\.[0-9]+)?)(h|m|s|ms|f|t)$"); - private static final Pattern FONT_SIZE = Pattern.compile("^(([0-9]*.)?[0-9]+)(px|em|%)$"); - static final Pattern SIGNED_PERCENTAGE = Pattern.compile("^([-+]?\\d+\\.?\\d*?)%$"); + private static final Pattern FONT_SIZE = Pattern.compile("^(([0-9]*\\.)?[0-9]+)(px|em|%)$"); + static final Pattern SIGNED_PERCENTAGE = Pattern.compile("^([-+]?\\d+(?:\\.\\d*)?)%$"); static final Pattern PERCENTAGE_COORDINATES = - Pattern.compile("^([-+]?\\d+\\.?\\d*?)% ([-+]?\\d+\\.?\\d*?)%$"); + Pattern.compile("^([-+]?\\d+(?:\\.\\d*)?)% ([-+]?\\d+(?:\\.\\d*)?)%$"); private static final Pattern PIXEL_COORDINATES = - Pattern.compile("^([-+]?\\d+\\.?\\d*?)px ([-+]?\\d+\\.?\\d*?)px$"); + Pattern.compile("^([-+]?\\d+(?:\\.\\d*)?)px ([-+]?\\d+(?:\\.\\d*)?)px$"); private static final Pattern CELL_RESOLUTION = Pattern.compile("^(\\d+) (\\d+)$"); private static final int DEFAULT_FRAME_RATE = 30; diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/ts/DefaultTsPayloadReaderFactory.java b/libraries/extractor/src/main/java/androidx/media3/extractor/ts/DefaultTsPayloadReaderFactory.java index d2aae227d69..bed32f05891 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/ts/DefaultTsPayloadReaderFactory.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/ts/DefaultTsPayloadReaderFactory.java @@ -294,7 +294,7 @@ private List getClosedCaptionFormats(EsInfo esInfo) { } // easy_reader(1), wide_aspect_ratio(1), reserved(6). - byte flags = (byte) scratchDescriptorData.readUnsignedByte(); + byte flags = scratchDescriptorData.readByte(); // Skip reserved (8). scratchDescriptorData.skipBytes(1); diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/ts/H264Reader.java b/libraries/extractor/src/main/java/androidx/media3/extractor/ts/H264Reader.java index 53b44c2e10a..83581409eeb 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/ts/H264Reader.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/ts/H264Reader.java @@ -61,6 +61,9 @@ public final class H264Reader implements ElementaryStreamReader { // State that should not be reset on seek. private boolean hasOutputFormat; + private boolean hasPendingFormat; + private @MonotonicNonNull Format format; + private @C.StereoMode int stereoMode; // Per PES packet state that gets reset at the start of each PES packet. private long pesTimeUs; @@ -93,6 +96,7 @@ public H264Reader( pps = new NalUnitTargetBuffer(NalUnitUtil.H264_NAL_UNIT_TYPE_PPS, 128); sei = new NalUnitTargetBuffer(NalUnitUtil.H264_NAL_UNIT_TYPE_SEI, 128); pesTimeUs = C.TIME_UNSET; + stereoMode = Format.NO_VALUE; seiWrapper = new ParsableByteArray(); } @@ -188,13 +192,25 @@ public void endOfInputReached() { seiReader.flush(); // Simulate end of current NAL unit and start an AUD one to trigger output of current sample endNalUnit(totalBytesWritten, 0, 0, pesTimeUs); + if (!hasOutputFormat && hasPendingFormat) { + output.format(checkNotNull(format)); + hasOutputFormat = true; + hasPendingFormat = false; + } startNalUnit(totalBytesWritten, NalUnitUtil.H264_NAL_UNIT_TYPE_AUD, pesTimeUs); endNalUnit(totalBytesWritten, 0, 0, pesTimeUs); } - @RequiresNonNull("sampleReader") + @RequiresNonNull({"output", "sampleReader"}) private void startNalUnit(long position, int nalUnitType, long pesTimeUs) { - if (!hasOutputFormat || sampleReader.needsSpsPps()) { + if (hasPendingFormat + && nalUnitType >= NalUnitUtil.H264_NAL_UNIT_TYPE_NON_IDR + && nalUnitType <= NalUnitUtil.H264_NAL_UNIT_TYPE_IDR) { + output.format(checkNotNull(format)); + hasOutputFormat = true; + hasPendingFormat = false; + } + if (format == null || sampleReader.needsSpsPps()) { sps.startNalUnit(nalUnitType); pps.startNalUnit(nalUnitType); } @@ -204,7 +220,7 @@ private void startNalUnit(long position, int nalUnitType, long pesTimeUs) { @RequiresNonNull("sampleReader") private void nalUnitData(byte[] dataArray, int offset, int limit) { - if (!hasOutputFormat || sampleReader.needsSpsPps()) { + if (format == null || sampleReader.needsSpsPps()) { sps.appendToNalUnit(dataArray, offset, limit); pps.appendToNalUnit(dataArray, offset, limit); } @@ -214,10 +230,10 @@ private void nalUnitData(byte[] dataArray, int offset, int limit) { @RequiresNonNull({"output", "sampleReader"}) private void endNalUnit(long position, int offset, int discardPadding, long pesTimeUs) { - if (!hasOutputFormat || sampleReader.needsSpsPps()) { + if (format == null || sampleReader.needsSpsPps()) { sps.endNalUnit(discardPadding); pps.endNalUnit(discardPadding); - if (!hasOutputFormat) { + if (format == null) { if (sps.isCompleted() && pps.isCompleted()) { List initializationData = new ArrayList<>(); initializationData.add(Arrays.copyOf(sps.nalData, sps.nalLength)); @@ -229,7 +245,7 @@ private void endNalUnit(long position, int offset, int discardPadding, long pesT spsData.profileIdc, spsData.constraintsFlagsAndReservedZero2Bits, spsData.levelIdc); - output.format( + format = new Format.Builder() .setId(formatId) .setContainerMimeType(containerMimeType) @@ -248,8 +264,9 @@ private void endNalUnit(long position, int offset, int discardPadding, long pesT .setPixelWidthHeightRatio(spsData.pixelWidthHeightRatio) .setInitializationData(initializationData) .setMaxNumReorderSamples(spsData.maxNumReorderFrames) - .build()); - hasOutputFormat = true; + .setStereoMode(stereoMode) + .build(); + hasPendingFormat = true; seiReader.setReorderingQueueSize(spsData.maxNumReorderFrames); sampleReader.putSps(spsData); sampleReader.putPps(ppsData); @@ -269,6 +286,18 @@ private void endNalUnit(long position, int offset, int discardPadding, long pesT } if (sei.endNalUnit(discardPadding)) { int unescapedLength = NalUnitUtil.unescapeStream(sei.nalData, sei.nalLength); + @C.StereoMode + int parsedStereoMode = + NalUnitUtil.parseSeiStereoMode(sei.nalData, /* offset= */ 4, unescapedLength); + if (parsedStereoMode != Format.NO_VALUE + && parsedStereoMode != stereoMode + && (parsedStereoMode != C.STEREO_MODE_MONO || stereoMode != Format.NO_VALUE)) { + stereoMode = parsedStereoMode; + if (format != null) { + format = format.buildUpon().setStereoMode(stereoMode).build(); + hasPendingFormat = true; + } + } seiWrapper.reset(sei.nalData, unescapedLength); seiWrapper.setPosition(4); // NAL prefix and nal_unit() header. seiReader.consume(pesTimeUs, seiWrapper); @@ -318,7 +347,7 @@ private static final class SampleReader { private boolean sampleIsKeyframe; private boolean randomAccessIndicator; - public SampleReader( + private SampleReader( TrackOutput output, boolean allowNonIdrKeyframes, boolean detectAccessUnits) { this.output = output; this.allowNonIdrKeyframes = allowNonIdrKeyframes; @@ -332,25 +361,25 @@ public SampleReader( reset(); } - public boolean needsSpsPps() { + private boolean needsSpsPps() { return detectAccessUnits; } - public void putSps(NalUnitUtil.SpsData spsData) { + private void putSps(NalUnitUtil.SpsData spsData) { sps.append(spsData.seqParameterSetId, spsData); } - public void putPps(NalUnitUtil.PpsData ppsData) { + private void putPps(NalUnitUtil.PpsData ppsData) { pps.append(ppsData.picParameterSetId, ppsData); } - public void reset() { + private void reset() { isFilling = false; readingSample = false; sliceHeader.clear(); } - public void startNalUnit( + private void startNalUnit( long position, int type, long pesTimeUs, boolean randomAccessIndicator) { nalUnitType = type; nalUnitTimeUs = pesTimeUs; @@ -378,7 +407,7 @@ public void startNalUnit( * @param offset The offset of the data in {@code data}. * @param limit The limit (exclusive) of the data in {@code data}. */ - public void appendToNalUnit(byte[] data, int offset, int limit) { + private void appendToNalUnit(byte[] data, int offset, int limit) { if (!isFilling) { return; } @@ -503,7 +532,7 @@ public void appendToNalUnit(byte[] data, int offset, int limit) { isFilling = false; } - public boolean endNalUnit(long position, int offset, boolean hasOutputFormat) { + private boolean endNalUnit(long position, int offset, boolean hasOutputFormat) { if (nalUnitType == NalUnitUtil.H264_NAL_UNIT_TYPE_AUD || (detectAccessUnits && sliceHeader.isFirstVclNalUnitOfPicture(previousSliceHeader))) { // If the NAL unit ending is the start of a new sample, output the previous one. @@ -562,17 +591,17 @@ private static final class SliceHeaderData { private int deltaPicOrderCnt0; private int deltaPicOrderCnt1; - public void clear() { + private void clear() { hasSliceType = false; isComplete = false; } - public void setSliceType(int sliceType) { + private void setSliceType(int sliceType) { this.sliceType = sliceType; hasSliceType = true; } - public void setAll( + private void setAll( SpsData spsData, int nalRefIdc, int sliceType, @@ -605,7 +634,7 @@ public void setAll( hasSliceType = true; } - public boolean isISlice() { + private boolean isISlice() { return hasSliceType && (sliceType == SLICE_TYPE_ALL_I || sliceType == SLICE_TYPE_I); } diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/ts/H265Reader.java b/libraries/extractor/src/main/java/androidx/media3/extractor/ts/H265Reader.java index 38a81516212..22fb1990f42 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/ts/H265Reader.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/ts/H265Reader.java @@ -49,6 +49,9 @@ public final class H265Reader implements ElementaryStreamReader { // State that should not be reset on seek. private boolean hasOutputFormat; + private boolean hasPendingFormat; + private @MonotonicNonNull Format format; + private @C.StereoMode int stereoMode; // State that should be reset on seek. private final boolean[] prefixFlags; @@ -79,6 +82,7 @@ public H265Reader(SeiReader seiReader, String containerMimeType) { prefixSei = new NalUnitTargetBuffer(NalUnitUtil.H265_NAL_UNIT_TYPE_PREFIX_SEI, 128); suffixSei = new NalUnitTargetBuffer(NalUnitUtil.H265_NAL_UNIT_TYPE_SUFFIX_SEI, 128); pesTimeUs = C.TIME_UNSET; + stereoMode = Format.NO_VALUE; seiWrapper = new ParsableByteArray(); } @@ -179,13 +183,23 @@ public void endOfInputReached() { // Simulate end of current NAL unit and start an unspecified one to trigger output of current // sample endNalUnit(totalBytesWritten, 0, 0, pesTimeUs); + if (!hasOutputFormat && hasPendingFormat) { + output.format(checkNotNull(format)); + hasOutputFormat = true; + hasPendingFormat = false; + } startNalUnit(totalBytesWritten, 0, NalUnitUtil.H265_NAL_UNIT_TYPE_UNSPECIFIED, pesTimeUs); } - @RequiresNonNull("sampleReader") + @RequiresNonNull({"output", "sampleReader"}) private void startNalUnit(long position, int offset, int nalUnitType, long pesTimeUs) { sampleReader.startNalUnit(position, offset, nalUnitType, pesTimeUs, hasOutputFormat); - if (!hasOutputFormat) { + if (hasPendingFormat && nalUnitType < NalUnitUtil.H265_NAL_UNIT_TYPE_VPS) { + output.format(checkNotNull(format)); + hasOutputFormat = true; + hasPendingFormat = false; + } + if (format == null) { vps.startNalUnit(nalUnitType); sps.startNalUnit(nalUnitType); pps.startNalUnit(nalUnitType); @@ -197,7 +211,7 @@ private void startNalUnit(long position, int offset, int nalUnitType, long pesTi @RequiresNonNull("sampleReader") private void nalUnitData(byte[] dataArray, int offset, int limit) { sampleReader.readNalUnitData(dataArray, offset, limit); - if (!hasOutputFormat) { + if (format == null) { vps.appendToNalUnit(dataArray, offset, limit); sps.appendToNalUnit(dataArray, offset, limit); pps.appendToNalUnit(dataArray, offset, limit); @@ -209,20 +223,31 @@ private void nalUnitData(byte[] dataArray, int offset, int limit) { @RequiresNonNull({"output", "sampleReader"}) private void endNalUnit(long position, int offset, int discardPadding, long pesTimeUs) { sampleReader.endNalUnit(position, offset, hasOutputFormat); - if (!hasOutputFormat) { + if (format == null) { vps.endNalUnit(discardPadding); sps.endNalUnit(discardPadding); pps.endNalUnit(discardPadding); if (vps.isCompleted() && sps.isCompleted() && pps.isCompleted()) { - Format format = parseMediaFormat(formatId, vps, sps, pps, containerMimeType); - output.format(format); + format = parseMediaFormat(formatId, vps, sps, pps, containerMimeType, stereoMode); + hasPendingFormat = true; checkState(format.maxNumReorderSamples != Format.NO_VALUE); seiReader.setReorderingQueueSize(format.maxNumReorderSamples); - hasOutputFormat = true; } } if (prefixSei.endNalUnit(discardPadding)) { int unescapedLength = NalUnitUtil.unescapeStream(prefixSei.nalData, prefixSei.nalLength); + @C.StereoMode + int parsedStereoMode = + NalUnitUtil.parseSeiStereoMode(prefixSei.nalData, /* offset= */ 5, unescapedLength); + if (parsedStereoMode != Format.NO_VALUE + && parsedStereoMode != stereoMode + && (parsedStereoMode != C.STEREO_MODE_MONO || stereoMode != Format.NO_VALUE)) { + stereoMode = parsedStereoMode; + if (format != null) { + format = format.buildUpon().setStereoMode(stereoMode).build(); + hasPendingFormat = true; + } + } seiWrapper.reset(prefixSei.nalData, unescapedLength); // Skip the NAL prefix and type. @@ -244,7 +269,8 @@ private static Format parseMediaFormat( NalUnitTargetBuffer vps, NalUnitTargetBuffer sps, NalUnitTargetBuffer pps, - String containerMimeType) { + String containerMimeType, + @C.StereoMode int stereoMode) { // Build codec-specific data. byte[] csdData = new byte[vps.nalLength + sps.nalLength + pps.nalLength]; System.arraycopy(vps.nalData, 0, csdData, 0, vps.nalLength); @@ -288,6 +314,7 @@ private static Format parseMediaFormat( .setMaxNumReorderSamples(spsData.maxNumReorderPics) .setMaxSubLayers(spsData.maxSubLayersMinus1 + 1) .setInitializationData(Collections.singletonList(csdData)) + .setStereoMode(stereoMode) .build(); } @@ -323,11 +350,11 @@ private static final class SampleReader { private long sampleTimeUs; private boolean sampleIsKeyframe; - public SampleReader(TrackOutput output) { + private SampleReader(TrackOutput output) { this.output = output; } - public void reset() { + private void reset() { lookingForFirstSliceFlag = false; isFirstSlice = false; isFirstPrefixNalUnit = false; @@ -335,7 +362,7 @@ public void reset() { readingPrefix = false; } - public void startNalUnit( + private void startNalUnit( long position, int offset, int nalUnitType, long pesTimeUs, boolean hasOutputFormat) { isFirstSlice = false; isFirstPrefixNalUnit = false; @@ -364,7 +391,7 @@ public void startNalUnit( nalUnitHasKeyframeData || nalUnitType <= NalUnitUtil.H265_NAL_UNIT_TYPE_RASL_R; } - public void readNalUnitData(byte[] data, int offset, int limit) { + private void readNalUnitData(byte[] data, int offset, int limit) { if (lookingForFirstSliceFlag) { int headerOffset = offset + FIRST_SLICE_FLAG_OFFSET - nalUnitBytesRead; if (headerOffset < limit) { @@ -376,7 +403,7 @@ public void readNalUnitData(byte[] data, int offset, int limit) { } } - public void endNalUnit(long position, int offset, boolean hasOutputFormat) { + private void endNalUnit(long position, int offset, boolean hasOutputFormat) { if (readingPrefix && isFirstSlice) { // This sample has parameter sets. Reset the key-frame flag based on the first slice. sampleIsKeyframe = nalUnitHasKeyframeData; diff --git a/libraries/extractor/src/main/java/androidx/media3/extractor/ts/PsExtractor.java b/libraries/extractor/src/main/java/androidx/media3/extractor/ts/PsExtractor.java index f4f816ee08e..972deedc4ba 100644 --- a/libraries/extractor/src/main/java/androidx/media3/extractor/ts/PsExtractor.java +++ b/libraries/extractor/src/main/java/androidx/media3/extractor/ts/PsExtractor.java @@ -260,7 +260,11 @@ public int read(ExtractorInput input, PositionHolder seekPosition) throws IOExce if (elementaryStreamReader != null) { TrackIdGenerator idGenerator = new TrackIdGenerator(streamId, MAX_STREAM_ID_PLUS_ONE); elementaryStreamReader.createTracks(output, idGenerator); - payloadReader = new PesReader(elementaryStreamReader, timestampAdjuster); + payloadReader = + new PesReader( + elementaryStreamReader, + timestampAdjuster, + /* isPrivateStream1= */ streamId == PRIVATE_STREAM_1); psPayloadReaders.put(streamId, payloadReader); } } @@ -328,6 +332,10 @@ private static final class PesReader { private final ElementaryStreamReader pesPayloadReader; private final TimestampAdjuster timestampAdjuster; private final ParsableBitArray pesScratch; + // Whether this reader consumes the MPEG-PS private_stream_1 (0xBD), which in DVD-style content + // carries a sub-stream header (sub_stream_id, number_of_frame_headers and + // first_access_unit_pointer) before the audio elementary stream. + private final boolean isPrivateStream1; private boolean ptsFlag; private boolean dtsFlag; @@ -335,9 +343,13 @@ private static final class PesReader { private int extendedHeaderLength; private long timeUs; - public PesReader(ElementaryStreamReader pesPayloadReader, TimestampAdjuster timestampAdjuster) { + private PesReader( + ElementaryStreamReader pesPayloadReader, + TimestampAdjuster timestampAdjuster, + boolean isPrivateStream1) { this.pesPayloadReader = pesPayloadReader; this.timestampAdjuster = timestampAdjuster; + this.isPrivateStream1 = isPrivateStream1; pesScratch = new ParsableBitArray(new byte[PES_SCRATCH_SIZE]); } @@ -366,12 +378,39 @@ public void consume(ParsableByteArray data) throws ParserException { data.readBytes(pesScratch.data, 0, extendedHeaderLength); pesScratch.setPosition(0); parseHeaderExtension(); + // For DVD-style private_stream_1 (AC-3/DTS/LPCM), a sub-stream header follows the PES header + // and precedes the audio elementary stream. It must be stripped so it isn't fed into the + // audio reader as elementary data, which would corrupt the audio frame spanning this PES + // boundary (see ISO/IEC 13818-1 / DVD-Video PS spec). + maybeSkipPrivateStream1SubStreamHeader(data); pesPayloadReader.packetStarted(timeUs, TsPayloadReader.FLAG_DATA_ALIGNMENT_INDICATOR); pesPayloadReader.consume(data); // We always have complete PES packets with program stream. pesPayloadReader.packetFinished(); } + /** + * Skips the DVD-style private_stream_1 sub-stream header, if present, so that {@code data} is + * positioned at the start of the audio elementary stream. + * + *

    In DVD-Video and similar MPEG-PS content, each private_stream_1 (0xBD) PES packet begins + * with a 4-byte header: {@code sub_stream_id} (1 byte, 0x80-0x87 for AC-3, 0x88-0x8F for DTS, + * 0xA0-0xA7 for LPCM), {@code number_of_frame_headers} (1 byte) and {@code + * first_access_unit_pointer} (2 bytes). These bytes are not part of the audio elementary stream + * and must not be forwarded to the audio reader. + */ + private void maybeSkipPrivateStream1SubStreamHeader(ParsableByteArray data) { + if (!isPrivateStream1 || data.bytesLeft() < 4) { + return; + } + int subStreamId = data.peekUnsignedByte(); + // AC-3 (0x80-0x87), DTS (0x88-0x8F) and LPCM (0xA0-0xA7) sub-streams carry the 4-byte header. + if ((subStreamId >= 0x80 && subStreamId <= 0x8F) + || (subStreamId >= 0xA0 && subStreamId <= 0xA7)) { + data.skipBytes(4); + } + } + private void consumeEndOfInput() { pesPayloadReader.endOfInputReached(); } diff --git a/libraries/extractor/src/test/java/androidx/media3/extractor/DefaultExtractorsFactoryTest.java b/libraries/extractor/src/test/java/androidx/media3/extractor/DefaultExtractorsFactoryTest.java index aa4bf720aa6..a2c5851d9dc 100644 --- a/libraries/extractor/src/test/java/androidx/media3/extractor/DefaultExtractorsFactoryTest.java +++ b/libraries/extractor/src/test/java/androidx/media3/extractor/DefaultExtractorsFactoryTest.java @@ -15,9 +15,11 @@ */ package androidx.media3.extractor; +import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.truth.Truth.assertThat; import android.net.Uri; +import androidx.annotation.Nullable; import androidx.media3.common.MimeTypes; import androidx.media3.extractor.amr.AmrExtractor; import androidx.media3.extractor.avi.AviExtractor; @@ -40,8 +42,11 @@ import androidx.media3.extractor.ts.TsExtractor; import androidx.media3.extractor.wav.WavExtractor; import androidx.media3.extractor.webp.WebpExtractor; +import androidx.media3.test.utils.FakeExtractorInput; import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.io.EOFException; import java.util.ArrayList; +import java.util.Arrays; import java.util.Collections; import java.util.HashMap; import java.util.List; @@ -75,14 +80,14 @@ public void createExtractors_withoutMediaInfo_optimizesSniffingOrder() { AdtsExtractor.class, Ac3Extractor.class, Ac4Extractor.class, - Mp3Extractor.class, AviExtractor.class, JpegExtractor.class, PngExtractor.class, WebpExtractor.class, BmpExtractor.class, HeifExtractor.class, - AvifExtractor.class) + AvifExtractor.class, + Mp3Extractor.class) .inOrder(); } @@ -134,6 +139,38 @@ public void createExtractors_withMediaInfo_optimizesSniffingOrder() { .inOrder(); } + @Test + public void createExtractors_jpegWithEmbeddedMp3SyncBytes_sniffsJpegBeforeMp3() throws Exception { + // JPEG header (SOI + SOS) followed by a 417-byte MP3 frame. + byte[] data = + Arrays.copyOf( + new byte[] { + (byte) 0xFF, (byte) 0xD8, (byte) 0xFF, (byte) 0xDA, + (byte) 0xFF, (byte) 0xFB, (byte) 0x90, (byte) 0x64 + }, + 4 + 417); + FakeExtractorInput input = new FakeExtractorInput.Builder().setData(data).build(); + Extractor[] extractors = new DefaultExtractorsFactory().createExtractors(); + + @Nullable Extractor matchedExtractor = null; + for (Extractor extractor : extractors) { + input.resetPeekPosition(); + try { + if (extractor.sniff(input)) { + matchedExtractor = extractor; + break; + } + } catch (EOFException e) { + // Ignore EOF during sniffing. + } + } + + assertThat(checkNotNull(matchedExtractor).getUnderlyingImplementation()) + .isInstanceOf(JpegExtractor.class); + input.resetPeekPosition(); + assertThat(new Mp3Extractor().sniff(input)).isTrue(); + } + private static List> getUnderlyingExtractorClasses( Extractor[] extractors) { List> extractorClasses = new ArrayList<>(); diff --git a/libraries/extractor/src/test/java/androidx/media3/extractor/ExtractorUtilTest.java b/libraries/extractor/src/test/java/androidx/media3/extractor/ExtractorUtilTest.java index 3e37d711e36..d844099bc45 100644 --- a/libraries/extractor/src/test/java/androidx/media3/extractor/ExtractorUtilTest.java +++ b/libraries/extractor/src/test/java/androidx/media3/extractor/ExtractorUtilTest.java @@ -20,10 +20,12 @@ import android.net.Uri; import androidx.media3.common.C; +import androidx.media3.common.util.Util; import androidx.media3.datasource.DataSpec; import androidx.media3.test.utils.FakeDataSource; import androidx.test.ext.junit.runners.AndroidJUnit4; import java.io.EOFException; +import java.nio.ByteBuffer; import java.util.Arrays; import org.junit.Test; import org.junit.runner.RunWith; @@ -210,4 +212,132 @@ public void peekFullyQuietly_endReachedWithoutEndOfInputAllowed_throws() throws input, target, offset, length, /* allowEndOfInput= */ false)); assertThat(input.getPeekPosition()).isEqualTo(0); } + + @Test + public void getFramesPerEncodedSample_aacLc_returnsSampleCount() { + ByteBuffer buffer = ByteBuffer.allocate(0); + + int sampleCount = ExtractorUtil.getFramesPerEncodedSample(C.ENCODING_AAC_LC, buffer); + + assertThat(sampleCount).isEqualTo(1024); + } + + @Test + public void getFramesPerEncodedSample_aacHe_returnsSampleCount() { + ByteBuffer buffer = ByteBuffer.allocate(0); + + int sampleCountV1 = ExtractorUtil.getFramesPerEncodedSample(C.ENCODING_AAC_HE_V1, buffer); + int sampleCountV2 = ExtractorUtil.getFramesPerEncodedSample(C.ENCODING_AAC_HE_V2, buffer); + + assertThat(sampleCountV1).isEqualTo(2048); + assertThat(sampleCountV2).isEqualTo(2048); + } + + @Test + public void getFramesPerEncodedSample_aacXhe_returnsSampleCount() { + ByteBuffer buffer = ByteBuffer.allocate(0); + + int sampleCount = ExtractorUtil.getFramesPerEncodedSample(C.ENCODING_AAC_XHE, buffer); + + assertThat(sampleCount).isEqualTo(1024); + } + + @Test + public void getFramesPerEncodedSample_aacEld_returnsSampleCount() { + ByteBuffer buffer = ByteBuffer.allocate(0); + + int sampleCount = ExtractorUtil.getFramesPerEncodedSample(C.ENCODING_AAC_ELD, buffer); + + assertThat(sampleCount).isEqualTo(512); + } + + @Test + public void getFramesPerEncodedSample_mp3_returnsSampleCount() { + byte[] mp3FrameHeader = Util.getBytesFromHexString("FFFB9064"); + ByteBuffer buffer = ByteBuffer.wrap(mp3FrameHeader); + + int sampleCount = ExtractorUtil.getFramesPerEncodedSample(C.ENCODING_MP3, buffer); + + assertThat(sampleCount).isEqualTo(1152); + } + + @Test + public void getFramesPerEncodedSample_invalidMp3_throwsIllegalArgumentException() { + ByteBuffer buffer = ByteBuffer.allocate(4); + + assertThrows( + IllegalArgumentException.class, + () -> ExtractorUtil.getFramesPerEncodedSample(C.ENCODING_MP3, buffer)); + } + + @Test + public void getFramesPerEncodedSample_trueHdSyncframe_returnsSampleCount() { + byte[] trueHdSyncframeHeader = Util.getBytesFromHexString("C07504D8F8726FBA0097C00FB7520000"); + ByteBuffer buffer = ByteBuffer.wrap(trueHdSyncframeHeader); + + int sampleCount = ExtractorUtil.getFramesPerEncodedSample(C.ENCODING_DOLBY_TRUEHD, buffer); + + assertThat(sampleCount).isEqualTo(640); + } + + @Test + public void getFramesPerEncodedSample_trueHdNonSyncframe_returnsZero() { + byte[] trueHdNonSyncframeHeader = + Util.getBytesFromHexString("A025048860224E6F6DEDB6D5B6DBAFE6"); + ByteBuffer buffer = ByteBuffer.wrap(trueHdNonSyncframeHeader); + + int sampleCount = ExtractorUtil.getFramesPerEncodedSample(C.ENCODING_DOLBY_TRUEHD, buffer); + + assertThat(sampleCount).isEqualTo(0); + } + + @Test + public void getFramesPerEncodedSample_ac3_returnsSampleCount() { + byte[] ac3SyncframeHeader = Util.getBytesFromHexString("0B7700000000"); + ByteBuffer buffer = ByteBuffer.wrap(ac3SyncframeHeader); + + int sampleCount = ExtractorUtil.getFramesPerEncodedSample(C.ENCODING_AC3, buffer); + + assertThat(sampleCount).isEqualTo(1536); + } + + @Test + public void getFramesPerEncodedSample_ac4_returnsSampleCount() { + byte[] ac4SyncframeHeader = Util.getBytesFromHexString("AC400020000500000000000000000000"); + ByteBuffer buffer = ByteBuffer.wrap(ac4SyncframeHeader); + + int sampleCount = ExtractorUtil.getFramesPerEncodedSample(C.ENCODING_AC4, buffer); + + assertThat(sampleCount).isEqualTo(1600); + } + + @Test + public void getFramesPerEncodedSample_dts_returnsSampleCount() { + byte[] dtsFrameHeader = Util.getBytesFromHexString("7FFE8001003C"); + ByteBuffer buffer = ByteBuffer.wrap(dtsFrameHeader); + + int sampleCount = ExtractorUtil.getFramesPerEncodedSample(C.ENCODING_DTS, buffer); + + assertThat(sampleCount).isEqualTo(512); + } + + @Test + public void getFramesPerEncodedSample_opus_returnsSampleCount() { + byte[] oggOpusPacket = + Util.getBytesFromHexString("4F6767530000E001000000000000000000000200000036E1841601EA04"); + ByteBuffer buffer = ByteBuffer.wrap(oggOpusPacket); + + int sampleCount = ExtractorUtil.getFramesPerEncodedSample(C.ENCODING_OPUS, buffer); + + assertThat(sampleCount).isEqualTo(480); + } + + @Test + public void getFramesPerEncodedSample_pcmEncoding_throwsIllegalStateException() { + ByteBuffer buffer = ByteBuffer.allocate(0); + + assertThrows( + IllegalStateException.class, + () -> ExtractorUtil.getFramesPerEncodedSample(C.ENCODING_PCM_16BIT, buffer)); + } } diff --git a/libraries/extractor/src/test/java/androidx/media3/extractor/Id3PeekerTest.java b/libraries/extractor/src/test/java/androidx/media3/extractor/Id3PeekerTest.java index 6f7073ada25..03efd2986b1 100644 --- a/libraries/extractor/src/test/java/androidx/media3/extractor/Id3PeekerTest.java +++ b/libraries/extractor/src/test/java/androidx/media3/extractor/Id3PeekerTest.java @@ -164,6 +164,28 @@ public void peekId3Data_returnId3TagAccordingToGivenPredicate_ifId3TagPresent() assertThat(commentFrame.text).isEqualTo("text"); } + @Test + public void peekId3Data_withInvalidMajorVersion_returnsNull() throws IOException { + Id3Peeker id3Peeker = new Id3Peeker(); + byte[] data = new byte[] {'I', 'D', '3', 'M', '0', 'W', 0x78, 0x62, 0x32, 0x59, 0, 0, 0, 0}; + FakeExtractorInput input = new FakeExtractorInput.Builder().setData(data).build(); + + @Nullable Metadata metadata = id3Peeker.peekId3Data(input, /* id3FramePredicate= */ null, 128); + + assertThat(metadata).isNull(); + } + + @Test + public void peekId3Data_withTagLengthExceedingInputLength_returnsNull() throws IOException { + Id3Peeker id3Peeker = new Id3Peeker(); + byte[] data = new byte[] {'I', 'D', '3', 3, 0, 0, 0x78, 0x62, 0x32, 0x59, 0, 0, 0, 0}; + FakeExtractorInput input = new FakeExtractorInput.Builder().setData(data).build(); + + @Nullable Metadata metadata = id3Peeker.peekId3Data(input, /* id3FramePredicate= */ null, 128); + + assertThat(metadata).isNull(); + } + private static void assertApicFramesEqual(ApicFrame actual, ApicFrame expected) { assertThat(actual.mimeType).isEqualTo(expected.mimeType); assertThat(actual.description).isEqualTo(expected.description); diff --git a/libraries/extractor/src/test/java/androidx/media3/extractor/jpeg/MotionPhotoDescriptionTest.java b/libraries/extractor/src/test/java/androidx/media3/extractor/MotionPhotoDescriptionTest.java similarity index 99% rename from libraries/extractor/src/test/java/androidx/media3/extractor/jpeg/MotionPhotoDescriptionTest.java rename to libraries/extractor/src/test/java/androidx/media3/extractor/MotionPhotoDescriptionTest.java index 3d3c9a23b03..d8abdf4dd17 100644 --- a/libraries/extractor/src/test/java/androidx/media3/extractor/jpeg/MotionPhotoDescriptionTest.java +++ b/libraries/extractor/src/test/java/androidx/media3/extractor/MotionPhotoDescriptionTest.java @@ -13,7 +13,7 @@ * See the License for the specific language governing permissions and * limitations under the License. */ -package androidx.media3.extractor.jpeg; +package androidx.media3.extractor; import static com.google.common.truth.Truth.assertThat; diff --git a/libraries/extractor/src/test/java/androidx/media3/extractor/mkv/MatroskaExtractorTest.java b/libraries/extractor/src/test/java/androidx/media3/extractor/mkv/MatroskaExtractorTest.java index e1e87a779ab..1762af4c363 100644 --- a/libraries/extractor/src/test/java/androidx/media3/extractor/mkv/MatroskaExtractorTest.java +++ b/libraries/extractor/src/test/java/androidx/media3/extractor/mkv/MatroskaExtractorTest.java @@ -338,6 +338,14 @@ public void mkvSample_withChapters() throws Exception { simulationConfig); } + @Test + public void mkaSample_withNestedChapters() throws Exception { + ExtractorAsserts.assertBehavior( + getExtractorFactory(subtitlesParsedDuringExtraction), + "media/mka/bear-flac-16bit-nested-chapters.mka", + simulationConfig); + } + @Test public void mkvSampleLastCluster() throws Exception { ExtractorAsserts.assertBehavior( diff --git a/libraries/extractor/src/test/java/androidx/media3/extractor/mp3/XingFrameTest.java b/libraries/extractor/src/test/java/androidx/media3/extractor/mp3/XingFrameTest.java index c8e1cfa2ba5..3df9f03c0a5 100644 --- a/libraries/extractor/src/test/java/androidx/media3/extractor/mp3/XingFrameTest.java +++ b/libraries/extractor/src/test/java/androidx/media3/extractor/mp3/XingFrameTest.java @@ -74,61 +74,14 @@ public void computeDurationUs_withInvalidGaplessSampleCount_returnsTimeUnset() { assertThat(frame.computeDurationUs()).isEqualTo(C.TIME_UNSET); } - @Test - public void parse_withLameEncoderIdentifier_adjustsDelayAndPaddingForDecodedPcm() { - XingFrame frame = - createXingFrame( - INFO_FRAME_HEADER_DATA, - /* frameCount= */ 40, - /* dataSize= */ 8_541, - /* encoderDelay= */ 576, - /* encoderPadding= */ 1_404, - /* encoderIdentifier= */ "LAME3.99r"); - - assertThat(frame.encoderDelay).isEqualTo(1_105); - assertThat(frame.encoderPadding).isEqualTo(875); - } - - @Test - public void parse_withLameEncoderIdentifierAndSmallPadding_clampsPaddingToZero() { - XingFrame frame = - createXingFrame( - INFO_FRAME_HEADER_DATA, - /* frameCount= */ 40, - /* dataSize= */ 8_541, - /* encoderDelay= */ 576, - /* encoderPadding= */ 398, - /* encoderIdentifier= */ "LAME3.99r"); - - assertThat(frame.encoderDelay).isEqualTo(1_105); - assertThat(frame.encoderPadding).isEqualTo(0); - } - private static XingFrame createXingFrame( int headerData, int frameCount, int dataSize, int encoderDelay, int encoderPadding) { - return createXingFrame( - headerData, - frameCount, - dataSize, - encoderDelay, - encoderPadding, - /* encoderIdentifier= */ ""); - } - - private static XingFrame createXingFrame( - int headerData, - int frameCount, - int dataSize, - int encoderDelay, - int encoderPadding, - String encoderIdentifier) { int encoderDelayAndPadding = (encoderDelay << 12) | encoderPadding; ByteBuffer payload = ByteBuffer.allocate(4 + 4 + 4 + 11 + 8 + 2 + 3); payload.putInt(0x03); payload.putInt(frameCount); payload.putInt(dataSize); - payload.put(createFixedLengthEncoderIdentifier(encoderIdentifier)); - payload.position(payload.position() + 2 + 8 + 2); + payload.position(payload.position() + 11 + 8 + 2); payload.put((byte) (encoderDelayAndPadding >> 16)); payload.put((byte) (encoderDelayAndPadding >> 8)); payload.put((byte) encoderDelayAndPadding); @@ -140,16 +93,4 @@ private static XingFrame createXingFrame(int headerData, byte[] payload) { xingFrameHeader.setForHeaderData(headerData); return XingFrame.parse(xingFrameHeader, new ParsableByteArray(payload)); } - - private static byte[] createFixedLengthEncoderIdentifier(String encoderIdentifier) { - byte[] fixedLengthIdentifier = new byte[9]; - byte[] identifierBytes = Util.getUtf8Bytes(encoderIdentifier); - System.arraycopy( - identifierBytes, - /* srcPos= */ 0, - fixedLengthIdentifier, - /* destPos= */ 0, - Math.min(identifierBytes.length, fixedLengthIdentifier.length)); - return fixedLengthIdentifier; - } } diff --git a/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/BoxParserTest.java b/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/BoxParserTest.java index bb968cf088a..2e54f4b539b 100644 --- a/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/BoxParserTest.java +++ b/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/BoxParserTest.java @@ -17,16 +17,24 @@ import static com.google.common.truth.Truth.assertThat; import static java.nio.charset.StandardCharsets.UTF_8; +import static org.junit.Assert.assertThrows; import androidx.media3.common.C; +import androidx.media3.common.Format; import androidx.media3.common.Metadata; +import androidx.media3.common.MimeTypes; import androidx.media3.common.ParserException; import androidx.media3.common.util.ParsableByteArray; import androidx.media3.common.util.Util; import androidx.media3.container.Mp4Box; +import androidx.media3.container.Mp4LocationData; +import androidx.media3.container.OpusUtil; +import androidx.media3.extractor.GaplessInfoHolder; import androidx.media3.extractor.metadata.Chapter; import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.primitives.ImmutableLongArray; import java.nio.ByteBuffer; +import java.nio.ByteOrder; import org.junit.Test; import org.junit.runner.RunWith; @@ -365,6 +373,261 @@ public void parseChpl_truncatedData_returnsNull() { assertThat(metadata).isNull(); } + @Test + public void parseXyz_withTwoCoordinates_returnsLocationData() { + ParsableByteArray xyzBox = createXyzBox("+35.1345-15.1020/"); + + Metadata metadata = BoxParser.parseXyz(xyzBox); + + assertThat(metadata) + .isEqualTo( + new Metadata( + new Mp4LocationData(/* latitude= */ 35.1345f, /* longitude= */ -15.1020f))); + } + + @Test + public void parseXyz_withThreeCoordinates_returnsLocationDataWithAltitude() { + ParsableByteArray xyzBox = createXyzBox("+37.7749-122.4194+15.0000/"); + + Metadata metadata = BoxParser.parseXyz(xyzBox); + + assertThat(metadata) + .isEqualTo( + new Metadata( + new Mp4LocationData( + /* latitude= */ 37.7749f, /* longitude= */ -122.4194f, /* altitude= */ 15.0f))); + } + + @Test + public void parseXyz_withNegativeAltitudeAndCrs_returnsLocationData() { + ParsableByteArray xyzBox = createXyzBox("-12.3456+78.9012-100.5CRSWGS_84/"); + + Metadata metadata = BoxParser.parseXyz(xyzBox); + + assertThat(metadata) + .isEqualTo( + new Metadata( + new Mp4LocationData( + /* latitude= */ -12.3456f, + /* longitude= */ 78.9012f, + /* altitude= */ -100.5f))); + } + + @Test + public void parseXyz_withTwoCoordinatesAndCrs_returnsLocationData() { + ParsableByteArray xyzBox = createXyzBox("+37.7749-122.4194CRSWGS_84/"); + + Metadata metadata = BoxParser.parseXyz(xyzBox); + + assertThat(metadata) + .isEqualTo( + new Metadata( + new Mp4LocationData(/* latitude= */ 37.7749f, /* longitude= */ -122.4194f))); + } + + @Test + public void parseXyz_withInvalidFormat_returnsNull() { + ParsableByteArray xyzBox = createXyzBox("invalid-location"); + + Metadata metadata = BoxParser.parseXyz(xyzBox); + + assertThat(metadata).isNull(); + } + + @Test + public void parseXyz_withCoordinatesOutOfRange_returnsNull() { + ParsableByteArray xyzBox = createXyzBox("+95.0000-122.4194/"); + + Metadata metadata = BoxParser.parseXyz(xyzBox); + + assertThat(metadata).isNull(); + } + + @Test + public void parseXyz_withoutTerminatingSlash_returnsNull() { + ParsableByteArray xyzBox = createXyzBox("+37.7749-122.4194+15.0000"); + + Metadata metadata = BoxParser.parseXyz(xyzBox); + + assertThat(metadata).isNull(); + } + + @Test + public void parseXyz_withExtraCoordinates_returnsNull() { + ParsableByteArray xyzBox = createXyzBox("+10-20+30-40/"); + + Metadata metadata = BoxParser.parseXyz(xyzBox); + + assertThat(metadata).isNull(); + } + + @Test + public void parseStbl_withEmptyEditAndOmitTrackSampleTable_includesEmptyEditInDuration() + throws Exception { + Track track = + new Track.Builder() + .setTimescale(1_000) + .setMovieTimescale(1_000) + .setFormat(new Format.Builder().build()) + .setEditListDurations(ImmutableLongArray.of(200, 800)) + .setEditListMediaTimes(ImmutableLongArray.of(-1, 0)) + .build(); + Mp4Box.ContainerBox stblBox = new Mp4Box.ContainerBox(Mp4Box.TYPE_stbl, /* endPosition= */ 0); + stblBox.add(createFullBox(Mp4Box.TYPE_stsz, 100, 1)); // sample_size, sample_count + stblBox.add(createFullBox(Mp4Box.TYPE_stco, 1, 0)); // entry_count, chunk_offset + stblBox.add(createFullBox(Mp4Box.TYPE_stsc, 1, 1, 1, 1)); // count, chunk, samples, desc_index + stblBox.add(createFullBox(Mp4Box.TYPE_stts, 1, 1, 800)); // entry_count, sample_count, delta + + TrackSampleTable sampleTable = + BoxParser.parseStbl( + track, stblBox, new GaplessInfoHolder(), /* omitTrackSampleTable= */ true); + + assertThat(sampleTable.durationUs).isEqualTo(1_000_000); + } + + private static Mp4Box.LeafBox createFullBox(int type, int... fields) { + ByteBuffer buffer = + ByteBuffer.allocate(Mp4Box.FULL_HEADER_SIZE + fields.length * Integer.BYTES) + .put(new byte[Mp4Box.FULL_HEADER_SIZE]); + for (int field : fields) { + buffer.putInt(field); + } + return new Mp4Box.LeafBox(type, new ParsableByteArray(buffer.array())); + } + + @Test + public void parseTrak_withOpusDopsBox_convertsDopsToLittleEndianOpusHead() + throws ParserException { + byte[] dOpsBody = + ByteBuffer.allocate(11) + .order(ByteOrder.BIG_ENDIAN) + .put((byte) 0) // Version + .put((byte) 2) // OutputChannelCount + .putShort((short) 312) // PreSkip + .putInt(48_000) // InputSampleRate + .putShort((short) 258) // OutputGain + .put((byte) 0) // ChannelMappingFamily + .array(); + byte[] expectedOpusHead = + ByteBuffer.allocate(19) + .order(ByteOrder.LITTLE_ENDIAN) + .put("OpusHead".getBytes(UTF_8)) + .put((byte) 1) // Version + .put((byte) 2) // OutputChannelCount + .putShort((short) 312) // PreSkip + .putInt(48_000) // InputSampleRate + .putShort((short) 258) // OutputGain + .put((byte) 0) // ChannelMappingFamily + .array(); + Mp4Box.ContainerBox trak = createOpusTrakWithDopsBox(dOpsBody); + Mp4Box.LeafBox mvhd = + new Mp4Box.LeafBox( + Mp4Box.TYPE_mvhd, + new ParsableByteArray(ByteBuffer.allocate(24).putInt(/* index= */ 20, 1000).array())); + + Track track = + BoxParser.parseTrak( + trak, + mvhd, + /* duration= */ C.TIME_UNSET, + /* drmInitData= */ null, + /* ignoreEditLists= */ false, + /* isQuickTime= */ false); + + assertThat(track).isNotNull(); + assertThat(track.format.sampleMimeType).isEqualTo(MimeTypes.AUDIO_OPUS); + assertThat(track.format.initializationData).hasSize(3); + byte[] opusHead = track.format.initializationData.get(0); + assertThat(opusHead).isEqualTo(expectedOpusHead); + assertThat(OpusUtil.getPreSkipSamples(opusHead)).isEqualTo(312); + long preSkipNanos = + ByteBuffer.wrap(track.format.initializationData.get(1)) + .order(ByteOrder.nativeOrder()) + .getLong(); + assertThat(preSkipNanos).isEqualTo(6_500_000L); + } + + @Test + public void parseTrak_withTruncatedOpusDopsBox_throwsParserException() { + byte[] truncatedDopsBody = new byte[10]; // Minimum valid dOps payload size is 11 bytes. + Mp4Box.ContainerBox trak = createOpusTrakWithDopsBox(truncatedDopsBody); + Mp4Box.LeafBox mvhd = + new Mp4Box.LeafBox( + Mp4Box.TYPE_mvhd, + new ParsableByteArray(ByteBuffer.allocate(24).putInt(/* index= */ 20, 1000).array())); + + assertThrows( + ParserException.class, + () -> + BoxParser.parseTrak( + trak, + mvhd, + /* duration= */ C.TIME_UNSET, + /* drmInitData= */ null, + /* ignoreEditLists= */ false, + /* isQuickTime= */ false)); + } + + private static Mp4Box.ContainerBox createOpusTrakWithDopsBox(byte[] dOpsBody) { + int dOpsBoxSize = Mp4Box.HEADER_SIZE + dOpsBody.length; + int opusEntrySize = Mp4Box.HEADER_SIZE + 8 + 20 + dOpsBoxSize; + int stsdBoxSize = Mp4Box.FULL_HEADER_SIZE + 4 + opusEntrySize; + byte[] stsdBytes = + ByteBuffer.allocate(stsdBoxSize) + .putInt(stsdBoxSize) + .putInt(Mp4Box.TYPE_stsd) + .putInt(0) // version and flags + .putInt(1) // numberOfEntries + .putInt(opusEntrySize) + .putInt(Mp4Box.TYPE_Opus) + .put(new byte[6]) // reserved + .putShort((short) 1) // data_reference_index + .put(new byte[8]) // reserved + .putShort((short) 2) // channelCount + .putShort((short) 16) // sampleSize + .putShort((short) 0) // compressionId + .putShort((short) 0) // packetSize + .putInt(48_000 << 16) // sampleRate (16.16 fixed-point) + .putInt(dOpsBoxSize) + .putInt(Mp4Box.TYPE_dOps) + .put(dOpsBody) + .array(); + Mp4Box.ContainerBox stbl = new Mp4Box.ContainerBox(Mp4Box.TYPE_stbl, /* endPosition= */ 0); + stbl.add(new Mp4Box.LeafBox(Mp4Box.TYPE_stsd, new ParsableByteArray(stsdBytes))); + Mp4Box.ContainerBox minf = new Mp4Box.ContainerBox(Mp4Box.TYPE_minf, /* endPosition= */ 0); + minf.add(stbl); + Mp4Box.ContainerBox mdia = new Mp4Box.ContainerBox(Mp4Box.TYPE_mdia, /* endPosition= */ 0); + mdia.add( + new Mp4Box.LeafBox( + Mp4Box.TYPE_hdlr, + new ParsableByteArray( + ByteBuffer.allocate(20).putInt(/* index= */ 16, 0x736f756e).array()))); + mdia.add( + new Mp4Box.LeafBox( + Mp4Box.TYPE_mdhd, + new ParsableByteArray( + ByteBuffer.allocate(32).putInt(/* index= */ 20, 48_000).array()))); + mdia.add(minf); + Mp4Box.ContainerBox trak = new Mp4Box.ContainerBox(Mp4Box.TYPE_trak, /* endPosition= */ 0); + trak.add( + new Mp4Box.LeafBox( + Mp4Box.TYPE_tkhd, + new ParsableByteArray(ByteBuffer.allocate(92).putInt(/* index= */ 20, 1).array()))); + trak.add(mdia); + return trak; + } + + private static ParsableByteArray createXyzBox(String locationString) { + byte[] stringBytes = locationString.getBytes(UTF_8); + byte[] data = + ByteBuffer.allocate(2 + 2 + stringBytes.length) + .putShort((short) stringBytes.length) + .putShort((short) 0x15C7) // language code + .put(stringBytes) + .array(); + return new ParsableByteArray(data); + } + private static void verifyStz2Parsing(Mp4Box.LeafBox stz2Atom) { BoxParser.Stz2SampleSizeBox box = new BoxParser.Stz2SampleSizeBox(stz2Atom); assertThat(box.getSampleCount()).isEqualTo(4); diff --git a/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/FixedSampleSizeRechunkerTest.java b/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/FixedSampleSizeRechunkerTest.java new file mode 100644 index 00000000000..e6bc481e71b --- /dev/null +++ b/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/FixedSampleSizeRechunkerTest.java @@ -0,0 +1,45 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.extractor.mp4; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.test.ext.junit.runners.AndroidJUnit4; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit test for {@link FixedSampleSizeRechunker}. */ +@RunWith(AndroidJUnit4.class) +public class FixedSampleSizeRechunkerTest { + + @Test + public void rechunk_withFixedSampleSizeExceedingMaxSampleSize_doesNotThrowDivisionByZero() { + int fixedSampleSize = 10_000; // > MAX_SAMPLE_SIZE (8192) + long[] chunkOffsets = new long[] {100}; + int[] chunkSampleCounts = new int[] {5}; + long timestampDeltaInTimeUnits = 1000; + + FixedSampleSizeRechunker.Results results = + FixedSampleSizeRechunker.rechunk( + fixedSampleSize, chunkOffsets, chunkSampleCounts, timestampDeltaInTimeUnits); + + assertThat(results.offsets).isEqualTo(new long[] {100, 10100, 20100, 30100, 40100}); + assertThat(results.sizes).isEqualTo(new int[] {10000, 10000, 10000, 10000, 10000}); + assertThat(results.maximumSize).isEqualTo(10000); + assertThat(results.timestamps).isEqualTo(new long[] {0, 1000, 2000, 3000, 4000}); + assertThat(results.flags).isEqualTo(new int[] {1, 1, 1, 1, 1}); + } +} diff --git a/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/FragmentedMp4ExtractorNonParameterizedTest.java b/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/FragmentedMp4ExtractorNonParameterizedTest.java index 9d256169719..2d8f81523bc 100644 --- a/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/FragmentedMp4ExtractorNonParameterizedTest.java +++ b/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/FragmentedMp4ExtractorNonParameterizedTest.java @@ -16,19 +16,20 @@ package androidx.media3.extractor.mp4; import static com.google.common.truth.Truth.assertThat; +import static java.nio.charset.StandardCharsets.US_ASCII; -import androidx.media3.common.Format; -import androidx.media3.common.MimeTypes; import androidx.media3.extractor.SniffFailure; import androidx.media3.extractor.text.SubtitleParser; -import androidx.media3.test.utils.DumpFileAsserts; import androidx.media3.test.utils.FakeExtractorInput; import androidx.media3.test.utils.FakeExtractorOutput; +import androidx.media3.test.utils.FakeTrackOutput; import androidx.media3.test.utils.TestUtil; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.collect.ImmutableList; import com.google.common.collect.Iterables; +import com.google.common.primitives.Bytes; +import com.google.common.primitives.Longs; import java.io.IOException; import org.junit.Test; import org.junit.runner.RunWith; @@ -85,45 +86,40 @@ public void sniff_returnsFalseWithoutPeekingLargeStbl() throws Exception { } @Test - public void extract_h264WithoutGopParsingFlags() throws Exception { + public void read_withNegativeVersion1TfdtBaseMediaDecodeTime_outputsShiftedSampleTimestamps() + throws Exception { + byte[] data = + TestUtil.getByteArray( + ApplicationProvider.getApplicationContext(), "media/mp4/sample_ac3_fragmented.mp4"); + int tfdtTypeIndex = Bytes.indexOf(data, "tfdt".getBytes(US_ASCII)); + // Overwrite the 64-bit version 1 baseMediaDecodeTime with -336 (-7000 us at 48 kHz). + System.arraycopy(Longs.toByteArray(-336L), 0, data, tfdtTypeIndex + 8, 8); FragmentedMp4Extractor extractor = - new FragmentedMp4Extractor( - SubtitleParser.Factory.UNSUPPORTED, FragmentedMp4Extractor.FLAG_READ_MFRA_FOR_SEEK_MAP); - FakeExtractorOutput output = - TestUtil.extractAllSamplesFromFile( - extractor, - ApplicationProvider.getApplicationContext(), - "media/mp4/sample_fragmented.mp4"); + new FragmentedMp4Extractor(SubtitleParser.Factory.UNSUPPORTED); + + FakeExtractorOutput output = TestUtil.extractAllSamplesFromByteArray(extractor, data); - DumpFileAsserts.assertOutput( - ApplicationProvider.getApplicationContext(), - output, - "extractordumps/mp4/sample_fragmented.mp4_without_gop_parsing_flags"); + FakeTrackOutput trackOutput = output.trackOutputs.get(0); + assertThat(trackOutput.getSampleCount()).isEqualTo(9); + assertThat(trackOutput.getSampleTimeUs(0)).isEqualTo(-7_000L); + assertThat(trackOutput.getSampleTimeUs(1)).isEqualTo(25_000L); } @Test - public void extract_h265WithoutGopParsingFlags() throws Exception { - ImmutableList closedCaptions = - ImmutableList.of( - new Format.Builder().setSampleMimeType(MimeTypes.APPLICATION_CEA608).build()); + public void extract_withoutMehdOrEditList_prefersMdhdTrackDuration() throws Exception { FragmentedMp4Extractor extractor = - new FragmentedMp4Extractor( - SubtitleParser.Factory.UNSUPPORTED, - FragmentedMp4Extractor.FLAG_READ_MFRA_FOR_SEEK_MAP, - /* timestampAdjuster= */ null, - /* sideloadedTrack= */ null, - closedCaptions, - /* additionalEmsgTrackOutput= */ null); + new FragmentedMp4Extractor(SubtitleParser.Factory.UNSUPPORTED); + FakeExtractorOutput output = TestUtil.extractAllSamplesFromFile( extractor, ApplicationProvider.getApplicationContext(), - "media/mp4/fragmented_captions_h265.mp4"); + "media/mp4/sample_partially_fragmented.mp4"); - DumpFileAsserts.assertOutput( - ApplicationProvider.getApplicationContext(), - output, - "extractordumps/mp4/fragmented_captions_h265.mp4_without_gop_parsing_flags"); + // Both tracks have tkhd = 134_000 us and no mehd or edit list. + // Video track mdhd = 133_466 us (4004 / 30_000); audio track mdhd = 133_945 us (5907 / 44_100). + assertThat(output.trackOutputs.get(0).getDurationUs()).isEqualTo(133_466L); + assertThat(output.trackOutputs.get(1).getDurationUs()).isEqualTo(133_945L); } private static FakeExtractorInput createInputForSample(String sample) throws IOException { diff --git a/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/FragmentedMp4ExtractorParameterizedTest.java b/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/FragmentedMp4ExtractorParameterizedTest.java index e53c24e6059..a33b2a1e3ba 100644 --- a/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/FragmentedMp4ExtractorParameterizedTest.java +++ b/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/FragmentedMp4ExtractorParameterizedTest.java @@ -333,11 +333,7 @@ private void assertExtractorBehavior( private static ExtractorFactory getExtractorFactory( List closedCaptionFormats, boolean subtitlesParsedDuringExtraction) { SubtitleParser.Factory subtitleParserFactory; - @FragmentedMp4Extractor.Flags - int flags = - FragmentedMp4Extractor.FLAG_READ_MFRA_FOR_SEEK_MAP - | FragmentedMp4Extractor.FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES - | FragmentedMp4Extractor.FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES_H265; + @FragmentedMp4Extractor.Flags int flags = FragmentedMp4Extractor.FLAG_READ_MFRA_FOR_SEEK_MAP; if (subtitlesParsedDuringExtraction) { subtitleParserFactory = new DefaultSubtitleParserFactory(); } else { diff --git a/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/Mp4ExtractorNonParameterizedTest.java b/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/Mp4ExtractorNonParameterizedTest.java index 727ca01e482..5ce1f4ea486 100644 --- a/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/Mp4ExtractorNonParameterizedTest.java +++ b/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/Mp4ExtractorNonParameterizedTest.java @@ -18,11 +18,18 @@ import static com.google.common.truth.Truth.assertThat; import android.content.Context; +import androidx.media3.common.C; +import androidx.media3.common.DataReader; import androidx.media3.common.Format; +import androidx.media3.common.util.ParsableByteArray; import androidx.media3.extractor.Extractor; +import androidx.media3.extractor.ExtractorOutput; import androidx.media3.extractor.PositionHolder; +import androidx.media3.extractor.SeekMap; import androidx.media3.extractor.SniffFailure; import androidx.media3.extractor.TrackAwareSeekMap; +import androidx.media3.extractor.TrackOutput; +import androidx.media3.extractor.TrackOutput.CryptoData; import androidx.media3.extractor.text.DefaultSubtitleParserFactory; import androidx.media3.extractor.text.SubtitleParser; import androidx.media3.test.utils.DumpFileAsserts; @@ -35,6 +42,8 @@ import com.google.common.collect.ImmutableList; import com.google.common.collect.Iterables; import java.io.IOException; +import java.util.ArrayList; +import java.util.List; import org.junit.Test; import org.junit.runner.RunWith; @@ -142,8 +151,8 @@ public void getSeekPoints_withEmptyTracks_returnsValidInformation() throws Excep FakeExtractorOutput primaryTracksOutput = TestUtil.extractAllSamplesFromFile(mp4Extractor, context, inputFilePath); - String dumpFilePath = getDumpFilePath(inputFilePath, "_with_flag_read_auxiliary_tracks"); - DumpFileAsserts.assertOutput(context, primaryTracksOutput, dumpFilePath); + DumpFileAsserts.assertOutput( + context, primaryTracksOutput, "extractordumps/mp4/sample.mp4.0.dump"); } @Test @@ -241,31 +250,122 @@ public void extract_withOmitTrackSampleTableFlagAndNoStssBox_extractsCorrectMeta } @Test - public void extract_h264WithoutGopParsingFlags() throws Exception { + public void extract_withPoorlyMuxedIt35Track_extractsMetadataBeforeVideo() throws Exception { + Mp4Extractor extractor = new Mp4Extractor(SubtitleParser.Factory.UNSUPPORTED); + FakeExtractorInput input = createInputForSample("sample_with_it35_track.mp4"); + + List extractedTrackTypes = new ArrayList<>(); + List extractedTimeUs = new ArrayList<>(); + ExtractorOutput output = + new ExtractorOutput() { + @Override + public TrackOutput track(int id, int type) { + return new TrackOutput() { + private final byte[] scratch = new byte[4096]; + + @Override + public void format(Format format) {} + + @Override + public int sampleData( + DataReader input, int length, boolean allowEndOfInput, int sampleDataPart) + throws IOException { + return input.read(scratch, 0, Math.min(length, scratch.length)); + } + + @Override + public void sampleData(ParsableByteArray data, int length, int sampleDataPart) { + data.skipBytes(length); + } + + @Override + public void sampleMetadata( + long timeUs, int flags, int size, int offset, CryptoData cryptoData) { + extractedTrackTypes.add(type); + extractedTimeUs.add(timeUs); + } + }; + } + + @Override + public void endTracks() {} + + @Override + public void seekMap(SeekMap seekMap) {} + }; + + extractor.init(output); + PositionHolder seekPositionHolder = new PositionHolder(); + int readResult = Extractor.RESULT_CONTINUE; + + while (readResult != Extractor.RESULT_END_OF_INPUT) { + readResult = extractor.read(input, seekPositionHolder); + if (readResult == Extractor.RESULT_SEEK) { + input.setPosition((int) seekPositionHolder.position); + } + } + + // There are 30 video samples and 30 metadata samples in sample_with_it35_track.mp4. + assertThat(extractedTrackTypes).hasSize(60); + // Verify that IT35 metadata samples are loaded BEFORE any video samples that share or + // exceed their timestamp. + // Which means: If a metadata sample is loaded AFTER a video sample, its timestamp + // must strictly EXCEED the video sample's timestamp. + for (int i = 0; i < extractedTrackTypes.size(); i++) { + if (extractedTrackTypes.get(i) == C.TRACK_TYPE_VIDEO) { + long videoTimeUs = extractedTimeUs.get(i); + for (int j = i + 1; j < extractedTrackTypes.size(); j++) { + if (extractedTrackTypes.get(j) == C.TRACK_TYPE_METADATA) { + long metadataTimeUs = extractedTimeUs.get(j); + assertThat(metadataTimeUs).isGreaterThan(videoTimeUs); + } + } + } + } + } + + @Test + public void extract_withUuidBox_extractsXmpMetadata() throws Exception { Context context = ApplicationProvider.getApplicationContext(); - String inputFilePath = "media/mp4/sample.mp4"; + String inputFilePath = "media/mp4/sample_with_xmp_uuid.mp4"; Mp4Extractor mp4Extractor = - new Mp4Extractor(new DefaultSubtitleParserFactory(), /* flags= */ 0); + new Mp4Extractor(new DefaultSubtitleParserFactory(), Mp4Extractor.FLAG_READ_XMP_METADATA); FakeExtractorOutput output = TestUtil.extractAllSamplesFromFile(mp4Extractor, context, inputFilePath); - String dumpFilePath = getDumpFilePath(inputFilePath, "_without_gop_parsing_flags"); + String dumpFilePath = getDumpFilePath(inputFilePath, "_with_flag_read_xmp_metadata"); DumpFileAsserts.assertOutput(context, output, dumpFilePath); } @Test - public void extract_h265WithoutGopParsingFlags() throws Exception { + public void extract_trackDuration_prefersMdhdOnlyWhenNoEditListIsApplied() throws Exception { + Context context = ApplicationProvider.getApplicationContext(); + Mp4Extractor mp4Extractor = new Mp4Extractor(SubtitleParser.Factory.UNSUPPORTED); + + FakeExtractorOutput output = + TestUtil.extractAllSamplesFromFile( + mp4Extractor, context, "media/mp4/h265_4k_bframes_emulation_prevention.mp4"); + + // Video track has an edit list: tkhd = 3_738_600 us, mdhd = 3_738_577 us. + assertThat(output.trackOutputs.get(0).getDurationUs()).isEqualTo(3_738_600L); + // Audio track has no edit list: tkhd = 3_711_900 us, mdhd = 3_711_937 us. + assertThat(output.trackOutputs.get(1).getDurationUs()).isEqualTo(3_711_937L); + } + + @Test + public void extract_withIgnoreEditLists_prefersMdhdTrackDuration() throws Exception { Context context = ApplicationProvider.getApplicationContext(); - String inputFilePath = "media/mp4/h265_bframes.mp4"; Mp4Extractor mp4Extractor = - new Mp4Extractor(new DefaultSubtitleParserFactory(), /* flags= */ 0); + new Mp4Extractor( + SubtitleParser.Factory.UNSUPPORTED, Mp4Extractor.FLAG_WORKAROUND_IGNORE_EDIT_LISTS); FakeExtractorOutput output = - TestUtil.extractAllSamplesFromFile(mp4Extractor, context, inputFilePath); + TestUtil.extractAllSamplesFromFile( + mp4Extractor, context, "media/mp4/h265_4k_bframes_emulation_prevention.mp4"); - String dumpFilePath = getDumpFilePath(inputFilePath, "_without_gop_parsing_flags"); - DumpFileAsserts.assertOutput(context, output, dumpFilePath); + // Video track's edit list is ignored: tkhd = 3_738_600 us, mdhd = 3_738_577 us. + assertThat(output.trackOutputs.get(0).getDurationUs()).isEqualTo(3_738_577L); } private static String getDumpFilePath(String inputFilePath, String suffix) { diff --git a/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/Mp4ExtractorParameterizedTest.java b/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/Mp4ExtractorParameterizedTest.java index 83f3e5d2d07..6e7762aa882 100644 --- a/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/Mp4ExtractorParameterizedTest.java +++ b/libraries/extractor/src/test/java/androidx/media3/extractor/mp4/Mp4ExtractorParameterizedTest.java @@ -377,10 +377,7 @@ private void assertExtractorBehavior(String file, int peekLimit) throws IOExcept private static ExtractorAsserts.ExtractorFactory getExtractorFactory( boolean subtitlesParsedDuringExtraction) { SubtitleParser.Factory subtitleParserFactory; - @Mp4Extractor.Flags - int flags = - Mp4Extractor.FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES - | Mp4Extractor.FLAG_READ_WITHIN_GOP_SAMPLE_DEPENDENCIES_H265; + @Mp4Extractor.Flags int flags = 0; if (subtitlesParsedDuringExtraction) { subtitleParserFactory = new DefaultSubtitleParserFactory(); } else { diff --git a/libraries/extractor/src/test/java/androidx/media3/extractor/text/subrip/SubripParserTest.java b/libraries/extractor/src/test/java/androidx/media3/extractor/text/subrip/SubripParserTest.java index b8cab4f943c..628e6d04b6a 100644 --- a/libraries/extractor/src/test/java/androidx/media3/extractor/text/subrip/SubripParserTest.java +++ b/libraries/extractor/src/test/java/androidx/media3/extractor/text/subrip/SubripParserTest.java @@ -17,24 +17,39 @@ import static androidx.media3.common.Format.CUE_REPLACEMENT_BEHAVIOR_MERGE; import static com.google.common.truth.Truth.assertThat; - +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.ArgumentMatchers.anyInt; +import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.ArgumentMatchers.same; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.never; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.when; + +import androidx.media3.common.Format; +import androidx.media3.common.MimeTypes; import androidx.media3.common.text.Cue; +import androidx.media3.extractor.text.CharsetDetector; import androidx.media3.extractor.text.CuesWithTiming; import androidx.media3.extractor.text.SubtitleParser; import androidx.media3.extractor.text.SubtitleParser.OutputOptions; import androidx.media3.test.utils.TestUtil; import androidx.test.core.app.ApplicationProvider; -import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.collect.ImmutableList; import com.google.common.collect.Iterables; +import com.google.common.primitives.Bytes; +import com.google.testing.junit.testparameterinjector.TestParameter; import java.io.IOException; +import java.nio.charset.Charset; +import java.nio.charset.StandardCharsets; import java.util.ArrayList; import java.util.List; import org.junit.Test; import org.junit.runner.RunWith; +import org.robolectric.RobolectricTestParameterInjector; /** Unit test for {@link SubripParser}. */ -@RunWith(AndroidJUnit4.class) +@RunWith(RobolectricTestParameterInjector.class) public final class SubripParserTest { private static final String EMPTY_FILE = "media/subrip/empty"; @@ -54,6 +69,9 @@ public final class SubripParserTest { "media/subrip/typical_no_hours_and_millis"; private static final String TYPICAL_BAD_TIMESTAMPS = "media/subrip/typical_bad_timestamps"; + private static final Format STANDALONE_SUBRIP_FORMAT = + new Format.Builder().setContainerMimeType(MimeTypes.APPLICATION_SUBRIP).build(); + @Test public void cueReplacementBehaviorIsMerge() { SubripParser parser = new SubripParser(); @@ -132,8 +150,46 @@ public void parseTypical_cuesAfterTimeThenCuesBefore() throws IOException { } @Test - public void parseTypicalWithByteOrderMark() throws IOException { - SubripParser parser = new SubripParser(); + public void parseTypical_charsetDetectorReturnsNull_assumesUtf8() throws IOException { + SubripParser parser = + new SubripParser( + /* charsetDetector= */ (data, offset, length) -> null, STANDALONE_SUBRIP_FORMAT); + byte[] bytes = TestUtil.getByteArray(ApplicationProvider.getApplicationContext(), TYPICAL_FILE); + + ImmutableList allCues = parseAllCues(parser, bytes); + + assertThat(allCues).hasSize(3); + assertTypicalCue1(allCues.get(0)); + assertTypicalCue2(allCues.get(1)); + assertTypicalCue3(allCues.get(2)); + } + + @Test + public void parseTypicalWithByteOrderMark_doesNotCallCharsetDetector() throws IOException { + CharsetDetector charsetDetector = mock(CharsetDetector.class); + SubripParser parser = new SubripParser(charsetDetector, STANDALONE_SUBRIP_FORMAT); + byte[] bytes = + TestUtil.getByteArray( + ApplicationProvider.getApplicationContext(), TYPICAL_WITH_BYTE_ORDER_MARK); + + ImmutableList allCues = parseAllCues(parser, bytes); + + assertThat(allCues).hasSize(3); + assertTypicalCue1(allCues.get(0)); + assertTypicalCue2(allCues.get(1)); + assertTypicalCue3(allCues.get(2)); + verify(charsetDetector, never()) + .detect(/* data= */ any(), /* offset= */ anyInt(), /* length= */ anyInt()); + } + + @Test + public void parseTypicalWithByteOrderMark_inMkvContainer_doesNotCallCharsetDetector() + throws IOException { + CharsetDetector charsetDetector = mock(CharsetDetector.class); + SubripParser parser = + new SubripParser( + charsetDetector, + new Format.Builder().setContainerMimeType(MimeTypes.VIDEO_MATROSKA).build()); byte[] bytes = TestUtil.getByteArray( ApplicationProvider.getApplicationContext(), TYPICAL_WITH_BYTE_ORDER_MARK); @@ -144,6 +200,8 @@ public void parseTypicalWithByteOrderMark() throws IOException { assertTypicalCue1(allCues.get(0)); assertTypicalCue2(allCues.get(1)); assertTypicalCue3(allCues.get(2)); + verify(charsetDetector, never()) + .detect(/* data= */ any(), /* offset= */ anyInt(), /* length= */ anyInt()); } @Test @@ -233,6 +291,28 @@ public void parseTypicalUtf16LittleEndian() throws IOException { assertTypicalCue3(allCues.get(2)); } + @Test + public void parseTypicalUtf16LittleEndian_inMkv_doesNotCallCharsetDetector_producesGarbage() + throws IOException { + CharsetDetector charsetDetector = mock(CharsetDetector.class); + SubripParser parser = + new SubripParser( + charsetDetector, + new Format.Builder().setContainerMimeType(MimeTypes.VIDEO_MATROSKA).build()); + byte[] bytes = + TestUtil.getByteArray(ApplicationProvider.getApplicationContext(), TYPICAL_UTF16LE); + + ImmutableList unused = parseAllCues(parser, bytes); + + // Don't assert the specific output, since it's effectively undefined behaviour. This test + // exists to assert we don't call CharsetDetector for every subtitle sample from a container + // where the charset is defined by the spec - because most CharsetDetector implementations + // are statistical (and so rely on longer documents than a single subtitle) and also potentially + // slow (so we don't want to call it for every text sample). + verify(charsetDetector, never()) + .detect(/* data= */ any(), /* offset= */ anyInt(), /* length= */ anyInt()); + } + @Test public void parseTypicalUtf16BigEndian() throws IOException { SubripParser parser = new SubripParser(); @@ -303,6 +383,64 @@ public void parseTypicalBadTimestamps() throws IOException { assertTypicalCue1(Iterables.getOnlyElement(allCues)); } + private enum CharsetTestCase { + GB18030("起来 快起来", Charset.forName("GB18030")), + EUC_KR("이것은 한국어 자막 테스트입니다.", Charset.forName("EUC-KR")), + SHIFT_JIS("これは日本語の字幕テストです。", Charset.forName("Shift_JIS")), + UTF_8("This is a UTF-8 subtitle.", StandardCharsets.UTF_8), + US_ASCII("This is an ASCII subtitle.", StandardCharsets.US_ASCII); + + private final Charset charset; + private final String text; + + private CharsetTestCase(String text, Charset charset) { + this.charset = charset; + this.text = text; + } + } + + @Test + public void parse_withCharsetDetector_outputsDecodedText( + @TestParameter CharsetTestCase testCase) { + byte[] bytes = + ("1\r\n" + "00:00:00,000 --> 00:00:05,000\r\n" + testCase.text + "\r\n") + .getBytes(testCase.charset); + SubripParser parser = + new SubripParser( + /* charsetDetector= */ (data, offset, length) -> testCase.charset, + STANDALONE_SUBRIP_FORMAT); + + ImmutableList allCues = parseAllCues(parser, bytes); + + assertThat(allCues).hasSize(1); + assertThat(allCues.get(0).cues.get(0).text.toString()).isEqualTo(testCase.text); + } + + @Test + public void parse_atOffsetWithCharsetDetector_passesRequestedRangeToDetector() { + Charset charset = Charset.forName("GB18030"); + int offset = 5; + int padding = 7; + String text = "这是一个字幕测试。"; + byte[] bytes = + Bytes.concat( + new byte[offset], + ("1\r\n" + "00:00:00,000 --> 00:00:05,000\r\n" + text + "\r\n").getBytes(charset), + new byte[padding]); + int subtitleLength = bytes.length - offset - padding; + CharsetDetector charsetDetector = mock(CharsetDetector.class); + when(charsetDetector.detect(/* data= */ any(), /* offset= */ anyInt(), /* length= */ anyInt())) + .thenReturn(charset); + SubripParser parser = new SubripParser(charsetDetector, STANDALONE_SUBRIP_FORMAT); + + List allCues = new ArrayList<>(); + parser.parse(bytes, offset, subtitleLength, OutputOptions.allCues(), allCues::add); + + assertThat(allCues).hasSize(1); + assertThat(allCues.get(0).cues.get(0).text.toString()).isEqualTo(text); + verify(charsetDetector).detect(same(bytes), eq(offset), eq(subtitleLength)); + } + private static ImmutableList parseAllCues(SubtitleParser parser, byte[] data) { ImmutableList.Builder cues = ImmutableList.builder(); parser.parse(data, OutputOptions.allCues(), cues::add); diff --git a/libraries/extractor/src/test/java/androidx/media3/extractor/text/ttml/TtmlParserTest.java b/libraries/extractor/src/test/java/androidx/media3/extractor/text/ttml/TtmlParserTest.java index 7816876ba7b..8ca46588f56 100644 --- a/libraries/extractor/src/test/java/androidx/media3/extractor/text/ttml/TtmlParserTest.java +++ b/libraries/extractor/src/test/java/androidx/media3/extractor/text/ttml/TtmlParserTest.java @@ -18,6 +18,7 @@ import static androidx.media3.test.utils.truth.SpannedSubject.assertThat; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.truth.Truth.assertThat; +import static java.nio.charset.StandardCharsets.UTF_8; import android.text.Layout; import android.text.Spanned; @@ -713,6 +714,68 @@ public void fontSizeWithEmptyValueIsIgnored() throws Exception { assertThat(spanned).hasNoAbsoluteSizeSpanBetween(0, spanned.length()); } + @Test + public void parse_fontSizeWithNonDotSeparator_ignoresFontSize() { + String ttml = + "" + + "

    " + + "

    comma decimal

    " + + "
    "; + TtmlParser ttmlParser = new TtmlParser(); + List allCues = new ArrayList<>(); + + ttmlParser.parse(ttml.getBytes(UTF_8), OutputOptions.allCues(), allCues::add); + + assertThat(allCues).hasSize(1); + Spanned spanned = getOnlyCueTextAtIndex(allCues, 0); + assertThat(spanned.toString()).isEqualTo("comma decimal"); + assertThat(spanned).hasNoRelativeSizeSpanBetween(0, spanned.length()); + assertThat(spanned).hasNoAbsoluteSizeSpanBetween(0, spanned.length()); + } + + @Test + public void parse_longInvalidStyleAndRegionNumericAttributes_ignoresAttributes() { + String longMalformedNumber = "9".repeat(100_000) + "X"; + String ttml = + "" + + "" + + "" + + "" + + "
    " + + "

    text

    " + + "
    "; + TtmlParser ttmlParser = new TtmlParser(); + List allCues = new ArrayList<>(); + + ttmlParser.parse(ttml.getBytes(UTF_8), OutputOptions.allCues(), allCues::add); + + assertThat(allCues).hasSize(1); + Spanned spanned = getOnlyCueTextAtIndex(allCues, 0); + assertThat(spanned.toString()).isEqualTo("text"); + } + @Test public void frameRate() throws Exception { ImmutableList allCues = getAllCues(FRAME_RATE_TTML_FILE); diff --git a/libraries/extractor/src/test/java/androidx/media3/extractor/ts/H264ReaderTest.java b/libraries/extractor/src/test/java/androidx/media3/extractor/ts/H264ReaderTest.java new file mode 100644 index 00000000000..bf5ddeb29fb --- /dev/null +++ b/libraries/extractor/src/test/java/androidx/media3/extractor/ts/H264ReaderTest.java @@ -0,0 +1,279 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.extractor.ts; + +import static androidx.media3.extractor.ts.TsPayloadReader.FLAG_DATA_ALIGNMENT_INDICATOR; +import static androidx.media3.test.utils.TestUtil.createByteArray; +import static com.google.common.truth.Truth.assertThat; + +import androidx.media3.common.C; +import androidx.media3.common.Format; +import androidx.media3.common.MimeTypes; +import androidx.media3.common.util.ParsableByteArray; +import androidx.media3.extractor.ts.TsPayloadReader.TrackIdGenerator; +import androidx.media3.test.utils.FakeExtractorOutput; +import androidx.media3.test.utils.FakeTrackOutput; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import com.google.common.primitives.Bytes; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link H264Reader}. */ +@RunWith(AndroidJUnit4.class) +public final class H264ReaderTest { + + private static final byte[] nalStartCode = createByteArray(0x00, 0x00, 0x00, 0x01); + + private static final byte[] audNalUnit = createByteArray(0x09, 0xF0); + + private static final byte[] spsNalUnit = + createByteArray( + 0x67, 0x4D, 0x40, 0x16, 0xEC, 0xA0, 0x50, 0x17, 0xFC, 0xB8, 0x0A, 0x90, 0x91, 0x00, 0x00, + 0x7E, 0xA0); + + private static final byte[] ppsNalUnit = createByteArray(0x68, 0xEE, 0x3C, 0x80); + + // H.264 SEI (0x06) with payloadType = 45 (0x2D), payloadSize = 3 (0x03), side-by-side (type = 3) + private static final byte[] seiFpaSideBySideNalUnit = + createByteArray(0x06, 0x2D, 0x03, 0x81, 0x80, 0x00, 0x80); + + // H.264 SEI (0x06) with payloadType = 45 (0x2D), payloadSize = 3 (0x03), top-bottom (type = 4) + private static final byte[] seiFpaTopBottomNalUnit = + createByteArray(0x06, 0x2D, 0x03, 0x82, 0x00, 0x00, 0x80); + + private static final byte[] idrSliceNalUnit = createByteArray(0x65, 0x88, 0x84, 0x00, 0x33, 0xFF); + + private FakeTrackOutput trackOutput; + private H264Reader reader; + + @Before + public void setUp() { + FakeExtractorOutput extractorOutput = new FakeExtractorOutput(); + SeiReader seiReader = new SeiReader(ImmutableList.of(), MimeTypes.VIDEO_MP2T); + reader = + new H264Reader( + seiReader, + /* allowNonIdrKeyframes= */ false, + /* detectAccessUnits= */ false, + MimeTypes.VIDEO_MP2T); + reader.createTracks(extractorOutput, new TrackIdGenerator(/* firstTrackId= */ 0, 1)); + trackOutput = extractorOutput.track(/* id= */ 0, C.TRACK_TYPE_VIDEO); + } + + @Test + public void consume_withFramePackingSeiAfterSpsPps_updatesFormatStereoMode() { + byte[] streamData = + Bytes.concat( + nalStartCode, + audNalUnit, + nalStartCode, + spsNalUnit, + nalStartCode, + ppsNalUnit, + nalStartCode, + seiFpaSideBySideNalUnit, + nalStartCode, + idrSliceNalUnit); + + reader.packetStarted(/* pesTimeUs= */ 1_000_000L, FLAG_DATA_ALIGNMENT_INDICATOR); + reader.consume(new ParsableByteArray(streamData)); + reader.endOfInputReached(); + + assertThat(trackOutput.getSampleCount()).isEqualTo(1); + assertThat(trackOutput.lastFormat).isNotNull(); + assertThat(trackOutput.lastFormat.stereoMode).isEqualTo(C.STEREO_MODE_LEFT_RIGHT); + } + + @Test + public void consume_withFramePackingSeiBeforeSpsPps_outputsFormatWithStereoMode() { + byte[] streamData = + Bytes.concat( + nalStartCode, + audNalUnit, + nalStartCode, + seiFpaTopBottomNalUnit, + nalStartCode, + spsNalUnit, + nalStartCode, + ppsNalUnit, + nalStartCode, + idrSliceNalUnit); + + reader.packetStarted(/* pesTimeUs= */ 1_000_000L, FLAG_DATA_ALIGNMENT_INDICATOR); + reader.consume(new ParsableByteArray(streamData)); + reader.endOfInputReached(); + + assertThat(trackOutput.getSampleCount()).isEqualTo(1); + assertThat(trackOutput.lastFormat).isNotNull(); + assertThat(trackOutput.lastFormat.stereoMode).isEqualTo(C.STEREO_MODE_TOP_BOTTOM); + } + + @Test + public void consume_withoutFramePackingSei_outputsFormatWithNoValueStereoMode() { + byte[] streamData = + Bytes.concat( + nalStartCode, + audNalUnit, + nalStartCode, + spsNalUnit, + nalStartCode, + ppsNalUnit, + nalStartCode, + idrSliceNalUnit); + + reader.packetStarted(/* pesTimeUs= */ 1_000_000L, FLAG_DATA_ALIGNMENT_INDICATOR); + reader.consume(new ParsableByteArray(streamData)); + reader.endOfInputReached(); + + assertThat(trackOutput.getSampleCount()).isEqualTo(1); + assertThat(trackOutput.lastFormat).isNotNull(); + assertThat(trackOutput.lastFormat.stereoMode).isEqualTo(Format.NO_VALUE); + } + + @Test + public void consume_withFramePackingSeiOnSecondSample_updatesFormatStereoMode() { + byte[] firstSampleData = + Bytes.concat( + nalStartCode, + audNalUnit, + nalStartCode, + spsNalUnit, + nalStartCode, + ppsNalUnit, + nalStartCode, + idrSliceNalUnit); + byte[] secondSampleData = + Bytes.concat( + nalStartCode, + audNalUnit, + nalStartCode, + seiFpaSideBySideNalUnit, + nalStartCode, + idrSliceNalUnit); + + reader.packetStarted(/* pesTimeUs= */ 1_000_000L, FLAG_DATA_ALIGNMENT_INDICATOR); + reader.consume(new ParsableByteArray(firstSampleData)); + + assertThat(trackOutput.lastFormat).isNotNull(); + assertThat(trackOutput.lastFormat.stereoMode).isEqualTo(Format.NO_VALUE); + + reader.packetStarted(/* pesTimeUs= */ 2_000_000L, FLAG_DATA_ALIGNMENT_INDICATOR); + reader.consume(new ParsableByteArray(secondSampleData)); + reader.endOfInputReached(); + + assertThat(trackOutput.getSampleCount()).isEqualTo(2); + assertThat(trackOutput.lastFormat.stereoMode).isEqualTo(C.STEREO_MODE_LEFT_RIGHT); + } + + @Test + public void consume_withDetectAccessUnitsAndSeiOnSecondSample_updatesFormatAfterFirstSample() { + FakeExtractorOutput extractorOutput = new FakeExtractorOutput(); + SeiReader seiReader = new SeiReader(ImmutableList.of(), MimeTypes.VIDEO_MP2T); + H264Reader accessUnitReader = + new H264Reader( + seiReader, + /* allowNonIdrKeyframes= */ false, + /* detectAccessUnits= */ true, + MimeTypes.VIDEO_MP2T); + accessUnitReader.createTracks(extractorOutput, new TrackIdGenerator(/* firstTrackId= */ 0, 1)); + FakeTrackOutput accessUnitTrackOutput = extractorOutput.track(/* id= */ 0, C.TRACK_TYPE_VIDEO); + byte[] firstSampleData = + Bytes.concat( + nalStartCode, spsNalUnit, nalStartCode, ppsNalUnit, nalStartCode, idrSliceNalUnit); + byte[] secondSampleData = + Bytes.concat(nalStartCode, seiFpaSideBySideNalUnit, nalStartCode, idrSliceNalUnit); + + accessUnitReader.packetStarted(/* pesTimeUs= */ 1_000_000L, FLAG_DATA_ALIGNMENT_INDICATOR); + accessUnitReader.consume(new ParsableByteArray(firstSampleData)); + + assertThat(accessUnitTrackOutput.lastFormat).isNotNull(); + assertThat(accessUnitTrackOutput.lastFormat.stereoMode).isEqualTo(Format.NO_VALUE); + + accessUnitReader.packetStarted(/* pesTimeUs= */ 2_000_000L, FLAG_DATA_ALIGNMENT_INDICATOR); + accessUnitReader.consume(new ParsableByteArray(secondSampleData)); + accessUnitReader.endOfInputReached(); + + assertThat(accessUnitTrackOutput.getSampleCount()).isEqualTo(2); + assertThat(accessUnitTrackOutput.lastFormat.stereoMode).isEqualTo(C.STEREO_MODE_LEFT_RIGHT); + } + + @Test + public void consume_withSpsPpsBeforeAudAndFramePackingSei_outputsFormatWithStereoMode() { + byte[] streamData = + Bytes.concat( + nalStartCode, + audNalUnit, + nalStartCode, + spsNalUnit, + nalStartCode, + ppsNalUnit, + nalStartCode, + audNalUnit, + nalStartCode, + seiFpaSideBySideNalUnit, + nalStartCode, + idrSliceNalUnit); + + reader.packetStarted(/* pesTimeUs= */ 1_000_000L, FLAG_DATA_ALIGNMENT_INDICATOR); + reader.consume(new ParsableByteArray(streamData)); + reader.endOfInputReached(); + + assertThat(trackOutput.getSampleCount()).isEqualTo(1); + assertThat(trackOutput.lastFormat).isNotNull(); + assertThat(trackOutput.lastFormat.stereoMode).isEqualTo(C.STEREO_MODE_LEFT_RIGHT); + } + + @Test + public void consume_withFramePackingCancelSeiAfter3dSample_updatesFormatToMono() { + // H.264 SEI (0x06) with payloadType = 45 (0x2D), payloadSize = 1 (0x01), cancel_flag = 1 (0xC0) + byte[] seiFpaCancelNalUnit = createByteArray(0x06, 0x2D, 0x01, 0xC0, 0x80); + byte[] firstSampleData = + Bytes.concat( + nalStartCode, + audNalUnit, + nalStartCode, + spsNalUnit, + nalStartCode, + ppsNalUnit, + nalStartCode, + seiFpaSideBySideNalUnit, + nalStartCode, + idrSliceNalUnit); + byte[] secondSampleData = + Bytes.concat( + nalStartCode, + audNalUnit, + nalStartCode, + seiFpaCancelNalUnit, + nalStartCode, + idrSliceNalUnit); + + reader.packetStarted(/* pesTimeUs= */ 1_000_000L, FLAG_DATA_ALIGNMENT_INDICATOR); + reader.consume(new ParsableByteArray(firstSampleData)); + + assertThat(trackOutput.lastFormat).isNotNull(); + assertThat(trackOutput.lastFormat.stereoMode).isEqualTo(C.STEREO_MODE_LEFT_RIGHT); + + reader.packetStarted(/* pesTimeUs= */ 2_000_000L, FLAG_DATA_ALIGNMENT_INDICATOR); + reader.consume(new ParsableByteArray(secondSampleData)); + reader.endOfInputReached(); + + assertThat(trackOutput.getSampleCount()).isEqualTo(2); + assertThat(trackOutput.lastFormat.stereoMode).isEqualTo(C.STEREO_MODE_MONO); + } +} diff --git a/libraries/extractor/src/test/java/androidx/media3/extractor/ts/H265ReaderTest.java b/libraries/extractor/src/test/java/androidx/media3/extractor/ts/H265ReaderTest.java new file mode 100644 index 00000000000..893762c8f37 --- /dev/null +++ b/libraries/extractor/src/test/java/androidx/media3/extractor/ts/H265ReaderTest.java @@ -0,0 +1,243 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.extractor.ts; + +import static androidx.media3.extractor.ts.TsPayloadReader.FLAG_DATA_ALIGNMENT_INDICATOR; +import static androidx.media3.test.utils.TestUtil.createByteArray; +import static com.google.common.truth.Truth.assertThat; + +import androidx.media3.common.C; +import androidx.media3.common.Format; +import androidx.media3.common.MimeTypes; +import androidx.media3.common.util.ParsableByteArray; +import androidx.media3.extractor.ts.TsPayloadReader.TrackIdGenerator; +import androidx.media3.test.utils.FakeExtractorOutput; +import androidx.media3.test.utils.FakeTrackOutput; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import com.google.common.primitives.Bytes; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link H265Reader}. */ +@RunWith(AndroidJUnit4.class) +public final class H265ReaderTest { + + private static final byte[] nalStartCode = createByteArray(0x00, 0x00, 0x00, 0x01); + + private static final byte[] vpsNalUnit = + createByteArray( + 0x40, 0x01, 0x0C, 0x01, 0xFF, 0xFF, 0x22, 0x20, 0x00, 0x00, 0x03, 0x00, 0xB0, 0x00, 0x00, + 0x03, 0x00, 0x00, 0x03, 0x00, 0x99, 0x2C, 0x09); + + private static final byte[] spsNalUnit = + createByteArray( + 0x42, 0x01, 0x01, 0x22, 0x20, 0x00, 0x00, 0x03, 0x00, 0xB0, 0x00, 0x00, 0x03, 0x00, 0x00, + 0x03, 0x00, 0x99, 0xA0, 0x01, 0xE0, 0x20, 0x02, 0x1C, 0x4D, 0x94, 0xBB, 0xB4, 0xA3, 0x32, + 0xAA, 0xC0, 0x5A, 0x84, 0x89, 0x04, 0x8A, 0x00, 0x00, 0x07, 0xD2, 0x00, 0x00, 0xBB, 0x80, + 0xE4, 0x68, 0x7C, 0x9C, 0x00, 0x01, 0x2E, 0x1F, 0x80, 0x00, 0x21, 0xFD, 0x30, 0x00, 0x02, + 0x5C, 0x3F, 0x00, 0x00, 0x43, 0xFA, 0x62); + + private static final byte[] ppsNalUnit = + createByteArray(0x44, 0x01, 0xC0, 0xF7, 0xC0, 0xCC, 0x90); + + // H.265 PREFIX_SEI (0x4E, 0x01) with payloadType = 45 (0x2D), payloadSize = 3 (0x03), + // side-by-side (type = 3) + private static final byte[] prefixSeiFpaSideBySideNalUnit = + createByteArray(0x4E, 0x01, 0x2D, 0x03, 0x81, 0x80, 0x00, 0x80); + + // H.265 PREFIX_SEI (0x4E, 0x01) with payloadType = 45 (0x2D), payloadSize = 3 (0x03), + // top-bottom (type = 4) + private static final byte[] prefixSeiFpaTopBottomNalUnit = + createByteArray(0x4E, 0x01, 0x2D, 0x03, 0x82, 0x00, 0x00, 0x80); + + private static final byte[] idrSliceNalUnit = createByteArray(0x26, 0x01, 0xAF, 0x08, 0x40, 0x00); + + private FakeTrackOutput trackOutput; + private H265Reader reader; + + @Before + public void setUp() { + FakeExtractorOutput extractorOutput = new FakeExtractorOutput(); + SeiReader seiReader = new SeiReader(ImmutableList.of(), MimeTypes.VIDEO_MP2T); + reader = new H265Reader(seiReader, MimeTypes.VIDEO_MP2T); + reader.createTracks(extractorOutput, new TrackIdGenerator(/* firstTrackId= */ 0, 1)); + trackOutput = extractorOutput.track(/* id= */ 0, C.TRACK_TYPE_VIDEO); + } + + @Test + public void consume_withFramePackingPrefixSeiAfterVpsSpsPps_updatesFormatStereoMode() { + byte[] streamData = + Bytes.concat( + nalStartCode, + vpsNalUnit, + nalStartCode, + spsNalUnit, + nalStartCode, + ppsNalUnit, + nalStartCode, + prefixSeiFpaSideBySideNalUnit, + nalStartCode, + idrSliceNalUnit); + + reader.packetStarted(/* pesTimeUs= */ 1_000_000L, FLAG_DATA_ALIGNMENT_INDICATOR); + reader.consume(new ParsableByteArray(streamData)); + reader.endOfInputReached(); + + assertThat(trackOutput.getSampleCount()).isEqualTo(1); + assertThat(trackOutput.lastFormat).isNotNull(); + assertThat(trackOutput.lastFormat.stereoMode).isEqualTo(C.STEREO_MODE_LEFT_RIGHT); + } + + @Test + public void consume_withFramePackingPrefixSeiBeforeVpsSpsPps_outputsFormatWithStereoMode() { + byte[] streamData = + Bytes.concat( + nalStartCode, + prefixSeiFpaTopBottomNalUnit, + nalStartCode, + vpsNalUnit, + nalStartCode, + spsNalUnit, + nalStartCode, + ppsNalUnit, + nalStartCode, + idrSliceNalUnit); + + reader.packetStarted(/* pesTimeUs= */ 1_000_000L, FLAG_DATA_ALIGNMENT_INDICATOR); + reader.consume(new ParsableByteArray(streamData)); + reader.endOfInputReached(); + + assertThat(trackOutput.getSampleCount()).isEqualTo(1); + assertThat(trackOutput.lastFormat).isNotNull(); + assertThat(trackOutput.lastFormat.stereoMode).isEqualTo(C.STEREO_MODE_TOP_BOTTOM); + } + + @Test + public void consume_withoutFramePackingPrefixSei_outputsFormatWithNoValueStereoMode() { + byte[] streamData = + Bytes.concat( + nalStartCode, + vpsNalUnit, + nalStartCode, + spsNalUnit, + nalStartCode, + ppsNalUnit, + nalStartCode, + idrSliceNalUnit); + + reader.packetStarted(/* pesTimeUs= */ 1_000_000L, FLAG_DATA_ALIGNMENT_INDICATOR); + reader.consume(new ParsableByteArray(streamData)); + reader.endOfInputReached(); + + assertThat(trackOutput.getSampleCount()).isEqualTo(1); + assertThat(trackOutput.lastFormat).isNotNull(); + assertThat(trackOutput.lastFormat.stereoMode).isEqualTo(Format.NO_VALUE); + } + + @Test + public void consume_withFramePackingPrefixSeiOnSecondSample_updatesFormatStereoMode() { + byte[] firstSampleData = + Bytes.concat( + nalStartCode, + vpsNalUnit, + nalStartCode, + spsNalUnit, + nalStartCode, + ppsNalUnit, + nalStartCode, + idrSliceNalUnit); + byte[] secondSampleData = + Bytes.concat(nalStartCode, prefixSeiFpaSideBySideNalUnit, nalStartCode, idrSliceNalUnit); + + reader.packetStarted(/* pesTimeUs= */ 1_000_000L, FLAG_DATA_ALIGNMENT_INDICATOR); + reader.consume(new ParsableByteArray(firstSampleData)); + + assertThat(trackOutput.lastFormat).isNotNull(); + assertThat(trackOutput.lastFormat.stereoMode).isEqualTo(Format.NO_VALUE); + + reader.packetStarted(/* pesTimeUs= */ 2_000_000L, FLAG_DATA_ALIGNMENT_INDICATOR); + reader.consume(new ParsableByteArray(secondSampleData)); + reader.endOfInputReached(); + + assertThat(trackOutput.getSampleCount()).isEqualTo(2); + assertThat(trackOutput.lastFormat.stereoMode).isEqualTo(C.STEREO_MODE_LEFT_RIGHT); + } + + @Test + public void consume_withVpsSpsPpsBetweenAudsAndPrefixSei_outputsFormatWithStereoMode() { + byte[] audNalUnit = createByteArray(0x46, 0x01, 0x50); + byte[] streamData = + Bytes.concat( + nalStartCode, + audNalUnit, + nalStartCode, + vpsNalUnit, + nalStartCode, + spsNalUnit, + nalStartCode, + ppsNalUnit, + nalStartCode, + audNalUnit, + nalStartCode, + prefixSeiFpaSideBySideNalUnit, + nalStartCode, + idrSliceNalUnit); + + reader.packetStarted(/* pesTimeUs= */ 1_000_000L, FLAG_DATA_ALIGNMENT_INDICATOR); + reader.consume(new ParsableByteArray(streamData)); + reader.endOfInputReached(); + + assertThat(trackOutput.getSampleCount()).isEqualTo(1); + assertThat(trackOutput.lastFormat).isNotNull(); + assertThat(trackOutput.lastFormat.stereoMode).isEqualTo(C.STEREO_MODE_LEFT_RIGHT); + } + + @Test + public void consume_withFramePackingCancelPrefixSeiAfter3dSample_updatesFormatToMono() { + // H.265 PREFIX_SEI (0x4E, 0x01) with payloadType = 45 (0x2D), payloadSize = 1 (0x01), + // cancel_flag = 1 (0xC0) + byte[] prefixSeiFpaCancelNalUnit = createByteArray(0x4E, 0x01, 0x2D, 0x01, 0xC0, 0x80); + byte[] firstSampleData = + Bytes.concat( + nalStartCode, + vpsNalUnit, + nalStartCode, + spsNalUnit, + nalStartCode, + ppsNalUnit, + nalStartCode, + prefixSeiFpaSideBySideNalUnit, + nalStartCode, + idrSliceNalUnit); + byte[] secondSampleData = + Bytes.concat(nalStartCode, prefixSeiFpaCancelNalUnit, nalStartCode, idrSliceNalUnit); + + reader.packetStarted(/* pesTimeUs= */ 1_000_000L, FLAG_DATA_ALIGNMENT_INDICATOR); + reader.consume(new ParsableByteArray(firstSampleData)); + + assertThat(trackOutput.lastFormat).isNotNull(); + assertThat(trackOutput.lastFormat.stereoMode).isEqualTo(C.STEREO_MODE_LEFT_RIGHT); + + reader.packetStarted(/* pesTimeUs= */ 2_000_000L, FLAG_DATA_ALIGNMENT_INDICATOR); + reader.consume(new ParsableByteArray(secondSampleData)); + reader.endOfInputReached(); + + assertThat(trackOutput.getSampleCount()).isEqualTo(2); + assertThat(trackOutput.lastFormat.stereoMode).isEqualTo(C.STEREO_MODE_MONO); + } +} diff --git a/libraries/extractor/src/test/java/androidx/media3/extractor/ts/PsExtractorTest.java b/libraries/extractor/src/test/java/androidx/media3/extractor/ts/PsExtractorTest.java index 4f82fd4f945..d329249a62b 100644 --- a/libraries/extractor/src/test/java/androidx/media3/extractor/ts/PsExtractorTest.java +++ b/libraries/extractor/src/test/java/androidx/media3/extractor/ts/PsExtractorTest.java @@ -44,4 +44,12 @@ public void sampleWithH262AndMpegAudio() throws Exception { public void sampleWithAc3() throws Exception { ExtractorAsserts.assertBehavior(PsExtractor::new, "media/ts/sample_ac3.ps", simulationConfig); } + + @Test + public void extract_withAc3StraddlingPesBoundaries_stripsSubStreamHeader() throws Exception { + // Regression test for a private_stream_1 (0xBD) sample whose AC-3 frames straddle PES packet + // boundaries, so the DVD sub-stream header preceding each packet's payload must be stripped. + ExtractorAsserts.assertBehavior( + PsExtractor::new, "media/ts/sample_ac3_straddle.ps", simulationConfig); + } } diff --git a/libraries/inspector/src/androidTest/java/androidx/media3/inspector/MediaExtractorCompatTest.java b/libraries/inspector/src/androidTest/java/androidx/media3/inspector/MediaExtractorCompatTest.java index 89a555ca26f..2e8f4c169d8 100644 --- a/libraries/inspector/src/androidTest/java/androidx/media3/inspector/MediaExtractorCompatTest.java +++ b/libraries/inspector/src/androidTest/java/androidx/media3/inspector/MediaExtractorCompatTest.java @@ -21,6 +21,7 @@ import static androidx.media3.common.MimeTypes.VIDEO_H264; import static androidx.media3.common.MimeTypes.VIDEO_MP4; import static androidx.media3.test.utils.TestUtil.buildTestData; +import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.truth.Truth.assertThat; import static org.junit.Assert.assertThrows; @@ -34,6 +35,8 @@ import android.media.metrics.PlaybackSession; import android.net.Uri; import android.os.PersistableBundle; +import android.system.ErrnoException; +import android.system.Os; import androidx.annotation.Nullable; import androidx.media3.common.C; import androidx.media3.common.DrmInitData; @@ -63,6 +66,8 @@ import com.google.common.io.Files; import com.google.common.primitives.Bytes; import java.io.File; +import java.io.FileDescriptor; +import java.io.FileInputStream; import java.io.IOException; import java.nio.ByteBuffer; import java.util.ArrayList; @@ -483,7 +488,7 @@ public void seek_withUninterleavedFile_seeksToTheRightPosition() throws IOExcept // We don't use the global mediaExtractorCompat because we want to use a real extractor in this // case, which is the Mp4 extractor. MediaExtractorCompat mediaExtractorCompat = - new MediaExtractorCompat(ApplicationProvider.getApplicationContext()); + new MediaExtractorCompat((Context) ApplicationProvider.getApplicationContext()); // The asset is an uninterleaved mp4 file. mediaExtractorCompat.setDataSource( Uri.parse("asset:///media/mp4/mv_with_2_top_shots.mp4"), /* offset= */ 0); @@ -1163,8 +1168,73 @@ public void getTrackFormat_withProfileAndLevelInCodecString_returnsExpectedProfi .isEqualTo(MediaCodecInfo.CodecProfileLevel.AVCLevel1); } + @Test + public void setDataSource_withFileDescriptor_closesDuplicatedFileDescriptorOnRelease() + throws IOException { + fakeExtractor.addReadAction( + (input, seekPosition) -> { + extractorOutput.endTracks(); + return Extractor.RESULT_END_OF_INPUT; + }); + File file = tempFolder.newFile(); + try (FileInputStream inputStream = new FileInputStream(file)) { + FileDescriptor fd = inputStream.getFD(); + mediaExtractorCompat.setDataSource(fd); + assertThrows(IllegalStateException.class, () -> mediaExtractorCompat.setDataSource(fd)); + } + + assertThat(getOpenFileDescriptorCount(file)).isEqualTo(1); + mediaExtractorCompat.release(); + assertThat(getOpenFileDescriptorCount(file)).isEqualTo(0); + } + + // Happy-path extraction with the Context-less MediaExtractorCompat() constructor is tested by + // MediaExtractorContractTest. + + @Test + public void setDataSource_withoutContextAndAssetUri_throwsIOException() { + MediaExtractorCompat extractor = new MediaExtractorCompat(); + Uri assetUri = Uri.parse("asset:///media/mp4/sample.mp4"); + + assertThrows(IOException.class, () -> extractor.setDataSource(assetUri, /* offset= */ 0)); + extractor.release(); + } + + @Test + public void setDataSource_withoutContextAndFileAssetUri_throwsIOException() { + MediaExtractorCompat extractor = new MediaExtractorCompat(); + Uri fileAssetUri = Uri.parse("file:///android_asset/media/mp4/sample.mp4"); + + assertThrows(IOException.class, () -> extractor.setDataSource(fileAssetUri, /* offset= */ 0)); + extractor.release(); + } + + @Test + public void setDataSource_withoutContextAndContentUri_throwsIOException() { + MediaExtractorCompat extractor = new MediaExtractorCompat(); + + assertThrows( + IOException.class, + () -> extractor.setDataSource("content://authority/media/mp4/sample.mp4")); + extractor.release(); + } + // Internal methods. + private static int getOpenFileDescriptorCount(File file) { + int count = 0; + for (File fdEntry : checkNotNull(new File("/proc/self/fd").listFiles())) { + try { + if (Os.readlink(fdEntry.getAbsolutePath()).endsWith(file.getName())) { + count++; + } + } catch (ErrnoException e) { + // FD may have been closed concurrently; ignore. + } + } + return count; + } + private void assertReadSample(int trackIndex, long timeUs, int size, byte... sampleData) { assertThat(mediaExtractorCompat.getSampleTrackIndex()).isEqualTo(trackIndex); assertThat(mediaExtractorCompat.getSampleTime()).isEqualTo(timeUs); diff --git a/libraries/inspector/src/androidTest/java/androidx/media3/inspector/MediaExtractorContractTest.java b/libraries/inspector/src/androidTest/java/androidx/media3/inspector/MediaExtractorContractTest.java index 3ea179a9fbd..78cbb939693 100644 --- a/libraries/inspector/src/androidTest/java/androidx/media3/inspector/MediaExtractorContractTest.java +++ b/libraries/inspector/src/androidTest/java/androidx/media3/inspector/MediaExtractorContractTest.java @@ -42,7 +42,7 @@ import androidx.media3.test.utils.TestUtil; import androidx.test.core.app.ApplicationProvider; import androidx.test.filters.SdkSuppress; -import com.google.common.base.Function; +import com.google.common.base.Supplier; import com.google.common.collect.ImmutableList; import com.google.common.io.Files; import java.io.File; @@ -85,12 +85,11 @@ public class MediaExtractorContractTest { 0x72, 0x73, 0x2A, 0x02, 0x53, 0x44); @Parameters(name = "{0}") - public static ImmutableList> - mediaExtractorProxyFactories() { + public static ImmutableList> mediaExtractorProxyFactories() { return ImmutableList.of( - new Function() { + new Supplier() { @Override - public MediaExtractorProxy apply(Context context) { + public MediaExtractorProxy get() { return new FrameworkMediaExtractorProxy(); } @@ -99,10 +98,10 @@ public String toString() { return FrameworkMediaExtractorProxy.class.getSimpleName(); } }, - new Function() { + new Supplier() { @Override - public MediaExtractorProxy apply(Context context) { - return new CompatMediaExtractorProxy(context); + public MediaExtractorProxy get() { + return new CompatMediaExtractorProxy(); } @Override @@ -113,7 +112,7 @@ public String toString() { } @Rule public final TemporaryFolder tempFolder = new TemporaryFolder(); - @Parameter public Function mediaExtractorProxyFactory; + @Parameter public Supplier mediaExtractorProxyFactory; private MediaExtractorProxy mediaExtractorProxy; private Context context; @@ -121,7 +120,7 @@ public String toString() { @Before public void setUp() { context = ApplicationProvider.getApplicationContext(); - mediaExtractorProxy = mediaExtractorProxyFactory.apply(context); + mediaExtractorProxy = mediaExtractorProxyFactory.get(); } @After @@ -130,36 +129,44 @@ public void tearDown() { } @Test - @SdkSuppress(minSdkVersion = 24) - public void setDataSource_withAssetFileDescriptor_returnsCorrectTrackCount() throws IOException { - AssetFileDescriptor afd = context.getAssets().openFd("media/mp4/sample.mp4"); - - mediaExtractorProxy.setDataSource(afd); + public void setDataSource_withAssetFileDescriptor_extractsSamplesAfterClosingFd() + throws IOException { + try (AssetFileDescriptor afd = context.getAssets().openFd("media/mp4/sample.mp4")) { + mediaExtractorProxy.setDataSource(afd); + } assertThat(mediaExtractorProxy.getTrackCount()).isEqualTo(2); + mediaExtractorProxy.selectTrack(0); + mediaExtractorProxy.seekTo(/* timeUs= */ 0, MediaExtractorCompat.SEEK_TO_CLOSEST_SYNC); + assertThat(mediaExtractorProxy.advance()).isTrue(); } @Test - public void setDataSource_withFileDescriptor_returnsCorrectTrackCount() throws IOException { + public void setDataSource_withFileDescriptor_extractsSamplesAfterClosingFd() throws IOException { File file = tempFolder.newFile(); Files.write(TestUtil.getByteArray(context, /* fileName= */ "media/mp4/sample.mp4"), file); - try (FileInputStream inputStream = new FileInputStream(file)) { mediaExtractorProxy.setDataSource(inputStream.getFD()); } assertThat(mediaExtractorProxy.getTrackCount()).isEqualTo(2); + mediaExtractorProxy.selectTrack(0); + mediaExtractorProxy.seekTo(/* timeUs= */ 0, MediaExtractorCompat.SEEK_TO_CLOSEST_SYNC); + assertThat(mediaExtractorProxy.advance()).isTrue(); } @Test - public void setDataSource_withFileDescriptorOffsetAndLength_returnsCorrectTrackCount() + public void setDataSource_withFileDescriptorOffsetAndLength_extractsSamplesAfterClosingFd() throws IOException { - AssetFileDescriptor afd = context.getAssets().openFd("media/mp4/sample.mp4"); - - mediaExtractorProxy.setDataSource( - afd.getFileDescriptor(), afd.getStartOffset(), afd.getLength()); + try (AssetFileDescriptor afd = context.getAssets().openFd("media/mp4/sample.mp4")) { + mediaExtractorProxy.setDataSource( + afd.getFileDescriptor(), afd.getStartOffset(), afd.getLength()); + } assertThat(mediaExtractorProxy.getTrackCount()).isEqualTo(2); + mediaExtractorProxy.selectTrack(0); + mediaExtractorProxy.seekTo(/* timeUs= */ 0, MediaExtractorCompat.SEEK_TO_CLOSEST_SYNC); + assertThat(mediaExtractorProxy.advance()).isTrue(); } @Test @@ -236,13 +243,16 @@ public void setDataSource_withHttpUrlAsStringAndHeaders_returnsCorrectTrackCount } @Test - public void setDataSource_withContentUri_returnsCorrectTrackCount() throws IOException { + public void setDataSource_withContentUri_extractsSamples() throws IOException { Uri contentUri = AssetContentProvider.buildUri(AUTHORITY, "media/mp4/sample.mp4", /* pipeMode= */ false); mediaExtractorProxy.setDataSource(context, contentUri, /* headers= */ null); assertThat(mediaExtractorProxy.getTrackCount()).isEqualTo(2); + mediaExtractorProxy.selectTrack(0); + mediaExtractorProxy.seekTo(/* timeUs= */ 0, MediaExtractorCompat.SEEK_TO_CLOSEST_SYNC); + assertThat(mediaExtractorProxy.advance()).isTrue(); } @Test @@ -447,8 +457,6 @@ public void selectTrack(int trackIndex) { } @Override - @SuppressWarnings("UseSdkSuppress") // https://issuetracker.google.com/382253664 - @RequiresApi(24) public void setDataSource(AssetFileDescriptor assetFileDescriptor) throws IOException { mediaExtractor.setDataSource(assetFileDescriptor); } @@ -503,8 +511,8 @@ private static class CompatMediaExtractorProxy implements MediaExtractorProxy { private final MediaExtractorCompat mediaExtractorCompat; - public CompatMediaExtractorProxy(Context context) { - this.mediaExtractorCompat = new MediaExtractorCompat(context); + private CompatMediaExtractorProxy() { + this.mediaExtractorCompat = new MediaExtractorCompat(); } @Override @@ -615,8 +623,6 @@ public void selectTrack(int trackIndex) { } @Override - @SuppressWarnings("UseSdkSuppress") // https://issuetracker.google.com/382253664 - @RequiresApi(24) public void setDataSource(AssetFileDescriptor assetFileDescriptor) throws IOException { mediaExtractorCompat.setDataSource(assetFileDescriptor); } @@ -714,8 +720,6 @@ private interface MediaExtractorProxy { void selectTrack(int trackIndex); - @SuppressWarnings("UseSdkSuppress") // https://issuetracker.google.com/382253664 - @RequiresApi(24) void setDataSource(AssetFileDescriptor assetFileDescriptor) throws IOException; void setDataSource(FileDescriptor fileDescriptor) throws IOException; diff --git a/libraries/inspector/src/main/java/androidx/media3/inspector/MediaExtractorCompat.java b/libraries/inspector/src/main/java/androidx/media3/inspector/MediaExtractorCompat.java index 277c4bb377e..585ebb605d4 100644 --- a/libraries/inspector/src/main/java/androidx/media3/inspector/MediaExtractorCompat.java +++ b/libraries/inspector/src/main/java/androidx/media3/inspector/MediaExtractorCompat.java @@ -16,6 +16,8 @@ package androidx.media3.inspector; import static androidx.annotation.VisibleForTesting.NONE; +import static com.google.common.base.Preconditions.checkNotNull; +import static com.google.common.base.Preconditions.checkState; import android.content.Context; import android.content.res.AssetFileDescriptor; @@ -34,8 +36,13 @@ import androidx.media3.common.DrmInitData; import androidx.media3.common.MediaLibraryInfo; import androidx.media3.common.util.UnstableApi; +import androidx.media3.common.util.Util; import androidx.media3.datasource.DataSource; +import androidx.media3.datasource.DataSpec; import androidx.media3.datasource.DefaultDataSource; +import androidx.media3.datasource.DefaultHttpDataSource; +import androidx.media3.datasource.FileDataSource; +import androidx.media3.datasource.TransferListener; import androidx.media3.exoplayer.source.BundledExtractorsAdapter; import androidx.media3.exoplayer.source.ProgressiveMediaExtractor; import androidx.media3.exoplayer.source.UnrecognizedInputFormatException; @@ -43,11 +50,14 @@ import androidx.media3.extractor.DefaultExtractorsFactory; import androidx.media3.extractor.Extractor; import androidx.media3.extractor.ExtractorsFactory; +import com.google.common.base.Ascii; +import com.google.common.collect.ImmutableMap; import java.io.FileDescriptor; import java.io.IOException; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.nio.ByteBuffer; +import java.util.List; import java.util.Map; import java.util.UUID; @@ -85,11 +95,41 @@ public final class MediaExtractorCompat { private final MediaExtractorCompatInternal delegate; - /** Creates a new instance. */ + /** + * Creates a new instance using a {@link DefaultExtractorsFactory} and a default {@link + * DataSource.Factory} that supports local files and HTTP/HTTPS streams. + * + *

    Note: Without a {@link Context}, {@code content://} and {@code android.resource://} URIs are + * only supported via {@link #setDataSource(Context, Uri, Map)}. For full {@link + * DefaultDataSource} support (including {@code asset://} URIs), use {@link + * #MediaExtractorCompat(Context)}. + */ + public MediaExtractorCompat() { + this(new DefaultExtractorsFactory(), MediaExtractorDataSource.FACTORY); + } + + /** + * Creates a new instance using a {@link DefaultExtractorsFactory} and a {@link + * DefaultDataSource.Factory} configured with the given {@link Context}. + */ public MediaExtractorCompat(Context context) { this(new DefaultExtractorsFactory(), new DefaultDataSource.Factory(context)); } + /** + * Creates a new instance using the given {@link ProgressiveMediaExtractor} and a default {@link + * DataSource.Factory} that supports local files and HTTP/HTTPS streams. + * + *

    Note: Without a {@link Context}, {@code content://} and {@code android.resource://} URIs are + * only supported via {@link #setDataSource(Context, Uri, Map)}. For full {@link + * DefaultDataSource} support (including {@code asset://} URIs), use {@link + * #MediaExtractorCompat(ProgressiveMediaExtractor, DataSource.Factory)} with a {@link + * DefaultDataSource.Factory}. + */ + public MediaExtractorCompat(ProgressiveMediaExtractor progressiveMediaExtractor) { + this(progressiveMediaExtractor, MediaExtractorDataSource.FACTORY); + } + /** * Creates a new instance using the given {@link ExtractorsFactory} to create the {@linkplain * Extractor extractors} to use for obtaining media samples from a {@link DataSource} generated by @@ -170,6 +210,9 @@ public void setDataSource(AssetFileDescriptor assetFileDescriptor) throws IOExce /** * Sets the data source using the media stream obtained from the provided {@link FileDescriptor}. * + *

    Note: The caller is responsible for closing the {@link FileDescriptor}. It is safe to do so + * immediately after this method returns. + * * @param fileDescriptor The {@link FileDescriptor} for the file to extract from. * @throws IOException If an error occurs while extracting the media. * @throws UnrecognizedInputFormatException If none of the available extractors successfully @@ -184,6 +227,9 @@ public void setDataSource(FileDescriptor fileDescriptor) throws IOException { * Sets the data source using the media stream obtained from the provided {@link FileDescriptor}, * with a specified {@code offset} and {@code length}. * + *

    Note: The caller is responsible for closing the {@link FileDescriptor}. It is safe to do so + * immediately after this method returns. + * * @param fileDescriptor The {@link FileDescriptor} for the file to extract from. * @param offset The offset into the file where the data to be extracted starts, in bytes. * @param length The length of the data to be extracted, in bytes, or {@link C#LENGTH_UNSET} if it @@ -441,4 +487,74 @@ public Map getPsshInfo() { public Allocator getAllocator() { return delegate.getAllocator(); } + + /** + * A {@link DataSource} that supports only the URI schemes handled by framework {@link + * MediaExtractor} when no {@link Context} is provided (local files and HTTP/HTTPS streams). + */ + private static final class MediaExtractorDataSource implements DataSource { + + private static final DataSource.Factory FACTORY = MediaExtractorDataSource::new; + + @Nullable private DataSource fileDataSource; + @Nullable private DataSource httpDataSource; + @Nullable private DataSource currentDataSource; + + @Override + public void addTransferListener(TransferListener transferListener) { + // Do nothing: MediaExtractorCompatInternal never registers a TransferListener. + } + + @Override + public long open(DataSpec dataSpec) throws IOException { + checkState(currentDataSource == null); + String scheme = dataSpec.uri.getScheme(); + if (Util.isLocalFileUri(dataSpec.uri)) { + String uriPath = dataSpec.uri.getPath(); + if (uriPath != null && uriPath.startsWith("/android_asset/")) { + throw new IOException("Asset URIs require a Context: " + dataSpec.uri); + } + if (fileDataSource == null) { + fileDataSource = new FileDataSource(); + } + currentDataSource = fileDataSource; + } else if (scheme != null + && (Ascii.equalsIgnoreCase(scheme, "http") || Ascii.equalsIgnoreCase(scheme, "https"))) { + if (httpDataSource == null) { + httpDataSource = new DefaultHttpDataSource.Factory().createDataSource(); + } + currentDataSource = httpDataSource; + } else { + throw new IOException("Unsupported URI scheme without a Context: " + scheme); + } + return currentDataSource.open(dataSpec); + } + + @Override + public int read(byte[] buffer, int offset, int length) throws IOException { + return checkNotNull(currentDataSource).read(buffer, offset, length); + } + + @Override + @Nullable + public Uri getUri() { + return currentDataSource == null ? null : currentDataSource.getUri(); + } + + @Override + public Map> getResponseHeaders() { + return currentDataSource == null ? ImmutableMap.of() : currentDataSource.getResponseHeaders(); + } + + @Override + public void close() throws IOException { + if (currentDataSource != null) { + try { + currentDataSource.close(); + } finally { + currentDataSource = null; + } + } + } + } } diff --git a/libraries/inspector/src/main/java/androidx/media3/inspector/MediaExtractorCompatInternal.java b/libraries/inspector/src/main/java/androidx/media3/inspector/MediaExtractorCompatInternal.java index a52decb0105..ec59a1e1b3c 100644 --- a/libraries/inspector/src/main/java/androidx/media3/inspector/MediaExtractorCompatInternal.java +++ b/libraries/inspector/src/main/java/androidx/media3/inspector/MediaExtractorCompatInternal.java @@ -35,6 +35,8 @@ import android.media.metrics.LogSessionId; import android.net.Uri; import android.os.PersistableBundle; +import android.system.ErrnoException; +import android.system.Os; import android.util.Pair; import android.util.SparseArray; import androidx.annotation.Nullable; @@ -113,6 +115,7 @@ private boolean hasBeenPrepared; private long offsetInCurrentFile; @Nullable private DataSource currentDataSource; + @Nullable private FileDescriptor duplicatedFileDescriptor; @Nullable private SeekPoint pendingSeek; @Nullable private SeekMap seekMap; @@ -201,6 +204,9 @@ /** * Sets the data source using the media stream obtained from the provided {@link FileDescriptor}. * + *

    Note: The caller is responsible for closing the {@link FileDescriptor}. It is safe to do so + * immediately after this method returns. + * * @param fileDescriptor The {@link FileDescriptor} for the file to extract from. * @throws IOException If an error occurs while extracting the media. * @throws UnrecognizedInputFormatException If none of the available extractors successfully @@ -215,6 +221,9 @@ * Sets the data source using the media stream obtained from the provided {@link FileDescriptor}, * with a specified {@code offset} and {@code length}. * + *

    Note: The caller is responsible for closing the {@link FileDescriptor}. It is safe to do so + * immediately after this method returns. + * * @param fileDescriptor The {@link FileDescriptor} for the file to extract from. * @param offset The offset into the file where the data to be extracted starts, in bytes. * @param length The length of the data to be extracted, in bytes, or {@link C#LENGTH_UNSET} if it @@ -226,8 +235,14 @@ */ /* package */ void setDataSource(FileDescriptor fileDescriptor, long offset, long length) throws IOException { + checkState(!hasBeenPrepared); + try { + duplicatedFileDescriptor = Os.dup(fileDescriptor); + } catch (ErrnoException e) { + throw new IOException("Failed to duplicate FileDescriptor.", e); + } FileDescriptorDataSource fileDescriptorDataSource = - new FileDescriptorDataSource(fileDescriptor, offset, length); + new FileDescriptorDataSource(duplicatedFileDescriptor, offset, length); prepareDataSource(fileDescriptorDataSource, buildDataSpec(Uri.EMPTY, /* position= */ 0)); } @@ -322,38 +337,42 @@ private void prepareDataSource(DataSource dataSource, DataSpec dataSpec) throws offsetInCurrentFile = dataSpec.position; currentDataSource = dataSource; - long length = currentDataSource.open(dataSpec); - progressiveMediaExtractor.init( - currentDataSource, - checkNotNull(currentDataSource.getUri()), - currentDataSource.getResponseHeaders(), - /* position= */ 0, - length, - new ExtractorOutputImpl()); - - boolean preparing = true; - Throwable error = null; - while (preparing) { - int result; - try { - result = progressiveMediaExtractor.read(positionHolder); - } catch (Exception | OutOfMemoryError e) { - // This value is ignored but initializes result to avoid static analysis errors. - result = Extractor.RESULT_END_OF_INPUT; - error = e; - } - preparing = !tracksEnded || upstreamFormatsCount < sampleQueues.size() || seekMap == null; - if (error != null || (preparing && result == Extractor.RESULT_END_OF_INPUT)) { - // TODO(b/178501820): Support files with incomplete track information. - release(); // Release resources as soon as possible, in case we are low on memory. - String message = - error != null - ? "Exception encountered while parsing input media." - : "Reached end of input before preparation completed."; - throw ParserException.createForMalformedContainer(message, /* cause= */ error); - } else if (result == Extractor.RESULT_SEEK) { - reopenCurrentDataSource(positionHolder.position); + try { + long length = currentDataSource.open(dataSpec); + progressiveMediaExtractor.init( + currentDataSource, + checkNotNull(currentDataSource.getUri()), + currentDataSource.getResponseHeaders(), + /* position= */ 0, + length, + new ExtractorOutputImpl()); + + boolean preparing = true; + Throwable error = null; + while (preparing) { + int result; + try { + result = progressiveMediaExtractor.read(positionHolder); + } catch (Exception | OutOfMemoryError e) { + // This value is ignored but initializes result to avoid static analysis errors. + result = Extractor.RESULT_END_OF_INPUT; + error = e; + } + preparing = !tracksEnded || upstreamFormatsCount < sampleQueues.size() || seekMap == null; + if (error != null || (preparing && result == Extractor.RESULT_END_OF_INPUT)) { + // TODO(b/178501820): Support files with incomplete track information. + String message = + error != null + ? "Exception encountered while parsing input media." + : "Reached end of input before preparation completed."; + throw ParserException.createForMalformedContainer(message, /* cause= */ error); + } else if (result == Extractor.RESULT_SEEK) { + reopenCurrentDataSource(positionHolder.position); + } } + } catch (Exception | OutOfMemoryError e) { + release(); // Release resources as soon as possible, in case we are low on memory. + throw e; } // At this point, we know how many tracks we have, and their format. } @@ -373,6 +392,14 @@ private void prepareDataSource(DataSource dataSource, DataSpec dataSpec) throws pendingSeek = null; DataSourceUtil.closeQuietly(currentDataSource); currentDataSource = null; + if (duplicatedFileDescriptor != null) { + try { + Os.close(duplicatedFileDescriptor); + } catch (ErrnoException e) { + Log.e(TAG, "Failed to close duplicated file descriptor.", e); + } + duplicatedFileDescriptor = null; + } } /** Returns the number of tracks found in the data source. */ diff --git a/libraries/inspector/src/main/java/androidx/media3/inspector/MetadataRetriever.java b/libraries/inspector/src/main/java/androidx/media3/inspector/MetadataRetriever.java index 4384547e545..84146818300 100644 --- a/libraries/inspector/src/main/java/androidx/media3/inspector/MetadataRetriever.java +++ b/libraries/inspector/src/main/java/androidx/media3/inspector/MetadataRetriever.java @@ -117,7 +117,9 @@ public MetadataRetriever build() { .setAdtsExtractorFlags(AdtsExtractor.FLAG_ENABLE_CONSTANT_BITRATE_SEEKING) .setAmrExtractorFlags(AmrExtractor.FLAG_ENABLE_CONSTANT_BITRATE_SEEKING) .setMp4ExtractorFlags( - Mp4Extractor.FLAG_READ_SEF_DATA | Mp4Extractor.FLAG_OMIT_TRACK_SAMPLE_TABLE); + Mp4Extractor.FLAG_READ_SEF_DATA + | Mp4Extractor.FLAG_OMIT_TRACK_SAMPLE_TABLE + | Mp4Extractor.FLAG_READ_XMP_METADATA); mediaSourceFactory = new DefaultMediaSourceFactory(context, extractorsFactory); } MetadataRetrieverInternal internalRetriever = diff --git a/libraries/inspector/src/main/java/androidx/media3/inspector/MetadataRetrieverInternal.java b/libraries/inspector/src/main/java/androidx/media3/inspector/MetadataRetrieverInternal.java index eb45ca1fdc0..916815c0e6d 100644 --- a/libraries/inspector/src/main/java/androidx/media3/inspector/MetadataRetrieverInternal.java +++ b/libraries/inspector/src/main/java/androidx/media3/inspector/MetadataRetrieverInternal.java @@ -31,10 +31,14 @@ import androidx.media3.common.Timeline; import androidx.media3.common.util.Clock; import androidx.media3.common.util.HandlerWrapper; +import androidx.media3.common.util.Util; import androidx.media3.exoplayer.LoadingInfo; import androidx.media3.exoplayer.analytics.PlayerId; +import androidx.media3.exoplayer.source.LoadEventInfo; +import androidx.media3.exoplayer.source.MediaLoadData; import androidx.media3.exoplayer.source.MediaPeriod; import androidx.media3.exoplayer.source.MediaSource; +import androidx.media3.exoplayer.source.MediaSourceEventListener; import androidx.media3.exoplayer.source.TrackGroupArray; import androidx.media3.exoplayer.upstream.Allocator; import androidx.media3.exoplayer.upstream.BandwidthMeter; @@ -322,6 +326,7 @@ private final class MediaSourceHandlerCallback implements Handler.Callback { private static final int ERROR_POLL_INTERVAL_MS = 100; private final MediaSourceCaller mediaSourceCaller; + private final MediaSourceEventListener mediaSourceEventListener; private @MonotonicNonNull MediaSource mediaSource; private @MonotonicNonNull MediaPeriod mediaPeriod; private @MonotonicNonNull Timeline timeline; @@ -329,6 +334,20 @@ private final class MediaSourceHandlerCallback implements Handler.Callback { private MediaSourceHandlerCallback() { mediaSourceCaller = new MediaSourceCaller(); + mediaSourceEventListener = + new MediaSourceEventListener() { + @Override + public void onLoadError( + int windowIndex, + @Nullable MediaSource.MediaPeriodId mediaPeriodId, + LoadEventInfo loadEventInfo, + MediaLoadData mediaLoadData, + IOException error, + boolean wasCanceled) { + // Trigger an immediate check to see if this error fails source/period preparation. + mediaSourceHandler.sendEmptyMessage(MESSAGE_CHECK_FOR_FAILURE); + } + }; } @Override @@ -341,6 +360,8 @@ public boolean handleMessage(Message msg) { case MESSAGE_PREPARE_SOURCE: MediaItem mediaItem = (MediaItem) msg.obj; mediaSource = mediaSourceFactory.createMediaSource(mediaItem); + mediaSource.addEventListener( + Util.createHandlerForCurrentLooper(), mediaSourceEventListener); mediaSource.prepareSource(mediaSourceCaller, PlayerId.UNSET, BandwidthMeter.NO_OP); mediaSourceHandler.sendEmptyMessage(MESSAGE_CHECK_FOR_FAILURE); return true; @@ -351,6 +372,7 @@ public boolean handleMessage(Message msg) { } else { mediaPeriod.maybeThrowPrepareError(); } + mediaSourceHandler.removeMessages(MESSAGE_CHECK_FOR_FAILURE); mediaSourceHandler.sendEmptyMessageDelayed( MESSAGE_CHECK_FOR_FAILURE, /* delayMs= */ ERROR_POLL_INTERVAL_MS); } catch (IOException e) { @@ -368,6 +390,7 @@ public boolean handleMessage(Message msg) { checkNotNull(mediaSource).releasePeriod(mediaPeriod); } if (mediaSource != null) { + mediaSource.removeEventListener(mediaSourceEventListener); mediaSource.releaseSource(mediaSourceCaller); } } finally { diff --git a/libraries/inspector/src/test/java/androidx/media3/inspector/MetadataRetrieverTest.java b/libraries/inspector/src/test/java/androidx/media3/inspector/MetadataRetrieverTest.java index 28bb7556878..2355bf40214 100644 --- a/libraries/inspector/src/test/java/androidx/media3/inspector/MetadataRetrieverTest.java +++ b/libraries/inspector/src/test/java/androidx/media3/inspector/MetadataRetrieverTest.java @@ -39,6 +39,7 @@ import androidx.media3.exoplayer.source.DefaultMediaSourceFactory; import androidx.media3.exoplayer.source.MediaSource; import androidx.media3.exoplayer.source.TrackGroupArray; +import androidx.media3.exoplayer.source.UnrecognizedInputFormatException; import androidx.media3.extractor.DefaultExtractorsFactory; import androidx.media3.extractor.metadata.MotionPhotoMetadata; import androidx.media3.extractor.metadata.mp4.SlowMotionData; @@ -182,7 +183,7 @@ public void retrieveMetadata_heicMotionPhoto_outputsExpectedResult() throws Exce new MotionPhotoMetadata( /* photoStartPosition= */ 0, /* photoSize= */ 28_853, - /* photoPresentationTimestampUs= */ C.TIME_UNSET, + /* photoPresentationTimestampUs= */ 0, /* videoStartPosition= */ 28_869, /* videoSize= */ 28_803); @@ -295,7 +296,7 @@ public void retrieveMetadata_sefSlowMotionAvc_outputsExpectedResult() throws Exc assertThat(timeline.getWindowCount()).isEqualTo(1); // Duration - assertThat(durationUs).isEqualTo(2_152_000); + assertThat(durationUs).isEqualTo(2_151_750); } } @@ -351,7 +352,7 @@ public void retrieveMetadata_sefSlowMotionHevc_outputsExpectedResult() throws Ex assertThat(timeline.getWindowCount()).isEqualTo(1); // Duration - assertThat(durationUs).isEqualTo(11_793_400); + assertThat(durationUs).isEqualTo(11_793_411); } } @@ -586,4 +587,21 @@ public void retrieveMetadata_mediaSourceThrowsRuntimeException_failsFuture() { assertThat(thrown).hasCauseThat().isSameInstanceAs(runtimeException); } } + + @Test + public void retrieveMetadata_malformedMediaItemWithDefaultClock_failsFast() throws Exception { + MediaItem mediaItem = MediaItem.fromUri("data:,fake"); + + // Deliberately omit setClock(clock) to verify Clock.DEFAULT fails fast when SystemClock does + // not advance (as in Robolectric), so the 100ms delayed poll never fires. + try (MetadataRetriever retriever = new MetadataRetriever.Builder(context, mediaItem).build()) { + ListenableFuture trackGroupsFuture = retriever.retrieveTrackGroups(); + + ExecutionException thrown = + assertThrows( + ExecutionException.class, + () -> trackGroupsFuture.get(TEST_TIMEOUT_SEC, TimeUnit.SECONDS)); + assertThat(thrown).hasCauseThat().isInstanceOf(UnrecognizedInputFormatException.class); + } + } } diff --git a/libraries/inspector_frame/src/androidTest/java/androidx/media3/inspector/frame/FrameExtractorTest.java b/libraries/inspector_frame/src/androidTest/java/androidx/media3/inspector/frame/FrameExtractorTest.java index a8f1ea329f5..a186f11eb12 100644 --- a/libraries/inspector_frame/src/androidTest/java/androidx/media3/inspector/frame/FrameExtractorTest.java +++ b/libraries/inspector_frame/src/androidTest/java/androidx/media3/inspector/frame/FrameExtractorTest.java @@ -99,95 +99,110 @@ public void setUpTestId() { @Test public void extractFrame_oneFrame_returnsNearest() throws Exception { + Frame frame; + int initialRenderedOutputBufferCount; + int finalRenderedOutputBufferCount; try (FrameExtractor frameExtractor = new FrameExtractor.Builder(context, MediaItem.fromUri(FILE_PATH)).build()) { DecoderCounters initialCounters = frameExtractor.getDecoderCounters().get(TIMEOUT_SECONDS, SECONDS); - int initialRenderedOutputBufferCount = + initialRenderedOutputBufferCount = initialCounters != null ? initialCounters.renderedOutputBufferCount : 0; ListenableFuture frameFuture = frameExtractor.getFrame(/* positionMs= */ 8_500); - Frame frame = frameFuture.get(TIMEOUT_SECONDS, SECONDS); - Bitmap actualBitmap = frame.bitmap; - Bitmap expectedBitmap = - readBitmap( - /* assetString= */ GOLDEN_ASSET_FOLDER_PATH - + "sample_with_increasing_timestamps_360p_8.531.png"); - maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual", actualBitmap, /* path= */ null); - - assertThat(frame.presentationTimeMs).isEqualTo(8_531); - assertBitmapsAreSimilar(expectedBitmap, actualBitmap, PSNR_THRESHOLD); + frame = frameFuture.get(TIMEOUT_SECONDS, SECONDS); DecoderCounters finalCounters = frameExtractor.getDecoderCounters().get(TIMEOUT_SECONDS, SECONDS); - assertThat(finalCounters.renderedOutputBufferCount - initialRenderedOutputBufferCount) - .isAtMost(2); + finalRenderedOutputBufferCount = + finalCounters != null ? finalCounters.renderedOutputBufferCount : 0; } + Bitmap actualBitmap = frame.bitmap; + Bitmap expectedBitmap = + readBitmap( + /* assetString= */ GOLDEN_ASSET_FOLDER_PATH + + "sample_with_increasing_timestamps_360p_8.531.png"); + maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual", actualBitmap, /* path= */ null); + + assertThat(frame.presentationTimeMs).isEqualTo(8_531); + assertBitmapsAreSimilar(expectedBitmap, actualBitmap, PSNR_THRESHOLD); + assertThat(finalRenderedOutputBufferCount - initialRenderedOutputBufferCount).isAtMost(2); } @Test public void extractFrame_oneFrameWithPresentationEffect_returnsScaledFrame() throws Exception { + Frame frame; + int initialRenderedOutputBufferCount; + int finalRenderedOutputBufferCount; try (FrameExtractor frameExtractor = new FrameExtractor.Builder(context, MediaItem.fromUri(FILE_PATH)) .setEffects(ImmutableList.of(Presentation.createForHeight(180))) .build()) { DecoderCounters initialCounters = frameExtractor.getDecoderCounters().get(TIMEOUT_SECONDS, SECONDS); - int initialRenderedOutputBufferCount = + initialRenderedOutputBufferCount = initialCounters != null ? initialCounters.renderedOutputBufferCount : 0; ListenableFuture frameFuture = frameExtractor.getFrame(/* positionMs= */ 8_500); - Frame frame = frameFuture.get(TIMEOUT_SECONDS, SECONDS); - Bitmap actualBitmap = frame.bitmap; - Bitmap expectedBitmap = - readBitmap( - /* assetString= */ GOLDEN_ASSET_FOLDER_PATH - + "sample_with_increasing_timestamps_360p_8.531_scaled_to_180p.png"); - maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual", actualBitmap, /* path= */ null); - - assertThat(frame.presentationTimeMs).isEqualTo(8_531); - assertBitmapsAreSimilar(expectedBitmap, actualBitmap, PSNR_THRESHOLD); + frame = frameFuture.get(TIMEOUT_SECONDS, SECONDS); DecoderCounters finalCounters = frameExtractor.getDecoderCounters().get(TIMEOUT_SECONDS, SECONDS); - assertThat(finalCounters.renderedOutputBufferCount - initialRenderedOutputBufferCount) - .isAtMost(2); + finalRenderedOutputBufferCount = + finalCounters != null ? finalCounters.renderedOutputBufferCount : 0; } + Bitmap actualBitmap = frame.bitmap; + Bitmap expectedBitmap = + readBitmap( + /* assetString= */ GOLDEN_ASSET_FOLDER_PATH + + "sample_with_increasing_timestamps_360p_8.531_scaled_to_180p.png"); + maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual", actualBitmap, /* path= */ null); + + assertThat(frame.presentationTimeMs).isEqualTo(8_531); + assertBitmapsAreSimilar(expectedBitmap, actualBitmap, PSNR_THRESHOLD); + assertThat(finalRenderedOutputBufferCount - initialRenderedOutputBufferCount).isAtMost(2); } @Test public void extractFrame_pastDuration_returnsLastFrame() throws Exception { + Frame frame; + int initialRenderedOutputBufferCount; + int finalRenderedOutputBufferCount; try (FrameExtractor frameExtractor = new FrameExtractor.Builder(context, MediaItem.fromUri(FILE_PATH)).build()) { DecoderCounters initialCounters = frameExtractor.getDecoderCounters().get(TIMEOUT_SECONDS, SECONDS); - int initialRenderedOutputBufferCount = + initialRenderedOutputBufferCount = initialCounters != null ? initialCounters.renderedOutputBufferCount : 0; ListenableFuture frameFuture = frameExtractor.getFrame(/* positionMs= */ 200_000); - Frame frame = frameFuture.get(TIMEOUT_SECONDS, SECONDS); - Bitmap actualBitmap = frame.bitmap; - int lastVideoFramePresentationTimeMs = 17_029; - Bitmap expectedBitmap = - readBitmap( - /* assetString= */ GOLDEN_ASSET_FOLDER_PATH - + "sample_with_increasing_timestamps_360p_17.029.png"); - maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual", actualBitmap, /* path= */ null); - - assertThat(frame.presentationTimeMs).isEqualTo(lastVideoFramePresentationTimeMs); - assertBitmapsAreSimilar(expectedBitmap, actualBitmap, PSNR_THRESHOLD); + frame = frameFuture.get(TIMEOUT_SECONDS, SECONDS); DecoderCounters finalCounters = frameExtractor.getDecoderCounters().get(TIMEOUT_SECONDS, SECONDS); - assertThat(finalCounters.renderedOutputBufferCount - initialRenderedOutputBufferCount) - .isAtMost(2); + finalRenderedOutputBufferCount = + finalCounters != null ? finalCounters.renderedOutputBufferCount : 0; } + Bitmap actualBitmap = frame.bitmap; + int lastVideoFramePresentationTimeMs = 17_029; + Bitmap expectedBitmap = + readBitmap( + /* assetString= */ GOLDEN_ASSET_FOLDER_PATH + + "sample_with_increasing_timestamps_360p_17.029.png"); + maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual", actualBitmap, /* path= */ null); + + assertThat(frame.presentationTimeMs).isEqualTo(lastVideoFramePresentationTimeMs); + assertBitmapsAreSimilar(expectedBitmap, actualBitmap, PSNR_THRESHOLD); + assertThat(finalRenderedOutputBufferCount - initialRenderedOutputBufferCount).isAtMost(2); } @Test public void extractFrame_repeatedPositionMs_returnsTheSameFrame() throws Exception { + ImmutableList expectedFramePositionsMs = ImmutableList.of(0L, 0L, 33L, 66L, 66L); + List extractedFrames = new ArrayList<>(); + int initialRenderedOutputBufferCount; + int finalRenderedOutputBufferCount; try (FrameExtractor frameExtractor = new FrameExtractor.Builder(context, MediaItem.fromUri(FILE_PATH)).build()) { DecoderCounters initialCounters = frameExtractor.getDecoderCounters().get(TIMEOUT_SECONDS, SECONDS); - int initialRenderedOutputBufferCount = + initialRenderedOutputBufferCount = initialCounters != null ? initialCounters.renderedOutputBufferCount : 0; ImmutableList requestedFramePositionsMs = ImmutableList.of(0L, 0L, 33L, 34L, 34L); - ImmutableList expectedFramePositionsMs = ImmutableList.of(0L, 0L, 33L, 66L, 66L); List> frameFutures = new ArrayList<>(); for (long positionMs : requestedFramePositionsMs) { @@ -196,37 +211,44 @@ public void extractFrame_repeatedPositionMs_returnsTheSameFrame() throws Excepti for (int i = 0; i < expectedFramePositionsMs.size(); i++) { ListenableFuture frameListenableFuture = frameFutures.get(i); Frame frame = frameListenableFuture.get(TIMEOUT_SECONDS, SECONDS); - maybeSaveTestBitmap( - testId, /* bitmapLabel= */ "actual_" + i, frame.bitmap, /* path= */ null); - Bitmap expectedBitmap = - readBitmap( - /* assetString= */ GOLDEN_ASSET_FOLDER_PATH - + "sample_with_increasing_timestamps_360p_" - + String.format(Locale.US, "%.3f", frame.presentationTimeMs / 1000f) - + ".png"); - - assertBitmapsAreSimilar(expectedBitmap, frame.bitmap, PSNR_THRESHOLD); - assertThat(frame.presentationTimeMs).isEqualTo(expectedFramePositionsMs.get(i)); + extractedFrames.add(frame); } DecoderCounters finalCounters = frameExtractor.getDecoderCounters().get(TIMEOUT_SECONDS, SECONDS); - assertThat(finalCounters.renderedOutputBufferCount - initialRenderedOutputBufferCount) - .isAtMost(3); + finalRenderedOutputBufferCount = + finalCounters != null ? finalCounters.renderedOutputBufferCount : 0; + } + for (int i = 0; i < expectedFramePositionsMs.size(); i++) { + Frame frame = extractedFrames.get(i); + maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual_" + i, frame.bitmap, /* path= */ null); + Bitmap expectedBitmap = + readBitmap( + /* assetString= */ GOLDEN_ASSET_FOLDER_PATH + + "sample_with_increasing_timestamps_360p_" + + String.format(Locale.US, "%.3f", frame.presentationTimeMs / 1000f) + + ".png"); + + assertBitmapsAreSimilar(expectedBitmap, frame.bitmap, PSNR_THRESHOLD); + assertThat(frame.presentationTimeMs).isEqualTo(expectedFramePositionsMs.get(i)); } + assertThat(finalRenderedOutputBufferCount - initialRenderedOutputBufferCount).isAtMost(3); } @Test public void extractFrame_repeatedPositionMsAndClosestSync_returnsTheSameFrame() throws Exception { + ImmutableList expectedFramePositionsMs = ImmutableList.of(0L, 0L, 0L, 0L, 0L); + List extractedFrames = new ArrayList<>(); + int initialRenderedOutputBufferCount; + int finalRenderedOutputBufferCount; try (FrameExtractor frameExtractor = new FrameExtractor.Builder(context, MediaItem.fromUri(FILE_PATH)) .setSeekParameters(CLOSEST_SYNC) .build()) { DecoderCounters initialCounters = frameExtractor.getDecoderCounters().get(TIMEOUT_SECONDS, SECONDS); - int initialRenderedOutputBufferCount = + initialRenderedOutputBufferCount = initialCounters != null ? initialCounters.renderedOutputBufferCount : 0; ImmutableList requestedFramePositionsMs = ImmutableList.of(0L, 0L, 33L, 34L, 34L); - ImmutableList expectedFramePositionsMs = ImmutableList.of(0L, 0L, 0L, 0L, 0L); List> frameFutures = new ArrayList<>(); for (long positionMs : requestedFramePositionsMs) { @@ -235,23 +257,27 @@ public void extractFrame_repeatedPositionMsAndClosestSync_returnsTheSameFrame() for (int i = 0; i < expectedFramePositionsMs.size(); i++) { ListenableFuture frameListenableFuture = frameFutures.get(i); Frame frame = frameListenableFuture.get(TIMEOUT_SECONDS, SECONDS); - maybeSaveTestBitmap( - testId, /* bitmapLabel= */ "actual_" + i, frame.bitmap, /* path= */ null); - Bitmap expectedBitmap = - readBitmap( - /* assetString= */ GOLDEN_ASSET_FOLDER_PATH - + "sample_with_increasing_timestamps_360p_" - + String.format(Locale.US, "%.3f", frame.presentationTimeMs / 1000f) - + ".png"); - - assertBitmapsAreSimilar(expectedBitmap, frame.bitmap, PSNR_THRESHOLD); - assertThat(frame.presentationTimeMs).isEqualTo(expectedFramePositionsMs.get(i)); + extractedFrames.add(frame); } DecoderCounters finalCounters = frameExtractor.getDecoderCounters().get(TIMEOUT_SECONDS, SECONDS); - assertThat(finalCounters.renderedOutputBufferCount - initialRenderedOutputBufferCount) - .isAtMost(1); + finalRenderedOutputBufferCount = + finalCounters != null ? finalCounters.renderedOutputBufferCount : 0; } + for (int i = 0; i < expectedFramePositionsMs.size(); i++) { + Frame frame = extractedFrames.get(i); + maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual_" + i, frame.bitmap, /* path= */ null); + Bitmap expectedBitmap = + readBitmap( + /* assetString= */ GOLDEN_ASSET_FOLDER_PATH + + "sample_with_increasing_timestamps_360p_" + + String.format(Locale.US, "%.3f", frame.presentationTimeMs / 1000f) + + ".png"); + + assertBitmapsAreSimilar(expectedBitmap, frame.bitmap, PSNR_THRESHOLD); + assertThat(frame.presentationTimeMs).isEqualTo(expectedFramePositionsMs.get(i)); + } + assertThat(finalRenderedOutputBufferCount - initialRenderedOutputBufferCount).isAtMost(1); } @Test @@ -380,29 +406,33 @@ public void frameExtractor_releaseOnPlayerLooper_returns() { @Test public void extractFrame_oneFrameRotated_returnsFrameInCorrectOrientation() throws Exception { + Frame frame; + int initialRenderedOutputBufferCount; + int finalRenderedOutputBufferCount; try (FrameExtractor frameExtractor = new FrameExtractor.Builder(context, MediaItem.fromUri(MP4_TRIM_OPTIMIZATION_270.uri)) .build()) { DecoderCounters initialCounters = frameExtractor.getDecoderCounters().get(TIMEOUT_SECONDS, SECONDS); - int initialRenderedOutputBufferCount = + initialRenderedOutputBufferCount = initialCounters != null ? initialCounters.renderedOutputBufferCount : 0; ListenableFuture frameFuture = frameExtractor.getFrame(/* positionMs= */ 0); - Frame frame = frameFuture.get(TIMEOUT_SECONDS, SECONDS); - Bitmap actualBitmap = frame.bitmap; - Bitmap expectedBitmap = - readBitmap( - /* assetString= */ GOLDEN_ASSET_FOLDER_PATH - + "internal_emulator_transformer_output_180_rotated_0.000.png"); - maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual", actualBitmap, /* path= */ null); - - assertThat(frame.presentationTimeMs).isEqualTo(0); - assertBitmapsAreSimilar(expectedBitmap, actualBitmap, PSNR_THRESHOLD); + frame = frameFuture.get(TIMEOUT_SECONDS, SECONDS); DecoderCounters finalCounters = frameExtractor.getDecoderCounters().get(TIMEOUT_SECONDS, SECONDS); - assertThat(finalCounters.renderedOutputBufferCount - initialRenderedOutputBufferCount) - .isAtMost(1); + finalRenderedOutputBufferCount = + finalCounters != null ? finalCounters.renderedOutputBufferCount : 0; } + Bitmap actualBitmap = frame.bitmap; + Bitmap expectedBitmap = + readBitmap( + /* assetString= */ GOLDEN_ASSET_FOLDER_PATH + + "internal_emulator_transformer_output_180_rotated_0.000.png"); + maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual", actualBitmap, /* path= */ null); + + assertThat(frame.presentationTimeMs).isEqualTo(0); + assertBitmapsAreSimilar(expectedBitmap, actualBitmap, PSNR_THRESHOLD); + assertThat(finalRenderedOutputBufferCount - initialRenderedOutputBufferCount).isAtMost(1); } @Test @@ -616,82 +646,94 @@ public void extractThumbnail_whenThumbnailMetadataTimestampIsZero_returnsCorrect testId, /* inputFormat= */ MP4_ADVANCED_ASSET.videoFormat, /* outputFormat= */ null); + Frame frame; + int initialRenderedOutputBufferCount; + int finalRenderedOutputBufferCount; try (FrameExtractor frameExtractor = new FrameExtractor.Builder(context, MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)).build()) { DecoderCounters initialCounters = frameExtractor.getDecoderCounters().get(TIMEOUT_SECONDS, SECONDS); - int initialRenderedOutputBufferCount = + initialRenderedOutputBufferCount = initialCounters != null ? initialCounters.renderedOutputBufferCount : 0; ListenableFuture frameFuture = frameExtractor.getThumbnail(); - Frame frame = frameFuture.get(TIMEOUT_SECONDS, SECONDS); - Bitmap actualBitmap = frame.bitmap; - Bitmap expectedBitmap = - readBitmap(/* assetString= */ GOLDEN_ASSET_FOLDER_PATH + "sample_0.000.png"); - maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual", actualBitmap, /* path= */ null); - - assertThat(frame.presentationTimeMs).isEqualTo(0); - assertBitmapsAreSimilar(expectedBitmap, actualBitmap, PSNR_THRESHOLD); + frame = frameFuture.get(TIMEOUT_SECONDS, SECONDS); DecoderCounters finalCounters = frameExtractor.getDecoderCounters().get(TIMEOUT_SECONDS, SECONDS); - assertThat(finalCounters.renderedOutputBufferCount - initialRenderedOutputBufferCount) - .isAtMost(1); + finalRenderedOutputBufferCount = + finalCounters != null ? finalCounters.renderedOutputBufferCount : 0; } + Bitmap actualBitmap = frame.bitmap; + Bitmap expectedBitmap = + readBitmap(/* assetString= */ GOLDEN_ASSET_FOLDER_PATH + "sample_0.000.png"); + maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual", actualBitmap, /* path= */ null); + + assertThat(frame.presentationTimeMs).isEqualTo(0); + assertBitmapsAreSimilar(expectedBitmap, actualBitmap, PSNR_THRESHOLD); + assertThat(finalRenderedOutputBufferCount - initialRenderedOutputBufferCount).isAtMost(1); } @Test public void extractThumbnail_whenThumbnailMetadataTimestampIsNonZero_returnsCorrectFrame() throws Exception { + Frame frame; + int initialRenderedOutputBufferCount; + int finalRenderedOutputBufferCount; try (FrameExtractor frameExtractor = new FrameExtractor.Builder(context, MediaItem.fromUri(FILE_PATH)).build()) { DecoderCounters initialCounters = frameExtractor.getDecoderCounters().get(TIMEOUT_SECONDS, SECONDS); - int initialRenderedOutputBufferCount = + initialRenderedOutputBufferCount = initialCounters != null ? initialCounters.renderedOutputBufferCount : 0; ListenableFuture frameFuture = frameExtractor.getThumbnail(); - Frame frame = frameFuture.get(TIMEOUT_SECONDS, SECONDS); - Bitmap actualBitmap = frame.bitmap; - Bitmap expectedBitmap = - readBitmap( - /* assetString= */ GOLDEN_ASSET_FOLDER_PATH - + "sample_with_increasing_timestamps_360p_8.331.png"); - maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual", actualBitmap, /* path= */ null); - - assertThat(frame.presentationTimeMs).isEqualTo(8_331); - assertBitmapsAreSimilar(expectedBitmap, actualBitmap, PSNR_THRESHOLD); + frame = frameFuture.get(TIMEOUT_SECONDS, SECONDS); DecoderCounters finalCounters = frameExtractor.getDecoderCounters().get(TIMEOUT_SECONDS, SECONDS); - assertThat(finalCounters.renderedOutputBufferCount - initialRenderedOutputBufferCount) - .isAtMost(2); + finalRenderedOutputBufferCount = + finalCounters != null ? finalCounters.renderedOutputBufferCount : 0; } + Bitmap actualBitmap = frame.bitmap; + Bitmap expectedBitmap = + readBitmap( + /* assetString= */ GOLDEN_ASSET_FOLDER_PATH + + "sample_with_increasing_timestamps_360p_8.331.png"); + maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual", actualBitmap, /* path= */ null); + + assertThat(frame.presentationTimeMs).isEqualTo(8_331); + assertBitmapsAreSimilar(expectedBitmap, actualBitmap, PSNR_THRESHOLD); + assertThat(finalRenderedOutputBufferCount - initialRenderedOutputBufferCount).isAtMost(2); } @Test public void extractThumbnail_whenThumbnailMetadataIsMissing_returnsCorrectFrame() throws Exception { + Frame frame; + int initialRenderedOutputBufferCount; + int finalRenderedOutputBufferCount; try (FrameExtractor frameExtractor = new FrameExtractor.Builder( context, MediaItem.fromUri(MP4_ONLY_PREROLL_SYNC_SAMPLE_EDIT_LIST.uri)) .build()) { DecoderCounters initialCounters = frameExtractor.getDecoderCounters().get(TIMEOUT_SECONDS, SECONDS); - int initialRenderedOutputBufferCount = + initialRenderedOutputBufferCount = initialCounters != null ? initialCounters.renderedOutputBufferCount : 0; ListenableFuture frameFuture = frameExtractor.getThumbnail(); - Frame frame = frameFuture.get(TIMEOUT_SECONDS, SECONDS); - Bitmap actualBitmap = frame.bitmap; - Bitmap expectedBitmap = - readBitmap( - /* assetString= */ GOLDEN_ASSET_FOLDER_PATH - + "sample_edit_list_only_preroll_sync_sample_0.000.png"); - maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual", actualBitmap, /* path= */ null); - - assertThat(frame.presentationTimeMs).isEqualTo(0); - assertBitmapsAreSimilar(expectedBitmap, actualBitmap, PSNR_THRESHOLD); + frame = frameFuture.get(TIMEOUT_SECONDS, SECONDS); DecoderCounters finalCounters = frameExtractor.getDecoderCounters().get(TIMEOUT_SECONDS, SECONDS); - assertThat(finalCounters.renderedOutputBufferCount - initialRenderedOutputBufferCount) - .isAtMost(1); + finalRenderedOutputBufferCount = + finalCounters != null ? finalCounters.renderedOutputBufferCount : 0; } + Bitmap actualBitmap = frame.bitmap; + Bitmap expectedBitmap = + readBitmap( + /* assetString= */ GOLDEN_ASSET_FOLDER_PATH + + "sample_edit_list_only_preroll_sync_sample_0.000.png"); + maybeSaveTestBitmap(testId, /* bitmapLabel= */ "actual", actualBitmap, /* path= */ null); + + assertThat(frame.presentationTimeMs).isEqualTo(0); + assertBitmapsAreSimilar(expectedBitmap, actualBitmap, PSNR_THRESHOLD); + assertThat(finalRenderedOutputBufferCount - initialRenderedOutputBufferCount).isAtMost(1); } @Test diff --git a/libraries/inspector_frame/src/main/java/androidx/media3/inspector/frame/FrameExtractorInternal.java b/libraries/inspector_frame/src/main/java/androidx/media3/inspector/frame/FrameExtractorInternal.java index 486fed09618..443b730fbf5 100644 --- a/libraries/inspector_frame/src/main/java/androidx/media3/inspector/frame/FrameExtractorInternal.java +++ b/libraries/inspector_frame/src/main/java/androidx/media3/inspector/frame/FrameExtractorInternal.java @@ -36,6 +36,7 @@ import android.graphics.ColorSpace; import android.graphics.Gainmap; import android.graphics.Matrix; +import android.media.MediaFormat; import android.opengl.GLES20; import android.opengl.GLES30; import android.os.Handler; @@ -1173,6 +1174,26 @@ protected void renderOutputBufferV21( super.renderOutputBufferV21(codec, index, presentationTimeUs, releaseTimeNs); } + @Override + protected MediaFormat getMediaFormat( + Format format, + String codecMimeType, + CodecMaxValues codecMaxValues, + float codecOperatingRate, + boolean deviceNeedsNoPostProcessWorkaround, + int tunnelingAudioSessionId) { + MediaFormat mediaFormat = + super.getMediaFormat( + format, + codecMimeType, + codecMaxValues, + codecOperatingRate, + deviceNeedsNoPostProcessWorkaround, + tunnelingAudioSessionId); + mediaFormat.setInteger(MediaFormat.KEY_PRIORITY, C.MEDIA_CODEC_PRIORITY_NON_REALTIME); + return mediaFormat; + } + @Override protected void onPositionReset( long positionUs, boolean joining, boolean sampleStreamIsResetToKeyFrame) diff --git a/libraries/muxer/src/androidTest/java/androidx/media3/muxer/MediaMuxerContractTest.java b/libraries/muxer/src/androidTest/java/androidx/media3/muxer/MediaMuxerContractTest.java index 94466c970cb..3c4e1eaf345 100644 --- a/libraries/muxer/src/androidTest/java/androidx/media3/muxer/MediaMuxerContractTest.java +++ b/libraries/muxer/src/androidTest/java/androidx/media3/muxer/MediaMuxerContractTest.java @@ -21,6 +21,7 @@ import static androidx.media3.common.MimeTypes.AUDIO_AAC; import static androidx.media3.common.util.MediaFormatUtil.createMediaFormatFromFormat; import static androidx.media3.muxer.MediaMuxerCompat.OUTPUT_FORMAT_MP4; +import static androidx.media3.muxer.MediaMuxerCompat.OUTPUT_FORMAT_WEBM; import static androidx.media3.test.utils.AssetInfo.AMR_NB_3GP_ASSET; import static androidx.media3.test.utils.AssetInfo.AMR_WB_3GP_ASSET; import static androidx.media3.test.utils.AssetInfo.H263_3GP_ASSET; @@ -31,6 +32,7 @@ import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_WITH_INCREASING_TIMESTAMPS; import static androidx.media3.test.utils.AssetInfo.MPEG4_MP4_ASSET; import static com.google.common.base.Preconditions.checkNotNull; +import static com.google.common.collect.Iterables.getOnlyElement; import static com.google.common.truth.Truth.assertThat; import static com.google.common.truth.Truth.assertWithMessage; import static org.junit.Assert.assertThrows; @@ -45,8 +47,10 @@ import androidx.annotation.RequiresApi; import androidx.media3.common.C; import androidx.media3.common.Format; +import androidx.media3.common.MimeTypes; import androidx.media3.common.util.MediaFormatUtil; import androidx.media3.container.Mp4LocationData; +import androidx.media3.extractor.mkv.MatroskaExtractor; import androidx.media3.extractor.mp4.Mp4Extractor; import androidx.media3.extractor.text.DefaultSubtitleParserFactory; import androidx.media3.inspector.MediaExtractorCompat; @@ -57,7 +61,6 @@ import androidx.test.core.app.ApplicationProvider; import androidx.test.filters.SdkSuppress; import com.google.common.collect.ImmutableList; -import com.google.common.collect.Iterables; import com.google.common.io.BaseEncoding; import com.google.errorprone.annotations.Immutable; import com.google.testing.junit.testparameterinjector.TestParameter; @@ -172,9 +175,9 @@ public void createMp4File_withFileDescriptorOutput_writesCorrectTracksAndSamples new Mp4Extractor(new DefaultSubtitleParserFactory()), checkNotNull(outputFilePath)); assertThat(fakeExtractorOutput.numberOfTracks) .isEqualTo(MP4_ASSET_WITH_INCREASING_TIMESTAMPS.trackCount); - Iterables.getOnlyElement(fakeExtractorOutput.getTrackOutputsForType(C.TRACK_TYPE_VIDEO)) + getOnlyElement(fakeExtractorOutput.getTrackOutputsForType(C.TRACK_TYPE_VIDEO)) .assertSampleCount(MP4_ASSET_WITH_INCREASING_TIMESTAMPS.videoFrameCount); - Iterables.getOnlyElement(fakeExtractorOutput.getTrackOutputsForType(C.TRACK_TYPE_AUDIO)) + getOnlyElement(fakeExtractorOutput.getTrackOutputsForType(C.TRACK_TYPE_AUDIO)) .assertSampleCount(MP4_ASSET_WITH_INCREASING_TIMESTAMPS.audioSampleCount); } @@ -502,8 +505,7 @@ public void createMp4File_withLocationData_writesCorrectLocation() throws Except TestUtil.extractAllSamplesFromFilePath( new Mp4Extractor(new DefaultSubtitleParserFactory()), checkNotNull(outputFilePath)); Format videoTrackFormat = - Iterables.getOnlyElement(fakeExtractorOutput.getTrackOutputsForType(C.TRACK_TYPE_VIDEO)) - .lastFormat; + getOnlyElement(fakeExtractorOutput.getTrackOutputsForType(C.TRACK_TYPE_VIDEO)).lastFormat; Mp4LocationData actualLocationData = videoTrackFormat.metadata.getFirstEntryOfType(Mp4LocationData.class); assertThat(actualLocationData.latitude).isEqualTo(latitude); @@ -532,8 +534,7 @@ public void createMp4File_withNegativeLocationData_writesCorrectLocation() throw TestUtil.extractAllSamplesFromFilePath( new Mp4Extractor(new DefaultSubtitleParserFactory()), checkNotNull(outputFilePath)); Format videoTrackFormat = - Iterables.getOnlyElement(fakeExtractorOutput.getTrackOutputsForType(C.TRACK_TYPE_VIDEO)) - .lastFormat; + getOnlyElement(fakeExtractorOutput.getTrackOutputsForType(C.TRACK_TYPE_VIDEO)).lastFormat; Mp4LocationData actualLocationData = videoTrackFormat.metadata.getFirstEntryOfType(Mp4LocationData.class); assertThat(actualLocationData.latitude).isEqualTo(latitude); @@ -577,11 +578,31 @@ public void createMp4File_withOrientationData_writesCorrectOrientation() throws TestUtil.extractAllSamplesFromFilePath( new Mp4Extractor(new DefaultSubtitleParserFactory()), checkNotNull(outputFilePath)); Format videoTrackFormat = - Iterables.getOnlyElement(fakeExtractorOutput.getTrackOutputsForType(C.TRACK_TYPE_VIDEO)) - .lastFormat; + getOnlyElement(fakeExtractorOutput.getTrackOutputsForType(C.TRACK_TYPE_VIDEO)).lastFormat; assertThat(videoTrackFormat.rotationDegrees).isEqualTo(orientation); } + @Test + public void createWebmFile_withValidWebmAsset_writesCorrectTracksAndSamples() throws Exception { + String outputFilePath = tempFolder.newFile().getAbsolutePath(); + + MediaMuxerProxy mediaMuxerProxy = + muxerFactoryEnum.value.create(outputFilePath, OUTPUT_FORMAT_WEBM); + try { + feedDataToMuxer(context, mediaMuxerProxy, "asset:///media/mkv/bbb_960x540_60fps_vp8.webm"); + } finally { + mediaMuxerProxy.release(); + } + + FakeExtractorOutput fakeExtractorOutput = + TestUtil.extractAllSamplesFromFilePath( + new MatroskaExtractor(new DefaultSubtitleParserFactory()), + checkNotNull(outputFilePath)); + Format videoTrackFormat = + getOnlyElement(fakeExtractorOutput.getTrackOutputsForType(C.TRACK_TYPE_VIDEO)).lastFormat; + assertThat(videoTrackFormat.sampleMimeType).isEqualTo(MimeTypes.VIDEO_VP9); + } + private static void feedDataToMuxer(Context context, MediaMuxerProxy muxer, String inputFilePath) throws IOException { MediaExtractorCompat extractor = new MediaExtractorCompat(context); diff --git a/libraries/muxer/src/main/java/androidx/media3/muxer/AnnexBUtils.java b/libraries/muxer/src/main/java/androidx/media3/muxer/AnnexBUtils.java index f43b73477ab..ef946a746a8 100644 --- a/libraries/muxer/src/main/java/androidx/media3/muxer/AnnexBUtils.java +++ b/libraries/muxer/src/main/java/androidx/media3/muxer/AnnexBUtils.java @@ -16,9 +16,12 @@ package androidx.media3.muxer; import static androidx.media3.muxer.Boxes.getDolbyVisionProfileAndLevel; +import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.base.Preconditions.checkState; +import android.util.Pair; +import androidx.annotation.Nullable; import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; import com.google.common.collect.ImmutableList; @@ -110,7 +113,9 @@ public static boolean doesSampleContainAnnexBNalUnits(Format format) { checkNotNull(sampleMimeType); if (sampleMimeType.equals(MimeTypes.VIDEO_DOLBY_VISION)) { // Dolby vision with AV1 profile does not contain Nal units. - int profile = checkNotNull(getDolbyVisionProfileAndLevel(format)).first; + @Nullable Pair profileAndLevel = getDolbyVisionProfileAndLevel(format); + checkArgument(profileAndLevel != null, "Invalid Dolby Vision format: %s", format.codecs); + int profile = profileAndLevel.first; // Dolby vision with Profile 10 is equivalent to DolbyVisionProfileDvav110 of framework // media codec constants. return profile != 10; diff --git a/libraries/muxer/src/main/java/androidx/media3/muxer/Boxes.java b/libraries/muxer/src/main/java/androidx/media3/muxer/Boxes.java index c60ad3a19f0..573e0e0c991 100644 --- a/libraries/muxer/src/main/java/androidx/media3/muxer/Boxes.java +++ b/libraries/muxer/src/main/java/androidx/media3/muxer/Boxes.java @@ -16,6 +16,10 @@ package androidx.media3.muxer; import static androidx.media3.common.MimeTypes.allSamplesAreSyncSamples; +import static androidx.media3.common.util.Util.getPcmFrameSize; +import static androidx.media3.common.util.Util.isEncodingLinearPcm; +import static androidx.media3.common.util.Util.isFloatPcmEncoding; +import static androidx.media3.common.util.Util.isPcmEncodingBigEndian; import static androidx.media3.muxer.MuxerUtil.UNSIGNED_INT_MAX_VALUE; import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkNotNull; @@ -37,15 +41,16 @@ import androidx.media3.container.MdtaMetadataEntry; import androidx.media3.container.Mp4LocationData; import androidx.media3.container.NalUnitUtil; +import androidx.media3.container.XmpData; import androidx.media3.muxer.FragmentedMp4Writer.SampleMetadata; import com.google.common.base.Splitter; import com.google.common.collect.ImmutableList; import com.google.common.collect.Iterables; import com.google.common.collect.Lists; -import com.google.common.primitives.Bytes; import com.google.common.primitives.Ints; import java.math.RoundingMode; import java.nio.ByteBuffer; +import java.nio.ByteOrder; import java.util.ArrayDeque; import java.util.ArrayList; import java.util.Arrays; @@ -55,6 +60,7 @@ import java.util.Locale; import java.util.Map; import java.util.Objects; +import java.util.UUID; import org.checkerframework.checker.nullness.qual.PolyNull; /** Writes out various types of boxes as per MP4 (ISO/IEC 14496-12) standards. */ @@ -62,6 +68,9 @@ /** Total number of bytes in an integer. */ private static final int BYTES_PER_INTEGER = 4; + /** Total number of bytes in a long. */ + private static final int BYTES_PER_LONG = 8; + /** Box size (4 bytes) + Box name (4 bytes) */ public static final int BOX_HEADER_SIZE = 8; @@ -77,6 +86,9 @@ /** The size (in bytes) of the tfhd box content. */ public static final int TFHD_BOX_CONTENT_SIZE = 4 * BYTES_PER_INTEGER; + /** The size (in bytes) of the tfdt box content. */ + public static final int TFDT_BOX_CONTENT_SIZE = BYTES_PER_INTEGER + BYTES_PER_LONG; + /** The maximum size (in bytes) of boxes that have fixed sizes. */ private static final int MAX_FIXED_LEAF_BOX_SIZE = 200; @@ -101,25 +113,6 @@ private Boxes() {} - public static final ImmutableList XMP_UUID = - ImmutableList.of( - (byte) 0xBE, - (byte) 0x7A, - (byte) 0xCF, - (byte) 0xCB, - (byte) 0x97, - (byte) 0xA9, - (byte) 0x42, - (byte) 0xE8, - (byte) 0x9C, - (byte) 0x71, - (byte) 0x99, - (byte) 0x94, - (byte) 0x91, - (byte) 0xE3, - (byte) 0xAF, - (byte) 0xAC); - /** Returns the moov box. */ @SuppressWarnings("InlinedApi") public static ByteBuffer moov( @@ -318,7 +311,7 @@ public static ByteBuffer moov( if (metadataCollector.xmpData != null) { return BoxUtils.concatenateBuffers( - moovBox, uuid(XMP_UUID, ByteBuffer.wrap(metadataCollector.xmpData.data))); + moovBox, uuid(XmpData.XMP_UUID, ByteBuffer.wrap(metadataCollector.xmpData.data))); } else { // No need for another copy if there is no XMP to be appended. return moovBox; @@ -635,7 +628,10 @@ public static ByteBuffer udta(@Nullable Mp4LocationData location) { } String locationString = - Util.formatInvariant("%+.4f%+.4f/", location.latitude, location.longitude); + location.altitude != Mp4LocationData.ALTITUDE_UNSET + ? Util.formatInvariant( + "%+.4f%+.4f%+.4f/", location.latitude, location.longitude, location.altitude) + : Util.formatInvariant("%+.4f%+.4f/", location.latitude, location.longitude); ByteBuffer xyzBoxContents = ByteBuffer.allocate(locationString.length() + 2 + 2); xyzBoxContents.putShort((short) (xyzBoxContents.capacity() - 4)); @@ -728,10 +724,13 @@ public static ByteBuffer meta(ByteBuffer... subBoxes) { * *

    This box is used for XMP and other metadata. */ - public static ByteBuffer uuid(List uuid, ByteBuffer contents) { + public static ByteBuffer uuid(UUID uuid, ByteBuffer contents) { checkArgument(contents.remaining() > 0); - return BoxUtils.wrapBoxesIntoBox( - "uuid", ImmutableList.of(ByteBuffer.wrap(Bytes.toArray(uuid)), contents)); + ByteBuffer uuidBuffer = ByteBuffer.allocate(16); + uuidBuffer.putLong(uuid.getMostSignificantBits()); + uuidBuffer.putLong(uuid.getLeastSignificantBits()); + uuidBuffer.flip(); + return BoxUtils.wrapBoxesIntoBox("uuid", ImmutableList.of(uuidBuffer, contents)); } /** Returns an audio sample entry box based on the MIME type. */ @@ -744,19 +743,21 @@ public static ByteBuffer audioSampleEntry(Format format) { contents.putInt(0x0); // reserved contents.putShort((short) 0x0); // reserved - contents.putShort((short) 0x1); // data ref index + contents.putShort((short) 0x1); // data ref index (points to the first entry in dref table) contents.putInt(0x0); // reserved contents.putInt(0x0); // reserved final boolean isIamf = Objects.equals(format.sampleMimeType, MimeTypes.AUDIO_IAMF); final int channelCount = isIamf ? 0 : format.channelCount; contents.putShort((short) channelCount); + // In AudioSampleEntry v0, sample_size is fixed to 16 as a template value (ISO/IEC 14496-12). + // The actual PCM bit depth for fpcm/ipcm is specified in the pcmC box. contents.putShort((short) 16); // sample size contents.putShort((short) 0x0); // predefined contents.putShort((short) 0x0); // reserved final int sampleRate = isIamf ? 0 : format.sampleRate; - contents.putInt(sampleRate << 16); + contents.putInt(sampleRate << 16); // 16.16 fixed-point format. contents.put(codecSpecificBox); @@ -779,10 +780,20 @@ public static ByteBuffer codecSpecificBox(Format format) { return damrBox(/* mode= */ (short) 0x83FF); // mode set: all enabled for AMR-WB case MimeTypes.AUDIO_OPUS: return dOpsBox(format); + case MimeTypes.AUDIO_E_AC3: + case MimeTypes.AUDIO_E_AC3_JOC: + return dec3Box(format); case MimeTypes.AUDIO_IAMF: return iacbBox(format); case MimeTypes.AUDIO_RAW: - return ByteBuffer.allocate(0); // No codec specific box for raw audio. + if (!isEncodingLinearPcm(format.pcmEncoding) || format.pcmEncoding == C.ENCODING_PCM_8BIT) { + throw new IllegalArgumentException("Unsupported encoding: " + format.pcmEncoding); + } + if (format.pcmEncoding == C.ENCODING_PCM_16BIT + || format.pcmEncoding == C.ENCODING_PCM_16BIT_BIG_ENDIAN) { + return ByteBuffer.allocate(0); + } + return pcmCBox(format); case MimeTypes.VIDEO_H263: return d263Box(format); case MimeTypes.VIDEO_H264: @@ -1132,19 +1143,34 @@ public static List calculateSampleCompositionTimeOffsets( /** Returns the stsz (sample size) box. */ public static ByteBuffer stsz(List writtenSamples) { - ByteBuffer contents = ByteBuffer.allocate(writtenSamples.size() * 4 + MAX_FIXED_LEAF_BOX_SIZE); - - contents.putInt(0x0); // version and flags + boolean sameSampleSize = !writtenSamples.isEmpty() && writtenSamples.get(0).size != 0; + int firstSampleSize = writtenSamples.isEmpty() ? 0 : writtenSamples.get(0).size; + if (sameSampleSize) { + for (int i = 1; i < writtenSamples.size(); i++) { + if (writtenSamples.get(i).size != firstSampleSize) { + sameSampleSize = false; + break; + } + } + } - // TODO: b/270583563 - Consider optimizing for identically-sized samples. - // sample_size: specifying the default sample size. Set to zero to indicate that the samples - // have different sizes and they are stored in the sample size table. - contents.putInt(0); + int capacity = + sameSampleSize + ? MAX_FIXED_LEAF_BOX_SIZE + : (writtenSamples.size() * 4 + MAX_FIXED_LEAF_BOX_SIZE); + ByteBuffer contents = ByteBuffer.allocate(capacity); - contents.putInt(writtenSamples.size()); // sample_count + contents.putInt(0x0); // version and flags - for (int i = 0; i < writtenSamples.size(); i++) { - contents.putInt(writtenSamples.get(i).size); + if (sameSampleSize) { + contents.putInt(firstSampleSize); // sample_size + contents.putInt(writtenSamples.size()); // sample_count + } else { + contents.putInt(0); // sample_size + contents.putInt(writtenSamples.size()); // sample_count + for (int i = 0; i < writtenSamples.size(); i++) { + contents.putInt(writtenSamples.get(i).size); // entry_size + } } contents.flip(); @@ -1264,8 +1290,13 @@ public static ByteBuffer stbl(ByteBuffer... subBoxes) { return BoxUtils.wrapBoxesIntoBox("stbl", Arrays.asList(subBoxes)); } - /** Creates the ftyp box. */ - public static ByteBuffer ftyp() { + /** + * Creates the ftyp box. + * + * @param additionalCompatibleBrands Additional compatible brands to declare, beyond the default + * set ({@code isom}, {@code iso2}, {@code mp41}). + */ + /* package */ static ByteBuffer ftyp(List additionalCompatibleBrands) { List boxBytes = new ArrayList<>(); String majorVersion = "isom"; @@ -1277,9 +1308,15 @@ public static ByteBuffer ftyp() { minorBytes.flip(); boxBytes.add(minorBytes); - String[] compatibleBrands = {"isom", "iso2", "mp41"}; + List compatibleBrands = new ArrayList<>(Arrays.asList("isom", "iso2", "mp41")); + compatibleBrands.addAll(additionalCompatibleBrands); for (String compatibleBrand : compatibleBrands) { - boxBytes.add(ByteBuffer.wrap(Util.getUtf8Bytes(compatibleBrand))); + byte[] compatibleBrandBytes = Util.getUtf8Bytes(compatibleBrand); + checkArgument( + compatibleBrandBytes.length == 4, + "Compatible brand must be 4 bytes: %s", + compatibleBrand); + boxBytes.add(ByteBuffer.wrap(compatibleBrandBytes)); } return BoxUtils.wrapBoxesIntoBox("ftyp", boxBytes); @@ -1301,8 +1338,17 @@ public static ByteBuffer mfhd(int sequenceNumber) { } /** Returns a track fragment (traf) box. */ - public static ByteBuffer traf(ByteBuffer tfhdBox, ByteBuffer trunBox) { - return BoxUtils.wrapBoxesIntoBox("traf", ImmutableList.of(tfhdBox, trunBox)); + public static ByteBuffer traf(ByteBuffer tfhdBox, ByteBuffer tfdtBox, ByteBuffer trunBox) { + return BoxUtils.wrapBoxesIntoBox("traf", ImmutableList.of(tfhdBox, tfdtBox, trunBox)); + } + + /** Returns a track fragment base media decode time (tfdt) box. */ + public static ByteBuffer tfdt(long baseMediaDecodeTime) { + ByteBuffer contents = ByteBuffer.allocate(TFDT_BOX_CONTENT_SIZE); + contents.putInt(0x01000000); // Version 1 (64-bit), flags = 0 + contents.putLong(baseMediaDecodeTime); + contents.flip(); + return BoxUtils.wrapIntoBox("tfdt", contents); } /** Returns a track fragment header (tfhd) box. */ @@ -1356,7 +1402,7 @@ public static ByteBuffer trun( return BoxUtils.wrapIntoBox("trun", contents); } - /** Returns the size required for {@link #trun(int, List, int, boolean)} box content. */ + /** Returns the size required for {@link #trun} box content. */ public static int getTrunBoxContentSize(int sampleCount, boolean hasBFrame) { int trunBoxFixedSize = 3 * BYTES_PER_INTEGER; int intWrittenPerSample = hasBFrame ? 4 : 3; @@ -1407,7 +1453,7 @@ private static ByteBuffer trefTypeBox(int referenceType, List trackIds) contents.putInt(trackIds.get(i)); } contents.flip(); - byte[] typeBytes = Util.toByteArray(referenceType); + byte[] typeBytes = Ints.toByteArray(referenceType); return BoxUtils.wrapIntoBox(typeBytes, contents); } @@ -1523,6 +1569,13 @@ private static ByteBuffer avcCBox(Format format) { return BoxUtils.wrapIntoBox("avcC", contents); } + private static ByteBuffer dec3Box(Format format) { + checkArgument(!format.initializationData.isEmpty(), "csd-0 not found in format for dec3 box."); + byte[] csd0 = format.initializationData.get(0); + checkArgument(csd0.length > 0, "csd-0 is empty for dec3 box."); + return BoxUtils.wrapIntoBox("dec3", ByteBuffer.wrap(csd0)); + } + /** Returns the hvcC box as per ISO/IEC 14496-15: 8.3.3.1.2. */ private static ByteBuffer hvcCBox(Format format) { // For H.265, all three codec-specific NALUs (VPS, SPS, PPS) are packed into csd-0. @@ -1658,7 +1711,7 @@ private static ByteBuffer doviBox(int profile, byte[] csd) { /** Returns a dolby vision box as per Dolby Vision ISO media format. */ private static ByteBuffer doviSpecificBox(Format format) { @Nullable Pair profileAndLevel = getDolbyVisionProfileAndLevel(format); - checkNotNull(profileAndLevel, "Can't identify Dolby vision profile"); + checkArgument(profileAndLevel != null, "Can't identify Dolby vision profile"); ByteBuffer avcHevcBox = profileAndLevel.first <= 8 ? hvcCBox(format) : avcCBox(format); byte[] dolbyVisionCsd = CodecSpecificDataUtil.buildDolbyVisionInitializationData( @@ -1812,8 +1865,8 @@ private static ByteBuffer colrBox(ColorInfo colorInfo) { /** Returns codec specific fourcc for Dolby vision. */ private static String getDoviFourcc(Format format) { - Pair profileAndLevel = getDolbyVisionProfileAndLevel(format); - checkNotNull(profileAndLevel, "Dolby Vision profile and level is not found."); + @Nullable Pair profileAndLevel = getDolbyVisionProfileAndLevel(format); + checkArgument(profileAndLevel != null, "Dolby Vision profile and level is not found."); switch (profileAndLevel.first) { case 5: return "dvh1"; @@ -1845,15 +1898,23 @@ private static String codecSpecificFourcc(Format format) { return "s263"; case MimeTypes.AUDIO_OPUS: return "Opus"; + case MimeTypes.AUDIO_E_AC3: + case MimeTypes.AUDIO_E_AC3_JOC: + return "ec-3"; case MimeTypes.AUDIO_IAMF: return "iamf"; case MimeTypes.AUDIO_RAW: + if (!isEncodingLinearPcm(format.pcmEncoding) || format.pcmEncoding == C.ENCODING_PCM_8BIT) { + throw new IllegalArgumentException("Unsupported encoding: " + format.pcmEncoding); + } if (format.pcmEncoding == C.ENCODING_PCM_16BIT) { return "sowt"; } else if (format.pcmEncoding == C.ENCODING_PCM_16BIT_BIG_ENDIAN) { return "twos"; + } else if (isFloatPcmEncoding(format.pcmEncoding)) { + return "fpcm"; } else { - throw new IllegalArgumentException("Unsupported PCM encoding: " + format.pcmEncoding); + return "ipcm"; } case MimeTypes.VIDEO_H264: return "avc1"; @@ -1874,6 +1935,27 @@ private static String codecSpecificFourcc(Format format) { } } + /** Returns the pcmC (PCM configuration) box. */ + private static ByteBuffer pcmCBox(Format format) { + checkArgument(Objects.equals(format.sampleMimeType, MimeTypes.AUDIO_RAW)); + // 16-bit integer PCM does not write a pcmC box and is written with 'sowt' or 'twos' sample + // entry instead for wider compatibility. + checkArgument( + isEncodingLinearPcm(format.pcmEncoding) + && format.pcmEncoding != C.ENCODING_PCM_8BIT + && format.pcmEncoding != C.ENCODING_PCM_16BIT + && format.pcmEncoding != C.ENCODING_PCM_16BIT_BIG_ENDIAN); + + ByteBuffer contents = ByteBuffer.allocate(6); + contents.putInt(0x0); // version (0) and flags (0) + // Format flags: bit 0 indicates Little-Endian (0x01), 0x00 is Big-Endian. + byte formatFlags = isPcmEncodingBigEndian(format.pcmEncoding) ? (byte) 0x00 : (byte) 0x01; + contents.put(formatFlags); + contents.put((byte) (getPcmFrameSize(format.pcmEncoding, /* channelCount= */ 1) * 8)); + contents.flip(); + return BoxUtils.wrapIntoBox("pcmC", contents); + } + /** Returns the esds box. */ private static ByteBuffer esdsBox(Format format) { checkArgument( @@ -1999,29 +2081,39 @@ private static ByteBuffer iacbBox(Format format) { return BoxUtils.wrapIntoBox("iacb", contents); } - /** Returns the audio dOps box for Opus codec as per RFC-7845: 5.1. */ + /** + * Returns the audio {@code dOps} box for Opus codec as per Opus-in-ISOBMFF Section 4.3.2, + * converted from {@code csd-0} ({@code OpusHead} per RFC 7845 Section 5.1). + */ private static ByteBuffer dOpsBox(Format format) { checkArgument( !format.initializationData.isEmpty(), "csd-0 not found in the format for dOps box."); int opusHeadSignatureLength = 8; + int minimumDopsPayloadSize = 11; byte[] csd0 = CodecSpecificDataUtil.getOpusInitializationData(format); - // As csd0 contains 'OpusHead' in first 8 bytes, csd0 length should be greater than 8. - checkArgument(csd0.length >= opusHeadSignatureLength); - ByteBuffer contents = ByteBuffer.allocate(csd0.length); - // Skip 8 bytes containing "OpusHead". - contents.put( - /* src */ csd0, - /* offset */ opusHeadSignatureLength, - /* length */ csd0.length - opusHeadSignatureLength); - - // For encapsulation of OPUS in MP4, the version byte (byte 0) in dOps box should be 0. - // (See https://opus-codec.org/docs/opus_in_isobmff.html, Section 4.3.2 Opus Specific Box). - // And for Ogg containers, the version byte is - // expected to be 1 (See https://www.rfc-editor.org/rfc/rfc7845#section-5.1, Section 5.1.2). - // The contents are otherwise identical. - checkState(contents.get(0) == 0 || contents.get(0) == 1); - contents.put(0, (byte) 0); + // As csd0 contains 'OpusHead' in first 8 bytes followed by at least 11 bytes of header payload, + // csd0 length must be at least 19. + checkArgument(csd0.length >= opusHeadSignatureLength + minimumDopsPayloadSize); + int dOpsPayloadSize = csd0.length - opusHeadSignatureLength; + ByteBuffer csd0Buffer = + ByteBuffer.wrap(csd0, opusHeadSignatureLength, dOpsPayloadSize) + .order(ByteOrder.LITTLE_ENDIAN); + + // For encapsulation of OPUS in MP4, the version byte in dOps box should be 0 and multi-byte + // fields (PreSkip, InputSampleRate, OutputGain) are big-endian + // (See https://opus-codec.org/docs/opus_in_isobmff.html, Section 4.3.2 Opus Specific Box), + // whereas RFC 7845 OpusHead uses version 1 and little-endian byte order + // (See https://www.rfc-editor.org/rfc/rfc7845#section-5.1). + byte version = csd0Buffer.get(); + checkState(version == 0 || version == 1); + ByteBuffer contents = ByteBuffer.allocate(dOpsPayloadSize); + contents.put((byte) 0); // Version (0 in dOps) + contents.put(csd0Buffer.get()); // OutputChannelCount + contents.putShort(csd0Buffer.getShort()); // PreSkip + contents.putInt(csd0Buffer.getInt()); // InputSampleRate + contents.putShort(csd0Buffer.getShort()); // OutputGain + contents.put(csd0Buffer); // ChannelMappingFamily and optional ChannelMappingTable contents.flip(); @@ -2099,19 +2191,33 @@ private static long findMinimumPresentationTimestampUsAcrossTracks(List t return minInputPtsUs != Long.MAX_VALUE ? minInputPtsUs : C.TIME_UNSET; } + /** Returns whether the Dolby Vision profile of the given {@link Format} is supported. */ + /* package */ static boolean isDolbyVisionProfileSupported(Format format) { + @Nullable Pair profileAndLevel = getDolbyVisionProfileAndLevel(format); + return profileAndLevel != null + && (profileAndLevel.first == 5 || profileAndLevel.first == 8 || profileAndLevel.first == 9); + } + /** Returns profile and level of dolby vision */ @Nullable /* package */ static Pair getDolbyVisionProfileAndLevel(Format format) { - checkNotNull(format.codecs, "Codec string is null for Dolby Vision format."); + if (format.codecs == null) { + return null; + } List parts = Splitter.on('.').splitToList(format.codecs); if (parts.size() < 3) { // The codec has fewer parts than required by the Dolby Vision codec string format. Log.w(TAG, "Invalid Dolby Vision codec string: " + format.codecs); return null; } - int profile = Integer.parseInt(parts.get(1)); - int level = Integer.parseInt(parts.get(2)); - return Pair.create(profile, level); + try { + int profile = Integer.parseInt(parts.get(1)); + int level = Integer.parseInt(parts.get(2)); + return Pair.create(profile, level); + } catch (NumberFormatException e) { + Log.w(TAG, "Invalid Dolby Vision codec string: " + format.codecs, e); + return null; + } } /** Returns H263 profile and level from codec string. */ @@ -2133,4 +2239,83 @@ private static Pair getH263ProfileAndLevel(Format format) { return DEFAULT_H263_PROFILE_AND_LEVEL; } } + + /** Represents an entry in the 'tfra' (Track Fragment Random Access) box. */ + public static class TfraEntry { + public final long time; + public final long moofOffset; + public final int trafNumber; + public final int trunNumber; + public final int sampleNumber; + + public TfraEntry(long time, long moofOffset, int trafNumber) { + this.time = time; + this.moofOffset = moofOffset; + this.trafNumber = trafNumber; + this.trunNumber = 1; + this.sampleNumber = 1; + } + } + + /** Returns a 'tfra' (Track Fragment Random Access) box. */ + public static ByteBuffer tfra(int trackId, List entries) { + int headerSize = 4; // version (1 byte) + flags (3 bytes) + int fixedFieldsSize = + 4 // track_ID (4 bytes) + + 4 // length_size_of_traf_num (2 bits), length_size_of_trun_num (2 bits), + // length_size_of_sample_num (2 bits), reserved (26 bits) + + 4; // number_of_entry (4 bytes) + + int entrySize = 8 + 8 + 1 + 1 + 1; + int contentSize = headerSize + fixedFieldsSize + (entries.size() * entrySize); + + ByteBuffer contents = ByteBuffer.allocate(contentSize); + contents.putInt(1 << 24); // version 1 (byte 0 = 0x01), flags = 0 (bytes 1-3 = 0x000000) + contents.putInt(trackId); + // Reserved (26 bits = 0) + length_size_of_traf_num (2 bits = 0: 1 byte) + // + length_size_of_trun_num (2 bits = 0: 1 byte) + length_size_of_sample_num (2 bits = 0: 1 + // byte). + contents.putInt(0x00000000); + contents.putInt(entries.size()); + + for (int i = 0; i < entries.size(); i++) { + TfraEntry entry = entries.get(i); + checkState(entry.trafNumber <= 255, "trafNumber must fit in 1 byte"); + checkState(entry.trunNumber <= 255, "trunNumber must fit in 1 byte"); + checkState(entry.sampleNumber <= 255, "sampleNumber must fit in 1 byte"); + contents.putLong(entry.time); + contents.putLong(entry.moofOffset); + contents.put((byte) entry.trafNumber); // 1-based index of traf box in enclosing moof + contents.put((byte) entry.trunNumber); // 1-based index of trun box in enclosing traf + contents.put((byte) entry.sampleNumber); // 1-based index of sample in enclosing trun + } + + contents.flip(); + return BoxUtils.wrapIntoBox("tfra", contents); + } + + /** Returns an 'mfro' (Movie Fragment Random Access Offset) box. */ + public static ByteBuffer mfro(int mfraSize) { + ByteBuffer contents = ByteBuffer.allocate(8); + contents.putInt(0x00000000); // version 0, flags = 0 + contents.putInt(mfraSize); + contents.flip(); + return BoxUtils.wrapIntoBox("mfro", contents); + } + + /** Returns an 'mfra' (Movie Fragment Random Access) box containing all tfra boxes and mfro. */ + public static ByteBuffer mfra(List tfraBoxes) { + int totalTfraSize = 0; + for (int i = 0; i < tfraBoxes.size(); i++) { + totalTfraSize += tfraBoxes.get(i).remaining(); + } + int mfroBoxSize = BOX_HEADER_SIZE + 8; // 16 bytes total + int mfraBoxSize = BOX_HEADER_SIZE + totalTfraSize + mfroBoxSize; + + List allBoxes = new ArrayList<>(tfraBoxes.size() + 1); + allBoxes.addAll(tfraBoxes); + allBoxes.add(mfro(mfraBoxSize)); + + return BoxUtils.wrapBoxesIntoBox("mfra", allBoxes); + } } diff --git a/libraries/muxer/src/main/java/androidx/media3/muxer/FragmentedMp4Muxer.java b/libraries/muxer/src/main/java/androidx/media3/muxer/FragmentedMp4Muxer.java index f64e1f3da53..905886570fd 100644 --- a/libraries/muxer/src/main/java/androidx/media3/muxer/FragmentedMp4Muxer.java +++ b/libraries/muxer/src/main/java/androidx/media3/muxer/FragmentedMp4Muxer.java @@ -36,6 +36,7 @@ import java.nio.ByteBuffer; import java.nio.channels.Channels; import java.nio.channels.WritableByteChannel; +import java.util.Objects; /** * A muxer for creating a fragmented MP4 file. @@ -52,7 +53,7 @@ *

  • H.265 (HEVC) *
  • VP9 *
  • APV - *
  • Dolby Vision + *
  • Dolby Vision (profiles 5, 8 and 9) * *
  • Audio Codecs: *
      @@ -97,6 +98,7 @@ public static final class Builder { private long fragmentDurationMs; private boolean sampleCopyEnabled; + private boolean fragmentsBetweenKeyFramesEnabled; /** * @deprecated Use {@link FragmentedMp4Muxer.Builder#Builder(WritableByteChannel)} instead. @@ -120,6 +122,7 @@ public Builder(WritableByteChannel outputChannel) { this.outputChannel = outputChannel; fragmentDurationMs = DEFAULT_FRAGMENT_DURATION_MS; sampleCopyEnabled = true; + fragmentsBetweenKeyFramesEnabled = false; } /** @@ -152,9 +155,29 @@ public Builder setSampleCopyingEnabled(boolean enabled) { return this; } + /** + * Sets whether fragments can start at non-key frames. + * + *

      When enabled, a fragment is written once its video samples span the {@linkplain + * #setFragmentDurationMs fragment duration}, even if the next sample is not a key frame. This + * allows the output to grow while samples arrive when key frames are farther apart than the + * fragment duration. Fragments start at non-key frames only when the video {@link Format} has + * {@link Format#maxNumReorderSamples} set to {@code 0}; tracks that may contain B-frames still + * end fragments only at key frames. Fragments that start at non-key frames are not listed as + * random access points in the {@code mfra} box. + * + *

      The default value is {@code false}. + */ + @CanIgnoreReturnValue + public Builder setFragmentsBetweenKeyFramesEnabled(boolean enabled) { + this.fragmentsBetweenKeyFramesEnabled = enabled; + return this; + } + /** Builds a {@link FragmentedMp4Muxer} instance. */ public FragmentedMp4Muxer build() { - return new FragmentedMp4Muxer(outputChannel, fragmentDurationMs, sampleCopyEnabled); + return new FragmentedMp4Muxer( + outputChannel, fragmentDurationMs, sampleCopyEnabled, fragmentsBetweenKeyFramesEnabled); } } @@ -180,7 +203,9 @@ public FragmentedMp4Muxer build() { MimeTypes.AUDIO_IAMF, MimeTypes.AUDIO_OPUS, MimeTypes.AUDIO_VORBIS, - MimeTypes.AUDIO_RAW); + MimeTypes.AUDIO_RAW, + MimeTypes.AUDIO_E_AC3, + MimeTypes.AUDIO_E_AC3_JOC); // LINT.ThenChange(Boxes.java:codec_specific_boxes) @@ -189,7 +214,10 @@ public FragmentedMp4Muxer build() { private final SparseArray trackIdToTrack; private FragmentedMp4Muxer( - WritableByteChannel outputChannel, long fragmentDurationMs, boolean sampleCopyEnabled) { + WritableByteChannel outputChannel, + long fragmentDurationMs, + boolean sampleCopyEnabled, + boolean fragmentsBetweenKeyFramesEnabled) { metadataCollector = new MetadataCollector(); fragmentedMp4Writer = new FragmentedMp4Writer( @@ -197,7 +225,8 @@ private FragmentedMp4Muxer( metadataCollector, AnnexBToAvccConverter.DEFAULT, fragmentDurationMs, - sampleCopyEnabled); + sampleCopyEnabled, + fragmentsBetweenKeyFramesEnabled); trackIdToTrack = new SparseArray<>(); } @@ -211,9 +240,17 @@ private FragmentedMp4Muxer( * @param format The {@link Format} for the track. * @return A unique track id. The track id is non-negative. It should be used in {@link * #writeSampleData}. + * @throws MuxerException If an error occurs while adding track. */ @Override - public int addTrack(Format format) { + public int addTrack(Format format) throws MuxerException { + if (Objects.equals(format.sampleMimeType, MimeTypes.VIDEO_DOLBY_VISION) + && !Boxes.isDolbyVisionProfileSupported(format)) { + throw new MuxerException( + "Unsupported Dolby Vision profile for format: " + format, + new IllegalArgumentException("Unsupported Dolby Vision codecs: " + format.codecs)); + } + // TODO: b/561659631 - Validate track format eagerly when adding track. Track track = fragmentedMp4Writer.addTrack(/* sortKey= */ 1, format); trackIdToTrack.append(track.id, track); return track.id; diff --git a/libraries/muxer/src/main/java/androidx/media3/muxer/FragmentedMp4Writer.java b/libraries/muxer/src/main/java/androidx/media3/muxer/FragmentedMp4Writer.java index d64b16ac9e4..10b8b14749b 100644 --- a/libraries/muxer/src/main/java/androidx/media3/muxer/FragmentedMp4Writer.java +++ b/libraries/muxer/src/main/java/androidx/media3/muxer/FragmentedMp4Writer.java @@ -19,6 +19,7 @@ import static androidx.media3.muxer.Av1ConfigUtil.createAv1CodecConfigurationRecord; import static androidx.media3.muxer.Boxes.BOX_HEADER_SIZE; import static androidx.media3.muxer.Boxes.MFHD_BOX_CONTENT_SIZE; +import static androidx.media3.muxer.Boxes.TFDT_BOX_CONTENT_SIZE; import static androidx.media3.muxer.Boxes.TFHD_BOX_CONTENT_SIZE; import static androidx.media3.muxer.Boxes.getTrunBoxContentSize; import static androidx.media3.muxer.Mp4Muxer.LAST_SAMPLE_DURATION_BEHAVIOR_SET_FROM_END_OF_STREAM_BUFFER_OR_DUPLICATE_PREVIOUS; @@ -100,16 +101,20 @@ public long getPosition() { private final AnnexBToAvccConverter annexBToAvccConverter; private final long fragmentDurationUs; private final boolean sampleCopyEnabled; + private final boolean fragmentsBetweenKeyFramesEnabled; private final @Mp4Muxer.LastSampleDurationBehavior int lastSampleDurationBehavior; private final List tracks; private final LinearByteBufferAllocator linearByteBufferAllocator; + private boolean isClosed; private @MonotonicNonNull Track videoTrack; private int currentFragmentSequenceNumber; private boolean headerCreated; private long minInputPresentationTimeUs; private long maxTrackDurationUs; private int nextTrackId; + private long lastVideoSampleTimeUs; + private boolean videoSamplesOutOfOrder; /** * Creates an instance. @@ -121,50 +126,70 @@ public long getPosition() { * AVCC format (which uses length prefixes). * @param fragmentDurationMs The fragment duration (in milliseconds). * @param sampleCopyEnabled Whether sample copying is enabled. + * @param fragmentsBetweenKeyFramesEnabled Whether fragments can start at non-key frames. */ public FragmentedMp4Writer( WritableByteChannel outputChannel, MetadataCollector metadataCollector, AnnexBToAvccConverter annexBToAvccConverter, long fragmentDurationMs, - boolean sampleCopyEnabled) { + boolean sampleCopyEnabled, + boolean fragmentsBetweenKeyFramesEnabled) { this.outputChannel = new PositionTrackingOutputChannel(outputChannel); this.metadataCollector = metadataCollector; this.annexBToAvccConverter = annexBToAvccConverter; this.fragmentDurationUs = fragmentDurationMs * 1_000; this.sampleCopyEnabled = sampleCopyEnabled; + this.fragmentsBetweenKeyFramesEnabled = fragmentsBetweenKeyFramesEnabled; lastSampleDurationBehavior = LAST_SAMPLE_DURATION_BEHAVIOR_SET_FROM_END_OF_STREAM_BUFFER_OR_DUPLICATE_PREVIOUS; tracks = new ArrayList<>(); minInputPresentationTimeUs = Long.MAX_VALUE; + lastVideoSampleTimeUs = C.TIME_UNSET; currentFragmentSequenceNumber = 1; + nextTrackId = 1; linearByteBufferAllocator = new LinearByteBufferAllocator(/* initialCapacity= */ 0); } public Track addTrack(int sortKey, Format format) { + checkState(!isClosed, "FragmentedMp4Writer is closed."); Track track = new Track(nextTrackId++, format, sampleCopyEnabled); tracks.add(track); if (MimeTypes.isVideo(format.sampleMimeType)) { videoTrack = track; + videoSamplesOutOfOrder = format.maxNumReorderSamples != 0; } return track; } public void writeSampleData(Track track, ByteBuffer byteBuffer, BufferInfo bufferInfo) throws IOException { - if (Objects.equals(track.format.sampleMimeType, MimeTypes.VIDEO_AV1) + checkState(!isClosed, "FragmentedMp4Writer is closed."); + boolean isSampleEmpty = bufferInfo.size == 0 || !byteBuffer.hasRemaining(); + if (!isSampleEmpty + && Objects.equals(track.format.sampleMimeType, MimeTypes.VIDEO_AV1) && track.format.initializationData.isEmpty() && track.parsedCsd == null) { track.parsedCsd = createAv1CodecConfigurationRecord(byteBuffer.duplicate()); } + if (fragmentsBetweenKeyFramesEnabled && track.equals(videoTrack) && !isSampleEmpty) { + if (lastVideoSampleTimeUs != C.TIME_UNSET + && bufferInfo.presentationTimeUs < lastVideoSampleTimeUs) { + videoSamplesOutOfOrder = true; + } + lastVideoSampleTimeUs = bufferInfo.presentationTimeUs; + } + if (!isSampleEmpty && shouldFlushPendingSamples(track, bufferInfo)) { + createFragment(/* nextFragmentFirstSampleTimeUs= */ bufferInfo.presentationTimeUs); + } + track.writeSampleData(byteBuffer, bufferInfo); + if (track.pendingSamplesBufferInfo.isEmpty()) { + return; + } if (!headerCreated) { createHeader(); headerCreated = true; } - if (shouldFlushPendingSamples(track, bufferInfo)) { - createFragment(); - } - track.writeSampleData(byteBuffer, bufferInfo); BufferInfo firstPendingSample = checkNotNull(track.pendingSamplesBufferInfo.peekFirst()); BufferInfo lastPendingSample = checkNotNull(track.pendingSamplesBufferInfo.peekLast()); minInputPresentationTimeUs = @@ -175,14 +200,39 @@ public void writeSampleData(Track track, ByteBuffer byteBuffer, BufferInfo buffe lastPendingSample.presentationTimeUs - firstPendingSample.presentationTimeUs); } + /** + * Finalizes output and closes the writer. + * + *

      This method can be called only once. The {@link FragmentedMp4Writer} cannot be used anymore + * once this method returns. + * + * @throws IOException If an error occurs while finalizing output or closing the output channel. + */ public void close() throws IOException { + checkState(!isClosed, "FragmentedMp4Writer is closed."); + isClosed = true; try { - createFragment(); + // There is no next fragment, so the last sample duration keeps the end of track behaviour. + createFragment(/* nextFragmentFirstSampleTimeUs= */ C.TIME_UNSET); + writeMfraBox(); } finally { outputChannel.close(); } } + private void writeMfraBox() throws IOException { + List tfraBoxes = new ArrayList<>(); + for (int i = 0; i < tracks.size(); i++) { + Track track = tracks.get(i); + if (!track.tfraEntries.isEmpty()) { + tfraBoxes.add(Boxes.tfra(track.id, track.tfraEntries)); + } + } + if (!tfraBoxes.isEmpty()) { + outputChannel.write(Boxes.mfra(tfraBoxes)); + } + } + private static ImmutableList createTrafBoxes( List trackInfos, long moofBoxStartPosition) { ImmutableList.Builder trafBoxes = new ImmutableList.Builder<>(); @@ -196,6 +246,7 @@ private static ImmutableList createTrafBoxes( trafBoxes.add( Boxes.traf( Boxes.tfhd(currentTrackInfo.trackId, /* baseDataOffset= */ moofBoxStartPosition), + Boxes.tfdt(currentTrackInfo.baseMediaDecodeTime), Boxes.trun( currentTrackInfo.trackFormat, currentTrackInfo.pendingSamplesMetadata, @@ -212,15 +263,18 @@ private static int calculateMoofBoxSize(List trackInfos) { mfhd traf tfhd + tfdt trun traf tfhd + tfdt trun */ int moofBoxHeaderSize = BOX_HEADER_SIZE; int mfhdBoxSize = BOX_HEADER_SIZE + MFHD_BOX_CONTENT_SIZE; int trafBoxHeaderSize = BOX_HEADER_SIZE; int tfhdBoxSize = BOX_HEADER_SIZE + TFHD_BOX_CONTENT_SIZE; + int tfdtBoxSize = BOX_HEADER_SIZE + TFDT_BOX_CONTENT_SIZE; int trunBoxHeaderFixedSize = BOX_HEADER_SIZE; int trafBoxesSize = 0; for (int i = 0; i < trackInfos.size(); i++) { @@ -228,27 +282,30 @@ private static int calculateMoofBoxSize(List trackInfos) { int trunBoxSize = trunBoxHeaderFixedSize + getTrunBoxContentSize(trackInfo.pendingSamplesMetadata.size(), trackInfo.hasBFrame); - trafBoxesSize += trafBoxHeaderSize + tfhdBoxSize + trunBoxSize; + trafBoxesSize += trafBoxHeaderSize + tfhdBoxSize + tfdtBoxSize + trunBoxSize; } return moofBoxHeaderSize + mfhdBoxSize + trafBoxesSize; } private void createHeader() throws IOException { - outputChannel.write(Boxes.ftyp()); + outputChannel.write(Boxes.ftyp(MuxerUtil.getFtypCompatibleBrands(tracks))); outputChannel.write( Boxes.moov( tracks, metadataCollector, /* isFragmentedMp4= */ true, lastSampleDurationBehavior)); } private boolean shouldFlushPendingSamples(Track track, BufferInfo nextSampleBufferInfo) { + // LINT.IfChange(fragmentation_logic) // If video track is present then fragment will be created based on group of pictures and // track's duration so far. if (videoTrack != null) { - // Video samples can be written only when complete group of pictures are present. + // Video samples can be written only when complete group of pictures are present, unless + // fragments can start at non-key frames. if (track.equals(videoTrack) && track.hadKeyframe - && ((nextSampleBufferInfo.flags & C.BUFFER_FLAG_KEY_FRAME) > 0)) { + && ((nextSampleBufferInfo.flags & C.BUFFER_FLAG_KEY_FRAME) > 0 + || canEndFragmentBeforeNonKeyFrame(nextSampleBufferInfo))) { BufferInfo firstPendingSample = checkNotNull(track.pendingSamplesBufferInfo.peekFirst()); BufferInfo lastPendingSample = checkNotNull(track.pendingSamplesBufferInfo.peekLast()); return lastPendingSample.presentationTimeUs - firstPendingSample.presentationTimeUs @@ -258,9 +315,25 @@ private boolean shouldFlushPendingSamples(Track track, BufferInfo nextSampleBuff } else { return maxTrackDurationUs >= fragmentDurationUs; } + // LINT.ThenChange(:mfra_indexing_logic) + } + + /** Returns whether a fragment can end before a non-key video sample. */ + private boolean canEndFragmentBeforeNonKeyFrame(BufferInfo nextSampleBufferInfo) { + if (!fragmentsBetweenKeyFramesEnabled || videoSamplesOutOfOrder) { + return false; + } + BufferInfo lastPendingSample = checkNotNull(videoTrack).pendingSamplesBufferInfo.peekLast(); + return lastPendingSample != null + && nextSampleBufferInfo.presentationTimeUs > lastPendingSample.presentationTimeUs; } - private void createFragment() throws IOException { + /** + * Writes all pending samples into a new fragment. + * + *

      Pass {@link C#TIME_UNSET} for the last fragment. + */ + private void createFragment(long nextFragmentFirstSampleTimeUs) throws IOException { /* Each fragment looks like: moof mfhd @@ -272,9 +345,33 @@ private void createFragment() throws IOException { trun mdat */ - ImmutableList trackInfos = processAllTracks(); - ImmutableList trafBoxes = - createTrafBoxes(trackInfos, /* moofBoxStartPosition= */ outputChannel.getPosition()); + long moofBoxStartPosition = outputChannel.getPosition(); + // The 1-based index of the 'traf' box within the 'moof' box for the current track. + int activeTrafIndex = 1; + for (int i = 0; i < tracks.size(); i++) { + Track track = tracks.get(i); + if (!track.pendingSamplesBufferInfo.isEmpty()) { + BufferInfo firstPendingSample = checkNotNull(track.pendingSamplesBufferInfo.peekFirst()); + // LINT.IfChange(mfra_indexing_logic) + // Index video keyframes, audio keyframes, and the first sample of metadata/text tracks. + boolean isRandomAccessPoint = + (firstPendingSample.flags & C.BUFFER_FLAG_KEY_FRAME) != 0 + || (!MimeTypes.isVideo(track.format.sampleMimeType) + && !MimeTypes.isAudio(track.format.sampleMimeType)); + // LINT.ThenChange(:fragmentation_logic) + if (isRandomAccessPoint) { + long ptsInTimescale = + Util.scaleLargeTimestamp( + firstPendingSample.presentationTimeUs, track.videoUnitTimebase(), 1_000_000L); + track.tfraEntries.add( + new Boxes.TfraEntry(ptsInTimescale, moofBoxStartPosition, activeTrafIndex)); + } + activeTrafIndex++; + } + } + + ImmutableList trackInfos = processAllTracks(nextFragmentFirstSampleTimeUs); + ImmutableList trafBoxes = createTrafBoxes(trackInfos, moofBoxStartPosition); if (trafBoxes.isEmpty()) { return; } @@ -321,17 +418,21 @@ private void writeMdatBox(List trackInfos) throws IOExceptio linearByteBufferAllocator.reset(); } - private ImmutableList processAllTracks() { + private ImmutableList processAllTracks(long nextFragmentFirstSampleTimeUs) { ImmutableList.Builder trackInfos = new ImmutableList.Builder<>(); for (int i = 0; i < tracks.size(); i++) { - if (!tracks.get(i).pendingSamplesBufferInfo.isEmpty()) { - trackInfos.add(processTrack(/* trackId= */ i + 1, tracks.get(i))); + Track track = tracks.get(i); + if (!track.pendingSamplesBufferInfo.isEmpty()) { + long trackNextSampleTimeUs = + track.equals(videoTrack) ? nextFragmentFirstSampleTimeUs : C.TIME_UNSET; + trackInfos.add(processTrack(track.id, track, trackNextSampleTimeUs)); } } return trackInfos.build(); } - private ProcessedTrackInfo processTrack(int trackId, Track track) { + private ProcessedTrackInfo processTrack( + int trackId, Track track, long nextFragmentFirstSampleTimeUs) { checkState(track.pendingSamplesByteBuffer.size() == track.pendingSamplesBufferInfo.size()); ImmutableList.Builder pendingSamplesByteBuffer = new ImmutableList.Builder<>(); @@ -360,12 +461,16 @@ private ProcessedTrackInfo processTrack(int trackId, Track track) { boolean hasBFrame = false; ImmutableList pendingSamplesBufferInfo = pendingSamplesBufferInfoBuilder.build(); + long fragmentEndTimestampUs = + nextFragmentFirstSampleTimeUs != C.TIME_UNSET + ? nextFragmentFirstSampleTimeUs + : track.endOfStreamTimestampUs; List sampleDurations = Boxes.convertPresentationTimestampsToDurationsVu( pendingSamplesBufferInfo, track.videoUnitTimebase(), LAST_SAMPLE_DURATION_BEHAVIOR_SET_FROM_END_OF_STREAM_BUFFER_OR_DUPLICATE_PREVIOUS, - track.endOfStreamTimestampUs); + fragmentEndTimestampUs); List sampleCompositionTimeOffsets = Boxes.calculateSampleCompositionTimeOffsets( @@ -386,10 +491,24 @@ private ProcessedTrackInfo processTrack(int trackId, Track track) { hasBFrame ? sampleCompositionTimeOffsets.get(i) : 0)); } + BufferInfo firstPendingSample = checkNotNull(pendingSamplesBufferInfo.get(0)); + // The baseMediaDecodeTime is scaled to the track's timescale (unit timebase) per + // ISO/IEC 14496-12 Section 8.8.8 and CMAF ISO/IEC 23000-19 Section 7.5.16. Initial + // presentation timestamps can be negative (e.g. audio pre-roll or encoder delay in Opus or + // IAMF streams). Since baseMediaDecodeTime in tfdt is an unsigned integer, it is clamped to 0. + long baseMediaDecodeTime = + max( + 0L, + Util.scaleLargeTimestamp( + /* timestamp= */ firstPendingSample.presentationTimeUs, + /* multiplier= */ track.videoUnitTimebase(), + /* divisor= */ 1_000_000L)); + return new ProcessedTrackInfo( trackId, track.format, totalSamplesSize, + baseMediaDecodeTime, hasBFrame, pendingSamplesByteBuffer.build(), pendingSamplesMetadata.build()); @@ -399,6 +518,7 @@ private static class ProcessedTrackInfo { public final int trackId; public final Format trackFormat; public final int totalSamplesSize; + public final long baseMediaDecodeTime; public final boolean hasBFrame; public final ImmutableList pendingSamplesByteBuffer; public final ImmutableList pendingSamplesMetadata; @@ -407,12 +527,14 @@ public ProcessedTrackInfo( int trackId, Format trackFormat, int totalSamplesSize, + long baseMediaDecodeTime, boolean hasBFrame, ImmutableList pendingSamplesByteBuffer, ImmutableList pendingSamplesMetadata) { this.trackId = trackId; this.trackFormat = trackFormat; this.totalSamplesSize = totalSamplesSize; + this.baseMediaDecodeTime = baseMediaDecodeTime; this.hasBFrame = hasBFrame; this.pendingSamplesByteBuffer = pendingSamplesByteBuffer; this.pendingSamplesMetadata = pendingSamplesMetadata; diff --git a/libraries/muxer/src/main/java/androidx/media3/muxer/MediaMuxerCompat.java b/libraries/muxer/src/main/java/androidx/media3/muxer/MediaMuxerCompat.java index 5bd076f8c25..f113d9b1e41 100644 --- a/libraries/muxer/src/main/java/androidx/media3/muxer/MediaMuxerCompat.java +++ b/libraries/muxer/src/main/java/androidx/media3/muxer/MediaMuxerCompat.java @@ -16,7 +16,6 @@ package androidx.media3.muxer; import static androidx.media3.muxer.MuxerUtil.getMuxerBufferInfoFromMediaCodecBufferInfo; -import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkState; import static java.lang.annotation.ElementType.TYPE_USE; @@ -48,30 +47,47 @@ * A drop-in replacement for {@link MediaMuxer} that provides similar functionality, based on the * {@code media3.muxer} logic. * - *

      Currently only MP4 file format is supported. + *

      Supported file formats are {@link #OUTPUT_FORMAT_MP4 MP4} and {@link #OUTPUT_FORMAT_WEBM + * WEBM}. * *

      Supported codecs are: * *

        - *
      • Video Codecs: + *
      • For MP4 file format: *
          - *
        • AV1 - *
        • MPEG-4 - *
        • H.263 - *
        • H.264 (AVC) - *
        • H.265 (HEVC) - *
        • VP9 - *
        • APV - *
        • Dolby Vision + *
        • Video Codecs: + *
            + *
          • AV1 + *
          • MPEG-4 + *
          • H.263 + *
          • H.264 (AVC) + *
          • H.265 (HEVC) + *
          • VP9 + *
          • APV + *
          • Dolby Vision + *
          + *
        • Audio Codecs: + *
            + *
          • AAC + *
          • AMR-NB (Narrowband AMR) + *
          • AMR-WB (Wideband AMR) + *
          • Opus + *
          • Vorbis + *
          • Raw Audio + *
          *
        - *
      • Audio Codecs: + *
      • For WebM file format: *
          - *
        • AAC - *
        • AMR-NB (Narrowband AMR) - *
        • AMR-WB (Wideband AMR) - *
        • Opus - *
        • Vorbis - *
        • Raw Audio + *
        • Video Codecs: + *
            + *
          • VP8 + *
          • VP9 + *
          + *
        • Audio Codecs: + *
            + *
          • Opus + *
          • Vorbis + *
          *
        *
      * @@ -84,15 +100,19 @@ public final class MediaMuxerCompat { @Documented @Retention(RetentionPolicy.SOURCE) @Target(TYPE_USE) - @IntDef({OUTPUT_FORMAT_MP4}) + @IntDef({OUTPUT_FORMAT_MP4, OUTPUT_FORMAT_WEBM}) @UnstableApi public @interface OutputFormat {} /** The MP4 file format. */ public static final int OUTPUT_FORMAT_MP4 = MediaMuxer.OutputFormat.MUXER_OUTPUT_MPEG_4; + /** The WebM file format. */ + public static final int OUTPUT_FORMAT_WEBM = MediaMuxer.OutputFormat.MUXER_OUTPUT_WEBM; + @Nullable private final FileDescriptor fileDescriptor; private final Muxer muxer; + private final CompatMetadataDelegate compatMetadataDelegate; private boolean startedMuxer; private boolean closedMuxer; @@ -116,6 +136,7 @@ public MediaMuxerCompat(FileDescriptor fileDescriptor, @OutputFormat int outputF throw new IOException("Failed to create a copy of FileDescriptor", e); } muxer = createMuxer(new FileOutputStream(this.fileDescriptor), outputFormat); + compatMetadataDelegate = createCompatMetadataDelegate(outputFormat); } /** @@ -128,6 +149,7 @@ public MediaMuxerCompat(FileDescriptor fileDescriptor, @OutputFormat int outputF public MediaMuxerCompat(String filePath, @OutputFormat int outputFormat) throws IOException { fileDescriptor = null; muxer = createMuxer(new FileOutputStream(filePath), outputFormat); + compatMetadataDelegate = createCompatMetadataDelegate(outputFormat); } /** @@ -160,17 +182,7 @@ public void start() { public int addTrack(MediaFormat format) { checkState(!startedMuxer); try { - float captureFps = - MediaFormatUtil.getFloatFromIntOrFloat( - format, MediaFormat.KEY_CAPTURE_RATE, C.RATE_UNSET); - if (captureFps != C.RATE_UNSET) { - MdtaMetadataEntry captureFpsMetadata = - new MdtaMetadataEntry( - MdtaMetadataEntry.KEY_ANDROID_CAPTURE_FPS, - /* value= */ Util.toByteArray(captureFps), - MdtaMetadataEntry.TYPE_INDICATOR_FLOAT32); - muxer.addMetadataEntry(captureFpsMetadata); - } + compatMetadataDelegate.handleAddTrackMetadata(format, muxer); return muxer.addTrack(MediaFormatUtil.createFormatFromMediaFormat(format)); } catch (MuxerException e) { throw new RuntimeException(e); @@ -216,7 +228,7 @@ public void setLocation( @FloatRange(from = -90.0, to = 90.0) float latitude, @FloatRange(from = -180.0, to = 180.0) float longitude) { checkState(!startedMuxer); - muxer.addMetadataEntry(new Mp4LocationData(latitude, longitude)); + compatMetadataDelegate.setLocation(latitude, longitude, muxer); } /** @@ -230,7 +242,7 @@ public void setLocation( */ public void setOrientationHint(int degrees) { checkState(!startedMuxer); - muxer.addMetadataEntry(new Mp4OrientationData(degrees)); + compatMetadataDelegate.setOrientationHint(degrees, muxer); } /** @@ -273,7 +285,94 @@ private void closeMuxer() { private static Muxer createMuxer( FileOutputStream fileOutputStream, @OutputFormat int outputFormat) { - checkArgument(outputFormat == OUTPUT_FORMAT_MP4); - return new Mp4Muxer.Builder(SeekableMuxerOutput.of(fileOutputStream)).build(); + SeekableMuxerOutput muxerOutput = SeekableMuxerOutput.of(fileOutputStream); + switch (outputFormat) { + case OUTPUT_FORMAT_MP4: + return new Mp4Muxer.Builder(muxerOutput).build(); + case OUTPUT_FORMAT_WEBM: + return new WebmMuxer.Builder(muxerOutput).build(); + default: + throw new IllegalArgumentException("Unsupported output format: " + outputFormat); + } + } + + private static CompatMetadataDelegate createCompatMetadataDelegate( + @OutputFormat int outputFormat) { + switch (outputFormat) { + case OUTPUT_FORMAT_MP4: + return new Mp4CompatMetadataDelegate(); + case OUTPUT_FORMAT_WEBM: + return new CompatMetadataDelegate() {}; + default: + throw new IllegalArgumentException("Unsupported output format: " + outputFormat); + } + } + + /** Encapsulates MP4-specific metadata operations for {@link MediaMuxerCompat}. */ + private static final class Mp4CompatMetadataDelegate implements CompatMetadataDelegate { + @Override + public void handleAddTrackMetadata(MediaFormat format, Muxer muxer) { + float captureFps = + MediaFormatUtil.getFloatFromIntOrFloat( + format, MediaFormat.KEY_CAPTURE_RATE, C.RATE_UNSET); + if (captureFps != C.RATE_UNSET) { + MdtaMetadataEntry captureFpsMetadata = + new MdtaMetadataEntry( + MdtaMetadataEntry.KEY_ANDROID_CAPTURE_FPS, + /* value= */ Util.toByteArray(captureFps), + MdtaMetadataEntry.TYPE_INDICATOR_FLOAT32); + muxer.addMetadataEntry(captureFpsMetadata); + } + } + + @Override + public void setLocation( + @FloatRange(from = -90.0, to = 90.0) float latitude, + @FloatRange(from = -180.0, to = 180.0) float longitude, + Muxer muxer) { + muxer.addMetadataEntry(new Mp4LocationData(latitude, longitude)); + } + + @Override + public void setOrientationHint(int degrees, Muxer muxer) { + muxer.addMetadataEntry(new Mp4OrientationData(degrees)); + } + } + + /** Delegate interface for container-specific metadata handling in {@link MediaMuxerCompat}. */ + private interface CompatMetadataDelegate { + /** + * Extracts track metadata from {@code format} and adds it to {@code muxer}. + * + *

      The default implementation is a no-op. + * + * @param format The track {@link MediaFormat}. + * @param muxer The target {@link Muxer}. + */ + default void handleAddTrackMetadata(MediaFormat format, Muxer muxer) {} + + /** + * Sets the location of the media file on {@code muxer}. + * + *

      The default implementation is a no-op. + * + * @param latitude The latitude, in degrees. + * @param longitude The longitude, in degrees. + * @param muxer The target {@link Muxer}. + */ + default void setLocation( + @FloatRange(from = -90.0, to = 90.0) float latitude, + @FloatRange(from = -180.0, to = 180.0) float longitude, + Muxer muxer) {} + + /** + * Sets the orientation hint for the media file on {@code muxer}. + * + *

      The default implementation is a no-op. + * + * @param degrees The orientation, in degrees. + * @param muxer The target {@link Muxer}. + */ + default void setOrientationHint(int degrees, Muxer muxer) {} } } diff --git a/libraries/muxer/src/main/java/androidx/media3/muxer/Mp4Muxer.java b/libraries/muxer/src/main/java/androidx/media3/muxer/Mp4Muxer.java index ee0421739e8..2b7d091ba0c 100644 --- a/libraries/muxer/src/main/java/androidx/media3/muxer/Mp4Muxer.java +++ b/libraries/muxer/src/main/java/androidx/media3/muxer/Mp4Muxer.java @@ -57,6 +57,7 @@ import java.nio.channels.FileChannel; import java.util.ArrayList; import java.util.List; +import java.util.Objects; import org.checkerframework.checker.nullness.qual.EnsuresNonNull; /** @@ -74,7 +75,7 @@ *

    • H.265 (HEVC) *
    • VP9 *
    • APV - *
    • Dolby Vision + *
    • Dolby Vision (profiles 5, 8 and 9) *
    *
  • Audio Codecs: *
      @@ -414,7 +415,9 @@ public Mp4Muxer build() { MimeTypes.AUDIO_OPUS, MimeTypes.AUDIO_VORBIS, MimeTypes.AUDIO_RAW, - MimeTypes.AUDIO_IAMF); + MimeTypes.AUDIO_IAMF, + MimeTypes.AUDIO_E_AC3, + MimeTypes.AUDIO_E_AC3_JOC); // LINT.ThenChange(Boxes.java:codec_specific_boxes) @@ -482,7 +485,10 @@ private Mp4Muxer( * {@inheritDoc} * *

      Tracks can be added at any point before the muxer is closed, even after writing samples to - * other tracks. + * other tracks. Note that a track requiring an additional {@code ftyp} compatible brand (for + * example a {@link MimeTypes#VIDEO_DOLBY_VISION} track, which requires the {@code dby1} brand) + * that is added after samples have already been written to another track will not retroactively + * update the {@code ftyp} box. * *

      The order of tracks remains same in which they are added. * @@ -507,7 +513,10 @@ public int addTrack(Format format) throws MuxerException { * Adds a track of the given media format. * *

      Tracks can be added at any point before the muxer is closed, even after writing samples to - * other tracks. + * other tracks. Note that a track requiring an additional {@code ftyp} compatible brand (for + * example a {@link MimeTypes#VIDEO_DOLBY_VISION} track, which requires the {@code dby1} brand) + * that is added after samples have already been written to another track will not retroactively + * update the {@code ftyp} box. * *

      The final order of tracks is determined by the provided sort key. Tracks with a lower sort * key will be written before tracks with a higher sort key. Ordering between tracks with the same @@ -520,6 +529,13 @@ public int addTrack(Format format) throws MuxerException { * @throws MuxerException If an error occurs while adding track. */ public int addTrack(int sortKey, Format format) throws MuxerException { + if (Objects.equals(format.sampleMimeType, MimeTypes.VIDEO_DOLBY_VISION) + && !Boxes.isDolbyVisionProfileSupported(format)) { + throw new MuxerException( + "Unsupported Dolby Vision profile for format: " + format, + new IllegalArgumentException("Unsupported Dolby Vision codecs: " + format.codecs)); + } + // TODO: b/561659631 - Validate track format eagerly when adding track. Track track; if (outputFileFormat == FILE_FORMAT_MP4_WITH_AUXILIARY_TRACKS_EXTENSION && isAuxiliaryTrack(format)) { diff --git a/libraries/muxer/src/main/java/androidx/media3/muxer/Mp4Writer.java b/libraries/muxer/src/main/java/androidx/media3/muxer/Mp4Writer.java index f54d70f7747..3c5ae2a7ffb 100644 --- a/libraries/muxer/src/main/java/androidx/media3/muxer/Mp4Writer.java +++ b/libraries/muxer/src/main/java/androidx/media3/muxer/Mp4Writer.java @@ -117,6 +117,8 @@ public Mp4Writer( linearByteBufferAllocator = new LinearByteBufferAllocator(/* initialCapacity= */ 0); } + private boolean isClosed; + /** * Adds a track of the given {@link Format}. * @@ -126,6 +128,7 @@ public Mp4Writer( * @return A unique {@link Track}. It should be used in {@link #writeSampleData}. */ public Track addTrack(int trackId, int sortKey, Format format) { + checkState(!isClosed, "Mp4Writer is closed."); Track track = new Track(trackId, format, sortKey, sampleCopyEnabled); tracks.add(track); Collections.sort(tracks, (a, b) -> Integer.compare(a.sortKey, b.sortKey)); @@ -143,6 +146,7 @@ public Track addTrack(int trackId, int sortKey, Format format) { * @return A unique {@link Track}. It should be used in {@link #writeSampleData}. */ public Track addAuxiliaryTrack(int trackId, int sortKey, Format format) { + checkState(!isClosed, "Mp4Writer is closed."); Track track = new Track(trackId, format, sortKey, sampleCopyEnabled); auxiliaryTracks.add(track); Collections.sort(auxiliaryTracks, (a, b) -> Integer.compare(a.sortKey, b.sortKey)); @@ -159,6 +163,7 @@ public Track addAuxiliaryTrack(int trackId, int sortKey, Format format) { */ public void writeSampleData(Track track, ByteBuffer byteBuffer, BufferInfo bufferInfo) throws IOException { + checkState(!isClosed, "Mp4Writer is closed."); if (track.format.initializationData.isEmpty() && track.parsedCsd == null) { if (Objects.equals(track.format.sampleMimeType, MimeTypes.VIDEO_AV1)) { track.parsedCsd = createAv1CodecConfigurationRecord(byteBuffer.duplicate()); @@ -174,6 +179,7 @@ public void writeSampleData(Track track, ByteBuffer byteBuffer, BufferInfo buffe boolean primaryTrackSampleWritten = tracks.contains(track); long currentSampleTimestampUs = bufferInfo.presentationTimeUs; if (primaryTrackSampleWritten + && hasWrittenSamples.get() && canWriteMoovAtStart && (currentSampleTimestampUs - lastMoovWrittenAtSampleTimestampUs >= MOOV_BOX_UPDATE_INTERVAL_US)) { @@ -186,10 +192,13 @@ public void writeSampleData(Track track, ByteBuffer byteBuffer, BufferInfo buffe /** * Writes all the pending samples and the final moov box to the output {@link FileChannel}. * - *

      This should be done before closing the file. The output {@link FileChannel} can be closed - * after calling this method. + *

      This method can be called only once, before closing the file. The {@link Mp4Writer} cannot + * be used anymore once this method is called. The output {@link FileChannel} can be closed after + * calling this method. */ public void finishWritingSamplesAndFinalizeMoovBox() throws IOException { + checkState(!isClosed, "Mp4Writer is closed."); + isClosed = true; for (int i = 0; i < tracks.size(); i++) { writePendingTrackSamples(tracks.get(i)); } @@ -230,7 +239,7 @@ private void writeAxteBox() throws IOException { private ByteBuffer getAxteBox() { // The axte box will have one ftyp and one moov box. - ByteBuffer ftypBox = Boxes.ftyp(); + ByteBuffer ftypBox = Boxes.ftyp(MuxerUtil.getFtypCompatibleBrands(auxiliaryTracks)); MetadataCollector auxiliaryTracksMetadataCollector = new MetadataCollector(); populateAuxiliaryTracksMetadata( auxiliaryTracksMetadataCollector, @@ -313,7 +322,7 @@ public void finalizeMoovBox() throws IOException { private void writeHeader() throws IOException { muxerOutput.setPosition(0L); - muxerOutput.write(Boxes.ftyp()); + muxerOutput.write(Boxes.ftyp(MuxerUtil.getFtypCompatibleBrands(tracks))); if (freeSpaceAfterFtypInBytes > 0) { muxerOutput.write( @@ -436,7 +445,6 @@ private boolean maybeWritePendingTrackSamples(List tracks) throws IOExcep boolean newSamplesWritten = false; for (int i = 0; i < tracks.size(); i++) { Track track = tracks.get(i); - // TODO: b/270583563 - Check if we need to consider the global timestamp instead. if (track.pendingSamplesBufferInfo.size() > 2) { BufferInfo firstSampleInfo = checkNotNull(track.pendingSamplesBufferInfo.peekFirst()); BufferInfo lastSampleInfo = checkNotNull(track.pendingSamplesBufferInfo.peekLast()); diff --git a/libraries/muxer/src/main/java/androidx/media3/muxer/Muxer.java b/libraries/muxer/src/main/java/androidx/media3/muxer/Muxer.java index f7703ec01be..31e98c7dbae 100644 --- a/libraries/muxer/src/main/java/androidx/media3/muxer/Muxer.java +++ b/libraries/muxer/src/main/java/androidx/media3/muxer/Muxer.java @@ -15,12 +15,14 @@ */ package androidx.media3.muxer; +import android.os.ParcelFileDescriptor; import androidx.media3.common.C; import androidx.media3.common.Format; import androidx.media3.common.Metadata; import androidx.media3.common.MimeTypes; import androidx.media3.common.util.UnstableApi; import com.google.common.collect.ImmutableList; +import java.io.IOException; import java.nio.ByteBuffer; /** A muxer for producing media container files. */ @@ -36,6 +38,37 @@ interface Factory { */ Muxer create(String path) throws MuxerException; + /** + * Returns a new {@link Muxer}. + * + *

      The caller transfers ownership of the {@link ParcelFileDescriptor} to the {@link Factory}. + * If this method returns successfully, the created {@link Muxer} is responsible for closing the + * descriptor when {@link Muxer#close()} is called. If an error occurs and a {@link + * MuxerException} is thrown, the descriptor is closed before the exception is thrown. + * + *

      Whether the provided {@link ParcelFileDescriptor} must be seekable depends on the + * container format and {@link Muxer} implementation. For example, standard MP4 muxers require + * seekable descriptors (such as those opened with {@code "rwt"} or {@code "rw"}), whereas + * streaming muxers (such as fragmented MP4 muxers) also accept non-seekable streams like pipes. + * + * @param pfd the {@link ParcelFileDescriptor} to write output to + * @throws MuxerException if an error occurs opening the output for writing or if the descriptor + * does not satisfy the seekability requirements of the muxer + */ + default Muxer create(ParcelFileDescriptor pfd) throws MuxerException { + MuxerException exception = + new MuxerException( + "ParcelFileDescriptor output is not supported", new UnsupportedOperationException()); + if (pfd != null) { + try { + pfd.close(); + } catch (IOException e) { + exception.addSuppressed(e); + } + } + throw exception; + } + /** * Returns the supported sample {@linkplain MimeTypes MIME types} for the given {@link * C.TrackType}. diff --git a/libraries/muxer/src/main/java/androidx/media3/muxer/MuxerUtil.java b/libraries/muxer/src/main/java/androidx/media3/muxer/MuxerUtil.java index 0c8882e63fb..c5152d0b7a4 100644 --- a/libraries/muxer/src/main/java/androidx/media3/muxer/MuxerUtil.java +++ b/libraries/muxer/src/main/java/androidx/media3/muxer/MuxerUtil.java @@ -34,6 +34,8 @@ import androidx.media3.container.Mp4OrientationData; import androidx.media3.container.Mp4TimestampData; import androidx.media3.container.XmpData; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableSet; import com.google.common.primitives.Longs; import java.io.FileInputStream; import java.io.IOException; @@ -43,6 +45,7 @@ import java.nio.channels.WritableByteChannel; import java.util.List; import java.util.Locale; +import java.util.Objects; /** Utility methods for muxer. */ @UnstableApi @@ -147,6 +150,32 @@ public static void createMotionPhotoFromJpegImageAndBmffVideo( || format.auxiliaryTrackType == C.AUXILIARY_TRACK_TYPE_DEPTH_METADATA); } + /** Returns the additional ftyp compatible brands that should be declared for the given tracks. */ + /* package */ static ImmutableList getFtypCompatibleBrands(List tracks) { + ImmutableSet.Builder compatibleBrands = new ImmutableSet.Builder<>(); + for (int i = 0; i < tracks.size(); i++) { + if (isDolbyTrack(tracks.get(i).format)) { + compatibleBrands.add("dby1"); + } + } + return compatibleBrands.build().asList(); + } + + /** + * Returns whether the given {@linkplain Format track format} is a Dolby format (Dolby Vision + * video, or Dolby AC-3, EAC3, EAC3-JOC, AC-4 or TrueHD audio) that requires the {@code dby1} + * compatible brand in the ftyp box. + */ + private static boolean isDolbyTrack(Format format) { + String sampleMimeType = format.sampleMimeType; + return Objects.equals(sampleMimeType, MimeTypes.VIDEO_DOLBY_VISION) + || Objects.equals(sampleMimeType, MimeTypes.AUDIO_AC3) + || Objects.equals(sampleMimeType, MimeTypes.AUDIO_AC4) + || Objects.equals(sampleMimeType, MimeTypes.AUDIO_E_AC3) + || Objects.equals(sampleMimeType, MimeTypes.AUDIO_E_AC3_JOC) + || Objects.equals(sampleMimeType, MimeTypes.AUDIO_TRUEHD); + } + /** Returns a {@link MdtaMetadataEntry} for the auxiliary tracks offset metadata. */ /* package */ static MdtaMetadataEntry getAuxiliaryTracksOffsetMetadata(long offset) { return new MdtaMetadataEntry( diff --git a/libraries/muxer/src/main/java/androidx/media3/muxer/SeekableMuxerOutput.java b/libraries/muxer/src/main/java/androidx/media3/muxer/SeekableMuxerOutput.java index bd5abf64ded..34cf7e7c777 100644 --- a/libraries/muxer/src/main/java/androidx/media3/muxer/SeekableMuxerOutput.java +++ b/libraries/muxer/src/main/java/androidx/media3/muxer/SeekableMuxerOutput.java @@ -15,6 +15,7 @@ */ package androidx.media3.muxer; +import android.os.ParcelFileDescriptor; import androidx.media3.common.util.UnstableApi; import java.io.FileNotFoundException; import java.io.FileOutputStream; @@ -30,6 +31,20 @@ static SeekableMuxerOutput of(FileOutputStream fileOutputStream) { return new FileOutputStreamSeekableMuxerOutput(fileOutputStream); } + /** + * Creates a {@link SeekableMuxerOutput} from the given {@link ParcelFileDescriptor}. + * + *

      The caller transfers ownership of the {@link ParcelFileDescriptor} to the returned {@link + * SeekableMuxerOutput}. The descriptor is automatically closed when {@link + * SeekableMuxerOutput#close()} is called. + * + * @param pfd the {@link ParcelFileDescriptor} to write output to + */ + static SeekableMuxerOutput of(ParcelFileDescriptor pfd) { + return new FileOutputStreamSeekableMuxerOutput( + new ParcelFileDescriptor.AutoCloseOutputStream(pfd)); + } + /** Creates a {@link SeekableMuxerOutput} from the given file path. */ static SeekableMuxerOutput of(String filePath) throws FileNotFoundException { return new FileOutputStreamSeekableMuxerOutput(new FileOutputStream(filePath)); diff --git a/libraries/muxer/src/main/java/androidx/media3/muxer/Track.java b/libraries/muxer/src/main/java/androidx/media3/muxer/Track.java index 257f53684e2..882004c60b8 100644 --- a/libraries/muxer/src/main/java/androidx/media3/muxer/Track.java +++ b/libraries/muxer/src/main/java/androidx/media3/muxer/Track.java @@ -38,6 +38,7 @@ public final List writtenSamples; public final List writtenChunkOffsets; public final List writtenChunkSampleCounts; + public final List tfraEntries; public final Deque pendingSamplesBufferInfo; public final Deque pendingSamplesByteBuffer; // Map from the reference type (e.g. "cdsc") to the list of referenced track ids. @@ -69,6 +70,7 @@ public Track(int trackId, Format format, int sortKey, boolean sampleCopyEnabled) writtenSamples = new ArrayList<>(); writtenChunkOffsets = new ArrayList<>(); writtenChunkSampleCounts = new ArrayList<>(); + tfraEntries = new ArrayList<>(); pendingSamplesBufferInfo = new ArrayDeque<>(); pendingSamplesByteBuffer = new ArrayDeque<>(); trackReferences = new HashMap<>(); diff --git a/libraries/muxer/src/main/java/androidx/media3/muxer/WebmConstants.java b/libraries/muxer/src/main/java/androidx/media3/muxer/WebmConstants.java index ab8a1891566..2f7461e42d4 100644 --- a/libraries/muxer/src/main/java/androidx/media3/muxer/WebmConstants.java +++ b/libraries/muxer/src/main/java/androidx/media3/muxer/WebmConstants.java @@ -40,6 +40,14 @@ */ public static final long MKV_UNKNOWN_LENGTH = 0x01ffffffffffffffL; + /** + * The standard ISO 639-2 / BCP 47 undetermined language code ("und"). + * + *

      The Matroska + * Specification defines this code as a fallback when no language is specified. + */ + public static final String DEFAULT_TRACK_LANGUAGE = "und"; + /** * EBML element IDs are in form of VINT and are defined in Matroska specifications. diff --git a/libraries/muxer/src/main/java/androidx/media3/muxer/WebmElements.java b/libraries/muxer/src/main/java/androidx/media3/muxer/WebmElements.java index f0c66c6f213..ef485a11a3f 100644 --- a/libraries/muxer/src/main/java/androidx/media3/muxer/WebmElements.java +++ b/libraries/muxer/src/main/java/androidx/media3/muxer/WebmElements.java @@ -295,7 +295,9 @@ private static List getCommonTrackEntry( trackEntry.add(createUnsignedIntElement(MkvEbmlElement.TRACK_NUMBER, trackNumber)); trackEntry.add(createUnsignedIntElement(MkvEbmlElement.TRACK_UID, uid)); trackEntry.add(createUnsignedIntElement(MkvEbmlElement.FLAG_LACING, /* false */ 0)); - trackEntry.add(createStringElement(MkvEbmlElement.LANGUAGE, checkNotNull(format.language))); + String language = + format.language != null ? format.language : WebmConstants.DEFAULT_TRACK_LANGUAGE; + trackEntry.add(createStringElement(MkvEbmlElement.LANGUAGE, language)); trackEntry.add( createStringElement( MkvEbmlElement.CODEC_ID, getCodecId(checkNotNull(format.sampleMimeType)))); diff --git a/libraries/muxer/src/main/java/androidx/media3/muxer/WebmWriter.java b/libraries/muxer/src/main/java/androidx/media3/muxer/WebmWriter.java index 593c1a80798..7a954ac6673 100644 --- a/libraries/muxer/src/main/java/androidx/media3/muxer/WebmWriter.java +++ b/libraries/muxer/src/main/java/androidx/media3/muxer/WebmWriter.java @@ -114,6 +114,7 @@ // sample. private final SparseArray prevPresentationTimeOfTrack; + private boolean isClosed; private boolean writtenSegmentHeader; private long trackElementStart; private long infoElementStart; @@ -145,6 +146,7 @@ * @return A unique {@link Track}. It should be used in {@link #writeSampleData}. */ public Track addTrack(int trackId, Format format) { + checkState(!isClosed, "WebmWriter is closed."); // Tracks can only be added before writing any samples. checkArgument(!writtenSegmentHeader); @@ -165,6 +167,7 @@ public Track addTrack(int trackId, Format format) { */ public void writeSampleData(Track track, ByteBuffer byteBuffer, BufferInfo bufferInfo) throws IOException { + checkState(!isClosed, "WebmWriter is closed."); if (!writtenSegmentHeader) { writeSegmentHeader(); writtenSegmentHeader = true; @@ -302,11 +305,17 @@ private void createCluster() throws IOException { /** * Finalizes the output and closes WebmWriter. * - *

      The WebmWriter cannot be used anymore once this method returns. + *

      This method can be called only once. The {@link WebmWriter} cannot be used anymore once this + * method returns. * * @throws IOException If the WebmWriter fails to finish writing the output. */ public void close() throws IOException { + checkState(!isClosed, "WebmWriter is closed."); + isClosed = true; + if (!writtenSegmentHeader) { + return; + } // Write the last cluster. createCluster(); // Write Cues master element. diff --git a/libraries/muxer/src/test/java/androidx/media3/muxer/BoxesTest.java b/libraries/muxer/src/test/java/androidx/media3/muxer/BoxesTest.java index 1f4e414f19f..e896ccd4625 100644 --- a/libraries/muxer/src/test/java/androidx/media3/muxer/BoxesTest.java +++ b/libraries/muxer/src/test/java/androidx/media3/muxer/BoxesTest.java @@ -15,12 +15,18 @@ */ package androidx.media3.muxer; +import static androidx.media3.common.util.Util.getPcmFormat; +import static androidx.media3.common.util.Util.getPcmFrameSize; +import static androidx.media3.common.util.Util.isFloatPcmEncoding; +import static androidx.media3.common.util.Util.isPcmEncodingBigEndian; import static androidx.media3.muxer.Mp4Muxer.LAST_SAMPLE_DURATION_BEHAVIOR_SET_FROM_END_OF_STREAM_BUFFER_OR_DUPLICATE_PREVIOUS; import static androidx.media3.muxer.Mp4Muxer.LAST_SAMPLE_DURATION_BEHAVIOR_SET_TO_ZERO; import static androidx.media3.muxer.MuxerTestUtil.FAKE_AUDIO_FORMAT; import static androidx.media3.muxer.MuxerTestUtil.FAKE_CSD_0; import static androidx.media3.muxer.MuxerTestUtil.FAKE_VIDEO_FORMAT; +import static androidx.media3.test.utils.TestUtil.createByteArray; import static com.google.common.truth.Truth.assertThat; +import static java.nio.charset.StandardCharsets.UTF_8; import static java.util.Arrays.asList; import static org.junit.Assert.assertThrows; @@ -30,20 +36,24 @@ import androidx.media3.common.ColorInfo; import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; +import androidx.media3.common.util.ParsableByteArray; import androidx.media3.common.util.Util; import androidx.media3.container.MdtaMetadataEntry; import androidx.media3.container.Mp4LocationData; +import androidx.media3.container.XmpData; import androidx.media3.muxer.FragmentedMp4Writer.SampleMetadata; import androidx.media3.test.utils.DumpFileAsserts; import androidx.media3.test.utils.DumpableMp4Box; import androidx.media3.test.utils.TestUtil; import androidx.test.core.app.ApplicationProvider; -import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.collect.ImmutableList; import com.google.common.io.BaseEncoding; import com.google.common.truth.Correspondence; +import com.google.testing.junit.testparameterinjector.TestParameter; +import com.google.testing.junit.testparameterinjector.TestParameterValuesProvider; import java.io.IOException; import java.nio.ByteBuffer; +import java.nio.ByteOrder; import java.util.ArrayList; import java.util.HashMap; import java.util.LinkedHashMap; @@ -52,9 +62,10 @@ import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; +import org.robolectric.RobolectricTestParameterInjector; /** Unit tests for {@link Boxes}. */ -@RunWith(AndroidJUnit4.class) +@RunWith(RobolectricTestParameterInjector.class) public class BoxesTest { // A typical timescale is ~90_000. We're using 100_000 here to simplify calculations. // This makes one time unit equal to 10 microseconds. @@ -232,6 +243,17 @@ public void createUdtaBox_matchesExpected() throws IOException { context, dumpableBox, MuxerTestUtil.getExpectedMp4DumpFilePath("udta_box")); } + @Test + public void createUdtaBox_withAltitude_containsAltitudeString() { + Mp4LocationData mp4Location = new Mp4LocationData(33.0f, -120f, 150.5f); + + ByteBuffer udtaBox = Boxes.udta(mp4Location); + + String boxContentsString = + new String(udtaBox.array(), udtaBox.arrayOffset(), udtaBox.remaining(), UTF_8); + assertThat(boxContentsString).contains("+33.0000-120.0000+150.5000/"); + } + @Test public void createKeysBox_matchesExpected() throws Exception { List metadataEntries = new ArrayList<>(); @@ -279,7 +301,7 @@ public void createUuidBox_forXmpData_matchesExpected() throws Exception { ByteBuffer xmpData = ByteBuffer.wrap(TestUtil.getByteArray(context, "media/xmp/sample_datetime_xmp.xmp")); - ByteBuffer xmpUuidBox = Boxes.uuid(Boxes.XMP_UUID, xmpData); + ByteBuffer xmpUuidBox = Boxes.uuid(XmpData.XMP_UUID, xmpData); DumpableMp4Box dumpableBox = new DumpableMp4Box(xmpUuidBox); DumpFileAsserts.assertOutput( @@ -290,7 +312,7 @@ public void createUuidBox_forXmpData_matchesExpected() throws Exception { public void createuuidBox_withEmptyXmpData_throws() { ByteBuffer xmpData = ByteBuffer.allocate(0); - assertThrows(IllegalArgumentException.class, () -> Boxes.uuid(Boxes.XMP_UUID, xmpData)); + assertThrows(IllegalArgumentException.class, () -> Boxes.uuid(XmpData.XMP_UUID, xmpData)); } @Test @@ -380,6 +402,39 @@ public void createAudioSampleEntryBox_forOpus_matchesExpected() throws Exception MuxerTestUtil.getExpectedMp4DumpFilePath("audio_sample_entry_box_opus")); } + @Test + public void createAudioSampleEntryBox_forOpus_convertsOpusHeadToBigEndianDops() { + byte[] opusHead = + ByteBuffer.allocate(19) + .order(ByteOrder.LITTLE_ENDIAN) + .put("OpusHead".getBytes(UTF_8)) + .put((byte) 1) // Version + .put((byte) 2) // OutputChannelCount + .putShort((short) 312) // PreSkip + .putInt(48_000) // InputSampleRate + .putShort((short) 258) // OutputGain (non-palindromic 0x0102) + .put((byte) 0) // ChannelMappingFamily + .array(); + Format format = + FAKE_AUDIO_FORMAT + .buildUpon() + .setSampleMimeType(MimeTypes.AUDIO_OPUS) + .setInitializationData(ImmutableList.of(opusHead)) + .build(); + + ByteBuffer audioSampleEntryBox = Boxes.audioSampleEntry(format); + + // dOps box payload starts at offset 44 (8 Opus box header + 28 AudioSampleEntry + 8 dOps box + // header). Verify multi-byte fields are big-endian per Opus-in-ISOBMFF Section 4.3.2. + assertThat(audioSampleEntryBox.get(/* index= */ 44)).isEqualTo((byte) 0); // Version + assertThat(audioSampleEntryBox.get(/* index= */ 45)).isEqualTo((byte) 2); // OutputChannelCount + assertThat(audioSampleEntryBox.getShort(/* index= */ 46)).isEqualTo((short) 312); // PreSkip + assertThat(audioSampleEntryBox.getInt(/* index= */ 48)).isEqualTo(48_000); // InputSampleRate + assertThat(audioSampleEntryBox.getShort(/* index= */ 52)).isEqualTo((short) 258); // OutputGain + assertThat(audioSampleEntryBox.get(/* index= */ 54)) + .isEqualTo((byte) 0); // ChannelMappingFamily + } + @Test public void createAudioSampleEntryBox_forVorbis_matchesExpected() throws Exception { Format format = @@ -403,6 +458,48 @@ public void createAudioSampleEntryBox_forVorbis_matchesExpected() throws Excepti MuxerTestUtil.getExpectedMp4DumpFilePath("audio_sample_entry_box_vorbis")); } + @Test + public void createAudioSampleEntryBox_withPcmCBox_matchesExpected( + @TestParameter(valuesProvider = PcmCEncodingValuesProvider.class) @C.PcmEncoding + int pcmEncoding) { + Format format = getPcmFormat(pcmEncoding, /* channels= */ 2, /* sampleRate= */ 48_000); + + ByteBuffer audioSampleEntryBox = Boxes.audioSampleEntry(format); + ParsableByteArray parsableByteArray = + new ParsableByteArray(createByteArray(audioSampleEntryBox)); + + assertThat(parsableByteArray.readInt()).isEqualTo(50); // Box size + String expectedFourcc = isFloatPcmEncoding(pcmEncoding) ? "fpcm" : "ipcm"; + assertThat(parsableByteArray.readString(4)).isEqualTo(expectedFourcc); + parsableByteArray.skipBytes(6); // reserved + assertThat(parsableByteArray.readUnsignedShort()) + .isEqualTo(1); // data reference index (points to first dref table entry) + parsableByteArray.skipBytes(8); // reserved + assertThat(parsableByteArray.readUnsignedShort()).isEqualTo(2); // channel count + // In AudioSampleEntry v0, sample_size is fixed to 16 as a template value (ISO/IEC 14496-12). + // The actual PCM bit depth for fpcm/ipcm is specified in the pcmC box. + assertThat(parsableByteArray.readUnsignedShort()).isEqualTo(16); // sample size + parsableByteArray.skipBytes(4); // predefined + reserved + assertThat(parsableByteArray.readInt()) + .isEqualTo(48_000 << 16); // sample_rate in 16.16 fixed-point format + // pcmC box + assertThat(parsableByteArray.readInt()).isEqualTo(14); // pcmC size + assertThat(parsableByteArray.readString(4)).isEqualTo("pcmC"); + assertThat(parsableByteArray.readInt()).isEqualTo(0); // version + flags + byte expectedFlags = isPcmEncodingBigEndian(pcmEncoding) ? (byte) 0 : (byte) 1; + assertThat(parsableByteArray.readUnsignedByte()).isEqualTo(expectedFlags); + int expectedBitDepth = getPcmFrameSize(pcmEncoding, /* channelCount= */ 1) * 8; + assertThat(parsableByteArray.readUnsignedByte()).isEqualTo(expectedBitDepth); + assertThat(parsableByteArray.bytesLeft()).isEqualTo(0); + } + + @Test + public void createAudioSampleEntryBox_with8BitPcm_throws() { + Format format = getPcmFormat(C.ENCODING_PCM_8BIT, /* channels= */ 2, /* sampleRate= */ 48_000); + + assertThrows(IllegalArgumentException.class, () -> Boxes.audioSampleEntry(format)); + } + @Test public void createAudioSampleEntryBox_withUnknownAudioFormat_throws() { // The audio format contains an unknown MIME type. @@ -857,6 +954,39 @@ public void createStszBox_matchesExpected() throws IOException { context, dumpableBox, MuxerTestUtil.getExpectedMp4DumpFilePath("stsz_box")); } + @Test + public void stsz_withIdenticallySizedSamples_matchesExpected() throws IOException { + List sampleBufferInfos = createBufferInfoListWithSampleSizes(150, 150, 150, 150); + + ByteBuffer stszBox = Boxes.stsz(sampleBufferInfos); + + DumpableMp4Box dumpableBox = new DumpableMp4Box(stszBox); + DumpFileAsserts.assertOutput( + context, + dumpableBox, + MuxerTestUtil.getExpectedMp4DumpFilePath("stsz_box_with_identically_sized_samples")); + } + + @Test + public void stsz_withIdenticallySizedSamples_omitsSampleTableAndSetsHeaderSampleSize() + throws Exception { + List sampleBufferInfos = createBufferInfoListWithSampleSizes(150, 150, 150, 150); + + ByteBuffer stszBox = Boxes.stsz(sampleBufferInfos); + + // Verify total box size is 20 bytes (8-byte header + 12-byte payload, omitting 4x4 entry array) + assertThat(stszBox.remaining()).isEqualTo(20); + + // Parse ISO 14496-12 fields: version/flags (4 bytes), sample_size (4 bytes), sample_count (4 + // bytes) + stszBox.position(12); + int sampleSize = stszBox.getInt(); + int sampleCount = stszBox.getInt(); + + assertThat(sampleSize).isEqualTo(150); + assertThat(sampleCount).isEqualTo(4); + } + @Test public void createStscBox_withDifferentChunks_matchesExpected() throws IOException { ImmutableList chunkSampleCounts = ImmutableList.of(100, 500, 200, 100); @@ -916,13 +1046,44 @@ public void createStssBox_matchesExpected() throws IOException { @Test public void createFtypBox_matchesExpected() throws IOException { - ByteBuffer ftypBox = Boxes.ftyp(); + ByteBuffer ftypBox = Boxes.ftyp(/* additionalCompatibleBrands= */ ImmutableList.of()); DumpableMp4Box dumpableBox = new DumpableMp4Box(ftypBox); DumpFileAsserts.assertOutput( context, dumpableBox, MuxerTestUtil.getExpectedMp4DumpFilePath("ftyp_box")); } + @Test + public void createFtypBox_withInvalidCompatibleBrandLength_throws() { + assertThrows( + IllegalArgumentException.class, + () -> Boxes.ftyp(/* additionalCompatibleBrands= */ ImmutableList.of("dby"))); + } + + @Test + public void createFtypBox_forDolbyVision_containsDby1CompatibleBrand() { + ByteBuffer ftypBox = Boxes.ftyp(/* additionalCompatibleBrands= */ ImmutableList.of("dby1")); + + assertThat(ftypBox.getInt()).isEqualTo(32); + byte[] type = new byte[4]; + ftypBox.get(type); + assertThat(type).isEqualTo(Util.getUtf8Bytes("ftyp")); + assertThat(ftypBox.getInt()).isEqualTo(Util.getIntegerCodeForString("isom")); + assertThat(ftypBox.getInt()).isEqualTo(0x020000); + + List compatibleBrands = new ArrayList<>(); + while (ftypBox.hasRemaining()) { + compatibleBrands.add(ftypBox.getInt()); + } + assertThat(compatibleBrands) + .containsExactly( + Util.getIntegerCodeForString("isom"), + Util.getIntegerCodeForString("iso2"), + Util.getIntegerCodeForString("mp41"), + Util.getIntegerCodeForString("dby1")) + .inOrder(); + } + @Test public void createMfhdBox_matchesExpected() throws IOException { ByteBuffer mfhdBox = Boxes.mfhd(/* sequenceNumber= */ 5); @@ -1061,6 +1222,27 @@ public void createIacbBox_matchesExpected() { assertThat(configObus).isEqualTo(new byte[] {0x11, 0x22, 0x33}); } + @Test + public void createDec3Box_withAudioEAc3Joc_matchesExpectedDec3Box() { + Format format = + FAKE_AUDIO_FORMAT + .buildUpon() + .setSampleMimeType(MimeTypes.AUDIO_E_AC3_JOC) + .setInitializationData(ImmutableList.of(new byte[] {0x00, 0x00, 0x00, 0x03, 0x00})) + .build(); + + ByteBuffer dec3Box = Boxes.codecSpecificBox(format); + + int expectedTotalSize = 4 + 4 + 5; + assertThat(dec3Box.getInt()).isEqualTo(expectedTotalSize); + byte[] type = new byte[4]; + dec3Box.get(type); + assertThat(type).isEqualTo(Util.getUtf8Bytes("dec3")); + byte[] payload = new byte[5]; + dec3Box.get(payload); + assertThat(payload).isEqualTo(new byte[] {0x00, 0x00, 0x00, 0x03, 0x00}); + } + @Test public void createAudioSampleEntryBox_forIamf_matchesExpected() { Format format = @@ -1135,6 +1317,25 @@ public void createTrefBox_withMultipleTrackReferences_matchesExpected() throws I MuxerTestUtil.getExpectedMp4DumpFilePath("tref_box_with_multiple_track_reference")); } + @Test + public void tfdt_withBaseMediaDecodeTime_matchesExpected() { + long baseMediaDecodeTime = 90_000L; + + ByteBuffer tfdtBox = Boxes.tfdt(baseMediaDecodeTime); + + // Box size (8-byte header + 12-byte payload = 20 bytes total). + assertThat(tfdtBox.remaining()).isEqualTo(20); + assertThat(tfdtBox.getInt()).isEqualTo(20); + // Box header type ("tfdt"). + byte[] boxType = new byte[4]; + tfdtBox.get(boxType); + assertThat(new String(boxType, UTF_8)).isEqualTo("tfdt"); + // FullBox version 1 (64-bit decode time support) and flags (0). + assertThat(tfdtBox.getInt()).isEqualTo(0x01000000); + // 64-bit baseMediaDecodeTime payload. + assertThat(tfdtBox.getLong()).isEqualTo(baseMediaDecodeTime); + } + private static List durationsVuToPresentationTimestamps( List durationsVu, int videoUnitTimescale) { List timestampsUs = new ArrayList<>(); @@ -1185,4 +1386,19 @@ private static List createBufferInfoListWithSomeKeyFrames() { return bufferInfoList; } + + private static final class PcmCEncodingValuesProvider extends TestParameterValuesProvider { + @Override + protected ImmutableList provideValues(TestParameterValuesProvider.Context context) { + return ImmutableList.of( + value(C.ENCODING_PCM_FLOAT).withName("ENCODING_PCM_FLOAT"), + value(C.ENCODING_PCM_FLOAT_BIG_ENDIAN).withName("ENCODING_PCM_FLOAT_BIG_ENDIAN"), + value(C.ENCODING_PCM_DOUBLE).withName("ENCODING_PCM_DOUBLE"), + value(C.ENCODING_PCM_DOUBLE_BIG_ENDIAN).withName("ENCODING_PCM_DOUBLE_BIG_ENDIAN"), + value(C.ENCODING_PCM_24BIT).withName("ENCODING_PCM_24BIT"), + value(C.ENCODING_PCM_24BIT_BIG_ENDIAN).withName("ENCODING_PCM_24BIT_BIG_ENDIAN"), + value(C.ENCODING_PCM_32BIT).withName("ENCODING_PCM_32BIT"), + value(C.ENCODING_PCM_32BIT_BIG_ENDIAN).withName("ENCODING_PCM_32BIT_BIG_ENDIAN")); + } + } } diff --git a/libraries/muxer/src/test/java/androidx/media3/muxer/FragmentedMp4MuxerEndToEndParameterizedTest.java b/libraries/muxer/src/test/java/androidx/media3/muxer/FragmentedMp4MuxerEndToEndParameterizedTest.java index 9a617dbf544..da0ce041bed 100644 --- a/libraries/muxer/src/test/java/androidx/media3/muxer/FragmentedMp4MuxerEndToEndParameterizedTest.java +++ b/libraries/muxer/src/test/java/androidx/media3/muxer/FragmentedMp4MuxerEndToEndParameterizedTest.java @@ -62,6 +62,8 @@ public class FragmentedMp4MuxerEndToEndParameterizedTest { // Audio Codecs private static final String AMR_NB_3GP = "3gp/bbb_mono_8kHz_12.2kbps_amrnb.3gp"; private static final String AMR_WB_3GP = "3gp/bbb_mono_16kHz_23.05kbps_amrwb.3gp"; + private static final String E_AC3_MP4 = "mp4/sample_eac3.mp4"; + private static final String E_AC3_JOC_MP4 = "mp4/sample_eac3joc.mp4"; private static final String OPUS_OGG = "ogg/bbb_6ch_8kHz_opus.ogg"; private static final String VORBIS_OGG = "ogg/bbb_1ch_16kHz_q10_vorbis.ogg"; private static final String IAMF_OPUS_MP4 = "mp4/5p1_iamf_opus.mp4"; @@ -89,6 +91,8 @@ public static ImmutableList mediaSamples() { AMR_NB_3GP, IAMF_OPUS_MP4, AMR_WB_3GP, + E_AC3_MP4, + E_AC3_JOC_MP4, OPUS_OGG, VORBIS_OGG, RAW_WAV); diff --git a/libraries/muxer/src/test/java/androidx/media3/muxer/FragmentedMp4MuxerEndToEndTest.java b/libraries/muxer/src/test/java/androidx/media3/muxer/FragmentedMp4MuxerEndToEndTest.java index bbac80f42f0..75f9489d5ac 100644 --- a/libraries/muxer/src/test/java/androidx/media3/muxer/FragmentedMp4MuxerEndToEndTest.java +++ b/libraries/muxer/src/test/java/androidx/media3/muxer/FragmentedMp4MuxerEndToEndTest.java @@ -15,10 +15,24 @@ */ package androidx.media3.muxer; +import static androidx.media3.common.util.Util.getPcmFormat; +import static androidx.media3.common.util.Util.getPcmFrameSize; +import static androidx.media3.common.util.Util.isPcmEncodingBigEndian; +import static androidx.media3.muxer.MuxerTestUtil.FAKE_VIDEO_FORMAT; import static androidx.media3.muxer.MuxerTestUtil.feedInputDataToMuxer; +import static androidx.media3.muxer.MuxerTestUtil.getFakeSampleAndSampleInfo; +import static androidx.media3.test.utils.TestUtil.buildDoubleTestSamples; +import static androidx.media3.test.utils.TestUtil.buildFloatTestSamples; +import static androidx.media3.test.utils.TestUtil.buildTestData; +import static androidx.media3.test.utils.TestUtil.createByteArray; +import static androidx.media3.test.utils.TestUtil.createByteBuffer; import static com.google.common.base.Preconditions.checkNotNull; +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; import android.content.Context; +import android.util.Pair; +import androidx.media3.common.C; import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; import androidx.media3.container.Mp4TimestampData; @@ -27,19 +41,26 @@ import androidx.media3.test.utils.DumpFileAsserts; import androidx.media3.test.utils.DumpableMp4Box; import androidx.media3.test.utils.FakeExtractorOutput; +import androidx.media3.test.utils.FakeTrackOutput; import androidx.media3.test.utils.TestUtil; import androidx.test.core.app.ApplicationProvider; -import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import com.google.testing.junit.testparameterinjector.TestParameter; +import com.google.testing.junit.testparameterinjector.TestParameterValuesProvider; +import java.io.File; import java.io.FileOutputStream; import java.nio.ByteBuffer; +import java.nio.ByteOrder; +import java.util.Random; import org.junit.Rule; import org.junit.Test; import org.junit.rules.TemporaryFolder; import org.junit.runner.RunWith; +import org.robolectric.RobolectricTestParameterInjector; import org.robolectric.annotation.Config; /** End to end instrumentation tests for {@link FragmentedMp4Muxer}. */ -@RunWith(AndroidJUnit4.class) +@RunWith(RobolectricTestParameterInjector.class) public class FragmentedMp4MuxerEndToEndTest { private static final String H264_MP4 = "mp4/sample_no_bframes.mp4"; private static final String H265_HDR10_MP4 = "mp4/hdr10-720p.mp4"; @@ -79,6 +100,76 @@ public void createFragmentedMp4File_fromInputFileSampleData_matchesExpectedBoxSt + "_fragmented_box_structure")); } + @Test + public void createFragmentedMp4File_withDolbyVisionTrack_ftypContainsDby1CompatibleBrand() + throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + Format dolbyVisionFormat = + FAKE_VIDEO_FORMAT + .buildUpon() + .setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION) + .setCodecs("dvav.09.02") + .build(); + Pair sampleAndSampleInfo = + getFakeSampleAndSampleInfo(/* presentationTimeUs= */ 0L, /* isVideo= */ true); + + try (FragmentedMp4Muxer muxer = + new FragmentedMp4Muxer.Builder(new FileOutputStream(outputFilePath).getChannel()).build()) { + int trackId = muxer.addTrack(dolbyVisionFormat); + muxer.writeSampleData(trackId, sampleAndSampleInfo.first, sampleAndSampleInfo.second); + } + + byte[] outputFileBytes = TestUtil.getByteArrayFromFilePath(outputFilePath); + assertThat(MuxerTestUtil.ftypBoxContainsCompatibleBrand(outputFileBytes, "dby1")).isTrue(); + } + + @Test + public void createFragmentedMp4File_withPcmCBox_writesAndExtractsCorrectSamples( + @TestParameter(valuesProvider = PcmCEncodingValuesProvider.class) @C.PcmEncoding + int pcmEncoding) + throws Exception { + File outputFile = temporaryFolder.newFile(); + Format pcmFormat = getPcmFormat(pcmEncoding, /* channels= */ 1, /* sampleRate= */ 48_000); + ByteBuffer sampleBuffer = getTestSamplesForEncoding(pcmEncoding); + byte[] sampleData = createByteArray(sampleBuffer.duplicate()); + BufferInfo bufferInfo = + new BufferInfo(/* presentationTimeUs= */ 0L, sampleData.length, C.BUFFER_FLAG_KEY_FRAME); + + try (FragmentedMp4Muxer muxer = + new FragmentedMp4Muxer.Builder(new FileOutputStream(outputFile).getChannel()).build()) { + int trackId = muxer.addTrack(pcmFormat); + muxer.writeSampleData(trackId, sampleBuffer, bufferInfo); + } + + FakeExtractorOutput extractorOutput = + TestUtil.extractAllSamplesFromFilePath( + new FragmentedMp4Extractor(new DefaultSubtitleParserFactory()), outputFile.getPath()); + FakeTrackOutput trackOutput = extractorOutput.trackOutputs.valueAt(0); + Format extractedFormat = checkNotNull(trackOutput.lastFormat); + assertThat(extractedFormat.sampleMimeType).isEqualTo(MimeTypes.AUDIO_RAW); + assertThat(extractedFormat.pcmEncoding).isEqualTo(pcmEncoding); + assertThat(trackOutput.getSampleData(/* index= */ 0)).isEqualTo(sampleData); + } + + @Test + public void createFragmentedMp4File_with8BitPcm_throws() throws Exception { + File outputFile = temporaryFolder.newFile(); + Format pcmFormat = + getPcmFormat(C.ENCODING_PCM_8BIT, /* channels= */ 1, /* sampleRate= */ 48_000); + ByteBuffer sampleBuffer = ByteBuffer.allocate(100); + BufferInfo bufferInfo = + new BufferInfo(/* presentationTimeUs= */ 0L, 100, C.BUFFER_FLAG_KEY_FRAME); + + try (FragmentedMp4Muxer muxer = + new FragmentedMp4Muxer.Builder(new FileOutputStream(outputFile).getChannel()).build()) { + int trackId = muxer.addTrack(pcmFormat); + // Header boxes and format validation are deferred until the first sample is written. + assertThrows( + IllegalArgumentException.class, + () -> muxer.writeSampleData(trackId, sampleBuffer, bufferInfo)); + } + } + @Test public void createFragmentedMp4File_fromAudioOnlyInputFile_writesExpectedFragments() throws Exception { @@ -243,4 +334,410 @@ public void createFragmentedMp4File_withSomeUnknownTrack_writesAsTextMetadataTra dumpableMp4Box, MuxerTestUtil.getExpectedMp4DumpFilePath("fragmented_mp4_with_unknown_track.mp4")); } + + @Test + public void write_singleTrack_extractorSeekMapIsSeekable() throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + + try (FragmentedMp4Muxer fragmentedMp4Muxer = + new FragmentedMp4Muxer.Builder(new FileOutputStream(outputFilePath).getChannel()).build()) { + feedInputDataToMuxer(context, fragmentedMp4Muxer, MEDIA_ASSET_DIRECTORY + H264_MP4); + } + + FragmentedMp4Extractor extractor = + new FragmentedMp4Extractor( + new DefaultSubtitleParserFactory(), FragmentedMp4Extractor.FLAG_READ_MFRA_FOR_SEEK_MAP); + + FakeExtractorOutput fakeExtractorOutput = + TestUtil.extractAllSamplesFromFilePath(extractor, outputFilePath); + + assertThat(fakeExtractorOutput.seekMap.isSeekable()).isTrue(); + assertThat(fakeExtractorOutput.seekMap.getSeekPoints(/* timeUs= */ 0).first.position) + .isGreaterThan(0); + assertThat(fakeExtractorOutput.seekMap.getSeekPoints(/* timeUs= */ 500_000L).first.position) + .isGreaterThan(0); + } + + @Test + public void write_fragmentedMp4_extractorParsesSampleTimestampsCorrectly() throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + + try (FragmentedMp4Muxer fragmentedMp4Muxer = + new FragmentedMp4Muxer.Builder(new FileOutputStream(outputFilePath).getChannel()).build()) { + feedInputDataToMuxer(context, fragmentedMp4Muxer, MEDIA_ASSET_DIRECTORY + H264_MP4); + } + + FragmentedMp4Extractor extractor = + new FragmentedMp4Extractor(new DefaultSubtitleParserFactory()); + FakeExtractorOutput fakeExtractorOutput = + TestUtil.extractAllSamplesFromFilePath(extractor, outputFilePath); + + // 43990L is the initial sample presentation timestamp (in microseconds) extracted from + // sample_no_bframes.mp4 after FragmentedMp4Extractor parses the written tfdt box. + FakeTrackOutput trackOutput = fakeExtractorOutput.trackOutputs.get(0); + assertThat(trackOutput.getSampleTimeUs(0)).isEqualTo(43990L); + } + + /** + * A fragmented MP4 file states its timeline twice: as the {@code tfdt} of each fragment, and as + * the sum of the {@code trun} sample durations. The two must agree. + * + *

      Frames within a group of pictures are 33367us apart, but only 32000us separate the last + * frame of a group from the next key frame, so the last sample duration of a fragment cannot be + * copied from the preceding one. + */ + @Test + public void writeSampleData_withShorterFrameDurationBeforeKeyFrame_fragmentsAreContiguous() + throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + long frameDurationUs = 33_367L; + long frameDurationBeforeKeyFrameUs = 32_000L; + int framesPerGroupOfPictures = 5; + int groupOfPicturesCount = 3; + + try (FragmentedMp4Muxer muxer = + new FragmentedMp4Muxer.Builder(new FileOutputStream(outputFilePath).getChannel()) + .setFragmentDurationMs(100) + .build()) { + int trackId = muxer.addTrack(FAKE_VIDEO_FORMAT); + long presentationTimeUs = 0L; + for (int i = 0; i < groupOfPicturesCount; i++) { + for (int j = 0; j < framesPerGroupOfPictures; j++) { + ByteBuffer sampleData = + getFakeSampleAndSampleInfo(presentationTimeUs, /* isVideo= */ true).first; + muxer.writeSampleData( + trackId, + sampleData, + new BufferInfo( + presentationTimeUs, + sampleData.remaining(), + /* flags= */ j == 0 ? C.BUFFER_FLAG_KEY_FRAME : 0)); + presentationTimeUs += + j == framesPerGroupOfPictures - 1 ? frameDurationBeforeKeyFrameUs : frameDurationUs; + } + } + } + + FakeTrackOutput fromTfdt = + TestUtil.extractAllSamplesFromFilePath( + new FragmentedMp4Extractor(new DefaultSubtitleParserFactory()), outputFilePath) + .trackOutputs + .get(0); + FakeTrackOutput fromDurations = + TestUtil.extractAllSamplesFromFilePath( + new FragmentedMp4Extractor( + new DefaultSubtitleParserFactory(), + FragmentedMp4Extractor.FLAG_WORKAROUND_IGNORE_TFDT_BOX), + outputFilePath) + .trackOutputs + .get(0); + ImmutableList.Builder tfdtSampleTimesUs = ImmutableList.builder(); + for (int i = 0; i < fromTfdt.getSampleCount(); i++) { + tfdtSampleTimesUs.add(fromTfdt.getSampleTimeUs(i)); + } + ImmutableList.Builder durationSampleTimesUs = ImmutableList.builder(); + for (int i = 0; i < fromDurations.getSampleCount(); i++) { + durationSampleTimesUs.add(fromDurations.getSampleTimeUs(i)); + } + assertThat(durationSampleTimesUs.build()).isEqualTo(tfdtSampleTimesUs.build()); + } + + @Test + public void write_negativeInitialSampleTimestamp_clampsBaseMediaDecodeTimeToZero() + throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + + try (FragmentedMp4Muxer fragmentedMp4Muxer = + new FragmentedMp4Muxer.Builder(new FileOutputStream(outputFilePath).getChannel()).build()) { + int trackId = fragmentedMp4Muxer.addTrack(MuxerTestUtil.FAKE_AUDIO_FORMAT); + ByteBuffer sampleData = ByteBuffer.allocate(10); + fragmentedMp4Muxer.writeSampleData( + trackId, + sampleData, + new BufferInfo( + /* presentationTimeUs= */ -1000L, + /* size= */ sampleData.remaining(), + /* flags= */ C.BUFFER_FLAG_KEY_FRAME)); + } + + FragmentedMp4Extractor extractor = + new FragmentedMp4Extractor(new DefaultSubtitleParserFactory()); + FakeExtractorOutput fakeExtractorOutput = + TestUtil.extractAllSamplesFromFilePath(extractor, outputFilePath); + + FakeTrackOutput trackOutput = fakeExtractorOutput.trackOutputs.get(0); + assertThat(trackOutput.getSampleTimeUs(0)).isEqualTo(0L); + } + + @Test + public void write_multiTrack_extractorSeekMapIsSeekable() throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + + try (FragmentedMp4Muxer fragmentedMp4Muxer = + new FragmentedMp4Muxer.Builder(new FileOutputStream(outputFilePath).getChannel()).build()) { + feedInputDataToMuxer(context, fragmentedMp4Muxer, MEDIA_ASSET_DIRECTORY + H264_MP4); + feedInputDataToMuxer(context, fragmentedMp4Muxer, MEDIA_ASSET_DIRECTORY + AUDIO_ONLY_MP4); + } + + FragmentedMp4Extractor extractor = + new FragmentedMp4Extractor( + new DefaultSubtitleParserFactory(), FragmentedMp4Extractor.FLAG_READ_MFRA_FOR_SEEK_MAP); + + FakeExtractorOutput fakeExtractorOutput = + TestUtil.extractAllSamplesFromFilePath(extractor, outputFilePath); + + assertThat(fakeExtractorOutput.numberOfTracks).isEqualTo(2); + assertThat(fakeExtractorOutput.seekMap.isSeekable()).isTrue(); + assertThat(fakeExtractorOutput.seekMap.getSeekPoints(/* timeUs= */ 0).first.position) + .isGreaterThan(0); + assertThat(fakeExtractorOutput.seekMap.getSeekPoints(/* timeUs= */ 500_000L).first.position) + .isGreaterThan(0); + } + + // Non-video/metadata tracks treat all samples as random access points (!MimeTypes.isVideo(...) == + // true), + // which records tfra entries into the mfra box and allows the extractor to build a seekable + // SeekMap. + @Test + public void write_withEmptyTrackInFragment_correctlySetsActiveTrafIndex() throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + byte[] sampleData = new byte[] {0x00, 0x00, 0x00, 0x01, 0x65, 0x01}; + Format track1Format = new Format.Builder().setSampleMimeType("meta1").build(); + Format track2Format = new Format.Builder().setSampleMimeType("meta2").build(); + Format track3Format = new Format.Builder().setSampleMimeType("meta3").build(); + + try (FragmentedMp4Muxer muxer = + new FragmentedMp4Muxer.Builder(new FileOutputStream(outputFilePath).getChannel()).build()) { + int track1Id = muxer.addTrack(track1Format); + int unusedTrack2Id = muxer.addTrack(track2Format); + int track3Id = muxer.addTrack(track3Format); + + // Track 1 gets a sample. + muxer.writeSampleData( + track1Id, + ByteBuffer.wrap(sampleData), + new BufferInfo( + /* presentationTimeUs= */ 0L, /* size= */ sampleData.length, /* flags= */ 0)); + + // Track 2 receives NO samples in this fragment. + + // Track 3 gets a sample. + muxer.writeSampleData( + track3Id, + ByteBuffer.wrap(sampleData), + new BufferInfo( + /* presentationTimeUs= */ 0L, /* size= */ sampleData.length, /* flags= */ 0)); + } + + FragmentedMp4Extractor extractor = + new FragmentedMp4Extractor( + new DefaultSubtitleParserFactory(), FragmentedMp4Extractor.FLAG_READ_MFRA_FOR_SEEK_MAP); + + FakeExtractorOutput fakeExtractorOutput = + TestUtil.extractAllSamplesFromFilePath(extractor, outputFilePath); + + assertThat(fakeExtractorOutput.seekMap.isSeekable()).isTrue(); + assertThat(fakeExtractorOutput.seekMap.getSeekPoints(/* timeUs= */ 0).first.position) + .isGreaterThan(0); + } + + @Test + public void createMp4File_withAudioEAc3Joc_createsDec3Box() throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + byte[] expectedDec3Payload = new byte[] {0x00, 0x00, 0x00, 0x03, 0x00}; + Format eac3JocFormat = + new Format.Builder() + .setSampleMimeType(MimeTypes.AUDIO_E_AC3_JOC) + .setInitializationData(ImmutableList.of(expectedDec3Payload)) + .build(); + byte[] sampleData = new byte[] {0x00, 0x01, 0x02, 0x03}; + + try (FragmentedMp4Muxer fragmentedMp4Muxer = + new FragmentedMp4Muxer.Builder(new FileOutputStream(outputFilePath).getChannel()).build()) { + int audioTrack = fragmentedMp4Muxer.addTrack(eac3JocFormat); + fragmentedMp4Muxer.writeSampleData( + audioTrack, + ByteBuffer.wrap(sampleData), + new BufferInfo( + /* presentationTimeUs= */ 0L, + /* size= */ sampleData.length, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME)); + } + + FragmentedMp4Extractor extractor = + new FragmentedMp4Extractor(new DefaultSubtitleParserFactory()); + FakeExtractorOutput extractorOutput = + TestUtil.extractAllSamplesFromFilePath(extractor, outputFilePath); + Format extractedFormat = checkNotNull(extractorOutput.trackOutputs.valueAt(0).lastFormat); + assertThat(extractedFormat.sampleMimeType).isEqualTo(MimeTypes.AUDIO_E_AC3); + assertThat(extractedFormat.initializationData).hasSize(1); + assertThat(extractedFormat.initializationData.get(0)).isEqualTo(expectedDec3Payload); + } + + @Test + public void writeFragmentedMp4File_onlyEndOfStreamSample_doesNotCrash() throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + Format format = new Format.Builder().setSampleMimeType("xyz").build(); + try (FragmentedMp4Muxer muxer = + new FragmentedMp4Muxer.Builder(new FileOutputStream(outputFilePath).getChannel()).build()) { + int trackId = muxer.addTrack(format); + BufferInfo endOfStreamBufferInfo = + new BufferInfo( + /* presentationTimeUs= */ 2_000_000L, /* size= */ 0, C.BUFFER_FLAG_END_OF_STREAM); + muxer.writeSampleData(trackId, ByteBuffer.allocate(0), endOfStreamBufferInfo); + } + + byte[] outputFileBytes = TestUtil.getByteArrayFromFilePath(outputFilePath); + assertThat(outputFileBytes).isEmpty(); + } + + @Test + public void addTrack_afterClose_throws() throws Exception { + FragmentedMp4Muxer muxer = + new FragmentedMp4Muxer.Builder(new FileOutputStream(temporaryFolder.newFile()).getChannel()) + .build(); + muxer.close(); + Format format = new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build(); + + assertThrows(IllegalStateException.class, () -> muxer.addTrack(format)); + } + + @Test + public void writeSampleData_afterClose_throws() throws Exception { + FragmentedMp4Muxer muxer = + new FragmentedMp4Muxer.Builder(new FileOutputStream(temporaryFolder.newFile()).getChannel()) + .build(); + Format format = new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build(); + int trackId = muxer.addTrack(format); + muxer.close(); + BufferInfo bufferInfo = + new BufferInfo(/* presentationTimeUs= */ 0, /* size= */ 0, /* flags= */ 0); + ByteBuffer sampleData = ByteBuffer.allocate(0); + + assertThrows( + IllegalStateException.class, () -> muxer.writeSampleData(trackId, sampleData, bufferInfo)); + } + + @Test + public void close_afterClose_throws() throws Exception { + FragmentedMp4Muxer muxer = + new FragmentedMp4Muxer.Builder(new FileOutputStream(temporaryFolder.newFile()).getChannel()) + .build(); + muxer.close(); + + assertThrows(IllegalStateException.class, muxer::close); + } + + @Test + public void addTrack_withDolbyVisionProfile20_throws() throws Exception { + Format dolbyVisionProfile20Format = + FAKE_VIDEO_FORMAT + .buildUpon() + .setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION) + .setCodecs("dvh1.20.01") + .build(); + + try (FragmentedMp4Muxer muxer = + new FragmentedMp4Muxer.Builder(new FileOutputStream(temporaryFolder.newFile()).getChannel()) + .build()) { + assertThrows(MuxerException.class, () -> muxer.addTrack(dolbyVisionProfile20Format)); + } + } + + @Test + public void addTrack_withDolbyVisionNullCodecs_throws() throws Exception { + Format dolbyVisionNullCodecsFormat = + FAKE_VIDEO_FORMAT + .buildUpon() + .setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION) + .setCodecs(null) + .build(); + + try (FragmentedMp4Muxer muxer = + new FragmentedMp4Muxer.Builder(new FileOutputStream(temporaryFolder.newFile()).getChannel()) + .build()) { + assertThrows(MuxerException.class, () -> muxer.addTrack(dolbyVisionNullCodecsFormat)); + } + } + + @Test + public void addTrack_withDolbyVisionMalformedCodecs_throws() throws Exception { + Format dolbyVisionMalformedCodecsFormat = + FAKE_VIDEO_FORMAT + .buildUpon() + .setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION) + .setCodecs("dvhe.xx.01") + .build(); + + try (FragmentedMp4Muxer muxer = + new FragmentedMp4Muxer.Builder(new FileOutputStream(temporaryFolder.newFile()).getChannel()) + .build()) { + assertThrows(MuxerException.class, () -> muxer.addTrack(dolbyVisionMalformedCodecsFormat)); + } + } + + @Test + public void createFragmentedMp4File_withMultipleTracks_preservesTrackIds() throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + try (FragmentedMp4Muxer muxer = + new FragmentedMp4Muxer.Builder(new FileOutputStream(outputFilePath).getChannel()).build()) { + int videoTrackId = muxer.addTrack(FAKE_VIDEO_FORMAT); + int audioTrackId = muxer.addTrack(MuxerTestUtil.FAKE_AUDIO_FORMAT); + + assertThat(videoTrackId).isEqualTo(1); + assertThat(audioTrackId).isEqualTo(2); + + Pair videoSample = + getFakeSampleAndSampleInfo(/* presentationTimeUs= */ 0L, /* isVideo= */ true); + muxer.writeSampleData(videoTrackId, videoSample.first, videoSample.second); + + Pair audioSample = + getFakeSampleAndSampleInfo(/* presentationTimeUs= */ 0L, /* isVideo= */ false); + muxer.writeSampleData(audioTrackId, audioSample.first, audioSample.second); + } + + FragmentedMp4Extractor extractor = + new FragmentedMp4Extractor(new DefaultSubtitleParserFactory()); + FakeExtractorOutput fakeExtractorOutput = + TestUtil.extractAllSamplesFromFilePath(extractor, outputFilePath); + + assertThat(fakeExtractorOutput.numberOfTracks).isEqualTo(2); + assertThat(fakeExtractorOutput.trackOutputs.get(0).lastFormat.sampleMimeType) + .isEqualTo(FAKE_VIDEO_FORMAT.sampleMimeType); + assertThat(fakeExtractorOutput.trackOutputs.get(1).lastFormat.sampleMimeType) + .isEqualTo(MuxerTestUtil.FAKE_AUDIO_FORMAT.sampleMimeType); + } + + private static ByteBuffer getTestSamplesForEncoding(@C.PcmEncoding int pcmEncoding) { + Random random = new Random(/* seed= */ 0); + int sampleCount = 100; + ByteBuffer sampleBuffer; + if (pcmEncoding == C.ENCODING_PCM_DOUBLE || pcmEncoding == C.ENCODING_PCM_DOUBLE_BIG_ENDIAN) { + sampleBuffer = createByteBuffer(buildDoubleTestSamples(sampleCount, random)); + } else if (pcmEncoding == C.ENCODING_PCM_FLOAT + || pcmEncoding == C.ENCODING_PCM_FLOAT_BIG_ENDIAN) { + sampleBuffer = createByteBuffer(buildFloatTestSamples(sampleCount, random)); + } else { + int bytesPerSample = getPcmFrameSize(pcmEncoding, /* channelCount= */ 1); + byte[] rawBytes = buildTestData(sampleCount * bytesPerSample, random); + sampleBuffer = createByteBuffer(rawBytes); + } + return sampleBuffer.order( + isPcmEncodingBigEndian(pcmEncoding) ? ByteOrder.BIG_ENDIAN : ByteOrder.LITTLE_ENDIAN); + } + + private static final class PcmCEncodingValuesProvider extends TestParameterValuesProvider { + @Override + protected ImmutableList provideValues(TestParameterValuesProvider.Context context) { + return ImmutableList.of( + value(C.ENCODING_PCM_FLOAT).withName("ENCODING_PCM_FLOAT"), + value(C.ENCODING_PCM_FLOAT_BIG_ENDIAN).withName("ENCODING_PCM_FLOAT_BIG_ENDIAN"), + value(C.ENCODING_PCM_DOUBLE).withName("ENCODING_PCM_DOUBLE"), + value(C.ENCODING_PCM_DOUBLE_BIG_ENDIAN).withName("ENCODING_PCM_DOUBLE_BIG_ENDIAN"), + value(C.ENCODING_PCM_24BIT).withName("ENCODING_PCM_24BIT"), + value(C.ENCODING_PCM_24BIT_BIG_ENDIAN).withName("ENCODING_PCM_24BIT_BIG_ENDIAN"), + value(C.ENCODING_PCM_32BIT).withName("ENCODING_PCM_32BIT"), + value(C.ENCODING_PCM_32BIT_BIG_ENDIAN).withName("ENCODING_PCM_32BIT_BIG_ENDIAN")); + } + } } diff --git a/libraries/muxer/src/test/java/androidx/media3/muxer/FragmentedMp4MuxerTest.java b/libraries/muxer/src/test/java/androidx/media3/muxer/FragmentedMp4MuxerTest.java new file mode 100644 index 00000000000..551f337948f --- /dev/null +++ b/libraries/muxer/src/test/java/androidx/media3/muxer/FragmentedMp4MuxerTest.java @@ -0,0 +1,233 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.muxer; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.media3.common.C; +import androidx.media3.common.Format; +import androidx.media3.common.MimeTypes; +import com.google.common.collect.ImmutableList; +import java.io.ByteArrayOutputStream; +import java.nio.ByteBuffer; +import java.nio.channels.Channels; +import java.nio.charset.StandardCharsets; +import java.util.ArrayList; +import java.util.List; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.RobolectricTestRunner; + +/** Tests for fragment boundaries in {@link FragmentedMp4Muxer}. */ +@RunWith(RobolectricTestRunner.class) +public class FragmentedMp4MuxerTest { + + private static final byte[] SPS = hex("000000016742c00ad9047b0110000003001000000303c0f1226480"); + private static final byte[] PPS = hex("0000000168cb83cb20"); + private static final byte[] KEY_FRAME = hex("0000000165888400ffee"); + private static final byte[] NON_KEY_FRAME = hex("00000001419a0011"); + private static final Format VIDEO_FORMAT = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H264) + .setWidth(64) + .setHeight(48) + .setMaxNumReorderSamples(0) + .setInitializationData(ImmutableList.of(SPS, PPS)) + .build(); + + private static final long FRAME_DURATION_US = 33_333; + private static final int FRAME_COUNT = 90; + private static final long FRAGMENT_DURATION_MS = 1_000; + private static final int SYNC_SAMPLE_FLAGS = 0x02000000; + private static final int NON_SYNC_SAMPLE_FLAGS = 0x01010000; + + @Test + public void writeSampleData_singleKeyFrame_writesFragmentsWhileSamplesArrive() throws Exception { + ByteArrayOutputStream output = new ByteArrayOutputStream(); + FragmentedMp4Muxer muxer = buildMuxer(output, /* fragmentsBetweenKeyFrames= */ true); + + writeVideo(muxer, presentationOrderTimestampsUs()); + int fragmentsBeforeClose = children(output.toByteArray(), 0, output.size(), "moof").size(); + muxer.close(); + byte[] file = output.toByteArray(); + + assertThat(fragmentsBeforeClose).isEqualTo(2); + assertThat(firstSampleFlags(file)) + .containsExactly(SYNC_SAMPLE_FLAGS, NON_SYNC_SAMPLE_FLAGS, NON_SYNC_SAMPLE_FLAGS) + .inOrder(); + assertThat(sampleCount(file)).isEqualTo(FRAME_COUNT); + } + + @Test + public void writeSampleData_disabled_writesSingleKeyFrameVideoOnClose() throws Exception { + ByteArrayOutputStream output = new ByteArrayOutputStream(); + FragmentedMp4Muxer muxer = buildMuxer(output, /* fragmentsBetweenKeyFrames= */ false); + + writeVideo(muxer, presentationOrderTimestampsUs()); + int fragmentsBeforeClose = children(output.toByteArray(), 0, output.size(), "moof").size(); + muxer.close(); + byte[] file = output.toByteArray(); + + assertThat(fragmentsBeforeClose).isEqualTo(0); + assertThat(firstSampleFlags(file)).containsExactly(SYNC_SAMPLE_FLAGS); + assertThat(sampleCount(file)).isEqualTo(FRAME_COUNT); + } + + @Test + public void writeSampleData_bFrames_waitsForKeyFrame() throws Exception { + ByteArrayOutputStream output = new ByteArrayOutputStream(); + FragmentedMp4Muxer muxer = buildMuxer(output, /* fragmentsBetweenKeyFrames= */ true); + Format reorderedVideoFormat = + VIDEO_FORMAT.buildUpon().setMaxNumReorderSamples(/* maxNumReorderSamples= */ 2).build(); + long[] timestampsUs = new long[FRAME_COUNT]; + for (int i = 0; i < FRAME_COUNT; i++) { + int frameIndex = i == 0 ? 0 : i % 3 == 1 ? i + 2 : i - 1; + timestampsUs[i] = frameIndex * FRAME_DURATION_US; + } + + writeVideo(muxer, reorderedVideoFormat, timestampsUs); + muxer.close(); + byte[] file = output.toByteArray(); + + assertThat(firstSampleFlags(file)).containsExactly(SYNC_SAMPLE_FLAGS); + assertThat(sampleCount(file)).isEqualTo(FRAME_COUNT); + } + + @Test + public void writeSampleData_emptySampleBeforeKeyFrame_doesNotFlushEmptySample() throws Exception { + ByteArrayOutputStream output = new ByteArrayOutputStream(); + FragmentedMp4Muxer muxer = buildMuxer(output, /* fragmentsBetweenKeyFrames= */ true); + int trackId = muxer.addTrack(VIDEO_FORMAT); + for (int i = 0; i < 32; i++) { + writeVideoSample(muxer, trackId, i * FRAME_DURATION_US, /* isKeyFrame= */ i == 0); + } + muxer.writeSampleData( + trackId, + ByteBuffer.allocate(0), + new BufferInfo(/* presentationTimeUs= */ 2_000_000, /* size= */ 0, /* flags= */ 0)); + + writeVideoSample(muxer, trackId, /* presentationTimeUs= */ 2_100_000, /* isKeyFrame= */ true); + int fragmentsBeforeClose = children(output.toByteArray(), 0, output.size(), "moof").size(); + muxer.close(); + byte[] file = output.toByteArray(); + + assertThat(fragmentsBeforeClose).isEqualTo(1); + assertThat(firstSampleFlags(file)).containsExactly(SYNC_SAMPLE_FLAGS, SYNC_SAMPLE_FLAGS); + assertThat(sampleCount(file)).isEqualTo(33); + } + + private static FragmentedMp4Muxer buildMuxer( + ByteArrayOutputStream output, boolean fragmentsBetweenKeyFrames) { + return new FragmentedMp4Muxer.Builder(Channels.newChannel(output)) + .setFragmentDurationMs(FRAGMENT_DURATION_MS) + .setFragmentsBetweenKeyFramesEnabled(fragmentsBetweenKeyFrames) + .build(); + } + + private static long[] presentationOrderTimestampsUs() { + long[] timestampsUs = new long[FRAME_COUNT]; + for (int i = 0; i < FRAME_COUNT; i++) { + timestampsUs[i] = i * FRAME_DURATION_US; + } + return timestampsUs; + } + + private static void writeVideo(FragmentedMp4Muxer muxer, long[] timestampsUs) throws Exception { + writeVideo(muxer, VIDEO_FORMAT, timestampsUs); + } + + private static void writeVideo(FragmentedMp4Muxer muxer, Format videoFormat, long[] timestampsUs) + throws Exception { + int trackId = muxer.addTrack(videoFormat); + for (int i = 0; i < timestampsUs.length; i++) { + writeVideoSample(muxer, trackId, timestampsUs[i], /* isKeyFrame= */ i == 0); + } + } + + private static void writeVideoSample( + FragmentedMp4Muxer muxer, int trackId, long presentationTimeUs, boolean isKeyFrame) + throws Exception { + byte[] sample = isKeyFrame ? KEY_FRAME : NON_KEY_FRAME; + muxer.writeSampleData( + trackId, + ByteBuffer.wrap(sample), + new BufferInfo( + presentationTimeUs, sample.length, isKeyFrame ? C.BUFFER_FLAG_KEY_FRAME : 0)); + } + + private static List firstSampleFlags(byte[] file) { + List flags = new ArrayList<>(); + for (int[] trun : truns(file)) { + int trunFlags = readInt(file, trun[0]) & 0xFFFFFF; + assertThat(trunFlags & 0x400).isNotEqualTo(0); + int position = trun[0] + 8; + if ((trunFlags & 0x1) != 0) { + position += 4; + } + if ((trunFlags & 0x100) != 0) { + position += 4; + } + if ((trunFlags & 0x200) != 0) { + position += 4; + } + flags.add(readInt(file, position)); + } + return flags; + } + + private static int sampleCount(byte[] file) { + int count = 0; + for (int[] trun : truns(file)) { + count += readInt(file, trun[0] + 4); + } + return count; + } + + private static List truns(byte[] file) { + List truns = new ArrayList<>(); + for (int[] moof : children(file, 0, file.length, "moof")) { + for (int[] traf : children(file, moof[0], moof[1], "traf")) { + truns.addAll(children(file, traf[0], traf[1], "trun")); + } + } + return truns; + } + + private static List children(byte[] file, int start, int end, String type) { + List boxes = new ArrayList<>(); + int position = start; + while (position + 8 <= end) { + int size = readInt(file, position); + if (new String(file, position + 4, 4, StandardCharsets.US_ASCII).equals(type)) { + boxes.add(new int[] {position + 8, position + size}); + } + position += size; + } + return boxes; + } + + private static int readInt(byte[] data, int position) { + return ByteBuffer.wrap(data, position, 4).getInt(); + } + + private static byte[] hex(String hex) { + byte[] bytes = new byte[hex.length() / 2]; + for (int i = 0; i < bytes.length; i++) { + bytes[i] = (byte) Integer.parseInt(hex.substring(2 * i, 2 * i + 2), 16); + } + return bytes; + } +} diff --git a/libraries/muxer/src/test/java/androidx/media3/muxer/MediaMuxerCompatTest.java b/libraries/muxer/src/test/java/androidx/media3/muxer/MediaMuxerCompatTest.java index 2a4432a6a0a..d18c5d0d682 100644 --- a/libraries/muxer/src/test/java/androidx/media3/muxer/MediaMuxerCompatTest.java +++ b/libraries/muxer/src/test/java/androidx/media3/muxer/MediaMuxerCompatTest.java @@ -18,9 +18,11 @@ import static android.media.MediaFormat.KEY_CAPTURE_RATE; import static android.media.MediaFormat.KEY_MIME; import static androidx.media3.muxer.MediaMuxerCompat.OUTPUT_FORMAT_MP4; +import static androidx.media3.muxer.MediaMuxerCompat.OUTPUT_FORMAT_WEBM; import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_WITH_INCREASING_TIMESTAMPS; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; import android.content.Context; import android.media.MediaCodec; @@ -33,6 +35,8 @@ import androidx.media3.common.MimeTypes; import androidx.media3.common.util.Util; import androidx.media3.container.MdtaMetadataEntry; +import androidx.media3.container.Mp4LocationData; +import androidx.media3.extractor.mkv.MatroskaExtractor; import androidx.media3.extractor.mp4.Mp4Extractor; import androidx.media3.extractor.text.DefaultSubtitleParserFactory; import androidx.media3.inspector.MediaExtractorCompat; @@ -85,16 +89,199 @@ public void createMp4File_withCaptureRateKey_writesCorrectCaptureFps() throws Ex assertThat(actualCaptureFps).isEqualTo(expectedCaptureFps); } + @Test + public void createWebmFile_withValidWebmAsset_writesSamplesSuccessfully() throws Exception { + String outputFilePath = tempFolder.newFile().getAbsolutePath(); + + MediaMuxerCompat mediaMuxerCompat = new MediaMuxerCompat(outputFilePath, OUTPUT_FORMAT_WEBM); + try { + // Calling setLocation and setOrientationHint are safe no-ops for WebM and do not throw. + mediaMuxerCompat.setLocation(/* latitude= */ 37.4220f, /* longitude= */ -122.0841f); + mediaMuxerCompat.setOrientationHint(90); + feedDataToMuxer( + context, + mediaMuxerCompat, + "asset:///media/mkv/bbb_960x540_60fps_vp8.webm", + /* captureFps= */ C.RATE_UNSET); + } finally { + mediaMuxerCompat.release(); + } + + FakeExtractorOutput fakeExtractorOutput = + TestUtil.extractAllSamplesFromFilePath( + new MatroskaExtractor(new DefaultSubtitleParserFactory()), + checkNotNull(outputFilePath)); + Format videoTrackFormat = + Iterables.getOnlyElement(fakeExtractorOutput.getTrackOutputsForType(C.TRACK_TYPE_VIDEO)) + .lastFormat; + assertThat(videoTrackFormat.sampleMimeType).isEqualTo(MimeTypes.VIDEO_VP9); + assertThat(videoTrackFormat.rotationDegrees).isEqualTo(0); + } + + @Test + public void createWebmFile_withNullLanguage_writesUndefinedLanguage() throws Exception { + String outputFilePath = tempFolder.newFile().getAbsolutePath(); + MediaMuxerCompat mediaMuxerCompat = new MediaMuxerCompat(outputFilePath, OUTPUT_FORMAT_WEBM); + MediaFormat videoFormat = + MediaFormat.createVideoFormat(MediaFormat.MIMETYPE_VIDEO_VP8, 1280, 720); + + try { + int trackIndex = mediaMuxerCompat.addTrack(videoFormat); + mediaMuxerCompat.start(); + ByteBuffer sampleBuffer = ByteBuffer.allocateDirect(100); + MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo(); + bufferInfo.set( + /* newOffset= */ 0, + /* newSize= */ 50, + /* newTimeUs= */ 1000L, + MediaCodec.BUFFER_FLAG_KEY_FRAME); + + mediaMuxerCompat.writeSampleData(trackIndex, sampleBuffer, bufferInfo); + } finally { + mediaMuxerCompat.release(); + } + + FakeExtractorOutput fakeExtractorOutput = + TestUtil.extractAllSamplesFromFilePath( + new MatroskaExtractor(new DefaultSubtitleParserFactory()), + checkNotNull(outputFilePath)); + Format videoTrackFormat = + Iterables.getOnlyElement(fakeExtractorOutput.getTrackOutputsForType(C.TRACK_TYPE_VIDEO)) + .lastFormat; + assertThat(videoTrackFormat.language).isEqualTo("und"); + } + + @Test + public void addTrack_withZeroFrameRateInMediaFormat_succeeds() throws Exception { + String outputFilePath = tempFolder.newFile().getAbsolutePath(); + MediaMuxerCompat mediaMuxerCompat = new MediaMuxerCompat(outputFilePath, OUTPUT_FORMAT_MP4); + MediaFormat videoFormat = + MediaFormat.createVideoFormat(MediaFormat.MIMETYPE_VIDEO_AVC, 1280, 720); + videoFormat.setInteger(MediaFormat.KEY_FRAME_RATE, 0); + + try { + int trackIndex = mediaMuxerCompat.addTrack(videoFormat); + assertThat(trackIndex).isAtLeast(0); + } finally { + mediaMuxerCompat.release(); + } + } + + @Test + public void addTrack_withNanFrameRateInMediaFormat_succeeds() throws Exception { + String outputFilePath = tempFolder.newFile().getAbsolutePath(); + MediaMuxerCompat mediaMuxerCompat = new MediaMuxerCompat(outputFilePath, OUTPUT_FORMAT_MP4); + MediaFormat videoFormat = + MediaFormat.createVideoFormat(MediaFormat.MIMETYPE_VIDEO_AVC, 1280, 720); + videoFormat.setFloat(MediaFormat.KEY_FRAME_RATE, Float.NaN); + + try { + int trackIndex = mediaMuxerCompat.addTrack(videoFormat); + assertThat(trackIndex).isAtLeast(0); + } finally { + mediaMuxerCompat.release(); + } + } + + @Test + public void createWebmFile_withUnsupportedTrack_throws() throws Exception { + String outputFilePath = tempFolder.newFile().getAbsolutePath(); + MediaMuxerCompat mediaMuxerCompat = new MediaMuxerCompat(outputFilePath, OUTPUT_FORMAT_WEBM); + MediaFormat unsupportedFormat = + MediaFormat.createVideoFormat(MediaFormat.MIMETYPE_VIDEO_AVC, 1280, 720); + + try { + assertThrows( + IllegalArgumentException.class, () -> mediaMuxerCompat.addTrack(unsupportedFormat)); + } finally { + try { + mediaMuxerCompat.release(); + } catch (RuntimeException exception) { + // TODO: b/544606733 - Gracefully handle closing empty WebM containers when no samples were + // written. + // Suppress finalization exception on uninitialized container to prevent teardown failures + // from masking the assertion error above. + } + } + } + + @Test + public void setLocation_withValidCoordinates_setsLocationDataOnOutput() throws Exception { + String outputFilePath = tempFolder.newFile().getAbsolutePath(); + float latitude = 37.7749f; + float longitude = -122.4194f; + + MediaMuxerCompat mediaMuxerCompat = new MediaMuxerCompat(outputFilePath, OUTPUT_FORMAT_MP4); + try { + mediaMuxerCompat.setLocation(latitude, longitude); + feedDataToMuxer( + context, mediaMuxerCompat, MP4_ASSET_WITH_INCREASING_TIMESTAMPS.uri, C.RATE_UNSET); + } finally { + mediaMuxerCompat.release(); + } + + FakeExtractorOutput fakeExtractorOutput = + TestUtil.extractAllSamplesFromFilePath( + new Mp4Extractor(new DefaultSubtitleParserFactory()), checkNotNull(outputFilePath)); + Format videoTrackFormat = + Iterables.getOnlyElement(fakeExtractorOutput.getTrackOutputsForType(C.TRACK_TYPE_VIDEO)) + .lastFormat; + Mp4LocationData locationData = findMetadataEntry(videoTrackFormat, Mp4LocationData.class); + assertThat(locationData).isNotNull(); + assertThat(locationData.latitude).isEqualTo(latitude); + assertThat(locationData.longitude).isEqualTo(longitude); + } + + @Test + public void setOrientationHint_withValidOrientation_setsOrientationDataOnOutput() + throws Exception { + String outputFilePath = tempFolder.newFile().getAbsolutePath(); + int orientation = 90; + + MediaMuxerCompat mediaMuxerCompat = new MediaMuxerCompat(outputFilePath, OUTPUT_FORMAT_MP4); + try { + mediaMuxerCompat.setOrientationHint(orientation); + feedDataToMuxer( + context, mediaMuxerCompat, MP4_ASSET_WITH_INCREASING_TIMESTAMPS.uri, C.RATE_UNSET); + } finally { + mediaMuxerCompat.release(); + } + + FakeExtractorOutput fakeExtractorOutput = + TestUtil.extractAllSamplesFromFilePath( + new Mp4Extractor(new DefaultSubtitleParserFactory()), checkNotNull(outputFilePath)); + Format videoTrackFormat = + Iterables.getOnlyElement(fakeExtractorOutput.getTrackOutputsForType(C.TRACK_TYPE_VIDEO)) + .lastFormat; + assertThat(videoTrackFormat.rotationDegrees).isEqualTo(orientation); + } + + @Nullable + private static T findMetadataEntry(Format format, Class clazz) { + if (format.metadata == null) { + return null; + } + for (int i = 0; i < format.metadata.length(); i++) { + Metadata.Entry metadataEntry = format.metadata.get(i); + if (clazz.isInstance(metadataEntry)) { + return clazz.cast(metadataEntry); + } + } + return null; + } + + @SuppressWarnings("PatternMatchingInstanceof") private static MdtaMetadataEntry findCaptureFpsMetadata(Format format) { if (format.metadata == null) { return null; } for (int i = 0; i < format.metadata.length(); i++) { Metadata.Entry metadataEntry = format.metadata.get(i); - if (metadataEntry instanceof MdtaMetadataEntry - && ((MdtaMetadataEntry) metadataEntry) - .key.equals(MdtaMetadataEntry.KEY_ANDROID_CAPTURE_FPS)) { - return (MdtaMetadataEntry) metadataEntry; + if (metadataEntry instanceof MdtaMetadataEntry) { + MdtaMetadataEntry mdtaMetadataEntry = (MdtaMetadataEntry) metadataEntry; + if (mdtaMetadataEntry.key.equals(MdtaMetadataEntry.KEY_ANDROID_CAPTURE_FPS)) { + return mdtaMetadataEntry; + } } } return null; diff --git a/libraries/muxer/src/test/java/androidx/media3/muxer/Mp4MuxerEndToEndParameterizedTest.java b/libraries/muxer/src/test/java/androidx/media3/muxer/Mp4MuxerEndToEndParameterizedTest.java index 632c6230c24..59de0bf71a5 100644 --- a/libraries/muxer/src/test/java/androidx/media3/muxer/Mp4MuxerEndToEndParameterizedTest.java +++ b/libraries/muxer/src/test/java/androidx/media3/muxer/Mp4MuxerEndToEndParameterizedTest.java @@ -62,6 +62,8 @@ public class Mp4MuxerEndToEndParameterizedTest { // Audio Codecs private static final String AMR_NB_3GP = "3gp/bbb_mono_8kHz_12.2kbps_amrnb.3gp"; private static final String AMR_WB_3GP = "3gp/bbb_mono_16kHz_23.05kbps_amrwb.3gp"; + private static final String E_AC3_MP4 = "mp4/sample_eac3.mp4"; + private static final String E_AC3_JOC_MP4 = "mp4/sample_eac3joc.mp4"; private static final String IAMF_OPUS_MP4 = "mp4/5p1_iamf_opus.mp4"; private static final String OPUS_OGG = "ogg/bbb_6ch_8kHz_opus.ogg"; private static final String VORBIS_OGG = "ogg/bbb_1ch_16kHz_q10_vorbis.ogg"; @@ -88,6 +90,8 @@ public static ImmutableList mediaSamples() { VP9_WEB, AMR_NB_3GP, AMR_WB_3GP, + E_AC3_MP4, + E_AC3_JOC_MP4, IAMF_OPUS_MP4, OPUS_OGG, VORBIS_OGG, diff --git a/libraries/muxer/src/test/java/androidx/media3/muxer/Mp4MuxerEndToEndTest.java b/libraries/muxer/src/test/java/androidx/media3/muxer/Mp4MuxerEndToEndTest.java index 09b81a33755..7c56164b350 100644 --- a/libraries/muxer/src/test/java/androidx/media3/muxer/Mp4MuxerEndToEndTest.java +++ b/libraries/muxer/src/test/java/androidx/media3/muxer/Mp4MuxerEndToEndTest.java @@ -15,6 +15,9 @@ */ package androidx.media3.muxer; +import static androidx.media3.common.util.Util.getPcmFormat; +import static androidx.media3.common.util.Util.getPcmFrameSize; +import static androidx.media3.common.util.Util.isPcmEncodingBigEndian; import static androidx.media3.muxer.Mp4Muxer.FILE_FORMAT_MP4_WITH_AUXILIARY_TRACKS_EXTENSION; import static androidx.media3.muxer.Mp4Muxer.LAST_SAMPLE_DURATION_BEHAVIOR_SET_FROM_END_OF_STREAM_BUFFER_OR_DUPLICATE_PREVIOUS; import static androidx.media3.muxer.MuxerTestUtil.FAKE_AUDIO_FORMAT; @@ -22,6 +25,11 @@ import static androidx.media3.muxer.MuxerTestUtil.XMP_SAMPLE_DATA; import static androidx.media3.muxer.MuxerTestUtil.feedInputDataToMuxer; import static androidx.media3.muxer.MuxerTestUtil.getFakeSampleAndSampleInfo; +import static androidx.media3.test.utils.TestUtil.buildDoubleTestSamples; +import static androidx.media3.test.utils.TestUtil.buildFloatTestSamples; +import static androidx.media3.test.utils.TestUtil.buildTestData; +import static androidx.media3.test.utils.TestUtil.createByteArray; +import static androidx.media3.test.utils.TestUtil.createByteBuffer; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.truth.Truth.assertThat; import static org.junit.Assert.assertThrows; @@ -43,20 +51,27 @@ import androidx.media3.test.utils.DumpFileAsserts; import androidx.media3.test.utils.DumpableMp4Box; import androidx.media3.test.utils.FakeExtractorOutput; +import androidx.media3.test.utils.FakeTrackOutput; import androidx.media3.test.utils.TestUtil; import androidx.test.core.app.ApplicationProvider; -import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.collect.ImmutableList; import com.google.common.collect.Iterables; +import com.google.testing.junit.testparameterinjector.TestParameter; +import com.google.testing.junit.testparameterinjector.TestParameterValuesProvider; +import java.io.File; +import java.io.FileOutputStream; import java.nio.ByteBuffer; +import java.nio.ByteOrder; +import java.util.Random; import org.junit.Rule; import org.junit.Test; import org.junit.rules.TemporaryFolder; import org.junit.runner.RunWith; +import org.robolectric.RobolectricTestParameterInjector; import org.robolectric.annotation.Config; /** End to end tests for {@link Mp4Muxer}. */ -@RunWith(AndroidJUnit4.class) +@RunWith(RobolectricTestParameterInjector.class) public class Mp4MuxerEndToEndTest { @Rule public final TemporaryFolder temporaryFolder = new TemporaryFolder(); @@ -65,6 +80,7 @@ public class Mp4MuxerEndToEndTest { private static final String H265_HDR10_MP4 = "hdr10-720p.mp4"; private static final String AV1_MP4 = "sample_av1.mp4"; private static final String VP9_MP4 = "bbb_800x640_768kbps_30fps_vp9.mp4"; + private static final String IT35_MP4 = "sample_with_it35_track.mp4"; private final Context context = ApplicationProvider.getApplicationContext(); @Test @@ -105,6 +121,73 @@ public void writeMp4File_withSampleAndMetadata_matchedExpectedBoxStructure() thr MuxerTestUtil.getExpectedMp4DumpFilePath("mp4_with_samples_and_metadata.mp4")); } + @Test + public void createMp4File_withDolbyVisionTrack_ftypContainsDby1CompatibleBrand() + throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + Format dolbyVisionFormat = + FAKE_VIDEO_FORMAT + .buildUpon() + .setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION) + .setCodecs("dvav.09.02") + .build(); + Pair sampleAndSampleInfo = + getFakeSampleAndSampleInfo(/* presentationTimeUs= */ 0L, /* isVideo= */ true); + + try (Mp4Muxer muxer = new Mp4Muxer.Builder(SeekableMuxerOutput.of(outputFilePath)).build()) { + int trackId = muxer.addTrack(dolbyVisionFormat); + muxer.writeSampleData(trackId, sampleAndSampleInfo.first, sampleAndSampleInfo.second); + } + + byte[] outputFileBytes = TestUtil.getByteArrayFromFilePath(outputFilePath); + assertThat(MuxerTestUtil.ftypBoxContainsCompatibleBrand(outputFileBytes, "dby1")).isTrue(); + } + + @Test + public void addTrack_withDolbyVisionProfile20_throws() throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + Format dolbyVisionProfile20Format = + FAKE_VIDEO_FORMAT + .buildUpon() + .setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION) + .setCodecs("dvh1.20.01") + .build(); + + try (Mp4Muxer muxer = new Mp4Muxer.Builder(SeekableMuxerOutput.of(outputFilePath)).build()) { + assertThrows(MuxerException.class, () -> muxer.addTrack(dolbyVisionProfile20Format)); + } + } + + @Test + public void addTrack_withDolbyVisionNullCodecs_throws() throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + Format dolbyVisionNullCodecsFormat = + FAKE_VIDEO_FORMAT + .buildUpon() + .setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION) + .setCodecs(null) + .build(); + + try (Mp4Muxer muxer = new Mp4Muxer.Builder(SeekableMuxerOutput.of(outputFilePath)).build()) { + assertThrows(MuxerException.class, () -> muxer.addTrack(dolbyVisionNullCodecsFormat)); + } + } + + @Test + public void addTrack_withDolbyVisionMalformedCodecs_throws() throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + Format dolbyVisionMalformedCodecsFormat = + FAKE_VIDEO_FORMAT + .buildUpon() + .setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION) + .setCodecs("dvhe.xx.01") + .build(); + + try (Mp4Muxer muxer = new Mp4Muxer.Builder(SeekableMuxerOutput.of(outputFilePath)).build()) { + assertThrows(MuxerException.class, () -> muxer.addTrack(dolbyVisionMalformedCodecsFormat)); + } + } + @Test public void createMp4File_addTrackAndMetadataButNoSamples_createsEmptyFile() throws Exception { String outputFilePath = temporaryFolder.newFile().getPath(); @@ -175,6 +258,30 @@ public void createVp9Mp4File_withoutCsd_matchesExpected() throws Exception { MuxerTestUtil.getExpectedMp4DumpFilePath(VP9_MP4 + "_without_csd")); } + @Test + public void createMp4File_withIt35_matchesExpected() throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + + try (Mp4Muxer mp4Muxer = new Mp4Muxer.Builder(SeekableMuxerOutput.of(outputFilePath)).build()) { + mp4Muxer.addMetadataEntry( + new Mp4TimestampData( + /* creationTimestampSeconds= */ 100_000_000L, + /* modificationTimestampSeconds= */ 500_000_000L)); + feedInputDataToMuxer(context, mp4Muxer, MP4_FILE_ASSET_DIRECTORY + IT35_MP4); + mp4Muxer.addTrackReference( + /* trackId= */ 1, + Mp4Muxer.TRACK_REFERENCE_TYPE_CDSC, + /* referencedTrackIds= */ ImmutableList.of(0)); + } + + FakeExtractorOutput fakeExtractorOutput = + TestUtil.extractAllSamplesFromFilePath( + new Mp4Extractor(new DefaultSubtitleParserFactory()), checkNotNull(outputFilePath)); + + DumpFileAsserts.assertOutput( + context, fakeExtractorOutput, MuxerTestUtil.getExpectedMp4DumpFilePath(IT35_MP4)); + } + @Test public void createMp4File_muxerNotClosed_createsPartiallyWrittenValidFile() throws Exception { String outputPath = temporaryFolder.newFile().getPath(); @@ -279,6 +386,77 @@ public void createMp4File_withDifferentTracksOffset_matchesExpected() throws Exc MuxerTestUtil.getExpectedMp4DumpFilePath("mp4_with_different_tracks_offset.mp4")); } + @Test + public void createMp4File_withAudioEAc3Joc_createsDec3Box() throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + byte[] expectedDec3Payload = new byte[] {0x00, 0x00, 0x00, 0x03, 0x00}; + Format eac3JocFormat = + FAKE_AUDIO_FORMAT + .buildUpon() + .setSampleMimeType(MimeTypes.AUDIO_E_AC3_JOC) + .setInitializationData(ImmutableList.of(expectedDec3Payload)) + .build(); + Pair sampleAndInfo = + getFakeSampleAndSampleInfo(/* presentationTimeUs= */ 0L, /* isVideo= */ false); + + try (Mp4Muxer mp4Muxer = new Mp4Muxer.Builder(SeekableMuxerOutput.of(outputFilePath)).build()) { + int audioTrack = mp4Muxer.addTrack(eac3JocFormat); + mp4Muxer.writeSampleData(audioTrack, sampleAndInfo.first, sampleAndInfo.second); + } + + FakeExtractorOutput extractorOutput = + TestUtil.extractAllSamplesFromFilePath( + new Mp4Extractor(new DefaultSubtitleParserFactory()), outputFilePath); + Format extractedFormat = checkNotNull(extractorOutput.trackOutputs.valueAt(0).lastFormat); + assertThat(extractedFormat.sampleMimeType).isEqualTo(MimeTypes.AUDIO_E_AC3); + assertThat(extractedFormat.initializationData).hasSize(1); + assertThat(extractedFormat.initializationData.get(0)).isEqualTo(expectedDec3Payload); + } + + @Test + public void createMp4File_withPcmCBox_writesAndExtractsCorrectSamples( + @TestParameter(valuesProvider = PcmCEncodingValuesProvider.class) @C.PcmEncoding + int pcmEncoding) + throws Exception { + File outputFile = temporaryFolder.newFile(); + Format pcmFormat = getPcmFormat(pcmEncoding, /* channels= */ 1, /* sampleRate= */ 48_000); + ByteBuffer sampleBuffer = getTestSamplesForEncoding(pcmEncoding); + byte[] sampleData = createByteArray(sampleBuffer.duplicate()); + BufferInfo bufferInfo = + new BufferInfo(/* presentationTimeUs= */ 0L, sampleData.length, C.BUFFER_FLAG_KEY_FRAME); + + try (Mp4Muxer mp4Muxer = + new Mp4Muxer.Builder(SeekableMuxerOutput.of(outputFile.getPath())).build()) { + int audioTrack = mp4Muxer.addTrack(pcmFormat); + mp4Muxer.writeSampleData(audioTrack, sampleBuffer, bufferInfo); + } + + FakeExtractorOutput extractorOutput = + TestUtil.extractAllSamplesFromFilePath( + new Mp4Extractor(new DefaultSubtitleParserFactory()), outputFile.getPath()); + FakeTrackOutput trackOutput = extractorOutput.trackOutputs.valueAt(0); + Format extractedFormat = checkNotNull(trackOutput.lastFormat); + assertThat(extractedFormat.sampleMimeType).isEqualTo(MimeTypes.AUDIO_RAW); + assertThat(extractedFormat.pcmEncoding).isEqualTo(pcmEncoding); + assertThat(trackOutput.getSampleData(/* index= */ 0)).isEqualTo(sampleData); + } + + @Test + public void createMp4File_with8BitPcm_throws() throws Exception { + File outputFile = temporaryFolder.newFile(); + Format pcmFormat = + getPcmFormat(C.ENCODING_PCM_8BIT, /* channels= */ 1, /* sampleRate= */ 48_000); + ByteBuffer sampleBuffer = ByteBuffer.allocate(100); + BufferInfo bufferInfo = + new BufferInfo(/* presentationTimeUs= */ 0L, 100, C.BUFFER_FLAG_KEY_FRAME); + + Mp4Muxer mp4Muxer = new Mp4Muxer.Builder(SeekableMuxerOutput.of(outputFile.getPath())).build(); + int audioTrack = mp4Muxer.addTrack(pcmFormat); + mp4Muxer.writeSampleData(audioTrack, sampleBuffer, bufferInfo); + // Moov box and format validation are deferred until the muxer is closed. + assertThrows(IllegalArgumentException.class, mp4Muxer::close); + } + @Test public void createMp4File_withNegativeTracksOffset_matchesExpected() throws Exception { String outputFilePath = temporaryFolder.newFile().getPath(); @@ -1201,4 +1379,119 @@ private static void writeFakeSamples(Mp4Muxer muxer, int trackId, int sampleCoun muxer.writeSampleData(trackId, sampleAndSampleInfo.first, sampleAndSampleInfo.second); } } + + @Test + public void writeMp4File_onlyEndOfStreamSample_doesNotCrash() throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + try (Mp4Muxer mp4Muxer = new Mp4Muxer.Builder(SeekableMuxerOutput.of(outputFilePath)).build()) { + int track = mp4Muxer.addTrack(FAKE_VIDEO_FORMAT); + BufferInfo endOfStreamBufferInfo = + new BufferInfo( + /* presentationTimeUs= */ 2_000_000L, /* size= */ 0, C.BUFFER_FLAG_END_OF_STREAM); + mp4Muxer.writeSampleData(track, ByteBuffer.allocate(0), endOfStreamBufferInfo); + } + + byte[] outputFileBytes = TestUtil.getByteArrayFromFilePath(outputFilePath); + assertThat(outputFileBytes).isEmpty(); + } + + @Test + public void writeSampleData_withIdenticallySizedAudioSamples_parsesCleanlyWithMp4Extractor() + throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + + try (Mp4Muxer mp4Muxer = new Mp4Muxer.Builder(SeekableMuxerOutput.of(outputFilePath)).build()) { + int audioTrack = mp4Muxer.addTrack(FAKE_AUDIO_FORMAT); + + // Write 20 audio samples with uniform 100-byte size. + for (int i = 0; i < 20; i++) { + ByteBuffer sampleData = ByteBuffer.allocate(100); + BufferInfo bufferInfo = + new BufferInfo( + /* presentationTimeUs= */ i * 20_000L, + /* size= */ 100, + /* flags= */ C.BUFFER_FLAG_KEY_FRAME); + mp4Muxer.writeSampleData(audioTrack, sampleData, bufferInfo); + } + } + + FakeExtractorOutput fakeExtractorOutput = + TestUtil.extractAllSamplesFromFilePath( + new Mp4Extractor(new DefaultSubtitleParserFactory()), outputFilePath); + + assertThat(fakeExtractorOutput.numberOfTracks).isEqualTo(1); + assertThat(fakeExtractorOutput.seekMap).isNotNull(); + } + + @Test + public void addTrack_afterClose_throws() throws Exception { + Mp4Muxer mp4Muxer = + new Mp4Muxer.Builder( + SeekableMuxerOutput.of(new FileOutputStream(temporaryFolder.newFile()))) + .build(); + mp4Muxer.close(); + + assertThrows(IllegalStateException.class, () -> mp4Muxer.addTrack(FAKE_AUDIO_FORMAT)); + } + + @Test + public void writeSampleData_afterClose_throws() throws Exception { + Mp4Muxer mp4Muxer = + new Mp4Muxer.Builder( + SeekableMuxerOutput.of(new FileOutputStream(temporaryFolder.newFile()))) + .build(); + int trackId = mp4Muxer.addTrack(FAKE_AUDIO_FORMAT); + mp4Muxer.close(); + BufferInfo bufferInfo = + new BufferInfo(/* presentationTimeUs= */ 0, /* size= */ 0, /* flags= */ 0); + ByteBuffer sampleData = ByteBuffer.allocate(0); + + assertThrows( + IllegalStateException.class, + () -> mp4Muxer.writeSampleData(trackId, sampleData, bufferInfo)); + } + + @Test + public void close_afterClose_throws() throws Exception { + Mp4Muxer mp4Muxer = + new Mp4Muxer.Builder( + SeekableMuxerOutput.of(new FileOutputStream(temporaryFolder.newFile()))) + .build(); + mp4Muxer.close(); + + assertThrows(IllegalStateException.class, mp4Muxer::close); + } + + private static ByteBuffer getTestSamplesForEncoding(@C.PcmEncoding int pcmEncoding) { + Random random = new Random(/* seed= */ 0); + int sampleCount = 100; + ByteBuffer sampleBuffer; + if (pcmEncoding == C.ENCODING_PCM_DOUBLE || pcmEncoding == C.ENCODING_PCM_DOUBLE_BIG_ENDIAN) { + sampleBuffer = createByteBuffer(buildDoubleTestSamples(sampleCount, random)); + } else if (pcmEncoding == C.ENCODING_PCM_FLOAT + || pcmEncoding == C.ENCODING_PCM_FLOAT_BIG_ENDIAN) { + sampleBuffer = createByteBuffer(buildFloatTestSamples(sampleCount, random)); + } else { + int bytesPerSample = getPcmFrameSize(pcmEncoding, /* channelCount= */ 1); + byte[] rawBytes = buildTestData(sampleCount * bytesPerSample, random); + sampleBuffer = createByteBuffer(rawBytes); + } + return sampleBuffer.order( + isPcmEncodingBigEndian(pcmEncoding) ? ByteOrder.BIG_ENDIAN : ByteOrder.LITTLE_ENDIAN); + } + + private static final class PcmCEncodingValuesProvider extends TestParameterValuesProvider { + @Override + protected ImmutableList provideValues(TestParameterValuesProvider.Context context) { + return ImmutableList.of( + value(C.ENCODING_PCM_FLOAT).withName("ENCODING_PCM_FLOAT"), + value(C.ENCODING_PCM_FLOAT_BIG_ENDIAN).withName("ENCODING_PCM_FLOAT_BIG_ENDIAN"), + value(C.ENCODING_PCM_DOUBLE).withName("ENCODING_PCM_DOUBLE"), + value(C.ENCODING_PCM_DOUBLE_BIG_ENDIAN).withName("ENCODING_PCM_DOUBLE_BIG_ENDIAN"), + value(C.ENCODING_PCM_24BIT).withName("ENCODING_PCM_24BIT"), + value(C.ENCODING_PCM_24BIT_BIG_ENDIAN).withName("ENCODING_PCM_24BIT_BIG_ENDIAN"), + value(C.ENCODING_PCM_32BIT).withName("ENCODING_PCM_32BIT"), + value(C.ENCODING_PCM_32BIT_BIG_ENDIAN).withName("ENCODING_PCM_32BIT_BIG_ENDIAN")); + } + } } diff --git a/libraries/muxer/src/test/java/androidx/media3/muxer/Mp4MuxerMetadataTest.java b/libraries/muxer/src/test/java/androidx/media3/muxer/Mp4MuxerMetadataTest.java index 92daeb37c65..e51965e35ad 100644 --- a/libraries/muxer/src/test/java/androidx/media3/muxer/Mp4MuxerMetadataTest.java +++ b/libraries/muxer/src/test/java/androidx/media3/muxer/Mp4MuxerMetadataTest.java @@ -20,9 +20,14 @@ import static androidx.media3.container.MdtaMetadataEntry.TYPE_INDICATOR_STRING; import static androidx.media3.muxer.MuxerTestUtil.FAKE_VIDEO_FORMAT; import static androidx.media3.muxer.MuxerTestUtil.XMP_SAMPLE_DATA; +import static com.google.common.truth.Truth.assertThat; import android.content.Context; import android.util.Pair; +import androidx.annotation.Nullable; +import androidx.media3.common.C; +import androidx.media3.common.Format; +import androidx.media3.common.Metadata; import androidx.media3.common.util.Util; import androidx.media3.container.MdtaMetadataEntry; import androidx.media3.container.Mp4LocationData; @@ -37,6 +42,7 @@ import androidx.media3.test.utils.TestUtil; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.Iterables; import java.nio.ByteBuffer; import org.junit.Rule; import org.junit.Test; @@ -174,6 +180,33 @@ public void writeMp4File_setLocation_setsSameLocation() throws Exception { MuxerTestUtil.getExpectedMp4DumpFilePath("mp4_with_location.mp4")); } + @Test + public void writeMp4File_setLocationWithAltitude_setsSameLocationWithAltitude() throws Exception { + String outputFilePath = temporaryFolder.newFile().getPath(); + float latitude = 33.0f; + float longitude = -120.0f; + float altitude = 150.5f; + + try (Mp4Muxer muxer = new Mp4Muxer.Builder(SeekableMuxerOutput.of(outputFilePath)).build()) { + muxer.addMetadataEntry( + new Mp4TimestampData( + /* creationTimestampSeconds= */ 1_000_000L, + /* modificationTimestampSeconds= */ 5_000_000L)); + int trackId = muxer.addTrack(/* sortKey= */ 0, FAKE_VIDEO_FORMAT); + muxer.writeSampleData(trackId, sampleAndSampleInfo.first, sampleAndSampleInfo.second); + muxer.addMetadataEntry(new Mp4LocationData(latitude, longitude, altitude)); + } + + FakeExtractorOutput fakeExtractorOutput = + TestUtil.extractAllSamplesFromFilePath( + new Mp4Extractor(new DefaultSubtitleParserFactory()), outputFilePath); + Format videoTrackFormat = + Iterables.getOnlyElement(fakeExtractorOutput.getTrackOutputsForType(C.TRACK_TYPE_VIDEO)) + .lastFormat; + Mp4LocationData locationData = findMetadataEntry(videoTrackFormat, Mp4LocationData.class); + assertThat(locationData).isEqualTo(new Mp4LocationData(latitude, longitude, altitude)); + } + @Test public void writeMp4File_locationNotSet_setsLocationToNull() throws Exception { String outputFilePath = temporaryFolder.newFile().getPath(); @@ -316,4 +349,19 @@ public void writeMp4File_addXmp_matchesExpected() throws Exception { DumpFileAsserts.assertOutput( context, dumpableBox, MuxerTestUtil.getExpectedMp4DumpFilePath("mp4_with_xmp.mp4")); } + + @Nullable + private static T findMetadataEntry( + Format format, Class metadataClazz) { + if (format.metadata == null) { + return null; + } + for (int i = 0; i < format.metadata.length(); i++) { + Metadata.Entry entry = format.metadata.get(i); + if (metadataClazz.isInstance(entry)) { + return metadataClazz.cast(entry); + } + } + return null; + } } diff --git a/libraries/muxer/src/test/java/androidx/media3/muxer/MuxerTestUtil.java b/libraries/muxer/src/test/java/androidx/media3/muxer/MuxerTestUtil.java index 185d1c60f08..e2849206d38 100644 --- a/libraries/muxer/src/test/java/androidx/media3/muxer/MuxerTestUtil.java +++ b/libraries/muxer/src/test/java/androidx/media3/muxer/MuxerTestUtil.java @@ -27,6 +27,7 @@ import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; import androidx.media3.common.util.MediaFormatUtil; +import androidx.media3.common.util.Util; import androidx.media3.inspector.MediaExtractorCompat; import com.google.common.collect.ImmutableList; import java.io.IOException; @@ -165,5 +166,23 @@ public static void feedInputDataToMuxer( extractor.release(); } + /** + * Returns whether the {@code ftyp} box at the start of {@code outputFileBytes} declares {@code + * compatibleBrand} as one of its compatible brands. + */ + public static boolean ftypBoxContainsCompatibleBrand( + byte[] outputFileBytes, String compatibleBrand) { + ByteBuffer buffer = ByteBuffer.wrap(outputFileBytes); + int ftypBoxSize = buffer.getInt(); + buffer.position(16); // Skip box size, "ftyp", major_brand and minor_version. + int compatibleBrandCode = Util.getIntegerCodeForString(compatibleBrand); + while (buffer.position() < ftypBoxSize) { + if (buffer.getInt() == compatibleBrandCode) { + return true; + } + } + return false; + } + private MuxerTestUtil() {} } diff --git a/libraries/muxer/src/test/java/androidx/media3/muxer/MuxerUtilTest.java b/libraries/muxer/src/test/java/androidx/media3/muxer/MuxerUtilTest.java new file mode 100644 index 00000000000..3bb0e9f0ef4 --- /dev/null +++ b/libraries/muxer/src/test/java/androidx/media3/muxer/MuxerUtilTest.java @@ -0,0 +1,144 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.muxer; + +import static androidx.media3.muxer.MuxerTestUtil.FAKE_AUDIO_FORMAT; +import static androidx.media3.muxer.MuxerTestUtil.FAKE_VIDEO_FORMAT; +import static com.google.common.truth.Truth.assertThat; + +import androidx.media3.common.Format; +import androidx.media3.common.MimeTypes; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link MuxerUtil}. */ +@RunWith(AndroidJUnit4.class) +public class MuxerUtilTest { + + @Test + public void getFtypCompatibleBrands_withNonDolbyTracks_returnsEmptyList() { + Track videoTrack = + new Track(/* trackId= */ 0, FAKE_VIDEO_FORMAT, /* sampleCopyEnabled= */ false); + Track audioTrack = + new Track(/* trackId= */ 1, FAKE_AUDIO_FORMAT, /* sampleCopyEnabled= */ false); + + ImmutableList compatibleBrands = + MuxerUtil.getFtypCompatibleBrands(ImmutableList.of(videoTrack, audioTrack)); + + assertThat(compatibleBrands).isEmpty(); + } + + @Test + public void getFtypCompatibleBrands_withDolbyVisionTrack_returnsDby1() { + Track dolbyVisionTrack = + new Track( + /* trackId= */ 0, + FAKE_VIDEO_FORMAT.buildUpon().setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION).build(), + /* sampleCopyEnabled= */ false); + + ImmutableList compatibleBrands = + MuxerUtil.getFtypCompatibleBrands(ImmutableList.of(dolbyVisionTrack)); + + assertThat(compatibleBrands).containsExactly("dby1"); + } + + @Test + public void getFtypCompatibleBrands_withMixedDolbyAndNonDolbyTracks_returnsDby1() { + Track videoTrack = + new Track(/* trackId= */ 0, FAKE_VIDEO_FORMAT, /* sampleCopyEnabled= */ false); + Track dolbyVisionTrack = + new Track( + /* trackId= */ 1, + FAKE_VIDEO_FORMAT.buildUpon().setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION).build(), + /* sampleCopyEnabled= */ false); + Track audioTrack = + new Track(/* trackId= */ 2, FAKE_AUDIO_FORMAT, /* sampleCopyEnabled= */ false); + + ImmutableList compatibleBrands = + MuxerUtil.getFtypCompatibleBrands( + ImmutableList.of(videoTrack, dolbyVisionTrack, audioTrack)); + + assertThat(compatibleBrands).containsExactly("dby1"); + } + + @Test + public void getFtypCompatibleBrands_withMultipleDolbyTracks_returnsSingleDby1() { + Format dolbyVisionFormat = + FAKE_VIDEO_FORMAT.buildUpon().setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION).build(); + Track dolbyVisionTrack1 = + new Track(/* trackId= */ 0, dolbyVisionFormat, /* sampleCopyEnabled= */ false); + Track dolbyVisionTrack2 = + new Track(/* trackId= */ 1, dolbyVisionFormat, /* sampleCopyEnabled= */ false); + Track ac4Track = + new Track( + /* trackId= */ 2, + FAKE_AUDIO_FORMAT.buildUpon().setSampleMimeType(MimeTypes.AUDIO_AC4).build(), + /* sampleCopyEnabled= */ false); + + ImmutableList compatibleBrands = + MuxerUtil.getFtypCompatibleBrands( + ImmutableList.of(dolbyVisionTrack1, dolbyVisionTrack2, ac4Track)); + + assertThat(compatibleBrands).containsExactly("dby1"); + } + + @Test + public void getFtypCompatibleBrands_withAc3Track_returnsDby1() { + assertThatTrackWithMimeTypeProducesDby1(MimeTypes.AUDIO_AC3); + } + + @Test + public void getFtypCompatibleBrands_withAc4Track_returnsDby1() { + assertThatTrackWithMimeTypeProducesDby1(MimeTypes.AUDIO_AC4); + } + + @Test + public void getFtypCompatibleBrands_withEac3Track_returnsDby1() { + assertThatTrackWithMimeTypeProducesDby1(MimeTypes.AUDIO_E_AC3); + } + + @Test + public void getFtypCompatibleBrands_withEac3JocTrack_returnsDby1() { + assertThatTrackWithMimeTypeProducesDby1(MimeTypes.AUDIO_E_AC3_JOC); + } + + @Test + public void getFtypCompatibleBrands_withTrueHdTrack_returnsDby1() { + assertThatTrackWithMimeTypeProducesDby1(MimeTypes.AUDIO_TRUEHD); + } + + @Test + public void getFtypCompatibleBrands_withEmptyTrackList_returnsEmptyList() { + ImmutableList compatibleBrands = MuxerUtil.getFtypCompatibleBrands(ImmutableList.of()); + + assertThat(compatibleBrands).isEmpty(); + } + + private static void assertThatTrackWithMimeTypeProducesDby1(String mimeType) { + Track track = + new Track( + /* trackId= */ 0, + FAKE_AUDIO_FORMAT.buildUpon().setSampleMimeType(mimeType).build(), + /* sampleCopyEnabled= */ false); + + ImmutableList compatibleBrands = + MuxerUtil.getFtypCompatibleBrands(ImmutableList.of(track)); + + assertThat(compatibleBrands).containsExactly("dby1"); + } +} diff --git a/libraries/muxer/src/test/java/androidx/media3/muxer/SeekableMuxerOutputTest.java b/libraries/muxer/src/test/java/androidx/media3/muxer/SeekableMuxerOutputTest.java new file mode 100644 index 00000000000..5d781fade9d --- /dev/null +++ b/libraries/muxer/src/test/java/androidx/media3/muxer/SeekableMuxerOutputTest.java @@ -0,0 +1,74 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.muxer; + +import static com.google.common.truth.Truth.assertThat; + +import android.os.ParcelFileDescriptor; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.io.File; +import java.nio.ByteBuffer; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import org.junit.runner.RunWith; + +/** Unit tests for {@link SeekableMuxerOutput}. */ +@RunWith(AndroidJUnit4.class) +public final class SeekableMuxerOutputTest { + + @Rule public final TemporaryFolder temporaryFolder = new TemporaryFolder(); + + @Test + public void ofAndClose_withParcelFileDescriptor_keepsDescriptorOpenUntilOutputClosed() + throws Exception { + File tempFile = temporaryFolder.newFile("test_output.mp4"); + + ParcelFileDescriptor pfd = + ParcelFileDescriptor.open(tempFile, ParcelFileDescriptor.MODE_READ_WRITE); + try (SeekableMuxerOutput output = SeekableMuxerOutput.of(pfd)) { + assertThat(output).isNotNull(); + assertThat(pfd.getFileDescriptor().valid()).isTrue(); + } + assertThat(pfd.getFileDescriptor().valid()).isFalse(); + } + + @Test + public void of_withParcelFileDescriptor_supportsSeekableOperations() throws Exception { + File tempFile = temporaryFolder.newFile("test_operations.mp4"); + + ParcelFileDescriptor pfd = + ParcelFileDescriptor.open(tempFile, ParcelFileDescriptor.MODE_READ_WRITE); + try (SeekableMuxerOutput output = SeekableMuxerOutput.of(pfd)) { + byte[] data = new byte[] {1, 2, 3, 4, 5}; + output.write(ByteBuffer.wrap(data)); + assertThat(output.getPosition()).isEqualTo(5); + assertThat(output.getSize()).isEqualTo(5); + + output.setPosition(2); + assertThat(output.getPosition()).isEqualTo(2); + + output.write(ByteBuffer.wrap(new byte[] {9, 9})); + assertThat(output.getPosition()).isEqualTo(4); + + output.truncate(3); + assertThat(output.getSize()).isEqualTo(3); + } + + assertThat(pfd.getFileDescriptor().valid()).isFalse(); + assertThat(tempFile.length()).isEqualTo(3); + } +} diff --git a/libraries/muxer/src/test/java/androidx/media3/muxer/WebmMuxerEndToEndTest.java b/libraries/muxer/src/test/java/androidx/media3/muxer/WebmMuxerEndToEndTest.java index 990e63ec8cb..57ca7990926 100644 --- a/libraries/muxer/src/test/java/androidx/media3/muxer/WebmMuxerEndToEndTest.java +++ b/libraries/muxer/src/test/java/androidx/media3/muxer/WebmMuxerEndToEndTest.java @@ -17,8 +17,12 @@ import static androidx.media3.muxer.MuxerTestUtil.feedInputDataToMuxer; import static com.google.common.base.Preconditions.checkNotNull; +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; import android.content.Context; +import androidx.media3.common.Format; +import androidx.media3.common.MimeTypes; import androidx.media3.extractor.mkv.MatroskaExtractor; import androidx.media3.extractor.text.DefaultSubtitleParserFactory; import androidx.media3.test.utils.DumpFileAsserts; @@ -27,6 +31,7 @@ import androidx.test.core.app.ApplicationProvider; import com.google.testing.junit.testparameterinjector.TestParameter; import java.io.FileOutputStream; +import java.nio.ByteBuffer; import org.junit.Rule; import org.junit.Test; import org.junit.rules.TemporaryFolder; @@ -80,4 +85,59 @@ public void createWebmFile_fromInputFileSampleData_matchesExpected() throws Exce fakeExtractorOutput, MuxerTestUtil.getExpectedDumpFilePath("webm/" + testFile.fileName)); } + + @Test + public void close_noSamplesWritten_createsEmptyFile() throws Exception { + String outputFilePath = temporaryFolder.newFile("empty.webm").getPath(); + Format format = new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_VP9).build(); + + try (WebmMuxer muxer = + new WebmMuxer.Builder(SeekableMuxerOutput.of(new FileOutputStream(outputFilePath))) + .build()) { + int unusedTrackId = muxer.addTrack(format); + } + + byte[] outputFileBytes = TestUtil.getByteArrayFromFilePath(outputFilePath); + assertThat(outputFileBytes).isEmpty(); + } + + @Test + public void addTrack_afterClose_throws() throws Exception { + WebmMuxer muxer = + new WebmMuxer.Builder( + SeekableMuxerOutput.of(new FileOutputStream(temporaryFolder.newFile()))) + .build(); + muxer.close(); + Format format = new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_VP9).build(); + + assertThrows(IllegalStateException.class, () -> muxer.addTrack(format)); + } + + @Test + public void writeSampleData_afterClose_throws() throws Exception { + WebmMuxer muxer = + new WebmMuxer.Builder( + SeekableMuxerOutput.of(new FileOutputStream(temporaryFolder.newFile()))) + .build(); + Format format = new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_VP9).build(); + int trackId = muxer.addTrack(format); + muxer.close(); + BufferInfo bufferInfo = + new BufferInfo(/* presentationTimeUs= */ 0, /* size= */ 0, /* flags= */ 0); + ByteBuffer sampleData = ByteBuffer.allocate(0); + + assertThrows( + IllegalStateException.class, () -> muxer.writeSampleData(trackId, sampleData, bufferInfo)); + } + + @Test + public void close_afterClose_throws() throws Exception { + WebmMuxer muxer = + new WebmMuxer.Builder( + SeekableMuxerOutput.of(new FileOutputStream(temporaryFolder.newFile()))) + .build(); + muxer.close(); + + assertThrows(IllegalStateException.class, muxer::close); + } } diff --git a/libraries/session/src/main/java/androidx/media3/session/ConnectedControllersManager.java b/libraries/session/src/main/java/androidx/media3/session/ConnectedControllersManager.java index 2218875e86d..ca0c5f980e1 100644 --- a/libraries/session/src/main/java/androidx/media3/session/ConnectedControllersManager.java +++ b/libraries/session/src/main/java/androidx/media3/session/ConnectedControllersManager.java @@ -336,9 +336,11 @@ public Tracks getLastSentTracks(ControllerInfo controllerInfo) { public void updateLastSentTimelineAndTracks( ControllerInfo controllerInfo, Timeline timeline, Tracks tracks) { verifyApplicationThread(); - ConnectedControllerRecord record = checkNotNull(controllerRecords.get(controllerInfo)); - record.lastSentTimeline = timeline; - record.lastSentTracks = tracks; + @Nullable ConnectedControllerRecord record = controllerRecords.get(controllerInfo); + if (record != null) { + record.lastSentTimeline = timeline; + record.lastSentTracks = tracks; + } } public void addToCommandQueue( diff --git a/libraries/session/src/main/java/androidx/media3/session/LegacyConversions.java b/libraries/session/src/main/java/androidx/media3/session/LegacyConversions.java index 53817e47a57..a9fa94d899c 100644 --- a/libraries/session/src/main/java/androidx/media3/session/LegacyConversions.java +++ b/libraries/session/src/main/java/androidx/media3/session/LegacyConversions.java @@ -87,6 +87,8 @@ import androidx.media3.session.legacy.PlaybackStateCompat.CustomAction; import androidx.media3.session.legacy.RatingCompat; import androidx.media3.session.legacy.VolumeProviderCompat; +import com.google.common.base.Joiner; +import com.google.common.base.Splitter; import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableSet; import java.util.ArrayList; @@ -134,7 +136,11 @@ MediaMetadataCompat.METADATA_KEY_BT_FOLDER_TYPE, MediaMetadataCompat.METADATA_KEY_ADVERTISEMENT, MediaMetadataCompat.METADATA_KEY_DOWNLOAD_STATUS, - MediaConstants.EXTRAS_KEY_MEDIA_TYPE_COMPAT); + MediaConstants.EXTRAS_KEY_MEDIA_TYPE_COMPAT, + MediaConstants.EXTRAS_KEY_PLAYLIST_ID, + MediaConstants.EXTRAS_KEY_PLAYLIST_TITLE, + MediaConstants.EXTRAS_KEY_PLAYLIST_ARTWORK_URI, + MediaConstants.EXTRAS_KEY_CATEGORIES); /** Exception thrown when the conversion between legacy and Media3 states fails. */ public static class ConversionException extends Exception { @@ -477,10 +483,57 @@ public static Period convertToPeriod(int windowIndex) { /** Creates {@link MediaMetadata} from the {@link CharSequence queue title}. */ public static MediaMetadata convertToMediaMetadata(@Nullable CharSequence queueTitle) { - if (queueTitle == null) { + return convertToPlaylistMetadata( + queueTitle, /* metadataCompat= */ null, /* descriptionCompat= */ null); + } + + /** + * Creates playlist {@link MediaMetadata} from the {@link CharSequence queue title} and fallback + * extras in {@link MediaMetadataCompat} or {@link MediaDescriptionCompat}. + * + * @param queueTitle The queue title of the platform session, or {@code null} if not set. + * @param metadataCompat The metadata of the current item being played from the platform session, + * or {@code null} if not available. + * @param descriptionCompat The description of the current item from the queue of the platform + * session, or {@code null} if not available. + * @return The converted playlist {@link MediaMetadata}, or {@link MediaMetadata#EMPTY} if neither + * title nor artwork URI is available. + */ + public static MediaMetadata convertToPlaylistMetadata( + @Nullable CharSequence queueTitle, + @Nullable MediaMetadataCompat metadataCompat, + @Nullable MediaDescriptionCompat descriptionCompat) { + @Nullable + Bundle descriptionExtras = descriptionCompat != null ? descriptionCompat.getExtras() : null; + + @Nullable String artworkUriString = null; + if (metadataCompat != null) { + artworkUriString = metadataCompat.getString(MediaConstants.EXTRAS_KEY_PLAYLIST_ARTWORK_URI); + } + if (TextUtils.isEmpty(artworkUriString) && descriptionExtras != null) { + artworkUriString = + descriptionExtras.getString(MediaConstants.EXTRAS_KEY_PLAYLIST_ARTWORK_URI); + } + + @Nullable CharSequence title = queueTitle; + if (TextUtils.isEmpty(title) && metadataCompat != null) { + title = metadataCompat.getText(MediaConstants.EXTRAS_KEY_PLAYLIST_TITLE); + } + if (TextUtils.isEmpty(title) && descriptionExtras != null) { + title = descriptionExtras.getCharSequence(MediaConstants.EXTRAS_KEY_PLAYLIST_TITLE); + } + + if (TextUtils.isEmpty(title) && TextUtils.isEmpty(artworkUriString)) { return MediaMetadata.EMPTY; } - return new MediaMetadata.Builder().setTitle(queueTitle).build(); + MediaMetadata.Builder builder = new MediaMetadata.Builder(); + if (!TextUtils.isEmpty(title)) { + builder.setTitle(title); + } + if (!TextUtils.isEmpty(artworkUriString)) { + builder.setArtworkUri(Uri.parse(artworkUriString)); + } + return builder.build(); } public static MediaMetadata convertToMediaMetadata( @@ -527,6 +580,28 @@ private static MediaMetadata convertToMediaMetadata( extras.remove(MediaConstants.EXTRAS_KEY_MEDIA_TYPE_COMPAT); } + if (extras != null && extras.containsKey(MediaConstants.EXTRAS_KEY_PLAYLIST_ID)) { + @Nullable String playlistId = extras.getString(MediaConstants.EXTRAS_KEY_PLAYLIST_ID); + if (!TextUtils.isEmpty(playlistId)) { + builder.setPlaylistId(playlistId); + } + extras.remove(MediaConstants.EXTRAS_KEY_PLAYLIST_ID); + } + + if (extras != null) { + extras.remove(MediaConstants.EXTRAS_KEY_PLAYLIST_TITLE); + extras.remove(MediaConstants.EXTRAS_KEY_PLAYLIST_ARTWORK_URI); + } + + if (extras != null && extras.containsKey(MediaConstants.EXTRAS_KEY_CATEGORIES)) { + @Nullable String categories = extras.getString(MediaConstants.EXTRAS_KEY_CATEGORIES); + if (!TextUtils.isEmpty(categories)) { + builder.setCategories( + Splitter.on(',').omitEmptyStrings().trimResults().splitToList(categories)); + } + extras.remove(MediaConstants.EXTRAS_KEY_CATEGORIES); + } + if (extras != null && extras.containsKey(DESCRIPTION_EXTRAS_KEY_CUSTOM_BROWSER_ACTION_ID_LIST)) { builder.setSupportedCommands( @@ -649,6 +724,21 @@ public static MediaMetadata convertToMediaMetadata( (int) metadataCompat.getLong(MediaConstants.EXTRAS_KEY_MEDIA_TYPE_COMPAT)); } + if (metadataCompat.containsKey(MediaConstants.EXTRAS_KEY_PLAYLIST_ID)) { + @Nullable String playlistId = metadataCompat.getString(MediaConstants.EXTRAS_KEY_PLAYLIST_ID); + if (!TextUtils.isEmpty(playlistId)) { + builder.setPlaylistId(playlistId); + } + } + + if (metadataCompat.containsKey(MediaConstants.EXTRAS_KEY_CATEGORIES)) { + @Nullable String categories = metadataCompat.getString(MediaConstants.EXTRAS_KEY_CATEGORIES); + if (!TextUtils.isEmpty(categories)) { + builder.setCategories( + Splitter.on(',').omitEmptyStrings().trimResults().splitToList(categories)); + } + } + builder.setIsPlayable(true); Bundle extras = metadataCompat.getBundle(); @@ -752,6 +842,15 @@ public static MediaMetadataCompat convertToMediaMetadataCompat( convertToExtraBtFolderType(metadata.folderType)); } + if (metadata.playlistId != null && !TextUtils.isEmpty(metadata.playlistId)) { + builder.putString(MediaConstants.EXTRAS_KEY_PLAYLIST_ID, metadata.playlistId); + } + + if (!metadata.categories.isEmpty()) { + builder.putString( + MediaConstants.EXTRAS_KEY_CATEGORIES, Joiner.on(',').join(metadata.categories)); + } + if (durationMs == C.TIME_UNSET && metadata.durationMs != null) { // If the actual media duration is unknown, use the manually declared value if available. durationMs = metadata.durationMs; @@ -823,6 +922,22 @@ public static MediaDescriptionCompat convertToMediaDescriptionCompat( MediaConstants.EXTRAS_KEY_MEDIA_TYPE_COMPAT, checkNotNull(metadata.mediaType)); } } + + if (metadata.playlistId != null && !TextUtils.isEmpty(metadata.playlistId)) { + if (extras == null) { + extras = new Bundle(); + } + extras.putString(MediaConstants.EXTRAS_KEY_PLAYLIST_ID, metadata.playlistId); + } + + if (!metadata.categories.isEmpty()) { + if (extras == null) { + extras = new Bundle(); + } + extras.putString( + MediaConstants.EXTRAS_KEY_CATEGORIES, Joiner.on(',').join(metadata.categories)); + } + if (!metadata.supportedCommands.isEmpty()) { if (extras == null) { extras = new Bundle(); @@ -1570,7 +1685,8 @@ public static DeviceInfo convertToDeviceInfo( == MediaControllerCompat.PlaybackInfo.PLAYBACK_TYPE_REMOTE ? DeviceInfo.PLAYBACK_TYPE_REMOTE : DeviceInfo.PLAYBACK_TYPE_LOCAL) - .setMaxVolume(playbackInfoCompat.getMaxVolume()) + // Legacy sessions may report an invalid negative maximum volume. Treat it as unspecified. + .setMaxVolume(max(0, playbackInfoCompat.getMaxVolume())) .setRoutingControllerId(playbackInfoCompat.getVolumeControlId()) .build(); } diff --git a/libraries/session/src/main/java/androidx/media3/session/MediaBrowser.java b/libraries/session/src/main/java/androidx/media3/session/MediaBrowser.java index 4f656494763..b36f2d76939 100644 --- a/libraries/session/src/main/java/androidx/media3/session/MediaBrowser.java +++ b/libraries/session/src/main/java/androidx/media3/session/MediaBrowser.java @@ -50,6 +50,12 @@ /** * Browses media content offered by a {@link MediaLibraryService} in addition to the {@link * MediaController} functions. + * + *

      Backward interoperability

      + * + *

      The browser is backwards-compatible when connected to a legacy {@code + * androidx.media.MediaBrowserServiceCompat} or {@code android.service.media.MediaBrowserService} + * (unless otherwise stated). */ public final class MediaBrowser extends MediaController { diff --git a/libraries/session/src/main/java/androidx/media3/session/MediaConstants.java b/libraries/session/src/main/java/androidx/media3/session/MediaConstants.java index 777d27e2717..5efc5dcb09c 100644 --- a/libraries/session/src/main/java/androidx/media3/session/MediaConstants.java +++ b/libraries/session/src/main/java/androidx/media3/session/MediaConstants.java @@ -21,6 +21,7 @@ import android.os.Parcelable; import androidx.media3.common.MediaItem; import androidx.media3.common.MediaMetadata; +import androidx.media3.common.Player; import androidx.media3.common.util.UnstableApi; import androidx.media3.session.MediaLibraryService.LibraryParams; import androidx.media3.session.MediaLibraryService.MediaLibrarySession; @@ -480,6 +481,67 @@ public final class MediaConstants { public static final String EXTRAS_KEY_MEDIA_TYPE_COMPAT = "androidx.media3.session.EXTRAS_KEY_MEDIA_TYPE_COMPAT"; + /** + * {@link Bundle} key used to indicate the suggested content type in {@link LibraryParams#extras} + * of a {@link MediaLibrarySession.Callback#onGetChildren(MediaLibrarySession, + * MediaSession.ControllerInfo, String, int, int, LibraryParams)} request. + * + *

      TYPE: String. + */ + @UnstableApi + public static final String EXTRAS_KEY_SUGGESTED_TYPE = + "androidx.media3.session.EXTRAS_KEY_SUGGESTED_TYPE"; + + /** + * {@link Bundle} key used to indicate the cache time-to-live (TTL) in seconds for suggestions in + * {@link LibraryParams#extras} of a {@link LibraryResult} returned by {@link + * MediaLibrarySession.Callback#onGetChildren(MediaLibrarySession, MediaSession.ControllerInfo, + * String, int, int, LibraryParams)}. + * + *

      TYPE: long. + */ + @UnstableApi + public static final String EXTRAS_KEY_TTL_SECONDS = + "androidx.media3.session.EXTRAS_KEY_TTL_SECONDS"; + + /** + * {@link Bundle} key used to indicate the {@link MediaMetadata#playlistId} in the legacy {@code + * android.support.v4.media.MediaDescriptionCompat} as a String extra and as a String value in + * {@code android.support.v4.media.MediaMetadataCompat}. + */ + @UnstableApi + public static final String EXTRAS_KEY_PLAYLIST_ID = + "androidx.media3.session.EXTRAS_KEY_PLAYLIST_ID"; + + /** + * {@link Bundle} key used to indicate the playlist title (corresponding to {@link + * Player#getPlaylistMetadata()}{@code .title}) in the legacy {@code + * android.support.v4.media.MediaDescriptionCompat} as a String extra and as a String value in + * {@code android.support.v4.media.MediaMetadataCompat}. + */ + @UnstableApi + public static final String EXTRAS_KEY_PLAYLIST_TITLE = + "androidx.media3.session.EXTRAS_KEY_PLAYLIST_TITLE"; + + /** + * {@link Bundle} key used to indicate the playlist artwork URI (corresponding to {@link + * Player#getPlaylistMetadata()}{@code .artworkUri}) in the legacy {@code + * android.support.v4.media.MediaDescriptionCompat} as a String extra and as a String value in + * {@code android.support.v4.media.MediaMetadataCompat}. + */ + @UnstableApi + public static final String EXTRAS_KEY_PLAYLIST_ARTWORK_URI = + "androidx.media3.session.EXTRAS_KEY_PLAYLIST_ARTWORK_URI"; + + /** + * {@link Bundle} key used to indicate the {@link MediaMetadata#categories} in the legacy {@code + * android.support.v4.media.MediaDescriptionCompat} as a comma-separated String extra and as a + * comma-separated String value in {@code android.support.v4.media.MediaMetadataCompat}. + */ + @UnstableApi + public static final String EXTRAS_KEY_CATEGORIES = + "androidx.media3.session.EXTRAS_KEY_CATEGORIES"; + /** * {@link Bundle} key used to indicate the {@link CommandButton.Icon} in the extras of the * platform {@link android.media.session.PlaybackState.CustomAction}. The corresponding value diff --git a/libraries/session/src/main/java/androidx/media3/session/MediaController.java b/libraries/session/src/main/java/androidx/media3/session/MediaController.java index 5248b8b4519..c9827c75935 100644 --- a/libraries/session/src/main/java/androidx/media3/session/MediaController.java +++ b/libraries/session/src/main/java/androidx/media3/session/MediaController.java @@ -41,6 +41,7 @@ import androidx.media3.common.AudioAttributes; import androidx.media3.common.C; import androidx.media3.common.DeviceInfo; +import androidx.media3.common.Flags; import androidx.media3.common.MediaItem; import androidx.media3.common.MediaLibraryInfo; import androidx.media3.common.MediaMetadata; @@ -170,6 +171,31 @@ * MediaSessionCompat.Callback.onRemoveQueueItem} callbacks. Check {@link #getAvailableCommands()} * to see if playlist modifications are {@linkplain * androidx.media3.common.Player#COMMAND_CHANGE_MEDIA_ITEMS supported} by the legacy session. + * + *

      When connected to a platform or legacy media session, some {@link Player} methods and features + * are not supported by the underlying legacy API: + * + *

        + *
      • Tracks and track selection: {@link #getCurrentTracks()} returns {@link + * androidx.media3.common.Tracks#EMPTY}, {@link #getTrackSelectionParameters()} returns {@link + * androidx.media3.common.TrackSelectionParameters#DEFAULT}, and {@link + * #setTrackSelectionParameters(TrackSelectionParameters)} is ignored. + *
      • Video surfaces: Setting a video surface via {@link #setVideoSurface(Surface)}, {@link + * #setVideoSurfaceHolder(SurfaceHolder)}, {@link #setVideoSurfaceView(SurfaceView)}, or + * {@link #setVideoTextureView(TextureView)} logs a warning and is ignored. + *
      • Player volume: {@link #getVolume()} returns {@code 1.0f}, and {@link #setVolume(float)} + * logs a warning and is ignored. Note that device volume control via {@link + * #setDeviceVolume(int)} is supported if the session provides remote volume handling. + *
      • Text cues: {@link #getCurrentCues()} returns {@link + * androidx.media3.common.text.CueGroup#EMPTY_TIME_ZERO}. + *
      • Playback suppression: {@link #getPlaybackSuppressionReason()} returns {@link + * Player#PLAYBACK_SUPPRESSION_REASON_NONE}. + *
      • Ad information: {@link #getCurrentAdGroupIndex()} and {@link #getCurrentAdIndexInAdGroup()} + * return {@link androidx.media3.common.C#INDEX_UNSET}. + *
      • Playlist metadata: {@link #setPlaylistMetadata(MediaMetadata)} is ignored. + *
      • Ratings: {@link #setRating(String, Rating)} is only supported for the currently playing + * media item; calling it for any other item is ignored. + *
      */ @DoNotMock public class MediaController implements Player { @@ -640,7 +666,8 @@ default void onError(MediaController controller, SessionError sessionError) {} + MediaLibraryInfo.VERSION_SLASHY + "] [" + Util.DEVICE_DEBUG_INFO - + "]"); + + "] " + + Flags.getLogString()); // Initialize default values. window = new Timeline.Window(); @@ -951,6 +978,13 @@ public final boolean getPlayWhenReady() { return isConnected() && impl.getPlayWhenReady(); } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, playback suppression reasons are not + * supported and this method always returns {@link Player#PLAYBACK_SUPPRESSION_REASON_NONE}. + */ @Override public final @PlaybackSuppressionReason int getPlaybackSuppressionReason() { verifyApplicationThread(); @@ -1060,12 +1094,26 @@ public final boolean isPlayingAd() { return isConnected() && impl.isPlayingAd(); } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, ad playback is not supported and this + * method always returns {@link androidx.media3.common.C#INDEX_UNSET}. + */ @Override public final int getCurrentAdGroupIndex() { verifyApplicationThread(); return isConnected() ? impl.getCurrentAdGroupIndex() : C.INDEX_UNSET; } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, ad playback is not supported and this + * method always returns {@link androidx.media3.common.C#INDEX_UNSET}. + */ @Override public final int getCurrentAdIndexInAdGroup() { verifyApplicationThread(); @@ -1126,6 +1174,11 @@ public final int getAudioSessionId() { * *

      If the user rating was {@code null}, the media item does not accept setting user rating. * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, setting a rating is only supported for + * the currently playing media item. If {@code mediaId} does not match the current media item's + * media ID, the call is ignored. + * * @param mediaId The non-empty {@link MediaItem#mediaId}. * @param rating The rating to set. * @return A {@link ListenableFuture} of {@link SessionResult} representing the pending @@ -1169,10 +1222,11 @@ public final ListenableFuture setRating(Rating rating) { *

      A command is not accepted if it is not a custom command or the command is not in the list of * {@linkplain #getAvailableSessionCommands() available session commands}. * - *

      Interoperability: When connected to {@code + *

      Interoperability: When sending a custom command to {@code * android.support.v4.media.session.MediaSessionCompat}, {@link SessionResult#resultCode} will - * return the custom result code from the {@code android.os.ResultReceiver#onReceiveResult(int, - * Bundle)} instead of the standard result codes defined in the {@link SessionResult}. + * always be {@link SessionResult#RESULT_SUCCESS} because the session has no way to send a result + * back from {@code + * android.support.v4.media.session.MediaSessionCompat.Callback#onCustomAction(String, Bundle)}. * * @param command The custom command. * @param args The additional arguments. May be empty. @@ -1199,10 +1253,12 @@ public final ListenableFuture sendCustomCommand( * Whether or not the session sends progress updates depends on the implementation of the session * callback that responds to the given {@link SessionCommand}. * - *

      Interoperability: When connected to {@code + *

      Interoperability: When sending a custom command to {@code * android.support.v4.media.session.MediaSessionCompat}, {@link SessionResult#resultCode} will - * return the custom result code from the {@code android.os.ResultReceiver#onReceiveResult(int, - * Bundle)} instead of the standard result codes defined in the {@link SessionResult}. + * always be {@link SessionResult#RESULT_SUCCESS} because the session has no way to send a result + * back from {@code + * android.support.v4.media.session.MediaSessionCompat.Callback#onCustomAction(String, Bundle)}, + * and {@code progressListener} is ignored. * * @param command The custom command. * @param args The additional arguments. May be empty. @@ -1237,8 +1293,8 @@ public final ListenableFuture sendCustomCommand( *

      Interoperability: When sending a custom command to {@code * android.support.v4.media.session.MediaSessionCompat}, {@link SessionResult#resultCode} will * always be {@link SessionResult#RESULT_SUCCESS} because the session has no way to send a result - * back from {@link - * androidx.media3.session.legacy.MediaSessionCompat.Callback#onCustomAction(String, Bundle)}. + * back from {@code + * android.support.v4.media.session.MediaSessionCompat.Callback#onCustomAction(String, Bundle)}. * * @param command The custom command. * @param mediaItem The media item for which the command is sent. @@ -1269,8 +1325,9 @@ public final ListenableFuture sendCustomCommand( *

      Interoperability: When sending a custom command to {@code * android.support.v4.media.session.MediaSessionCompat}, {@link SessionResult#resultCode} will * always be {@link SessionResult#RESULT_SUCCESS} because the session has no way to send a result - * back from {@link - * androidx.media3.session.legacy.MediaSessionCompat.Callback#onCustomAction(String, Bundle)}. + * back from {@code + * android.support.v4.media.session.MediaSessionCompat.Callback#onCustomAction(String, Bundle)}, + * and {@code progressListener} is ignored. * * @param command The custom command. * @param mediaItem The media item for which the command is sent. @@ -1431,6 +1488,13 @@ public final void setMediaItems( impl.setMediaItems(mediaItems, startIndex, startPositionMs); } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, setting playlist metadata is not + * supported and calls to this method are ignored. + */ @Override public final void setPlaylistMetadata(MediaMetadata playlistMetadata) { verifyApplicationThread(); @@ -1841,6 +1905,13 @@ public final Size getSurfaceSize() { return isConnected() ? impl.getSurfaceSize() : Size.UNKNOWN; } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, video surfaces are not supported and + * calls to this method are ignored. + */ @Override public final void clearVideoSurface() { verifyApplicationThread(); @@ -1851,6 +1922,13 @@ public final void clearVideoSurface() { impl.clearVideoSurface(); } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, video surfaces are not supported and + * calls to this method are ignored. + */ @Override public final void clearVideoSurface(@Nullable Surface surface) { verifyApplicationThread(); @@ -1861,6 +1939,13 @@ public final void clearVideoSurface(@Nullable Surface surface) { impl.clearVideoSurface(surface); } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, setting a video surface is not supported + * and calls to this method are ignored. + */ @Override public final void setVideoSurface(@Nullable Surface surface) { verifyApplicationThread(); @@ -1871,6 +1956,13 @@ public final void setVideoSurface(@Nullable Surface surface) { impl.setVideoSurface(surface); } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, setting a video surface is not supported + * and calls to this method are ignored. + */ @Override public final void setVideoSurfaceHolder(@Nullable SurfaceHolder surfaceHolder) { verifyApplicationThread(); @@ -1881,6 +1973,13 @@ public final void setVideoSurfaceHolder(@Nullable SurfaceHolder surfaceHolder) { impl.setVideoSurfaceHolder(surfaceHolder); } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, video surfaces are not supported and + * calls to this method are ignored. + */ @Override public final void clearVideoSurfaceHolder(@Nullable SurfaceHolder surfaceHolder) { verifyApplicationThread(); @@ -1891,6 +1990,13 @@ public final void clearVideoSurfaceHolder(@Nullable SurfaceHolder surfaceHolder) impl.clearVideoSurfaceHolder(surfaceHolder); } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, setting a video surface is not supported + * and calls to this method are ignored. + */ @Override public final void setVideoSurfaceView(@Nullable SurfaceView surfaceView) { verifyApplicationThread(); @@ -1901,6 +2007,13 @@ public final void setVideoSurfaceView(@Nullable SurfaceView surfaceView) { impl.setVideoSurfaceView(surfaceView); } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, video surfaces are not supported and + * calls to this method are ignored. + */ @Override public final void clearVideoSurfaceView(@Nullable SurfaceView surfaceView) { verifyApplicationThread(); @@ -1911,6 +2024,13 @@ public final void clearVideoSurfaceView(@Nullable SurfaceView surfaceView) { impl.clearVideoSurfaceView(surfaceView); } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, setting a video surface is not supported + * and calls to this method are ignored. + */ @Override public final void setVideoTextureView(@Nullable TextureView textureView) { verifyApplicationThread(); @@ -1921,6 +2041,13 @@ public final void setVideoTextureView(@Nullable TextureView textureView) { impl.setVideoTextureView(textureView); } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, video surfaces are not supported and + * calls to this method are ignored. + */ @Override public final void clearVideoTextureView(@Nullable TextureView textureView) { verifyApplicationThread(); @@ -1931,12 +2058,26 @@ public final void clearVideoTextureView(@Nullable TextureView textureView) { impl.clearVideoTextureView(textureView); } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, text cues are not supported and this + * method returns {@link CueGroup#EMPTY_TIME_ZERO}. + */ @Override public final CueGroup getCurrentCues() { verifyApplicationThread(); return isConnected() ? impl.getCurrentCues() : CueGroup.EMPTY_TIME_ZERO; } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, player volume is not supported and this + * method returns {@code 1.0f}. Remote device volume is accessible via {@link #getDeviceVolume()}. + */ @Override @FloatRange(from = 0, to = 1) public final float getVolume() { @@ -1944,6 +2085,14 @@ public final float getVolume() { return isConnected() ? impl.getVolume() : 1; } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, setting player volume is not supported + * and calls to this method are ignored. Remote device volume can be controlled via {@link + * #setDeviceVolume(int)}. + */ @Override public final void setVolume(@FloatRange(from = 0, to = 1) float volume) { verifyApplicationThread(); @@ -1955,6 +2104,14 @@ public final void setVolume(@FloatRange(from = 0, to = 1) float volume) { impl.setVolume(volume); } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, muting the player is not supported and + * calls to this method are ignored. Remote device volume can be muted via {@link + * #setDeviceMuted(boolean)}. + */ @Override public final void mute() { verifyApplicationThread(); @@ -1965,6 +2122,14 @@ public final void mute() { impl.mute(); } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, unmuting the player is not supported and + * calls to this method are ignored. Remote device volume can be unmuted via {@link + * #setDeviceMuted(boolean)}. + */ @Override public final void unmute() { verifyApplicationThread(); @@ -2115,12 +2280,26 @@ public final MediaMetadata getMediaMetadata() { return isConnected() ? impl.getMediaMetadata() : MediaMetadata.EMPTY; } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, tracks are not supported and this method + * returns {@link Tracks#EMPTY}. + */ @Override public final Tracks getCurrentTracks() { verifyApplicationThread(); return isConnected() ? impl.getCurrentTracks() : Tracks.EMPTY; } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, track selection parameters are not + * supported and this method returns {@link TrackSelectionParameters#DEFAULT}. + */ @Override public final TrackSelectionParameters getTrackSelectionParameters() { verifyApplicationThread(); @@ -2130,6 +2309,13 @@ public final TrackSelectionParameters getTrackSelectionParameters() { return impl.getTrackSelectionParameters(); } + /** + * {@inheritDoc} + * + *

      Interoperability: When connected to {@code + * android.support.v4.media.session.MediaSessionCompat}, track selection parameters are not + * supported and calls to this method are ignored. + */ @Override public final void setTrackSelectionParameters(TrackSelectionParameters parameters) { verifyApplicationThread(); diff --git a/libraries/session/src/main/java/androidx/media3/session/MediaControllerImplBase.java b/libraries/session/src/main/java/androidx/media3/session/MediaControllerImplBase.java index 7a95a8aef3d..0c12e1a5db8 100644 --- a/libraries/session/src/main/java/androidx/media3/session/MediaControllerImplBase.java +++ b/libraries/session/src/main/java/androidx/media3/session/MediaControllerImplBase.java @@ -2974,11 +2974,17 @@ void onPlayerInfoChanged(PlayerInfo newPlayerInfo, BundlingExclusions bundlingEx checkNotNull(connectedToken)); PlayerInfo finalPlayerInfo = playerInfo; + // Discontinuity events are only emitted if both timelines are not empty, or if it's a seek + // discontinuity which is emitted even when the player is empty. + boolean seekOrNonEmptyTimeline = + finalPlayerInfo.discontinuityReason == DISCONTINUITY_REASON_SEEK + || (!oldPlayerInfo.timeline.isEmpty() && !finalPlayerInfo.timeline.isEmpty()); @Nullable @Player.DiscontinuityReason Integer positionDiscontinuityReason = - (!oldPlayerInfo.oldPositionInfo.equals(newPlayerInfo.oldPositionInfo) - || !oldPlayerInfo.newPositionInfo.equals(newPlayerInfo.newPositionInfo)) + (seekOrNonEmptyTimeline + && (!oldPlayerInfo.oldPositionInfo.equals(finalPlayerInfo.oldPositionInfo) + || !oldPlayerInfo.newPositionInfo.equals(finalPlayerInfo.newPositionInfo))) ? finalPlayerInfo.discontinuityReason : null; diff --git a/libraries/session/src/main/java/androidx/media3/session/MediaControllerImplLegacy.java b/libraries/session/src/main/java/androidx/media3/session/MediaControllerImplLegacy.java index b5a6984c8e1..afe96214c70 100644 --- a/libraries/session/src/main/java/androidx/media3/session/MediaControllerImplLegacy.java +++ b/libraries/session/src/main/java/androidx/media3/session/MediaControllerImplLegacy.java @@ -68,6 +68,7 @@ import androidx.media3.session.LegacyConversions.ConversionException; import androidx.media3.session.legacy.MediaBrowserCompat; import androidx.media3.session.legacy.MediaControllerCompat; +import androidx.media3.session.legacy.MediaDescriptionCompat; import androidx.media3.session.legacy.MediaMetadataCompat; import androidx.media3.session.legacy.MediaSessionCompat; import androidx.media3.session.legacy.MediaSessionCompat.QueueItem; @@ -281,7 +282,7 @@ public void prepare() { /* discontinuityReason= */ null, /* mediaItemTransitionReason= */ null); - if (canInitializeLegacyPlaylist(controllerInfo)) { + if (canInitializeLegacyPlaylist(controllerInfo, legacyPlayerInfo.playbackStateCompat)) { initializeLegacyPlaylist(); } } @@ -701,8 +702,11 @@ public void setMediaItems(List mediaItems, int startIndex, long start maskedControllerInfo, /* discontinuityReason= */ null, /* mediaItemTransitionReason= */ null); - if (canInitializeLegacyPlaylist(controllerInfo)) { + if (canInitializeLegacyPlaylist(controllerInfo, legacyPlayerInfo.playbackStateCompat)) { initializeLegacyPlaylist(); + } else { + // Wait to initialize the new playlist (via playFrom*) until playWhenReady becomes true. + hasInitializedLegacyPlaylist = false; } } @@ -1318,7 +1322,7 @@ public void setPlayWhenReady(boolean playWhenReady) { } else { controllerCompat.getTransportControls().pause(); } - } else if (canInitializeLegacyPlaylist(controllerInfo)) { + } else if (canInitializeLegacyPlaylist(controllerInfo, legacyPlayerInfo.playbackStateCompat)) { initializeLegacyPlaylist(); } } @@ -1457,11 +1461,45 @@ private void connectToService() { }); } - private static boolean canInitializeLegacyPlaylist(ControllerInfo controllerInfo) { - return controllerInfo.playerInfo.playbackState != Player.STATE_IDLE - && !controllerInfo.playerInfo.timeline.isEmpty() - && (controllerInfo.availablePlayerCommands.contains(Player.COMMAND_PREPARE) - || controllerInfo.playerInfo.playWhenReady); + private static boolean isLegacyPlaylistInitialized(ControllerInfo controllerInfo) { + if (controllerInfo.playerInfo.playbackState == Player.STATE_IDLE + || controllerInfo.playerInfo.timeline.isEmpty()) { + return false; + } + QueueTimeline queueTimeline = (QueueTimeline) controllerInfo.playerInfo.timeline; + int currentIndex = controllerInfo.playerInfo.sessionPositionInfo.positionInfo.mediaItemIndex; + return queueTimeline.getQueueId(currentIndex) != QueueItem.UNKNOWN_ID; + } + + private static boolean canInitializeLegacyPlaylist( + ControllerInfo controllerInfo, @Nullable PlaybackStateCompat playbackStateCompat) { + if (controllerInfo.playerInfo.playbackState == Player.STATE_IDLE + || controllerInfo.playerInfo.timeline.isEmpty()) { + return false; + } + if (controllerInfo.playerInfo.playWhenReady) { + return true; + } + // When !playWhenReady, check whether the session supports the specific prepare action needed + // to eagerly initialize the playlist now (via prepare/prepareFrom*), or whether we must wait + // until playWhenReady becomes true to initialize via play/playFrom*. + if (playbackStateCompat == null) { + return false; + } + long actions = playbackStateCompat.getActions(); + QueueTimeline queueTimeline = (QueueTimeline) controllerInfo.playerInfo.timeline; + int currentIndex = controllerInfo.playerInfo.sessionPositionInfo.positionInfo.mediaItemIndex; + if (queueTimeline.getQueueId(currentIndex) != QueueItem.UNKNOWN_ID) { + return (actions & PlaybackStateCompat.ACTION_PREPARE) != 0; + } + MediaItem currentMediaItem = queueTimeline.getWindow(currentIndex, new Window()).mediaItem; + if (currentMediaItem.requestMetadata.mediaUri != null) { + return (actions & PlaybackStateCompat.ACTION_PREPARE_FROM_URI) != 0; + } else if (currentMediaItem.requestMetadata.searchQuery != null) { + return (actions & PlaybackStateCompat.ACTION_PREPARE_FROM_SEARCH) != 0; + } else { + return (actions & PlaybackStateCompat.ACTION_PREPARE_FROM_MEDIA_ID) != 0; + } } private void initializeLegacyPlaylist() { @@ -1665,7 +1703,7 @@ private void updateControllerInfo( if (hasNewLegacyPlayerInfo) { legacyPlayerInfo = newLegacyPlayerInfo; pendingLegacyPlayerInfo = legacyPlayerInfo; - if (canInitializeLegacyPlaylist(newControllerInfo)) { + if (isLegacyPlaylistInitialized(newControllerInfo)) { // New platform state already has initialized playlist. hasInitializedLegacyPlaylist = true; } @@ -1703,7 +1741,8 @@ private void updateControllerInfo( newControllerInfo.playerInfo.timelineChangeReason)); } - if (!TextUtils.equals(oldLegacyPlayerInfo.queueTitle, newLegacyPlayerInfo.queueTitle)) { + if (!oldControllerInfo.playerInfo.playlistMetadata.equals( + newControllerInfo.playerInfo.playlistMetadata)) { listeners.queueEvent( Player.EVENT_PLAYLIST_METADATA_CHANGED, (listener) -> @@ -2124,10 +2163,24 @@ private static ControllerInfo buildNewControllerInfo( sessionFlags, isSessionReady); - playlistMetadata = - oldLegacyPlayerInfo.queueTitle == newLegacyPlayerInfo.queueTitle - ? oldControllerInfo.playerInfo.playlistMetadata - : LegacyConversions.convertToMediaMetadata(newLegacyPlayerInfo.queueTitle); + if (!TextUtils.equals(oldLegacyPlayerInfo.queueTitle, newLegacyPlayerInfo.queueTitle) + || isMetadataCompatChanged + || isCurrentActiveQueueIdChanged + || isQueueChanged) { + int activeQueueIndex = findQueueItemIndex(newLegacyPlayerInfo.queue, newActiveQueueId); + @Nullable + MediaDescriptionCompat currentDescriptionCompat = + activeQueueIndex != C.INDEX_UNSET + ? newLegacyPlayerInfo.queue.get(activeQueueIndex).getDescription() + : null; + playlistMetadata = + LegacyConversions.convertToPlaylistMetadata( + newLegacyPlayerInfo.queueTitle, + newLegacyPlayerInfo.mediaMetadataCompat, + currentDescriptionCompat); + } else { + playlistMetadata = oldControllerInfo.playerInfo.playlistMetadata; + } repeatMode = LegacyConversions.convertToRepeatMode(newLegacyPlayerInfo.repeatMode); shuffleModeEnabled = LegacyConversions.convertToShuffleModeEnabled(newLegacyPlayerInfo.shuffleMode); diff --git a/libraries/session/src/main/java/androidx/media3/session/MediaLibraryService.java b/libraries/session/src/main/java/androidx/media3/session/MediaLibraryService.java index 4712828ece7..5a7099dbb71 100644 --- a/libraries/session/src/main/java/androidx/media3/session/MediaLibraryService.java +++ b/libraries/session/src/main/java/androidx/media3/session/MediaLibraryService.java @@ -187,7 +187,9 @@ public interface Callback extends MediaSession.Callback { * androidx.media.MediaBrowserServiceCompat.BrowserRoot}, then the main thread may be blocked * until the returned future is done. If your service may be queried by a legacy {@code * android.support.v4.media.MediaBrowserCompat}, you should ensure that the future completes - * quickly to avoid blocking the main thread for a long period of time. + * quickly to avoid blocking the main thread for a long period of time. In particular, the + * returned future must not be completed on or by posting to the application thread (or main + * thread), as this will cause a deadlock while the thread is blocked. * * @param session The session for this event. * @param browser The browser information. diff --git a/libraries/session/src/main/java/androidx/media3/session/MediaLibraryServiceLegacyStub.java b/libraries/session/src/main/java/androidx/media3/session/MediaLibraryServiceLegacyStub.java index f706da2691a..2624cb63fbd 100644 --- a/libraries/session/src/main/java/androidx/media3/session/MediaLibraryServiceLegacyStub.java +++ b/libraries/session/src/main/java/androidx/media3/session/MediaLibraryServiceLegacyStub.java @@ -120,47 +120,48 @@ private void getRootOnHandler( haveRoot.open(); return; } + boolean isSearchSessionCommandAvailable = + getConnectedControllersManager() + .isSessionCommandAvailable(controller, SessionCommand.COMMAND_CODE_LIBRARY_SEARCH); + ImmutableList commandButtonsForMediaItems = + librarySessionImpl.getCommandButtonsForMediaItems(); + @Nullable ArrayList browserActionBundles = null; + if (!commandButtonsForMediaItems.isEmpty()) { + browserActionBundles = new ArrayList<>(); + for (int i = 0; i < commandButtonsForMediaItems.size(); i++) { + CommandButton commandButton = commandButtonsForMediaItems.get(i); + if (commandButton.sessionCommand != null + && commandButton.sessionCommand.commandCode == SessionCommand.COMMAND_CODE_CUSTOM) { + browserActionBundles.add(LegacyConversions.convertToBundle(commandButton)); + } + } + } @Nullable LibraryParams params = LegacyConversions.convertToLibraryParams(librarySessionImpl.getContext(), rootHints); ListenableFuture> future = librarySessionImpl.onGetLibraryRootOnHandler(controller, params); + @Nullable ArrayList finalBrowserActionBundles = browserActionBundles; Futures.addCallback( future, new FutureCallback>() { @Override public void onSuccess(@Nullable LibraryResult result) { + // Running on arbitrary future callback. if (result != null && result.resultCode == RESULT_SUCCESS && result.value != null) { @Nullable Bundle extras = result.params != null ? LegacyConversions.convertToRootHints(result.params) : new Bundle(); - boolean isSearchSessionCommandAvailable = - getConnectedControllersManager() - .isSessionCommandAvailable( - controller, SessionCommand.COMMAND_CODE_LIBRARY_SEARCH); checkNotNull(extras) .putBoolean( BROWSER_SERVICE_EXTRAS_KEY_SEARCH_SUPPORTED, isSearchSessionCommandAvailable); - ImmutableList commandButtonsForMediaItems = - librarySessionImpl.getCommandButtonsForMediaItems(); - if (!commandButtonsForMediaItems.isEmpty()) { - ArrayList browserActionBundles = new ArrayList<>(); - for (int i = 0; i < commandButtonsForMediaItems.size(); i++) { - CommandButton commandButton = commandButtonsForMediaItems.get(i); - if (commandButton.sessionCommand != null - && commandButton.sessionCommand.commandCode - == SessionCommand.COMMAND_CODE_CUSTOM) { - browserActionBundles.add(LegacyConversions.convertToBundle(commandButton)); - } - } - if (!browserActionBundles.isEmpty()) { - extras.putParcelableArrayList( - BROWSER_SERVICE_EXTRAS_KEY_CUSTOM_BROWSER_ACTION_ROOT_LIST, - browserActionBundles); - } + if (finalBrowserActionBundles != null && !finalBrowserActionBundles.isEmpty()) { + extras.putParcelableArrayList( + BROWSER_SERVICE_EXTRAS_KEY_CUSTOM_BROWSER_ACTION_ROOT_LIST, + finalBrowserActionBundles); } rootReference.set(new BrowserRoot(result.value.mediaId, extras)); } else { @@ -174,12 +175,13 @@ public void onSuccess(@Nullable LibraryResult result) { @Override public void onFailure(Throwable t) { + // Running on arbitrary future callback. Log.e(TAG, "Couldn't get a result from onGetLibraryRoot", t); rootReference.set(MediaUtils.defaultBrowserRoot); haveRoot.open(); } }, - this::postOrRunOnApplicationHandler); + MoreExecutors.directExecutor()); } // TODO(b/192455639): Optimize potential multiple calls of diff --git a/libraries/session/src/main/java/androidx/media3/session/MediaNotificationManager.java b/libraries/session/src/main/java/androidx/media3/session/MediaNotificationManager.java index 89050a3878c..0e5bfcb355b 100644 --- a/libraries/session/src/main/java/androidx/media3/session/MediaNotificationManager.java +++ b/libraries/session/src/main/java/androidx/media3/session/MediaNotificationManager.java @@ -24,6 +24,7 @@ import android.app.ForegroundServiceStartNotAllowedException; import android.app.Notification; import android.app.NotificationManager; +import android.app.Service; import android.content.Context; import android.content.Intent; import android.content.pm.ServiceInfo; @@ -356,7 +357,7 @@ private void updateNotificationInternal( // Notification manager has to be updated first to avoid missing updates // (https://github.com/androidx/media/issues/192). notificationManager.notify(mediaNotification.notificationId, mediaNotification.notification); - Util.stopForeground(mediaSessionService, /* removeNotification= */ false); + mediaSessionService.stopForeground(Service.STOP_FOREGROUND_DETACH); } } @@ -364,7 +365,7 @@ private void updateNotificationInternal( private void removeNotification() { // To hide the notification on all API levels, we need to call both Service.stopForeground(true) // and notificationManagerCompat.cancel(notificationId). - Util.stopForeground(mediaSessionService, /* removeNotification= */ true); + mediaSessionService.stopForeground(Service.STOP_FOREGROUND_REMOVE); if (mediaNotification != null) { notificationManager.cancel(mediaNotification.notificationId); // Update the notification count so that if a pending notification callback arrives (e.g., a diff --git a/libraries/session/src/main/java/androidx/media3/session/MediaSession.java b/libraries/session/src/main/java/androidx/media3/session/MediaSession.java index ffc8692eafe..5b89128e42b 100644 --- a/libraries/session/src/main/java/androidx/media3/session/MediaSession.java +++ b/libraries/session/src/main/java/androidx/media3/session/MediaSession.java @@ -228,6 +228,38 @@ * android.media.session.MediaController} and legacy {@code * android.support.v4.media.session.MediaControllerCompat} instances. * + *

      While Media3 allows setting controller-specific configurations (such as available commands, + * custom layout, media button preferences, session activity, session extras, and playback + * exceptions) for each connected {@link MediaController}, all platform and legacy controllers + * interact with this single, shared platform session. + * + *

      Because all platform and legacy controllers share this one platform session, + * controller-specific settings cannot be pushed to individual platform or legacy controllers (calls + * to controller-specific setters with a {@linkplain ControllerInfo#LEGACY_CONTROLLER_VERSION legacy + * controller} are ignored). Instead, to dynamically configure the shared platform session, apps + * can: + * + *

        + *
      • Use {@linkplain #getMediaNotificationControllerInfo() the media notification controller} as + * the target controller in controller-specific methods (such as {@link + * #setAvailableCommands(ControllerInfo, SessionCommands, Player.Commands)}, {@link + * #setMediaButtonPreferences(ControllerInfo, List)}, {@link #setCustomLayout(ControllerInfo, + * List)}, {@link #setSessionActivity(ControllerInfo, PendingIntent)}, {@link + * #setSessionExtras(ControllerInfo, Bundle)}, or {@link #setPlaybackException(ControllerInfo, + * PlaybackException)}). When a media notification controller is connected, settings applied + * to it are routed directly to the shared platform session and reflected across all platform + * controllers. + *
      • Use session-wide methods such as {@link #setMediaButtonPreferences(List)}, {@link + * #setCustomLayout(List)}, {@link #setSessionExtras(Bundle)}, {@link + * #setSessionActivity(PendingIntent)}, {@link #setPlaybackException(PlaybackException)}, + * {@link #sendError(SessionError)}, and {@link #broadcastCustomCommand(SessionCommand, + * Bundle)}. + *
      • Configure initial commands, media button preferences, custom layout, session extras, or + * session activity for the platform session when the media notification controller connects + * in {@link Callback#onConnect} or {@link Callback#onConnectAsync} via {@link + * ConnectionResult.AcceptedResultBuilder}. + *
      + * *

      Backward compatibility with platform and legacy controller APIs *

      * @@ -936,6 +968,9 @@ public final PendingIntent getSessionActivity() { *

      Note: When a controller is connected to the session that has a version smaller than 1.6.0, * then setting the session activity to null has no effect on the controller side. * + *

      Interoperability: Setting the session activity also updates the session activity of the + * shared platform session. + * * @param activityPendingIntent The pending intent to start the session activity or null. * @throws IllegalArgumentException if the {@link PendingIntent} passed into this method is * {@linkplain PendingIntent#getActivity(Context, int, Intent, int) not an activity}. @@ -955,8 +990,9 @@ public final void setSessionActivity(@Nullable PendingIntent activityPendingInte * *

      Interoperability: This call has no effect when called for a {@linkplain * ControllerInfo#LEGACY_CONTROLLER_VERSION platform or legacy controller}. To set the session - * activity of the platform session use {@linkplain #getMediaNotificationControllerInfo() the - * media notification controller} as the target controller. + * activity of the platform session, use {@linkplain #getMediaNotificationControllerInfo() the + * media notification controller} as the target controller or use {@link + * #setSessionActivity(PendingIntent)}. * * @param controller The controller to send the session activity to. * @param activityPendingIntent The pending intent to start the session activity. @@ -1037,6 +1073,13 @@ public final SessionToken getToken() { /** * Returns the list of connected controllers. * + *

      Interoperability: The returned list includes connected platform or legacy controllers (where + * {@link ControllerInfo#getControllerVersion()} equals {@link + * ControllerInfo#LEGACY_CONTROLLER_VERSION}). Note that when the media notification controller is + * connected, requests from System UI are attributed to {@linkplain + * #getMediaNotificationControllerInfo() the media notification controller} and System UI is not + * included in this list. + * *

      This method must be called from the thread associated with the {@linkplain * Player#getApplicationLooper() application looper}. */ @@ -1055,6 +1098,10 @@ public final List getConnectedControllers() { * ConnectionResult#availablePlayerCommands available commands} in {@link Callback#onConnectAsync} * or set them via {@link #setAvailableCommands}. * + *

      Interoperability: For requests originating from System UI, this method returns the {@link + * ControllerInfo} of {@linkplain #getMediaNotificationControllerInfo() the media notification + * controller} when the notification controller is connected. + * *

      This method must be called on the {@linkplain Player#getApplicationLooper() application * thread} of the underlying player. * @@ -1107,6 +1154,13 @@ public boolean isMediaNotificationController(ControllerInfo controllerInfo) { * available commands of the underlying player to determine the playback actions of the platform * session (see {@code PlaybackStateCompat.getActions()}). * + *

      In addition to available commands and media button preferences, this controller info serves + * as the designated target to dynamically configure the single shared platform session for {@link + * #setSessionActivity(ControllerInfo, PendingIntent)}, {@link #setSessionExtras(ControllerInfo, + * Bundle)}, {@link #setPlaybackException(ControllerInfo, PlaybackException)}, and {@link + * #setCustomLayout(ControllerInfo, List)}. Configurations applied to this controller info update + * the shared platform session and reflect across all connected platform controllers. + * *

      This method must be called from the thread associated with the {@linkplain * Player#getApplicationLooper() application looper}. */ @@ -1121,6 +1175,13 @@ public ControllerInfo getMediaNotificationControllerInfo() { * *

      Note: This is not a security validation. * + *

      Interoperability: If an Automotive OS controller connects as a platform or legacy controller + * (see {@link ControllerInfo#LEGACY_CONTROLLER_VERSION}), it interacts via the shared platform + * session. In this case, controller-specific settings (such as custom layout or media button + * preferences) cannot be set directly for the controller; configure the shared platform session + * instead (e.g. via {@link #setMediaButtonPreferences(List)} or {@linkplain + * #getMediaNotificationControllerInfo() the media notification controller}). + * * @param controllerInfo The controller info of the connected controller. * @return True if the controller into belongs to a connected Automotive OS controller. */ @@ -1135,6 +1196,13 @@ public final boolean isAutomotiveController(ControllerInfo controllerInfo) { * *

      Note: This is not a security validation. * + *

      Interoperability: If an Android Auto companion app controller connects as a platform or + * legacy controller (see {@link ControllerInfo#LEGACY_CONTROLLER_VERSION}), it interacts via the + * shared platform session. In this case, controller-specific settings (such as custom layout or + * media button preferences) cannot be set directly for the controller; configure the shared + * platform session instead (e.g. via {@link #setMediaButtonPreferences(List)} or {@linkplain + * #getMediaNotificationControllerInfo() the media notification controller}). + * * @param controllerInfo The controller info of the connected controller. * @return True if the controller into belongs to a connected Auto companion client app. */ @@ -1168,7 +1236,9 @@ public final boolean isAutoCompanionController(ControllerInfo controllerInfo) { * {@code false} if the available commands of the controller do not allow to use a button. * *

      Interoperability: This call has no effect when called for a {@linkplain - * ControllerInfo#LEGACY_CONTROLLER_VERSION platform or legacy controller}. + * ControllerInfo#LEGACY_CONTROLLER_VERSION platform or legacy controller}. To set the custom + * layout of the platform session, use {@linkplain #getMediaNotificationControllerInfo() the media + * notification controller} as the target controller or use {@link #setCustomLayout(List)}. * * @param controller The controller for which to set the custom layout. * @param layout The ordered list of {@linkplain CommandButton command buttons}. @@ -1233,7 +1303,10 @@ public final void setCustomLayout(List layout) { * {@code false} if the available commands of the controller do not allow to use a button. * *

      Interoperability: This call has no effect when called for a {@linkplain - * ControllerInfo#LEGACY_CONTROLLER_VERSION platform or legacy controller}. + * ControllerInfo#LEGACY_CONTROLLER_VERSION platform or legacy controller}. To set the media + * button preferences of the platform session, use {@linkplain + * #getMediaNotificationControllerInfo() the media notification controller} as the target + * controller or use {@link #setMediaButtonPreferences(List)}. * * @param controller The controller for which to set the media button preferences. * @param mediaButtonPreferences The ordered list of {@linkplain CommandButton command buttons}. @@ -1290,6 +1363,12 @@ public final void setMediaButtonPreferences(List mediaButtonPrefe * PlaybackException#areErrorInfosEqual(PlaybackException, PlaybackException) equal error info} to * the previously set exception for the given controller, results in a no-op. * + *

      Interoperability: This call has no effect when called for a {@linkplain + * ControllerInfo#LEGACY_CONTROLLER_VERSION platform or legacy controller}. To set the playback + * exception of the platform session, use {@linkplain #getMediaNotificationControllerInfo() the + * media notification controller} as the target controller or use {@link + * #setPlaybackException(PlaybackException)}. + * * @param controllerInfo The controller for which to set the playback exception. * @param playbackException The {@link PlaybackException} or null. */ @@ -1326,6 +1405,13 @@ public final void setPlaybackException(@Nullable PlaybackException playbackExcep * Player#getAvailableCommands() available commands} of the underlying {@link Player} and the * controller will only be able to call the commonly available commands. * + *

      Interoperability: Calling this method for a {@linkplain + * ControllerInfo#LEGACY_CONTROLLER_VERSION platform or legacy controller} updates internal + * command permissions for that controller, but does not alter the visible playback actions or + * custom actions of the shared platform session. To update the available commands and actions of + * the shared platform session, use {@linkplain #getMediaNotificationControllerInfo() the media + * notification controller} as the target controller. + * * @param controller The controller to change allowed commands. * @param sessionCommands The new available session commands. * @param playerCommands The new available player commands. @@ -1432,7 +1518,9 @@ public final void setSessionExtras(Bundle sessionExtras) { *

      This call immediately returns and doesn't wait for a result from the controller. * *

      Interoperability: This call has no effect when called for a {@linkplain - * ControllerInfo#LEGACY_CONTROLLER_VERSION platform or legacy controller}. + * ControllerInfo#LEGACY_CONTROLLER_VERSION platform or legacy controller}. To set the session + * extras of the platform session, use {@linkplain #getMediaNotificationControllerInfo() the media + * notification controller} as the target controller or use {@link #setSessionExtras(Bundle)}. * * @param controller The controller to send the extras to. * @param sessionExtras The session extras. @@ -1498,8 +1586,6 @@ public final ListenableFuture sendCustomCommand( * * @param controllerInfo The controller to send the error to. * @param sessionError The session error. - * @throws IllegalArgumentException thrown if an error is attempted to be sent to a legacy - * controller. */ @UnstableApi public final void sendError(ControllerInfo controllerInfo, SessionError sessionError) { @@ -1669,6 +1755,14 @@ public interface Callback { * legacy controllers will connect to the session, you should ensure that the callback returns * quickly to avoid blocking the main thread for a long period of time. * + *

      Custom layouts, media button preferences, session extras, and session activities returned + * in the {@link ConnectionResult} for a platform or legacy controller are ignored by the + * platform session. To configure initial commands, media button preferences, or custom actions + * for the shared platform session, configure the {@link ConnectionResult} returned when + * {@linkplain #getMediaNotificationControllerInfo() the media notification controller} + * connects. When the notification controller is connected, System UI does not invoke this + * callback and automatically adopts the platform session configuration. + * * @param session The session for this event. * @param controller The {@linkplain ControllerInfo controller} information. * @return A {@link ListenableFuture} of the {@link ConnectionResult}. @@ -1717,7 +1811,17 @@ default ConnectionResult onConnect(MediaSession session, ControllerInfo controll * legacy controller} is connecting to the session then this callback may block the main thread, * even if it's called on a different application thread. If it's possible that platform or * legacy controllers will connect to the session, you should ensure that the callback returns - * quickly to avoid blocking the main thread for a long period of time. + * quickly to avoid blocking the main thread for a long period of time. In particular, the + * returned future must not be completed on or by posting to the application thread (or main + * thread), as this will cause a deadlock while the thread is blocked. + * + *

      Custom layouts, media button preferences, session extras, and session activities returned + * in the {@link ConnectionResult} for a platform or legacy controller are ignored by the + * platform session. To configure initial commands, media button preferences, or custom actions + * for the shared platform session, configure the {@link ConnectionResult} returned when + * {@linkplain #getMediaNotificationControllerInfo() the media notification controller} + * connects. When the notification controller is connected, System UI does not invoke this + * callback and automatically adopts the platform session configuration. * * @param session The session for this event. * @param controller The {@linkplain ControllerInfo controller} information. @@ -1827,11 +1931,15 @@ default ListenableFuture onSetRating( * available session commands} in {@link #onConnectAsync} or set via {@link * #setAvailableCommands}. * - *

      Interoperability: This will be also called by {@code - * android.support.v4.media.MediaBrowserCompat.sendCustomAction()}. If so, {@code extras} from - * {@code android.support.v4.media.MediaBrowserCompat.sendCustomAction()} will be considered as - * {@code args} and the custom command will have {@code null} {@link - * SessionCommand#customExtras}. + *

      Interoperability: This will also be called when custom action buttons in the media + * notification or on Android Auto / Automotive OS are clicked, or when legacy controller APIs + * are used (such as {@code MediaControllerCompat.TransportControls#sendCustomAction(String, + * Bundle)}, {@code MediaControllerCompat#sendCommand(String, Bundle, ResultReceiver)}, or + * {@code android.support.v4.media.MediaBrowserCompat#sendCustomAction(String, Bundle, + * CustomActionCallback)}). If the controller connects as a platform or legacy controller (note + * that the media notification, Android Auto, and Automotive OS may connect as either platform + * or Media3 controllers), the incoming extras will be considered as {@code args} and the custom + * command will have empty {@link SessionCommand#customExtras}. * *

      Return a {@link ListenableFuture} to send a {@link SessionResult} back to the controller * asynchronously. You can also return a {@link SessionResult} directly by using Guava's {@link @@ -1870,11 +1978,15 @@ default ListenableFuture onCustomCommand( * available session commands} in {@link #onConnectAsync} or set via {@link * #setAvailableCommands}. * - *

      Interoperability: This will be also called by {@code - * android.support.v4.media.MediaBrowserCompat.sendCustomAction()}. If so, {@code extras} from - * {@code android.support.v4.media.MediaBrowserCompat.sendCustomAction()} will be considered as - * {@code args} and the custom command will have {@code null} {@link - * SessionCommand#customExtras}. + *

      Interoperability: This will also be called when custom action buttons in the media + * notification or on Android Auto / Automotive OS are clicked, or when legacy controller APIs + * are used (such as {@code MediaControllerCompat.TransportControls#sendCustomAction(String, + * Bundle)}, {@code MediaControllerCompat#sendCommand(String, Bundle, ResultReceiver)}, or + * {@code android.support.v4.media.MediaBrowserCompat#sendCustomAction(String, Bundle, + * CustomActionCallback)}). If the controller connects as a platform or legacy controller (note + * that the media notification, Android Auto, and Automotive OS may connect as either platform + * or Media3 controllers), the incoming extras will be considered as {@code args} and the custom + * command will have empty {@link SessionCommand#customExtras}. * *

      Return a {@link ListenableFuture} to send a {@link SessionResult} back to the controller * asynchronously. You can also return a {@link SessionResult} directly by using Guava's {@link @@ -2303,6 +2415,14 @@ public AcceptedResultBuilder(MediaSession mediaSession, ControllerInfo controlle *

      The default is {@link ConnectionResult#DEFAULT_SESSION_AND_LIBRARY_COMMANDS} for a * {@link MediaLibrarySession} and {@link ConnectionResult#DEFAULT_SESSION_COMMANDS} for a * {@link MediaSession}. + * + *

      Interoperability: When accepting a connection from a {@linkplain + * ControllerInfo#LEGACY_CONTROLLER_VERSION platform or legacy controller}, this sets the + * internal command permissions for that controller, but does not alter the visible custom + * actions of the shared platform session. Setting available session commands when accepting + * {@linkplain MediaSession#getMediaNotificationControllerInfo() the media notification + * controller} sets the session commands of the shared platform session, which determines + * which custom actions are enabled on the platform session. */ @CanIgnoreReturnValue public AcceptedResultBuilder setAvailableSessionCommands( @@ -2321,6 +2441,16 @@ public AcceptedResultBuilder setAvailableSessionCommands( * available to a controller. * *

      The default is {@link ConnectionResult#DEFAULT_PLAYER_COMMANDS}. + * + *

      Interoperability: When accepting a connection from a {@linkplain + * ControllerInfo#LEGACY_CONTROLLER_VERSION platform or legacy controller}, this sets the + * internal player command permissions for that controller, but does not alter the visible + * playback actions of the shared platform session. Setting available player commands when + * accepting {@linkplain MediaSession#getMediaNotificationControllerInfo() the media + * notification controller} sets the available player commands of the shared platform session, + * which determines the visible playback actions ({@link + * android.support.v4.media.session.PlaybackStateCompat#getActions()}) of the platform + * session. */ @CanIgnoreReturnValue public AcceptedResultBuilder setAvailablePlayerCommands( @@ -2344,6 +2474,12 @@ public AcceptedResultBuilder setAvailablePlayerCommands( * commands}. On the controller side, the {@linkplain CommandButton#isEnabled enabled} flag is * set to {@code false} if the available commands of the controller do not allow to use a * button. + * + *

      Interoperability: This setting has no effect when accepting a connection from a + * {@linkplain ControllerInfo#LEGACY_CONTROLLER_VERSION platform or legacy controller}. + * Setting a custom layout when accepting {@linkplain + * MediaSession#getMediaNotificationControllerInfo() the media notification controller} sets + * the custom layout of the shared platform session. */ @CanIgnoreReturnValue public AcceptedResultBuilder setCustomLayout(@Nullable List customLayout) { @@ -2363,6 +2499,12 @@ public AcceptedResultBuilder setCustomLayout(@Nullable List custo * available session commands}. On the controller side, the {@linkplain * CommandButton#isEnabled enabled} flag is set to {@code false} if the available commands of * the controller do not allow to use a button. + * + *

      Interoperability: This setting has no effect when accepting a connection from a + * {@linkplain ControllerInfo#LEGACY_CONTROLLER_VERSION platform or legacy controller}. + * Setting media button preferences when accepting {@linkplain + * MediaSession#getMediaNotificationControllerInfo() the media notification controller} sets + * the media button preferences and custom actions of the shared platform session. */ @CanIgnoreReturnValue public AcceptedResultBuilder setMediaButtonPreferences( @@ -2377,6 +2519,12 @@ public AcceptedResultBuilder setMediaButtonPreferences( * of the session}. * *

      The default is null to indicate that the extras of the session should be used. + * + *

      Interoperability: This setting has no effect when accepting a connection from a + * {@linkplain ControllerInfo#LEGACY_CONTROLLER_VERSION platform or legacy controller}. + * Setting session extras when accepting {@linkplain + * MediaSession#getMediaNotificationControllerInfo() the media notification controller} sets + * the extras of the shared platform session. */ @CanIgnoreReturnValue public AcceptedResultBuilder setSessionExtras(Bundle sessionExtras) { @@ -2389,6 +2537,12 @@ public AcceptedResultBuilder setSessionExtras(Bundle sessionExtras) { * session activity of the session}. * *

      The default is null to indicate that the session activity of the session should be used. + * + *

      Interoperability: This setting has no effect when accepting a connection from a + * {@linkplain ControllerInfo#LEGACY_CONTROLLER_VERSION platform or legacy controller}. + * Setting a session activity when accepting {@linkplain + * MediaSession#getMediaNotificationControllerInfo() the media notification controller} sets + * the session activity of the shared platform session. */ @CanIgnoreReturnValue public AcceptedResultBuilder setSessionActivity(@Nullable PendingIntent sessionActivity) { diff --git a/libraries/session/src/main/java/androidx/media3/session/MediaSessionImpl.java b/libraries/session/src/main/java/androidx/media3/session/MediaSessionImpl.java index 63b11877395..fba75308925 100644 --- a/libraries/session/src/main/java/androidx/media3/session/MediaSessionImpl.java +++ b/libraries/session/src/main/java/androidx/media3/session/MediaSessionImpl.java @@ -28,6 +28,7 @@ import static android.view.KeyEvent.KEYCODE_MEDIA_STOP; import static androidx.media3.common.Player.COMMAND_CHANGE_MEDIA_ITEMS; import static androidx.media3.common.Player.COMMAND_SET_MEDIA_ITEM; +import static androidx.media3.common.Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED; import static androidx.media3.common.util.Util.postOrRun; import static androidx.media3.session.MediaSessionStub.UNKNOWN_SEQUENCE_NUMBER; import static androidx.media3.session.SessionError.ERROR_SESSION_DISCONNECTED; @@ -65,6 +66,7 @@ import androidx.concurrent.futures.ResolvableFuture; import androidx.media3.common.AudioAttributes; import androidx.media3.common.DeviceInfo; +import androidx.media3.common.Flags; import androidx.media3.common.MediaItem; import androidx.media3.common.MediaLibraryInfo; import androidx.media3.common.MediaMetadata; @@ -194,7 +196,8 @@ public MediaSessionImpl( + MediaLibraryInfo.VERSION_SLASHY + "] [" + Util.DEVICE_DEBUG_INFO - + "]"); + + "] " + + Flags.getLogString()); this.instance = instance; this.context = context; sessionId = id; @@ -650,10 +653,27 @@ public void setPlaybackException(@Nullable PlaybackException playbackException) // Do not check for equality and return as a no-op if equal. Some controller may have a // different exception set individually that we want to override. this.playbackException = playbackException; + boolean mediaNotificationControllerConnected = false; ImmutableList connectedControllers = sessionStub.getConnectedControllersManager().getConnectedControllers(); for (int i = 0; i < connectedControllers.size(); i++) { - setPlaybackExceptionOnHandler(connectedControllers.get(i), playbackException); + ControllerInfo controller = connectedControllers.get(i); + if (isMediaNotificationController(controller)) { + mediaNotificationControllerConnected = true; + } + setPlaybackExceptionOnHandler(controller, playbackException); + } + if (!mediaNotificationControllerConnected) { + sessionLegacyStub.setPlaybackException( + playbackException, + playbackException != null + ? createPlayerCommandsForCustomErrorState( + sessionLegacyStub.getAvailablePlayerCommands()) + : null); + if (playbackException == null) { + sessionLegacyStub.updateLegacySessionPlaybackState(playerWrapper); + sessionLegacyStub.maybeUpdateFlags(playerWrapper); + } } }); } @@ -1017,6 +1037,24 @@ private void updateLegacySessionConfigurationOnConnectOnHandler( } sessionLegacyStub.setAvailableCommands( connectionResult.availableSessionCommands, connectionResult.availablePlayerCommands); + if (playbackException != null) { + sessionLegacyStub.setPlaybackException( + playbackException, + createPlayerCommandsForCustomErrorState(connectionResult.availablePlayerCommands)); + } + Bundle sessionExtras = + connectionResult.sessionExtras != null + ? connectionResult.sessionExtras + : instance.getSessionExtras(); + dispatchRemoteControllerTaskToLegacyStub( + (callback, seq) -> callback.onSessionExtrasChanged(seq, sessionExtras)); + @Nullable + PendingIntent sessionActivity = + connectionResult.sessionActivity != null + ? connectionResult.sessionActivity + : instance.getSessionActivity(); + dispatchRemoteControllerTaskToLegacyStub( + (callback, seq) -> callback.onSessionActivityChanged(seq, sessionActivity)); } } @@ -1147,10 +1185,17 @@ protected void setSessionActivity(@Nullable PendingIntent sessionActivity) { postOrRunOnApplicationHandler( () -> { this.sessionActivity = sessionActivity; + dispatchRemoteControllerTaskToLegacyStub( + (callback, seq) -> callback.onSessionActivityChanged(seq, sessionActivity)); ImmutableList connectedControllers = sessionStub.getConnectedControllersManager().getConnectedControllers(); for (int i = 0; i < connectedControllers.size(); i++) { - setSessionActivityOnHandler(connectedControllers.get(i), sessionActivity); + ControllerInfo controller = connectedControllers.get(i); + if (controller.getControllerVersion() >= 3) { + dispatchRemoteControllerTaskWithoutReturn( + controller, + (callback, seq) -> callback.onSessionActivityChanged(seq, sessionActivity)); + } } }); } @@ -1158,20 +1203,20 @@ protected void setSessionActivity(@Nullable PendingIntent sessionActivity) { @UnstableApi protected void setSessionActivity( ControllerInfo controller, @Nullable PendingIntent sessionActivity) { - postOrRunOnApplicationHandler(() -> setSessionActivityOnHandler(controller, sessionActivity)); - } - - private void setSessionActivityOnHandler( - ControllerInfo controller, @Nullable PendingIntent sessionActivity) { - if (controller.getControllerVersion() >= 3 - && sessionStub.getConnectedControllersManager().isConnected(controller)) { - dispatchRemoteControllerTaskWithoutReturn( - controller, (callback, seq) -> callback.onSessionActivityChanged(seq, sessionActivity)); - if (isMediaNotificationController(controller)) { - dispatchRemoteControllerTaskToLegacyStub( - (callback, seq) -> callback.onSessionActivityChanged(seq, sessionActivity)); - } - } + postOrRunOnApplicationHandler( + () -> { + if (sessionStub.getConnectedControllersManager().isConnected(controller)) { + if (controller.getControllerVersion() >= 3) { + dispatchRemoteControllerTaskWithoutReturn( + controller, + (callback, seq) -> callback.onSessionActivityChanged(seq, sessionActivity)); + } + if (isMediaNotificationController(controller)) { + dispatchRemoteControllerTaskToLegacyStub( + (callback, seq) -> callback.onSessionActivityChanged(seq, sessionActivity)); + } + } + }); } protected ControllerInfo resolveControllerInfoForCallback(ControllerInfo controller) { @@ -1300,7 +1345,12 @@ protected MediaSessionServiceLegacyStub getLegacyBrowserService() { @Override public void onSuccess(Boolean result) { if (result) { - handleMediaControllerPlayRequestInternal(controller, callOnPlayerInteractionFinished); + callWithControllerForCurrentRequestSet( + controller, + () -> + handleMediaControllerPlayRequestInternal( + controller, callOnPlayerInteractionFinished)) + .run(); } } @@ -1378,10 +1428,15 @@ public void onFailure(Throwable t) { t); } // Play as requested even if playback resumption fails. - Util.handlePlayButtonAction(playerWrapper); - if (callOnPlayerInteractionFinished) { - onPlayerInteractionFinishedOnHandler(controllerForRequest, playCommand); - } + callWithControllerForCurrentRequestSet( + controllerForRequest, + () -> { + Util.handlePlayButtonAction(playerWrapper); + if (callOnPlayerInteractionFinished) { + onPlayerInteractionFinishedOnHandler(controllerForRequest, playCommand); + } + }) + .run(); } }, this::postOrRunOnApplicationHandler); @@ -2010,9 +2065,18 @@ public void onTimelineChanged(Timeline timeline, @Player.TimelineChangeReason in if (player == null) { return; } + SessionPositionInfo sessionPositionInfo = player.createSessionPositionInfo(); + if (reason == TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED + && session.playerInfo.timeline.isEmpty()) { + session.playerInfo = + new PlayerInfo.Builder(session.playerInfo) + .setOldPositionInfo(session.playerInfo.newPositionInfo) + .setNewPositionInfo(sessionPositionInfo.positionInfo) + .build(); + } session.playerInfo = session.playerInfo.copyWithTimelineAndSessionPositionInfo( - timeline, player.createSessionPositionInfo(), reason); + timeline, sessionPositionInfo, reason); session.onPlayerInfoChangedHandler.sendPlayerInfoChangedMessage( /* excludeTimeline= */ false, /* excludeTracks= */ true); session.dispatchRemoteControllerTaskToLegacyStub( diff --git a/libraries/session/src/main/java/androidx/media3/session/MediaSessionLegacyStub.java b/libraries/session/src/main/java/androidx/media3/session/MediaSessionLegacyStub.java index 3f162bc13d0..a03ede39b4a 100644 --- a/libraries/session/src/main/java/androidx/media3/session/MediaSessionLegacyStub.java +++ b/libraries/session/src/main/java/androidx/media3/session/MediaSessionLegacyStub.java @@ -88,7 +88,6 @@ import androidx.media3.common.Rating; import androidx.media3.common.Timeline; import androidx.media3.common.util.Log; -import androidx.media3.common.util.NullableType; import androidx.media3.common.util.Util; import androidx.media3.session.MediaSession.ControllerCb; import androidx.media3.session.MediaSession.ControllerInfo; @@ -109,6 +108,7 @@ import com.google.common.util.concurrent.ListenableFuture; import com.google.common.util.concurrent.MoreExecutors; import java.util.ArrayList; +import java.util.Arrays; import java.util.List; import java.util.Objects; import java.util.concurrent.CancellationException; @@ -148,7 +148,6 @@ private final boolean playIfSuppressed; private volatile long connectionTimeoutMs; - @Nullable private FutureCallback pendingBitmapLoadCallback; private int sessionFlags; @Nullable private LegacyError legacyError; private Bundle legacyExtras; @@ -200,74 +199,25 @@ public MediaSessionLegacyStub( updateCustomLayoutAndLegacyExtrasForMediaButtonPreferences(); } - // Assume an app that intentionally puts a `MediaButtonReceiver` into the manifest has - // implemented some kind of resumption of the last recently played media item. - broadcastReceiverComponentName = queryPackageManagerForMediaButtonReceiver(context); - @Nullable ComponentName receiverComponentName = broadcastReceiverComponentName; - boolean isReceiverComponentAService = false; - if (receiverComponentName == null || SDK_INT < 31) { - // Below API 26, media button events are sent to the receiver at runtime also. We always want - // these to arrive at the service at runtime. release() then set the receiver for restart if - // available. - receiverComponentName = - getServiceComponentByAction(context, MediaLibraryService.SERVICE_INTERFACE); - if (receiverComponentName == null) { - receiverComponentName = - getServiceComponentByAction(context, MediaSessionService.SERVICE_INTERFACE); - } - isReceiverComponentAService = - receiverComponentName != null - && !Objects.equals(receiverComponentName, broadcastReceiverComponentName); - } - Intent intent = new Intent(Intent.ACTION_MEDIA_BUTTON, sessionUri); - PendingIntent mediaButtonIntent; - if (receiverComponentName == null) { - // Neither a media button receiver from the app manifest nor a service available that could - // handle media button events. Create a runtime receiver and a pending intent for it. - runtimeBroadcastReceiver = new MediaButtonReceiver(); - IntentFilter filter = new IntentFilter(Intent.ACTION_MEDIA_BUTTON); - filter.addDataScheme(castNonNull(sessionUri.getScheme())); - Util.registerReceiverNotExported(context, runtimeBroadcastReceiver, filter); - // Create a pending intent to be broadcast to the receiver. - intent.setPackage(context.getPackageName()); - mediaButtonIntent = - PendingIntent.getBroadcast( - context, /* requestCode= */ 0, intent, PENDING_INTENT_FLAG_MUTABLE); - // Creates a fake ComponentName for MediaSessionCompat in pre-L or without a service. - receiverComponentName = new ComponentName(context, context.getClass()); - } else { - intent.setComponent(receiverComponentName); - mediaButtonIntent = - isReceiverComponentAService - ? (SDK_INT >= 26 - ? PendingIntent.getForegroundService( - context, /* requestCode= */ 0, intent, PENDING_INTENT_FLAG_MUTABLE) - : PendingIntent.getService( - context, /* requestCode= */ 0, intent, PENDING_INTENT_FLAG_MUTABLE)) - : PendingIntent.getBroadcast( - context, /* requestCode= */ 0, intent, PENDING_INTENT_FLAG_MUTABLE); - runtimeBroadcastReceiver = null; - } - String sessionCompatId = TextUtils.join( DEFAULT_MEDIA_SESSION_TAG_DELIM, new String[] {DEFAULT_MEDIA_SESSION_TAG_PREFIX, session.getId()}); sessionCompat = new MediaSessionCompat( - context, - sessionCompatId, - SDK_INT < 31 ? receiverComponentName : null, - SDK_INT < 31 ? mediaButtonIntent : null, - /* sessionInfo= */ tokenExtras, - packageNameOverride); - if (SDK_INT >= 31 && broadcastReceiverComponentName != null) { - Api31.setMediaButtonBroadcastReceiver(sessionCompat, broadcastReceiverComponentName); - } + context, sessionCompatId, /* sessionInfo= */ tokenExtras, packageNameOverride); + + // Assume an app that intentionally puts a `MediaButtonReceiver` into the manifest has + // implemented some kind of resumption of the last recently played media item. + broadcastReceiverComponentName = queryPackageManagerForMediaButtonReceiver(context); + runtimeBroadcastReceiver = initializeMediaButtonReceiver(context, sessionUri); if (sessionActivity != null) { sessionCompat.setSessionActivity(sessionActivity); } + if (!this.legacyExtras.isEmpty()) { + sessionCompat.setExtras(this.legacyExtras); + } @SuppressWarnings("nullness:assignment") @Initialized @@ -348,6 +298,16 @@ public void setPlatformMediaButtonPreferences( updateCustomLayoutAndLegacyExtrasForMediaButtonPreferencesAndInformExtrasChanged(); } + /** Returns the available player commands of the platform session. */ + public Player.Commands getAvailablePlayerCommands() { + return availablePlayerCommands; + } + + /** Returns the legacy extras of the platform session. */ + public Bundle getLegacyExtras() { + return new Bundle(legacyExtras); + } + /** * Sets or clears a playback exception override for the platform session. * @@ -441,6 +401,63 @@ private static ComponentName queryPackageManagerForMediaButtonReceiver(Context c } } + @SuppressWarnings("PendingIntentMutability") // We can't use SaferPendingIntent + @Nullable + private MediaButtonReceiver initializeMediaButtonReceiver(Context context, Uri sessionUri) { + if (SDK_INT >= 31) { + if (broadcastReceiverComponentName != null) { + Api31.setMediaButtonBroadcastReceiver(sessionCompat, broadcastReceiverComponentName); + } + return null; + } + // Below API 26, media button events are sent to the receiver at runtime also. We always want + // these to arrive at the service at runtime. release() then set the receiver for restart if + // available. + @Nullable + ComponentName serviceComponentName = + getServiceComponentByAction(context, MediaLibraryService.SERVICE_INTERFACE); + if (serviceComponentName == null) { + serviceComponentName = + getServiceComponentByAction(context, MediaSessionService.SERVICE_INTERFACE); + } + Intent intent = new Intent(Intent.ACTION_MEDIA_BUTTON, sessionUri); + try { + PendingIntent mediaButtonIntent; + if (serviceComponentName == null) { + // Neither a media button receiver from the app manifest nor a service available that could + // handle media button events. Create a runtime receiver and a pending intent for it. + intent.setPackage(context.getPackageName()); + mediaButtonIntent = + PendingIntent.getBroadcast( + context, /* requestCode= */ 0, intent, PENDING_INTENT_FLAG_MUTABLE); + } else { + intent.setComponent(serviceComponentName); + mediaButtonIntent = + SDK_INT >= 26 + ? PendingIntent.getForegroundService( + context, /* requestCode= */ 0, intent, PENDING_INTENT_FLAG_MUTABLE) + : PendingIntent.getService( + context, /* requestCode= */ 0, intent, PENDING_INTENT_FLAG_MUTABLE); + } + sessionCompat.setMediaButtonReceiver(mediaButtonIntent); + if (serviceComponentName == null) { + MediaButtonReceiver runtimeReceiver = new MediaButtonReceiver(); + IntentFilter filter = new IntentFilter(Intent.ACTION_MEDIA_BUTTON); + filter.addDataScheme(castNonNull(sessionUri.getScheme())); + Util.registerReceiverNotExported(context, runtimeReceiver, filter); + return runtimeReceiver; + } + return null; + } catch (SecurityException e) { + Log.w( + TAG, + "Failed to set media button receiver PendingIntent; media button events may not be" + + " delivered after releasing the session.", + e); + return null; + } + } + /** Starts to receive commands. */ public void start() { sessionCompat.setActive(true); @@ -451,18 +468,23 @@ public void release() { if (SDK_INT < 31) { if (broadcastReceiverComponentName == null) { // No broadcast receiver available. Playback resumption not supported. - setMediaButtonReceiver(sessionCompat, /* mediaButtonReceiverIntent= */ null); + sessionCompat.setMediaButtonReceiver(/* mbr= */ null); } else { - // Override the runtime receiver with the broadcast receiver for playback resumption. - Intent intent = new Intent(Intent.ACTION_MEDIA_BUTTON, sessionImpl.getUri()); - intent.setComponent(broadcastReceiverComponentName); - PendingIntent mediaButtonReceiverIntent = - PendingIntent.getBroadcast( - sessionImpl.getContext(), - /* requestCode= */ 0, - intent, - PENDING_INTENT_FLAG_MUTABLE); - setMediaButtonReceiver(sessionCompat, mediaButtonReceiverIntent); + try { + // Override the runtime receiver with the broadcast receiver for playback resumption. + Intent intent = new Intent(Intent.ACTION_MEDIA_BUTTON, sessionImpl.getUri()); + intent.setComponent(broadcastReceiverComponentName); + PendingIntent mediaButtonReceiverIntent = + PendingIntent.getBroadcast( + sessionImpl.getContext(), + /* requestCode= */ 0, + intent, + PENDING_INTENT_FLAG_MUTABLE); + sessionCompat.setMediaButtonReceiver(mediaButtonReceiverIntent); + } catch (SecurityException e) { + Log.w( + TAG, "Failed to set media button receiver PendingIntent for playback resumption.", e); + } } } if (runtimeBroadcastReceiver != null) { @@ -472,6 +494,7 @@ public void release() { androidAutoObserver.release(); } // No check for COMMAND_RELEASE needed as MediaControllers can always be released. + controllerLegacyCbForBroadcast.onDisconnected(/* seq= */ 0); sessionCompat.release(); connectedControllersManager.release(); } @@ -1243,12 +1266,6 @@ private static void setMetadata( sessionCompat.setMetadata(metadataCompat); } - @SuppressWarnings("nullness:argument") // MediaSessionCompat didn't annotate @Nullable. - private static void setMediaButtonReceiver( - MediaSessionCompat sessionCompat, @Nullable PendingIntent mediaButtonReceiverIntent) { - sessionCompat.setMediaButtonReceiver(mediaButtonReceiverIntent); - } - @SuppressWarnings("nullness:argument") // MediaSessionCompat didn't annotate @Nullable. private static void setQueue(MediaSessionCompat sessionCompat, @Nullable List queue) { sessionCompat.setQueue(queue); @@ -1387,11 +1404,15 @@ private final class ControllerLegacyCbForBroadcast implements ControllerCb { private String lastMediaId; @Nullable private Uri lastMediaUri; private long lastDurationMs; + @Nullable private FutureCallback pendingBitmapLoadCallback; + @Nullable private Runnable pendingBitmapFuturesTask; + private ArrayMap> queueBitmapFutures; public ControllerLegacyCbForBroadcast() { lastMediaMetadata = MediaMetadata.EMPTY; lastMediaId = MediaItem.DEFAULT_MEDIA_ID; lastDurationMs = C.TIME_UNSET; + queueBitmapFutures = new ArrayMap<>(); } /** @@ -1417,6 +1438,9 @@ public void onAvailableCommandsChangedFromPlayer(int seq, Player.Commands availa @Override public void onDisconnected(int seq) { // Calling MediaSessionCompat#release() is already done in release(). + pendingBitmapLoadCallback = null; + pendingBitmapFuturesTask = null; + queueBitmapFutures.clear(); } @Override @@ -1632,52 +1656,76 @@ public void onTimelineChanged( updateMetadataIfChanged(); } + @SuppressWarnings("FutureReturnValueIgnored") private void updateQueue(Timeline timeline) { + pendingBitmapFuturesTask = null; if (!isQueueEnabled() || timeline.isEmpty()) { + queueBitmapFutures = new ArrayMap<>(); setQueue(sessionCompat, /* queue= */ null); return; } List mediaItemList = LegacyConversions.convertToMediaItemList(timeline); - List<@NullableType ListenableFuture> bitmapFutures = new ArrayList<>(); - final AtomicInteger resultCount = new AtomicInteger(0); + // Reuse bitmap futures for identical artwork within this update and from the previous one. + ArrayMap> newQueueBitmapFutures = new ArrayMap<>(); + for (int i = 0; i < mediaItemList.size(); i++) { + @Nullable byte[] artworkData = mediaItemList.get(i).mediaMetadata.artworkData; + if (artworkData == null) { + continue; + } + ByteArrayKey key = new ByteArrayKey(artworkData); + if (!newQueueBitmapFutures.containsKey(key)) { + @Nullable ListenableFuture bitmapFuture = queueBitmapFutures.get(key); + if (bitmapFuture == null || bitmapFuture.isCancelled()) { + bitmapFuture = sessionImpl.getBitmapLoader().decodeBitmap(artworkData); + } + newQueueBitmapFutures.put(key, bitmapFuture); + } + } + queueBitmapFutures = newQueueBitmapFutures; + if (newQueueBitmapFutures.isEmpty()) { + handleBitmapFuturesAllCompletedAndSetQueue(newQueueBitmapFutures, mediaItemList); + return; + } + AtomicInteger resultCount = new AtomicInteger(0); Runnable handleBitmapFuturesTask = - () -> { - int completedBitmapFutureCount = resultCount.incrementAndGet(); - if (completedBitmapFutureCount == mediaItemList.size()) { - handleBitmapFuturesAllCompletedAndSetQueue(bitmapFutures, mediaItemList); + new Runnable() { + @Override + public void run() { + if (this != pendingBitmapFuturesTask) { + return; + } + int completedBitmapFutureCount = resultCount.incrementAndGet(); + if (completedBitmapFutureCount == newQueueBitmapFutures.size()) { + pendingBitmapFuturesTask = null; + handleBitmapFuturesAllCompletedAndSetQueue(newQueueBitmapFutures, mediaItemList); + } } }; - - for (int i = 0; i < mediaItemList.size(); i++) { - MediaItem mediaItem = mediaItemList.get(i); - MediaMetadata metadata = mediaItem.mediaMetadata; - if (metadata.artworkData == null) { - bitmapFutures.add(null); - handleBitmapFuturesTask.run(); - } else { - ListenableFuture bitmapFuture = - sessionImpl.getBitmapLoader().decodeBitmap(metadata.artworkData); - bitmapFutures.add(bitmapFuture); - bitmapFuture.addListener( - handleBitmapFuturesTask, sessionImpl.getApplicationHandler()::post); - } + pendingBitmapFuturesTask = handleBitmapFuturesTask; + for (int i = 0; i < newQueueBitmapFutures.size(); i++) { + newQueueBitmapFutures + .valueAt(i) + .addListener(handleBitmapFuturesTask, sessionImpl.getApplicationHandler()::post); } } private void handleBitmapFuturesAllCompletedAndSetQueue( - List<@NullableType ListenableFuture> bitmapFutures, List mediaItems) { + ArrayMap> bitmapFutures, + List mediaItems) { List queueItemList = new ArrayList<>(); - for (int i = 0; i < bitmapFutures.size(); i++) { - @Nullable ListenableFuture future = bitmapFutures.get(i); + for (int i = 0; i < mediaItems.size(); i++) { + MediaItem mediaItem = mediaItems.get(i); + @Nullable byte[] artworkData = mediaItem.mediaMetadata.artworkData; @Nullable Bitmap bitmap = null; - if (future != null) { + if (artworkData != null) { try { - bitmap = Futures.getDone(future); + bitmap = + Futures.getDone(checkNotNull(bitmapFutures.get(new ByteArrayKey(artworkData)))); } catch (CancellationException | ExecutionException e) { Log.d(TAG, "Failed to get bitmap", e); } } - queueItemList.add(LegacyConversions.convertToQueueItem(mediaItems.get(i), i, bitmap)); + queueItemList.add(LegacyConversions.convertToQueueItem(mediaItem, i, bitmap)); } // Framework MediaSession#setQueue() uses ParceledListSlice, @@ -1779,10 +1827,10 @@ private void updateMetadataIfChanged() { lastDurationMs = newDurationMs; @Nullable Bitmap artworkBitmap = null; + pendingBitmapLoadCallback = null; ListenableFuture bitmapFuture = sessionImpl.getBitmapLoader().loadBitmapFromMetadata(newMediaMetadata); if (bitmapFuture != null) { - pendingBitmapLoadCallback = null; if (bitmapFuture.isDone()) { try { artworkBitmap = Futures.getDone(bitmapFuture); @@ -1797,6 +1845,7 @@ public void onSuccess(Bitmap result) { if (this != pendingBitmapLoadCallback) { return; } + pendingBitmapLoadCallback = null; setMetadata( sessionCompat, LegacyConversions.convertToMediaMetadataCompat( @@ -1813,6 +1862,7 @@ public void onFailure(Throwable t) { if (this != pendingBitmapLoadCallback) { return; } + pendingBitmapLoadCallback = null; Log.w(TAG, getBitmapLoadErrorMessage(t)); } }; @@ -1829,6 +1879,33 @@ public void onFailure(Throwable t) { } } + /** Map key wrapping a byte array, using content-based equality. */ + private static final class ByteArrayKey { + private final byte[] data; + private final int hashCode; + + private ByteArrayKey(byte[] data) { + this.data = data; + this.hashCode = Arrays.hashCode(data); + } + + @Override + public boolean equals(@Nullable Object other) { + if (this == other) { + return true; + } + if (!(other instanceof ByteArrayKey)) { + return false; + } + return Arrays.equals(data, ((ByteArrayKey) other).data); + } + + @Override + public int hashCode() { + return hashCode; + } + } + private static class ConnectionTimeoutHandler extends Handler { private static final int MSG_CONNECTION_TIMED_OUT = 1001; diff --git a/libraries/session/src/main/java/androidx/media3/session/MediaSessionManager.java b/libraries/session/src/main/java/androidx/media3/session/MediaSessionManager.java index 513432f3a27..a8b377c8727 100644 --- a/libraries/session/src/main/java/androidx/media3/session/MediaSessionManager.java +++ b/libraries/session/src/main/java/androidx/media3/session/MediaSessionManager.java @@ -15,6 +15,7 @@ */ package androidx.media3.session; +import static android.os.Build.VERSION.SDK_INT; import static com.google.common.base.Predicates.notNull; import static com.google.common.util.concurrent.Futures.immediateFuture; import static com.google.common.util.concurrent.MoreExecutors.directExecutor; @@ -25,24 +26,20 @@ import android.content.pm.PackageManager; import android.media.session.MediaController; import android.media.session.MediaSession; -import android.os.Build; import android.os.Handler; import android.os.Looper; import android.provider.Settings; import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; import androidx.media3.common.util.Log; import androidx.media3.common.util.NullableType; import androidx.media3.common.util.UnstableApi; import androidx.media3.common.util.Util; -import com.google.common.base.Supplier; -import com.google.common.base.Suppliers; import com.google.common.collect.ImmutableList; import com.google.common.collect.Iterables; import com.google.common.util.concurrent.FutureCallback; import com.google.common.util.concurrent.Futures; import com.google.common.util.concurrent.ListenableFuture; -import com.google.errorprone.annotations.CanIgnoreReturnValue; -import java.lang.reflect.Method; import java.util.ArrayList; import java.util.HashMap; import java.util.List; @@ -72,76 +69,6 @@ public interface OnActiveSessionsChangedListener { private static final Map listenersMap = new HashMap<>(); - // TODO: b/500320224 - Remove reflection once compileSdk is bumped to 37. - private static final Supplier<@NullableType Method> - GET_ACTIVE_SESSIONS_FOR_PACKAGE_METHOD_SUPPLIER = - Suppliers.memoize( - new Supplier<@NullableType Method>() { - @Override - public @NullableType Method get() { - if (Build.VERSION.SDK_INT < 37) { - return null; - } - try { - return android.media.session.MediaSessionManager.class.getMethod( - "getActiveSessionsForPackage", String.class, ComponentName.class); - } catch (ReflectiveOperationException e) { - Log.w(TAG, "Failed to resolve getActiveSessionsForPackage method", e); - return null; - } - } - }); - - private static final Supplier<@NullableType Method> - ADD_ON_ACTIVE_SESSIONS_FOR_PACKAGE_CHANGED_LISTENER_METHOD_SUPPLIER = - Suppliers.memoize( - new Supplier<@NullableType Method>() { - @Override - public @NullableType Method get() { - if (Build.VERSION.SDK_INT < 37) { - return null; - } - try { - return android.media.session.MediaSessionManager.class.getMethod( - "addOnActiveSessionsForPackageChangedListener", - String.class, - Executor.class, - android.media.session.MediaSessionManager.OnActiveSessionsChangedListener - .class); - } catch (ReflectiveOperationException e) { - Log.w( - TAG, - "Failed to resolve addOnActiveSessionsForPackageChangedListener method", - e); - return null; - } - } - }); - - private static final Supplier<@NullableType Method> - REMOVE_ON_ACTIVE_SESSIONS_FOR_PACKAGE_CHANGED_LISTENER_METHOD_SUPPLIER = - Suppliers.memoize( - new Supplier<@NullableType Method>() { - @Override - public @NullableType Method get() { - if (Build.VERSION.SDK_INT < 37) { - return null; - } - try { - return android.media.session.MediaSessionManager.class.getMethod( - "removeOnActiveSessionsForPackageChangedListener", - android.media.session.MediaSessionManager.OnActiveSessionsChangedListener - .class); - } catch (ReflectiveOperationException e) { - Log.w( - TAG, - "Failed to resolve removeOnActiveSessionsForPackageChangedListener method", - e); - return null; - } - } - }); - private MediaSessionManager() { // Prevent instantiation. } @@ -185,13 +112,13 @@ public static ListenableFuture> getActiveSessions( "Requires MEDIA_CONTENT_CONTROL or enabled notification listener"); } - if (Build.VERSION.SDK_INT < 37 && isOwnPackage && !hasPermission) { + if (SDK_INT < 37 && isOwnPackage && !hasPermission) { return immediateFuture(ImmutableList.of()); } - if (packageName != null) { + if (SDK_INT >= 37 && packageName != null) { ListenableFuture> sessionsForPackage = - getActiveSessionsForPackageReflection( + Api37.getActiveSessionsForPackage( applicationContext, platformManager, packageName, notificationListener); if (sessionsForPackage != null) { return sessionsForPackage; @@ -246,7 +173,7 @@ public static void addOnActiveSessionsChangedListener( "Requires MEDIA_CONTENT_CONTROL or enabled notification listener"); } - if (Build.VERSION.SDK_INT < 37 && isOwnPackage && !hasPermission) { + if (SDK_INT < 37 && isOwnPackage && !hasPermission) { return; } @@ -286,11 +213,10 @@ public void onFailure(Throwable t) { } boolean isPackageSpecific = false; - if (packageName != null) { - if (tryAddOnActiveSessionsForPackageChangedListenerReflection( - platformManager, packageName, platformListener, executor)) { - isPackageSpecific = true; - } + if (SDK_INT >= 37 && packageName != null) { + Api37.addOnActiveSessionsForPackageChangedListener( + platformManager, packageName, executor, platformListener); + isPackageSpecific = true; } listenersMap.put(listener, new ListenerWrapper(platformListener, isPackageSpecific)); @@ -319,8 +245,8 @@ public static void removeOnActiveSessionsChangedListener( } if (wrapper != null) { - if (wrapper.isPackageSpecific) { - tryRemoveOnActiveSessionsForPackageChangedListenerReflection( + if (wrapper.isPackageSpecific && SDK_INT >= 37) { + Api37.removeOnActiveSessionsForPackageChangedListener( platformManager, wrapper.platformListener); } else { platformManager.removeOnActiveSessionsChangedListener(wrapper.platformListener); @@ -340,7 +266,7 @@ public static void removeOnActiveSessionsChangedListener( */ public static ListenableFuture<@NullableType SessionToken> getMediaKeyEventSession( Context context) { - if (Build.VERSION.SDK_INT < 33) { + if (SDK_INT < 33) { return immediateFuture(null); } Context applicationContext = context.getApplicationContext(); @@ -389,7 +315,7 @@ private static boolean hasEnabledNotificationListener( if (notificationListener == null) { return false; } - if (Build.VERSION.SDK_INT >= 27) { + if (SDK_INT >= 27) { NotificationManager notificationManager = context.getSystemService(NotificationManager.class); return notificationManager.isNotificationListenerAccessGranted(notificationListener); } @@ -407,68 +333,44 @@ private static boolean hasEnabledNotificationListener( return false; } - // TODO: b/500320224 - Remove reflection once compileSdk is bumped to 37. - @Nullable - @SuppressWarnings("nullness") - private static ListenableFuture> getActiveSessionsForPackageReflection( - Context context, - android.media.session.MediaSessionManager platformManager, - String packageName, - @Nullable ComponentName notificationListener) { - Method method = GET_ACTIVE_SESSIONS_FOR_PACKAGE_METHOD_SUPPLIER.get(); - if (method == null) { - return null; - } - try { - List tokens = (List) method.invoke(platformManager, packageName, notificationListener); + @RequiresApi(37) + private static final class Api37 { + private Api37() {} + + @Nullable + @SuppressWarnings("nullness") + private static ListenableFuture> getActiveSessionsForPackage( + Context context, + android.media.session.MediaSessionManager platformManager, + String packageName, + @Nullable ComponentName notificationListener) { + List tokens = + platformManager.getActiveSessionsForPackage(packageName, notificationListener); if (tokens == null) { return null; } List> tokenFutures = new ArrayList<>(tokens.size()); - for (Object tokenObj : tokens) { - MediaSession.Token platformToken = (MediaSession.Token) tokenObj; + for (MediaSession.Token platformToken : tokens) { tokenFutures.add(SessionToken.createSessionToken(context, platformToken)); } return convertFuturesToTokens(tokenFutures); - } catch (ReflectiveOperationException e) { - Log.w(TAG, "Failed to call getActiveSessionsForPackage via reflection", e); - return null; } - } - private static boolean tryAddOnActiveSessionsForPackageChangedListenerReflection( - android.media.session.MediaSessionManager platformManager, - String packageName, - android.media.session.MediaSessionManager.OnActiveSessionsChangedListener platformListener, - Executor executor) { - Method method = ADD_ON_ACTIVE_SESSIONS_FOR_PACKAGE_CHANGED_LISTENER_METHOD_SUPPLIER.get(); - if (method == null) { - return false; + private static void addOnActiveSessionsForPackageChangedListener( + android.media.session.MediaSessionManager platformManager, + String packageName, + Executor executor, + android.media.session.MediaSessionManager.OnActiveSessionsChangedListener + platformListener) { + platformManager.addOnActiveSessionsForPackageChangedListener( + packageName, executor, platformListener); } - try { - method.invoke(platformManager, packageName, executor, platformListener); - return true; - } catch (ReflectiveOperationException e) { - Log.w(TAG, "Failed to call addOnActiveSessionsForPackageChangedListener via reflection", e); - return false; - } - } - @CanIgnoreReturnValue - private static boolean tryRemoveOnActiveSessionsForPackageChangedListenerReflection( - android.media.session.MediaSessionManager platformManager, - android.media.session.MediaSessionManager.OnActiveSessionsChangedListener platformListener) { - Method method = REMOVE_ON_ACTIVE_SESSIONS_FOR_PACKAGE_CHANGED_LISTENER_METHOD_SUPPLIER.get(); - if (method == null) { - return false; - } - try { - method.invoke(platformManager, platformListener); - return true; - } catch (ReflectiveOperationException e) { - Log.w( - TAG, "Failed to call removeOnActiveSessionsForPackageChangedListener via reflection", e); - return false; + private static void removeOnActiveSessionsForPackageChangedListener( + android.media.session.MediaSessionManager platformManager, + android.media.session.MediaSessionManager.OnActiveSessionsChangedListener + platformListener) { + platformManager.removeOnActiveSessionsForPackageChangedListener(platformListener); } } diff --git a/libraries/session/src/main/java/androidx/media3/session/MediaSessionService.java b/libraries/session/src/main/java/androidx/media3/session/MediaSessionService.java index ef12d462137..c242d6ac4f2 100644 --- a/libraries/session/src/main/java/androidx/media3/session/MediaSessionService.java +++ b/libraries/session/src/main/java/androidx/media3/session/MediaSessionService.java @@ -110,6 +110,7 @@ *

        *
      1. Service Lifecycle *
      2. Supporting Multiple Sessions + *
      3. Backward Interoperability *
      * *

      Service Lifecycle

      @@ -158,6 +159,21 @@ * session in this service. In that case, {@link #onGetSession(ControllerInfo)} will be called to * decide which session to handle the connection request. Pick the best session among the added * sessions, or create a new session and return it from {@link #onGetSession(ControllerInfo)}. + * + *

      Backward Interoperability

      + * + *

      When a legacy controller (such as {@code + * android.support.v4.media.session.MediaControllerCompat}) connects, it connects using the session + * token published by the underlying platform session rather than binding directly to {@link + * MediaSessionService}. + * + *

      When a legacy media browser (such as {@code android.support.v4.media.MediaBrowserCompat}) + * binds to the service using the {@code android.media.browse.MediaBrowserService} action, {@link + * #onGetSession(ControllerInfo)} is called with a {@link ControllerInfo} whose package name is + * {@link android.service.media.MediaBrowserService#SERVICE_INTERFACE}. Returning a {@link + * MediaSession} allows the legacy browser to connect and control playback, but media browsing + * operations are not supported. If media browsing is required, extend {@link MediaLibraryService} + * and return a {@link MediaLibraryService.MediaLibrarySession}. */ public abstract class MediaSessionService extends LifecycleService { @@ -321,7 +337,10 @@ public void onCreate() { *

    • When a legacy {@link android.media.browse.MediaBrowser} or a {@code * android.support.v4.media.MediaBrowserCompat} tries to connect, the package name will be * {@link android.service.media.MediaBrowserService#SERVICE_INTERFACE}. If you want to allow - * the service to be bound by the legacy media browsers, do not return {@code null}. + * the service to be bound by the legacy media browsers, do not return {@code null}. Note + * that returning a {@link MediaSession} allows legacy browsers to control playback, but + * browsing operations are not supported unless the session is a {@link + * MediaLibraryService.MediaLibrarySession}. *
    * *

    For those special cases, the values returned by {@link ControllerInfo#getUid()} and {@link @@ -587,17 +606,26 @@ public int onStartCommand(@Nullable Intent intent, int flags, int startId) { @SuppressLint("InlinedApi") // Using compile time constant FOREGROUND_SERVICE_TYPE_MEDIA_PLAYBACK private void stopSelfSafely() { - Pair shutdownNotification = - getMediaNotificationManager().createShutdownNotification(/* context= */ this); - Util.setForegroundServiceNotification( - /* service= */ this, - /* notificationId= */ shutdownNotification.first, - /* notification= */ shutdownNotification.second, - ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PLAYBACK, - /* foregroundServiceManifestType= */ "mediaPlayback"); - getMediaNotificationManager().disableUserEngagedTimeout(); - Util.stopForeground(/* service= */ this, /* removeNotification= */ true); - stopSelf(); + try { + Pair shutdownNotification = + getMediaNotificationManager().createShutdownNotification(/* context= */ this); + Util.setForegroundServiceNotification( + /* service= */ this, + /* notificationId= */ shutdownNotification.first, + /* notification= */ shutdownNotification.second, + ServiceInfo.FOREGROUND_SERVICE_TYPE_MEDIA_PLAYBACK, + /* foregroundServiceManifestType= */ "mediaPlayback"); + getMediaNotificationManager().disableUserEngagedTimeout(); + stopForeground(STOP_FOREGROUND_REMOVE); + } catch (IllegalStateException e) { + if (SDK_INT >= 31 && e instanceof ForegroundServiceStartNotAllowedException) { + onForegroundServiceStartNotAllowedException(); + } else { + throw e; + } + } finally { + stopSelf(); + } } private static ControllerInfo createFallbackMediaButtonCaller(Intent mediaButtonIntent) { diff --git a/libraries/session/src/main/java/androidx/media3/session/MediaSessionServiceLegacyStub.java b/libraries/session/src/main/java/androidx/media3/session/MediaSessionServiceLegacyStub.java index be4cbd60241..761b2618ff3 100644 --- a/libraries/session/src/main/java/androidx/media3/session/MediaSessionServiceLegacyStub.java +++ b/libraries/session/src/main/java/androidx/media3/session/MediaSessionServiceLegacyStub.java @@ -31,6 +31,7 @@ import com.google.common.util.concurrent.FutureCallback; import com.google.common.util.concurrent.Futures; import com.google.common.util.concurrent.ListenableFuture; +import com.google.common.util.concurrent.MoreExecutors; import java.util.List; import java.util.concurrent.atomic.AtomicReference; @@ -83,25 +84,20 @@ public BrowserRoot onGetRoot( new FutureCallback() { @Override public void onSuccess(MediaSession.ConnectionResult result) { + // Running on arbitrary future callback. resultReference.set(result); - if (result.isAccepted) { - connectedControllersManager.addController( - info, - controller, - result.availableSessionCommands, - result.availablePlayerCommands); - } haveResult.open(); } @Override public void onFailure(Throwable t) { + // Running on arbitrary future callback. // Should not happen, onConnect should at least return a rejected result. resultReference.set(MediaSession.ConnectionResult.reject()); haveResult.open(); } }, - this::postOrRunOnApplicationHandler); + MoreExecutors.directExecutor()); }); try { haveResult.block(); @@ -110,9 +106,17 @@ public void onFailure(Throwable t) { return null; } MediaSession.ConnectionResult result = resultReference.get(); - if (!result.isAccepted) { + if (result == null || !result.isAccepted) { return null; } + postOrRunOnApplicationHandler( + () -> { + if (sessionImpl.isReleased()) { + return; + } + connectedControllersManager.addController( + info, controller, result.availableSessionCommands, result.availablePlayerCommands); + }); // No library root, but keep browser compat connected to allow getting session. return MediaUtils.defaultBrowserRoot; } diff --git a/libraries/session/src/main/java/androidx/media3/session/MediaStyleNotificationHelper.java b/libraries/session/src/main/java/androidx/media3/session/MediaStyleNotificationHelper.java index c8354239aea..6abb7c00ab5 100644 --- a/libraries/session/src/main/java/androidx/media3/session/MediaStyleNotificationHelper.java +++ b/libraries/session/src/main/java/androidx/media3/session/MediaStyleNotificationHelper.java @@ -17,21 +17,17 @@ import static android.Manifest.permission.MEDIA_CONTENT_CONTROL; import static android.os.Build.VERSION.SDK_INT; -import static androidx.core.app.NotificationCompat.COLOR_DEFAULT; import static com.google.common.base.Preconditions.checkArgument; import android.annotation.SuppressLint; import android.app.Notification; import android.app.PendingIntent; import android.os.Bundle; -import android.view.View; -import android.widget.RemoteViews; import androidx.annotation.DrawableRes; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; import androidx.annotation.RequiresPermission; import androidx.core.app.NotificationBuilderWithBuilderAccessor; -import androidx.core.graphics.drawable.IconCompat; import androidx.media3.common.util.Log; import androidx.media3.common.util.NullableType; import androidx.media3.common.util.UnstableApi; @@ -115,9 +111,6 @@ public static SessionToken getSessionToken(Notification notification) { } } - private static final int MAX_MEDIA_BUTTONS_IN_COMPACT = 3; - private static final int MAX_MEDIA_BUTTONS = 5; - /* package */ final MediaSession session; /* package */ int @NullableType [] actionsToShowInCompact; @@ -217,83 +210,6 @@ public void apply(NotificationBuilderWithBuilderAccessor builder) { builder.getBuilder().addExtras(bundle); } } - - /* package */ RemoteViews generateContentView() { - RemoteViews view = - applyStandardTemplate( - /* showSmallIcon= */ false, getContentViewLayoutResource(), /* fitIn1U= */ true); - - final int numActions = mBuilder.mActions.size(); - if (actionsToShowInCompact != null) { - int[] actions = actionsToShowInCompact; - final int numActionsInCompact = Math.min(actions.length, MAX_MEDIA_BUTTONS_IN_COMPACT); - view.removeAllViews(androidx.media3.session.R.id.media_actions); - if (numActionsInCompact > 0) { - for (int i = 0; i < numActionsInCompact; i++) { - if (i >= numActions) { - throw new IllegalArgumentException( - String.format( - "setShowActionsInCompactView: action %d out of bounds (max %d)", - i, numActions - 1)); - } - - final androidx.core.app.NotificationCompat.Action action = - mBuilder.mActions.get(actions[i]); - final RemoteViews button = generateMediaActionButton(action); - view.addView(androidx.media3.session.R.id.media_actions, button); - } - } - } - view.setViewVisibility(androidx.media3.session.R.id.end_padder, View.VISIBLE); - return view; - } - - private RemoteViews generateMediaActionButton( - androidx.core.app.NotificationCompat.Action action) { - final boolean tombstone = (action.getActionIntent() == null); - RemoteViews button = - new RemoteViews( - mBuilder.mContext.getPackageName(), - androidx.media3.session.R.layout.media3_notification_media_action); - IconCompat iconCompat = action.getIconCompat(); - if (iconCompat != null) { - button.setImageViewResource(androidx.media3.session.R.id.action0, iconCompat.getResId()); - } - if (!tombstone) { - button.setOnClickPendingIntent( - androidx.media3.session.R.id.action0, action.getActionIntent()); - } - button.setContentDescription(androidx.media3.session.R.id.action0, action.getTitle()); - return button; - } - - /* package */ int getContentViewLayoutResource() { - return androidx.media3.session.R.layout.media3_notification_template_media; - } - - /* package */ RemoteViews generateBigContentView() { - final int actionCount = Math.min(mBuilder.mActions.size(), MAX_MEDIA_BUTTONS); - RemoteViews big = - applyStandardTemplate( - /* showSmallIcon= */ false, - getBigContentViewLayoutResource(actionCount), - /* fitIn1U= */ false); - - big.removeAllViews(androidx.media3.session.R.id.media_actions); - if (actionCount > 0) { - for (int i = 0; i < actionCount; i++) { - final RemoteViews button = generateMediaActionButton(mBuilder.mActions.get(i)); - big.addView(androidx.media3.session.R.id.media_actions, button); - } - } - return big; - } - - /* package */ int getBigContentViewLayoutResource(int actionCount) { - return actionCount <= 3 - ? androidx.media3.session.R.layout.media3_notification_template_big_media_narrow - : androidx.media3.session.R.layout.media3_notification_template_big_media; - } } /** @@ -338,12 +254,8 @@ public DecoratedMediaCustomViewStyle(MediaSession session) { @Override public void apply(NotificationBuilderWithBuilderAccessor builder) { - if (SDK_INT < 24) { - super.apply(builder); - return; - } Notification.DecoratedMediaCustomViewStyle style = - Api24Impl.createDecoratedMediaCustomViewStyle(); + new Notification.DecoratedMediaCustomViewStyle(); if (actionsToShowInCompact != null) { style.setShowActionsInCompactView(actionsToShowInCompact); } @@ -358,110 +270,6 @@ public void apply(NotificationBuilderWithBuilderAccessor builder) { builder.getBuilder().addExtras(bundle); } } - - @Override - @Nullable - @SuppressWarnings("nullness:override.return") // NotificationCompat doesn't annotate @Nullable - public RemoteViews makeContentView(NotificationBuilderWithBuilderAccessor builder) { - if (SDK_INT >= 24) { - // No custom content view required - return null; - } - boolean hasContentView = mBuilder.getContentView() != null; - boolean createCustomContent = hasContentView || mBuilder.getBigContentView() != null; - if (createCustomContent) { - RemoteViews contentView = generateContentView(); - if (hasContentView) { - buildIntoRemoteViews(contentView, mBuilder.getContentView()); - } - setBackgroundColor(contentView); - return contentView; - } - return null; - } - - @Override - /* package */ int getContentViewLayoutResource() { - return mBuilder.getContentView() != null - ? androidx.media3.session.R.layout.media3_notification_template_media_custom - : super.getContentViewLayoutResource(); - } - - @Override - @Nullable - @SuppressWarnings("nullness:override.return") // NotificationCompat doesn't annotate @Nullable - public RemoteViews makeBigContentView(NotificationBuilderWithBuilderAccessor builder) { - if (SDK_INT >= 24) { - // No custom big content view required - return null; - } - RemoteViews innerView = - mBuilder.getBigContentView() != null - ? mBuilder.getBigContentView() - : mBuilder.getContentView(); - if (innerView == null) { - // No expandable notification - return null; - } - RemoteViews bigContentView = generateBigContentView(); - buildIntoRemoteViews(bigContentView, innerView); - setBackgroundColor(bigContentView); - return bigContentView; - } - - @Override - /* package */ int getBigContentViewLayoutResource(int actionCount) { - return actionCount <= 3 - ? androidx.media3.session.R.layout.media3_notification_template_big_media_narrow_custom - : androidx.media3.session.R.layout.media3_notification_template_big_media_custom; - } - - @Override - @Nullable - @SuppressWarnings("nullness:override.return") // NotificationCompat doesn't annotate @Nullable - public RemoteViews makeHeadsUpContentView(NotificationBuilderWithBuilderAccessor builder) { - if (SDK_INT >= 24) { - // No custom heads up content view required - return null; - } - RemoteViews innerView = - mBuilder.getHeadsUpContentView() != null - ? mBuilder.getHeadsUpContentView() - : mBuilder.getContentView(); - if (innerView == null) { - // No expandable notification - return null; - } - RemoteViews headsUpContentView = generateBigContentView(); - buildIntoRemoteViews(headsUpContentView, innerView); - setBackgroundColor(headsUpContentView); - return headsUpContentView; - } - - private void setBackgroundColor(RemoteViews views) { - int color = - mBuilder.getColor() != COLOR_DEFAULT - ? mBuilder.getColor() - : mBuilder - .mContext - .getResources() - .getColor( - androidx.media3.session.R.color - .notification_material_background_media_default_color); - views.setInt( - androidx.media3.session.R.id.status_bar_latest_event_content, - "setBackgroundColor", - color); - } - } - - @RequiresApi(24) - private static class Api24Impl { - private Api24Impl() {} - - public static Notification.DecoratedMediaCustomViewStyle createDecoratedMediaCustomViewStyle() { - return new Notification.DecoratedMediaCustomViewStyle(); - } } @RequiresApi(34) diff --git a/libraries/session/src/main/java/androidx/media3/session/MediaUtils.java b/libraries/session/src/main/java/androidx/media3/session/MediaUtils.java index 628ced88796..c48e6827136 100644 --- a/libraries/session/src/main/java/androidx/media3/session/MediaUtils.java +++ b/libraries/session/src/main/java/androidx/media3/session/MediaUtils.java @@ -15,9 +15,9 @@ */ package androidx.media3.session; -import static androidx.core.util.Preconditions.checkState; import static androidx.media3.common.Player.COMMAND_CHANGE_MEDIA_ITEMS; import static androidx.media3.common.util.Util.castNonNull; +import static com.google.common.base.Preconditions.checkState; import static java.lang.Math.min; import android.os.Parcel; diff --git a/libraries/session/src/main/java/androidx/media3/session/artwork/BitmapProcessor.java b/libraries/session/src/main/java/androidx/media3/session/artwork/BitmapProcessor.java index d60f20569cd..2a2981b0e5d 100644 --- a/libraries/session/src/main/java/androidx/media3/session/artwork/BitmapProcessor.java +++ b/libraries/session/src/main/java/androidx/media3/session/artwork/BitmapProcessor.java @@ -27,7 +27,7 @@ /** Utility class for processing bitmaps, such as downsizing and re-compression. */ @UnstableApi -/* package */ final class BitmapProcessor { +public final class BitmapProcessor { private BitmapProcessor() {} diff --git a/libraries/session/src/main/java/androidx/media3/session/legacy/MediaBrowserServiceCompat.java b/libraries/session/src/main/java/androidx/media3/session/legacy/MediaBrowserServiceCompat.java index 208b3c2f13d..3b9e964462a 100644 --- a/libraries/session/src/main/java/androidx/media3/session/legacy/MediaBrowserServiceCompat.java +++ b/libraries/session/src/main/java/androidx/media3/session/legacy/MediaBrowserServiceCompat.java @@ -301,16 +301,7 @@ public void onLoadChildren(String parentId, final ResultWrapper> re void onResultSent(@Nullable List list) { List parcelList; if (list == null) { - // A null children list here indicates that the requested parentId - // is invalid. Unfortunately before API 24 the platform - // MediaBrowserService's implementation of Result inside - // performLoadChildren (invoked below) throws an exception if - // given a null list (b/19127753). This means there's no clear - // way to communicate an invalid parentId, so in order to avoid - // an exception below API 24 we transform null to an empty list - // here (meaning it looks like parentId is valid but has no - // children). - parcelList = Build.VERSION.SDK_INT >= 24 ? null : Collections.emptyList(); + parcelList = null; } else { parcelList = new ArrayList<>(list.size()); for (MediaBrowserCompat.MediaItem item : list) { diff --git a/libraries/session/src/main/java/androidx/media3/session/legacy/MediaControllerCompat.java b/libraries/session/src/main/java/androidx/media3/session/legacy/MediaControllerCompat.java index 2ac982e364a..4b070fe04c2 100644 --- a/libraries/session/src/main/java/androidx/media3/session/legacy/MediaControllerCompat.java +++ b/libraries/session/src/main/java/androidx/media3/session/legacy/MediaControllerCompat.java @@ -175,7 +175,7 @@ public MediaControllerCompat(Context context, MediaSessionCompat.Token sessionTo if (Build.VERSION.SDK_INT >= 29) { impl = new MediaControllerImplApi29(context, sessionToken); } else { - impl = new MediaControllerImplApi23(context, sessionToken); + impl = new MediaControllerImplBase(context, sessionToken); } } @@ -1249,7 +1249,7 @@ interface MediaControllerImpl { Object getMediaController(); } - static class MediaControllerImplApi23 implements MediaControllerImpl { + static class MediaControllerImplBase implements MediaControllerImpl { protected final MediaController controllerFwk; final Object lock = new Object(); @@ -1265,7 +1265,7 @@ static class MediaControllerImplApi23 implements MediaControllerImpl { // Calling method from constructor @SuppressWarnings({"assignment.type.incompatible", "method.invocation.invalid"}) - MediaControllerImplApi23(Context context, MediaSessionCompat.Token sessionToken) { + MediaControllerImplBase(Context context, MediaSessionCompat.Token sessionToken) { this.sessionToken = sessionToken; controllerFwk = new MediaController(context, this.sessionToken.getToken()); if (this.sessionToken.getExtraBinder() == null) { @@ -1328,10 +1328,8 @@ public TransportControls getTransportControls() { MediaController.TransportControls controlsFwk = controllerFwk.getTransportControls(); if (Build.VERSION.SDK_INT >= 29) { return new TransportControlsApi29(controlsFwk); - } else if (Build.VERSION.SDK_INT >= 24) { - return new TransportControlsApi24(controlsFwk); } else { - return new TransportControlsApi23(controlsFwk); + return new TransportControlsBase(controlsFwk); } } @@ -1566,16 +1564,16 @@ void processPendingCallbacksLocked() { } private static class ExtraBinderRequestResultReceiver extends ResultReceiver { - private final WeakReference mediaControllerImpl; + private final WeakReference mediaControllerImpl; - ExtraBinderRequestResultReceiver(MediaControllerImplApi23 mediaControllerImpl) { + ExtraBinderRequestResultReceiver(MediaControllerImplBase mediaControllerImpl) { super(null /* handler */); this.mediaControllerImpl = new WeakReference<>(mediaControllerImpl); } @Override protected void onReceiveResult(int resultCode, Bundle resultData) { - MediaControllerImplApi23 mediaControllerImpl = this.mediaControllerImpl.get(); + MediaControllerImplBase mediaControllerImpl = this.mediaControllerImpl.get(); if (mediaControllerImpl == null || resultData == null) { return; } @@ -1593,7 +1591,7 @@ protected void onReceiveResult(int resultCode, Bundle resultData) { } @RequiresApi(29) - static class MediaControllerImplApi29 extends MediaControllerImplApi23 { + static class MediaControllerImplApi29 extends MediaControllerImplBase { MediaControllerImplApi29(Context context, MediaSessionCompat.Token sessionToken) { super(context, sessionToken); } @@ -1609,40 +1607,34 @@ public Bundle getSessionInfo() { } } - static class TransportControlsApi23 extends TransportControls { + static class TransportControlsBase extends TransportControls { protected final MediaController.TransportControls controlsFwk; - TransportControlsApi23(MediaController.TransportControls controlsFwk) { + TransportControlsBase(MediaController.TransportControls controlsFwk) { this.controlsFwk = controlsFwk; } @Override public void prepare() { - sendCustomAction(MediaSessionCompat.ACTION_PREPARE, null); + controlsFwk.prepare(); } + @SuppressWarnings("argument.type.incompatible") // Framework controller is missing annotation @Override public void prepareFromMediaId(String mediaId, @Nullable Bundle extras) { - Bundle bundle = new Bundle(); - bundle.putString(MediaSessionCompat.ACTION_ARGUMENT_MEDIA_ID, mediaId); - bundle.putBundle(MediaSessionCompat.ACTION_ARGUMENT_EXTRAS, extras); - sendCustomAction(MediaSessionCompat.ACTION_PREPARE_FROM_MEDIA_ID, bundle); + controlsFwk.prepareFromMediaId(mediaId, extras); } + @SuppressWarnings("argument.type.incompatible") // Platform controller accepts null extra @Override public void prepareFromSearch(String query, @Nullable Bundle extras) { - Bundle bundle = new Bundle(); - bundle.putString(MediaSessionCompat.ACTION_ARGUMENT_QUERY, query); - bundle.putBundle(MediaSessionCompat.ACTION_ARGUMENT_EXTRAS, extras); - sendCustomAction(MediaSessionCompat.ACTION_PREPARE_FROM_SEARCH, bundle); + controlsFwk.prepareFromSearch(query, extras); } + @SuppressWarnings("argument.type.incompatible") // Platform controller accepts null extra @Override public void prepareFromUri(Uri uri, @Nullable Bundle extras) { - Bundle bundle = new Bundle(); - bundle.putParcelable(MediaSessionCompat.ACTION_ARGUMENT_URI, uri); - bundle.putBundle(MediaSessionCompat.ACTION_ARGUMENT_EXTRAS, extras); - sendCustomAction(MediaSessionCompat.ACTION_PREPARE_FROM_URI, bundle); + controlsFwk.prepareFromUri(uri, extras); } @Override @@ -1761,38 +1753,8 @@ public void sendCustomAction(String action, @Nullable Bundle args) { } } - @RequiresApi(24) - static class TransportControlsApi24 extends TransportControlsApi23 { - TransportControlsApi24(MediaController.TransportControls controlsFwk) { - super(controlsFwk); - } - - @Override - public void prepare() { - controlsFwk.prepare(); - } - - @SuppressWarnings("argument.type.incompatible") // Framework controller is missing annotation - @Override - public void prepareFromMediaId(String mediaId, @Nullable Bundle extras) { - controlsFwk.prepareFromMediaId(mediaId, extras); - } - - @SuppressWarnings("argument.type.incompatible") // Platform controller accepts null extra - @Override - public void prepareFromSearch(String query, @Nullable Bundle extras) { - controlsFwk.prepareFromSearch(query, extras); - } - - @SuppressWarnings("argument.type.incompatible") // Platform controller accepts null extra - @Override - public void prepareFromUri(Uri uri, @Nullable Bundle extras) { - controlsFwk.prepareFromUri(uri, extras); - } - } - @RequiresApi(29) - static class TransportControlsApi29 extends TransportControlsApi24 { + static class TransportControlsApi29 extends TransportControlsBase { TransportControlsApi29(MediaController.TransportControls controlsFwk) { super(controlsFwk); } diff --git a/libraries/session/src/main/java/androidx/media3/session/legacy/MediaSessionCompat.java b/libraries/session/src/main/java/androidx/media3/session/legacy/MediaSessionCompat.java index 44a4f27c2ca..1376f1e8f97 100644 --- a/libraries/session/src/main/java/androidx/media3/session/legacy/MediaSessionCompat.java +++ b/libraries/session/src/main/java/androidx/media3/session/legacy/MediaSessionCompat.java @@ -26,8 +26,6 @@ import android.annotation.SuppressLint; import android.app.Activity; import android.app.PendingIntent; -import android.content.BroadcastReceiver; -import android.content.ComponentName; import android.content.Context; import android.content.Intent; import android.media.MediaDescription; @@ -66,7 +64,6 @@ import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.ref.WeakReference; -import java.lang.reflect.Constructor; import java.lang.reflect.Field; import java.lang.reflect.Method; import java.util.ArrayList; @@ -293,26 +290,17 @@ public class MediaSessionCompat { public static final String KEY_SESSION2_TOKEN = "android.support.v4.media.session.SESSION_TOKEN2"; /** - * Creates a new session with a specified media button receiver (a component name and/or a pending - * intent). You must call {@link #release()} when finished with the session. + * Creates a new session. You must call {@link #release()} when finished with the session. * *

    The session will automatically be registered with the system but will not be published until * {@link #setActive(boolean) setActive(true)} is called. * - *

    For API 20 or earlier, note that a media button receiver is required for handling {@link - * Intent#ACTION_MEDIA_BUTTON}. This constructor will attempt to find an appropriate {@link - * BroadcastReceiver} from your manifest if it's not specified. See {@link MediaButtonReceiver} - * for more details. The {@code sessionInfo} can include additional unchanging information about - * this session. For example, it can include the version of the application, or other app-specific - * unchanging information. + *

    The {@code sessionInfo} can include additional unchanging information about this session. + * For example, it can include the version of the application, or other app-specific unchanging + * information. * * @param context The context to use to create the session. * @param tag A short name for debugging purposes. - * @param mbrComponent The component name for your media button receiver. - * @param mbrIntent The PendingIntent for your receiver component that handles media button - * events. This is optional and will be used on between {@link - * android.os.Build.VERSION_CODES#JELLY_BEAN_MR2} and {@link - * android.os.Build.VERSION_CODES#KITKAT_WATCH} instead of the component name. * @param sessionInfo A bundle for additional information about this session, or {@link * Bundle#EMPTY} if none. Controllers can get this information by calling {@link * MediaControllerCompat#getSessionInfo()}. An {@link IllegalArgumentException} will be thrown @@ -328,8 +316,6 @@ public class MediaSessionCompat { public MediaSessionCompat( Context context, String tag, - @Nullable ComponentName mbrComponent, - @Nullable PendingIntent mbrIntent, @Nullable Bundle sessionInfo, @Nullable String packageNameOverride) { if (TextUtils.isEmpty(tag)) { @@ -342,25 +328,6 @@ public MediaSessionCompat( "must have OVERRIDE_MEDIA_SESSION_OWNER permission to override package name"); } - if (mbrComponent == null) { - mbrComponent = MediaButtonReceiver.getMediaButtonReceiverComponent(context); - if (mbrComponent == null) { - Log.i(TAG, "Couldn't find a unique registered media button receiver in the given context."); - } - } - if (mbrComponent != null && mbrIntent == null) { - // construct a PendingIntent for the media button - Intent mediaButtonIntent = new Intent(Intent.ACTION_MEDIA_BUTTON); - // the associated intent will be handled by the component being registered - mediaButtonIntent.setComponent(mbrComponent); - mbrIntent = - PendingIntent.getBroadcast( - context, - 0 /* requestCode, ignored */, - mediaButtonIntent, - Build.VERSION.SDK_INT >= 31 ? PendingIntent.FLAG_MUTABLE : 0); - } - if (VERSION.SDK_INT >= 37) { impl = new MediaSessionImplApi37(context, tag, sessionInfo, packageNameOverride); } else if (Build.VERSION.SDK_INT >= 29) { @@ -374,7 +341,6 @@ public MediaSessionCompat( Looper myLooper = Looper.myLooper(); Handler handler = new Handler(myLooper != null ? myLooper : Looper.getMainLooper()); setCallback(new Callback() {}, handler); - impl.setMediaButtonReceiver(mbrIntent); controller = new MediaControllerCompat(context, this); } @@ -414,7 +380,7 @@ public void setSessionActivity(@Nullable PendingIntent pi) { * * @param mbr The {@link PendingIntent} to send the media button event to. */ - public void setMediaButtonReceiver(PendingIntent mbr) { + public void setMediaButtonReceiver(@Nullable PendingIntent mbr) { impl.setMediaButtonReceiver(mbr); } @@ -736,8 +702,7 @@ public abstract static class Callback { @Nullable @GuardedBy("lock") - @SuppressWarnings("WeakerAccess") /* synthetic access */ - CallbackHandler callbackHandler; + private CallbackHandler callbackHandler; public Callback() { callbackFwk = new MediaSessionCallback(); @@ -1364,7 +1329,6 @@ public void onCustomAction(String action, @Nullable Bundle extras) { clearCurrentControllerInfo(sessionImpl); } - @RequiresApi(24) @Override public void onPrepare() { MediaSessionImplApi23 sessionImpl = getSessionImplIfCallbackIsSet(); @@ -1376,7 +1340,6 @@ public void onPrepare() { clearCurrentControllerInfo(sessionImpl); } - @RequiresApi(24) @Override public void onPrepareFromMediaId(@Nullable String mediaId, @Nullable Bundle extras) { MediaSessionImplApi23 sessionImpl = getSessionImplIfCallbackIsSet(); @@ -1389,7 +1352,6 @@ public void onPrepareFromMediaId(@Nullable String mediaId, @Nullable Bundle extr clearCurrentControllerInfo(sessionImpl); } - @RequiresApi(24) @Override public void onPrepareFromSearch(@Nullable String query, @Nullable Bundle extras) { MediaSessionImplApi23 sessionImpl = getSessionImplIfCallbackIsSet(); @@ -1402,7 +1364,6 @@ public void onPrepareFromSearch(@Nullable String query, @Nullable Bundle extras) clearCurrentControllerInfo(sessionImpl); } - @RequiresApi(24) @Override public void onPrepareFromUri(@Nullable Uri uri, @Nullable Bundle extras) { MediaSessionImplApi23 sessionImpl = getSessionImplIfCallbackIsSet(); @@ -1885,7 +1846,7 @@ interface MediaSessionImpl { static class MediaSessionImplApi23 implements MediaSessionImpl { final MediaSession sessionFwk; - final ExtraSession extraSession; + private final ExtraSession extraSession; final Token token; final Object lock = new Object(); @Nullable Bundle sessionInfo; @@ -2135,17 +2096,13 @@ public void setCurrentControllerInfo(@Nullable RemoteUserInfo remoteUserInfo) { @Nullable @Override public String getCallingPackage() { - if (android.os.Build.VERSION.SDK_INT < 24) { - return null; - } else { - try { - Method getCallingPackageMethod = sessionFwk.getClass().getMethod("getCallingPackage"); - return (String) getCallingPackageMethod.invoke(sessionFwk); - } catch (Exception e) { - Log.e(TAG, "Cannot execute MediaSession.getCallingPackage()", e); - } - return null; + try { + Method getCallingPackageMethod = sessionFwk.getClass().getMethod("getCallingPackage"); + return (String) getCallingPackageMethod.invoke(sessionFwk); + } catch (Exception e) { + Log.e(TAG, "Cannot execute MediaSession.getCallingPackage()", e); } + return null; } @Nullable @@ -2316,23 +2273,15 @@ static class MediaSessionImplApi37 extends MediaSessionImplApi29 { } @Override + @SuppressLint("MissingPermission") // Checked at runtime in MediaSessionCompat constructor. public MediaSession createFwkMediaSession( Context context, String tag, @Nullable Bundle sessionInfo, @Nullable String packageNameOverride) { if (packageNameOverride != null) { - // TODO: b/500320224- Replace reflection with actual API once the compileSdk version is - // bumped to 37. - try { - Constructor constructor = - MediaSession.class.getConstructor( - Context.class, String.class, Bundle.class, String.class); - return constructor.newInstance( - context, tag, sessionInfo == null ? Bundle.EMPTY : sessionInfo, packageNameOverride); - } catch (ReflectiveOperationException e) { - throw new IllegalStateException("Unable to create session with overridden owner", e); - } + return new MediaSession( + context, tag, sessionInfo == null ? Bundle.EMPTY : sessionInfo, packageNameOverride); } else { return new MediaSession(context, tag, sessionInfo); } diff --git a/libraries/session/src/main/java/androidx/media3/session/legacy/MediaSessionManager.java b/libraries/session/src/main/java/androidx/media3/session/legacy/MediaSessionManager.java index 4cee171908e..c128a5985a5 100644 --- a/libraries/session/src/main/java/androidx/media3/session/legacy/MediaSessionManager.java +++ b/libraries/session/src/main/java/androidx/media3/session/legacy/MediaSessionManager.java @@ -244,11 +244,6 @@ public boolean isTrustedForMediaControl(MediaSessionManager.RemoteUserInfoImpl u // but it's hidden so we cannot use it. int remoteUserUid = userInfo.getUid(); try { - if (SDK_INT == 23 && userInfo.getPackageName().equals(RemoteUserInfo.LEGACY_CONTROLLER)) { - // On API 23, there is no package name or caller propagation to allow caller checks for - // framework controllers, and the best we can do is to simply trust all controllers. - return true; - } ApplicationInfo applicationInfo = context.getPackageManager().getApplicationInfo(userInfo.getPackageName(), 0); if (applicationInfo == null) { diff --git a/libraries/session/src/main/res/layout/media3_notification_template_big_media.xml b/libraries/session/src/main/res/layout/media3_notification_template_big_media.xml deleted file mode 100644 index 63887c1e336..00000000000 --- a/libraries/session/src/main/res/layout/media3_notification_template_big_media.xml +++ /dev/null @@ -1,50 +0,0 @@ - - - - - - - - - - - diff --git a/libraries/session/src/main/res/layout/media3_notification_template_big_media_custom.xml b/libraries/session/src/main/res/layout/media3_notification_template_big_media_custom.xml deleted file mode 100644 index 904e0363b1a..00000000000 --- a/libraries/session/src/main/res/layout/media3_notification_template_big_media_custom.xml +++ /dev/null @@ -1,97 +0,0 @@ - - - - - - - - - - - - - - - - - - diff --git a/libraries/session/src/main/res/layout/media3_notification_template_big_media_narrow.xml b/libraries/session/src/main/res/layout/media3_notification_template_big_media_narrow.xml deleted file mode 100644 index 8a165b89c5e..00000000000 --- a/libraries/session/src/main/res/layout/media3_notification_template_big_media_narrow.xml +++ /dev/null @@ -1,57 +0,0 @@ - - - - - - - - - - - - - - diff --git a/libraries/session/src/main/res/layout/media3_notification_template_big_media_narrow_custom.xml b/libraries/session/src/main/res/layout/media3_notification_template_big_media_narrow_custom.xml deleted file mode 100644 index 9518a73be75..00000000000 --- a/libraries/session/src/main/res/layout/media3_notification_template_big_media_narrow_custom.xml +++ /dev/null @@ -1,104 +0,0 @@ - - - - - - - - - - - - - - - - - - - - - diff --git a/libraries/session/src/main/res/layout/media3_notification_template_lines_media.xml b/libraries/session/src/main/res/layout/media3_notification_template_lines_media.xml deleted file mode 100644 index 62aa29c2f28..00000000000 --- a/libraries/session/src/main/res/layout/media3_notification_template_lines_media.xml +++ /dev/null @@ -1,111 +0,0 @@ - - - - - - - - - - - - - - - diff --git a/libraries/session/src/main/res/layout/media3_notification_template_media.xml b/libraries/session/src/main/res/layout/media3_notification_template_media.xml deleted file mode 100644 index 575af959b01..00000000000 --- a/libraries/session/src/main/res/layout/media3_notification_template_media.xml +++ /dev/null @@ -1,45 +0,0 @@ - - - - - - - - - - - diff --git a/libraries/session/src/main/res/layout/media3_notification_template_media_custom.xml b/libraries/session/src/main/res/layout/media3_notification_template_media_custom.xml deleted file mode 100644 index 744f74696c5..00000000000 --- a/libraries/session/src/main/res/layout/media3_notification_template_media_custom.xml +++ /dev/null @@ -1,92 +0,0 @@ - - - - - - - - - - - - - - - - - - diff --git a/libraries/session/src/main/res/values-gl/strings.xml b/libraries/session/src/main/res/values-gl/strings.xml index fe8cfef4f4c..ddffac96b64 100755 --- a/libraries/session/src/main/res/values-gl/strings.xml +++ b/libraries/session/src/main/res/values-gl/strings.xml @@ -17,7 +17,7 @@ Aquí non se pode acceder a ese contido Esta aplicación non pode realizar a acción Ese contido está bloqueado - Requírese acceso premium + Requírese acceso prémium Non se poden saltar máis pistas Datos de entrada incorrectos Erro de entrada/saída diff --git a/libraries/session/src/test/java/androidx/media3/session/ConnectionResultTest.java b/libraries/session/src/test/java/androidx/media3/session/ConnectionResultTest.java index 77a327ab795..6e8e088f08a 100644 --- a/libraries/session/src/test/java/androidx/media3/session/ConnectionResultTest.java +++ b/libraries/session/src/test/java/androidx/media3/session/ConnectionResultTest.java @@ -17,7 +17,9 @@ import static com.google.common.truth.Truth.assertThat; +import android.app.PendingIntent; import android.content.Context; +import android.content.Intent; import android.os.Bundle; import androidx.annotation.Nullable; import androidx.media3.common.Player; @@ -47,6 +49,8 @@ public void acceptedResultBuilder_builtWidthMediaSession_correctDefaults() { assertThat(connectionResult.availablePlayerCommands) .isEqualTo(MediaSession.ConnectionResult.DEFAULT_PLAYER_COMMANDS); assertThat(connectionResult.customLayout).isNull(); + assertThat(connectionResult.sessionExtras).isNull(); + assertThat(connectionResult.sessionActivity).isNull(); assertThat(connectionResult.isAccepted).isTrue(); mediaSession.getPlayer().release(); @@ -58,17 +62,26 @@ public void acceptedResultBuilder_builtWidthMediaSession_correctlyOverridden() { Context context = ApplicationProvider.getApplicationContext(); MediaSession mediaSession = new MediaSession.Builder(context, new TestExoPlayerBuilder(context).build()).build(); + Bundle extras = new Bundle(); + extras.putString("key", "value"); + PendingIntent sessionActivity = + PendingIntent.getActivity( + context, /* requestCode= */ 0, new Intent(), PendingIntent.FLAG_IMMUTABLE); MediaSession.ConnectionResult connectionResult = new MediaSession.ConnectionResult.AcceptedResultBuilder(mediaSession) .setAvailableSessionCommands(SessionCommands.EMPTY) .setAvailablePlayerCommands(Player.Commands.EMPTY) .setCustomLayout(ImmutableList.of()) + .setSessionExtras(extras) + .setSessionActivity(sessionActivity) .build(); assertThat(connectionResult.availableSessionCommands.commands).isEmpty(); assertThat(connectionResult.availablePlayerCommands.size()).isEqualTo(0); assertThat(connectionResult.customLayout).isEmpty(); + assertThat(connectionResult.sessionExtras).isEqualTo(extras); + assertThat(connectionResult.sessionActivity).isEqualTo(sessionActivity); assertThat(connectionResult.isAccepted).isTrue(); mediaSession.getPlayer().release(); diff --git a/libraries/session/src/test/java/androidx/media3/session/LegacyConversionsTest.java b/libraries/session/src/test/java/androidx/media3/session/LegacyConversionsTest.java index b036541b036..96307fd6947 100644 --- a/libraries/session/src/test/java/androidx/media3/session/LegacyConversionsTest.java +++ b/libraries/session/src/test/java/androidx/media3/session/LegacyConversionsTest.java @@ -15,10 +15,14 @@ */ package androidx.media3.session; +import static androidx.media3.session.MediaConstants.EXTRAS_KEY_CATEGORIES; import static androidx.media3.session.MediaConstants.EXTRAS_KEY_COMMAND_BUTTON_ICON_COMPAT; import static androidx.media3.session.MediaConstants.EXTRAS_KEY_COMMAND_BUTTON_ICON_URI_COMPAT; import static androidx.media3.session.MediaConstants.EXTRAS_KEY_COMPLETION_STATUS; import static androidx.media3.session.MediaConstants.EXTRAS_KEY_MEDIA_TYPE_COMPAT; +import static androidx.media3.session.MediaConstants.EXTRAS_KEY_PLAYLIST_ARTWORK_URI; +import static androidx.media3.session.MediaConstants.EXTRAS_KEY_PLAYLIST_ID; +import static androidx.media3.session.MediaConstants.EXTRAS_KEY_PLAYLIST_TITLE; import static androidx.media3.session.MediaConstants.EXTRAS_VALUE_COMPLETION_STATUS_PARTIALLY_PLAYED; import static androidx.media3.session.MediaConstants.EXTRA_KEY_ROOT_CHILDREN_BROWSABLE_ONLY; import static androidx.media3.session.legacy.MediaBrowserCompat.MediaItem.FLAG_BROWSABLE; @@ -29,17 +33,22 @@ import static com.google.common.truth.Truth.assertThat; import static java.util.concurrent.TimeUnit.SECONDS; import static org.junit.Assert.fail; +import static org.robolectric.Shadows.shadowOf; import android.content.Context; import android.graphics.Bitmap; import android.graphics.BitmapFactory; +import android.media.AudioAttributes; +import android.media.session.MediaController; import android.net.Uri; import android.os.Bundle; import android.service.media.MediaBrowserService; import android.text.SpannableString; import android.text.SpannedString; +import android.text.TextUtils; import androidx.annotation.Nullable; import androidx.media3.common.C; +import androidx.media3.common.DeviceInfo; import androidx.media3.common.HeartRating; import androidx.media3.common.MediaItem; import androidx.media3.common.MediaMetadata; @@ -51,6 +60,7 @@ import androidx.media3.common.util.BitmapLoader; import androidx.media3.datasource.DataSourceBitmapLoader; import androidx.media3.session.legacy.MediaBrowserCompat; +import androidx.media3.session.legacy.MediaControllerCompat; import androidx.media3.session.legacy.MediaDescriptionCompat; import androidx.media3.session.legacy.MediaMetadataCompat; import androidx.media3.session.legacy.MediaSessionCompat; @@ -68,6 +78,7 @@ import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.annotation.Config; +import org.robolectric.shadows.PlaybackInfoBuilder; /** Tests for {@link LegacyConversions}. */ @RunWith(AndroidJUnit4.class) @@ -163,6 +174,8 @@ public void convertToMediaDescriptionCompat_setsExpectedValues() { .setAlbumTitle("testAlbumTitle") .setWriter("testWriter") .setMediaType(MediaMetadata.MEDIA_TYPE_MUSIC) + .setPlaylistId("testPlaylistId") + .setCategories(ImmutableList.of("cat1", "cat2")) .setDurationMs(10_000L) .setExtras(extras) .build(); @@ -180,6 +193,10 @@ public void convertToMediaDescriptionCompat_setsExpectedValues() { .isEqualTo(MediaMetadata.MEDIA_TYPE_MUSIC); assertThat(descriptionCompat.getExtras().getInt(EXTRAS_KEY_COMPLETION_STATUS)) .isEqualTo(EXTRAS_VALUE_COMPLETION_STATUS_PARTIALLY_PLAYED); + assertThat(descriptionCompat.getExtras().getString(EXTRAS_KEY_PLAYLIST_ID)) + .isEqualTo("testPlaylistId"); + assertThat(descriptionCompat.getExtras().getString(EXTRAS_KEY_CATEGORIES)) + .isEqualTo("cat1,cat2"); } @Test @@ -469,7 +486,7 @@ public void convertToMediaMetadata_displayTitleSet_usesDisplayTitleDisplayDescri assertThat(mediaMetadata.title.toString()).isEqualTo("displayTitle"); assertThat(mediaMetadata.description.toString()).isEqualTo("displayDescription"); assertThat(mediaMetadata.subtitle.toString()).isEqualTo("displaySubtitle"); - assertThat(mediaMetadata.displayTitle).isEqualTo("displayTitle"); + assertThat(TextUtils.equals(mediaMetadata.displayTitle, "displayTitle")).isTrue(); } @Test @@ -489,7 +506,7 @@ public void convertToMediaMetadata_displayTitleNotSet_usesPreferredDescriptionOr assertThat(mediaMetadata.title.toString()).isEqualTo("title"); assertThat(mediaMetadata.subtitle.toString()).isEqualTo("artist"); assertThat(mediaMetadata.description.toString()).isEqualTo("album"); - assertThat(mediaMetadata.displayTitle).isEqualTo("title"); + assertThat(TextUtils.equals(mediaMetadata.displayTitle, "title")).isTrue(); } @Test @@ -519,6 +536,160 @@ public void convertToMediaMetadata_displayTitleNotSet_usesPreferredDescriptionOr assertThat(mediaMetadata.artworkData).isNotNull(); } + @Test + public void convertToMediaMetadata_withPlaylistIdInMediaMetadataCompat_assignedToPlaylistId() { + MediaMetadataCompat testMediaMetadataCompat = + new MediaMetadataCompat.Builder().putString(EXTRAS_KEY_PLAYLIST_ID, "playlist123").build(); + + MediaMetadata mediaMetadata = + LegacyConversions.convertToMediaMetadata(testMediaMetadataCompat, RatingCompat.RATING_NONE); + + assertThat(mediaMetadata.playlistId).isEqualTo("playlist123"); + assertThat(mediaMetadata.extras).isNull(); + } + + @Test + public void convertToMediaMetadata_withEmptyPlaylistIdInMediaMetadataCompat_ignoresPlaylistId() { + MediaMetadataCompat testMediaMetadataCompat = + new MediaMetadataCompat.Builder().putString(EXTRAS_KEY_PLAYLIST_ID, "").build(); + + MediaMetadata mediaMetadata = + LegacyConversions.convertToMediaMetadata(testMediaMetadataCompat, RatingCompat.RATING_NONE); + + assertThat(mediaMetadata.playlistId).isNull(); + assertThat(mediaMetadata.extras).isNull(); + } + + @Test + public void convertToMediaMetadata_withPlaylistIdInMediaDescriptionCompat_assignedToPlaylistId() { + Bundle extras = new Bundle(); + extras.putString(EXTRAS_KEY_PLAYLIST_ID, "playlist123"); + MediaDescriptionCompat descriptionCompat = + new MediaDescriptionCompat.Builder().setExtras(extras).build(); + + MediaMetadata mediaMetadata = + LegacyConversions.convertToMediaMetadata(descriptionCompat, RatingCompat.RATING_NONE); + + assertThat(mediaMetadata.playlistId).isEqualTo("playlist123"); + assertThat(mediaMetadata.extras).isNull(); + } + + @Test + public void + convertToMediaMetadata_withEmptyPlaylistIdInMediaDescriptionCompat_ignoresPlaylistId() { + Bundle extras = new Bundle(); + extras.putString(EXTRAS_KEY_PLAYLIST_ID, ""); + MediaDescriptionCompat descriptionCompat = + new MediaDescriptionCompat.Builder().setExtras(extras).build(); + + MediaMetadata mediaMetadata = + LegacyConversions.convertToMediaMetadata(descriptionCompat, RatingCompat.RATING_NONE); + + assertThat(mediaMetadata.playlistId).isNull(); + assertThat(mediaMetadata.extras).isNull(); + } + + @Test + public void convertToMediaMetadata_withCategoriesInMediaMetadataCompat_assignedToCategories() { + MediaMetadataCompat testMediaMetadataCompat = + new MediaMetadataCompat.Builder().putString(EXTRAS_KEY_CATEGORIES, "rock,indie").build(); + + MediaMetadata mediaMetadata = + LegacyConversions.convertToMediaMetadata(testMediaMetadataCompat, RatingCompat.RATING_NONE); + + assertThat(mediaMetadata.categories).containsExactly("rock", "indie").inOrder(); + assertThat(mediaMetadata.extras).isNull(); + } + + @Test + public void convertToMediaMetadata_withCategoriesInMediaDescriptionCompat_assignedToCategories() { + Bundle extras = new Bundle(); + extras.putString(EXTRAS_KEY_CATEGORIES, "rock,indie"); + MediaDescriptionCompat descriptionCompat = + new MediaDescriptionCompat.Builder().setExtras(extras).build(); + + MediaMetadata mediaMetadata = + LegacyConversions.convertToMediaMetadata(descriptionCompat, RatingCompat.RATING_NONE); + + assertThat(mediaMetadata.categories).containsExactly("rock", "indie").inOrder(); + assertThat(mediaMetadata.extras).isNull(); + } + + @Test + public void + convertToPlaylistMetadata_withPlaylistFallbacksInMediaMetadataCompat_setsTitleAndArtworkUri() { + MediaMetadataCompat metadataCompat = + new MediaMetadataCompat.Builder() + .putString(EXTRAS_KEY_PLAYLIST_TITLE, "Fallback Playlist Title") + .putString(EXTRAS_KEY_PLAYLIST_ARTWORK_URI, "https://example.com/playlist.jpg") + .build(); + + MediaMetadata playlistMetadata = + LegacyConversions.convertToPlaylistMetadata( + /* queueTitle= */ null, metadataCompat, /* descriptionCompat= */ null); + + assertThat(playlistMetadata.title.toString()).isEqualTo("Fallback Playlist Title"); + assertThat(playlistMetadata.artworkUri) + .isEqualTo(Uri.parse("https://example.com/playlist.jpg")); + } + + @Test + public void + convertToPlaylistMetadata_withPlaylistFallbacksInMediaDescriptionCompat_setsTitleAndArtworkUri() { + Bundle extras = new Bundle(); + extras.putString(EXTRAS_KEY_PLAYLIST_TITLE, "Description Playlist Title"); + extras.putString(EXTRAS_KEY_PLAYLIST_ARTWORK_URI, "https://example.com/desc_playlist.jpg"); + MediaDescriptionCompat descriptionCompat = + new MediaDescriptionCompat.Builder().setExtras(extras).build(); + + MediaMetadata playlistMetadata = + LegacyConversions.convertToPlaylistMetadata( + /* queueTitle= */ null, /* metadataCompat= */ null, descriptionCompat); + + assertThat(playlistMetadata.title.toString()).isEqualTo("Description Playlist Title"); + assertThat(playlistMetadata.artworkUri) + .isEqualTo(Uri.parse("https://example.com/desc_playlist.jpg")); + } + + @Test + public void convertToPlaylistMetadata_withQueueTitleAndFallbackTitle_prefersQueueTitle() { + MediaMetadataCompat metadataCompat = + new MediaMetadataCompat.Builder() + .putString(EXTRAS_KEY_PLAYLIST_TITLE, "Fallback Playlist Title") + .putString(EXTRAS_KEY_PLAYLIST_ARTWORK_URI, "https://example.com/playlist.jpg") + .build(); + + MediaMetadata playlistMetadata = + LegacyConversions.convertToPlaylistMetadata( + "Session Queue Title", metadataCompat, /* descriptionCompat= */ null); + + assertThat(playlistMetadata.title.toString()).isEqualTo("Session Queue Title"); + assertThat(playlistMetadata.artworkUri) + .isEqualTo(Uri.parse("https://example.com/playlist.jpg")); + } + + @Test + public void convertToMediaMetadata_withPlaylistFallbacksInItemMetadata_removesFromItemExtras() { + MediaMetadataCompat metadataCompat = + new MediaMetadataCompat.Builder() + .putString(EXTRAS_KEY_PLAYLIST_TITLE, "Fallback Playlist Title") + .putString(EXTRAS_KEY_PLAYLIST_ARTWORK_URI, "https://example.com/playlist.jpg") + .build(); + Bundle extras = new Bundle(); + extras.putString(EXTRAS_KEY_PLAYLIST_TITLE, "Description Playlist Title"); + extras.putString(EXTRAS_KEY_PLAYLIST_ARTWORK_URI, "https://example.com/desc_playlist.jpg"); + MediaDescriptionCompat descriptionCompat = + new MediaDescriptionCompat.Builder().setExtras(extras).build(); + + MediaMetadata fromMetadataCompat = + LegacyConversions.convertToMediaMetadata(metadataCompat, RatingCompat.RATING_NONE); + MediaMetadata fromDescriptionCompat = + LegacyConversions.convertToMediaMetadata(descriptionCompat, RatingCompat.RATING_NONE); + + assertThat(fromMetadataCompat.extras).isNull(); + assertThat(fromDescriptionCompat.extras).isNull(); + } + @Test public void convertToMediaMetadataCompat_withMediaType_setsMediaType() { MediaItem mediaItem = @@ -539,6 +710,47 @@ public void convertToMediaMetadataCompat_withMediaType_setsMediaType() { .isEqualTo(MediaMetadata.MEDIA_TYPE_MUSIC); } + @Test + public void convertToMediaMetadataCompat_withPlaylistId_setsPlaylistIdExtra() { + MediaItem mediaItem = + new MediaItem.Builder() + .setMediaMetadata(new MediaMetadata.Builder().setPlaylistId("playlistId-1").build()) + .build(); + + MediaMetadataCompat mediaMetadataCompat = + LegacyConversions.convertToMediaMetadataCompat( + mediaItem.mediaMetadata, + "mediaId", + Uri.parse("http://www.example.com"), + /* durationMs= */ C.TIME_UNSET, + /* artworkBitmap= */ null); + + assertThat(mediaMetadataCompat.getString(MediaConstants.EXTRAS_KEY_PLAYLIST_ID)) + .isEqualTo("playlistId-1"); + } + + @Test + public void convertToMediaMetadataCompat_withCategories_setsCategoriesExtra() { + MediaItem mediaItem = + new MediaItem.Builder() + .setMediaMetadata( + new MediaMetadata.Builder() + .setCategories(ImmutableList.of("category1", "category2")) + .build()) + .build(); + + MediaMetadataCompat mediaMetadataCompat = + LegacyConversions.convertToMediaMetadataCompat( + mediaItem.mediaMetadata, + "mediaId", + Uri.parse("http://www.example.com"), + /* durationMs= */ C.TIME_UNSET, + /* artworkBitmap= */ null); + + assertThat(mediaMetadataCompat.getString(EXTRAS_KEY_CATEGORIES)) + .isEqualTo("category1,category2"); + } + @Test public void convertToMediaMetadataCompat_populatesExtrasFromMediaMetadata() { Bundle extras = new Bundle(); @@ -1560,6 +1772,28 @@ public void convertToPlaybackException_populatesUnsetMessageFromErrorCode() { .isNotEmpty(); } + @Test + public void convertToDeviceInfo_withNegativeMaxVolume_returnsUnspecifiedMaxVolume() { + MediaSessionCompat sessionCompat = + new MediaSessionCompat( + context, "tag", /* sessionInfo= */ null, /* packageNameOverride= */ null); + MediaControllerCompat controllerCompat = new MediaControllerCompat(context, sessionCompat); + shadowOf((MediaController) controllerCompat.getMediaController()) + .setPlaybackInfo( + PlaybackInfoBuilder.newBuilder() + .setVolumeType(MediaController.PlaybackInfo.PLAYBACK_TYPE_REMOTE) + .setMaxVolume(-1) + .setAudioAttributes(new AudioAttributes.Builder().build()) + .build()); + + DeviceInfo deviceInfo = + LegacyConversions.convertToDeviceInfo(controllerCompat.getPlaybackInfo()); + sessionCompat.release(); + + assertThat(deviceInfo.playbackType).isEqualTo(DeviceInfo.PLAYBACK_TYPE_REMOTE); + assertThat(deviceInfo.maxVolume).isEqualTo(0); + } + private static PlaybackStateCompat createErrorPlaybackStateCompatWithoutMessage( @PlaybackStateCompat.ErrorCode int errorCode) { return new PlaybackStateCompat.Builder() diff --git a/libraries/session/src/test/java/androidx/media3/session/MediaSessionServiceTest.java b/libraries/session/src/test/java/androidx/media3/session/MediaSessionServiceTest.java index b5cd0352486..74f3d90bea1 100644 --- a/libraries/session/src/test/java/androidx/media3/session/MediaSessionServiceTest.java +++ b/libraries/session/src/test/java/androidx/media3/session/MediaSessionServiceTest.java @@ -40,6 +40,7 @@ import com.google.common.collect.ImmutableList; import com.google.common.util.concurrent.Futures; import com.google.common.util.concurrent.ListenableFuture; +import com.google.common.util.concurrent.SettableFuture; import com.google.testing.junit.testparameterinjector.TestParameter; import java.util.ArrayList; import java.util.List; @@ -945,6 +946,58 @@ public NotificationChannelInfo getNotificationChannelInfo() { serviceController.destroy(); } + @Test + public void + triggerNotificationUpdate_beforeMediaNotificationControllerConnected_updatesNotificationOnceConnected() { + ExoPlayer player = new TestExoPlayerBuilder(context).build(); + SettableFuture connectionResultFuture = SettableFuture.create(); + MediaSession session = + new MediaSession.Builder(context, player) + .setCallback( + new MediaSession.Callback() { + @Override + public ListenableFuture onConnectAsync( + MediaSession session, MediaSession.ControllerInfo controller) { + if (session.isMediaNotificationController(controller)) { + return connectionResultFuture; + } + return immediateFuture( + new MediaSession.ConnectionResult.AcceptedResultBuilder(session, controller) + .build()); + } + }) + .build(); + ServiceController serviceController = Robolectric.buildService(TestService.class); + TestService service = serviceController.create().get(); + service.setMediaNotificationProvider( + new DefaultMediaNotificationProvider( + service, + /* notificationIdProvider= */ unused -> 2000, + DefaultMediaNotificationProvider.DEFAULT_CHANNEL_ID, + DefaultMediaNotificationProvider.DEFAULT_CHANNEL_NAME_RESOURCE_ID)); + service.addSession(session); + player.setMediaItem(MediaItem.fromUri("asset:///media/mp4/sample.mp4")); + player.prepare(); + ShadowLooper.idleMainLooper(); + + service.triggerNotificationUpdate(); + ShadowLooper.idleMainLooper(); + StatusBarNotification notificationBeforeConnected = getStatusBarNotification(2000); + + connectionResultFuture.set( + MediaSession.ConnectionResult.accept( + MediaSession.ConnectionResult.DEFAULT_SESSION_COMMANDS, + MediaSession.ConnectionResult.DEFAULT_PLAYER_COMMANDS)); + ShadowLooper.idleMainLooper(); + + StatusBarNotification notificationAfterConnected = getStatusBarNotification(2000); + assertThat(notificationBeforeConnected).isNull(); + assertThat(notificationAfterConnected).isNotNull(); + session.release(); + player.release(); + serviceController.destroy(); + } + @Nullable private StatusBarNotification getStatusBarNotification(int notificationId) { for (StatusBarNotification notification : notificationManager.getActiveNotifications()) { diff --git a/libraries/session/src/test/java/androidx/media3/session/MediaSessionUnitTest.java b/libraries/session/src/test/java/androidx/media3/session/MediaSessionUnitTest.java index a55be7b4523..c1825fe7dc4 100644 --- a/libraries/session/src/test/java/androidx/media3/session/MediaSessionUnitTest.java +++ b/libraries/session/src/test/java/androidx/media3/session/MediaSessionUnitTest.java @@ -17,20 +17,43 @@ import static androidx.test.core.app.ApplicationProvider.getApplicationContext; import static com.google.common.truth.Truth.assertThat; - +import static com.google.common.util.concurrent.Futures.immediateFailedFuture; +import static com.google.common.util.concurrent.Futures.immediateFuture; +import static org.robolectric.Shadows.shadowOf; + +import android.app.PendingIntent; +import android.content.ComponentName; +import android.content.Context; +import android.content.Intent; +import android.content.IntentFilter; import android.graphics.Bitmap; import android.graphics.Color; +import android.net.Uri; import android.os.Bundle; +import android.support.v4.media.session.PlaybackStateCompat; +import androidx.media3.common.MediaItem; import androidx.media3.common.MediaLibraryInfo; +import androidx.media3.common.MediaMetadata; +import androidx.media3.common.PlaybackException; +import androidx.media3.common.Player; import androidx.media3.common.util.BitmapLoader; import androidx.media3.session.legacy.MediaSessionManager; import androidx.media3.test.utils.TestExoPlayerBuilder; import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import com.google.common.util.concurrent.ListenableFuture; +import com.google.common.util.concurrent.SettableFuture; import java.io.ByteArrayOutputStream; +import java.util.Objects; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicInteger; +import java.util.concurrent.atomic.AtomicReference; import org.junit.After; import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; +import org.robolectric.annotation.Config; +import org.robolectric.shadows.ShadowLooper; /** Tests for {@link MediaSession}. */ @RunWith(AndroidJUnit4.class) @@ -262,6 +285,577 @@ public void getBitmapLoader_defaultConfiguration_decodesToExactLimit() throws Ex assertThat(decodedBitmap.getHeight()).isEqualTo(limit); } + @Test + public void prepare_withControllerReleasedInListenerCallback_doesNotCrash() throws Exception { + AtomicReference controllerReference = new AtomicReference<>(); + Player.Listener listener = + new Player.Listener() { + @Override + public void onPlaybackStateChanged(int playbackState) { + MediaController controller = controllerReference.get(); + if (controller != null) { + controller.release(); + } + } + }; + MediaController controller = + new MediaController.Builder(getApplicationContext(), session.getToken()).buildAsync().get(); + controller.addListener(listener); + controllerReference.set(controller); + + session.getPlayer().prepare(); + ShadowLooper.idleMainLooper(); + } + + @Test + public void setSessionActivity_updatesSessionActivity() { + Intent intent = new Intent("test_action"); + PendingIntent sessionActivity = + PendingIntent.getActivity( + getApplicationContext(), /* requestCode= */ 0, intent, PendingIntent.FLAG_IMMUTABLE); + + session.setSessionActivity(sessionActivity); + ShadowLooper.idleMainLooper(); + + assertThat(session.getSessionActivity()).isEqualTo(sessionActivity); + + session.setSessionActivity(null); + ShadowLooper.idleMainLooper(); + + assertThat(session.getSessionActivity()).isNull(); + } + + @Test + public void builder_withSessionExtras_setsLegacyExtras() { + Bundle extras = new Bundle(); + extras.putString("test_key", "test_value"); + MediaSession testSession = + new MediaSession.Builder( + getApplicationContext(), new TestExoPlayerBuilder(getApplicationContext()).build()) + .setId("session_with_extras") + .setSessionExtras(extras) + .build(); + + assertThat( + testSession + .getImpl() + .getMediaSessionLegacyStub() + .getLegacyExtras() + .getString("test_key")) + .isEqualTo("test_value"); + + testSession.release(); + } + + @SuppressWarnings("deprecation") // Using MediaSessionCompat to test platform interoperability + @Test + public void setPlaybackException_standaloneSession_setsAndClearsPlatformSessionErrorState() { + PlaybackException exception = + new PlaybackException( + "error_msg", /* cause= */ null, PlaybackException.ERROR_CODE_REMOTE_ERROR); + + session.setPlaybackException(exception); + ShadowLooper.idleMainLooper(); + + assertThat( + session + .getImpl() + .getMediaSessionLegacyStub() + .getSessionCompat() + .getController() + .getPlaybackState() + .getState()) + .isEqualTo(PlaybackStateCompat.STATE_ERROR); + + session.setPlaybackException(null); + ShadowLooper.idleMainLooper(); + + assertThat( + session + .getImpl() + .getMediaSessionLegacyStub() + .getSessionCompat() + .getController() + .getPlaybackState() + .getState()) + .isNotEqualTo(PlaybackStateCompat.STATE_ERROR); + } + + @Test + @Config(sdk = {30, 31}) + public void buildAndRelease_pendingIntentThrowsSecurityException_doesNotThrow() { + Context context = getApplicationContext(); + ComponentName receiverComponent = + new ComponentName(context, "androidx.media3.session.MediaButtonReceiver"); + shadowOf(context.getPackageManager()).addReceiverIfNotPresent(receiverComponent); + shadowOf(context.getPackageManager()) + .addIntentFilterForReceiver( + receiverComponent, new IntentFilter(Intent.ACTION_MEDIA_BUTTON)); + // Seed ShadowPendingIntent with a broadcast PendingIntent whose filterEquals throws + // SecurityException whenever PendingIntent.getBroadcast is called for ACTION_MEDIA_BUTTON. + PendingIntent.getBroadcast( + context, + /* requestCode= */ 0, + new Intent() { + @Override + public boolean filterEquals(Intent other) { + if (other != null && Objects.equals(other.getAction(), Intent.ACTION_MEDIA_BUTTON)) { + throw new SecurityException("Too many PendingIntent created"); + } + return super.filterEquals(other); + } + }, + /* flags= */ PendingIntent.FLAG_IMMUTABLE); + Player player = new TestExoPlayerBuilder(context).build(); + + MediaSession testSession = + new MediaSession.Builder(context, player) + .setId("session_pending_intent_security_exception") + .build(); + testSession.release(); + + assertThat(testSession.getImpl().isReleased()).isTrue(); + player.release(); + } + + @Test + public void setMediaItems_withDuplicateArtwork_decodesEachUniqueArtworkOnce() { + Context context = getApplicationContext(); + Player player = new TestExoPlayerBuilder(context).build(); + AtomicInteger decodeCount = new AtomicInteger(); + MediaSession testSession = + new MediaSession.Builder(context, player) + .setId("session_duplicate_artwork") + .setBitmapLoader(createCountingBitmapLoader(decodeCount)) + .build(); + // The current item has no artwork, so only queue artwork is decoded. + ImmutableList mediaItems = + ImmutableList.of( + createMediaItemWithoutArtwork("item_0"), + createMediaItemWithArtwork("item_1", new byte[] {1}), + createMediaItemWithArtwork("item_2", new byte[] {2}), + createMediaItemWithArtwork("item_3", new byte[] {1})); + + player.setMediaItems(mediaItems); + ShadowLooper.idleMainLooper(); + + assertThat(decodeCount.get()).isEqualTo(2); + testSession.release(); + player.release(); + } + + @Test + public void addMediaItem_withExistingQueueArtwork_onlyDecodesNewArtwork() { + Context context = getApplicationContext(); + Player player = new TestExoPlayerBuilder(context).build(); + AtomicInteger decodeCount = new AtomicInteger(); + MediaSession testSession = + new MediaSession.Builder(context, player) + .setId("session_add_media_item") + .setBitmapLoader(createCountingBitmapLoader(decodeCount)) + .build(); + player.setMediaItems( + ImmutableList.of( + createMediaItemWithoutArtwork("item_0"), + createMediaItemWithArtwork("item_1", new byte[] {1}), + createMediaItemWithArtwork("item_2", new byte[] {2}))); + ShadowLooper.idleMainLooper(); + int decodeCountBeforeAdd = decodeCount.get(); + + player.addMediaItem(createMediaItemWithArtwork("item_3", new byte[] {3})); + ShadowLooper.idleMainLooper(); + + assertThat(decodeCount.get() - decodeCountBeforeAdd).isEqualTo(1); + testSession.release(); + player.release(); + } + + @Test + public void addMediaItem_withArtworkPreviouslyRemovedFromQueue_decodesArtworkAgain() { + Context context = getApplicationContext(); + Player player = new TestExoPlayerBuilder(context).build(); + AtomicInteger decodeCount = new AtomicInteger(); + MediaSession testSession = + new MediaSession.Builder(context, player) + .setId("session_re_add_removed_artwork") + .setBitmapLoader(createCountingBitmapLoader(decodeCount)) + .build(); + MediaItem mediaItemToReAdd = createMediaItemWithArtwork("item_1", new byte[] {1}); + player.setMediaItems( + ImmutableList.of( + createMediaItemWithoutArtwork("item_0"), + mediaItemToReAdd, + createMediaItemWithArtwork("item_2", new byte[] {2}), + createMediaItemWithArtwork("item_3", new byte[] {3}))); + ShadowLooper.idleMainLooper(); + player.removeMediaItem(/* index= */ 1); + ShadowLooper.idleMainLooper(); + int decodeCountBeforeAdd = decodeCount.get(); + + player.addMediaItem(mediaItemToReAdd); + ShadowLooper.idleMainLooper(); + + assertThat(decodeCount.get() - decodeCountBeforeAdd).isEqualTo(1); + testSession.release(); + player.release(); + } + + @Test + public void setMediaItems_whenPreviousQueueArtworkFinishesLate_doesNotOverwriteNewerQueue() { + Context context = getApplicationContext(); + Player player = new TestExoPlayerBuilder(context).build(); + SettableFuture slowArtworkFuture = SettableFuture.create(); + Bitmap bitmap = Bitmap.createBitmap(/* width= */ 10, /* height= */ 10, Bitmap.Config.ARGB_8888); + BitmapLoader bitmapLoader = + new BitmapLoader() { + @Override + public boolean supportsMimeType(String mimeType) { + return true; + } + + @Override + public ListenableFuture decodeBitmap(byte[] data) { + return data[0] == 1 ? slowArtworkFuture : immediateFuture(bitmap); + } + + @Override + public ListenableFuture loadBitmap(Uri uri) { + return immediateFailedFuture(new UnsupportedOperationException()); + } + }; + MediaSession testSession = + new MediaSession.Builder(context, player) + .setId("session_late_queue_artwork") + .setBitmapLoader(bitmapLoader) + .build(); + AtomicBoolean staleQueueConverted = new AtomicBoolean(); + MediaItem staleQueueItem = + new MediaItem.Builder() + .setMediaId("item_1") + .setUri("http://example.com/item_1") + .setMediaMetadata( + new MediaMetadata.Builder() + .setTitle(createAccessTrackingCharSequence("item_1", staleQueueConverted)) + .setArtworkData(new byte[] {1}, MediaMetadata.PICTURE_TYPE_FRONT_COVER) + .build()) + .build(); + AtomicBoolean newerQueueConverted = new AtomicBoolean(); + MediaItem newerQueueItem = + new MediaItem.Builder() + .setMediaId("item_2") + .setUri("http://example.com/item_2") + .setMediaMetadata( + new MediaMetadata.Builder() + .setTitle(createAccessTrackingCharSequence("item_2", newerQueueConverted)) + .setArtworkData(new byte[] {2}, MediaMetadata.PICTURE_TYPE_FRONT_COVER) + .build()) + .build(); + player.setMediaItems(ImmutableList.of(createMediaItemWithoutArtwork("item_0"), staleQueueItem)); + ShadowLooper.idleMainLooper(); + + player.setMediaItems(ImmutableList.of(createMediaItemWithoutArtwork("item_0"), newerQueueItem)); + ShadowLooper.idleMainLooper(); + slowArtworkFuture.set(bitmap); + ShadowLooper.idleMainLooper(); + + assertThat(newerQueueConverted.get()).isTrue(); + assertThat(staleQueueConverted.get()).isFalse(); + testSession.release(); + player.release(); + } + + @Test + public void + setMediaItems_toItemsWithoutArtworkWhenPreviousQueueArtworkFinishesLate_doesNotOverwriteNewerQueue() { + Context context = getApplicationContext(); + Player player = new TestExoPlayerBuilder(context).build(); + SettableFuture slowArtworkFuture = SettableFuture.create(); + BitmapLoader bitmapLoader = + new BitmapLoader() { + @Override + public boolean supportsMimeType(String mimeType) { + return true; + } + + @Override + public ListenableFuture decodeBitmap(byte[] data) { + return slowArtworkFuture; + } + + @Override + public ListenableFuture loadBitmap(Uri uri) { + return immediateFailedFuture(new UnsupportedOperationException()); + } + }; + MediaSession testSession = + new MediaSession.Builder(context, player) + .setId("session_late_queue_artwork_to_no_artwork") + .setBitmapLoader(bitmapLoader) + .build(); + AtomicBoolean staleQueueConverted = new AtomicBoolean(); + MediaItem staleQueueItem = + new MediaItem.Builder() + .setMediaId("item_1") + .setUri("http://example.com/item_1") + .setMediaMetadata( + new MediaMetadata.Builder() + .setTitle(createAccessTrackingCharSequence("item_1", staleQueueConverted)) + .setArtworkData(new byte[] {1}, MediaMetadata.PICTURE_TYPE_FRONT_COVER) + .build()) + .build(); + AtomicBoolean newerQueueConverted = new AtomicBoolean(); + MediaItem newerQueueItem = + new MediaItem.Builder() + .setMediaId("item_2") + .setUri("http://example.com/item_2") + .setMediaMetadata( + new MediaMetadata.Builder() + .setTitle(createAccessTrackingCharSequence("item_2", newerQueueConverted)) + .build()) + .build(); + player.setMediaItems(ImmutableList.of(createMediaItemWithoutArtwork("item_0"), staleQueueItem)); + ShadowLooper.idleMainLooper(); + + player.setMediaItems(ImmutableList.of(createMediaItemWithoutArtwork("item_0"), newerQueueItem)); + ShadowLooper.idleMainLooper(); + slowArtworkFuture.set( + Bitmap.createBitmap(/* width= */ 10, /* height= */ 10, Bitmap.Config.ARGB_8888)); + ShadowLooper.idleMainLooper(); + + assertThat(newerQueueConverted.get()).isTrue(); + assertThat(staleQueueConverted.get()).isFalse(); + testSession.release(); + player.release(); + } + + @Test + public void + seekToNextMediaItem_toItemWithoutArtworkBeforePreviousArtworkLoads_doesNotSetPreviousArtwork() { + Context context = getApplicationContext(); + Player player = new TestExoPlayerBuilder(context).build(); + SettableFuture artworkFuture = SettableFuture.create(); + BitmapLoader bitmapLoader = + new BitmapLoader() { + @Override + public boolean supportsMimeType(String mimeType) { + return true; + } + + @Override + public ListenableFuture decodeBitmap(byte[] data) { + return artworkFuture; + } + + @Override + public ListenableFuture loadBitmap(Uri uri) { + return immediateFailedFuture(new UnsupportedOperationException()); + } + }; + MediaSession testSession = + new MediaSession.Builder(context, player) + .setId("session_stale_metadata_artwork") + .setBitmapLoader(bitmapLoader) + .build(); + AtomicBoolean notificationRefreshRequired = new AtomicBoolean(); + testSession + .getImpl() + .setMediaSessionListener( + new MediaSession.Listener() { + @Override + public void onNotificationRefreshRequired(MediaSession session) { + notificationRefreshRequired.set(true); + } + + @Override + public ListenableFuture onPlayRequested(MediaSession session) { + return immediateFuture(true); + } + }); + player.setMediaItems( + ImmutableList.of( + createMediaItemWithArtwork("item_0", new byte[] {1}), + createMediaItemWithoutArtwork("item_1"))); + ShadowLooper.idleMainLooper(); + + player.seekToNextMediaItem(); + ShadowLooper.idleMainLooper(); + artworkFuture.set( + Bitmap.createBitmap(/* width= */ 10, /* height= */ 10, Bitmap.Config.ARGB_8888)); + ShadowLooper.idleMainLooper(); + + assertThat(notificationRefreshRequired.get()).isFalse(); + testSession.release(); + player.release(); + } + + @Test + public void release_withPendingArtworkLoads_ignoresLateCompletions() { + Context context = getApplicationContext(); + Player player = new TestExoPlayerBuilder(context).build(); + SettableFuture metadataArtworkFuture = SettableFuture.create(); + SettableFuture queueArtworkFuture = SettableFuture.create(); + BitmapLoader bitmapLoader = + new BitmapLoader() { + @Override + public boolean supportsMimeType(String mimeType) { + return true; + } + + @Override + public ListenableFuture decodeBitmap(byte[] data) { + return data[0] == 1 ? metadataArtworkFuture : queueArtworkFuture; + } + + @Override + public ListenableFuture loadBitmap(Uri uri) { + return immediateFailedFuture(new UnsupportedOperationException()); + } + }; + MediaSession testSession = + new MediaSession.Builder(context, player) + .setId("session_release_pending_artwork") + .setBitmapLoader(bitmapLoader) + .build(); + AtomicBoolean notificationRefreshRequired = new AtomicBoolean(); + testSession + .getImpl() + .setMediaSessionListener( + new MediaSession.Listener() { + @Override + public void onNotificationRefreshRequired(MediaSession session) { + notificationRefreshRequired.set(true); + } + + @Override + public ListenableFuture onPlayRequested(MediaSession session) { + return immediateFuture(true); + } + }); + AtomicBoolean queueConverted = new AtomicBoolean(); + MediaItem queueItem = + new MediaItem.Builder() + .setMediaId("item_1") + .setUri("http://example.com/item_1") + .setMediaMetadata( + new MediaMetadata.Builder() + .setTitle(createAccessTrackingCharSequence("item_1", queueConverted)) + .setArtworkData(new byte[] {2}, MediaMetadata.PICTURE_TYPE_FRONT_COVER) + .build()) + .build(); + player.setMediaItems( + ImmutableList.of(createMediaItemWithArtwork("item_0", new byte[] {1}), queueItem)); + ShadowLooper.idleMainLooper(); + + testSession.release(); + metadataArtworkFuture.set( + Bitmap.createBitmap(/* width= */ 10, /* height= */ 10, Bitmap.Config.ARGB_8888)); + queueArtworkFuture.set( + Bitmap.createBitmap(/* width= */ 10, /* height= */ 10, Bitmap.Config.ARGB_8888)); + ShadowLooper.idleMainLooper(); + + assertThat(notificationRefreshRequired.get()).isFalse(); + assertThat(queueConverted.get()).isFalse(); + player.release(); + } + + @Test + public void onDisconnected_withCachedQueueArtwork_clearsCachedBitmaps() { + Context context = getApplicationContext(); + Player player = new TestExoPlayerBuilder(context).build(); + AtomicInteger decodeCount = new AtomicInteger(); + MediaSession testSession = + new MediaSession.Builder(context, player) + .setId("session_disconnected_clears_cache") + .setBitmapLoader(createCountingBitmapLoader(decodeCount)) + .build(); + player.setMediaItems( + ImmutableList.of( + createMediaItemWithoutArtwork("item_0"), + createMediaItemWithArtwork("item_1", new byte[] {1}), + createMediaItemWithArtwork("item_2", new byte[] {2}))); + ShadowLooper.idleMainLooper(); + int decodeCountBeforeDisconnect = decodeCount.get(); + + testSession + .getImpl() + .getMediaSessionLegacyStub() + .getControllerLegacyCbForBroadcast() + .onDisconnected(/* seq= */ 0); + player.addMediaItem(createMediaItemWithArtwork("item_3", new byte[] {3})); + ShadowLooper.idleMainLooper(); + + assertThat(decodeCount.get() - decodeCountBeforeDisconnect).isEqualTo(3); + testSession.release(); + player.release(); + } + + private static BitmapLoader createCountingBitmapLoader(AtomicInteger decodeCount) { + return new BitmapLoader() { + @Override + public boolean supportsMimeType(String mimeType) { + return true; + } + + @Override + public ListenableFuture decodeBitmap(byte[] data) { + decodeCount.incrementAndGet(); + return immediateFuture( + Bitmap.createBitmap(/* width= */ 10, /* height= */ 10, Bitmap.Config.ARGB_8888)); + } + + @Override + public ListenableFuture loadBitmap(Uri uri) { + return immediateFailedFuture(new UnsupportedOperationException()); + } + }; + } + + private static MediaItem createMediaItemWithoutArtwork(String mediaId) { + return new MediaItem.Builder() + .setMediaId(mediaId) + .setUri("http://example.com/" + mediaId) + .build(); + } + + private static MediaItem createMediaItemWithArtwork(String mediaId, byte[] artworkData) { + return new MediaItem.Builder() + .setMediaId(mediaId) + .setUri("http://example.com/" + mediaId) + .setMediaMetadata( + new MediaMetadata.Builder() + .setArtworkData(artworkData, MediaMetadata.PICTURE_TYPE_FRONT_COVER) + .build()) + .build(); + } + + private static CharSequence createAccessTrackingCharSequence( + String value, AtomicBoolean accessed) { + return new CharSequence() { + @Override + public int length() { + accessed.set(true); + return value.length(); + } + + @Override + public char charAt(int index) { + accessed.set(true); + return value.charAt(index); + } + + @Override + public CharSequence subSequence(int start, int end) { + accessed.set(true); + return value.subSequence(start, end); + } + + @Override + public String toString() { + return value; + } + }; + } + private static MediaSession.ControllerInfo createMinimalLegacyControllerInfo( MediaSessionManager.RemoteUserInfo remoteUserInfo) { return new MediaSession.ControllerInfo( diff --git a/libraries/session/src/test/java/androidx/media3/session/SimpleBitmapLoaderTest.java b/libraries/session/src/test/java/androidx/media3/session/SimpleBitmapLoaderTest.java index 4fff8443b6c..fbb5742b5dc 100644 --- a/libraries/session/src/test/java/androidx/media3/session/SimpleBitmapLoaderTest.java +++ b/libraries/session/src/test/java/androidx/media3/session/SimpleBitmapLoaderTest.java @@ -21,6 +21,7 @@ import android.graphics.Bitmap; import android.graphics.BitmapFactory; import android.net.Uri; +import androidx.annotation.Nullable; import androidx.media3.common.MediaMetadata; import androidx.media3.test.utils.TestUtil; import androidx.test.core.app.ApplicationProvider; @@ -36,6 +37,7 @@ import okhttp3.mockwebserver.MockResponse; import okhttp3.mockwebserver.MockWebServer; import okio.Buffer; +import org.junit.After; import org.junit.Rule; import org.junit.Test; import org.junit.function.ThrowingRunnable; @@ -52,6 +54,15 @@ public class SimpleBitmapLoaderTest { @Rule public final TemporaryFolder tempFolder = new TemporaryFolder(); + @Nullable private MockWebServer mockWebServer; + + @After + public void tearDown() throws Exception { + if (mockWebServer != null) { + mockWebServer.shutdown(); + } + } + @Test public void loadData() throws Exception { SimpleBitmapLoader bitmapLoader = @@ -86,7 +97,7 @@ public void loadData_withInvalidData_throwsException() { public void load_httpUri_loadsImage() throws Exception { SimpleBitmapLoader bitmapLoader = new SimpleBitmapLoader(MoreExecutors.newDirectExecutorService()); - MockWebServer mockWebServer = new MockWebServer(); + mockWebServer = new MockWebServer(); byte[] imageData = TestUtil.getByteArray(ApplicationProvider.getApplicationContext(), TEST_IMAGE_PATH); Buffer responseBody = new Buffer().write(imageData); @@ -105,7 +116,7 @@ public void load_httpUri_loadsImage() throws Exception { public void load_httpUriAndServerError_throwsException() { SimpleBitmapLoader bitmapLoader = new SimpleBitmapLoader(MoreExecutors.newDirectExecutorService()); - MockWebServer mockWebServer = new MockWebServer(); + mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse().setResponseCode(404)); ListenableFuture future = @@ -172,7 +183,7 @@ public void load_unhandledUriScheme_throwsException() { public void loadBitmapFromMetadata_decodeFromArtworkData() throws Exception { byte[] imageData = TestUtil.getByteArray(ApplicationProvider.getApplicationContext(), TEST_IMAGE_PATH); - MockWebServer mockWebServer = new MockWebServer(); + mockWebServer = new MockWebServer(); Uri uri = Uri.parse(mockWebServer.url("test_path").toString()); // Set both artworkData and artworkUri MediaMetadata metadata = @@ -196,7 +207,7 @@ public void loadBitmapFromMetadata_decodeFromArtworkData() throws Exception { public void loadBitmapFromMetadata_loadFromArtworkUri() throws Exception { byte[] imageData = TestUtil.getByteArray(ApplicationProvider.getApplicationContext(), TEST_IMAGE_PATH); - MockWebServer mockWebServer = new MockWebServer(); + mockWebServer = new MockWebServer(); Buffer responseBody = new Buffer().write(imageData); mockWebServer.enqueue(new MockResponse().setResponseCode(200).setBody(responseBody)); Uri uri = Uri.parse(mockWebServer.url("test_path").toString()); diff --git a/libraries/session/src/test/java/androidx/media3/session/SizeAvoidingBitmapLoaderTest.java b/libraries/session/src/test/java/androidx/media3/session/SizeAvoidingBitmapLoaderTest.java index 9e62740b22f..507c810b304 100644 --- a/libraries/session/src/test/java/androidx/media3/session/SizeAvoidingBitmapLoaderTest.java +++ b/libraries/session/src/test/java/androidx/media3/session/SizeAvoidingBitmapLoaderTest.java @@ -22,6 +22,7 @@ import android.content.Context; import android.graphics.Bitmap; import android.net.Uri; +import androidx.annotation.Nullable; import androidx.media3.common.MediaMetadata; import androidx.media3.common.util.BitmapLoader; import androidx.media3.datasource.DataSourceBitmapLoader; @@ -32,6 +33,7 @@ import okhttp3.mockwebserver.MockResponse; import okhttp3.mockwebserver.MockWebServer; import okio.Buffer; +import org.junit.After; import org.junit.Before; import org.junit.Rule; import org.junit.Test; @@ -47,12 +49,20 @@ public class SizeAvoidingBitmapLoaderTest { @Rule public final TemporaryFolder tempFolder = new TemporaryFolder(); private Context context; + @Nullable private MockWebServer mockWebServer; @Before public void setUp() { context = ApplicationProvider.getApplicationContext(); } + @After + public void tearDown() throws Exception { + if (mockWebServer != null) { + mockWebServer.shutdown(); + } + } + @Test public void overridesAllMethods() throws NoSuchMethodException { assertSubclassOverridesAllMethods(BitmapLoader.class, SizeAvoidingBitmapLoader.class); @@ -117,7 +127,7 @@ public void loadBitmap_sizeBelowLimit_keepsSize() throws Exception { /* avoidSizes= */ ImmutableIntArray.of(200)); byte[] imageData = TestUtil.getByteArray(context, TEST_IMAGE_PATH); Buffer responseBody = new Buffer().write(imageData); - MockWebServer mockWebServer = new MockWebServer(); + mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse().setResponseCode(200).setBody(responseBody)); Uri uri = Uri.parse(mockWebServer.url("test_path").toString()); @@ -137,7 +147,7 @@ public void loadBitmap_sizeAboveLimit_keepsSize() throws Exception { /* avoidSizes= */ ImmutableIntArray.of(200)); byte[] imageData = TestUtil.getByteArray(context, TEST_IMAGE_PATH); Buffer responseBody = new Buffer().write(imageData); - MockWebServer mockWebServer = new MockWebServer(); + mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse().setResponseCode(200).setBody(responseBody)); Uri uri = Uri.parse(mockWebServer.url("test_path").toString()); @@ -157,7 +167,7 @@ public void loadBitmap_sizeAtLimit_changesSize() throws Exception { /* avoidSizes= */ ImmutableIntArray.of(200)); byte[] imageData = TestUtil.getByteArray(context, TEST_IMAGE_PATH); Buffer responseBody = new Buffer().write(imageData); - MockWebServer mockWebServer = new MockWebServer(); + mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse().setResponseCode(200).setBody(responseBody)); Uri uri = Uri.parse(mockWebServer.url("test_path").toString()); diff --git a/libraries/session/src/test/java/androidx/media3/session/SizeLimitedBitmapLoaderTest.java b/libraries/session/src/test/java/androidx/media3/session/SizeLimitedBitmapLoaderTest.java index 68c4daac122..b7b86184bbf 100644 --- a/libraries/session/src/test/java/androidx/media3/session/SizeLimitedBitmapLoaderTest.java +++ b/libraries/session/src/test/java/androidx/media3/session/SizeLimitedBitmapLoaderTest.java @@ -25,6 +25,7 @@ import android.graphics.BitmapFactory; import android.graphics.Matrix; import android.net.Uri; +import androidx.annotation.Nullable; import androidx.media3.common.MediaMetadata; import androidx.media3.common.util.BitmapLoader; import androidx.media3.datasource.DataSourceBitmapLoader; @@ -37,6 +38,7 @@ import okhttp3.mockwebserver.MockResponse; import okhttp3.mockwebserver.MockWebServer; import okio.Buffer; +import org.junit.After; import org.junit.Before; import org.junit.Rule; import org.junit.Test; @@ -55,12 +57,20 @@ public class SizeLimitedBitmapLoaderTest { @Rule public final TemporaryFolder tempFolder = new TemporaryFolder(); private Context context; + @Nullable private MockWebServer mockWebServer; @Before public void setUp() { context = ApplicationProvider.getApplicationContext(); } + @After + public void tearDown() throws Exception { + if (mockWebServer != null) { + mockWebServer.shutdown(); + } + } + @Test public void overridesAllMethods() throws NoSuchMethodException { assertSubclassOverridesAllMethods(BitmapLoader.class, SizeLimitedBitmapLoader.class); @@ -88,7 +98,7 @@ public void loadBitmapWithLimit() throws Exception { new DataSourceBitmapLoader.Builder(context).build(), limit, /* makeShared= */ false); byte[] imageData = TestUtil.getByteArray(context, TEST_IMAGE_PATH); Buffer responseBody = new Buffer().write(imageData); - MockWebServer mockWebServer = new MockWebServer(); + mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse().setResponseCode(200).setBody(responseBody)); Uri uri = Uri.parse(mockWebServer.url("test_path").toString()); Bitmap expectedBitmap = getExpectedBitmap(imageData, limit); @@ -109,7 +119,7 @@ public void loadBitmapWithLimitWithDifferentUris() throws Exception { byte[] imageData2 = TestUtil.getByteArray(context, SECOND_TEST_IMAGE_PATH); Buffer responseBody1 = new Buffer().write(imageData1); Buffer responseBody2 = new Buffer().write(imageData2); - MockWebServer mockWebServer = new MockWebServer(); + mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse().setResponseCode(200).setBody(responseBody1)); mockWebServer.enqueue(new MockResponse().setResponseCode(200).setBody(responseBody2)); Uri uri1 = Uri.parse(mockWebServer.url("test_path_1").toString()); @@ -135,7 +145,7 @@ public void loadBitmapWithLimitWithInvalidUri() { SizeLimitedBitmapLoader sizeLimitedBitmapLoader = new SizeLimitedBitmapLoader( new DataSourceBitmapLoader.Builder(context).build(), limit, /* makeShared= */ false); - MockWebServer mockWebServer = new MockWebServer(); + mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse().setResponseCode(404)); Uri uri = Uri.parse(mockWebServer.url("test_path").toString()); @@ -159,7 +169,7 @@ public void loadBitmapFromMetadata_forwardsCallToWrappedLoader() throws Exceptio SizeLimitedBitmapLoader sizeLimitedBitmapLoader = new SizeLimitedBitmapLoader(testBitmapLoader, limit, /* makeShared= */ false); Buffer responseBody = new Buffer().write(imageData); - MockWebServer mockWebServer = new MockWebServer(); + mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse().setResponseCode(200).setBody(responseBody)); Uri uri = Uri.parse(mockWebServer.url("test_path").toString()); MediaMetadata metadata = new MediaMetadata.Builder().setArtworkUri(uri).build(); @@ -211,7 +221,7 @@ public void loadBitmap_sizeUnderLimitAndMakeShared_returnsImmutableInstance() th new DataSourceBitmapLoader.Builder(context).build(), limit, /* makeShared= */ true); byte[] imageData = TestUtil.getByteArray(context, TEST_IMAGE_PATH); Buffer responseBody = new Buffer().write(imageData); - MockWebServer mockWebServer = new MockWebServer(); + mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse().setResponseCode(200).setBody(responseBody)); Uri uri = Uri.parse(mockWebServer.url("test_path").toString()); @@ -231,7 +241,7 @@ public void loadBitmap_sizeOverLimitAndMakeShared_returnsImmutableInstance() thr new DataSourceBitmapLoader.Builder(context).build(), limit, /* makeShared= */ true); byte[] imageData = TestUtil.getByteArray(context, TEST_IMAGE_PATH); Buffer responseBody = new Buffer().write(imageData); - MockWebServer mockWebServer = new MockWebServer(); + mockWebServer = new MockWebServer(); mockWebServer.enqueue(new MockResponse().setResponseCode(200).setBody(responseBody)); Uri uri = Uri.parse(mockWebServer.url("test_path").toString()); diff --git a/libraries/test_data/src/test/assets/audiosinkdumps/mp4/sample_mhm1_prefaudiolang.mp4.nonasciicmd.asi.dump b/libraries/test_data/src/test/assets/audiosinkdumps/mp4/sample_mhm1_prefaudiolang.mp4.nonasciicmd.asi.dump new file mode 100644 index 00000000000..154240907a6 --- /dev/null +++ b/libraries/test_data/src/test/assets/audiosinkdumps/mp4/sample_mhm1_prefaudiolang.mp4.nonasciicmd.asi.dump @@ -0,0 +1,46 @@ +MPEG-H ASI 0 = + + + + + + + + + p1 + + + + + + + g1 + + + + + + g2 + + + + + + g4 + + + + + + g3 + + + + + s1 + + + + + + diff --git a/libraries/test_data/src/test/assets/audiosinkdumps/mp4/sample_mhm1_prefaudiolang.mp4.nonasciicmd.audiosink.dump b/libraries/test_data/src/test/assets/audiosinkdumps/mp4/sample_mhm1_prefaudiolang.mp4.nonasciicmd.audiosink.dump new file mode 100644 index 00000000000..2794bf7bff6 --- /dev/null +++ b/libraries/test_data/src/test/assets/audiosinkdumps/mp4/sample_mhm1_prefaudiolang.mp4.nonasciicmd.audiosink.dump @@ -0,0 +1,170 @@ +AudioSink: + buffer count = 41 + discontinuity: + config: + pcmEncoding = pcm-16 + channelCount = 2 + sampleRate = 48000 + buffer #0: + time = 1000000000000 + channel[0] = 282783674 + channel[1] = 282783674 + buffer #1: + time = 1000000021333 + channel[0] = -1443166734 + channel[1] = -1443166734 + buffer #2: + time = 1000000042666 + channel[0] = -664167358 + channel[1] = -664167358 + buffer #3: + time = 1000000064000 + channel[0] = -1357830080 + channel[1] = -1357830080 + buffer #4: + time = 1000000085333 + channel[0] = -517045533 + channel[1] = -517045533 + buffer #5: + time = 1000000106666 + channel[0] = -238596381 + channel[1] = -238596381 + buffer #6: + time = 1000000128000 + channel[0] = 216065051 + channel[1] = 216065051 + buffer #7: + time = 1000000149333 + channel[0] = -876388976 + channel[1] = -876388976 + buffer #8: + time = 1000000170666 + channel[0] = 19660189 + channel[1] = 19660189 + buffer #9: + time = 1000000192000 + channel[0] = 1646453118 + channel[1] = 1646453118 + buffer #10: + time = 1000000213333 + channel[0] = 755521285 + channel[1] = 755521285 + buffer #11: + time = 1000000234666 + channel[0] = -1278578714 + channel[1] = -1278578714 + buffer #12: + time = 1000000256000 + channel[0] = -505412467 + channel[1] = -505412467 + buffer #13: + time = 1000000277333 + channel[0] = 1268823513 + channel[1] = 1268823513 + buffer #14: + time = 1000000298666 + channel[0] = -279779843 + channel[1] = -279779843 + buffer #15: + time = 1000000320000 + channel[0] = 965962998 + channel[1] = 965962998 + buffer #16: + time = 1000000341333 + channel[0] = 1916683672 + channel[1] = 1916683672 + buffer #17: + time = 1000000362666 + channel[0] = 1159431915 + channel[1] = 1159431915 + buffer #18: + time = 1000000384000 + channel[0] = -1569393028 + channel[1] = -1569393028 + buffer #19: + time = 1000000405333 + channel[0] = -144016301 + channel[1] = -144016301 + buffer #20: + time = 1000000426666 + channel[0] = 2049508790 + channel[1] = 2049508790 + buffer #21: + time = 1000000448000 + channel[0] = 1540502013 + channel[1] = 1540502013 + buffer #22: + time = 1000000469333 + channel[0] = 1803818321 + channel[1] = 1803818321 + buffer #23: + time = 1000000490666 + channel[0] = 1249500551 + channel[1] = 1249500551 + buffer #24: + time = 1000000512000 + channel[0] = -741827833 + channel[1] = -741827833 + buffer #25: + time = 1000000533333 + channel[0] = 1510404518 + channel[1] = 1510404518 + buffer #26: + time = 1000000554666 + channel[0] = 640973283 + channel[1] = 640973283 + buffer #27: + time = 1000000576000 + channel[0] = 667927860 + channel[1] = 667927860 + buffer #28: + time = 1000000597333 + channel[0] = -1400252340 + channel[1] = -1400252340 + buffer #29: + time = 1000000618666 + channel[0] = -109250957 + channel[1] = -109250957 + buffer #30: + time = 1000000640000 + channel[0] = -1213175193 + channel[1] = -1213175193 + buffer #31: + time = 1000000661333 + channel[0] = 130930266 + channel[1] = 130930266 + buffer #32: + time = 1000000682666 + channel[0] = 635431592 + channel[1] = 635431592 + buffer #33: + time = 1000000704000 + channel[0] = 66494029 + channel[1] = 66494029 + buffer #34: + time = 1000000725333 + channel[0] = 1712210053 + channel[1] = 1712210053 + buffer #35: + time = 1000000746666 + channel[0] = -1176226604 + channel[1] = -1176226604 + buffer #36: + time = 1000000768000 + channel[0] = 960464384 + channel[1] = 960464384 + buffer #37: + time = 1000000789333 + channel[0] = -712964186 + channel[1] = -712964186 + buffer #38: + time = 1000000810666 + channel[0] = -1435730719 + channel[1] = -1435730719 + buffer #39: + time = 1000000832000 + channel[0] = -1658363839 + channel[1] = -1658363839 + buffer #40: + time = 1000000853333 + data = 4096 zeroes diff --git a/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.0.dump b/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.0.dump index bbd970829e5..25a786b35f1 100644 --- a/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.0.dump @@ -78,11 +78,11 @@ track 0: data = length 116, hash 92DC289E sample 11: time = 299400 - flags = 0 + flags = 67108864 data = length 48, hash 4A8EA0DD sample 12: time = 365933 - flags = 0 + flags = 67108864 data = length 53, hash 33956169 sample 13: time = 432466 @@ -102,11 +102,11 @@ track 0: data = length 101, hash BF2D0E0D sample 17: time = 499000 - flags = 0 + flags = 67108864 data = length 65, hash 4D38F821 sample 18: time = 565533 - flags = 0 + flags = 67108864 data = length 52, hash 185FA7F3 sample 19: time = 698600 @@ -118,7 +118,7 @@ track 0: data = length 60, hash BC84E6AD sample 21: time = 665333 - flags = 0 + flags = 67108864 data = length 51, hash 43D309D8 sample 22: time = 765133 @@ -126,7 +126,7 @@ track 0: data = length 170, hash 340FC1CF sample 23: time = 731866 - flags = 0 + flags = 67108864 data = length 44, hash FEF28DB8 sample 24: time = 798400 @@ -162,11 +162,11 @@ track 0: data = length 91, hash 2AEDA634 sample 32: time = 998000 - flags = 0 + flags = 67108864 data = length 60, hash 9479657E sample 33: time = 1064533 - flags = 0 + flags = 67108864 data = length 43, hash CE267529 sample 34: time = 1131066 @@ -423,5 +423,5 @@ track 1024: sample count = 0 format 0: containerMimeType = image/heic - metadata = entries=[Motion photo metadata: photoStartPosition=0, photoSize=28853, photoPresentationTimestampUs=-9223372036854775807, videoStartPosition=28869, videoSize=28803] + metadata = entries=[Motion photo metadata: photoStartPosition=0, photoSize=28853, photoPresentationTimestampUs=0, videoStartPosition=28869, videoSize=28803] tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.1.dump b/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.1.dump index f3294889ef2..269a0696981 100644 --- a/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.1.dump @@ -78,11 +78,11 @@ track 0: data = length 116, hash 92DC289E sample 11: time = 299400 - flags = 0 + flags = 67108864 data = length 48, hash 4A8EA0DD sample 12: time = 365933 - flags = 0 + flags = 67108864 data = length 53, hash 33956169 sample 13: time = 432466 @@ -102,11 +102,11 @@ track 0: data = length 101, hash BF2D0E0D sample 17: time = 499000 - flags = 0 + flags = 67108864 data = length 65, hash 4D38F821 sample 18: time = 565533 - flags = 0 + flags = 67108864 data = length 52, hash 185FA7F3 sample 19: time = 698600 @@ -118,7 +118,7 @@ track 0: data = length 60, hash BC84E6AD sample 21: time = 665333 - flags = 0 + flags = 67108864 data = length 51, hash 43D309D8 sample 22: time = 765133 @@ -126,7 +126,7 @@ track 0: data = length 170, hash 340FC1CF sample 23: time = 731866 - flags = 0 + flags = 67108864 data = length 44, hash FEF28DB8 sample 24: time = 798400 @@ -162,11 +162,11 @@ track 0: data = length 91, hash 2AEDA634 sample 32: time = 998000 - flags = 0 + flags = 67108864 data = length 60, hash 9479657E sample 33: time = 1064533 - flags = 0 + flags = 67108864 data = length 43, hash CE267529 sample 34: time = 1131066 @@ -355,5 +355,5 @@ track 1024: sample count = 0 format 0: containerMimeType = image/heic - metadata = entries=[Motion photo metadata: photoStartPosition=0, photoSize=28853, photoPresentationTimestampUs=-9223372036854775807, videoStartPosition=28869, videoSize=28803] + metadata = entries=[Motion photo metadata: photoStartPosition=0, photoSize=28853, photoPresentationTimestampUs=0, videoStartPosition=28869, videoSize=28803] tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.2.dump b/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.2.dump index 63537640a88..f308f27dc68 100644 --- a/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.2.dump @@ -78,11 +78,11 @@ track 0: data = length 116, hash 92DC289E sample 11: time = 299400 - flags = 0 + flags = 67108864 data = length 48, hash 4A8EA0DD sample 12: time = 365933 - flags = 0 + flags = 67108864 data = length 53, hash 33956169 sample 13: time = 432466 @@ -102,11 +102,11 @@ track 0: data = length 101, hash BF2D0E0D sample 17: time = 499000 - flags = 0 + flags = 67108864 data = length 65, hash 4D38F821 sample 18: time = 565533 - flags = 0 + flags = 67108864 data = length 52, hash 185FA7F3 sample 19: time = 698600 @@ -118,7 +118,7 @@ track 0: data = length 60, hash BC84E6AD sample 21: time = 665333 - flags = 0 + flags = 67108864 data = length 51, hash 43D309D8 sample 22: time = 765133 @@ -126,7 +126,7 @@ track 0: data = length 170, hash 340FC1CF sample 23: time = 731866 - flags = 0 + flags = 67108864 data = length 44, hash FEF28DB8 sample 24: time = 798400 @@ -162,11 +162,11 @@ track 0: data = length 91, hash 2AEDA634 sample 32: time = 998000 - flags = 0 + flags = 67108864 data = length 60, hash 9479657E sample 33: time = 1064533 - flags = 0 + flags = 67108864 data = length 43, hash CE267529 sample 34: time = 1131066 @@ -279,5 +279,5 @@ track 1024: sample count = 0 format 0: containerMimeType = image/heic - metadata = entries=[Motion photo metadata: photoStartPosition=0, photoSize=28853, photoPresentationTimestampUs=-9223372036854775807, videoStartPosition=28869, videoSize=28803] + metadata = entries=[Motion photo metadata: photoStartPosition=0, photoSize=28853, photoPresentationTimestampUs=0, videoStartPosition=28869, videoSize=28803] tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.3.dump b/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.3.dump index 3166e912a6b..8e2c48f14f7 100644 --- a/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.3.dump @@ -78,11 +78,11 @@ track 0: data = length 116, hash 92DC289E sample 11: time = 299400 - flags = 0 + flags = 67108864 data = length 48, hash 4A8EA0DD sample 12: time = 365933 - flags = 0 + flags = 67108864 data = length 53, hash 33956169 sample 13: time = 432466 @@ -102,11 +102,11 @@ track 0: data = length 101, hash BF2D0E0D sample 17: time = 499000 - flags = 0 + flags = 67108864 data = length 65, hash 4D38F821 sample 18: time = 565533 - flags = 0 + flags = 67108864 data = length 52, hash 185FA7F3 sample 19: time = 698600 @@ -118,7 +118,7 @@ track 0: data = length 60, hash BC84E6AD sample 21: time = 665333 - flags = 0 + flags = 67108864 data = length 51, hash 43D309D8 sample 22: time = 765133 @@ -126,7 +126,7 @@ track 0: data = length 170, hash 340FC1CF sample 23: time = 731866 - flags = 0 + flags = 67108864 data = length 44, hash FEF28DB8 sample 24: time = 798400 @@ -162,11 +162,11 @@ track 0: data = length 91, hash 2AEDA634 sample 32: time = 998000 - flags = 0 + flags = 67108864 data = length 60, hash 9479657E sample 33: time = 1064533 - flags = 0 + flags = 67108864 data = length 43, hash CE267529 sample 34: time = 1131066 @@ -207,5 +207,5 @@ track 1024: sample count = 0 format 0: containerMimeType = image/heic - metadata = entries=[Motion photo metadata: photoStartPosition=0, photoSize=28853, photoPresentationTimestampUs=-9223372036854775807, videoStartPosition=28869, videoSize=28803] + metadata = entries=[Motion photo metadata: photoStartPosition=0, photoSize=28853, photoPresentationTimestampUs=0, videoStartPosition=28869, videoSize=28803] tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.unknown_length.dump index 5898b5c13fe..3ca9be14722 100644 --- a/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/heif/sample_MP.heic.unknown_length.dump @@ -8,5 +8,5 @@ track 1024: sample count = 0 format 0: containerMimeType = image/heic - metadata = entries=[Motion photo metadata: photoStartPosition=0, photoSize=28853, photoPresentationTimestampUs=-9223372036854775807, videoStartPosition=28869, videoSize=28803] + metadata = entries=[Motion photo metadata: photoStartPosition=0, photoSize=28853, photoPresentationTimestampUs=0, videoStartPosition=28869, videoSize=28803] tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/jpeg/pixel-motion-photo-2-hevc-tracks.jpg.0.dump b/libraries/test_data/src/test/assets/extractordumps/jpeg/pixel-motion-photo-2-hevc-tracks.jpg.0.dump index a1fe0e1c51b..35536d76483 100644 --- a/libraries/test_data/src/test/assets/extractordumps/jpeg/pixel-motion-photo-2-hevc-tracks.jpg.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/jpeg/pixel-motion-photo-2-hevc-tracks.jpg.0.dump @@ -270,7 +270,7 @@ track 0: track 1: total output bytes = 151315 sample count = 1 - track duration = 466800 + track duration = 466833 format 0: averageBitrate = 2593047 id = 2 @@ -546,7 +546,7 @@ track 2: track 3: total output bytes = 59 sample count = 1 - track duration = 1133700 + track duration = 1133744 format 0: averageBitrate = 416 id = 4 diff --git a/libraries/test_data/src/test/assets/extractordumps/jpeg/pixel-motion-photo-2-hevc-tracks.jpg.1.dump b/libraries/test_data/src/test/assets/extractordumps/jpeg/pixel-motion-photo-2-hevc-tracks.jpg.1.dump index 7e90a904e2d..6e39ff5aab0 100644 --- a/libraries/test_data/src/test/assets/extractordumps/jpeg/pixel-motion-photo-2-hevc-tracks.jpg.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/jpeg/pixel-motion-photo-2-hevc-tracks.jpg.1.dump @@ -214,7 +214,7 @@ track 0: track 1: total output bytes = 151315 sample count = 1 - track duration = 466800 + track duration = 466833 format 0: averageBitrate = 2593047 id = 2 @@ -410,7 +410,7 @@ track 2: track 3: total output bytes = 59 sample count = 1 - track duration = 1133700 + track duration = 1133744 format 0: averageBitrate = 416 id = 4 diff --git a/libraries/test_data/src/test/assets/extractordumps/jpeg/pixel-motion-photo-2-hevc-tracks.jpg.2.dump b/libraries/test_data/src/test/assets/extractordumps/jpeg/pixel-motion-photo-2-hevc-tracks.jpg.2.dump index 660bb7ddf88..31cf6183418 100644 --- a/libraries/test_data/src/test/assets/extractordumps/jpeg/pixel-motion-photo-2-hevc-tracks.jpg.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/jpeg/pixel-motion-photo-2-hevc-tracks.jpg.2.dump @@ -130,7 +130,7 @@ track 0: track 1: total output bytes = 151315 sample count = 1 - track duration = 466800 + track duration = 466833 format 0: averageBitrate = 2593047 id = 2 @@ -242,7 +242,7 @@ track 2: track 3: total output bytes = 59 sample count = 1 - track duration = 1133700 + track duration = 1133744 format 0: averageBitrate = 416 id = 4 diff --git a/libraries/test_data/src/test/assets/extractordumps/jpeg/pixel-motion-photo-2-hevc-tracks.jpg.3.dump b/libraries/test_data/src/test/assets/extractordumps/jpeg/pixel-motion-photo-2-hevc-tracks.jpg.3.dump index c9980ff112f..b0010bb7f37 100644 --- a/libraries/test_data/src/test/assets/extractordumps/jpeg/pixel-motion-photo-2-hevc-tracks.jpg.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/jpeg/pixel-motion-photo-2-hevc-tracks.jpg.3.dump @@ -46,7 +46,7 @@ track 0: track 1: total output bytes = 151315 sample count = 1 - track duration = 466800 + track duration = 466833 format 0: averageBitrate = 2593047 id = 2 @@ -94,7 +94,7 @@ track 2: track 3: total output bytes = 59 sample count = 1 - track duration = 1133700 + track duration = 1133744 format 0: averageBitrate = 416 id = 4 diff --git a/libraries/test_data/src/test/assets/extractordumps/mka/bear-flac-16bit-nested-chapters.mka.0.dump b/libraries/test_data/src/test/assets/extractordumps/mka/bear-flac-16bit-nested-chapters.mka.0.dump new file mode 100644 index 00000000000..d506860fed6 --- /dev/null +++ b/libraries/test_data/src/test/assets/extractordumps/mka/bear-flac-16bit-nested-chapters.mka.0.dump @@ -0,0 +1,140 @@ +seekMap: + isSeekable = true + duration = 2740000 + getPosition(0) = [[timeUs=1000, position=5961]] + getPosition(1) = [[timeUs=1000, position=5961]] + getPosition(1370000) = [[timeUs=769000, position=5961], [timeUs=1537000, position=5961]] + getPosition(2740000) = [[timeUs=2305000, position=5961]] +numberOfTracks = 1 +track 1: + total output bytes = 110791 + sample count = 29 + format 0: + id = 1 + containerMimeType = video/x-matroska + sampleMimeType = audio/flac + channelCount = 2 + sampleRate = 48000 + pcmEncoding = pcm-16 + selectionFlags = [default] + language = und + metadata = entries=[Chapter: startTimeMs=0, endTimeMs=1000, title={ lang=en, 'Introduction' }, Chapter: startTimeMs=200, endTimeMs=500, title={ lang=en, 'Getting Started' }, Chapter: startTimeMs=1200, endTimeMs=1500, title={ lang=en, 'Advanced Techniques' }, Chapter: startTimeMs=1000, endTimeMs=2000, title={ lang=en, 'Main Content' }, Chapter: startTimeMs=550, endTimeMs=850, title={ lang=en, 'Basic Concepts' }, Chapter: startTimeMs=1600, endTimeMs=1900, title={ lang=en, 'Practical Examples' }, Chapter: startTimeMs=2000, endTimeMs=2500, title={ lang=en, 'Conclusion' }] + initializationData: + data = length 42, hash 46DC6DB0 + sample 0: + time = 1000 + flags = 1 + data = length 3608, hash EE7EFC6C + sample 1: + time = 97000 + flags = 1 + data = length 4278, hash 7BAF8A9 + sample 2: + time = 193000 + flags = 1 + data = length 3763, hash 40C70127 + sample 3: + time = 289000 + flags = 1 + data = length 3568, hash 51FBCC44 + sample 4: + time = 385000 + flags = 1 + data = length 4339, hash FFA8D5A2 + sample 5: + time = 481000 + flags = 1 + data = length 3788, hash 5F5CFFDD + sample 6: + time = 577000 + flags = 1 + data = length 3714, hash DD46DF94 + sample 7: + time = 673000 + flags = 1 + data = length 3766, hash 4D96187C + sample 8: + time = 769000 + flags = 1 + data = length 3806, hash 86DB14D4 + sample 9: + time = 865000 + flags = 1 + data = length 3755, hash AF217312 + sample 10: + time = 961000 + flags = 1 + data = length 3824, hash 37A61E58 + sample 11: + time = 1057000 + flags = 1 + data = length 4785, hash F5200FA7 + sample 12: + time = 1153000 + flags = 1 + data = length 4619, hash 8D64F73D + sample 13: + time = 1249000 + flags = 1 + data = length 3915, hash 73500C2C + sample 14: + time = 1345000 + flags = 1 + data = length 4763, hash 2FCDE683 + sample 15: + time = 1441000 + flags = 1 + data = length 4154, hash A5AB5A5D + sample 16: + time = 1537000 + flags = 1 + data = length 4547, hash BF9A246A + sample 17: + time = 1633000 + flags = 1 + data = length 4005, hash 83EE3482 + sample 18: + time = 1729000 + flags = 1 + data = length 3905, hash E6EE5342 + sample 19: + time = 1825000 + flags = 1 + data = length 3797, hash AE6F03B2 + sample 20: + time = 1921000 + flags = 1 + data = length 3863, hash E022115D + sample 21: + time = 2017000 + flags = 1 + data = length 3958, hash B3F0059 + sample 22: + time = 2113000 + flags = 1 + data = length 3856, hash 81DEB55F + sample 23: + time = 2209000 + flags = 1 + data = length 3958, hash EC7E2889 + sample 24: + time = 2305000 + flags = 1 + data = length 3937, hash 2CC2FD97 + sample 25: + time = 2401000 + flags = 1 + data = length 3934, hash F80FF227 + sample 26: + time = 2497000 + flags = 1 + data = length 3954, hash 17C9E273 + sample 27: + time = 2593000 + flags = 1 + data = length 2299, hash 3F1A98BF + sample 28: + time = 2689000 + flags = 1 + data = length 333, hash FC52D488 +tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mka/bear-flac-16bit-nested-chapters.mka.1.dump b/libraries/test_data/src/test/assets/extractordumps/mka/bear-flac-16bit-nested-chapters.mka.1.dump new file mode 100644 index 00000000000..d506860fed6 --- /dev/null +++ b/libraries/test_data/src/test/assets/extractordumps/mka/bear-flac-16bit-nested-chapters.mka.1.dump @@ -0,0 +1,140 @@ +seekMap: + isSeekable = true + duration = 2740000 + getPosition(0) = [[timeUs=1000, position=5961]] + getPosition(1) = [[timeUs=1000, position=5961]] + getPosition(1370000) = [[timeUs=769000, position=5961], [timeUs=1537000, position=5961]] + getPosition(2740000) = [[timeUs=2305000, position=5961]] +numberOfTracks = 1 +track 1: + total output bytes = 110791 + sample count = 29 + format 0: + id = 1 + containerMimeType = video/x-matroska + sampleMimeType = audio/flac + channelCount = 2 + sampleRate = 48000 + pcmEncoding = pcm-16 + selectionFlags = [default] + language = und + metadata = entries=[Chapter: startTimeMs=0, endTimeMs=1000, title={ lang=en, 'Introduction' }, Chapter: startTimeMs=200, endTimeMs=500, title={ lang=en, 'Getting Started' }, Chapter: startTimeMs=1200, endTimeMs=1500, title={ lang=en, 'Advanced Techniques' }, Chapter: startTimeMs=1000, endTimeMs=2000, title={ lang=en, 'Main Content' }, Chapter: startTimeMs=550, endTimeMs=850, title={ lang=en, 'Basic Concepts' }, Chapter: startTimeMs=1600, endTimeMs=1900, title={ lang=en, 'Practical Examples' }, Chapter: startTimeMs=2000, endTimeMs=2500, title={ lang=en, 'Conclusion' }] + initializationData: + data = length 42, hash 46DC6DB0 + sample 0: + time = 1000 + flags = 1 + data = length 3608, hash EE7EFC6C + sample 1: + time = 97000 + flags = 1 + data = length 4278, hash 7BAF8A9 + sample 2: + time = 193000 + flags = 1 + data = length 3763, hash 40C70127 + sample 3: + time = 289000 + flags = 1 + data = length 3568, hash 51FBCC44 + sample 4: + time = 385000 + flags = 1 + data = length 4339, hash FFA8D5A2 + sample 5: + time = 481000 + flags = 1 + data = length 3788, hash 5F5CFFDD + sample 6: + time = 577000 + flags = 1 + data = length 3714, hash DD46DF94 + sample 7: + time = 673000 + flags = 1 + data = length 3766, hash 4D96187C + sample 8: + time = 769000 + flags = 1 + data = length 3806, hash 86DB14D4 + sample 9: + time = 865000 + flags = 1 + data = length 3755, hash AF217312 + sample 10: + time = 961000 + flags = 1 + data = length 3824, hash 37A61E58 + sample 11: + time = 1057000 + flags = 1 + data = length 4785, hash F5200FA7 + sample 12: + time = 1153000 + flags = 1 + data = length 4619, hash 8D64F73D + sample 13: + time = 1249000 + flags = 1 + data = length 3915, hash 73500C2C + sample 14: + time = 1345000 + flags = 1 + data = length 4763, hash 2FCDE683 + sample 15: + time = 1441000 + flags = 1 + data = length 4154, hash A5AB5A5D + sample 16: + time = 1537000 + flags = 1 + data = length 4547, hash BF9A246A + sample 17: + time = 1633000 + flags = 1 + data = length 4005, hash 83EE3482 + sample 18: + time = 1729000 + flags = 1 + data = length 3905, hash E6EE5342 + sample 19: + time = 1825000 + flags = 1 + data = length 3797, hash AE6F03B2 + sample 20: + time = 1921000 + flags = 1 + data = length 3863, hash E022115D + sample 21: + time = 2017000 + flags = 1 + data = length 3958, hash B3F0059 + sample 22: + time = 2113000 + flags = 1 + data = length 3856, hash 81DEB55F + sample 23: + time = 2209000 + flags = 1 + data = length 3958, hash EC7E2889 + sample 24: + time = 2305000 + flags = 1 + data = length 3937, hash 2CC2FD97 + sample 25: + time = 2401000 + flags = 1 + data = length 3934, hash F80FF227 + sample 26: + time = 2497000 + flags = 1 + data = length 3954, hash 17C9E273 + sample 27: + time = 2593000 + flags = 1 + data = length 2299, hash 3F1A98BF + sample 28: + time = 2689000 + flags = 1 + data = length 333, hash FC52D488 +tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mka/bear-flac-16bit-nested-chapters.mka.2.dump b/libraries/test_data/src/test/assets/extractordumps/mka/bear-flac-16bit-nested-chapters.mka.2.dump new file mode 100644 index 00000000000..d506860fed6 --- /dev/null +++ b/libraries/test_data/src/test/assets/extractordumps/mka/bear-flac-16bit-nested-chapters.mka.2.dump @@ -0,0 +1,140 @@ +seekMap: + isSeekable = true + duration = 2740000 + getPosition(0) = [[timeUs=1000, position=5961]] + getPosition(1) = [[timeUs=1000, position=5961]] + getPosition(1370000) = [[timeUs=769000, position=5961], [timeUs=1537000, position=5961]] + getPosition(2740000) = [[timeUs=2305000, position=5961]] +numberOfTracks = 1 +track 1: + total output bytes = 110791 + sample count = 29 + format 0: + id = 1 + containerMimeType = video/x-matroska + sampleMimeType = audio/flac + channelCount = 2 + sampleRate = 48000 + pcmEncoding = pcm-16 + selectionFlags = [default] + language = und + metadata = entries=[Chapter: startTimeMs=0, endTimeMs=1000, title={ lang=en, 'Introduction' }, Chapter: startTimeMs=200, endTimeMs=500, title={ lang=en, 'Getting Started' }, Chapter: startTimeMs=1200, endTimeMs=1500, title={ lang=en, 'Advanced Techniques' }, Chapter: startTimeMs=1000, endTimeMs=2000, title={ lang=en, 'Main Content' }, Chapter: startTimeMs=550, endTimeMs=850, title={ lang=en, 'Basic Concepts' }, Chapter: startTimeMs=1600, endTimeMs=1900, title={ lang=en, 'Practical Examples' }, Chapter: startTimeMs=2000, endTimeMs=2500, title={ lang=en, 'Conclusion' }] + initializationData: + data = length 42, hash 46DC6DB0 + sample 0: + time = 1000 + flags = 1 + data = length 3608, hash EE7EFC6C + sample 1: + time = 97000 + flags = 1 + data = length 4278, hash 7BAF8A9 + sample 2: + time = 193000 + flags = 1 + data = length 3763, hash 40C70127 + sample 3: + time = 289000 + flags = 1 + data = length 3568, hash 51FBCC44 + sample 4: + time = 385000 + flags = 1 + data = length 4339, hash FFA8D5A2 + sample 5: + time = 481000 + flags = 1 + data = length 3788, hash 5F5CFFDD + sample 6: + time = 577000 + flags = 1 + data = length 3714, hash DD46DF94 + sample 7: + time = 673000 + flags = 1 + data = length 3766, hash 4D96187C + sample 8: + time = 769000 + flags = 1 + data = length 3806, hash 86DB14D4 + sample 9: + time = 865000 + flags = 1 + data = length 3755, hash AF217312 + sample 10: + time = 961000 + flags = 1 + data = length 3824, hash 37A61E58 + sample 11: + time = 1057000 + flags = 1 + data = length 4785, hash F5200FA7 + sample 12: + time = 1153000 + flags = 1 + data = length 4619, hash 8D64F73D + sample 13: + time = 1249000 + flags = 1 + data = length 3915, hash 73500C2C + sample 14: + time = 1345000 + flags = 1 + data = length 4763, hash 2FCDE683 + sample 15: + time = 1441000 + flags = 1 + data = length 4154, hash A5AB5A5D + sample 16: + time = 1537000 + flags = 1 + data = length 4547, hash BF9A246A + sample 17: + time = 1633000 + flags = 1 + data = length 4005, hash 83EE3482 + sample 18: + time = 1729000 + flags = 1 + data = length 3905, hash E6EE5342 + sample 19: + time = 1825000 + flags = 1 + data = length 3797, hash AE6F03B2 + sample 20: + time = 1921000 + flags = 1 + data = length 3863, hash E022115D + sample 21: + time = 2017000 + flags = 1 + data = length 3958, hash B3F0059 + sample 22: + time = 2113000 + flags = 1 + data = length 3856, hash 81DEB55F + sample 23: + time = 2209000 + flags = 1 + data = length 3958, hash EC7E2889 + sample 24: + time = 2305000 + flags = 1 + data = length 3937, hash 2CC2FD97 + sample 25: + time = 2401000 + flags = 1 + data = length 3934, hash F80FF227 + sample 26: + time = 2497000 + flags = 1 + data = length 3954, hash 17C9E273 + sample 27: + time = 2593000 + flags = 1 + data = length 2299, hash 3F1A98BF + sample 28: + time = 2689000 + flags = 1 + data = length 333, hash FC52D488 +tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mka/bear-flac-16bit-nested-chapters.mka.3.dump b/libraries/test_data/src/test/assets/extractordumps/mka/bear-flac-16bit-nested-chapters.mka.3.dump new file mode 100644 index 00000000000..d506860fed6 --- /dev/null +++ b/libraries/test_data/src/test/assets/extractordumps/mka/bear-flac-16bit-nested-chapters.mka.3.dump @@ -0,0 +1,140 @@ +seekMap: + isSeekable = true + duration = 2740000 + getPosition(0) = [[timeUs=1000, position=5961]] + getPosition(1) = [[timeUs=1000, position=5961]] + getPosition(1370000) = [[timeUs=769000, position=5961], [timeUs=1537000, position=5961]] + getPosition(2740000) = [[timeUs=2305000, position=5961]] +numberOfTracks = 1 +track 1: + total output bytes = 110791 + sample count = 29 + format 0: + id = 1 + containerMimeType = video/x-matroska + sampleMimeType = audio/flac + channelCount = 2 + sampleRate = 48000 + pcmEncoding = pcm-16 + selectionFlags = [default] + language = und + metadata = entries=[Chapter: startTimeMs=0, endTimeMs=1000, title={ lang=en, 'Introduction' }, Chapter: startTimeMs=200, endTimeMs=500, title={ lang=en, 'Getting Started' }, Chapter: startTimeMs=1200, endTimeMs=1500, title={ lang=en, 'Advanced Techniques' }, Chapter: startTimeMs=1000, endTimeMs=2000, title={ lang=en, 'Main Content' }, Chapter: startTimeMs=550, endTimeMs=850, title={ lang=en, 'Basic Concepts' }, Chapter: startTimeMs=1600, endTimeMs=1900, title={ lang=en, 'Practical Examples' }, Chapter: startTimeMs=2000, endTimeMs=2500, title={ lang=en, 'Conclusion' }] + initializationData: + data = length 42, hash 46DC6DB0 + sample 0: + time = 1000 + flags = 1 + data = length 3608, hash EE7EFC6C + sample 1: + time = 97000 + flags = 1 + data = length 4278, hash 7BAF8A9 + sample 2: + time = 193000 + flags = 1 + data = length 3763, hash 40C70127 + sample 3: + time = 289000 + flags = 1 + data = length 3568, hash 51FBCC44 + sample 4: + time = 385000 + flags = 1 + data = length 4339, hash FFA8D5A2 + sample 5: + time = 481000 + flags = 1 + data = length 3788, hash 5F5CFFDD + sample 6: + time = 577000 + flags = 1 + data = length 3714, hash DD46DF94 + sample 7: + time = 673000 + flags = 1 + data = length 3766, hash 4D96187C + sample 8: + time = 769000 + flags = 1 + data = length 3806, hash 86DB14D4 + sample 9: + time = 865000 + flags = 1 + data = length 3755, hash AF217312 + sample 10: + time = 961000 + flags = 1 + data = length 3824, hash 37A61E58 + sample 11: + time = 1057000 + flags = 1 + data = length 4785, hash F5200FA7 + sample 12: + time = 1153000 + flags = 1 + data = length 4619, hash 8D64F73D + sample 13: + time = 1249000 + flags = 1 + data = length 3915, hash 73500C2C + sample 14: + time = 1345000 + flags = 1 + data = length 4763, hash 2FCDE683 + sample 15: + time = 1441000 + flags = 1 + data = length 4154, hash A5AB5A5D + sample 16: + time = 1537000 + flags = 1 + data = length 4547, hash BF9A246A + sample 17: + time = 1633000 + flags = 1 + data = length 4005, hash 83EE3482 + sample 18: + time = 1729000 + flags = 1 + data = length 3905, hash E6EE5342 + sample 19: + time = 1825000 + flags = 1 + data = length 3797, hash AE6F03B2 + sample 20: + time = 1921000 + flags = 1 + data = length 3863, hash E022115D + sample 21: + time = 2017000 + flags = 1 + data = length 3958, hash B3F0059 + sample 22: + time = 2113000 + flags = 1 + data = length 3856, hash 81DEB55F + sample 23: + time = 2209000 + flags = 1 + data = length 3958, hash EC7E2889 + sample 24: + time = 2305000 + flags = 1 + data = length 3937, hash 2CC2FD97 + sample 25: + time = 2401000 + flags = 1 + data = length 3934, hash F80FF227 + sample 26: + time = 2497000 + flags = 1 + data = length 3954, hash 17C9E273 + sample 27: + time = 2593000 + flags = 1 + data = length 2299, hash 3F1A98BF + sample 28: + time = 2689000 + flags = 1 + data = length 333, hash FC52D488 +tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mka/bear-flac-16bit-nested-chapters.mka.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mka/bear-flac-16bit-nested-chapters.mka.unknown_length.dump new file mode 100644 index 00000000000..d506860fed6 --- /dev/null +++ b/libraries/test_data/src/test/assets/extractordumps/mka/bear-flac-16bit-nested-chapters.mka.unknown_length.dump @@ -0,0 +1,140 @@ +seekMap: + isSeekable = true + duration = 2740000 + getPosition(0) = [[timeUs=1000, position=5961]] + getPosition(1) = [[timeUs=1000, position=5961]] + getPosition(1370000) = [[timeUs=769000, position=5961], [timeUs=1537000, position=5961]] + getPosition(2740000) = [[timeUs=2305000, position=5961]] +numberOfTracks = 1 +track 1: + total output bytes = 110791 + sample count = 29 + format 0: + id = 1 + containerMimeType = video/x-matroska + sampleMimeType = audio/flac + channelCount = 2 + sampleRate = 48000 + pcmEncoding = pcm-16 + selectionFlags = [default] + language = und + metadata = entries=[Chapter: startTimeMs=0, endTimeMs=1000, title={ lang=en, 'Introduction' }, Chapter: startTimeMs=200, endTimeMs=500, title={ lang=en, 'Getting Started' }, Chapter: startTimeMs=1200, endTimeMs=1500, title={ lang=en, 'Advanced Techniques' }, Chapter: startTimeMs=1000, endTimeMs=2000, title={ lang=en, 'Main Content' }, Chapter: startTimeMs=550, endTimeMs=850, title={ lang=en, 'Basic Concepts' }, Chapter: startTimeMs=1600, endTimeMs=1900, title={ lang=en, 'Practical Examples' }, Chapter: startTimeMs=2000, endTimeMs=2500, title={ lang=en, 'Conclusion' }] + initializationData: + data = length 42, hash 46DC6DB0 + sample 0: + time = 1000 + flags = 1 + data = length 3608, hash EE7EFC6C + sample 1: + time = 97000 + flags = 1 + data = length 4278, hash 7BAF8A9 + sample 2: + time = 193000 + flags = 1 + data = length 3763, hash 40C70127 + sample 3: + time = 289000 + flags = 1 + data = length 3568, hash 51FBCC44 + sample 4: + time = 385000 + flags = 1 + data = length 4339, hash FFA8D5A2 + sample 5: + time = 481000 + flags = 1 + data = length 3788, hash 5F5CFFDD + sample 6: + time = 577000 + flags = 1 + data = length 3714, hash DD46DF94 + sample 7: + time = 673000 + flags = 1 + data = length 3766, hash 4D96187C + sample 8: + time = 769000 + flags = 1 + data = length 3806, hash 86DB14D4 + sample 9: + time = 865000 + flags = 1 + data = length 3755, hash AF217312 + sample 10: + time = 961000 + flags = 1 + data = length 3824, hash 37A61E58 + sample 11: + time = 1057000 + flags = 1 + data = length 4785, hash F5200FA7 + sample 12: + time = 1153000 + flags = 1 + data = length 4619, hash 8D64F73D + sample 13: + time = 1249000 + flags = 1 + data = length 3915, hash 73500C2C + sample 14: + time = 1345000 + flags = 1 + data = length 4763, hash 2FCDE683 + sample 15: + time = 1441000 + flags = 1 + data = length 4154, hash A5AB5A5D + sample 16: + time = 1537000 + flags = 1 + data = length 4547, hash BF9A246A + sample 17: + time = 1633000 + flags = 1 + data = length 4005, hash 83EE3482 + sample 18: + time = 1729000 + flags = 1 + data = length 3905, hash E6EE5342 + sample 19: + time = 1825000 + flags = 1 + data = length 3797, hash AE6F03B2 + sample 20: + time = 1921000 + flags = 1 + data = length 3863, hash E022115D + sample 21: + time = 2017000 + flags = 1 + data = length 3958, hash B3F0059 + sample 22: + time = 2113000 + flags = 1 + data = length 3856, hash 81DEB55F + sample 23: + time = 2209000 + flags = 1 + data = length 3958, hash EC7E2889 + sample 24: + time = 2305000 + flags = 1 + data = length 3937, hash 2CC2FD97 + sample 25: + time = 2401000 + flags = 1 + data = length 3934, hash F80FF227 + sample 26: + time = 2497000 + flags = 1 + data = length 3954, hash 17C9E273 + sample 27: + time = 2593000 + flags = 1 + data = length 2299, hash 3F1A98BF + sample 28: + time = 2689000 + flags = 1 + data = length 333, hash FC52D488 +tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.0.dump b/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.0.dump index a20b1826dbd..7558a7ebf54 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.0.dump @@ -1,7 +1,10 @@ seekMap: - isSeekable = false + isSeekable = true duration = 1072000 - getPosition(0) = [[timeUs=0, position=0]] + getPosition(0) = [[timeUs=67000, position=5562]] + getPosition(1) = [[timeUs=67000, position=5562]] + getPosition(536000) = [[timeUs=67000, position=5562]] + getPosition(1072000) = [[timeUs=67000, position=5562]] numberOfTracks = 2 track 1: total output bytes = 89502 diff --git a/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.1.dump b/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.1.dump index fbda13d6b45..7558a7ebf54 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.1.dump @@ -1,7 +1,10 @@ seekMap: - isSeekable = false + isSeekable = true duration = 1072000 - getPosition(0) = [[timeUs=0, position=0]] + getPosition(0) = [[timeUs=67000, position=5562]] + getPosition(1) = [[timeUs=67000, position=5562]] + getPosition(536000) = [[timeUs=67000, position=5562]] + getPosition(1072000) = [[timeUs=67000, position=5562]] numberOfTracks = 2 track 1: total output bytes = 89502 @@ -18,110 +21,11 @@ track 1: chromaBitdepth = 8 selectionFlags = [default] language = und - label = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee + label = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee labels: lang = und - value = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee + value = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee + metadata = entries=[ThumbnailMetadata: presentationTimeUs=67000] initializationData: data = length 30, hash F6F3D010 data = length 10, hash 7A0D0F2B @@ -150,7 +54,6 @@ eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee flags = 0 data = length 721, hash C50D1C73 sample 6: - time = 23400 time = 234000 flags = 0 data = length 519, hash 65FE1911 @@ -258,110 +161,11 @@ eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee chromaBitdepth = 8 selectionFlags = [default] language = und - label = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee + label = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee labels: lang = und - value = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee + value = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee + metadata = entries=[ThumbnailMetadata: presentationTimeUs=67000] initializationData: data = length 30, hash F6F3D010 data = length 10, hash 7A0D0F2B @@ -422,7 +226,6 @@ track 2: data = length 418, hash D69A9251 sample 11: time = 512489 - fl flags = 1 data = length 418, hash BCFB758D sample 12: diff --git a/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.2.dump b/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.2.dump index fbda13d6b45..7558a7ebf54 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.2.dump @@ -1,7 +1,10 @@ seekMap: - isSeekable = false + isSeekable = true duration = 1072000 - getPosition(0) = [[timeUs=0, position=0]] + getPosition(0) = [[timeUs=67000, position=5562]] + getPosition(1) = [[timeUs=67000, position=5562]] + getPosition(536000) = [[timeUs=67000, position=5562]] + getPosition(1072000) = [[timeUs=67000, position=5562]] numberOfTracks = 2 track 1: total output bytes = 89502 @@ -18,110 +21,11 @@ track 1: chromaBitdepth = 8 selectionFlags = [default] language = und - label = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee + label = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee labels: lang = und - value = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee + value = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee + metadata = entries=[ThumbnailMetadata: presentationTimeUs=67000] initializationData: data = length 30, hash F6F3D010 data = length 10, hash 7A0D0F2B @@ -150,7 +54,6 @@ eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee flags = 0 data = length 721, hash C50D1C73 sample 6: - time = 23400 time = 234000 flags = 0 data = length 519, hash 65FE1911 @@ -258,110 +161,11 @@ eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee chromaBitdepth = 8 selectionFlags = [default] language = und - label = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee + label = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee labels: lang = und - value = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee + value = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee + metadata = entries=[ThumbnailMetadata: presentationTimeUs=67000] initializationData: data = length 30, hash F6F3D010 data = length 10, hash 7A0D0F2B @@ -422,7 +226,6 @@ track 2: data = length 418, hash D69A9251 sample 11: time = 512489 - fl flags = 1 data = length 418, hash BCFB758D sample 12: diff --git a/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.3.dump b/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.3.dump index fbda13d6b45..7558a7ebf54 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.3.dump @@ -1,7 +1,10 @@ seekMap: - isSeekable = false + isSeekable = true duration = 1072000 - getPosition(0) = [[timeUs=0, position=0]] + getPosition(0) = [[timeUs=67000, position=5562]] + getPosition(1) = [[timeUs=67000, position=5562]] + getPosition(536000) = [[timeUs=67000, position=5562]] + getPosition(1072000) = [[timeUs=67000, position=5562]] numberOfTracks = 2 track 1: total output bytes = 89502 @@ -18,110 +21,11 @@ track 1: chromaBitdepth = 8 selectionFlags = [default] language = und - label = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee + label = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee labels: lang = und - value = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee + value = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee + metadata = entries=[ThumbnailMetadata: presentationTimeUs=67000] initializationData: data = length 30, hash F6F3D010 data = length 10, hash 7A0D0F2B @@ -150,7 +54,6 @@ eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee flags = 0 data = length 721, hash C50D1C73 sample 6: - time = 23400 time = 234000 flags = 0 data = length 519, hash 65FE1911 @@ -258,110 +161,11 @@ eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee chromaBitdepth = 8 selectionFlags = [default] language = und - label = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee + label = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee labels: lang = und - value = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee -eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee + value = eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee + metadata = entries=[ThumbnailMetadata: presentationTimeUs=67000] initializationData: data = length 30, hash F6F3D010 data = length 10, hash 7A0D0F2B @@ -422,7 +226,6 @@ track 2: data = length 418, hash D69A9251 sample 11: time = 512489 - fl flags = 1 data = length 418, hash BCFB758D sample 12: diff --git a/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.unknown_length.dump index a20b1826dbd..7558a7ebf54 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mkv/sample_last_cluster.mkv.unknown_length.dump @@ -1,7 +1,10 @@ seekMap: - isSeekable = false + isSeekable = true duration = 1072000 - getPosition(0) = [[timeUs=0, position=0]] + getPosition(0) = [[timeUs=67000, position=5562]] + getPosition(1) = [[timeUs=67000, position=5562]] + getPosition(536000) = [[timeUs=67000, position=5562]] + getPosition(1072000) = [[timeUs=67000, position=5562]] numberOfTracks = 2 track 1: total output bytes = 89502 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.0.dump index e5032488d71..bf3a6e0e45f 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.0.dump @@ -16,8 +16,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TIT2: description=null: values=[Test title], TPE1: description=null: values=[Test Artist], TALB: description=null: values=[Test Album], TXXX: description=Test description: values=[Test user info], COMM: language=eng, description=Test description, text=Test comment, WXXX: url=Test URL, TSSE: description=null: values=[Lavf58.29.100], MLLT, PRIV: owner=test@gmail.com, UNKN, GEOB: mimeType=test/mime, filename=Testfilename.txt, description=Test description, CHAP, CHAP, CTOC, APIC: mimeType=image/jpeg, description=Test description, TCON: description=null: values=[9]] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.1.dump index f9246a4f175..eafac7a6cca 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.1.dump @@ -16,8 +16,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TIT2: description=null: values=[Test title], TPE1: description=null: values=[Test Artist], TALB: description=null: values=[Test Album], TXXX: description=Test description: values=[Test user info], COMM: language=eng, description=Test description, text=Test comment, WXXX: url=Test URL, TSSE: description=null: values=[Lavf58.29.100], MLLT, PRIV: owner=test@gmail.com, UNKN, GEOB: mimeType=test/mime, filename=Testfilename.txt, description=Test description, CHAP, CHAP, CTOC, APIC: mimeType=image/jpeg, description=Test description, TCON: description=null: values=[9]] sample 0: time = 943000 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.2.dump index cad819be786..3f0892a6b4b 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.2.dump @@ -16,8 +16,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TIT2: description=null: values=[Test title], TPE1: description=null: values=[Test Artist], TALB: description=null: values=[Test Album], TXXX: description=Test description: values=[Test user info], COMM: language=eng, description=Test description, text=Test comment, WXXX: url=Test URL, TSSE: description=null: values=[Lavf58.29.100], MLLT, PRIV: owner=test@gmail.com, UNKN, GEOB: mimeType=test/mime, filename=Testfilename.txt, description=Test description, CHAP, CHAP, CTOC, APIC: mimeType=image/jpeg, description=Test description, TCON: description=null: values=[9]] sample 0: time = 1879000 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.3.dump index 1cdc8931fdd..fe26e0b0756 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.3.dump @@ -16,7 +16,7 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TIT2: description=null: values=[Test title], TPE1: description=null: values=[Test Artist], TALB: description=null: values=[Test Album], TXXX: description=Test description: values=[Test user info], COMM: language=eng, description=Test description, text=Test comment, WXXX: url=Test URL, TSSE: description=null: values=[Lavf58.29.100], MLLT, PRIV: owner=test@gmail.com, UNKN, GEOB: mimeType=test/mime, filename=Testfilename.txt, description=Test description, CHAP, CHAP, CTOC, APIC: mimeType=image/jpeg, description=Test description, TCON: description=null: values=[9]] tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.unknown_length.dump index e5032488d71..bf3a6e0e45f 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3-numeric-genre.mp3.unknown_length.dump @@ -16,8 +16,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TIT2: description=null: values=[Test title], TPE1: description=null: values=[Test Artist], TALB: description=null: values=[Test Album], TXXX: description=Test description: values=[Test user info], COMM: language=eng, description=Test description, text=Test comment, WXXX: url=Test URL, TSSE: description=null: values=[Lavf58.29.100], MLLT, PRIV: owner=test@gmail.com, UNKN, GEOB: mimeType=test/mime, filename=Testfilename.txt, description=Test description, CHAP, CHAP, CTOC, APIC: mimeType=image/jpeg, description=Test description, TCON: description=null: values=[9]] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.0.dump index 8a177206bc4..8b84b79047d 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.0.dump @@ -16,8 +16,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 sample 0: time = 0 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.1.dump index d8463020d34..2a75c2bb3f7 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.1.dump @@ -16,8 +16,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 sample 0: time = 943000 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.2.dump index c2db2ba9949..8cf7cddb906 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.2.dump @@ -16,8 +16,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 sample 0: time = 1879000 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.3.dump index 5335ff543de..8f32b9426a9 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.3.dump @@ -16,6 +16,6 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.unknown_length.dump index 8a177206bc4..8b84b79047d 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-disabled.unknown_length.dump @@ -16,8 +16,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 sample 0: time = 0 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.0.dump index 8977fe00bdb..069b98e317f 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.0.dump @@ -16,8 +16,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TIT2: description=null: values=[Test title], TPE1: description=null: values=[Test Artist], TALB: description=null: values=[Test Album], TXXX: description=Test description: values=[Test user info], COMM: language=eng, description=Test description, text=Test comment, WXXX: url=Test URL, TSSE: description=null: values=[Lavf58.29.100], MLLT, PRIV: owner=test@gmail.com, UNKN, GEOB: mimeType=test/mime, filename=Testfilename.txt, description=Test description, CHAP, CHAP, CTOC, APIC: mimeType=image/jpeg, description=Test description, TCON: description=null: values=[Gorpcore]] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.1.dump index 69bff6aa324..5059438ec55 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.1.dump @@ -16,8 +16,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TIT2: description=null: values=[Test title], TPE1: description=null: values=[Test Artist], TALB: description=null: values=[Test Album], TXXX: description=Test description: values=[Test user info], COMM: language=eng, description=Test description, text=Test comment, WXXX: url=Test URL, TSSE: description=null: values=[Lavf58.29.100], MLLT, PRIV: owner=test@gmail.com, UNKN, GEOB: mimeType=test/mime, filename=Testfilename.txt, description=Test description, CHAP, CHAP, CTOC, APIC: mimeType=image/jpeg, description=Test description, TCON: description=null: values=[Gorpcore]] sample 0: time = 943000 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.2.dump index 6d0259bca77..16419ce6395 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.2.dump @@ -16,8 +16,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TIT2: description=null: values=[Test title], TPE1: description=null: values=[Test Artist], TALB: description=null: values=[Test Album], TXXX: description=Test description: values=[Test user info], COMM: language=eng, description=Test description, text=Test comment, WXXX: url=Test URL, TSSE: description=null: values=[Lavf58.29.100], MLLT, PRIV: owner=test@gmail.com, UNKN, GEOB: mimeType=test/mime, filename=Testfilename.txt, description=Test description, CHAP, CHAP, CTOC, APIC: mimeType=image/jpeg, description=Test description, TCON: description=null: values=[Gorpcore]] sample 0: time = 1879000 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.3.dump index f78c455ac83..a0273aac08a 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.3.dump @@ -16,7 +16,7 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TIT2: description=null: values=[Test title], TPE1: description=null: values=[Test Artist], TALB: description=null: values=[Test Album], TXXX: description=Test description: values=[Test user info], COMM: language=eng, description=Test description, text=Test comment, WXXX: url=Test URL, TSSE: description=null: values=[Lavf58.29.100], MLLT, PRIV: owner=test@gmail.com, UNKN, GEOB: mimeType=test/mime, filename=Testfilename.txt, description=Test description, CHAP, CHAP, CTOC, APIC: mimeType=image/jpeg, description=Test description, TCON: description=null: values=[Gorpcore]] tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.unknown_length.dump index 8977fe00bdb..069b98e317f 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-id3.mp3.id3-enabled.unknown_length.dump @@ -16,8 +16,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TIT2: description=null: values=[Test title], TPE1: description=null: values=[Test Artist], TALB: description=null: values=[Test Album], TXXX: description=Test description: values=[Test user info], COMM: language=eng, description=Test description, text=Test comment, WXXX: url=Test URL, TSSE: description=null: values=[Lavf58.29.100], MLLT, PRIV: owner=test@gmail.com, UNKN, GEOB: mimeType=test/mime, filename=Testfilename.txt, description=Test description, CHAP, CHAP, CTOC, APIC: mimeType=image/jpeg, description=Test description, TCON: description=null: values=[Gorpcore]] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.0.dump index d3b22bcee8b..8cd6fb558dd 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.0.dump @@ -14,8 +14,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.0.dump index d9eaa80c992..64f8f9ed283 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.0.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.1.dump index 9723530e25c..80a3ec9a36a 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.1.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 934996 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.2.dump index 947c5285561..dedf6814967 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.2.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 1862989 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.3.dump index 61d8ee3b527..b04efec744f 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.3.dump @@ -17,7 +17,7 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.unknown_length.dump index d9eaa80c992..64f8f9ed283 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking-always.unknown_length.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.0.dump index d9eaa80c992..64f8f9ed283 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.0.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.1.dump index 9723530e25c..80a3ec9a36a 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.1.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 934996 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.2.dump index 947c5285561..dedf6814967 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.2.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 1862989 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.3.dump index 61d8ee3b527..b04efec744f 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.3.dump @@ -17,7 +17,7 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.unknown_length.dump index d9eaa80c992..64f8f9ed283 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.cbr-seeking.unknown_length.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.0.dump index ea1f0a28a0f..be1d0b8adfa 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.0.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.1.dump index 4fe163ebb97..e75965fd12d 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.1.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 840000 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.2.dump index 9b2c3379720..58e2d8bc1c3 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.2.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 1800000 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.3.dump index 5cd9e8bd34a..5d9a95e4f10 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.3.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 2760000 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.unknown_length.dump index ea1f0a28a0f..be1d0b8adfa 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.index-seeking.unknown_length.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.unknown_length.dump index d3b22bcee8b..8cd6fb558dd 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-no-toc.mp3.unknown_length.dump @@ -14,8 +14,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.0.dump index 3e7f7191e53..bed9d0b0a06 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.0.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100], ReplayGain Xing/Info: peak=8.14048E-40, field 1=GainField{name=1, originator=3, gain=23.3}, field 2=null] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.1.dump index c8c00e55a9e..0342663e4b5 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.1.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100], ReplayGain Xing/Info: peak=8.14048E-40, field 1=GainField{name=1, originator=3, gain=23.3}, field 2=null] sample 0: time = 904201 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.2.dump index 7d9902edcae..af626679dbe 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.2.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100], ReplayGain Xing/Info: peak=8.14048E-40, field 1=GainField{name=1, originator=3, gain=23.3}, field 2=null] sample 0: time = 1832197 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.3.dump index 350beed88ab..72df85048d8 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.3.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100], ReplayGain Xing/Info: peak=8.14048E-40, field 1=GainField{name=1, originator=3, gain=23.3}, field 2=null] sample 0: time = 2760193 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.unknown_length.dump index 3e7f7191e53..bed9d0b0a06 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-accurate.mp3.unknown_length.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100], ReplayGain Xing/Info: peak=8.14048E-40, field 1=GainField{name=1, originator=3, gain=23.3}, field 2=null] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.0.dump index a6e94cee698..634863540ab 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.0.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100], ReplayGain Xing/Info: peak=0.0, field 1=GainField{name=1, originator=3, gain=23.0}, field 2=null] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.1.dump index 6a9fdd7ae91..754806a1c88 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.1.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100], ReplayGain Xing/Info: peak=0.0, field 1=GainField{name=1, originator=3, gain=23.0}, field 2=null] sample 0: time = 904201 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.2.dump index 22c27662b17..7865ce431f4 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.2.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100], ReplayGain Xing/Info: peak=0.0, field 1=GainField{name=1, originator=3, gain=23.0}, field 2=null] sample 0: time = 1832197 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.3.dump index 6fb8f06d006..1d412a06dd0 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.3.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100], ReplayGain Xing/Info: peak=0.0, field 1=GainField{name=1, originator=3, gain=23.0}, field 2=null] sample 0: time = 2760193 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.unknown_length.dump index a6e94cee698..634863540ab 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header-replaygain-fast.mp3.unknown_length.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100], ReplayGain Xing/Info: peak=0.0, field 1=GainField{name=1, originator=3, gain=23.0}, field 2=null] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.0.dump index 0243117e737..9b127a3ef9d 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.0.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.1.dump index 09a86aa2119..2b8c349f6d7 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.1.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 949845 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.2.dump index cdee0946c7a..b5a7768aa5c 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.2.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 1870631 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.3.dump index bd080496d90..b68b9816f5c 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.3.dump @@ -17,7 +17,7 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.unknown_length.dump index 0243117e737..9b127a3ef9d 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/bear-vbr-xing-header.mp3.unknown_length.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 2 sampleRate = 48000 - encoderDelay = 1105 - encoderPadding = 47 + encoderDelay = 576 + encoderPadding = 576 metadata = entries=[TSSE: description=null: values=[Lavf58.29.100]] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.0.dump index f82c0236cc7..dac98674a35 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.0.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 1 sampleRate = 44100 - encoderDelay = 1105 - encoderPadding = 875 + encoderDelay = 576 + encoderPadding = 1404 metadata = entries=[TSSE: description=null: values=[Lavf58.45.100]] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.1.dump index b68e5ffd095..fb72775fde8 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.1.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 1 sampleRate = 44100 - encoderDelay = 1105 - encoderPadding = 875 + encoderDelay = 576 + encoderPadding = 1404 metadata = entries=[TSSE: description=null: values=[Lavf58.45.100]] sample 0: time = 341911 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.2.dump index 352e3ae2816..55b1a2e30fd 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.2.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 1 sampleRate = 44100 - encoderDelay = 1105 - encoderPadding = 875 + encoderDelay = 576 + encoderPadding = 1404 metadata = entries=[TSSE: description=null: values=[Lavf58.45.100]] sample 0: time = 696674 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.3.dump index bae80251fe4..9b173c46721 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.3.dump @@ -17,7 +17,7 @@ track 0: maxInputSize = 4096 channelCount = 1 sampleRate = 44100 - encoderDelay = 1105 - encoderPadding = 875 + encoderDelay = 576 + encoderPadding = 1404 metadata = entries=[TSSE: description=null: values=[Lavf58.45.100]] tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.unknown_length.dump index f82c0236cc7..dac98674a35 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header-pcut-frame.mp3.unknown_length.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 1 sampleRate = 44100 - encoderDelay = 1105 - encoderPadding = 875 + encoderDelay = 576 + encoderPadding = 1404 metadata = entries=[TSSE: description=null: values=[Lavf58.45.100]] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.0.dump index e8cfdb42f15..1d172c38918 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.0.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 1 sampleRate = 44100 - encoderDelay = 1105 - encoderPadding = 875 + encoderDelay = 576 + encoderPadding = 1404 metadata = entries=[TSSE: description=null: values=[Lavf58.45.100]] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.1.dump index 664ff36a1c6..d1d83bd2077 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.1.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 1 sampleRate = 44100 - encoderDelay = 1105 - encoderPadding = 875 + encoderDelay = 576 + encoderPadding = 1404 metadata = entries=[TSSE: description=null: values=[Lavf58.45.100]] sample 0: time = 349911 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.2.dump index 923bdc7a7fb..070022b73ff 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.2.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 1 sampleRate = 44100 - encoderDelay = 1105 - encoderPadding = 875 + encoderDelay = 576 + encoderPadding = 1404 metadata = entries=[TSSE: description=null: values=[Lavf58.45.100]] sample 0: time = 674940 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.3.dump index 32d80bb26ef..1dbaf26572d 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.3.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 1 sampleRate = 44100 - encoderDelay = 1105 - encoderPadding = 875 + encoderDelay = 576 + encoderPadding = 1404 metadata = entries=[TSSE: description=null: values=[Lavf58.45.100]] sample 0: time = 974967 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.unknown_length.dump index e8cfdb42f15..1d172c38918 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp3/test-cbr-info-header.mp3.unknown_length.dump @@ -17,8 +17,8 @@ track 0: maxInputSize = 4096 channelCount = 1 sampleRate = 44100 - encoderDelay = 1105 - encoderPadding = 875 + encoderDelay = 576 + encoderPadding = 1404 metadata = entries=[TSSE: description=null: values=[Lavf58.45.100]] sample 0: time = 0 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/fragmented_captions_h265.mp4_without_gop_parsing_flags b/libraries/test_data/src/test/assets/extractordumps/mp4/fragmented_captions_h265.mp4_without_gop_parsing_flags deleted file mode 100644 index 655cfb32d3c..00000000000 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/fragmented_captions_h265.mp4_without_gop_parsing_flags +++ /dev/null @@ -1,288 +0,0 @@ -seekMap: - isSeekable = true - duration = UNSET TIME - getPosition(0) = [[timeUs=0, position=3203]] - getPosition(1) = [[timeUs=0, position=3203]] -numberOfTracks = 2 -track 0: - total output bytes = 24020 - sample count = 61 - format 0: - id = 1 - containerMimeType = video/mp4 - sampleMimeType = video/hevc - codecs = hvc1.1.6.L60.90 - maxNumReorderSamples = 2 - width = 416 - height = 234 - decodedWidth = 416 - decodedHeight = 240 - maxSubLayers = 1 - colorInfo: - colorSpace = 2 - colorRange = 2 - colorTransfer = 3 - lumaBitdepth = 8 - chromaBitdepth = 8 - language = und - metadata = entries=[TSSE: description=null: values=[Lavf60.16.100], Mp4Timestamp: creation time=0, modification time=0, timescale=1000] - initializationData: - data = length 2426, hash 25737613 - sample 0: - time = 66733 - flags = 1 - data = length 5220, hash 5F0CCC08 - sample 1: - time = 233566 - flags = 0 - data = length 615, hash CBFF4D3C - sample 2: - time = 166833 - flags = 0 - data = length 217, hash B600557D - sample 3: - time = 100100 - flags = 0 - data = length 177, hash 99B5D77F - sample 4: - time = 133466 - flags = 0 - data = length 190, hash AF7F85A2 - sample 5: - time = 200200 - flags = 0 - data = length 225, hash BA8F2E19 - sample 6: - time = 333666 - flags = 0 - data = length 330, hash DCBABAE8 - sample 7: - time = 300300 - flags = 0 - data = length 183, hash 15A2DB25 - sample 8: - time = 266933 - flags = 0 - data = length 154, hash 81018D0E - sample 9: - time = 400400 - flags = 0 - data = length 576, hash C9C5389F - sample 10: - time = 367033 - flags = 0 - data = length 173, hash 2D567C7E - sample 11: - time = 500500 - flags = 0 - data = length 356, hash A6D37B60 - sample 12: - time = 467133 - flags = 0 - data = length 188, hash 11AB1597 - sample 13: - time = 433766 - flags = 0 - data = length 174, hash 18C46868 - sample 14: - time = 567233 - flags = 0 - data = length 331, hash EEA184AF - sample 15: - time = 533866 - flags = 0 - data = length 213, hash 3FAB17EE - sample 16: - time = 667333 - flags = 0 - data = length 371, hash D0ECAE97 - sample 17: - time = 633966 - flags = 0 - data = length 179, hash 796F24AF - sample 18: - time = 600600 - flags = 0 - data = length 154, hash FD43F0C6 - sample 19: - time = 734066 - flags = 0 - data = length 622, hash 8AD9409C - sample 20: - time = 700700 - flags = 0 - data = length 195, hash 4EFA8FFD - sample 21: - time = 900900 - flags = 0 - data = length 424, hash 5F9D809C - sample 22: - time = 834166 - flags = 0 - data = length 187, hash FB5137FD - sample 23: - time = 767433 - flags = 0 - data = length 162, hash 7B352EA - sample 24: - time = 800800 - flags = 0 - data = length 173, hash 42EBA5AC - sample 25: - time = 867533 - flags = 0 - data = length 205, hash 15D2186B - sample 26: - time = 1001000 - flags = 0 - data = length 342, hash 70E9AD53 - sample 27: - time = 967633 - flags = 0 - data = length 200, hash 2C923950 - sample 28: - time = 934266 - flags = 0 - data = length 150, hash 242935D6 - sample 29: - time = 1167833 - flags = 0 - data = length 802, hash 60213433 - sample 30: - time = 1101100 - flags = 0 - data = length 203, hash 21B384E5 - sample 31: - time = 1034366 - flags = 0 - data = length 185, hash B2735872 - sample 32: - time = 1067733 - flags = 0 - data = length 1310, hash 8D4F7E35 - sample 33: - time = 1134466 - flags = 0 - data = length 176, hash D1D91C73 - sample 34: - time = 1334666 - flags = 0 - data = length 258, hash 92056B3A - sample 35: - time = 1267933 - flags = 0 - data = length 214, hash 86C47DEC - sample 36: - time = 1201200 - flags = 0 - data = length 201, hash AFA1B6D4 - sample 37: - time = 1234566 - flags = 0 - data = length 146, hash 3E1B1C72 - sample 38: - time = 1301300 - flags = 0 - data = length 173, hash FC0B911D - sample 39: - time = 1401400 - flags = 0 - data = length 617, hash 35AAFB9 - sample 40: - time = 1368033 - flags = 0 - data = length 179, hash E9838582 - sample 41: - time = 1468133 - flags = 0 - data = length 395, hash 8F5373AC - sample 42: - time = 1434766 - flags = 0 - data = length 184, hash 10ED9B63 - sample 43: - time = 1634966 - flags = 0 - data = length 460, hash ED8ECDE7 - sample 44: - time = 1568233 - flags = 0 - data = length 184, hash BF273413 - sample 45: - time = 1501500 - flags = 0 - data = length 162, hash FE32A96B - sample 46: - time = 1534866 - flags = 0 - data = length 201, hash BBB627F2 - sample 47: - time = 1601600 - flags = 0 - data = length 169, hash AFD9804F - sample 48: - time = 1735066 - flags = 0 - data = length 668, hash 4B9FB50F - sample 49: - time = 1701700 - flags = 0 - data = length 219, hash E2A651A2 - sample 50: - time = 1668333 - flags = 0 - data = length 162, hash ECB52FD2 - sample 51: - time = 1901900 - flags = 0 - data = length 405, hash 7DA640B4 - sample 52: - time = 1835166 - flags = 0 - data = length 188, hash 46D5E9D4 - sample 53: - time = 1768433 - flags = 0 - data = length 167, hash 65F87E6A - sample 54: - time = 1801800 - flags = 0 - data = length 161, hash 2E37B5E0 - sample 55: - time = 1868533 - flags = 0 - data = length 199, hash D63CF14E - sample 56: - time = 2002000 - flags = 0 - data = length 282, hash E748555D - sample 57: - time = 1968633 - flags = 0 - data = length 199, hash F341EE39 - sample 58: - time = 1935266 - flags = 0 - data = length 166, hash 32F07BFF - sample 59: - time = 2068733 - flags = 0 - data = length 2246, hash E07DFCAE - sample 60: - time = 2035366 - flags = 0 - data = length 53, hash D9F70BD5 -track 100: - total output bytes = 45 - sample count = 2 - format 0: - sampleMimeType = application/cea-608 - sample 0: - time = 66733 - flags = 1 - data = length 36, hash 6E15DFAD - sample 1: - time = 1935266 - flags = 1 - data = length 9, hash 604EC6AA -tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.0.dump index 24fd90867b5..673cd29973b 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.0.dump @@ -485,7 +485,7 @@ track 0: track 1: total output bytes = 118975 sample count = 174 - track duration = 3711900 + track duration = 3711937 format 0: averageBitrate = 256416 peakBitrate = 256000 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.1.dump index 66aa28a0d10..2e525326ba9 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.1.dump @@ -353,7 +353,7 @@ track 0: track 1: total output bytes = 79194 sample count = 116 - track duration = 3711900 + track duration = 3711937 format 0: averageBitrate = 256416 peakBitrate = 256000 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.2.dump index d981f313e70..c504047c596 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.2.dump @@ -225,7 +225,7 @@ track 0: track 1: total output bytes = 39597 sample count = 58 - track duration = 3711900 + track duration = 3711937 format 0: averageBitrate = 256416 peakBitrate = 256000 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.3.dump index a3be4901b34..b66be82edab 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.3.dump @@ -97,7 +97,7 @@ track 0: track 1: total output bytes = 620 sample count = 1 - track duration = 3711900 + track duration = 3711937 format 0: averageBitrate = 256416 peakBitrate = 256000 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.unknown_length.dump index 24fd90867b5..673cd29973b 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/h265_4k_bframes_emulation_prevention.mp4.unknown_length.dump @@ -485,7 +485,7 @@ track 0: track 1: total output bytes = 118975 sample count = 174 - track duration = 3711900 + track duration = 3711937 format 0: averageBitrate = 256416 peakBitrate = 256000 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/h265_bframes.mp4_without_gop_parsing_flags b/libraries/test_data/src/test/assets/extractordumps/mp4/h265_bframes.mp4_without_gop_parsing_flags deleted file mode 100644 index 0c9b504ccf3..00000000000 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/h265_bframes.mp4_without_gop_parsing_flags +++ /dev/null @@ -1,156 +0,0 @@ -seekMap: - isSeekable = true - duration = 1000000 - getPosition(0) = [[timeUs=0, position=44]] - getPosition(1) = [[timeUs=0, position=44]] - getPosition(500000) = [[timeUs=0, position=44]] - getPosition(1000000) = [[timeUs=0, position=44]] -numberOfTracks = 1 -track 0: - total output bytes = 19395 - sample count = 30 - track duration = 1000000 - format 0: - averageBitrate = 155160 - peakBitrate = 155160 - id = 1 - containerMimeType = video/mp4 - sampleMimeType = video/hevc - codecs = hvc1.1.6.L90.90 - maxInputSize = 7848 - maxNumReorderSamples = 2 - width = 854 - height = 480 - decodedWidth = 856 - decodedHeight = 480 - frameRate = 30.00 - maxSubLayers = 1 - colorInfo: - colorRange = 2 - lumaBitdepth = 8 - chromaBitdepth = 8 - language = und - metadata = entries=[TSSE: description=null: values=[Lavf60.16.100], Mp4Timestamp: creation time=0, modification time=0, timescale=1000, ThumbnailMetadata: presentationTimeUs=0] - initializationData: - data = length 84, hash 6FF5034A - sample 0: - time = 0 - flags = 1 - data = length 2520, hash 9DF10E5A - sample 1: - time = 33333 - flags = 0 - data = length 1227, hash C3B78778 - sample 2: - time = 200000 - flags = 0 - data = length 7818, hash 3118F8A5 - sample 3: - time = 133333 - flags = 0 - data = length 2314, hash F06B088 - sample 4: - time = 66666 - flags = 0 - data = length 1066, hash 2F1BE2DB - sample 5: - time = 100000 - flags = 0 - data = length 106, hash CF881CF - sample 6: - time = 166666 - flags = 0 - data = length 69, hash 43B2E357 - sample 7: - time = 366666 - flags = 0 - data = length 304, hash 19293C74 - sample 8: - time = 300000 - flags = 0 - data = length 145, hash C3F16A - sample 9: - time = 233333 - flags = 0 - data = length 226, hash FDA9A38B - sample 10: - time = 266666 - flags = 0 - data = length 185, hash 9C800E54 - sample 11: - time = 333333 - flags = 0 - data = length 90, hash D1EA90C4 - sample 12: - time = 466666 - flags = 0 - data = length 298, hash F203E87C - sample 13: - time = 433333 - flags = 0 - data = length 276, hash 66430FFD - sample 14: - time = 400000 - flags = 0 - data = length 186, hash B985342A - sample 15: - time = 500000 - flags = 0 - data = length 279, hash 1EFA3459 - sample 16: - time = 666666 - flags = 0 - data = length 265, hash 7FCB3588 - sample 17: - time = 600000 - flags = 0 - data = length 214, hash EFA105EC - sample 18: - time = 533333 - flags = 0 - data = length 138, hash 5F4FDE79 - sample 19: - time = 566666 - flags = 0 - data = length 122, hash 951D945A - sample 20: - time = 633333 - flags = 0 - data = length 103, hash 50A4D1ED - sample 21: - time = 833333 - flags = 0 - data = length 241, hash 290CE032 - sample 22: - time = 766666 - flags = 0 - data = length 211, hash D120B486 - sample 23: - time = 700000 - flags = 0 - data = length 103, hash D33CDF67 - sample 24: - time = 733333 - flags = 0 - data = length 111, hash E8A78D5 - sample 25: - time = 800000 - flags = 0 - data = length 119, hash 3FA7645D - sample 26: - time = 966666 - flags = 0 - data = length 261, hash 53B1BD3B - sample 27: - time = 900000 - flags = 0 - data = length 142, hash B45973DC - sample 28: - time = 866666 - flags = 0 - data = length 88, hash 757187AE - sample 29: - time = 933333 - flags = 536870912 - data = length 168, hash 8220F9BC -tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.0.dump index 60815c9c302..b660a966c26 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.0.dump @@ -269,7 +269,7 @@ track 0: track 1: total output bytes = 151315 sample count = 1 - track duration = 466800 + track duration = 466833 format 0: averageBitrate = 2593047 id = 2 @@ -544,7 +544,7 @@ track 2: track 3: total output bytes = 59 sample count = 1 - track duration = 1133700 + track duration = 1133744 format 0: averageBitrate = 416 id = 4 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.1.dump index 07f435217f3..43038afe2b7 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.1.dump @@ -213,7 +213,7 @@ track 0: track 1: total output bytes = 151315 sample count = 1 - track duration = 466800 + track duration = 466833 format 0: averageBitrate = 2593047 id = 2 @@ -408,7 +408,7 @@ track 2: track 3: total output bytes = 59 sample count = 1 - track duration = 1133700 + track duration = 1133744 format 0: averageBitrate = 416 id = 4 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.2.dump index 9719dc739b9..75c66cee1b6 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.2.dump @@ -129,7 +129,7 @@ track 0: track 1: total output bytes = 151315 sample count = 1 - track duration = 466800 + track duration = 466833 format 0: averageBitrate = 2593047 id = 2 @@ -240,7 +240,7 @@ track 2: track 3: total output bytes = 59 sample count = 1 - track duration = 1133700 + track duration = 1133744 format 0: averageBitrate = 416 id = 4 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.3.dump index 96aa0b62ed9..055991ec7cf 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.3.dump @@ -45,7 +45,7 @@ track 0: track 1: total output bytes = 151315 sample count = 1 - track duration = 466800 + track duration = 466833 format 0: averageBitrate = 2593047 id = 2 @@ -92,7 +92,7 @@ track 2: track 3: total output bytes = 59 sample count = 1 - track duration = 1133700 + track duration = 1133744 format 0: averageBitrate = 416 id = 4 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.unknown_length.dump index 60815c9c302..b660a966c26 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/pixel-motion-photo-2-hevc-tracks.mp4.unknown_length.dump @@ -269,7 +269,7 @@ track 0: track 1: total output bytes = 151315 sample count = 1 - track duration = 466800 + track duration = 466833 format 0: averageBitrate = 2593047 id = 2 @@ -544,7 +544,7 @@ track 2: track 3: total output bytes = 59 sample count = 1 - track duration = 1133700 + track duration = 1133744 format 0: averageBitrate = 416 id = 4 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample.mp4_with_flag_read_auxiliary_tracks b/libraries/test_data/src/test/assets/extractordumps/mp4/sample.mp4_with_flag_read_auxiliary_tracks deleted file mode 100644 index 6d7ee3fb8bf..00000000000 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample.mp4_with_flag_read_auxiliary_tracks +++ /dev/null @@ -1,350 +0,0 @@ -seekMap: - isSeekable = true - duration = 1024000 - getPosition(0) = [[timeUs=0, position=2269]] - getPosition(1) = [[timeUs=0, position=2269]] - getPosition(512000) = [[timeUs=0, position=2269]] - getPosition(1024000) = [[timeUs=0, position=2269]] -numberOfTracks = 2 -track 0: - total output bytes = 89876 - sample count = 30 - track duration = 1001000 - format 0: - averageBitrate = 718290 - id = 1 - containerMimeType = video/mp4 - sampleMimeType = video/avc - codecs = avc1.64001F - maxInputSize = 36722 - maxNumReorderSamples = 2 - width = 1080 - height = 720 - frameRate = 29.97 - colorInfo: - lumaBitdepth = 8 - chromaBitdepth = 8 - language = und - metadata = entries=[TSSE: description=null: values=[Lavf56.1.0], xyz: latitude=40.68, longitude=-74.5, Mp4Timestamp: creation time=3547558895, modification time=3547558895, timescale=1000, ThumbnailMetadata: presentationTimeUs=0] - initializationData: - data = length 29, hash 4746B5D9 - data = length 10, hash 7A0D0F2B - sample 0: - time = 0 - flags = 1 - data = length 36692, hash D216076E - sample 1: - time = 66733 - flags = 0 - data = length 5312, hash D45D3CA0 - sample 2: - time = 33366 - flags = 0 - data = length 599, hash 1BE7812D - sample 3: - time = 200200 - flags = 0 - data = length 7735, hash 4490F110 - sample 4: - time = 133466 - flags = 0 - data = length 987, hash 560B5036 - sample 5: - time = 100100 - flags = 0 - data = length 673, hash ED7CD8C7 - sample 6: - time = 166833 - flags = 0 - data = length 523, hash 3020DF50 - sample 7: - time = 333666 - flags = 0 - data = length 6061, hash 736C72B2 - sample 8: - time = 266933 - flags = 0 - data = length 992, hash FE132F23 - sample 9: - time = 233566 - flags = 0 - data = length 623, hash 5B2C1816 - sample 10: - time = 300300 - flags = 0 - data = length 421, hash 742E69C1 - sample 11: - time = 433766 - flags = 0 - data = length 4899, hash F72F86A1 - sample 12: - time = 400400 - flags = 0 - data = length 568, hash 519A8E50 - sample 13: - time = 367033 - flags = 0 - data = length 620, hash 3990AA39 - sample 14: - time = 567233 - flags = 0 - data = length 5450, hash F06EC4AA - sample 15: - time = 500500 - flags = 0 - data = length 1051, hash 92DFA63A - sample 16: - time = 467133 - flags = 0 - data = length 874, hash 69587FB4 - sample 17: - time = 533866 - flags = 0 - data = length 781, hash 36BE495B - sample 18: - time = 700700 - flags = 0 - data = length 4725, hash AC0C8CD3 - sample 19: - time = 633966 - flags = 0 - data = length 1022, hash 5D8BFF34 - sample 20: - time = 600600 - flags = 0 - data = length 790, hash 99413A99 - sample 21: - time = 667333 - flags = 0 - data = length 610, hash 5E129290 - sample 22: - time = 834166 - flags = 0 - data = length 2751, hash 769974CB - sample 23: - time = 767433 - flags = 0 - data = length 745, hash B78A477A - sample 24: - time = 734066 - flags = 0 - data = length 621, hash CF741E7A - sample 25: - time = 800800 - flags = 0 - data = length 505, hash 1DB4894E - sample 26: - time = 967633 - flags = 0 - data = length 1268, hash C15348DC - sample 27: - time = 900900 - flags = 0 - data = length 880, hash C2DE85D0 - sample 28: - time = 867533 - flags = 0 - data = length 530, hash C98BC6A8 - sample 29: - time = 934266 - flags = 536870912 - data = length 568, hash 4FE5C8EA -track 1: - total output bytes = 9529 - sample count = 45 - track duration = 1024000 - format 0: - averageBitrate = 74502 - peakBitrate = 200000 - id = 2 - containerMimeType = video/mp4 - sampleMimeType = audio/mp4a-latm - codecs = mp4a.40.2 - maxInputSize = 294 - channelCount = 1 - sampleRate = 44100 - language = und - metadata = entries=[Mp4AlternateGroup: 1, TSSE: description=null: values=[Lavf56.1.0], xyz: latitude=40.68, longitude=-74.5, Mp4Timestamp: creation time=3547558895, modification time=3547558895, timescale=1000] - initializationData: - data = length 2, hash 5F7 - sample 0: - time = 44000 - flags = 1 - data = length 23, hash 47DE9131 - sample 1: - time = 67219 - flags = 1 - data = length 6, hash 31EC5206 - sample 2: - time = 90439 - flags = 1 - data = length 148, hash 894A176B - sample 3: - time = 113659 - flags = 1 - data = length 189, hash CEF235A1 - sample 4: - time = 136879 - flags = 1 - data = length 205, hash BBF5F7B0 - sample 5: - time = 160099 - flags = 1 - data = length 210, hash F278B193 - sample 6: - time = 183319 - flags = 1 - data = length 210, hash 82DA1589 - sample 7: - time = 206539 - flags = 1 - data = length 207, hash 5BE231DF - sample 8: - time = 229759 - flags = 1 - data = length 225, hash 18819EE1 - sample 9: - time = 252979 - flags = 1 - data = length 215, hash CA7FA67B - sample 10: - time = 276199 - flags = 1 - data = length 211, hash 581A1C18 - sample 11: - time = 299419 - flags = 1 - data = length 216, hash ADB88187 - sample 12: - time = 322639 - flags = 1 - data = length 229, hash 2E8BA4DC - sample 13: - time = 345859 - flags = 1 - data = length 232, hash 22F0C510 - sample 14: - time = 369079 - flags = 1 - data = length 235, hash 867AD0DC - sample 15: - time = 392299 - flags = 1 - data = length 231, hash 84E823A8 - sample 16: - time = 415519 - flags = 1 - data = length 226, hash 1BEF3A95 - sample 17: - time = 438739 - flags = 1 - data = length 216, hash EAA345AE - sample 18: - time = 461959 - flags = 1 - data = length 229, hash 6957411F - sample 19: - time = 485179 - flags = 1 - data = length 219, hash 41275022 - sample 20: - time = 508399 - flags = 1 - data = length 241, hash 6495DF96 - sample 21: - time = 531619 - flags = 1 - data = length 228, hash 63D95906 - sample 22: - time = 554839 - flags = 1 - data = length 238, hash 34F676F9 - sample 23: - time = 578058 - flags = 1 - data = length 234, hash E5CBC045 - sample 24: - time = 601278 - flags = 1 - data = length 231, hash 5FC43661 - sample 25: - time = 624498 - flags = 1 - data = length 217, hash 682708ED - sample 26: - time = 647718 - flags = 1 - data = length 239, hash D43780FC - sample 27: - time = 670938 - flags = 1 - data = length 243, hash C5E17980 - sample 28: - time = 694158 - flags = 1 - data = length 231, hash AC5837BA - sample 29: - time = 717378 - flags = 1 - data = length 230, hash 169EE895 - sample 30: - time = 740598 - flags = 1 - data = length 238, hash C48FF3F1 - sample 31: - time = 763818 - flags = 1 - data = length 225, hash 531E4599 - sample 32: - time = 787038 - flags = 1 - data = length 232, hash CB3C6B8D - sample 33: - time = 810258 - flags = 1 - data = length 243, hash F8C94C7 - sample 34: - time = 833478 - flags = 1 - data = length 232, hash A646A7D0 - sample 35: - time = 856698 - flags = 1 - data = length 237, hash E8B787A5 - sample 36: - time = 879918 - flags = 1 - data = length 228, hash 3FA7A29F - sample 37: - time = 903138 - flags = 1 - data = length 235, hash B9B33B0A - sample 38: - time = 926358 - flags = 1 - data = length 264, hash 71A4869E - sample 39: - time = 949578 - flags = 1 - data = length 257, hash D049B54C - sample 40: - time = 972798 - flags = 1 - data = length 227, hash 66757231 - sample 41: - time = 996018 - flags = 1 - data = length 227, hash BD374F1B - sample 42: - time = 1019238 - flags = 1 - data = length 235, hash 999477F6 - sample 43: - time = 1042458 - flags = 1 - data = length 229, hash FFF98DF0 - sample 44: - time = 1065678 - flags = 536870913 - data = length 6, hash 31B22286 -tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample.mp4_without_gop_parsing_flags b/libraries/test_data/src/test/assets/extractordumps/mp4/sample.mp4_without_gop_parsing_flags deleted file mode 100644 index 6d7ee3fb8bf..00000000000 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample.mp4_without_gop_parsing_flags +++ /dev/null @@ -1,350 +0,0 @@ -seekMap: - isSeekable = true - duration = 1024000 - getPosition(0) = [[timeUs=0, position=2269]] - getPosition(1) = [[timeUs=0, position=2269]] - getPosition(512000) = [[timeUs=0, position=2269]] - getPosition(1024000) = [[timeUs=0, position=2269]] -numberOfTracks = 2 -track 0: - total output bytes = 89876 - sample count = 30 - track duration = 1001000 - format 0: - averageBitrate = 718290 - id = 1 - containerMimeType = video/mp4 - sampleMimeType = video/avc - codecs = avc1.64001F - maxInputSize = 36722 - maxNumReorderSamples = 2 - width = 1080 - height = 720 - frameRate = 29.97 - colorInfo: - lumaBitdepth = 8 - chromaBitdepth = 8 - language = und - metadata = entries=[TSSE: description=null: values=[Lavf56.1.0], xyz: latitude=40.68, longitude=-74.5, Mp4Timestamp: creation time=3547558895, modification time=3547558895, timescale=1000, ThumbnailMetadata: presentationTimeUs=0] - initializationData: - data = length 29, hash 4746B5D9 - data = length 10, hash 7A0D0F2B - sample 0: - time = 0 - flags = 1 - data = length 36692, hash D216076E - sample 1: - time = 66733 - flags = 0 - data = length 5312, hash D45D3CA0 - sample 2: - time = 33366 - flags = 0 - data = length 599, hash 1BE7812D - sample 3: - time = 200200 - flags = 0 - data = length 7735, hash 4490F110 - sample 4: - time = 133466 - flags = 0 - data = length 987, hash 560B5036 - sample 5: - time = 100100 - flags = 0 - data = length 673, hash ED7CD8C7 - sample 6: - time = 166833 - flags = 0 - data = length 523, hash 3020DF50 - sample 7: - time = 333666 - flags = 0 - data = length 6061, hash 736C72B2 - sample 8: - time = 266933 - flags = 0 - data = length 992, hash FE132F23 - sample 9: - time = 233566 - flags = 0 - data = length 623, hash 5B2C1816 - sample 10: - time = 300300 - flags = 0 - data = length 421, hash 742E69C1 - sample 11: - time = 433766 - flags = 0 - data = length 4899, hash F72F86A1 - sample 12: - time = 400400 - flags = 0 - data = length 568, hash 519A8E50 - sample 13: - time = 367033 - flags = 0 - data = length 620, hash 3990AA39 - sample 14: - time = 567233 - flags = 0 - data = length 5450, hash F06EC4AA - sample 15: - time = 500500 - flags = 0 - data = length 1051, hash 92DFA63A - sample 16: - time = 467133 - flags = 0 - data = length 874, hash 69587FB4 - sample 17: - time = 533866 - flags = 0 - data = length 781, hash 36BE495B - sample 18: - time = 700700 - flags = 0 - data = length 4725, hash AC0C8CD3 - sample 19: - time = 633966 - flags = 0 - data = length 1022, hash 5D8BFF34 - sample 20: - time = 600600 - flags = 0 - data = length 790, hash 99413A99 - sample 21: - time = 667333 - flags = 0 - data = length 610, hash 5E129290 - sample 22: - time = 834166 - flags = 0 - data = length 2751, hash 769974CB - sample 23: - time = 767433 - flags = 0 - data = length 745, hash B78A477A - sample 24: - time = 734066 - flags = 0 - data = length 621, hash CF741E7A - sample 25: - time = 800800 - flags = 0 - data = length 505, hash 1DB4894E - sample 26: - time = 967633 - flags = 0 - data = length 1268, hash C15348DC - sample 27: - time = 900900 - flags = 0 - data = length 880, hash C2DE85D0 - sample 28: - time = 867533 - flags = 0 - data = length 530, hash C98BC6A8 - sample 29: - time = 934266 - flags = 536870912 - data = length 568, hash 4FE5C8EA -track 1: - total output bytes = 9529 - sample count = 45 - track duration = 1024000 - format 0: - averageBitrate = 74502 - peakBitrate = 200000 - id = 2 - containerMimeType = video/mp4 - sampleMimeType = audio/mp4a-latm - codecs = mp4a.40.2 - maxInputSize = 294 - channelCount = 1 - sampleRate = 44100 - language = und - metadata = entries=[Mp4AlternateGroup: 1, TSSE: description=null: values=[Lavf56.1.0], xyz: latitude=40.68, longitude=-74.5, Mp4Timestamp: creation time=3547558895, modification time=3547558895, timescale=1000] - initializationData: - data = length 2, hash 5F7 - sample 0: - time = 44000 - flags = 1 - data = length 23, hash 47DE9131 - sample 1: - time = 67219 - flags = 1 - data = length 6, hash 31EC5206 - sample 2: - time = 90439 - flags = 1 - data = length 148, hash 894A176B - sample 3: - time = 113659 - flags = 1 - data = length 189, hash CEF235A1 - sample 4: - time = 136879 - flags = 1 - data = length 205, hash BBF5F7B0 - sample 5: - time = 160099 - flags = 1 - data = length 210, hash F278B193 - sample 6: - time = 183319 - flags = 1 - data = length 210, hash 82DA1589 - sample 7: - time = 206539 - flags = 1 - data = length 207, hash 5BE231DF - sample 8: - time = 229759 - flags = 1 - data = length 225, hash 18819EE1 - sample 9: - time = 252979 - flags = 1 - data = length 215, hash CA7FA67B - sample 10: - time = 276199 - flags = 1 - data = length 211, hash 581A1C18 - sample 11: - time = 299419 - flags = 1 - data = length 216, hash ADB88187 - sample 12: - time = 322639 - flags = 1 - data = length 229, hash 2E8BA4DC - sample 13: - time = 345859 - flags = 1 - data = length 232, hash 22F0C510 - sample 14: - time = 369079 - flags = 1 - data = length 235, hash 867AD0DC - sample 15: - time = 392299 - flags = 1 - data = length 231, hash 84E823A8 - sample 16: - time = 415519 - flags = 1 - data = length 226, hash 1BEF3A95 - sample 17: - time = 438739 - flags = 1 - data = length 216, hash EAA345AE - sample 18: - time = 461959 - flags = 1 - data = length 229, hash 6957411F - sample 19: - time = 485179 - flags = 1 - data = length 219, hash 41275022 - sample 20: - time = 508399 - flags = 1 - data = length 241, hash 6495DF96 - sample 21: - time = 531619 - flags = 1 - data = length 228, hash 63D95906 - sample 22: - time = 554839 - flags = 1 - data = length 238, hash 34F676F9 - sample 23: - time = 578058 - flags = 1 - data = length 234, hash E5CBC045 - sample 24: - time = 601278 - flags = 1 - data = length 231, hash 5FC43661 - sample 25: - time = 624498 - flags = 1 - data = length 217, hash 682708ED - sample 26: - time = 647718 - flags = 1 - data = length 239, hash D43780FC - sample 27: - time = 670938 - flags = 1 - data = length 243, hash C5E17980 - sample 28: - time = 694158 - flags = 1 - data = length 231, hash AC5837BA - sample 29: - time = 717378 - flags = 1 - data = length 230, hash 169EE895 - sample 30: - time = 740598 - flags = 1 - data = length 238, hash C48FF3F1 - sample 31: - time = 763818 - flags = 1 - data = length 225, hash 531E4599 - sample 32: - time = 787038 - flags = 1 - data = length 232, hash CB3C6B8D - sample 33: - time = 810258 - flags = 1 - data = length 243, hash F8C94C7 - sample 34: - time = 833478 - flags = 1 - data = length 232, hash A646A7D0 - sample 35: - time = 856698 - flags = 1 - data = length 237, hash E8B787A5 - sample 36: - time = 879918 - flags = 1 - data = length 228, hash 3FA7A29F - sample 37: - time = 903138 - flags = 1 - data = length 235, hash B9B33B0A - sample 38: - time = 926358 - flags = 1 - data = length 264, hash 71A4869E - sample 39: - time = 949578 - flags = 1 - data = length 257, hash D049B54C - sample 40: - time = 972798 - flags = 1 - data = length 227, hash 66757231 - sample 41: - time = 996018 - flags = 1 - data = length 227, hash BD374F1B - sample 42: - time = 1019238 - flags = 1 - data = length 235, hash 999477F6 - sample 43: - time = 1042458 - flags = 1 - data = length 229, hash FFF98DF0 - sample 44: - time = 1065678 - flags = 536870913 - data = length 6, hash 31B22286 -tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.0.dump index 18be5b4841c..cebfdf9e3ad 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.0.dump @@ -21,6 +21,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086526, modification time=3662086526, timescale=48000] + initializationData: + data = length 5, hash 38742EF sample 0: time = 0 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.1.dump index 5553d6aacc5..b94bd054721 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.1.dump @@ -21,6 +21,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086526, modification time=3662086526, timescale=48000] + initializationData: + data = length 5, hash 38742EF sample 0: time = 576000 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.2.dump index 70065cd5b70..9bdb057269c 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.2.dump @@ -21,6 +21,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086526, modification time=3662086526, timescale=48000] + initializationData: + data = length 5, hash 38742EF sample 0: time = 1152000 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.3.dump index debb18e351b..cf30e637867 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.3.dump @@ -21,6 +21,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086526, modification time=3662086526, timescale=48000] + initializationData: + data = length 5, hash 38742EF sample 0: time = 1696000 flags = 536870913 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.unknown_length.dump index 18be5b4841c..cebfdf9e3ad 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3.mp4.unknown_length.dump @@ -21,6 +21,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086526, modification time=3662086526, timescale=48000] + initializationData: + data = length 5, hash 38742EF sample 0: time = 0 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.0.dump index d667a5b6bd3..75a49a12d08 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.0.dump @@ -19,6 +19,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086719, modification time=3662086719, timescale=1000] + initializationData: + data = length 5, hash 38742EF sample 0: time = 0 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.1.dump index b2fe4e15817..02e40e02a26 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.1.dump @@ -19,6 +19,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086719, modification time=3662086719, timescale=1000] + initializationData: + data = length 5, hash 38742EF sample 0: time = 576000 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.2.dump index 9b79dddd878..f18dc17dba6 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.2.dump @@ -19,6 +19,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086719, modification time=3662086719, timescale=1000] + initializationData: + data = length 5, hash 38742EF sample 0: time = 1152000 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.3.dump index e05add6a366..64d7b89915f 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.3.dump @@ -19,6 +19,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086719, modification time=3662086719, timescale=1000] + initializationData: + data = length 5, hash 38742EF sample 0: time = 1696000 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.unknown_length.dump index d667a5b6bd3..75a49a12d08 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3_fragmented.mp4.unknown_length.dump @@ -19,6 +19,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086719, modification time=3662086719, timescale=1000] + initializationData: + data = length 5, hash 38742EF sample 0: time = 0 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.0.dump index 722eacf3946..13ce08f4948 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.0.dump @@ -21,6 +21,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086940, modification time=3662086940, timescale=48000] + initializationData: + data = length 7, hash 8BA78FD3 sample 0: time = 0 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.1.dump index 0c4e9324c26..2b449a6a7bc 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.1.dump @@ -21,6 +21,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086940, modification time=3662086940, timescale=48000] + initializationData: + data = length 7, hash 8BA78FD3 sample 0: time = 672000 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.2.dump index 8391f766571..ca71fd43165 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.2.dump @@ -21,6 +21,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086940, modification time=3662086940, timescale=48000] + initializationData: + data = length 7, hash 8BA78FD3 sample 0: time = 1344000 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.3.dump index adac41cf3e0..640dd172db0 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.3.dump @@ -21,6 +21,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086940, modification time=3662086940, timescale=48000] + initializationData: + data = length 7, hash 8BA78FD3 sample 0: time = 2016000 flags = 536870913 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.unknown_length.dump index 722eacf3946..13ce08f4948 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc.mp4.unknown_length.dump @@ -21,6 +21,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086940, modification time=3662086940, timescale=48000] + initializationData: + data = length 7, hash 8BA78FD3 sample 0: time = 0 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.0.dump index 37ca0ee67ae..20860f5b0af 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.0.dump @@ -19,6 +19,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086977, modification time=3662086977, timescale=1000] + initializationData: + data = length 7, hash 8BA78FD3 sample 0: time = 0 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.1.dump index a95b4b0033a..e60d468c52c 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.1.dump @@ -19,6 +19,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086977, modification time=3662086977, timescale=1000] + initializationData: + data = length 7, hash 8BA78FD3 sample 0: time = 672000 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.2.dump index 3f5ff4f7381..cc9d4353c5d 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.2.dump @@ -19,6 +19,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086977, modification time=3662086977, timescale=1000] + initializationData: + data = length 7, hash 8BA78FD3 sample 0: time = 1344000 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.3.dump index e4372fd8373..9bfffca5a28 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.3.dump @@ -19,6 +19,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086977, modification time=3662086977, timescale=1000] + initializationData: + data = length 7, hash 8BA78FD3 sample 0: time = 2016000 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.unknown_length.dump index 37ca0ee67ae..20860f5b0af 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_eac3joc_fragmented.mp4.unknown_length.dump @@ -19,6 +19,8 @@ track 0: sampleRate = 48000 language = und metadata = entries=[Mp4Timestamp: creation time=3662086977, modification time=3662086977, timescale=1000] + initializationData: + data = length 7, hash 8BA78FD3 sample 0: time = 0 flags = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.0.dump index 0d76a548bbc..83b74b26927 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.0.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 1065600 + duration = 1065666 getPosition(0) = [[timeUs=0, position=44]] getPosition(1) = [[timeUs=0, position=44]] - getPosition(532800) = [[timeUs=0, position=44]] - getPosition(1065600) = [[timeUs=0, position=44]] + getPosition(532833) = [[timeUs=0, position=44]] + getPosition(1065666) = [[timeUs=0, position=44]] numberOfTracks = 2 track 0: total output bytes = 301392 sample count = 30 - track duration = 967600 + track duration = 967622 format 0: averageBitrate = 2491816 id = 1 @@ -154,7 +154,7 @@ track 0: track 1: total output bytes = 9529 sample count = 45 - track duration = 1065600 + track duration = 1065666 format 0: averageBitrate = 71535 id = 3 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.1.dump index 381072192f0..43d207fc3af 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.1.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 1065600 + duration = 1065666 getPosition(0) = [[timeUs=0, position=44]] getPosition(1) = [[timeUs=0, position=44]] - getPosition(532800) = [[timeUs=0, position=44]] - getPosition(1065600) = [[timeUs=0, position=44]] + getPosition(532833) = [[timeUs=0, position=44]] + getPosition(1065666) = [[timeUs=0, position=44]] numberOfTracks = 2 track 0: total output bytes = 301392 sample count = 30 - track duration = 967600 + track duration = 967622 format 0: averageBitrate = 2491816 id = 1 @@ -154,7 +154,7 @@ track 0: track 1: total output bytes = 7235 sample count = 32 - track duration = 1065600 + track duration = 1065666 format 0: averageBitrate = 71535 id = 3 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.2.dump index 402b816ea3f..22cf64f823f 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.2.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 1065600 + duration = 1065666 getPosition(0) = [[timeUs=0, position=44]] getPosition(1) = [[timeUs=0, position=44]] - getPosition(532800) = [[timeUs=0, position=44]] - getPosition(1065600) = [[timeUs=0, position=44]] + getPosition(532833) = [[timeUs=0, position=44]] + getPosition(1065666) = [[timeUs=0, position=44]] numberOfTracks = 2 track 0: total output bytes = 301392 sample count = 30 - track duration = 967600 + track duration = 967622 format 0: averageBitrate = 2491816 id = 1 @@ -154,7 +154,7 @@ track 0: track 1: total output bytes = 3776 sample count = 17 - track duration = 1065600 + track duration = 1065666 format 0: averageBitrate = 71535 id = 3 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.3.dump index f26028a12a8..ea39d1b82fa 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.3.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 1065600 + duration = 1065666 getPosition(0) = [[timeUs=0, position=44]] getPosition(1) = [[timeUs=0, position=44]] - getPosition(532800) = [[timeUs=0, position=44]] - getPosition(1065600) = [[timeUs=0, position=44]] + getPosition(532833) = [[timeUs=0, position=44]] + getPosition(1065666) = [[timeUs=0, position=44]] numberOfTracks = 2 track 0: total output bytes = 301392 sample count = 30 - track duration = 967600 + track duration = 967622 format 0: averageBitrate = 2491816 id = 1 @@ -152,9 +152,9 @@ track 0: flags = 536870912 data = length 14240, hash E190BF31 track 1: - total output bytes = 235 - sample count = 2 - track duration = 1065600 + total output bytes = 6 + sample count = 1 + track duration = 1065666 format 0: averageBitrate = 71535 id = 3 @@ -169,10 +169,6 @@ track 1: initializationData: data = length 2, hash 5F7 sample 0: - time = 1042437 - flags = 1 - data = length 229, hash FFF98DF0 - sample 1: time = 1065666 flags = 536870913 data = length 6, hash 31B22286 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.unknown_length.dump index 0d76a548bbc..83b74b26927 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_empty_track.mp4.unknown_length.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 1065600 + duration = 1065666 getPosition(0) = [[timeUs=0, position=44]] getPosition(1) = [[timeUs=0, position=44]] - getPosition(532800) = [[timeUs=0, position=44]] - getPosition(1065600) = [[timeUs=0, position=44]] + getPosition(532833) = [[timeUs=0, position=44]] + getPosition(1065666) = [[timeUs=0, position=44]] numberOfTracks = 2 track 0: total output bytes = 301392 sample count = 30 - track duration = 967600 + track duration = 967622 format 0: averageBitrate = 2491816 id = 1 @@ -154,7 +154,7 @@ track 0: track 1: total output bytes = 9529 sample count = 45 - track duration = 1065600 + track duration = 1065666 format 0: averageBitrate = 71535 id = 3 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented.mp4_without_gop_parsing_flags b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented.mp4_without_gop_parsing_flags deleted file mode 100644 index fec270d8678..00000000000 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented.mp4_without_gop_parsing_flags +++ /dev/null @@ -1,345 +0,0 @@ -seekMap: - isSeekable = true - duration = UNSET TIME - getPosition(0) = [[timeUs=0, position=1244]] - getPosition(1) = [[timeUs=0, position=1244]] -numberOfTracks = 2 -track 0: - total output bytes = 85933 - sample count = 30 - format 0: - id = 1 - containerMimeType = video/mp4 - sampleMimeType = video/avc - codecs = avc1.64001F - maxNumReorderSamples = 2 - width = 1080 - height = 720 - colorInfo: - lumaBitdepth = 8 - chromaBitdepth = 8 - language = und - metadata = entries=[TSSE: description=null: values=[Lavf54.20.4], Mp4Timestamp: creation time=3547558895, modification time=3547558895, timescale=1000] - initializationData: - data = length 29, hash 4746B5D9 - data = length 10, hash 7A0D0F2B - sample 0: - time = 66733 - flags = 1 - data = length 38070, hash B58E1AEE - sample 1: - time = 200200 - flags = 0 - data = length 8340, hash 8AC449FF - sample 2: - time = 133466 - flags = 0 - data = length 1295, hash C0DA5090 - sample 3: - time = 100100 - flags = 0 - data = length 469, hash D6E0A200 - sample 4: - time = 166833 - flags = 0 - data = length 564, hash E5F56C5B - sample 5: - time = 333666 - flags = 0 - data = length 6075, hash 8756E49E - sample 6: - time = 266933 - flags = 0 - data = length 847, hash DCC2B618 - sample 7: - time = 233566 - flags = 0 - data = length 455, hash B9CCE047 - sample 8: - time = 300300 - flags = 0 - data = length 467, hash 69806D94 - sample 9: - time = 467133 - flags = 0 - data = length 4549, hash 3944F501 - sample 10: - time = 400400 - flags = 0 - data = length 1087, hash 491BF106 - sample 11: - time = 367033 - flags = 0 - data = length 380, hash 5FED016A - sample 12: - time = 433766 - flags = 0 - data = length 455, hash 8A0610 - sample 13: - time = 600600 - flags = 0 - data = length 5190, hash B9031D8 - sample 14: - time = 533866 - flags = 0 - data = length 1071, hash 684E7DC8 - sample 15: - time = 500500 - flags = 0 - data = length 653, hash 8494F326 - sample 16: - time = 567233 - flags = 0 - data = length 485, hash 2CCC85F4 - sample 17: - time = 734066 - flags = 0 - data = length 4884, hash D16B6A96 - sample 18: - time = 667333 - flags = 0 - data = length 997, hash 164FF210 - sample 19: - time = 633966 - flags = 0 - data = length 640, hash F664125B - sample 20: - time = 700700 - flags = 0 - data = length 491, hash B5930C7C - sample 21: - time = 867533 - flags = 0 - data = length 2989, hash 92CF4FCF - sample 22: - time = 800800 - flags = 0 - data = length 838, hash 294A3451 - sample 23: - time = 767433 - flags = 0 - data = length 544, hash FCCE2DE6 - sample 24: - time = 834166 - flags = 0 - data = length 329, hash A654FFA1 - sample 25: - time = 1001000 - flags = 0 - data = length 1517, hash 5F7EBF8B - sample 26: - time = 934266 - flags = 0 - data = length 803, hash 7A5C4C1D - sample 27: - time = 900900 - flags = 0 - data = length 415, hash B31BBC3B - sample 28: - time = 967633 - flags = 0 - data = length 415, hash 850DFEA3 - sample 29: - time = 1034366 - flags = 0 - data = length 619, hash AB5E56CA -track 1: - total output bytes = 18257 - sample count = 46 - format 0: - peakBitrate = 200000 - id = 2 - containerMimeType = video/mp4 - sampleMimeType = audio/mp4a-latm - codecs = mp4a.40.2 - channelCount = 1 - sampleRate = 44100 - language = und - metadata = entries=[Mp4AlternateGroup: 1, TSSE: description=null: values=[Lavf54.20.4], Mp4Timestamp: creation time=3547558895, modification time=3547558895, timescale=1000] - initializationData: - data = length 5, hash 2B7623A - sample 0: - time = 0 - flags = 1 - data = length 18, hash 96519432 - sample 1: - time = 23219 - flags = 1 - data = length 4, hash EE9DF - sample 2: - time = 46439 - flags = 1 - data = length 4, hash EEDBF - sample 3: - time = 69659 - flags = 1 - data = length 157, hash E2F078F4 - sample 4: - time = 92879 - flags = 1 - data = length 371, hash B9471F94 - sample 5: - time = 116099 - flags = 1 - data = length 373, hash 2AB265CB - sample 6: - time = 139319 - flags = 1 - data = length 402, hash 1295477C - sample 7: - time = 162539 - flags = 1 - data = length 455, hash 2D8146C8 - sample 8: - time = 185759 - flags = 1 - data = length 434, hash F2C5D287 - sample 9: - time = 208979 - flags = 1 - data = length 450, hash 84143FCD - sample 10: - time = 232199 - flags = 1 - data = length 429, hash EF769D50 - sample 11: - time = 255419 - flags = 1 - data = length 450, hash EC3DE692 - sample 12: - time = 278639 - flags = 1 - data = length 447, hash 3E519E13 - sample 13: - time = 301859 - flags = 1 - data = length 457, hash 1E4F23A0 - sample 14: - time = 325079 - flags = 1 - data = length 447, hash A439EA97 - sample 15: - time = 348299 - flags = 1 - data = length 456, hash 1E9034C6 - sample 16: - time = 371519 - flags = 1 - data = length 398, hash 99DB7345 - sample 17: - time = 394739 - flags = 1 - data = length 474, hash 3F05F10A - sample 18: - time = 417959 - flags = 1 - data = length 416, hash C105EE09 - sample 19: - time = 441179 - flags = 1 - data = length 454, hash 5FDBE458 - sample 20: - time = 464399 - flags = 1 - data = length 438, hash 41A93AC3 - sample 21: - time = 487619 - flags = 1 - data = length 443, hash 10FDA652 - sample 22: - time = 510839 - flags = 1 - data = length 412, hash 1F791E25 - sample 23: - time = 534058 - flags = 1 - data = length 482, hash A6D983D - sample 24: - time = 557278 - flags = 1 - data = length 386, hash BED7392F - sample 25: - time = 580498 - flags = 1 - data = length 463, hash 5309F8C9 - sample 26: - time = 603718 - flags = 1 - data = length 394, hash 21C7321F - sample 27: - time = 626938 - flags = 1 - data = length 489, hash 71B4730D - sample 28: - time = 650158 - flags = 1 - data = length 403, hash D9C6DE89 - sample 29: - time = 673378 - flags = 1 - data = length 447, hash 9B14B73B - sample 30: - time = 696598 - flags = 1 - data = length 439, hash 4760D35B - sample 31: - time = 719818 - flags = 1 - data = length 463, hash 1601F88D - sample 32: - time = 743038 - flags = 1 - data = length 423, hash D4AE6773 - sample 33: - time = 766258 - flags = 1 - data = length 497, hash A3C674D3 - sample 34: - time = 789478 - flags = 1 - data = length 419, hash D3734A1F - sample 35: - time = 812698 - flags = 1 - data = length 474, hash DFB41F9 - sample 36: - time = 835918 - flags = 1 - data = length 413, hash 53E7CB9F - sample 37: - time = 859138 - flags = 1 - data = length 445, hash D15B0E39 - sample 38: - time = 882358 - flags = 1 - data = length 453, hash 77ED81E4 - sample 39: - time = 905578 - flags = 1 - data = length 545, hash 3321AEB9 - sample 40: - time = 928798 - flags = 1 - data = length 317, hash F557D0E - sample 41: - time = 952018 - flags = 1 - data = length 537, hash ED58CF7B - sample 42: - time = 975238 - flags = 1 - data = length 458, hash 51CDAA10 - sample 43: - time = 998458 - flags = 1 - data = length 465, hash CBA1EFD7 - sample 44: - time = 1021678 - flags = 1 - data = length 446, hash D6735B8A - sample 45: - time = 1044897 - flags = 1 - data = length 10, hash A453EEBE -tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.0.dump index f97a5b458b4..41e3efc65af 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.0.dump @@ -41,11 +41,11 @@ track 0: data = length 68, hash 83DC1BAB sample 3: time = 33333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 4: time = 100000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 5: time = 266666 @@ -57,11 +57,11 @@ track 0: data = length 70, hash 98EFD089 sample 7: time = 166666 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 8: time = 233333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 9: time = 400000 @@ -73,11 +73,11 @@ track 0: data = length 70, hash 7BAE9641 sample 11: time = 300000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 12: time = 366666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 13: time = 533333 @@ -89,11 +89,11 @@ track 0: data = length 70, hash 5B33157E sample 15: time = 433333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 16: time = 500000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 17: time = 666666 @@ -105,11 +105,11 @@ track 0: data = length 70, hash ECF81CBA sample 19: time = 566666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 20: time = 633333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 21: time = 800000 @@ -121,11 +121,11 @@ track 0: data = length 70, hash 1DB944F5 sample 23: time = 700000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 24: time = 766666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 25: time = 933333 @@ -137,11 +137,11 @@ track 0: data = length 70, hash AF7E4C32 sample 27: time = 833333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 28: time = 900000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 29: time = 1066666 @@ -153,11 +153,11 @@ track 0: data = length 70, hash F383DB6C sample 31: time = 966666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 32: time = 1033333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 33: time = 1200000 @@ -169,11 +169,11 @@ track 0: data = length 70, hash B92866C9 sample 35: time = 1100000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 36: time = 1166666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 37: time = 1333333 @@ -185,11 +185,11 @@ track 0: data = length 70, hash E9E98F05 sample 39: time = 1233333 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 40: time = 1300000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 41: time = 1466666 @@ -201,11 +201,11 @@ track 0: data = length 70, hash 7BAE9641 sample 43: time = 1366666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 44: time = 1433333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 45: time = 1600000 @@ -217,11 +217,11 @@ track 0: data = length 70, hash 5B33157E sample 47: time = 1500000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 48: time = 1566666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 49: time = 1733333 @@ -233,11 +233,11 @@ track 0: data = length 70, hash ECF81CBA sample 51: time = 1633333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 52: time = 1700000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 53: time = 1866666 @@ -249,11 +249,11 @@ track 0: data = length 70, hash 1DB944F5 sample 55: time = 1766666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 56: time = 1833333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 57: time = 2000000 @@ -265,11 +265,11 @@ track 0: data = length 70, hash AF7E4C32 sample 59: time = 1900000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 60: time = 1966666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 61: time = 2133333 @@ -281,11 +281,11 @@ track 0: data = length 70, hash F383DB6C sample 63: time = 2033333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 64: time = 2100000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 65: time = 2266666 @@ -297,11 +297,11 @@ track 0: data = length 70, hash B92866C9 sample 67: time = 2166666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 68: time = 2233333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 69: time = 2400000 @@ -313,11 +313,11 @@ track 0: data = length 70, hash E9E98F05 sample 71: time = 2300000 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 72: time = 2366666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 73: time = 2533333 @@ -329,11 +329,11 @@ track 0: data = length 70, hash 7BAE9641 sample 75: time = 2433333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 76: time = 2500000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 77: time = 2666666 @@ -345,11 +345,11 @@ track 0: data = length 70, hash 5B33157E sample 79: time = 2566666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 80: time = 2633333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 81: time = 2800000 @@ -361,11 +361,11 @@ track 0: data = length 70, hash ECF81CBA sample 83: time = 2700000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 84: time = 2766666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 85: time = 2933333 @@ -377,11 +377,11 @@ track 0: data = length 70, hash 1DB944F5 sample 87: time = 2833333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 88: time = 2900000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 89: time = 3066666 @@ -393,11 +393,11 @@ track 0: data = length 70, hash AF7E4C32 sample 91: time = 2966666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 92: time = 3033333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 93: time = 3200000 @@ -409,11 +409,11 @@ track 0: data = length 70, hash F383DB6C sample 95: time = 3100000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 96: time = 3166666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 97: time = 3333333 @@ -425,11 +425,11 @@ track 0: data = length 70, hash B92866C9 sample 99: time = 3233333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 100: time = 3300000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 101: time = 3466666 @@ -441,11 +441,11 @@ track 0: data = length 70, hash E9E98F05 sample 103: time = 3366666 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 104: time = 3433333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 105: time = 3600000 @@ -457,11 +457,11 @@ track 0: data = length 70, hash 7BAE9641 sample 107: time = 3500000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 108: time = 3566666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 109: time = 3733333 @@ -473,11 +473,11 @@ track 0: data = length 70, hash 5B33157E sample 111: time = 3633333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 112: time = 3700000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 113: time = 3866666 @@ -489,11 +489,11 @@ track 0: data = length 70, hash ECF81CBA sample 115: time = 3766666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 116: time = 3833333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 117: time = 4000000 @@ -505,11 +505,11 @@ track 0: data = length 70, hash 1DB944F5 sample 119: time = 3900000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 120: time = 3966666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 121: time = 4133333 @@ -521,11 +521,11 @@ track 0: data = length 70, hash AF7E4C32 sample 123: time = 4033333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 124: time = 4100000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 125: time = 4266666 @@ -537,11 +537,11 @@ track 0: data = length 70, hash F383DB6C sample 127: time = 4166666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 128: time = 4233333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 129: time = 4400000 @@ -553,11 +553,11 @@ track 0: data = length 70, hash B92866C9 sample 131: time = 4300000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 132: time = 4366666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 133: time = 4533333 @@ -569,11 +569,11 @@ track 0: data = length 70, hash E9E98F05 sample 135: time = 4433333 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 136: time = 4500000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 137: time = 4666666 @@ -585,11 +585,11 @@ track 0: data = length 70, hash 7BAE9641 sample 139: time = 4566666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 140: time = 4633333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 141: time = 4800000 @@ -601,11 +601,11 @@ track 0: data = length 70, hash 5B33157E sample 143: time = 4700000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 144: time = 4766666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 145: time = 4933333 @@ -617,11 +617,11 @@ track 0: data = length 70, hash ECF81CBA sample 147: time = 4833333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 148: time = 4900000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 149: time = 5066666 @@ -633,11 +633,11 @@ track 0: data = length 70, hash 1DB944F5 sample 151: time = 4966666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 152: time = 5033333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 153: time = 5200000 @@ -649,11 +649,11 @@ track 0: data = length 70, hash AF7E4C32 sample 155: time = 5100000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 156: time = 5166666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 157: time = 5333333 @@ -665,11 +665,11 @@ track 0: data = length 70, hash F383DB6C sample 159: time = 5233333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 160: time = 5300000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 161: time = 5466666 @@ -681,11 +681,11 @@ track 0: data = length 70, hash B92866C9 sample 163: time = 5366666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 164: time = 5433333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 165: time = 5600000 @@ -697,11 +697,11 @@ track 0: data = length 70, hash E9E98F05 sample 167: time = 5500000 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 168: time = 5566666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 169: time = 5733333 @@ -713,11 +713,11 @@ track 0: data = length 70, hash 7BAE9641 sample 171: time = 5633333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 172: time = 5700000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 173: time = 5866666 @@ -729,11 +729,11 @@ track 0: data = length 70, hash 5B33157E sample 175: time = 5766666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 176: time = 5833333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 177: time = 6000000 @@ -745,11 +745,11 @@ track 0: data = length 70, hash ECF81CBA sample 179: time = 5900000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 180: time = 5966666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 181: time = 6133333 @@ -761,11 +761,11 @@ track 0: data = length 70, hash 1DB944F5 sample 183: time = 6033333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 184: time = 6100000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 185: time = 6266666 @@ -777,11 +777,11 @@ track 0: data = length 70, hash AF7E4C32 sample 187: time = 6166666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 188: time = 6233333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 189: time = 6400000 @@ -793,11 +793,11 @@ track 0: data = length 70, hash F383DB6C sample 191: time = 6300000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 192: time = 6366666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 193: time = 6533333 @@ -809,11 +809,11 @@ track 0: data = length 70, hash B92866C9 sample 195: time = 6433333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 196: time = 6500000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 197: time = 6666666 @@ -825,11 +825,11 @@ track 0: data = length 70, hash E9E98F05 sample 199: time = 6566666 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 200: time = 6633333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 201: time = 6800000 @@ -841,11 +841,11 @@ track 0: data = length 70, hash 7BAE9641 sample 203: time = 6700000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 204: time = 6766666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 205: time = 6933333 @@ -857,11 +857,11 @@ track 0: data = length 70, hash 5B33157E sample 207: time = 6833333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 208: time = 6900000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 209: time = 7066666 @@ -873,11 +873,11 @@ track 0: data = length 70, hash ECF81CBA sample 211: time = 6966666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 212: time = 7033333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 213: time = 7200000 @@ -889,11 +889,11 @@ track 0: data = length 70, hash 1DB944F5 sample 215: time = 7100000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 216: time = 7166666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 217: time = 7333333 @@ -905,11 +905,11 @@ track 0: data = length 70, hash AF7E4C32 sample 219: time = 7233333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 220: time = 7300000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 221: time = 7466666 @@ -921,11 +921,11 @@ track 0: data = length 70, hash F383DB6C sample 223: time = 7366666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 224: time = 7433333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 225: time = 7600000 @@ -937,11 +937,11 @@ track 0: data = length 70, hash B92866C9 sample 227: time = 7500000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 228: time = 7566666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 229: time = 7733333 @@ -953,11 +953,11 @@ track 0: data = length 70, hash E9E98F05 sample 231: time = 7633333 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 232: time = 7700000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 233: time = 7866666 @@ -969,11 +969,11 @@ track 0: data = length 70, hash 7BAE9641 sample 235: time = 7766666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 236: time = 7833333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 237: time = 8000000 @@ -985,11 +985,11 @@ track 0: data = length 70, hash 5B33157E sample 239: time = 7900000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 240: time = 7966666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 241: time = 8133333 @@ -1001,11 +1001,11 @@ track 0: data = length 70, hash ECF81CBA sample 243: time = 8033333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 244: time = 8100000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 245: time = 8266666 @@ -1017,11 +1017,11 @@ track 0: data = length 70, hash 1DB944F5 sample 247: time = 8166666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 248: time = 8233333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 249: time = 8300000 @@ -1041,11 +1041,11 @@ track 0: data = length 68, hash 83DC1BAB sample 253: time = 8366666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 254: time = 8433333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 255: time = 8600000 @@ -1057,11 +1057,11 @@ track 0: data = length 70, hash 98EFD089 sample 257: time = 8500000 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 258: time = 8566666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 259: time = 8733333 @@ -1073,11 +1073,11 @@ track 0: data = length 70, hash 7BAE9641 sample 261: time = 8633333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 262: time = 8700000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 263: time = 8866666 @@ -1089,11 +1089,11 @@ track 0: data = length 70, hash 5B33157D sample 265: time = 8766666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 266: time = 8833333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 267: time = 9000000 @@ -1105,11 +1105,11 @@ track 0: data = length 70, hash ECF81CB9 sample 269: time = 8900000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 270: time = 8966666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 271: time = 9133333 @@ -1121,11 +1121,11 @@ track 0: data = length 70, hash 1DB944F5 sample 273: time = 9033333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 274: time = 9100000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 275: time = 9266666 @@ -1137,11 +1137,11 @@ track 0: data = length 70, hash AF7E4C31 sample 277: time = 9166666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 278: time = 9233333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 279: time = 9400000 @@ -1153,11 +1153,11 @@ track 0: data = length 70, hash F383DB6D sample 281: time = 9300000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 282: time = 9366666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 283: time = 9533333 @@ -1169,11 +1169,11 @@ track 0: data = length 70, hash B92866C9 sample 285: time = 9433333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 286: time = 9500000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 287: time = 9666666 @@ -1185,11 +1185,11 @@ track 0: data = length 70, hash E9E98F05 sample 289: time = 9566666 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 290: time = 9633333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 291: time = 9800000 @@ -1201,11 +1201,11 @@ track 0: data = length 70, hash 7BAE9641 sample 293: time = 9700000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 294: time = 9766666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 295: time = 9933333 @@ -1217,11 +1217,11 @@ track 0: data = length 70, hash 5B33157D sample 297: time = 9833333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 298: time = 9900000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 299: time = 10066666 @@ -1233,11 +1233,11 @@ track 0: data = length 70, hash ECF81CB9 sample 301: time = 9966666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 302: time = 10033333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 303: time = 10200000 @@ -1249,11 +1249,11 @@ track 0: data = length 70, hash 1DB944F5 sample 305: time = 10100000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 306: time = 10166666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 307: time = 10333333 @@ -1265,11 +1265,11 @@ track 0: data = length 70, hash AF7E4C31 sample 309: time = 10233333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 310: time = 10300000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 311: time = 10466666 @@ -1281,11 +1281,11 @@ track 0: data = length 70, hash F383DB6D sample 313: time = 10366666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 314: time = 10433333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 315: time = 10600000 @@ -1297,11 +1297,11 @@ track 0: data = length 70, hash B92866C9 sample 317: time = 10500000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 318: time = 10566666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 319: time = 10733333 @@ -1313,11 +1313,11 @@ track 0: data = length 70, hash E9E98F05 sample 321: time = 10633333 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 322: time = 10700000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 323: time = 10866666 @@ -1329,11 +1329,11 @@ track 0: data = length 70, hash 7BAE9641 sample 325: time = 10766666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 326: time = 10833333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 327: time = 11000000 @@ -1345,11 +1345,11 @@ track 0: data = length 70, hash 5B33157D sample 329: time = 10900000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 330: time = 10966666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 331: time = 11133333 @@ -1361,11 +1361,11 @@ track 0: data = length 70, hash ECF81CB9 sample 333: time = 11033333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 334: time = 11100000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 335: time = 11266666 @@ -1377,11 +1377,11 @@ track 0: data = length 70, hash 1DB944F5 sample 337: time = 11166666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 338: time = 11233333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 339: time = 11400000 @@ -1393,11 +1393,11 @@ track 0: data = length 70, hash AF7E4C31 sample 341: time = 11300000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 342: time = 11366666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 343: time = 11533333 @@ -1409,11 +1409,11 @@ track 0: data = length 70, hash F383DB6D sample 345: time = 11433333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 346: time = 11500000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 347: time = 11666666 @@ -1425,11 +1425,11 @@ track 0: data = length 70, hash B92866C9 sample 349: time = 11566666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 350: time = 11633333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 351: time = 11800000 @@ -1441,11 +1441,11 @@ track 0: data = length 70, hash E9E98F05 sample 353: time = 11700000 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 354: time = 11766666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 355: time = 11933333 @@ -1457,11 +1457,11 @@ track 0: data = length 70, hash 7BAE9641 sample 357: time = 11833333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 358: time = 11900000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 359: time = 12066666 @@ -1473,11 +1473,11 @@ track 0: data = length 70, hash 5B33157D sample 361: time = 11966666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 362: time = 12033333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 363: time = 12200000 @@ -1489,11 +1489,11 @@ track 0: data = length 70, hash ECF81CB9 sample 365: time = 12100000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 366: time = 12166666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 367: time = 12333333 @@ -1505,11 +1505,11 @@ track 0: data = length 70, hash 1DB944F5 sample 369: time = 12233333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 370: time = 12300000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 371: time = 12466666 @@ -1521,11 +1521,11 @@ track 0: data = length 70, hash AF7E4C31 sample 373: time = 12366666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 374: time = 12433333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 375: time = 12600000 @@ -1537,11 +1537,11 @@ track 0: data = length 70, hash F383DB6D sample 377: time = 12500000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 378: time = 12566666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 379: time = 12733333 @@ -1553,11 +1553,11 @@ track 0: data = length 70, hash B92866C9 sample 381: time = 12633333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 382: time = 12700000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 383: time = 12866666 @@ -1569,11 +1569,11 @@ track 0: data = length 70, hash E9E98F05 sample 385: time = 12766666 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 386: time = 12833333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 387: time = 13000000 @@ -1585,11 +1585,11 @@ track 0: data = length 70, hash 7BAE9641 sample 389: time = 12900000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 390: time = 12966666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 391: time = 13133333 @@ -1601,11 +1601,11 @@ track 0: data = length 70, hash 5B33157D sample 393: time = 13033333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 394: time = 13100000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 395: time = 13266666 @@ -1617,11 +1617,11 @@ track 0: data = length 70, hash ECF81CB9 sample 397: time = 13166666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 398: time = 13233333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 399: time = 13400000 @@ -1633,11 +1633,11 @@ track 0: data = length 70, hash 1DB944F5 sample 401: time = 13300000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 402: time = 13366666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 403: time = 13533333 @@ -1649,11 +1649,11 @@ track 0: data = length 70, hash AF7E4C31 sample 405: time = 13433333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 406: time = 13500000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 407: time = 13666666 @@ -1665,11 +1665,11 @@ track 0: data = length 70, hash F383DB6D sample 409: time = 13566666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 410: time = 13633333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 411: time = 13800000 @@ -1681,11 +1681,11 @@ track 0: data = length 70, hash B92866C9 sample 413: time = 13700000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 414: time = 13766666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 415: time = 13933333 @@ -1697,11 +1697,11 @@ track 0: data = length 70, hash E9E98F05 sample 417: time = 13833333 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 418: time = 13900000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 419: time = 14066666 @@ -1713,11 +1713,11 @@ track 0: data = length 70, hash 7BAE9641 sample 421: time = 13966666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 422: time = 14033333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 423: time = 14200000 @@ -1729,11 +1729,11 @@ track 0: data = length 70, hash 5B33157D sample 425: time = 14100000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 426: time = 14166666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 427: time = 14333333 @@ -1745,11 +1745,11 @@ track 0: data = length 70, hash ECF81CB9 sample 429: time = 14233333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 430: time = 14300000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 431: time = 14466666 @@ -1761,11 +1761,11 @@ track 0: data = length 70, hash 1DB944F5 sample 433: time = 14366666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 434: time = 14433333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 435: time = 14600000 @@ -1777,11 +1777,11 @@ track 0: data = length 70, hash AF7E4C31 sample 437: time = 14500000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 438: time = 14566666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 439: time = 14733333 @@ -1793,11 +1793,11 @@ track 0: data = length 70, hash F383DB6D sample 441: time = 14633333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 442: time = 14700000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 443: time = 14866666 @@ -1809,11 +1809,11 @@ track 0: data = length 70, hash B92866C9 sample 445: time = 14766666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 446: time = 14833333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 447: time = 15000000 @@ -1825,11 +1825,11 @@ track 0: data = length 70, hash E9E98F05 sample 449: time = 14900000 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 450: time = 14966666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 451: time = 15133333 @@ -1841,11 +1841,11 @@ track 0: data = length 70, hash 7BAE9641 sample 453: time = 15033333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 454: time = 15100000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 455: time = 15266666 @@ -1857,11 +1857,11 @@ track 0: data = length 70, hash 5B33157D sample 457: time = 15166666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 458: time = 15233333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 459: time = 15400000 @@ -1873,11 +1873,11 @@ track 0: data = length 70, hash ECF81CB9 sample 461: time = 15300000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 462: time = 15366666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 463: time = 15533333 @@ -1889,11 +1889,11 @@ track 0: data = length 70, hash 1DB944F5 sample 465: time = 15433333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 466: time = 15500000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 467: time = 15666666 @@ -1905,11 +1905,11 @@ track 0: data = length 70, hash AF7E4C31 sample 469: time = 15566666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 470: time = 15633333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 471: time = 15800000 @@ -1921,11 +1921,11 @@ track 0: data = length 70, hash F383DB6D sample 473: time = 15700000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 474: time = 15766666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 475: time = 15933333 @@ -1937,11 +1937,11 @@ track 0: data = length 70, hash B92866C9 sample 477: time = 15833333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 478: time = 15900000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 479: time = 16066666 @@ -1953,11 +1953,11 @@ track 0: data = length 70, hash E9E98F05 sample 481: time = 15966666 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 482: time = 16033333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 483: time = 16200000 @@ -1969,11 +1969,11 @@ track 0: data = length 70, hash 7BAE9641 sample 485: time = 16100000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 486: time = 16166666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 487: time = 16333333 @@ -1985,11 +1985,11 @@ track 0: data = length 70, hash 5B33157D sample 489: time = 16233333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 490: time = 16300000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 491: time = 16466666 @@ -2001,11 +2001,11 @@ track 0: data = length 70, hash ECF81CB9 sample 493: time = 16366666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 494: time = 16433333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 495: time = 16600000 @@ -2017,11 +2017,11 @@ track 0: data = length 70, hash 1DB944F5 sample 497: time = 16500000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 498: time = 16566666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 499: time = 16633333 @@ -2041,11 +2041,11 @@ track 0: data = length 68, hash 83DC1BAA sample 503: time = 16700000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 504: time = 16766666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 505: time = 16933333 @@ -2057,11 +2057,11 @@ track 0: data = length 70, hash 98EFD089 sample 507: time = 16833333 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 508: time = 16900000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 509: time = 17066666 @@ -2073,11 +2073,11 @@ track 0: data = length 70, hash 7BAE9640 sample 511: time = 16966666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 512: time = 17033333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 513: time = 17200000 @@ -2089,11 +2089,11 @@ track 0: data = length 70, hash 5B33157E sample 515: time = 17100000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 516: time = 17166666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 517: time = 17333333 @@ -2105,11 +2105,11 @@ track 0: data = length 70, hash ECF81CBA sample 519: time = 17233333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 520: time = 17300000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 521: time = 17466666 @@ -2121,11 +2121,11 @@ track 0: data = length 70, hash 1DB944F6 sample 523: time = 17366666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 524: time = 17433333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA5 sample 525: time = 17600000 @@ -2137,11 +2137,11 @@ track 0: data = length 70, hash AF7E4C32 sample 527: time = 17500000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 528: time = 17566666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 529: time = 17733333 @@ -2153,11 +2153,11 @@ track 0: data = length 70, hash F383DB6D sample 531: time = 17633333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 532: time = 17700000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DC sample 533: time = 17866666 @@ -2169,11 +2169,11 @@ track 0: data = length 70, hash B92866C8 sample 535: time = 17766666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 536: time = 17833333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 537: time = 18000000 @@ -2185,11 +2185,11 @@ track 0: data = length 70, hash E9E98F05 sample 539: time = 17900000 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 540: time = 17966666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 541: time = 18133333 @@ -2201,11 +2201,11 @@ track 0: data = length 70, hash 7BAE9640 sample 543: time = 18033333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 544: time = 18100000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 545: time = 18266666 @@ -2217,11 +2217,11 @@ track 0: data = length 70, hash 5B33157E sample 547: time = 18166666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 548: time = 18233333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 549: time = 18400000 @@ -2233,11 +2233,11 @@ track 0: data = length 70, hash ECF81CBA sample 551: time = 18300000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 552: time = 18366666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 553: time = 18533333 @@ -2249,11 +2249,11 @@ track 0: data = length 70, hash 1DB944F6 sample 555: time = 18433333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 556: time = 18500000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA5 sample 557: time = 18666666 @@ -2265,11 +2265,11 @@ track 0: data = length 70, hash AF7E4C32 sample 559: time = 18566666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 560: time = 18633333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 561: time = 18800000 @@ -2281,11 +2281,11 @@ track 0: data = length 70, hash F383DB6D sample 563: time = 18700000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 564: time = 18766666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DC sample 565: time = 18933333 @@ -2297,11 +2297,11 @@ track 0: data = length 70, hash B92866C8 sample 567: time = 18833333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 568: time = 18900000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 569: time = 19066666 @@ -2313,11 +2313,11 @@ track 0: data = length 70, hash E9E98F05 sample 571: time = 18966666 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 572: time = 19033333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 573: time = 19200000 @@ -2329,11 +2329,11 @@ track 0: data = length 70, hash 7BAE9640 sample 575: time = 19100000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 576: time = 19166666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 577: time = 19333333 @@ -2345,11 +2345,11 @@ track 0: data = length 70, hash 5B33157E sample 579: time = 19233333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 580: time = 19300000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 581: time = 19466666 @@ -2361,11 +2361,11 @@ track 0: data = length 70, hash ECF81CBA sample 583: time = 19366666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 584: time = 19433333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 585: time = 19600000 @@ -2377,11 +2377,11 @@ track 0: data = length 70, hash 1DB944F6 sample 587: time = 19500000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 588: time = 19566666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA5 sample 589: time = 19733333 @@ -2393,11 +2393,11 @@ track 0: data = length 70, hash AF7E4C32 sample 591: time = 19633333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 592: time = 19700000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 593: time = 19866666 @@ -2409,11 +2409,11 @@ track 0: data = length 70, hash F383DB6D sample 595: time = 19766666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 596: time = 19833333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DC sample 597: time = 19966666 @@ -2425,6 +2425,6 @@ track 0: data = length 70, hash 3AC762E9 sample 599: time = 19933333 - flags = 0 + flags = 67108864 data = length 68, hash B1C7769A tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.1.dump index f97a5b458b4..41e3efc65af 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.1.dump @@ -41,11 +41,11 @@ track 0: data = length 68, hash 83DC1BAB sample 3: time = 33333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 4: time = 100000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 5: time = 266666 @@ -57,11 +57,11 @@ track 0: data = length 70, hash 98EFD089 sample 7: time = 166666 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 8: time = 233333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 9: time = 400000 @@ -73,11 +73,11 @@ track 0: data = length 70, hash 7BAE9641 sample 11: time = 300000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 12: time = 366666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 13: time = 533333 @@ -89,11 +89,11 @@ track 0: data = length 70, hash 5B33157E sample 15: time = 433333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 16: time = 500000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 17: time = 666666 @@ -105,11 +105,11 @@ track 0: data = length 70, hash ECF81CBA sample 19: time = 566666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 20: time = 633333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 21: time = 800000 @@ -121,11 +121,11 @@ track 0: data = length 70, hash 1DB944F5 sample 23: time = 700000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 24: time = 766666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 25: time = 933333 @@ -137,11 +137,11 @@ track 0: data = length 70, hash AF7E4C32 sample 27: time = 833333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 28: time = 900000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 29: time = 1066666 @@ -153,11 +153,11 @@ track 0: data = length 70, hash F383DB6C sample 31: time = 966666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 32: time = 1033333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 33: time = 1200000 @@ -169,11 +169,11 @@ track 0: data = length 70, hash B92866C9 sample 35: time = 1100000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 36: time = 1166666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 37: time = 1333333 @@ -185,11 +185,11 @@ track 0: data = length 70, hash E9E98F05 sample 39: time = 1233333 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 40: time = 1300000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 41: time = 1466666 @@ -201,11 +201,11 @@ track 0: data = length 70, hash 7BAE9641 sample 43: time = 1366666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 44: time = 1433333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 45: time = 1600000 @@ -217,11 +217,11 @@ track 0: data = length 70, hash 5B33157E sample 47: time = 1500000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 48: time = 1566666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 49: time = 1733333 @@ -233,11 +233,11 @@ track 0: data = length 70, hash ECF81CBA sample 51: time = 1633333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 52: time = 1700000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 53: time = 1866666 @@ -249,11 +249,11 @@ track 0: data = length 70, hash 1DB944F5 sample 55: time = 1766666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 56: time = 1833333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 57: time = 2000000 @@ -265,11 +265,11 @@ track 0: data = length 70, hash AF7E4C32 sample 59: time = 1900000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 60: time = 1966666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 61: time = 2133333 @@ -281,11 +281,11 @@ track 0: data = length 70, hash F383DB6C sample 63: time = 2033333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 64: time = 2100000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 65: time = 2266666 @@ -297,11 +297,11 @@ track 0: data = length 70, hash B92866C9 sample 67: time = 2166666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 68: time = 2233333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 69: time = 2400000 @@ -313,11 +313,11 @@ track 0: data = length 70, hash E9E98F05 sample 71: time = 2300000 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 72: time = 2366666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 73: time = 2533333 @@ -329,11 +329,11 @@ track 0: data = length 70, hash 7BAE9641 sample 75: time = 2433333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 76: time = 2500000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 77: time = 2666666 @@ -345,11 +345,11 @@ track 0: data = length 70, hash 5B33157E sample 79: time = 2566666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 80: time = 2633333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 81: time = 2800000 @@ -361,11 +361,11 @@ track 0: data = length 70, hash ECF81CBA sample 83: time = 2700000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 84: time = 2766666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 85: time = 2933333 @@ -377,11 +377,11 @@ track 0: data = length 70, hash 1DB944F5 sample 87: time = 2833333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 88: time = 2900000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 89: time = 3066666 @@ -393,11 +393,11 @@ track 0: data = length 70, hash AF7E4C32 sample 91: time = 2966666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 92: time = 3033333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 93: time = 3200000 @@ -409,11 +409,11 @@ track 0: data = length 70, hash F383DB6C sample 95: time = 3100000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 96: time = 3166666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 97: time = 3333333 @@ -425,11 +425,11 @@ track 0: data = length 70, hash B92866C9 sample 99: time = 3233333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 100: time = 3300000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 101: time = 3466666 @@ -441,11 +441,11 @@ track 0: data = length 70, hash E9E98F05 sample 103: time = 3366666 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 104: time = 3433333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 105: time = 3600000 @@ -457,11 +457,11 @@ track 0: data = length 70, hash 7BAE9641 sample 107: time = 3500000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 108: time = 3566666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 109: time = 3733333 @@ -473,11 +473,11 @@ track 0: data = length 70, hash 5B33157E sample 111: time = 3633333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 112: time = 3700000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 113: time = 3866666 @@ -489,11 +489,11 @@ track 0: data = length 70, hash ECF81CBA sample 115: time = 3766666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 116: time = 3833333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 117: time = 4000000 @@ -505,11 +505,11 @@ track 0: data = length 70, hash 1DB944F5 sample 119: time = 3900000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 120: time = 3966666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 121: time = 4133333 @@ -521,11 +521,11 @@ track 0: data = length 70, hash AF7E4C32 sample 123: time = 4033333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 124: time = 4100000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 125: time = 4266666 @@ -537,11 +537,11 @@ track 0: data = length 70, hash F383DB6C sample 127: time = 4166666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 128: time = 4233333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 129: time = 4400000 @@ -553,11 +553,11 @@ track 0: data = length 70, hash B92866C9 sample 131: time = 4300000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 132: time = 4366666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 133: time = 4533333 @@ -569,11 +569,11 @@ track 0: data = length 70, hash E9E98F05 sample 135: time = 4433333 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 136: time = 4500000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 137: time = 4666666 @@ -585,11 +585,11 @@ track 0: data = length 70, hash 7BAE9641 sample 139: time = 4566666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 140: time = 4633333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 141: time = 4800000 @@ -601,11 +601,11 @@ track 0: data = length 70, hash 5B33157E sample 143: time = 4700000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 144: time = 4766666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 145: time = 4933333 @@ -617,11 +617,11 @@ track 0: data = length 70, hash ECF81CBA sample 147: time = 4833333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 148: time = 4900000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 149: time = 5066666 @@ -633,11 +633,11 @@ track 0: data = length 70, hash 1DB944F5 sample 151: time = 4966666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 152: time = 5033333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 153: time = 5200000 @@ -649,11 +649,11 @@ track 0: data = length 70, hash AF7E4C32 sample 155: time = 5100000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 156: time = 5166666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 157: time = 5333333 @@ -665,11 +665,11 @@ track 0: data = length 70, hash F383DB6C sample 159: time = 5233333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 160: time = 5300000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 161: time = 5466666 @@ -681,11 +681,11 @@ track 0: data = length 70, hash B92866C9 sample 163: time = 5366666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 164: time = 5433333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 165: time = 5600000 @@ -697,11 +697,11 @@ track 0: data = length 70, hash E9E98F05 sample 167: time = 5500000 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 168: time = 5566666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 169: time = 5733333 @@ -713,11 +713,11 @@ track 0: data = length 70, hash 7BAE9641 sample 171: time = 5633333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 172: time = 5700000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 173: time = 5866666 @@ -729,11 +729,11 @@ track 0: data = length 70, hash 5B33157E sample 175: time = 5766666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 176: time = 5833333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 177: time = 6000000 @@ -745,11 +745,11 @@ track 0: data = length 70, hash ECF81CBA sample 179: time = 5900000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 180: time = 5966666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 181: time = 6133333 @@ -761,11 +761,11 @@ track 0: data = length 70, hash 1DB944F5 sample 183: time = 6033333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 184: time = 6100000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 185: time = 6266666 @@ -777,11 +777,11 @@ track 0: data = length 70, hash AF7E4C32 sample 187: time = 6166666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 188: time = 6233333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 189: time = 6400000 @@ -793,11 +793,11 @@ track 0: data = length 70, hash F383DB6C sample 191: time = 6300000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 192: time = 6366666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 193: time = 6533333 @@ -809,11 +809,11 @@ track 0: data = length 70, hash B92866C9 sample 195: time = 6433333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 196: time = 6500000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 197: time = 6666666 @@ -825,11 +825,11 @@ track 0: data = length 70, hash E9E98F05 sample 199: time = 6566666 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 200: time = 6633333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 201: time = 6800000 @@ -841,11 +841,11 @@ track 0: data = length 70, hash 7BAE9641 sample 203: time = 6700000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 204: time = 6766666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 205: time = 6933333 @@ -857,11 +857,11 @@ track 0: data = length 70, hash 5B33157E sample 207: time = 6833333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 208: time = 6900000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 209: time = 7066666 @@ -873,11 +873,11 @@ track 0: data = length 70, hash ECF81CBA sample 211: time = 6966666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 212: time = 7033333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 213: time = 7200000 @@ -889,11 +889,11 @@ track 0: data = length 70, hash 1DB944F5 sample 215: time = 7100000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 216: time = 7166666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 217: time = 7333333 @@ -905,11 +905,11 @@ track 0: data = length 70, hash AF7E4C32 sample 219: time = 7233333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 220: time = 7300000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 221: time = 7466666 @@ -921,11 +921,11 @@ track 0: data = length 70, hash F383DB6C sample 223: time = 7366666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 224: time = 7433333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 225: time = 7600000 @@ -937,11 +937,11 @@ track 0: data = length 70, hash B92866C9 sample 227: time = 7500000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 228: time = 7566666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 229: time = 7733333 @@ -953,11 +953,11 @@ track 0: data = length 70, hash E9E98F05 sample 231: time = 7633333 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 232: time = 7700000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 233: time = 7866666 @@ -969,11 +969,11 @@ track 0: data = length 70, hash 7BAE9641 sample 235: time = 7766666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 236: time = 7833333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 237: time = 8000000 @@ -985,11 +985,11 @@ track 0: data = length 70, hash 5B33157E sample 239: time = 7900000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 240: time = 7966666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 241: time = 8133333 @@ -1001,11 +1001,11 @@ track 0: data = length 70, hash ECF81CBA sample 243: time = 8033333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 244: time = 8100000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 245: time = 8266666 @@ -1017,11 +1017,11 @@ track 0: data = length 70, hash 1DB944F5 sample 247: time = 8166666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 248: time = 8233333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 249: time = 8300000 @@ -1041,11 +1041,11 @@ track 0: data = length 68, hash 83DC1BAB sample 253: time = 8366666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 254: time = 8433333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 255: time = 8600000 @@ -1057,11 +1057,11 @@ track 0: data = length 70, hash 98EFD089 sample 257: time = 8500000 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 258: time = 8566666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 259: time = 8733333 @@ -1073,11 +1073,11 @@ track 0: data = length 70, hash 7BAE9641 sample 261: time = 8633333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 262: time = 8700000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 263: time = 8866666 @@ -1089,11 +1089,11 @@ track 0: data = length 70, hash 5B33157D sample 265: time = 8766666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 266: time = 8833333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 267: time = 9000000 @@ -1105,11 +1105,11 @@ track 0: data = length 70, hash ECF81CB9 sample 269: time = 8900000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 270: time = 8966666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 271: time = 9133333 @@ -1121,11 +1121,11 @@ track 0: data = length 70, hash 1DB944F5 sample 273: time = 9033333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 274: time = 9100000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 275: time = 9266666 @@ -1137,11 +1137,11 @@ track 0: data = length 70, hash AF7E4C31 sample 277: time = 9166666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 278: time = 9233333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 279: time = 9400000 @@ -1153,11 +1153,11 @@ track 0: data = length 70, hash F383DB6D sample 281: time = 9300000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 282: time = 9366666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 283: time = 9533333 @@ -1169,11 +1169,11 @@ track 0: data = length 70, hash B92866C9 sample 285: time = 9433333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 286: time = 9500000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 287: time = 9666666 @@ -1185,11 +1185,11 @@ track 0: data = length 70, hash E9E98F05 sample 289: time = 9566666 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 290: time = 9633333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 291: time = 9800000 @@ -1201,11 +1201,11 @@ track 0: data = length 70, hash 7BAE9641 sample 293: time = 9700000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 294: time = 9766666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 295: time = 9933333 @@ -1217,11 +1217,11 @@ track 0: data = length 70, hash 5B33157D sample 297: time = 9833333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 298: time = 9900000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 299: time = 10066666 @@ -1233,11 +1233,11 @@ track 0: data = length 70, hash ECF81CB9 sample 301: time = 9966666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 302: time = 10033333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 303: time = 10200000 @@ -1249,11 +1249,11 @@ track 0: data = length 70, hash 1DB944F5 sample 305: time = 10100000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 306: time = 10166666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 307: time = 10333333 @@ -1265,11 +1265,11 @@ track 0: data = length 70, hash AF7E4C31 sample 309: time = 10233333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 310: time = 10300000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 311: time = 10466666 @@ -1281,11 +1281,11 @@ track 0: data = length 70, hash F383DB6D sample 313: time = 10366666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 314: time = 10433333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 315: time = 10600000 @@ -1297,11 +1297,11 @@ track 0: data = length 70, hash B92866C9 sample 317: time = 10500000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 318: time = 10566666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 319: time = 10733333 @@ -1313,11 +1313,11 @@ track 0: data = length 70, hash E9E98F05 sample 321: time = 10633333 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 322: time = 10700000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 323: time = 10866666 @@ -1329,11 +1329,11 @@ track 0: data = length 70, hash 7BAE9641 sample 325: time = 10766666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 326: time = 10833333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 327: time = 11000000 @@ -1345,11 +1345,11 @@ track 0: data = length 70, hash 5B33157D sample 329: time = 10900000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 330: time = 10966666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 331: time = 11133333 @@ -1361,11 +1361,11 @@ track 0: data = length 70, hash ECF81CB9 sample 333: time = 11033333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 334: time = 11100000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 335: time = 11266666 @@ -1377,11 +1377,11 @@ track 0: data = length 70, hash 1DB944F5 sample 337: time = 11166666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 338: time = 11233333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 339: time = 11400000 @@ -1393,11 +1393,11 @@ track 0: data = length 70, hash AF7E4C31 sample 341: time = 11300000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 342: time = 11366666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 343: time = 11533333 @@ -1409,11 +1409,11 @@ track 0: data = length 70, hash F383DB6D sample 345: time = 11433333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 346: time = 11500000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 347: time = 11666666 @@ -1425,11 +1425,11 @@ track 0: data = length 70, hash B92866C9 sample 349: time = 11566666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 350: time = 11633333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 351: time = 11800000 @@ -1441,11 +1441,11 @@ track 0: data = length 70, hash E9E98F05 sample 353: time = 11700000 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 354: time = 11766666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 355: time = 11933333 @@ -1457,11 +1457,11 @@ track 0: data = length 70, hash 7BAE9641 sample 357: time = 11833333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 358: time = 11900000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 359: time = 12066666 @@ -1473,11 +1473,11 @@ track 0: data = length 70, hash 5B33157D sample 361: time = 11966666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 362: time = 12033333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 363: time = 12200000 @@ -1489,11 +1489,11 @@ track 0: data = length 70, hash ECF81CB9 sample 365: time = 12100000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 366: time = 12166666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 367: time = 12333333 @@ -1505,11 +1505,11 @@ track 0: data = length 70, hash 1DB944F5 sample 369: time = 12233333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 370: time = 12300000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 371: time = 12466666 @@ -1521,11 +1521,11 @@ track 0: data = length 70, hash AF7E4C31 sample 373: time = 12366666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 374: time = 12433333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 375: time = 12600000 @@ -1537,11 +1537,11 @@ track 0: data = length 70, hash F383DB6D sample 377: time = 12500000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 378: time = 12566666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 379: time = 12733333 @@ -1553,11 +1553,11 @@ track 0: data = length 70, hash B92866C9 sample 381: time = 12633333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 382: time = 12700000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 383: time = 12866666 @@ -1569,11 +1569,11 @@ track 0: data = length 70, hash E9E98F05 sample 385: time = 12766666 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 386: time = 12833333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 387: time = 13000000 @@ -1585,11 +1585,11 @@ track 0: data = length 70, hash 7BAE9641 sample 389: time = 12900000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 390: time = 12966666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 391: time = 13133333 @@ -1601,11 +1601,11 @@ track 0: data = length 70, hash 5B33157D sample 393: time = 13033333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 394: time = 13100000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 395: time = 13266666 @@ -1617,11 +1617,11 @@ track 0: data = length 70, hash ECF81CB9 sample 397: time = 13166666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 398: time = 13233333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 399: time = 13400000 @@ -1633,11 +1633,11 @@ track 0: data = length 70, hash 1DB944F5 sample 401: time = 13300000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 402: time = 13366666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 403: time = 13533333 @@ -1649,11 +1649,11 @@ track 0: data = length 70, hash AF7E4C31 sample 405: time = 13433333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 406: time = 13500000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 407: time = 13666666 @@ -1665,11 +1665,11 @@ track 0: data = length 70, hash F383DB6D sample 409: time = 13566666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 410: time = 13633333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 411: time = 13800000 @@ -1681,11 +1681,11 @@ track 0: data = length 70, hash B92866C9 sample 413: time = 13700000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 414: time = 13766666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 415: time = 13933333 @@ -1697,11 +1697,11 @@ track 0: data = length 70, hash E9E98F05 sample 417: time = 13833333 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 418: time = 13900000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 419: time = 14066666 @@ -1713,11 +1713,11 @@ track 0: data = length 70, hash 7BAE9641 sample 421: time = 13966666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 422: time = 14033333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 423: time = 14200000 @@ -1729,11 +1729,11 @@ track 0: data = length 70, hash 5B33157D sample 425: time = 14100000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 426: time = 14166666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 427: time = 14333333 @@ -1745,11 +1745,11 @@ track 0: data = length 70, hash ECF81CB9 sample 429: time = 14233333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 430: time = 14300000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 431: time = 14466666 @@ -1761,11 +1761,11 @@ track 0: data = length 70, hash 1DB944F5 sample 433: time = 14366666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 434: time = 14433333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 435: time = 14600000 @@ -1777,11 +1777,11 @@ track 0: data = length 70, hash AF7E4C31 sample 437: time = 14500000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 438: time = 14566666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 439: time = 14733333 @@ -1793,11 +1793,11 @@ track 0: data = length 70, hash F383DB6D sample 441: time = 14633333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 442: time = 14700000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 443: time = 14866666 @@ -1809,11 +1809,11 @@ track 0: data = length 70, hash B92866C9 sample 445: time = 14766666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 446: time = 14833333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 447: time = 15000000 @@ -1825,11 +1825,11 @@ track 0: data = length 70, hash E9E98F05 sample 449: time = 14900000 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 450: time = 14966666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 451: time = 15133333 @@ -1841,11 +1841,11 @@ track 0: data = length 70, hash 7BAE9641 sample 453: time = 15033333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 454: time = 15100000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 455: time = 15266666 @@ -1857,11 +1857,11 @@ track 0: data = length 70, hash 5B33157D sample 457: time = 15166666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 458: time = 15233333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 459: time = 15400000 @@ -1873,11 +1873,11 @@ track 0: data = length 70, hash ECF81CB9 sample 461: time = 15300000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 462: time = 15366666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 463: time = 15533333 @@ -1889,11 +1889,11 @@ track 0: data = length 70, hash 1DB944F5 sample 465: time = 15433333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 466: time = 15500000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 467: time = 15666666 @@ -1905,11 +1905,11 @@ track 0: data = length 70, hash AF7E4C31 sample 469: time = 15566666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 470: time = 15633333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 471: time = 15800000 @@ -1921,11 +1921,11 @@ track 0: data = length 70, hash F383DB6D sample 473: time = 15700000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 474: time = 15766666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 475: time = 15933333 @@ -1937,11 +1937,11 @@ track 0: data = length 70, hash B92866C9 sample 477: time = 15833333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 478: time = 15900000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 479: time = 16066666 @@ -1953,11 +1953,11 @@ track 0: data = length 70, hash E9E98F05 sample 481: time = 15966666 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 482: time = 16033333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 483: time = 16200000 @@ -1969,11 +1969,11 @@ track 0: data = length 70, hash 7BAE9641 sample 485: time = 16100000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 486: time = 16166666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 487: time = 16333333 @@ -1985,11 +1985,11 @@ track 0: data = length 70, hash 5B33157D sample 489: time = 16233333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 490: time = 16300000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 491: time = 16466666 @@ -2001,11 +2001,11 @@ track 0: data = length 70, hash ECF81CB9 sample 493: time = 16366666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 494: time = 16433333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 495: time = 16600000 @@ -2017,11 +2017,11 @@ track 0: data = length 70, hash 1DB944F5 sample 497: time = 16500000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 498: time = 16566666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 499: time = 16633333 @@ -2041,11 +2041,11 @@ track 0: data = length 68, hash 83DC1BAA sample 503: time = 16700000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 504: time = 16766666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 505: time = 16933333 @@ -2057,11 +2057,11 @@ track 0: data = length 70, hash 98EFD089 sample 507: time = 16833333 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 508: time = 16900000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 509: time = 17066666 @@ -2073,11 +2073,11 @@ track 0: data = length 70, hash 7BAE9640 sample 511: time = 16966666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 512: time = 17033333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 513: time = 17200000 @@ -2089,11 +2089,11 @@ track 0: data = length 70, hash 5B33157E sample 515: time = 17100000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 516: time = 17166666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 517: time = 17333333 @@ -2105,11 +2105,11 @@ track 0: data = length 70, hash ECF81CBA sample 519: time = 17233333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 520: time = 17300000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 521: time = 17466666 @@ -2121,11 +2121,11 @@ track 0: data = length 70, hash 1DB944F6 sample 523: time = 17366666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 524: time = 17433333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA5 sample 525: time = 17600000 @@ -2137,11 +2137,11 @@ track 0: data = length 70, hash AF7E4C32 sample 527: time = 17500000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 528: time = 17566666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 529: time = 17733333 @@ -2153,11 +2153,11 @@ track 0: data = length 70, hash F383DB6D sample 531: time = 17633333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 532: time = 17700000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DC sample 533: time = 17866666 @@ -2169,11 +2169,11 @@ track 0: data = length 70, hash B92866C8 sample 535: time = 17766666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 536: time = 17833333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 537: time = 18000000 @@ -2185,11 +2185,11 @@ track 0: data = length 70, hash E9E98F05 sample 539: time = 17900000 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 540: time = 17966666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 541: time = 18133333 @@ -2201,11 +2201,11 @@ track 0: data = length 70, hash 7BAE9640 sample 543: time = 18033333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 544: time = 18100000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 545: time = 18266666 @@ -2217,11 +2217,11 @@ track 0: data = length 70, hash 5B33157E sample 547: time = 18166666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 548: time = 18233333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 549: time = 18400000 @@ -2233,11 +2233,11 @@ track 0: data = length 70, hash ECF81CBA sample 551: time = 18300000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 552: time = 18366666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 553: time = 18533333 @@ -2249,11 +2249,11 @@ track 0: data = length 70, hash 1DB944F6 sample 555: time = 18433333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 556: time = 18500000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA5 sample 557: time = 18666666 @@ -2265,11 +2265,11 @@ track 0: data = length 70, hash AF7E4C32 sample 559: time = 18566666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 560: time = 18633333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 561: time = 18800000 @@ -2281,11 +2281,11 @@ track 0: data = length 70, hash F383DB6D sample 563: time = 18700000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 564: time = 18766666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DC sample 565: time = 18933333 @@ -2297,11 +2297,11 @@ track 0: data = length 70, hash B92866C8 sample 567: time = 18833333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 568: time = 18900000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 569: time = 19066666 @@ -2313,11 +2313,11 @@ track 0: data = length 70, hash E9E98F05 sample 571: time = 18966666 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 572: time = 19033333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 573: time = 19200000 @@ -2329,11 +2329,11 @@ track 0: data = length 70, hash 7BAE9640 sample 575: time = 19100000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 576: time = 19166666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 577: time = 19333333 @@ -2345,11 +2345,11 @@ track 0: data = length 70, hash 5B33157E sample 579: time = 19233333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 580: time = 19300000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 581: time = 19466666 @@ -2361,11 +2361,11 @@ track 0: data = length 70, hash ECF81CBA sample 583: time = 19366666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 584: time = 19433333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 585: time = 19600000 @@ -2377,11 +2377,11 @@ track 0: data = length 70, hash 1DB944F6 sample 587: time = 19500000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 588: time = 19566666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA5 sample 589: time = 19733333 @@ -2393,11 +2393,11 @@ track 0: data = length 70, hash AF7E4C32 sample 591: time = 19633333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 592: time = 19700000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 593: time = 19866666 @@ -2409,11 +2409,11 @@ track 0: data = length 70, hash F383DB6D sample 595: time = 19766666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 596: time = 19833333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DC sample 597: time = 19966666 @@ -2425,6 +2425,6 @@ track 0: data = length 70, hash 3AC762E9 sample 599: time = 19933333 - flags = 0 + flags = 67108864 data = length 68, hash B1C7769A tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.2.dump index f6faa08ecb3..75a2c1de75b 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.2.dump @@ -41,11 +41,11 @@ track 0: data = length 68, hash 83DC1BAB sample 3: time = 8366666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 4: time = 8433333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 5: time = 8600000 @@ -57,11 +57,11 @@ track 0: data = length 70, hash 98EFD089 sample 7: time = 8500000 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 8: time = 8566666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 9: time = 8733333 @@ -73,11 +73,11 @@ track 0: data = length 70, hash 7BAE9641 sample 11: time = 8633333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 12: time = 8700000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 13: time = 8866666 @@ -89,11 +89,11 @@ track 0: data = length 70, hash 5B33157D sample 15: time = 8766666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 16: time = 8833333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 17: time = 9000000 @@ -105,11 +105,11 @@ track 0: data = length 70, hash ECF81CB9 sample 19: time = 8900000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 20: time = 8966666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 21: time = 9133333 @@ -121,11 +121,11 @@ track 0: data = length 70, hash 1DB944F5 sample 23: time = 9033333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 24: time = 9100000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 25: time = 9266666 @@ -137,11 +137,11 @@ track 0: data = length 70, hash AF7E4C31 sample 27: time = 9166666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 28: time = 9233333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 29: time = 9400000 @@ -153,11 +153,11 @@ track 0: data = length 70, hash F383DB6D sample 31: time = 9300000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 32: time = 9366666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 33: time = 9533333 @@ -169,11 +169,11 @@ track 0: data = length 70, hash B92866C9 sample 35: time = 9433333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 36: time = 9500000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 37: time = 9666666 @@ -185,11 +185,11 @@ track 0: data = length 70, hash E9E98F05 sample 39: time = 9566666 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 40: time = 9633333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 41: time = 9800000 @@ -201,11 +201,11 @@ track 0: data = length 70, hash 7BAE9641 sample 43: time = 9700000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 44: time = 9766666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 45: time = 9933333 @@ -217,11 +217,11 @@ track 0: data = length 70, hash 5B33157D sample 47: time = 9833333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 48: time = 9900000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 49: time = 10066666 @@ -233,11 +233,11 @@ track 0: data = length 70, hash ECF81CB9 sample 51: time = 9966666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 52: time = 10033333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 53: time = 10200000 @@ -249,11 +249,11 @@ track 0: data = length 70, hash 1DB944F5 sample 55: time = 10100000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 56: time = 10166666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 57: time = 10333333 @@ -265,11 +265,11 @@ track 0: data = length 70, hash AF7E4C31 sample 59: time = 10233333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 60: time = 10300000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 61: time = 10466666 @@ -281,11 +281,11 @@ track 0: data = length 70, hash F383DB6D sample 63: time = 10366666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 64: time = 10433333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 65: time = 10600000 @@ -297,11 +297,11 @@ track 0: data = length 70, hash B92866C9 sample 67: time = 10500000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 68: time = 10566666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 69: time = 10733333 @@ -313,11 +313,11 @@ track 0: data = length 70, hash E9E98F05 sample 71: time = 10633333 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 72: time = 10700000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 73: time = 10866666 @@ -329,11 +329,11 @@ track 0: data = length 70, hash 7BAE9641 sample 75: time = 10766666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 76: time = 10833333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 77: time = 11000000 @@ -345,11 +345,11 @@ track 0: data = length 70, hash 5B33157D sample 79: time = 10900000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 80: time = 10966666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 81: time = 11133333 @@ -361,11 +361,11 @@ track 0: data = length 70, hash ECF81CB9 sample 83: time = 11033333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 84: time = 11100000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 85: time = 11266666 @@ -377,11 +377,11 @@ track 0: data = length 70, hash 1DB944F5 sample 87: time = 11166666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 88: time = 11233333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 89: time = 11400000 @@ -393,11 +393,11 @@ track 0: data = length 70, hash AF7E4C31 sample 91: time = 11300000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 92: time = 11366666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 93: time = 11533333 @@ -409,11 +409,11 @@ track 0: data = length 70, hash F383DB6D sample 95: time = 11433333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 96: time = 11500000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 97: time = 11666666 @@ -425,11 +425,11 @@ track 0: data = length 70, hash B92866C9 sample 99: time = 11566666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 100: time = 11633333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 101: time = 11800000 @@ -441,11 +441,11 @@ track 0: data = length 70, hash E9E98F05 sample 103: time = 11700000 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 104: time = 11766666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 105: time = 11933333 @@ -457,11 +457,11 @@ track 0: data = length 70, hash 7BAE9641 sample 107: time = 11833333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 108: time = 11900000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 109: time = 12066666 @@ -473,11 +473,11 @@ track 0: data = length 70, hash 5B33157D sample 111: time = 11966666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 112: time = 12033333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 113: time = 12200000 @@ -489,11 +489,11 @@ track 0: data = length 70, hash ECF81CB9 sample 115: time = 12100000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 116: time = 12166666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 117: time = 12333333 @@ -505,11 +505,11 @@ track 0: data = length 70, hash 1DB944F5 sample 119: time = 12233333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 120: time = 12300000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 121: time = 12466666 @@ -521,11 +521,11 @@ track 0: data = length 70, hash AF7E4C31 sample 123: time = 12366666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 124: time = 12433333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 125: time = 12600000 @@ -537,11 +537,11 @@ track 0: data = length 70, hash F383DB6D sample 127: time = 12500000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 128: time = 12566666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 129: time = 12733333 @@ -553,11 +553,11 @@ track 0: data = length 70, hash B92866C9 sample 131: time = 12633333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 132: time = 12700000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 133: time = 12866666 @@ -569,11 +569,11 @@ track 0: data = length 70, hash E9E98F05 sample 135: time = 12766666 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 136: time = 12833333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 137: time = 13000000 @@ -585,11 +585,11 @@ track 0: data = length 70, hash 7BAE9641 sample 139: time = 12900000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 140: time = 12966666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 141: time = 13133333 @@ -601,11 +601,11 @@ track 0: data = length 70, hash 5B33157D sample 143: time = 13033333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 144: time = 13100000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 145: time = 13266666 @@ -617,11 +617,11 @@ track 0: data = length 70, hash ECF81CB9 sample 147: time = 13166666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 148: time = 13233333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 149: time = 13400000 @@ -633,11 +633,11 @@ track 0: data = length 70, hash 1DB944F5 sample 151: time = 13300000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 152: time = 13366666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 153: time = 13533333 @@ -649,11 +649,11 @@ track 0: data = length 70, hash AF7E4C31 sample 155: time = 13433333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 156: time = 13500000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 157: time = 13666666 @@ -665,11 +665,11 @@ track 0: data = length 70, hash F383DB6D sample 159: time = 13566666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 160: time = 13633333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 161: time = 13800000 @@ -681,11 +681,11 @@ track 0: data = length 70, hash B92866C9 sample 163: time = 13700000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 164: time = 13766666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 165: time = 13933333 @@ -697,11 +697,11 @@ track 0: data = length 70, hash E9E98F05 sample 167: time = 13833333 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 168: time = 13900000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 169: time = 14066666 @@ -713,11 +713,11 @@ track 0: data = length 70, hash 7BAE9641 sample 171: time = 13966666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 172: time = 14033333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 173: time = 14200000 @@ -729,11 +729,11 @@ track 0: data = length 70, hash 5B33157D sample 175: time = 14100000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 176: time = 14166666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 177: time = 14333333 @@ -745,11 +745,11 @@ track 0: data = length 70, hash ECF81CB9 sample 179: time = 14233333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 180: time = 14300000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 181: time = 14466666 @@ -761,11 +761,11 @@ track 0: data = length 70, hash 1DB944F5 sample 183: time = 14366666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 184: time = 14433333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 185: time = 14600000 @@ -777,11 +777,11 @@ track 0: data = length 70, hash AF7E4C31 sample 187: time = 14500000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 188: time = 14566666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 189: time = 14733333 @@ -793,11 +793,11 @@ track 0: data = length 70, hash F383DB6D sample 191: time = 14633333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 192: time = 14700000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 193: time = 14866666 @@ -809,11 +809,11 @@ track 0: data = length 70, hash B92866C9 sample 195: time = 14766666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 196: time = 14833333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 197: time = 15000000 @@ -825,11 +825,11 @@ track 0: data = length 70, hash E9E98F05 sample 199: time = 14900000 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 200: time = 14966666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 201: time = 15133333 @@ -841,11 +841,11 @@ track 0: data = length 70, hash 7BAE9641 sample 203: time = 15033333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 204: time = 15100000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 205: time = 15266666 @@ -857,11 +857,11 @@ track 0: data = length 70, hash 5B33157D sample 207: time = 15166666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 208: time = 15233333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 209: time = 15400000 @@ -873,11 +873,11 @@ track 0: data = length 70, hash ECF81CB9 sample 211: time = 15300000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 212: time = 15366666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 213: time = 15533333 @@ -889,11 +889,11 @@ track 0: data = length 70, hash 1DB944F5 sample 215: time = 15433333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 216: time = 15500000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 217: time = 15666666 @@ -905,11 +905,11 @@ track 0: data = length 70, hash AF7E4C31 sample 219: time = 15566666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 220: time = 15633333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 221: time = 15800000 @@ -921,11 +921,11 @@ track 0: data = length 70, hash F383DB6D sample 223: time = 15700000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 224: time = 15766666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 225: time = 15933333 @@ -937,11 +937,11 @@ track 0: data = length 70, hash B92866C9 sample 227: time = 15833333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 228: time = 15900000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 229: time = 16066666 @@ -953,11 +953,11 @@ track 0: data = length 70, hash E9E98F05 sample 231: time = 15966666 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 232: time = 16033333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 233: time = 16200000 @@ -969,11 +969,11 @@ track 0: data = length 70, hash 7BAE9641 sample 235: time = 16100000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 236: time = 16166666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 237: time = 16333333 @@ -985,11 +985,11 @@ track 0: data = length 70, hash 5B33157D sample 239: time = 16233333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 240: time = 16300000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 241: time = 16466666 @@ -1001,11 +1001,11 @@ track 0: data = length 70, hash ECF81CB9 sample 243: time = 16366666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 244: time = 16433333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 245: time = 16600000 @@ -1017,11 +1017,11 @@ track 0: data = length 70, hash 1DB944F5 sample 247: time = 16500000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 248: time = 16566666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 249: time = 16633333 @@ -1041,11 +1041,11 @@ track 0: data = length 68, hash 83DC1BAA sample 253: time = 16700000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 254: time = 16766666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 255: time = 16933333 @@ -1057,11 +1057,11 @@ track 0: data = length 70, hash 98EFD089 sample 257: time = 16833333 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 258: time = 16900000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 259: time = 17066666 @@ -1073,11 +1073,11 @@ track 0: data = length 70, hash 7BAE9640 sample 261: time = 16966666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 262: time = 17033333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 263: time = 17200000 @@ -1089,11 +1089,11 @@ track 0: data = length 70, hash 5B33157E sample 265: time = 17100000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 266: time = 17166666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 267: time = 17333333 @@ -1105,11 +1105,11 @@ track 0: data = length 70, hash ECF81CBA sample 269: time = 17233333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 270: time = 17300000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 271: time = 17466666 @@ -1121,11 +1121,11 @@ track 0: data = length 70, hash 1DB944F6 sample 273: time = 17366666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 274: time = 17433333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA5 sample 275: time = 17600000 @@ -1137,11 +1137,11 @@ track 0: data = length 70, hash AF7E4C32 sample 277: time = 17500000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 278: time = 17566666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 279: time = 17733333 @@ -1153,11 +1153,11 @@ track 0: data = length 70, hash F383DB6D sample 281: time = 17633333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 282: time = 17700000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DC sample 283: time = 17866666 @@ -1169,11 +1169,11 @@ track 0: data = length 70, hash B92866C8 sample 285: time = 17766666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 286: time = 17833333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 287: time = 18000000 @@ -1185,11 +1185,11 @@ track 0: data = length 70, hash E9E98F05 sample 289: time = 17900000 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 290: time = 17966666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 291: time = 18133333 @@ -1201,11 +1201,11 @@ track 0: data = length 70, hash 7BAE9640 sample 293: time = 18033333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 294: time = 18100000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 295: time = 18266666 @@ -1217,11 +1217,11 @@ track 0: data = length 70, hash 5B33157E sample 297: time = 18166666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 298: time = 18233333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 299: time = 18400000 @@ -1233,11 +1233,11 @@ track 0: data = length 70, hash ECF81CBA sample 301: time = 18300000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 302: time = 18366666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 303: time = 18533333 @@ -1249,11 +1249,11 @@ track 0: data = length 70, hash 1DB944F6 sample 305: time = 18433333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 306: time = 18500000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA5 sample 307: time = 18666666 @@ -1265,11 +1265,11 @@ track 0: data = length 70, hash AF7E4C32 sample 309: time = 18566666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 310: time = 18633333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 311: time = 18800000 @@ -1281,11 +1281,11 @@ track 0: data = length 70, hash F383DB6D sample 313: time = 18700000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 314: time = 18766666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DC sample 315: time = 18933333 @@ -1297,11 +1297,11 @@ track 0: data = length 70, hash B92866C8 sample 317: time = 18833333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 318: time = 18900000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 319: time = 19066666 @@ -1313,11 +1313,11 @@ track 0: data = length 70, hash E9E98F05 sample 321: time = 18966666 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 322: time = 19033333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 323: time = 19200000 @@ -1329,11 +1329,11 @@ track 0: data = length 70, hash 7BAE9640 sample 325: time = 19100000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 326: time = 19166666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 327: time = 19333333 @@ -1345,11 +1345,11 @@ track 0: data = length 70, hash 5B33157E sample 329: time = 19233333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 330: time = 19300000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 331: time = 19466666 @@ -1361,11 +1361,11 @@ track 0: data = length 70, hash ECF81CBA sample 333: time = 19366666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 334: time = 19433333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 335: time = 19600000 @@ -1377,11 +1377,11 @@ track 0: data = length 70, hash 1DB944F6 sample 337: time = 19500000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 338: time = 19566666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA5 sample 339: time = 19733333 @@ -1393,11 +1393,11 @@ track 0: data = length 70, hash AF7E4C32 sample 341: time = 19633333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 342: time = 19700000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 343: time = 19866666 @@ -1409,11 +1409,11 @@ track 0: data = length 70, hash F383DB6D sample 345: time = 19766666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 346: time = 19833333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DC sample 347: time = 19966666 @@ -1425,6 +1425,6 @@ track 0: data = length 70, hash 3AC762E9 sample 349: time = 19933333 - flags = 0 + flags = 67108864 data = length 68, hash B1C7769A tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.3.dump index d47b135412c..501982849ba 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.3.dump @@ -41,11 +41,11 @@ track 0: data = length 68, hash 83DC1BAA sample 3: time = 16700000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 4: time = 16766666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 5: time = 16933333 @@ -57,11 +57,11 @@ track 0: data = length 70, hash 98EFD089 sample 7: time = 16833333 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 8: time = 16900000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 9: time = 17066666 @@ -73,11 +73,11 @@ track 0: data = length 70, hash 7BAE9640 sample 11: time = 16966666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 12: time = 17033333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 13: time = 17200000 @@ -89,11 +89,11 @@ track 0: data = length 70, hash 5B33157E sample 15: time = 17100000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 16: time = 17166666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 17: time = 17333333 @@ -105,11 +105,11 @@ track 0: data = length 70, hash ECF81CBA sample 19: time = 17233333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 20: time = 17300000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 21: time = 17466666 @@ -121,11 +121,11 @@ track 0: data = length 70, hash 1DB944F6 sample 23: time = 17366666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 24: time = 17433333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA5 sample 25: time = 17600000 @@ -137,11 +137,11 @@ track 0: data = length 70, hash AF7E4C32 sample 27: time = 17500000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 28: time = 17566666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 29: time = 17733333 @@ -153,11 +153,11 @@ track 0: data = length 70, hash F383DB6D sample 31: time = 17633333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 32: time = 17700000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DC sample 33: time = 17866666 @@ -169,11 +169,11 @@ track 0: data = length 70, hash B92866C8 sample 35: time = 17766666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 36: time = 17833333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 37: time = 18000000 @@ -185,11 +185,11 @@ track 0: data = length 70, hash E9E98F05 sample 39: time = 17900000 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 40: time = 17966666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 41: time = 18133333 @@ -201,11 +201,11 @@ track 0: data = length 70, hash 7BAE9640 sample 43: time = 18033333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 44: time = 18100000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 45: time = 18266666 @@ -217,11 +217,11 @@ track 0: data = length 70, hash 5B33157E sample 47: time = 18166666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 48: time = 18233333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 49: time = 18400000 @@ -233,11 +233,11 @@ track 0: data = length 70, hash ECF81CBA sample 51: time = 18300000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 52: time = 18366666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 53: time = 18533333 @@ -249,11 +249,11 @@ track 0: data = length 70, hash 1DB944F6 sample 55: time = 18433333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 56: time = 18500000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA5 sample 57: time = 18666666 @@ -265,11 +265,11 @@ track 0: data = length 70, hash AF7E4C32 sample 59: time = 18566666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 60: time = 18633333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 61: time = 18800000 @@ -281,11 +281,11 @@ track 0: data = length 70, hash F383DB6D sample 63: time = 18700000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 64: time = 18766666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DC sample 65: time = 18933333 @@ -297,11 +297,11 @@ track 0: data = length 70, hash B92866C8 sample 67: time = 18833333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 68: time = 18900000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 69: time = 19066666 @@ -313,11 +313,11 @@ track 0: data = length 70, hash E9E98F05 sample 71: time = 18966666 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 72: time = 19033333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 73: time = 19200000 @@ -329,11 +329,11 @@ track 0: data = length 70, hash 7BAE9640 sample 75: time = 19100000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 76: time = 19166666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 77: time = 19333333 @@ -345,11 +345,11 @@ track 0: data = length 70, hash 5B33157E sample 79: time = 19233333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 80: time = 19300000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 81: time = 19466666 @@ -361,11 +361,11 @@ track 0: data = length 70, hash ECF81CBA sample 83: time = 19366666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 84: time = 19433333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 85: time = 19600000 @@ -377,11 +377,11 @@ track 0: data = length 70, hash 1DB944F6 sample 87: time = 19500000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 88: time = 19566666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA5 sample 89: time = 19733333 @@ -393,11 +393,11 @@ track 0: data = length 70, hash AF7E4C32 sample 91: time = 19633333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 92: time = 19700000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 93: time = 19866666 @@ -409,11 +409,11 @@ track 0: data = length 70, hash F383DB6D sample 95: time = 19766666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 96: time = 19833333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DC sample 97: time = 19966666 @@ -425,6 +425,6 @@ track 0: data = length 70, hash 3AC762E9 sample 99: time = 19933333 - flags = 0 + flags = 67108864 data = length 68, hash B1C7769A tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.unknown_length.dump index f97a5b458b4..41e3efc65af 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_seekable_multiple_sidx.mp4.unknown_length.dump @@ -41,11 +41,11 @@ track 0: data = length 68, hash 83DC1BAB sample 3: time = 33333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 4: time = 100000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 5: time = 266666 @@ -57,11 +57,11 @@ track 0: data = length 70, hash 98EFD089 sample 7: time = 166666 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 8: time = 233333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 9: time = 400000 @@ -73,11 +73,11 @@ track 0: data = length 70, hash 7BAE9641 sample 11: time = 300000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 12: time = 366666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 13: time = 533333 @@ -89,11 +89,11 @@ track 0: data = length 70, hash 5B33157E sample 15: time = 433333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 16: time = 500000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 17: time = 666666 @@ -105,11 +105,11 @@ track 0: data = length 70, hash ECF81CBA sample 19: time = 566666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 20: time = 633333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 21: time = 800000 @@ -121,11 +121,11 @@ track 0: data = length 70, hash 1DB944F5 sample 23: time = 700000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 24: time = 766666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 25: time = 933333 @@ -137,11 +137,11 @@ track 0: data = length 70, hash AF7E4C32 sample 27: time = 833333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 28: time = 900000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 29: time = 1066666 @@ -153,11 +153,11 @@ track 0: data = length 70, hash F383DB6C sample 31: time = 966666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 32: time = 1033333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 33: time = 1200000 @@ -169,11 +169,11 @@ track 0: data = length 70, hash B92866C9 sample 35: time = 1100000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 36: time = 1166666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 37: time = 1333333 @@ -185,11 +185,11 @@ track 0: data = length 70, hash E9E98F05 sample 39: time = 1233333 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 40: time = 1300000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 41: time = 1466666 @@ -201,11 +201,11 @@ track 0: data = length 70, hash 7BAE9641 sample 43: time = 1366666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 44: time = 1433333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 45: time = 1600000 @@ -217,11 +217,11 @@ track 0: data = length 70, hash 5B33157E sample 47: time = 1500000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 48: time = 1566666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 49: time = 1733333 @@ -233,11 +233,11 @@ track 0: data = length 70, hash ECF81CBA sample 51: time = 1633333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 52: time = 1700000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 53: time = 1866666 @@ -249,11 +249,11 @@ track 0: data = length 70, hash 1DB944F5 sample 55: time = 1766666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 56: time = 1833333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 57: time = 2000000 @@ -265,11 +265,11 @@ track 0: data = length 70, hash AF7E4C32 sample 59: time = 1900000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 60: time = 1966666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 61: time = 2133333 @@ -281,11 +281,11 @@ track 0: data = length 70, hash F383DB6C sample 63: time = 2033333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 64: time = 2100000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 65: time = 2266666 @@ -297,11 +297,11 @@ track 0: data = length 70, hash B92866C9 sample 67: time = 2166666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 68: time = 2233333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 69: time = 2400000 @@ -313,11 +313,11 @@ track 0: data = length 70, hash E9E98F05 sample 71: time = 2300000 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 72: time = 2366666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 73: time = 2533333 @@ -329,11 +329,11 @@ track 0: data = length 70, hash 7BAE9641 sample 75: time = 2433333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 76: time = 2500000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 77: time = 2666666 @@ -345,11 +345,11 @@ track 0: data = length 70, hash 5B33157E sample 79: time = 2566666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 80: time = 2633333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 81: time = 2800000 @@ -361,11 +361,11 @@ track 0: data = length 70, hash ECF81CBA sample 83: time = 2700000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 84: time = 2766666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 85: time = 2933333 @@ -377,11 +377,11 @@ track 0: data = length 70, hash 1DB944F5 sample 87: time = 2833333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 88: time = 2900000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 89: time = 3066666 @@ -393,11 +393,11 @@ track 0: data = length 70, hash AF7E4C32 sample 91: time = 2966666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 92: time = 3033333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 93: time = 3200000 @@ -409,11 +409,11 @@ track 0: data = length 70, hash F383DB6C sample 95: time = 3100000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 96: time = 3166666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 97: time = 3333333 @@ -425,11 +425,11 @@ track 0: data = length 70, hash B92866C9 sample 99: time = 3233333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 100: time = 3300000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 101: time = 3466666 @@ -441,11 +441,11 @@ track 0: data = length 70, hash E9E98F05 sample 103: time = 3366666 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 104: time = 3433333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 105: time = 3600000 @@ -457,11 +457,11 @@ track 0: data = length 70, hash 7BAE9641 sample 107: time = 3500000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 108: time = 3566666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 109: time = 3733333 @@ -473,11 +473,11 @@ track 0: data = length 70, hash 5B33157E sample 111: time = 3633333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 112: time = 3700000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 113: time = 3866666 @@ -489,11 +489,11 @@ track 0: data = length 70, hash ECF81CBA sample 115: time = 3766666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 116: time = 3833333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 117: time = 4000000 @@ -505,11 +505,11 @@ track 0: data = length 70, hash 1DB944F5 sample 119: time = 3900000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 120: time = 3966666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 121: time = 4133333 @@ -521,11 +521,11 @@ track 0: data = length 70, hash AF7E4C32 sample 123: time = 4033333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 124: time = 4100000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 125: time = 4266666 @@ -537,11 +537,11 @@ track 0: data = length 70, hash F383DB6C sample 127: time = 4166666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 128: time = 4233333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 129: time = 4400000 @@ -553,11 +553,11 @@ track 0: data = length 70, hash B92866C9 sample 131: time = 4300000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 132: time = 4366666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 133: time = 4533333 @@ -569,11 +569,11 @@ track 0: data = length 70, hash E9E98F05 sample 135: time = 4433333 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 136: time = 4500000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 137: time = 4666666 @@ -585,11 +585,11 @@ track 0: data = length 70, hash 7BAE9641 sample 139: time = 4566666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 140: time = 4633333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 141: time = 4800000 @@ -601,11 +601,11 @@ track 0: data = length 70, hash 5B33157E sample 143: time = 4700000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 144: time = 4766666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 145: time = 4933333 @@ -617,11 +617,11 @@ track 0: data = length 70, hash ECF81CBA sample 147: time = 4833333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 148: time = 4900000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 149: time = 5066666 @@ -633,11 +633,11 @@ track 0: data = length 70, hash 1DB944F5 sample 151: time = 4966666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 152: time = 5033333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 153: time = 5200000 @@ -649,11 +649,11 @@ track 0: data = length 70, hash AF7E4C32 sample 155: time = 5100000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 156: time = 5166666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 157: time = 5333333 @@ -665,11 +665,11 @@ track 0: data = length 70, hash F383DB6C sample 159: time = 5233333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 160: time = 5300000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 161: time = 5466666 @@ -681,11 +681,11 @@ track 0: data = length 70, hash B92866C9 sample 163: time = 5366666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 164: time = 5433333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 165: time = 5600000 @@ -697,11 +697,11 @@ track 0: data = length 70, hash E9E98F05 sample 167: time = 5500000 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 168: time = 5566666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 169: time = 5733333 @@ -713,11 +713,11 @@ track 0: data = length 70, hash 7BAE9641 sample 171: time = 5633333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 172: time = 5700000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 173: time = 5866666 @@ -729,11 +729,11 @@ track 0: data = length 70, hash 5B33157E sample 175: time = 5766666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 176: time = 5833333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 177: time = 6000000 @@ -745,11 +745,11 @@ track 0: data = length 70, hash ECF81CBA sample 179: time = 5900000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 180: time = 5966666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 181: time = 6133333 @@ -761,11 +761,11 @@ track 0: data = length 70, hash 1DB944F5 sample 183: time = 6033333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 184: time = 6100000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 185: time = 6266666 @@ -777,11 +777,11 @@ track 0: data = length 70, hash AF7E4C32 sample 187: time = 6166666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 188: time = 6233333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 189: time = 6400000 @@ -793,11 +793,11 @@ track 0: data = length 70, hash F383DB6C sample 191: time = 6300000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 192: time = 6366666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 193: time = 6533333 @@ -809,11 +809,11 @@ track 0: data = length 70, hash B92866C9 sample 195: time = 6433333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 196: time = 6500000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 197: time = 6666666 @@ -825,11 +825,11 @@ track 0: data = length 70, hash E9E98F05 sample 199: time = 6566666 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 200: time = 6633333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 201: time = 6800000 @@ -841,11 +841,11 @@ track 0: data = length 70, hash 7BAE9641 sample 203: time = 6700000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 204: time = 6766666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 205: time = 6933333 @@ -857,11 +857,11 @@ track 0: data = length 70, hash 5B33157E sample 207: time = 6833333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 208: time = 6900000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 209: time = 7066666 @@ -873,11 +873,11 @@ track 0: data = length 70, hash ECF81CBA sample 211: time = 6966666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 212: time = 7033333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 213: time = 7200000 @@ -889,11 +889,11 @@ track 0: data = length 70, hash 1DB944F5 sample 215: time = 7100000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 216: time = 7166666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 217: time = 7333333 @@ -905,11 +905,11 @@ track 0: data = length 70, hash AF7E4C32 sample 219: time = 7233333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AD sample 220: time = 7300000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E2 sample 221: time = 7466666 @@ -921,11 +921,11 @@ track 0: data = length 70, hash F383DB6C sample 223: time = 7366666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 224: time = 7433333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 225: time = 7600000 @@ -937,11 +937,11 @@ track 0: data = length 70, hash B92866C9 sample 227: time = 7500000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C04 sample 228: time = 7566666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 229: time = 7733333 @@ -953,11 +953,11 @@ track 0: data = length 70, hash E9E98F05 sample 231: time = 7633333 - flags = 0 + flags = 67108864 data = length 68, hash 88475341 sample 232: time = 7700000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED74 sample 233: time = 7866666 @@ -969,11 +969,11 @@ track 0: data = length 70, hash 7BAE9641 sample 235: time = 7766666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 236: time = 7833333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B0 sample 237: time = 8000000 @@ -985,11 +985,11 @@ track 0: data = length 70, hash 5B33157E sample 239: time = 7900000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223FA sample 240: time = 7966666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2D sample 241: time = 8133333 @@ -1001,11 +1001,11 @@ track 0: data = length 70, hash ECF81CBA sample 243: time = 8033333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA35 sample 244: time = 8100000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 245: time = 8266666 @@ -1017,11 +1017,11 @@ track 0: data = length 70, hash 1DB944F5 sample 247: time = 8166666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD172 sample 248: time = 8233333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 249: time = 8300000 @@ -1041,11 +1041,11 @@ track 0: data = length 68, hash 83DC1BAB sample 253: time = 8366666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 254: time = 8433333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 255: time = 8600000 @@ -1057,11 +1057,11 @@ track 0: data = length 70, hash 98EFD089 sample 257: time = 8500000 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 258: time = 8566666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 259: time = 8733333 @@ -1073,11 +1073,11 @@ track 0: data = length 70, hash 7BAE9641 sample 261: time = 8633333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 262: time = 8700000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 263: time = 8866666 @@ -1089,11 +1089,11 @@ track 0: data = length 70, hash 5B33157D sample 265: time = 8766666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 266: time = 8833333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 267: time = 9000000 @@ -1105,11 +1105,11 @@ track 0: data = length 70, hash ECF81CB9 sample 269: time = 8900000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 270: time = 8966666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 271: time = 9133333 @@ -1121,11 +1121,11 @@ track 0: data = length 70, hash 1DB944F5 sample 273: time = 9033333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 274: time = 9100000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 275: time = 9266666 @@ -1137,11 +1137,11 @@ track 0: data = length 70, hash AF7E4C31 sample 277: time = 9166666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 278: time = 9233333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 279: time = 9400000 @@ -1153,11 +1153,11 @@ track 0: data = length 70, hash F383DB6D sample 281: time = 9300000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 282: time = 9366666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 283: time = 9533333 @@ -1169,11 +1169,11 @@ track 0: data = length 70, hash B92866C9 sample 285: time = 9433333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 286: time = 9500000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 287: time = 9666666 @@ -1185,11 +1185,11 @@ track 0: data = length 70, hash E9E98F05 sample 289: time = 9566666 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 290: time = 9633333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 291: time = 9800000 @@ -1201,11 +1201,11 @@ track 0: data = length 70, hash 7BAE9641 sample 293: time = 9700000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 294: time = 9766666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 295: time = 9933333 @@ -1217,11 +1217,11 @@ track 0: data = length 70, hash 5B33157D sample 297: time = 9833333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 298: time = 9900000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 299: time = 10066666 @@ -1233,11 +1233,11 @@ track 0: data = length 70, hash ECF81CB9 sample 301: time = 9966666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 302: time = 10033333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 303: time = 10200000 @@ -1249,11 +1249,11 @@ track 0: data = length 70, hash 1DB944F5 sample 305: time = 10100000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 306: time = 10166666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 307: time = 10333333 @@ -1265,11 +1265,11 @@ track 0: data = length 70, hash AF7E4C31 sample 309: time = 10233333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 310: time = 10300000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 311: time = 10466666 @@ -1281,11 +1281,11 @@ track 0: data = length 70, hash F383DB6D sample 313: time = 10366666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 314: time = 10433333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 315: time = 10600000 @@ -1297,11 +1297,11 @@ track 0: data = length 70, hash B92866C9 sample 317: time = 10500000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 318: time = 10566666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 319: time = 10733333 @@ -1313,11 +1313,11 @@ track 0: data = length 70, hash E9E98F05 sample 321: time = 10633333 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 322: time = 10700000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 323: time = 10866666 @@ -1329,11 +1329,11 @@ track 0: data = length 70, hash 7BAE9641 sample 325: time = 10766666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 326: time = 10833333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 327: time = 11000000 @@ -1345,11 +1345,11 @@ track 0: data = length 70, hash 5B33157D sample 329: time = 10900000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 330: time = 10966666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 331: time = 11133333 @@ -1361,11 +1361,11 @@ track 0: data = length 70, hash ECF81CB9 sample 333: time = 11033333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 334: time = 11100000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 335: time = 11266666 @@ -1377,11 +1377,11 @@ track 0: data = length 70, hash 1DB944F5 sample 337: time = 11166666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 338: time = 11233333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 339: time = 11400000 @@ -1393,11 +1393,11 @@ track 0: data = length 70, hash AF7E4C31 sample 341: time = 11300000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 342: time = 11366666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 343: time = 11533333 @@ -1409,11 +1409,11 @@ track 0: data = length 70, hash F383DB6D sample 345: time = 11433333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 346: time = 11500000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 347: time = 11666666 @@ -1425,11 +1425,11 @@ track 0: data = length 70, hash B92866C9 sample 349: time = 11566666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 350: time = 11633333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 351: time = 11800000 @@ -1441,11 +1441,11 @@ track 0: data = length 70, hash E9E98F05 sample 353: time = 11700000 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 354: time = 11766666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 355: time = 11933333 @@ -1457,11 +1457,11 @@ track 0: data = length 70, hash 7BAE9641 sample 357: time = 11833333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 358: time = 11900000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 359: time = 12066666 @@ -1473,11 +1473,11 @@ track 0: data = length 70, hash 5B33157D sample 361: time = 11966666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 362: time = 12033333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 363: time = 12200000 @@ -1489,11 +1489,11 @@ track 0: data = length 70, hash ECF81CB9 sample 365: time = 12100000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 366: time = 12166666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 367: time = 12333333 @@ -1505,11 +1505,11 @@ track 0: data = length 70, hash 1DB944F5 sample 369: time = 12233333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 370: time = 12300000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 371: time = 12466666 @@ -1521,11 +1521,11 @@ track 0: data = length 70, hash AF7E4C31 sample 373: time = 12366666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 374: time = 12433333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 375: time = 12600000 @@ -1537,11 +1537,11 @@ track 0: data = length 70, hash F383DB6D sample 377: time = 12500000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 378: time = 12566666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 379: time = 12733333 @@ -1553,11 +1553,11 @@ track 0: data = length 70, hash B92866C9 sample 381: time = 12633333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 382: time = 12700000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 383: time = 12866666 @@ -1569,11 +1569,11 @@ track 0: data = length 70, hash E9E98F05 sample 385: time = 12766666 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 386: time = 12833333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 387: time = 13000000 @@ -1585,11 +1585,11 @@ track 0: data = length 70, hash 7BAE9641 sample 389: time = 12900000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 390: time = 12966666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 391: time = 13133333 @@ -1601,11 +1601,11 @@ track 0: data = length 70, hash 5B33157D sample 393: time = 13033333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 394: time = 13100000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 395: time = 13266666 @@ -1617,11 +1617,11 @@ track 0: data = length 70, hash ECF81CB9 sample 397: time = 13166666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 398: time = 13233333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 399: time = 13400000 @@ -1633,11 +1633,11 @@ track 0: data = length 70, hash 1DB944F5 sample 401: time = 13300000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 402: time = 13366666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 403: time = 13533333 @@ -1649,11 +1649,11 @@ track 0: data = length 70, hash AF7E4C31 sample 405: time = 13433333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 406: time = 13500000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 407: time = 13666666 @@ -1665,11 +1665,11 @@ track 0: data = length 70, hash F383DB6D sample 409: time = 13566666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 410: time = 13633333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 411: time = 13800000 @@ -1681,11 +1681,11 @@ track 0: data = length 70, hash B92866C9 sample 413: time = 13700000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 414: time = 13766666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 415: time = 13933333 @@ -1697,11 +1697,11 @@ track 0: data = length 70, hash E9E98F05 sample 417: time = 13833333 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 418: time = 13900000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 419: time = 14066666 @@ -1713,11 +1713,11 @@ track 0: data = length 70, hash 7BAE9641 sample 421: time = 13966666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 422: time = 14033333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 423: time = 14200000 @@ -1729,11 +1729,11 @@ track 0: data = length 70, hash 5B33157D sample 425: time = 14100000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 426: time = 14166666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 427: time = 14333333 @@ -1745,11 +1745,11 @@ track 0: data = length 70, hash ECF81CB9 sample 429: time = 14233333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 430: time = 14300000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 431: time = 14466666 @@ -1761,11 +1761,11 @@ track 0: data = length 70, hash 1DB944F5 sample 433: time = 14366666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 434: time = 14433333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 435: time = 14600000 @@ -1777,11 +1777,11 @@ track 0: data = length 70, hash AF7E4C31 sample 437: time = 14500000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 438: time = 14566666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 439: time = 14733333 @@ -1793,11 +1793,11 @@ track 0: data = length 70, hash F383DB6D sample 441: time = 14633333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 442: time = 14700000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 443: time = 14866666 @@ -1809,11 +1809,11 @@ track 0: data = length 70, hash B92866C9 sample 445: time = 14766666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 446: time = 14833333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 447: time = 15000000 @@ -1825,11 +1825,11 @@ track 0: data = length 70, hash E9E98F05 sample 449: time = 14900000 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 450: time = 14966666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 451: time = 15133333 @@ -1841,11 +1841,11 @@ track 0: data = length 70, hash 7BAE9641 sample 453: time = 15033333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 454: time = 15100000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 455: time = 15266666 @@ -1857,11 +1857,11 @@ track 0: data = length 70, hash 5B33157D sample 457: time = 15166666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 458: time = 15233333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 459: time = 15400000 @@ -1873,11 +1873,11 @@ track 0: data = length 70, hash ECF81CB9 sample 461: time = 15300000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 462: time = 15366666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 463: time = 15533333 @@ -1889,11 +1889,11 @@ track 0: data = length 70, hash 1DB944F5 sample 465: time = 15433333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 466: time = 15500000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 467: time = 15666666 @@ -1905,11 +1905,11 @@ track 0: data = length 70, hash AF7E4C31 sample 469: time = 15566666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 470: time = 15633333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 471: time = 15800000 @@ -1921,11 +1921,11 @@ track 0: data = length 70, hash F383DB6D sample 473: time = 15700000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 474: time = 15766666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DD sample 475: time = 15933333 @@ -1937,11 +1937,11 @@ track 0: data = length 70, hash B92866C9 sample 477: time = 15833333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 478: time = 15900000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E638 sample 479: time = 16066666 @@ -1953,11 +1953,11 @@ track 0: data = length 70, hash E9E98F05 sample 481: time = 15966666 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 482: time = 16033333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 483: time = 16200000 @@ -1969,11 +1969,11 @@ track 0: data = length 70, hash 7BAE9641 sample 485: time = 16100000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97D sample 486: time = 16166666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 487: time = 16333333 @@ -1985,11 +1985,11 @@ track 0: data = length 70, hash 5B33157D sample 489: time = 16233333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 490: time = 16300000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 491: time = 16466666 @@ -2001,11 +2001,11 @@ track 0: data = length 70, hash ECF81CB9 sample 493: time = 16366666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 494: time = 16433333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C646A sample 495: time = 16600000 @@ -2017,11 +2017,11 @@ track 0: data = length 70, hash 1DB944F5 sample 497: time = 16500000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 498: time = 16566666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA6 sample 499: time = 16633333 @@ -2041,11 +2041,11 @@ track 0: data = length 68, hash 83DC1BAA sample 503: time = 16700000 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 504: time = 16766666 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 505: time = 16933333 @@ -2057,11 +2057,11 @@ track 0: data = length 70, hash 98EFD089 sample 507: time = 16833333 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 508: time = 16900000 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 509: time = 17066666 @@ -2073,11 +2073,11 @@ track 0: data = length 70, hash 7BAE9640 sample 511: time = 16966666 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 512: time = 17033333 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 513: time = 17200000 @@ -2089,11 +2089,11 @@ track 0: data = length 70, hash 5B33157E sample 515: time = 17100000 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 516: time = 17166666 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 517: time = 17333333 @@ -2105,11 +2105,11 @@ track 0: data = length 70, hash ECF81CBA sample 519: time = 17233333 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 520: time = 17300000 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 521: time = 17466666 @@ -2121,11 +2121,11 @@ track 0: data = length 70, hash 1DB944F6 sample 523: time = 17366666 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 524: time = 17433333 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA5 sample 525: time = 17600000 @@ -2137,11 +2137,11 @@ track 0: data = length 70, hash AF7E4C32 sample 527: time = 17500000 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 528: time = 17566666 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 529: time = 17733333 @@ -2153,11 +2153,11 @@ track 0: data = length 70, hash F383DB6D sample 531: time = 17633333 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 532: time = 17700000 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DC sample 533: time = 17866666 @@ -2169,11 +2169,11 @@ track 0: data = length 70, hash B92866C8 sample 535: time = 17766666 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 536: time = 17833333 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 537: time = 18000000 @@ -2185,11 +2185,11 @@ track 0: data = length 70, hash E9E98F05 sample 539: time = 17900000 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 540: time = 17966666 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 541: time = 18133333 @@ -2201,11 +2201,11 @@ track 0: data = length 70, hash 7BAE9640 sample 543: time = 18033333 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 544: time = 18100000 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 545: time = 18266666 @@ -2217,11 +2217,11 @@ track 0: data = length 70, hash 5B33157E sample 547: time = 18166666 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 548: time = 18233333 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 549: time = 18400000 @@ -2233,11 +2233,11 @@ track 0: data = length 70, hash ECF81CBA sample 551: time = 18300000 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 552: time = 18366666 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 553: time = 18533333 @@ -2249,11 +2249,11 @@ track 0: data = length 70, hash 1DB944F6 sample 555: time = 18433333 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 556: time = 18500000 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA5 sample 557: time = 18666666 @@ -2265,11 +2265,11 @@ track 0: data = length 70, hash AF7E4C32 sample 559: time = 18566666 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 560: time = 18633333 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 561: time = 18800000 @@ -2281,11 +2281,11 @@ track 0: data = length 70, hash F383DB6D sample 563: time = 18700000 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 564: time = 18766666 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DC sample 565: time = 18933333 @@ -2297,11 +2297,11 @@ track 0: data = length 70, hash B92866C8 sample 567: time = 18833333 - flags = 0 + flags = 67108864 data = length 68, hash 66214C05 sample 568: time = 18900000 - flags = 0 + flags = 67108864 data = length 68, hash 4605E639 sample 569: time = 19066666 @@ -2313,11 +2313,11 @@ track 0: data = length 70, hash E9E98F05 sample 571: time = 18966666 - flags = 0 + flags = 67108864 data = length 68, hash 88475340 sample 572: time = 19033333 - flags = 0 + flags = 67108864 data = length 68, hash 682BED75 sample 573: time = 19200000 @@ -2329,11 +2329,11 @@ track 0: data = length 70, hash 7BAE9640 sample 575: time = 19100000 - flags = 0 + flags = 67108864 data = length 68, hash E8DCF97C sample 576: time = 19166666 - flags = 0 + flags = 67108864 data = length 68, hash C8C193B1 sample 577: time = 19333333 @@ -2345,11 +2345,11 @@ track 0: data = length 70, hash 5B33157E sample 579: time = 19233333 - flags = 0 + flags = 67108864 data = length 68, hash 3D0223F9 sample 580: time = 19300000 - flags = 0 + flags = 67108864 data = length 68, hash 1CE6BE2E sample 581: time = 19466666 @@ -2361,11 +2361,11 @@ track 0: data = length 70, hash ECF81CBA sample 583: time = 19366666 - flags = 0 + flags = 67108864 data = length 68, hash 9D97CA36 sample 584: time = 19433333 - flags = 0 + flags = 67108864 data = length 68, hash 7D7C6469 sample 585: time = 19600000 @@ -2377,11 +2377,11 @@ track 0: data = length 70, hash 1DB944F6 sample 587: time = 19500000 - flags = 0 + flags = 67108864 data = length 68, hash BFBDD171 sample 588: time = 19566666 - flags = 0 + flags = 67108864 data = length 68, hash 9FA26BA5 sample 589: time = 19733333 @@ -2393,11 +2393,11 @@ track 0: data = length 70, hash AF7E4C32 sample 591: time = 19633333 - flags = 0 + flags = 67108864 data = length 68, hash 205377AE sample 592: time = 19700000 - flags = 0 + flags = 67108864 data = length 68, hash 3811E1 sample 593: time = 19866666 @@ -2409,11 +2409,11 @@ track 0: data = length 70, hash F383DB6D sample 595: time = 19766666 - flags = 0 + flags = 67108864 data = length 68, hash 8D5999A8 sample 596: time = 19833333 - flags = 0 + flags = 67108864 data = length 68, hash 6D3E33DC sample 597: time = 19966666 @@ -2425,6 +2425,6 @@ track 0: data = length 70, hash 3AC762E9 sample 599: time = 19933333 - flags = 0 + flags = 67108864 data = length 68, hash B1C7769A tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_sideloaded_track.mp4.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_sideloaded_track.mp4.0.dump index a9ef2b3b587..1f373b84593 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_sideloaded_track.mp4.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_sideloaded_track.mp4.0.dump @@ -23,11 +23,11 @@ track 0: data = length 1295, hash C0DA5090 sample 3: time = 100100 - flags = 0 + flags = 67108864 data = length 469, hash D6E0A200 sample 4: time = 166833 - flags = 0 + flags = 67108864 data = length 564, hash E5F56C5B sample 5: time = 333666 @@ -39,11 +39,11 @@ track 0: data = length 847, hash DCC2B618 sample 7: time = 233566 - flags = 0 + flags = 67108864 data = length 455, hash B9CCE047 sample 8: time = 300300 - flags = 0 + flags = 67108864 data = length 467, hash 69806D94 sample 9: time = 467133 @@ -55,11 +55,11 @@ track 0: data = length 1087, hash 491BF106 sample 11: time = 367033 - flags = 0 + flags = 67108864 data = length 380, hash 5FED016A sample 12: time = 433766 - flags = 0 + flags = 67108864 data = length 455, hash 8A0610 sample 13: time = 600600 @@ -71,11 +71,11 @@ track 0: data = length 1071, hash 684E7DC8 sample 15: time = 500500 - flags = 0 + flags = 67108864 data = length 653, hash 8494F326 sample 16: time = 567233 - flags = 0 + flags = 67108864 data = length 485, hash 2CCC85F4 sample 17: time = 734066 @@ -87,11 +87,11 @@ track 0: data = length 997, hash 164FF210 sample 19: time = 633966 - flags = 0 + flags = 67108864 data = length 640, hash F664125B sample 20: time = 700700 - flags = 0 + flags = 67108864 data = length 491, hash B5930C7C sample 21: time = 867533 @@ -103,11 +103,11 @@ track 0: data = length 838, hash 294A3451 sample 23: time = 767433 - flags = 0 + flags = 67108864 data = length 544, hash FCCE2DE6 sample 24: time = 834166 - flags = 0 + flags = 67108864 data = length 329, hash A654FFA1 sample 25: time = 1001000 @@ -119,11 +119,11 @@ track 0: data = length 803, hash 7A5C4C1D sample 27: time = 900900 - flags = 0 + flags = 67108864 data = length 415, hash B31BBC3B sample 28: time = 967633 - flags = 0 + flags = 67108864 data = length 415, hash 850DFEA3 sample 29: time = 1034366 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_sideloaded_track.mp4.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_sideloaded_track.mp4.unknown_length.dump index a9ef2b3b587..1f373b84593 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_sideloaded_track.mp4.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_fragmented_sideloaded_track.mp4.unknown_length.dump @@ -23,11 +23,11 @@ track 0: data = length 1295, hash C0DA5090 sample 3: time = 100100 - flags = 0 + flags = 67108864 data = length 469, hash D6E0A200 sample 4: time = 166833 - flags = 0 + flags = 67108864 data = length 564, hash E5F56C5B sample 5: time = 333666 @@ -39,11 +39,11 @@ track 0: data = length 847, hash DCC2B618 sample 7: time = 233566 - flags = 0 + flags = 67108864 data = length 455, hash B9CCE047 sample 8: time = 300300 - flags = 0 + flags = 67108864 data = length 467, hash 69806D94 sample 9: time = 467133 @@ -55,11 +55,11 @@ track 0: data = length 1087, hash 491BF106 sample 11: time = 367033 - flags = 0 + flags = 67108864 data = length 380, hash 5FED016A sample 12: time = 433766 - flags = 0 + flags = 67108864 data = length 455, hash 8A0610 sample 13: time = 600600 @@ -71,11 +71,11 @@ track 0: data = length 1071, hash 684E7DC8 sample 15: time = 500500 - flags = 0 + flags = 67108864 data = length 653, hash 8494F326 sample 16: time = 567233 - flags = 0 + flags = 67108864 data = length 485, hash 2CCC85F4 sample 17: time = 734066 @@ -87,11 +87,11 @@ track 0: data = length 997, hash 164FF210 sample 19: time = 633966 - flags = 0 + flags = 67108864 data = length 640, hash F664125B sample 20: time = 700700 - flags = 0 + flags = 67108864 data = length 491, hash B5930C7C sample 21: time = 867533 @@ -103,11 +103,11 @@ track 0: data = length 838, hash 294A3451 sample 23: time = 767433 - flags = 0 + flags = 67108864 data = length 544, hash FCCE2DE6 sample 24: time = 834166 - flags = 0 + flags = 67108864 data = length 329, hash A654FFA1 sample 25: time = 1001000 @@ -119,11 +119,11 @@ track 0: data = length 803, hash 7A5C4C1D sample 27: time = 900900 - flags = 0 + flags = 67108864 data = length 415, hash B31BBC3B sample 28: time = 967633 - flags = 0 + flags = 67108864 data = length 415, hash 850DFEA3 sample 29: time = 1034366 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.0.dump index 72410da0dd2..bc5c8986fc3 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.0.dump @@ -21,8 +21,8 @@ track 0: language = und metadata = entries=[Mp4AlternateGroup: 1, TSSE: description=null: values=[Lavf58.29.100], Mp4Timestamp: creation time=0, modification time=0, timescale=1000] initializationData: - data = length 19, hash 86852AE2 - data = length 8, hash 72CBCBF5 + data = length 19, hash BFE794DB + data = length 8, hash CA22068C data = length 8, hash 79C07075 sample 0: time = -6500 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.1.dump index 9ba1cd47f43..c7d7cd4b6e1 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.1.dump @@ -21,8 +21,8 @@ track 0: language = und metadata = entries=[Mp4AlternateGroup: 1, TSSE: description=null: values=[Lavf58.29.100], Mp4Timestamp: creation time=0, modification time=0, timescale=1000] initializationData: - data = length 19, hash 86852AE2 - data = length 8, hash 72CBCBF5 + data = length 19, hash BFE794DB + data = length 8, hash CA22068C data = length 8, hash 79C07075 sample 0: time = 483500 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.2.dump index b60d8c0e5bd..430608b38ac 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.2.dump @@ -21,8 +21,8 @@ track 0: language = und metadata = entries=[Mp4AlternateGroup: 1, TSSE: description=null: values=[Lavf58.29.100], Mp4Timestamp: creation time=0, modification time=0, timescale=1000] initializationData: - data = length 19, hash 86852AE2 - data = length 8, hash 72CBCBF5 + data = length 19, hash BFE794DB + data = length 8, hash CA22068C data = length 8, hash 79C07075 sample 0: time = 963500 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.3.dump index 5d2b03c3410..41443a07179 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.3.dump @@ -21,8 +21,8 @@ track 0: language = und metadata = entries=[Mp4AlternateGroup: 1, TSSE: description=null: values=[Lavf58.29.100], Mp4Timestamp: creation time=0, modification time=0, timescale=1000] initializationData: - data = length 19, hash 86852AE2 - data = length 8, hash 72CBCBF5 + data = length 19, hash BFE794DB + data = length 8, hash CA22068C data = length 8, hash 79C07075 sample 0: time = 993500 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.unknown_length.dump index 72410da0dd2..bc5c8986fc3 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus.mp4.unknown_length.dump @@ -21,8 +21,8 @@ track 0: language = und metadata = entries=[Mp4AlternateGroup: 1, TSSE: description=null: values=[Lavf58.29.100], Mp4Timestamp: creation time=0, modification time=0, timescale=1000] initializationData: - data = length 19, hash 86852AE2 - data = length 8, hash 72CBCBF5 + data = length 19, hash BFE794DB + data = length 8, hash CA22068C data = length 8, hash 79C07075 sample 0: time = -6500 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus_fragmented.mp4.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus_fragmented.mp4.0.dump index cbf6269565a..ba2984a071d 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus_fragmented.mp4.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus_fragmented.mp4.0.dump @@ -14,7 +14,7 @@ track 0: sampleRate = 16000 metadata = entries=[Mp4Timestamp: creation time=0, modification time=0, timescale=1000000] initializationData: - data = length 19, hash 4034F23B + data = length 19, hash C898FCE2 data = length 8, hash 94446F01 data = length 8, hash 79C07075 sample 0: diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus_fragmented.mp4.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus_fragmented.mp4.unknown_length.dump index cbf6269565a..ba2984a071d 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus_fragmented.mp4.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_opus_fragmented.mp4.unknown_length.dump @@ -14,7 +14,7 @@ track 0: sampleRate = 16000 metadata = entries=[Mp4Timestamp: creation time=0, modification time=0, timescale=1000000] initializationData: - data = length 19, hash 4034F23B + data = length 19, hash C898FCE2 data = length 8, hash 94446F01 data = length 8, hash 79C07075 sample 0: diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.0.dump index 400d475923b..23a3896a7a5 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.0.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 134000 + duration = 133945 getPosition(0) = [[timeUs=0, position=1428]] getPosition(1) = [[timeUs=0, position=1428]] - getPosition(67000) = [[timeUs=0, position=1428]] - getPosition(134000) = [[timeUs=0, position=1428]] + getPosition(66972) = [[timeUs=0, position=1428]] + getPosition(133945) = [[timeUs=0, position=1428]] numberOfTracks = 2 track 0: total output bytes = 87715 sample count = 30 - track duration = 134000 + track duration = 133466 format 0: averageBitrate = 2939595 id = 1 @@ -151,7 +151,7 @@ track 0: track 1: total output bytes = 10107 sample count = 45 - track duration = 134000 + track duration = 133945 format 0: averageBitrate = 1254 peakBitrate = 69000 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.1.dump index 400d475923b..23a3896a7a5 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.1.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 134000 + duration = 133945 getPosition(0) = [[timeUs=0, position=1428]] getPosition(1) = [[timeUs=0, position=1428]] - getPosition(67000) = [[timeUs=0, position=1428]] - getPosition(134000) = [[timeUs=0, position=1428]] + getPosition(66972) = [[timeUs=0, position=1428]] + getPosition(133945) = [[timeUs=0, position=1428]] numberOfTracks = 2 track 0: total output bytes = 87715 sample count = 30 - track duration = 134000 + track duration = 133466 format 0: averageBitrate = 2939595 id = 1 @@ -151,7 +151,7 @@ track 0: track 1: total output bytes = 10107 sample count = 45 - track duration = 134000 + track duration = 133945 format 0: averageBitrate = 1254 peakBitrate = 69000 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.2.dump index 400d475923b..23a3896a7a5 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.2.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 134000 + duration = 133945 getPosition(0) = [[timeUs=0, position=1428]] getPosition(1) = [[timeUs=0, position=1428]] - getPosition(67000) = [[timeUs=0, position=1428]] - getPosition(134000) = [[timeUs=0, position=1428]] + getPosition(66972) = [[timeUs=0, position=1428]] + getPosition(133945) = [[timeUs=0, position=1428]] numberOfTracks = 2 track 0: total output bytes = 87715 sample count = 30 - track duration = 134000 + track duration = 133466 format 0: averageBitrate = 2939595 id = 1 @@ -151,7 +151,7 @@ track 0: track 1: total output bytes = 10107 sample count = 45 - track duration = 134000 + track duration = 133945 format 0: averageBitrate = 1254 peakBitrate = 69000 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.3.dump index 400d475923b..23a3896a7a5 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.3.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 134000 + duration = 133945 getPosition(0) = [[timeUs=0, position=1428]] getPosition(1) = [[timeUs=0, position=1428]] - getPosition(67000) = [[timeUs=0, position=1428]] - getPosition(134000) = [[timeUs=0, position=1428]] + getPosition(66972) = [[timeUs=0, position=1428]] + getPosition(133945) = [[timeUs=0, position=1428]] numberOfTracks = 2 track 0: total output bytes = 87715 sample count = 30 - track duration = 134000 + track duration = 133466 format 0: averageBitrate = 2939595 id = 1 @@ -151,7 +151,7 @@ track 0: track 1: total output bytes = 10107 sample count = 45 - track duration = 134000 + track duration = 133945 format 0: averageBitrate = 1254 peakBitrate = 69000 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.unknown_length.dump index c4ea2a7ec9b..f63dc7c27a6 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_partially_fragmented.mp4.unknown_length.dump @@ -1,12 +1,12 @@ seekMap: isSeekable = false - duration = 134000 + duration = 133945 getPosition(0) = [[timeUs=0, position=1428]] numberOfTracks = 2 track 0: total output bytes = 87715 sample count = 30 - track duration = 134000 + track duration = 133466 format 0: averageBitrate = 2939595 id = 1 @@ -148,7 +148,7 @@ track 0: track 1: total output bytes = 10107 sample count = 45 - track duration = 134000 + track duration = 133945 format 0: averageBitrate = 1254 peakBitrate = 69000 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.0.dump index 2edf5ece182..bb59de1cd19 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.0.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 4166600 + duration = 4166622 getPosition(0) = [[timeUs=0, position=40]] getPosition(1) = [[timeUs=0, position=40], [timeUs=33333, position=52417]] - getPosition(2083300) = [[timeUs=2066644, position=4012385], [timeUs=2099977, position=4082637]] - getPosition(4166600) = [[timeUs=4133288, position=8385548]] + getPosition(2083311) = [[timeUs=2066644, position=4012385], [timeUs=2099977, position=4082637]] + getPosition(4166622) = [[timeUs=4133288, position=8385548]] numberOfTracks = 1 track 0: total output bytes = 8456438 sample count = 125 - track duration = 4166600 + track duration = 4166622 format 0: averageBitrate = 16236535 id = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.1.dump index 01a7e30110d..428e8715be5 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.1.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 4166600 + duration = 4166622 getPosition(0) = [[timeUs=0, position=40]] getPosition(1) = [[timeUs=0, position=40], [timeUs=33333, position=52417]] - getPosition(2083300) = [[timeUs=2066644, position=4012385], [timeUs=2099977, position=4082637]] - getPosition(4166600) = [[timeUs=4133288, position=8385548]] + getPosition(2083311) = [[timeUs=2066644, position=4012385], [timeUs=2099977, position=4082637]] + getPosition(4166622) = [[timeUs=4133288, position=8385548]] numberOfTracks = 1 track 0: total output bytes = 5890743 sample count = 84 - track duration = 4166600 + track duration = 4166622 format 0: averageBitrate = 16236535 id = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.2.dump index 3763bc1dd76..a362c6ed58f 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.2.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 4166600 + duration = 4166622 getPosition(0) = [[timeUs=0, position=40]] getPosition(1) = [[timeUs=0, position=40], [timeUs=33333, position=52417]] - getPosition(2083300) = [[timeUs=2066644, position=4012385], [timeUs=2099977, position=4082637]] - getPosition(4166600) = [[timeUs=4133288, position=8385548]] + getPosition(2083311) = [[timeUs=2066644, position=4012385], [timeUs=2099977, position=4082637]] + getPosition(4166622) = [[timeUs=4133288, position=8385548]] numberOfTracks = 1 track 0: total output bytes = 2962049 sample count = 42 - track duration = 4166600 + track duration = 4166622 format 0: averageBitrate = 16236535 id = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.3.dump index 77646e4db8b..cb57fe15534 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.3.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 4166600 + duration = 4166622 getPosition(0) = [[timeUs=0, position=40]] getPosition(1) = [[timeUs=0, position=40], [timeUs=33333, position=52417]] - getPosition(2083300) = [[timeUs=2066644, position=4012385], [timeUs=2099977, position=4082637]] - getPosition(4166600) = [[timeUs=4133288, position=8385548]] + getPosition(2083311) = [[timeUs=2066644, position=4012385], [timeUs=2099977, position=4082637]] + getPosition(4166622) = [[timeUs=4133288, position=8385548]] numberOfTracks = 1 track 0: total output bytes = 70930 sample count = 1 - track duration = 4166600 + track duration = 4166622 format 0: averageBitrate = 16236535 id = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.unknown_length.dump index 2edf5ece182..bb59de1cd19 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_apvc.mp4.unknown_length.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 4166600 + duration = 4166622 getPosition(0) = [[timeUs=0, position=40]] getPosition(1) = [[timeUs=0, position=40], [timeUs=33333, position=52417]] - getPosition(2083300) = [[timeUs=2066644, position=4012385], [timeUs=2099977, position=4082637]] - getPosition(4166600) = [[timeUs=4133288, position=8385548]] + getPosition(2083311) = [[timeUs=2066644, position=4012385], [timeUs=2099977, position=4082637]] + getPosition(4166622) = [[timeUs=4133288, position=8385548]] numberOfTracks = 1 track 0: total output bytes = 8456438 sample count = 125 - track duration = 4166600 + track duration = 4166622 format 0: averageBitrate = 16236535 id = 1 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.0.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.0.dump index 33c6b8391b0..c38651c61dc 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.0.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.0.dump @@ -154,7 +154,7 @@ track 0: track 1: total output bytes = 7350 sample count = 30 - track duration = 1003000 + track duration = 1002785 format 0: averageBitrate = 58637 id = 2 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.1.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.1.dump index d82f1a8e0eb..009117a9183 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.1.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.1.dump @@ -154,7 +154,7 @@ track 0: track 1: total output bytes = 4900 sample count = 20 - track duration = 1003000 + track duration = 1002785 format 0: averageBitrate = 58637 id = 2 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.2.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.2.dump index dbd9fdfdcd4..6db29ee2957 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.2.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.2.dump @@ -154,7 +154,7 @@ track 0: track 1: total output bytes = 2450 sample count = 10 - track duration = 1003000 + track duration = 1002785 format 0: averageBitrate = 58637 id = 2 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.3.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.3.dump index 2893f493c7e..259a548b8af 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.3.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.3.dump @@ -154,7 +154,7 @@ track 0: track 1: total output bytes = 245 sample count = 1 - track duration = 1003000 + track duration = 1002785 format 0: averageBitrate = 58637 id = 2 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.unknown_length.dump index 33c6b8391b0..c38651c61dc 100644 --- a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.unknown_length.dump +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_it35_track.mp4.unknown_length.dump @@ -154,7 +154,7 @@ track 0: track 1: total output bytes = 7350 sample count = 30 - track duration = 1003000 + track duration = 1002785 format 0: averageBitrate = 58637 id = 2 diff --git a/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_xmp_uuid.mp4_with_flag_read_xmp_metadata b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_xmp_uuid.mp4_with_flag_read_xmp_metadata new file mode 100644 index 00000000000..2b2a9be8c0c --- /dev/null +++ b/libraries/test_data/src/test/assets/extractordumps/mp4/sample_with_xmp_uuid.mp4_with_flag_read_xmp_metadata @@ -0,0 +1,891 @@ +seekMap: + isSeekable = true + duration = 2000177 + getPosition(0) = [[timeUs=0, position=44]] + getPosition(1) = [[timeUs=0, position=44], [timeUs=166688, position=44]] + getPosition(1000088) = [[timeUs=1000088, position=44]] + getPosition(2000177) = [[timeUs=1833500, position=44]] +numberOfTracks = 4 +track 0: + total output bytes = 62276630 + sample count = 60 + track duration = 2000177 + format 0: + averageBitrate = 249084476 + id = 1 + containerMimeType = video/mp4 + sampleMimeType = video/hevc + codecs = hvc1.2.4.H180 + maxInputSize = 2088529 + maxNumReorderSamples = 0 + width = 4032 + height = 2268 + decodedWidth = 4032 + decodedHeight = 2272 + frameRate = 30.00 + rotationDegrees = 90 + maxSubLayers = 1 + colorInfo: + colorSpace = 1 + colorRange = 2 + colorTransfer = 2 + lumaBitdepth = 10 + chromaBitdepth = 10 + metadata = entries=[mdta: key=com.android.capture.fps, value=30.0, mdta: key=com.android.manufacturer, value=Google, mdta: key=com.android.model, value=Pixel 10 Pro XL, Mp4Timestamp: creation time=3857914650, modification time=3857914650, timescale=10000, ThumbnailMetadata: presentationTimeUs=500044, XMP: 3c783a786d706d65746120786d6c6e733a783d2261646f62653a6e733a6d6574612f2220783a786d70746b3d2241646f626520584d5020436f726520352e312e302d6a63303033223e0a20203c7264663a52444620786d6c6e733a7264663d22687474703a2f2f7777772e77332e6f72672f313939392f30322f32322d7264662d73796e7461782d6e7323223e0a202020203c7264663a4465736372697074696f6e207264663a61626f75743d22220a2020202020202020786d6c6e733a4743616d6572613d22687474703a2f2f6e732e676f6f676c652e636f6d2f70686f746f732f312e302f63616d6572612f220a2020202020204743616d6572613a4746696c654d657461646174613d2247686349416843414478693443434541414141414141412b514441414f67494941513d3d222f3e0a20203c2f7264663a5244463e0a3c2f783a786d706d6574613e0a] + initializationData: + data = length 93, hash 37CEAAC1 + sample 0: + time = 0 + flags = 1 + data = length 776537, hash C7DC3493 + sample 1: + time = 33344 + flags = 0 + data = length 122942, hash 92CF159B + sample 2: + time = 66677 + flags = 0 + data = length 31099, hash 2B3011D5 + sample 3: + time = 100011 + flags = 0 + data = length 118562, hash BF18E1A0 + sample 4: + time = 133355 + flags = 0 + data = length 255031, hash A8A7501B + sample 5: + time = 166688 + flags = 1 + data = length 1783808, hash C21806C5 + sample 6: + time = 200022 + flags = 0 + data = length 339656, hash 82A13C5E + sample 7: + time = 233355 + flags = 0 + data = length 808790, hash 59733652 + sample 8: + time = 266700 + flags = 0 + data = length 933059, hash 25746C62 + sample 9: + time = 300033 + flags = 0 + data = length 930694, hash 930CDC11 + sample 10: + time = 333366 + flags = 1 + data = length 1795769, hash 26E88CF3 + sample 11: + time = 366700 + flags = 0 + data = length 822794, hash A325D3EF + sample 12: + time = 400044 + flags = 0 + data = length 967851, hash B5B23124 + sample 13: + time = 433377 + flags = 0 + data = length 983345, hash F4DB5768 + sample 14: + time = 466711 + flags = 0 + data = length 1004093, hash B4E8261D + sample 15: + time = 500044 + flags = 1 + data = length 2088499, hash 42E91E81 + sample 16: + time = 533388 + flags = 0 + data = length 1016727, hash BB8D8F4B + sample 17: + time = 566722 + flags = 0 + data = length 1018797, hash BFF9DCD0 + sample 18: + time = 600055 + flags = 0 + data = length 1016674, hash 2AACA1E0 + sample 19: + time = 633388 + flags = 0 + data = length 1013958, hash B8887EB6 + sample 20: + time = 666733 + flags = 1 + data = length 2074060, hash A8518AF6 + sample 21: + time = 700066 + flags = 0 + data = length 994982, hash 8F0F40F9 + sample 22: + time = 733400 + flags = 0 + data = length 980425, hash 911DF57B + sample 23: + time = 766744 + flags = 0 + data = length 968801, hash 727909DB + sample 24: + time = 800077 + flags = 0 + data = length 964202, hash 77E4B405 + sample 25: + time = 833411 + flags = 1 + data = length 2016117, hash 42CA9C92 + sample 26: + time = 866744 + flags = 0 + data = length 952516, hash 23A99250 + sample 27: + time = 900088 + flags = 0 + data = length 945971, hash 38376083 + sample 28: + time = 933422 + flags = 0 + data = length 941409, hash 1FB99BD1 + sample 29: + time = 966755 + flags = 0 + data = length 934808, hash D2C1B4C7 + sample 30: + time = 1000088 + flags = 1 + data = length 1989952, hash 100D6B7E + sample 31: + time = 1033433 + flags = 0 + data = length 930469, hash E9CF8ED0 + sample 32: + time = 1066766 + flags = 0 + data = length 924593, hash 6C9C2325 + sample 33: + time = 1100100 + flags = 0 + data = length 927048, hash BD636DF7 + sample 34: + time = 1133433 + flags = 0 + data = length 926655, hash E30EDE6E + sample 35: + time = 1166777 + flags = 1 + data = length 1984368, hash 8CE9C018 + sample 36: + time = 1200111 + flags = 0 + data = length 924707, hash AC073B84 + sample 37: + time = 1233444 + flags = 0 + data = length 918512, hash 53CF27C4 + sample 38: + time = 1266788 + flags = 0 + data = length 908927, hash 653F22F6 + sample 39: + time = 1300122 + flags = 0 + data = length 905577, hash 6C42F9CF + sample 40: + time = 1333455 + flags = 1 + data = length 1942633, hash 7517A99D + sample 41: + time = 1366788 + flags = 0 + data = length 885355, hash 2430DC54 + sample 42: + time = 1400133 + flags = 0 + data = length 874429, hash 75F8DB61 + sample 43: + time = 1433466 + flags = 0 + data = length 871636, hash B9F4A9E0 + sample 44: + time = 1466800 + flags = 0 + data = length 868265, hash D8716A73 + sample 45: + time = 1500133 + flags = 1 + data = length 1897358, hash 65098E3B + sample 46: + time = 1533477 + flags = 0 + data = length 870398, hash 85EB7538 + sample 47: + time = 1566811 + flags = 0 + data = length 868004, hash 8F00493D + sample 48: + time = 1600144 + flags = 0 + data = length 862131, hash DE8D496E + sample 49: + time = 1633477 + flags = 0 + data = length 861054, hash 9A22A6CB + sample 50: + time = 1666822 + flags = 1 + data = length 1879137, hash 73FD0FC7 + sample 51: + time = 1700155 + flags = 0 + data = length 854808, hash DDF125F4 + sample 52: + time = 1733488 + flags = 0 + data = length 851241, hash 77951D41 + sample 53: + time = 1766822 + flags = 0 + data = length 848790, hash 9FC9108A + sample 54: + time = 1800166 + flags = 0 + data = length 849399, hash 33A063EC + sample 55: + time = 1833500 + flags = 1 + data = length 1863895, hash 7ADB6656 + sample 56: + time = 1866833 + flags = 0 + data = length 847209, hash 8A034B0B + sample 57: + time = 1900177 + flags = 0 + data = length 847675, hash 70F44BAA + sample 58: + time = 1933511 + flags = 0 + data = length 843304, hash 96995FB9 + sample 59: + time = 1966844 + flags = 536870912 + data = length 847125, hash 55ABD83F +track 1: + total output bytes = 42694 + sample count = 83 + track duration = 1770700 + format 0: + averageBitrate = 192895 + peakBitrate = 192000 + id = 2 + containerMimeType = video/mp4 + sampleMimeType = audio/mp4a-latm + codecs = mp4a.40.2 + maxInputSize = 716 + channelCount = 2 + sampleRate = 48000 + metadata = entries=[mdta: key=com.android.manufacturer, value=Google, mdta: key=com.android.model, value=Pixel 10 Pro XL, Mp4Timestamp: creation time=3857914650, modification time=3857914650, timescale=10000, XMP: 3c783a786d706d65746120786d6c6e733a783d2261646f62653a6e733a6d6574612f2220783a786d70746b3d2241646f626520584d5020436f726520352e312e302d6a63303033223e0a20203c7264663a52444620786d6c6e733a7264663d22687474703a2f2f7777772e77332e6f72672f313939392f30322f32322d7264662d73796e7461782d6e7323223e0a202020203c7264663a4465736372697074696f6e207264663a61626f75743d22220a2020202020202020786d6c6e733a4743616d6572613d22687474703a2f2f6e732e676f6f676c652e636f6d2f70686f746f732f312e302f63616d6572612f220a2020202020204743616d6572613a4746696c654d657461646174613d2247686349416843414478693443434541414141414141412b514441414f67494941513d3d222f3e0a20203c2f7264663a5244463e0a3c2f783a786d706d6574613e0a] + initializationData: + data = length 2, hash 560 + sample 0: + time = 3900 + flags = 1 + data = length 686, hash 80AF87A + sample 1: + time = 25233 + flags = 1 + data = length 647, hash 334A7E78 + sample 2: + time = 46566 + flags = 1 + data = length 617, hash 521182F7 + sample 3: + time = 67900 + flags = 1 + data = length 586, hash 2079D91E + sample 4: + time = 89233 + flags = 1 + data = length 567, hash 81D70020 + sample 5: + time = 110566 + flags = 1 + data = length 563, hash 814BBCCF + sample 6: + time = 131900 + flags = 1 + data = length 535, hash CF742ED0 + sample 7: + time = 153233 + flags = 1 + data = length 530, hash BDC31758 + sample 8: + time = 174566 + flags = 1 + data = length 524, hash 4431237F + sample 9: + time = 195900 + flags = 1 + data = length 518, hash 1894CB2B + sample 10: + time = 217233 + flags = 1 + data = length 510, hash C5E82B2C + sample 11: + time = 238566 + flags = 1 + data = length 505, hash 560C4521 + sample 12: + time = 259900 + flags = 1 + data = length 530, hash A841527 + sample 13: + time = 281233 + flags = 1 + data = length 514, hash 370C89D4 + sample 14: + time = 302566 + flags = 1 + data = length 507, hash 92151404 + sample 15: + time = 323900 + flags = 1 + data = length 507, hash 217C4E22 + sample 16: + time = 345233 + flags = 1 + data = length 510, hash 7A1B4274 + sample 17: + time = 366566 + flags = 1 + data = length 521, hash 54172867 + sample 18: + time = 387900 + flags = 1 + data = length 501, hash E6526B97 + sample 19: + time = 409233 + flags = 1 + data = length 495, hash 87D39E0C + sample 20: + time = 430566 + flags = 1 + data = length 531, hash 7FCCE7A4 + sample 21: + time = 451900 + flags = 1 + data = length 478, hash 7482366C + sample 22: + time = 473233 + flags = 1 + data = length 454, hash E5A89635 + sample 23: + time = 494566 + flags = 1 + data = length 472, hash 9ACBF515 + sample 24: + time = 515900 + flags = 1 + data = length 496, hash CEC8755 + sample 25: + time = 537233 + flags = 1 + data = length 473, hash BA037A51 + sample 26: + time = 558566 + flags = 1 + data = length 460, hash 3E2761ED + sample 27: + time = 579900 + flags = 1 + data = length 470, hash E0AFA168 + sample 28: + time = 601233 + flags = 1 + data = length 478, hash 93914EB3 + sample 29: + time = 622566 + flags = 1 + data = length 486, hash 7657DC04 + sample 30: + time = 643900 + flags = 1 + data = length 492, hash 1760D740 + sample 31: + time = 665233 + flags = 1 + data = length 492, hash 1399E727 + sample 32: + time = 686566 + flags = 1 + data = length 494, hash 8FA7F2CD + sample 33: + time = 707900 + flags = 1 + data = length 497, hash 847E750F + sample 34: + time = 729233 + flags = 1 + data = length 498, hash 6B2C9E52 + sample 35: + time = 750566 + flags = 1 + data = length 536, hash 1B646A77 + sample 36: + time = 771900 + flags = 1 + data = length 499, hash 27324FD3 + sample 37: + time = 793233 + flags = 1 + data = length 498, hash B1164FE5 + sample 38: + time = 814566 + flags = 1 + data = length 494, hash F68BFCB7 + sample 39: + time = 835900 + flags = 1 + data = length 515, hash B3140BFC + sample 40: + time = 857233 + flags = 1 + data = length 504, hash 477F82F3 + sample 41: + time = 878566 + flags = 1 + data = length 507, hash FDFC8243 + sample 42: + time = 899900 + flags = 1 + data = length 510, hash C984FE57 + sample 43: + time = 921233 + flags = 1 + data = length 500, hash 1504DDCD + sample 44: + time = 942566 + flags = 1 + data = length 515, hash E1E59E3F + sample 45: + time = 963900 + flags = 1 + data = length 519, hash 887DC2EB + sample 46: + time = 985233 + flags = 1 + data = length 513, hash 85B14E0B + sample 47: + time = 1006566 + flags = 1 + data = length 511, hash 8FC0099A + sample 48: + time = 1027900 + flags = 1 + data = length 512, hash D72F6928 + sample 49: + time = 1049233 + flags = 1 + data = length 507, hash CF9E3DAD + sample 50: + time = 1070566 + flags = 1 + data = length 508, hash 2F2266D8 + sample 51: + time = 1091900 + flags = 1 + data = length 511, hash CDCBA78F + sample 52: + time = 1113233 + flags = 1 + data = length 505, hash 8E316EAE + sample 53: + time = 1134566 + flags = 1 + data = length 508, hash 1C203D1D + sample 54: + time = 1155900 + flags = 1 + data = length 510, hash 24830FFA + sample 55: + time = 1177233 + flags = 1 + data = length 508, hash E4FC9331 + sample 56: + time = 1198566 + flags = 1 + data = length 504, hash ACFA591 + sample 57: + time = 1219900 + flags = 1 + data = length 525, hash 9F1267B0 + sample 58: + time = 1241233 + flags = 1 + data = length 515, hash DDFC1FF + sample 59: + time = 1262566 + flags = 1 + data = length 518, hash A5D8747F + sample 60: + time = 1283900 + flags = 1 + data = length 514, hash B68096CA + sample 61: + time = 1305233 + flags = 1 + data = length 511, hash 51A76499 + sample 62: + time = 1326566 + flags = 1 + data = length 512, hash 9A2F0218 + sample 63: + time = 1347900 + flags = 1 + data = length 520, hash 6AE3265C + sample 64: + time = 1369233 + flags = 1 + data = length 512, hash CAA100B + sample 65: + time = 1390566 + flags = 1 + data = length 521, hash 9DD78CDC + sample 66: + time = 1411900 + flags = 1 + data = length 532, hash DCC57BF9 + sample 67: + time = 1433233 + flags = 1 + data = length 551, hash D5A400C8 + sample 68: + time = 1454566 + flags = 1 + data = length 507, hash 57075CC0 + sample 69: + time = 1475900 + flags = 1 + data = length 509, hash A49B4985 + sample 70: + time = 1497233 + flags = 1 + data = length 491, hash 4F66259D + sample 71: + time = 1518566 + flags = 1 + data = length 508, hash 40928732 + sample 72: + time = 1539900 + flags = 1 + data = length 506, hash 1867B5F3 + sample 73: + time = 1561233 + flags = 1 + data = length 506, hash FA08EF45 + sample 74: + time = 1582566 + flags = 1 + data = length 512, hash 10FAAF70 + sample 75: + time = 1603900 + flags = 1 + data = length 502, hash FEC73CD4 + sample 76: + time = 1625233 + flags = 1 + data = length 517, hash 3DF8FF55 + sample 77: + time = 1646566 + flags = 1 + data = length 509, hash 72CDD643 + sample 78: + time = 1667900 + flags = 1 + data = length 514, hash 612977A9 + sample 79: + time = 1689233 + flags = 1 + data = length 509, hash 6393AB9A + sample 80: + time = 1710566 + flags = 1 + data = length 513, hash 16090FF5 + sample 81: + time = 1731900 + flags = 1 + data = length 504, hash 2A9D797C + sample 82: + time = 1753233 + flags = 536870913 + data = length 518, hash A225FE5 +track 2: + total output bytes = 992708 + sample count = 60 + track duration = 2000177 + format 0: + averageBitrate = 3970481 + id = 3 + containerMimeType = video/mp4 + sampleMimeType = application/p11-image-meta + maxInputSize = 16647 + metadata = entries=[mdta: key=com.android.manufacturer, value=Google, mdta: key=com.android.model, value=Pixel 10 Pro XL, Mp4Timestamp: creation time=3857914650, modification time=3857914650, timescale=10000, XMP: 3c783a786d706d65746120786d6c6e733a783d2261646f62653a6e733a6d6574612f2220783a786d70746b3d2241646f626520584d5020436f726520352e312e302d6a63303033223e0a20203c7264663a52444620786d6c6e733a7264663d22687474703a2f2f7777772e77332e6f72672f313939392f30322f32322d7264662d73796e7461782d6e7323223e0a202020203c7264663a4465736372697074696f6e207264663a61626f75743d22220a2020202020202020786d6c6e733a4743616d6572613d22687474703a2f2f6e732e676f6f676c652e636f6d2f70686f746f732f312e302f63616d6572612f220a2020202020204743616d6572613a4746696c654d657461646174613d2247686349416843414478693443434541414141414141412b514441414f67494941513d3d222f3e0a20203c2f7264663a5244463e0a3c2f783a786d706d6574613e0a] + sample 0: + time = 0 + flags = 1 + data = length 16559, hash 7239301D + sample 1: + time = 33344 + flags = 1 + data = length 16539, hash 37829E40 + sample 2: + time = 66677 + flags = 1 + data = length 16509, hash BCAE3F87 + sample 3: + time = 100011 + flags = 1 + data = length 16559, hash 93949399 + sample 4: + time = 133355 + flags = 1 + data = length 16539, hash 1D0B4D1E + sample 5: + time = 166688 + flags = 1 + data = length 16509, hash F3A37E46 + sample 6: + time = 200022 + flags = 1 + data = length 16559, hash BFD4F05B + sample 7: + time = 233355 + flags = 1 + data = length 16539, hash 772F4AB3 + sample 8: + time = 266700 + flags = 1 + data = length 16615, hash 23F590D2 + sample 9: + time = 300033 + flags = 1 + data = length 16509, hash A49090C6 + sample 10: + time = 333366 + flags = 1 + data = length 16539, hash CCB0D68C + sample 11: + time = 366700 + flags = 1 + data = length 16559, hash F3D5E15B + sample 12: + time = 400044 + flags = 1 + data = length 16509, hash 612AE59E + sample 13: + time = 433377 + flags = 1 + data = length 16539, hash C5615CFF + sample 14: + time = 466711 + flags = 1 + data = length 16559, hash BFCB87BB + sample 15: + time = 500044 + flags = 1 + data = length 16559, hash 9C3C85E3 + sample 16: + time = 533388 + flags = 1 + data = length 16489, hash 1911DF16 + sample 17: + time = 566722 + flags = 1 + data = length 16615, hash BCF7B214 + sample 18: + time = 600055 + flags = 1 + data = length 16559, hash F4EE6C9F + sample 19: + time = 633388 + flags = 1 + data = length 16489, hash A5EF7480 + sample 20: + time = 666733 + flags = 1 + data = length 16559, hash 60CEF173 + sample 21: + time = 700066 + flags = 1 + data = length 16559, hash F221E534 + sample 22: + time = 733400 + flags = 1 + data = length 16539, hash 6169650E + sample 23: + time = 766744 + flags = 1 + data = length 16509, hash D2EFB9C5 + sample 24: + time = 800077 + flags = 1 + data = length 16559, hash 4F489665 + sample 25: + time = 833411 + flags = 1 + data = length 16539, hash FB1552E + sample 26: + time = 866744 + flags = 1 + data = length 16565, hash BFB81F93 + sample 27: + time = 900088 + flags = 1 + data = length 16559, hash BB9F3DFD + sample 28: + time = 933422 + flags = 1 + data = length 16539, hash 6D02528 + sample 29: + time = 966755 + flags = 1 + data = length 16559, hash A0D51DFE + sample 30: + time = 1000088 + flags = 1 + data = length 16509, hash 6E7D7288 + sample 31: + time = 1033433 + flags = 1 + data = length 16539, hash 4FD4FFD2 + sample 32: + time = 1066766 + flags = 1 + data = length 16559, hash 46BA2E79 + sample 33: + time = 1100100 + flags = 1 + data = length 16509, hash B09300F1 + sample 34: + time = 1133433 + flags = 1 + data = length 16539, hash AB518096 + sample 35: + time = 1166777 + flags = 1 + data = length 16615, hash FD1249A1 + sample 36: + time = 1200111 + flags = 1 + data = length 16559, hash C8ABDDCD + sample 37: + time = 1233444 + flags = 1 + data = length 16539, hash 4CF66F0C + sample 38: + time = 1266788 + flags = 1 + data = length 16559, hash 129634A6 + sample 39: + time = 1300122 + flags = 1 + data = length 16559, hash C2692190 + sample 40: + time = 1333455 + flags = 1 + data = length 16489, hash 7C77174E + sample 41: + time = 1366788 + flags = 1 + data = length 16559, hash 99409F2A + sample 42: + time = 1400133 + flags = 1 + data = length 16559, hash 5EF0D739 + sample 43: + time = 1433466 + flags = 1 + data = length 16489, hash ECE6B0E6 + sample 44: + time = 1466800 + flags = 1 + data = length 16617, hash F05CA44C + sample 45: + time = 1500133 + flags = 1 + data = length 16561, hash 81D472A9 + sample 46: + time = 1533477 + flags = 1 + data = length 16541, hash 58667D42 + sample 47: + time = 1566811 + flags = 1 + data = length 16511, hash F189EAE9 + sample 48: + time = 1600144 + flags = 1 + data = length 16561, hash 5DF95951 + sample 49: + time = 1633477 + flags = 1 + data = length 16541, hash 39BE7AC4 + sample 50: + time = 1666822 + flags = 1 + data = length 16511, hash 91A4596 + sample 51: + time = 1700155 + flags = 1 + data = length 16561, hash 54A6B382 + sample 52: + time = 1733488 + flags = 1 + data = length 16541, hash 22BB1B6B + sample 53: + time = 1766822 + flags = 1 + data = length 16617, hash 8B44075 + sample 54: + time = 1800166 + flags = 1 + data = length 16511, hash D85BA754 + sample 55: + time = 1833500 + flags = 1 + data = length 16541, hash 729AF2E7 + sample 56: + time = 1866833 + flags = 1 + data = length 16561, hash 3BF24F95 + sample 57: + time = 1900177 + flags = 1 + data = length 16511, hash D899AF57 + sample 58: + time = 1933511 + flags = 1 + data = length 16541, hash 2ED695B1 + sample 59: + time = 1966844 + flags = 536870913 + data = length 16561, hash 1962E97A +track 3: + total output bytes = 6864 + sample count = 1 + track duration = 0 + format 0: + id = 4 + containerMimeType = video/mp4 + sampleMimeType = application/p11-global-meta + maxInputSize = 6894 + metadata = entries=[mdta: key=com.android.manufacturer, value=Google, mdta: key=com.android.model, value=Pixel 10 Pro XL, Mp4Timestamp: creation time=3857914650, modification time=3857914650, timescale=10000, XMP: 3c783a786d706d65746120786d6c6e733a783d2261646f62653a6e733a6d6574612f2220783a786d70746b3d2241646f626520584d5020436f726520352e312e302d6a63303033223e0a20203c7264663a52444620786d6c6e733a7264663d22687474703a2f2f7777772e77332e6f72672f313939392f30322f32322d7264662d73796e7461782d6e7323223e0a202020203c7264663a4465736372697074696f6e207264663a61626f75743d22220a2020202020202020786d6c6e733a4743616d6572613d22687474703a2f2f6e732e676f6f676c652e636f6d2f70686f746f732f312e302f63616d6572612f220a2020202020204743616d6572613a4746696c654d657461646174613d2247686349416843414478693443434541414141414141412b514441414f67494941513d3d222f3e0a20203c2f7264663a5244463e0a3c2f783a786d706d6574613e0a] + sample 0: + time = 0 + flags = 536870913 + data = length 6864, hash 1A3CA7AA +tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/ts/sample_ac3_straddle.ps.0.dump b/libraries/test_data/src/test/assets/extractordumps/ts/sample_ac3_straddle.ps.0.dump new file mode 100644 index 00000000000..7451d588e0f --- /dev/null +++ b/libraries/test_data/src/test/assets/extractordumps/ts/sample_ac3_straddle.ps.0.dump @@ -0,0 +1,179 @@ +seekMap: + isSeekable = true + duration = 1201344 + getPosition(0) = [[timeUs=0, position=0]] + getPosition(1) = [[timeUs=1, position=0]] + getPosition(600672) = [[timeUs=600672, position=35805]] + getPosition(1201344) = [[timeUs=1201344, position=71799]] +numberOfTracks = 2 +track 189: + total output bytes = 19200 + sample count = 25 + format 0: + averageBitrate = 192000 + peakBitrate = 192000 + id = 189 + containerMimeType = video/mp2p + sampleMimeType = audio/ac3 + channelCount = 2 + sampleRate = 48000 + sample 0: + time = 61333 + flags = 1 + data = length 768, hash 4513D6ED + sample 1: + time = 93333 + flags = 1 + data = length 768, hash F7DD71AD + sample 2: + time = 157333 + flags = 1 + data = length 768, hash B46BDFC2 + sample 3: + time = 189333 + flags = 1 + data = length 768, hash 6E3DC97F + sample 4: + time = 221333 + flags = 1 + data = length 768, hash BCF0EE73 + sample 5: + time = 253333 + flags = 1 + data = length 768, hash 4FD0241F + sample 6: + time = 285333 + flags = 1 + data = length 768, hash DF256D70 + sample 7: + time = 317333 + flags = 1 + data = length 768, hash BD3A8741 + sample 8: + time = 349333 + flags = 1 + data = length 768, hash 6CDA87FA + sample 9: + time = 381333 + flags = 1 + data = length 768, hash D7542AD6 + sample 10: + time = 413333 + flags = 1 + data = length 768, hash 650B3B21 + sample 11: + time = 445333 + flags = 1 + data = length 768, hash 7AA9EA5 + sample 12: + time = 477333 + flags = 1 + data = length 768, hash 36E854AB + sample 13: + time = 509333 + flags = 1 + data = length 768, hash 1958F2B0 + sample 14: + time = 541333 + flags = 1 + data = length 768, hash E12C6DFC + sample 15: + time = 573333 + flags = 1 + data = length 768, hash 64E149B3 + sample 16: + time = 605333 + flags = 1 + data = length 768, hash A053FE3F + sample 17: + time = 637333 + flags = 1 + data = length 768, hash 326041DB + sample 18: + time = 669333 + flags = 1 + data = length 768, hash 865250F + sample 19: + time = 701333 + flags = 1 + data = length 768, hash B45ADB43 + sample 20: + time = 733333 + flags = 1 + data = length 768, hash 99626351 + sample 21: + time = 765333 + flags = 1 + data = length 768, hash BCED46AE + sample 22: + time = 797333 + flags = 1 + data = length 768, hash 63B6188F + sample 23: + time = 829333 + flags = 1 + data = length 768, hash 86C6C28B + sample 24: + time = 861333 + flags = 1 + data = length 768, hash A1D014D1 +track 224: + total output bytes = 52248 + sample count = 12 + format 0: + id = 224 + containerMimeType = video/mp2p + sampleMimeType = video/mpeg2 + width = 160 + height = 120 + initializationData: + data = length 22, hash 3B91971B + sample 0: + time = 66666 + flags = 1 + data = length 5986, hash 6A578416 + sample 1: + time = 133333 + flags = 0 + data = length 5571, hash A276297C + sample 2: + time = 200000 + flags = 0 + data = length 4475, hash 50693038 + sample 3: + time = 266666 + flags = 0 + data = length 5187, hash 8AD660F2 + sample 4: + time = 333333 + flags = 0 + data = length 4552, hash 773C4FC3 + sample 5: + time = 400000 + flags = 0 + data = length 5611, hash A70A9C09 + sample 6: + time = 466666 + flags = 0 + data = length 4740, hash F8C7264F + sample 7: + time = 533333 + flags = 0 + data = length 3679, hash BEF11657 + sample 8: + time = 600000 + flags = 0 + data = length 2910, hash 5F82B9AA + sample 9: + time = 666666 + flags = 0 + data = length 3357, hash 91082EE0 + sample 10: + time = 733333 + flags = 0 + data = length 2471, hash B011430D + sample 11: + time = 800000 + flags = 0 + data = length 3709, hash 381BE4ED +tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/ts/sample_ac3_straddle.ps.1.dump b/libraries/test_data/src/test/assets/extractordumps/ts/sample_ac3_straddle.ps.1.dump new file mode 100644 index 00000000000..d2b218bb612 --- /dev/null +++ b/libraries/test_data/src/test/assets/extractordumps/ts/sample_ac3_straddle.ps.1.dump @@ -0,0 +1,147 @@ +seekMap: + isSeekable = true + duration = 1201344 + getPosition(0) = [[timeUs=0, position=0]] + getPosition(1) = [[timeUs=1, position=0]] + getPosition(600672) = [[timeUs=600672, position=35805]] + getPosition(1201344) = [[timeUs=1201344, position=71799]] +numberOfTracks = 2 +track 189: + total output bytes = 16896 + sample count = 22 + format 0: + averageBitrate = 192000 + peakBitrate = 192000 + id = 189 + containerMimeType = video/mp2p + sampleMimeType = audio/ac3 + channelCount = 2 + sampleRate = 48000 + sample 0: + time = 157333 + flags = 1 + data = length 768, hash 6E3DC97F + sample 1: + time = 189333 + flags = 1 + data = length 768, hash BCF0EE73 + sample 2: + time = 253333 + flags = 1 + data = length 768, hash 4FD0241F + sample 3: + time = 285333 + flags = 1 + data = length 768, hash DF256D70 + sample 4: + time = 317333 + flags = 1 + data = length 768, hash BD3A8741 + sample 5: + time = 349333 + flags = 1 + data = length 768, hash 6CDA87FA + sample 6: + time = 381333 + flags = 1 + data = length 768, hash D7542AD6 + sample 7: + time = 413333 + flags = 1 + data = length 768, hash 650B3B21 + sample 8: + time = 445333 + flags = 1 + data = length 768, hash 7AA9EA5 + sample 9: + time = 477333 + flags = 1 + data = length 768, hash 36E854AB + sample 10: + time = 509333 + flags = 1 + data = length 768, hash 1958F2B0 + sample 11: + time = 541333 + flags = 1 + data = length 768, hash E12C6DFC + sample 12: + time = 573333 + flags = 1 + data = length 768, hash 64E149B3 + sample 13: + time = 605333 + flags = 1 + data = length 768, hash A053FE3F + sample 14: + time = 637333 + flags = 1 + data = length 768, hash 326041DB + sample 15: + time = 669333 + flags = 1 + data = length 768, hash 865250F + sample 16: + time = 701333 + flags = 1 + data = length 768, hash B45ADB43 + sample 17: + time = 733333 + flags = 1 + data = length 768, hash 99626351 + sample 18: + time = 765333 + flags = 1 + data = length 768, hash BCED46AE + sample 19: + time = 797333 + flags = 1 + data = length 768, hash 63B6188F + sample 20: + time = 829333 + flags = 1 + data = length 768, hash 86C6C28B + sample 21: + time = 861333 + flags = 1 + data = length 768, hash A1D014D1 +track 224: + total output bytes = 30037 + sample count = 7 + format 0: + id = 224 + containerMimeType = video/mp2p + sampleMimeType = video/mpeg2 + width = 160 + height = 120 + initializationData: + data = length 22, hash 3B91971B + sample 0: + time = 400000 + flags = 0 + data = length 5611, hash A70A9C09 + sample 1: + time = 466666 + flags = 0 + data = length 4740, hash F8C7264F + sample 2: + time = 533333 + flags = 0 + data = length 3679, hash BEF11657 + sample 3: + time = 600000 + flags = 0 + data = length 2910, hash 5F82B9AA + sample 4: + time = 666666 + flags = 0 + data = length 3357, hash 91082EE0 + sample 5: + time = 733333 + flags = 0 + data = length 2471, hash B011430D + sample 6: + time = 800000 + flags = 0 + data = length 3709, hash 381BE4ED +tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/ts/sample_ac3_straddle.ps.2.dump b/libraries/test_data/src/test/assets/extractordumps/ts/sample_ac3_straddle.ps.2.dump new file mode 100644 index 00000000000..2510b897078 --- /dev/null +++ b/libraries/test_data/src/test/assets/extractordumps/ts/sample_ac3_straddle.ps.2.dump @@ -0,0 +1,99 @@ +seekMap: + isSeekable = true + duration = 1201344 + getPosition(0) = [[timeUs=0, position=0]] + getPosition(1) = [[timeUs=1, position=0]] + getPosition(600672) = [[timeUs=600672, position=35805]] + getPosition(1201344) = [[timeUs=1201344, position=71799]] +numberOfTracks = 2 +track 189: + total output bytes = 13056 + sample count = 17 + format 0: + averageBitrate = 192000 + peakBitrate = 192000 + id = 189 + containerMimeType = video/mp2p + sampleMimeType = audio/ac3 + channelCount = 2 + sampleRate = 48000 + sample 0: + time = 317333 + flags = 1 + data = length 768, hash 6CDA87FA + sample 1: + time = 349333 + flags = 1 + data = length 768, hash D7542AD6 + sample 2: + time = 413333 + flags = 1 + data = length 768, hash 650B3B21 + sample 3: + time = 445333 + flags = 1 + data = length 768, hash 7AA9EA5 + sample 4: + time = 477333 + flags = 1 + data = length 768, hash 36E854AB + sample 5: + time = 509333 + flags = 1 + data = length 768, hash 1958F2B0 + sample 6: + time = 541333 + flags = 1 + data = length 768, hash E12C6DFC + sample 7: + time = 573333 + flags = 1 + data = length 768, hash 64E149B3 + sample 8: + time = 605333 + flags = 1 + data = length 768, hash A053FE3F + sample 9: + time = 637333 + flags = 1 + data = length 768, hash 326041DB + sample 10: + time = 669333 + flags = 1 + data = length 768, hash 865250F + sample 11: + time = 701333 + flags = 1 + data = length 768, hash B45ADB43 + sample 12: + time = 733333 + flags = 1 + data = length 768, hash 99626351 + sample 13: + time = 765333 + flags = 1 + data = length 768, hash BCED46AE + sample 14: + time = 797333 + flags = 1 + data = length 768, hash 63B6188F + sample 15: + time = 829333 + flags = 1 + data = length 768, hash 86C6C28B + sample 16: + time = 861333 + flags = 1 + data = length 768, hash A1D014D1 +track 224: + total output bytes = 0 + sample count = 0 + format 0: + id = 224 + containerMimeType = video/mp2p + sampleMimeType = video/mpeg2 + width = 160 + height = 120 + initializationData: + data = length 22, hash 3B91971B +tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/ts/sample_ac3_straddle.ps.3.dump b/libraries/test_data/src/test/assets/extractordumps/ts/sample_ac3_straddle.ps.3.dump new file mode 100644 index 00000000000..10e4a5069e2 --- /dev/null +++ b/libraries/test_data/src/test/assets/extractordumps/ts/sample_ac3_straddle.ps.3.dump @@ -0,0 +1,35 @@ +seekMap: + isSeekable = true + duration = 1201344 + getPosition(0) = [[timeUs=0, position=0]] + getPosition(1) = [[timeUs=1, position=0]] + getPosition(600672) = [[timeUs=600672, position=35805]] + getPosition(1201344) = [[timeUs=1201344, position=71799]] +numberOfTracks = 2 +track 189: + total output bytes = 768 + sample count = 1 + format 0: + averageBitrate = 192000 + peakBitrate = 192000 + id = 189 + containerMimeType = video/mp2p + sampleMimeType = audio/ac3 + channelCount = 2 + sampleRate = 48000 + sample 0: + time = 829333 + flags = 1 + data = length 768, hash A1D014D1 +track 224: + total output bytes = 0 + sample count = 0 + format 0: + id = 224 + containerMimeType = video/mp2p + sampleMimeType = video/mpeg2 + width = 160 + height = 120 + initializationData: + data = length 22, hash 3B91971B +tracksEnded = true diff --git a/libraries/test_data/src/test/assets/extractordumps/ts/sample_ac3_straddle.ps.unknown_length.dump b/libraries/test_data/src/test/assets/extractordumps/ts/sample_ac3_straddle.ps.unknown_length.dump new file mode 100644 index 00000000000..2e480b28aa4 --- /dev/null +++ b/libraries/test_data/src/test/assets/extractordumps/ts/sample_ac3_straddle.ps.unknown_length.dump @@ -0,0 +1,176 @@ +seekMap: + isSeekable = false + duration = UNSET TIME + getPosition(0) = [[timeUs=0, position=0]] +numberOfTracks = 2 +track 189: + total output bytes = 19200 + sample count = 25 + format 0: + averageBitrate = 192000 + peakBitrate = 192000 + id = 189 + containerMimeType = video/mp2p + sampleMimeType = audio/ac3 + channelCount = 2 + sampleRate = 48000 + sample 0: + time = 61333 + flags = 1 + data = length 768, hash 4513D6ED + sample 1: + time = 93333 + flags = 1 + data = length 768, hash F7DD71AD + sample 2: + time = 157333 + flags = 1 + data = length 768, hash B46BDFC2 + sample 3: + time = 189333 + flags = 1 + data = length 768, hash 6E3DC97F + sample 4: + time = 221333 + flags = 1 + data = length 768, hash BCF0EE73 + sample 5: + time = 253333 + flags = 1 + data = length 768, hash 4FD0241F + sample 6: + time = 285333 + flags = 1 + data = length 768, hash DF256D70 + sample 7: + time = 317333 + flags = 1 + data = length 768, hash BD3A8741 + sample 8: + time = 349333 + flags = 1 + data = length 768, hash 6CDA87FA + sample 9: + time = 381333 + flags = 1 + data = length 768, hash D7542AD6 + sample 10: + time = 413333 + flags = 1 + data = length 768, hash 650B3B21 + sample 11: + time = 445333 + flags = 1 + data = length 768, hash 7AA9EA5 + sample 12: + time = 477333 + flags = 1 + data = length 768, hash 36E854AB + sample 13: + time = 509333 + flags = 1 + data = length 768, hash 1958F2B0 + sample 14: + time = 541333 + flags = 1 + data = length 768, hash E12C6DFC + sample 15: + time = 573333 + flags = 1 + data = length 768, hash 64E149B3 + sample 16: + time = 605333 + flags = 1 + data = length 768, hash A053FE3F + sample 17: + time = 637333 + flags = 1 + data = length 768, hash 326041DB + sample 18: + time = 669333 + flags = 1 + data = length 768, hash 865250F + sample 19: + time = 701333 + flags = 1 + data = length 768, hash B45ADB43 + sample 20: + time = 733333 + flags = 1 + data = length 768, hash 99626351 + sample 21: + time = 765333 + flags = 1 + data = length 768, hash BCED46AE + sample 22: + time = 797333 + flags = 1 + data = length 768, hash 63B6188F + sample 23: + time = 829333 + flags = 1 + data = length 768, hash 86C6C28B + sample 24: + time = 861333 + flags = 1 + data = length 768, hash A1D014D1 +track 224: + total output bytes = 52248 + sample count = 12 + format 0: + id = 224 + containerMimeType = video/mp2p + sampleMimeType = video/mpeg2 + width = 160 + height = 120 + initializationData: + data = length 22, hash 3B91971B + sample 0: + time = 66666 + flags = 1 + data = length 5986, hash 6A578416 + sample 1: + time = 133333 + flags = 0 + data = length 5571, hash A276297C + sample 2: + time = 200000 + flags = 0 + data = length 4475, hash 50693038 + sample 3: + time = 266666 + flags = 0 + data = length 5187, hash 8AD660F2 + sample 4: + time = 333333 + flags = 0 + data = length 4552, hash 773C4FC3 + sample 5: + time = 400000 + flags = 0 + data = length 5611, hash A70A9C09 + sample 6: + time = 466666 + flags = 0 + data = length 4740, hash F8C7264F + sample 7: + time = 533333 + flags = 0 + data = length 3679, hash BEF11657 + sample 8: + time = 600000 + flags = 0 + data = length 2910, hash 5F82B9AA + sample 9: + time = 666666 + flags = 0 + data = length 3357, hash 91082EE0 + sample 10: + time = 733333 + flags = 0 + data = length 2471, hash B011430D + sample 11: + time = 800000 + flags = 0 + data = length 3709, hash 381BE4ED +tracksEnded = true diff --git a/libraries/test_data/src/test/assets/media/dash/empty-segment-timeline/sample.mpd b/libraries/test_data/src/test/assets/media/dash/empty-segment-timeline/sample.mpd new file mode 100644 index 00000000000..cf12169456c --- /dev/null +++ b/libraries/test_data/src/test/assets/media/dash/empty-segment-timeline/sample.mpd @@ -0,0 +1,14 @@ + + + + + + + + + + + diff --git a/libraries/test_data/src/test/assets/media/dash/multi-track-with-offset/multi-period.mpd b/libraries/test_data/src/test/assets/media/dash/multi-track-with-offset/multi-period.mpd new file mode 100644 index 00000000000..4f169dfb34a --- /dev/null +++ b/libraries/test_data/src/test/assets/media/dash/multi-track-with-offset/multi-period.mpd @@ -0,0 +1,39 @@ + + + + + + sample.video.mp4 + + + + + + + + sample.audio.mp4 + + + + + + + + + + period1.video.mp4 + + + + + + + + sample.audio.mp4 + + + + + + + diff --git a/libraries/test_data/src/test/assets/media/jpeg/ultraHdr_360x480.jpg b/libraries/test_data/src/test/assets/media/jpeg/ultraHdr_360x480.jpg new file mode 100644 index 00000000000..58f076d69e5 Binary files /dev/null and b/libraries/test_data/src/test/assets/media/jpeg/ultraHdr_360x480.jpg differ diff --git a/libraries/test_data/src/test/assets/media/mka/bear-flac-16bit-nested-chapters.mka b/libraries/test_data/src/test/assets/media/mka/bear-flac-16bit-nested-chapters.mka new file mode 100644 index 00000000000..6a963aad3f8 Binary files /dev/null and b/libraries/test_data/src/test/assets/media/mka/bear-flac-16bit-nested-chapters.mka differ diff --git a/libraries/test_data/src/test/assets/media/mp4/hlg_1280x720_primary_gradient.mp4 b/libraries/test_data/src/test/assets/media/mp4/hlg_1280x720_primary_gradient.mp4 new file mode 100644 index 00000000000..3bfaf5fe691 Binary files /dev/null and b/libraries/test_data/src/test/assets/media/mp4/hlg_1280x720_primary_gradient.mp4 differ diff --git a/libraries/test_data/src/test/assets/media/mp4/sample_av1_mono.mp4 b/libraries/test_data/src/test/assets/media/mp4/sample_av1_mono.mp4 deleted file mode 100644 index 7d3fe6ddae4..00000000000 Binary files a/libraries/test_data/src/test/assets/media/mp4/sample_av1_mono.mp4 and /dev/null differ diff --git a/libraries/test_data/src/test/assets/media/mp4/sample_av1_monochrome_sdr.mp4 b/libraries/test_data/src/test/assets/media/mp4/sample_av1_monochrome_sdr.mp4 new file mode 100644 index 00000000000..4b52800196f Binary files /dev/null and b/libraries/test_data/src/test/assets/media/mp4/sample_av1_monochrome_sdr.mp4 differ diff --git a/libraries/test_data/src/test/assets/media/mp4/sample_with_increasing_timestamps_320w_240h_5s_play_services_upscaled_000.png b/libraries/test_data/src/test/assets/media/mp4/sample_with_increasing_timestamps_320w_240h_5s_play_services_upscaled_000.png new file mode 100644 index 00000000000..651f87aa0bd Binary files /dev/null and b/libraries/test_data/src/test/assets/media/mp4/sample_with_increasing_timestamps_320w_240h_5s_play_services_upscaled_000.png differ diff --git a/libraries/test_data/src/test/assets/media/mp4/sample_with_xmp_uuid.mp4 b/libraries/test_data/src/test/assets/media/mp4/sample_with_xmp_uuid.mp4 new file mode 100644 index 00000000000..48d4c118208 Binary files /dev/null and b/libraries/test_data/src/test/assets/media/mp4/sample_with_xmp_uuid.mp4 differ diff --git a/libraries/test_data/src/test/assets/media/mpd/sample_mpd_content_steering_with_default_service_location b/libraries/test_data/src/test/assets/media/mpd/sample_mpd_content_steering_with_default_service_location new file mode 100644 index 00000000000..81f681c1acb --- /dev/null +++ b/libraries/test_data/src/test/assets/media/mpd/sample_mpd_content_steering_with_default_service_location @@ -0,0 +1,22 @@ + + + steering + manifest_loc1.mpd + manifest_loc2.mpd + + https://segments-loc1.com/ + https://segments-loc2.com/ + + + + + + https://segments-loc3.com/ + https://segments-loc4.com/ + + + + + diff --git a/libraries/test_data/src/test/assets/media/mpd/sample_mpd_content_steering_without_default_service_location b/libraries/test_data/src/test/assets/media/mpd/sample_mpd_content_steering_without_default_service_location new file mode 100644 index 00000000000..21045bf9bc5 --- /dev/null +++ b/libraries/test_data/src/test/assets/media/mpd/sample_mpd_content_steering_without_default_service_location @@ -0,0 +1,22 @@ + + + steering + manifest_loc1.mpd + manifest_loc2.mpd + + https://segments-loc1.com/ + https://segments-loc2.com/ + + + + + + https://segments-loc3.com/ + https://segments-loc4.com/ + + + + + diff --git a/libraries/test_data/src/test/assets/media/mpd/sample_mpd_different_representation_availabilities b/libraries/test_data/src/test/assets/media/mpd/sample_mpd_different_representation_availabilities new file mode 100644 index 00000000000..492c2f3bec8 --- /dev/null +++ b/libraries/test_data/src/test/assets/media/mpd/sample_mpd_different_representation_availabilities @@ -0,0 +1,40 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/libraries/test_data/src/test/assets/media/mpd/sample_mpd_trick_play_property_multiple_ids b/libraries/test_data/src/test/assets/media/mpd/sample_mpd_trick_play_property_multiple_ids new file mode 100644 index 00000000000..d1600b6bc86 --- /dev/null +++ b/libraries/test_data/src/test/assets/media/mpd/sample_mpd_trick_play_property_multiple_ids @@ -0,0 +1,24 @@ + + + + + + + + + + + + + + + + + + diff --git a/libraries/test_data/src/test/assets/media/mpd/sample_mpd_vod_location_and_track_fallback b/libraries/test_data/src/test/assets/media/mpd/sample_mpd_vod_location_and_track_fallback new file mode 100644 index 00000000000..1d45741fba6 --- /dev/null +++ b/libraries/test_data/src/test/assets/media/mpd/sample_mpd_vod_location_and_track_fallback @@ -0,0 +1,27 @@ + + + + + + + + + + + http://video.com/baseUrl/a/ + http://video.com/baseUrl/b/ + http://video.com/baseUrl/c/ + + + http://video.com/baseUrl/b/ + http://video.com/baseUrl/d/ + + + + diff --git a/libraries/test_data/src/test/assets/media/mpd/sample_mpd_vod_with_producer_reference_time b/libraries/test_data/src/test/assets/media/mpd/sample_mpd_vod_with_producer_reference_time new file mode 100644 index 00000000000..d82e60e1706 --- /dev/null +++ b/libraries/test_data/src/test/assets/media/mpd/sample_mpd_vod_with_producer_reference_time @@ -0,0 +1,23 @@ + + + + + + + + + + + + + + + + + + + diff --git a/libraries/test_data/src/test/assets/media/ts/sample_ac3_straddle.ps b/libraries/test_data/src/test/assets/media/ts/sample_ac3_straddle.ps new file mode 100644 index 00000000000..18e498a2020 Binary files /dev/null and b/libraries/test_data/src/test/assets/media/ts/sample_ac3_straddle.ps differ diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/5p1_iamf_opus.mp4_fragmented.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/5p1_iamf_opus.mp4_fragmented.dump index 7762974831b..be24ae79608 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/5p1_iamf_opus.mp4_fragmented.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/5p1_iamf_opus.mp4_fragmented.dump @@ -424,463 +424,463 @@ track 0: flags = 1 data = length 334, hash 4C607695 sample 102: - time = 2039979 + time = 2033500 flags = 1 data = length 332, hash E74186BB sample 103: - time = 2059979 + time = 2053500 flags = 1 data = length 337, hash 6CB8A18C sample 104: - time = 2079979 + time = 2073500 flags = 1 data = length 334, hash 4C2BDFD4 sample 105: - time = 2099979 + time = 2093500 flags = 1 data = length 333, hash A06E0080 sample 106: - time = 2119979 + time = 2113500 flags = 1 data = length 335, hash 7445D152 sample 107: - time = 2139979 + time = 2133500 flags = 1 data = length 332, hash E375990F sample 108: - time = 2159979 + time = 2153500 flags = 1 data = length 332, hash 3C908B20 sample 109: - time = 2179979 + time = 2173500 flags = 1 data = length 332, hash D62F9C3D sample 110: - time = 2199979 + time = 2193500 flags = 1 data = length 331, hash B66AFD97 sample 111: - time = 2219979 + time = 2213500 flags = 1 data = length 336, hash 77B2D8B7 sample 112: - time = 2239979 + time = 2233500 flags = 1 data = length 336, hash A011B0E2 sample 113: - time = 2259979 + time = 2253500 flags = 1 data = length 373, hash 6315BF93 sample 114: - time = 2279979 + time = 2273500 flags = 1 data = length 413, hash C3D99BA sample 115: - time = 2299979 + time = 2293500 flags = 1 data = length 294, hash C1F7ABB7 sample 116: - time = 2319979 + time = 2313500 flags = 1 data = length 472, hash DBA8563A sample 117: - time = 2339979 + time = 2333500 flags = 1 data = length 339, hash D9D7E93B sample 118: - time = 2359979 + time = 2353500 flags = 1 data = length 339, hash E8F21D67 sample 119: - time = 2379979 + time = 2373500 flags = 1 data = length 339, hash 9A3B9E7E sample 120: - time = 2399979 + time = 2393500 flags = 1 data = length 339, hash 64BF7C3B sample 121: - time = 2419979 + time = 2413500 flags = 1 data = length 339, hash E6E28BA0 sample 122: - time = 2439979 + time = 2433500 flags = 1 data = length 339, hash 9F6B8BB2 sample 123: - time = 2459979 + time = 2453500 flags = 1 data = length 20, hash 7BB4BD11 sample 124: - time = 2479979 + time = 2473500 flags = 1 data = length 20, hash 7BB4BD11 sample 125: - time = 2499979 + time = 2493500 flags = 1 data = length 20, hash 7BB4BD11 sample 126: - time = 2519979 + time = 2513500 flags = 1 data = length 20, hash 7BB4BD11 sample 127: - time = 2539979 + time = 2533500 flags = 1 data = length 20, hash 7BB4BD11 sample 128: - time = 2559979 + time = 2553500 flags = 1 data = length 20, hash 7BB4BD11 sample 129: - time = 2579979 + time = 2573500 flags = 1 data = length 20, hash 7BB4BD11 sample 130: - time = 2599979 + time = 2593500 flags = 1 data = length 20, hash 7BB4BD11 sample 131: - time = 2619979 + time = 2613500 flags = 1 data = length 20, hash 7BB4BD11 sample 132: - time = 2639979 + time = 2633500 flags = 1 data = length 20, hash 7BB4BD11 sample 133: - time = 2659979 + time = 2653500 flags = 1 data = length 20, hash 7BB4BD11 sample 134: - time = 2679979 + time = 2673500 flags = 1 data = length 20, hash 7BB4BD11 sample 135: - time = 2699979 + time = 2693500 flags = 1 data = length 20, hash 7BB4BD11 sample 136: - time = 2719979 + time = 2713500 flags = 1 data = length 20, hash 7BB4BD11 sample 137: - time = 2739979 + time = 2733500 flags = 1 data = length 20, hash 7BB4BD11 sample 138: - time = 2759979 + time = 2753500 flags = 1 data = length 20, hash 7BB4BD11 sample 139: - time = 2779979 + time = 2773500 flags = 1 data = length 20, hash 7BB4BD11 sample 140: - time = 2799979 + time = 2793500 flags = 1 data = length 20, hash 7BB4BD11 sample 141: - time = 2819979 + time = 2813500 flags = 1 data = length 20, hash 7BB4BD11 sample 142: - time = 2839979 + time = 2833500 flags = 1 data = length 20, hash 7BB4BD11 sample 143: - time = 2859979 + time = 2853500 flags = 1 data = length 20, hash 7BB4BD11 sample 144: - time = 2879979 + time = 2873500 flags = 1 data = length 20, hash 7BB4BD11 sample 145: - time = 2899979 + time = 2893500 flags = 1 data = length 20, hash 7BB4BD11 sample 146: - time = 2919979 + time = 2913500 flags = 1 data = length 20, hash 7BB4BD11 sample 147: - time = 2939979 + time = 2933500 flags = 1 data = length 20, hash 7BB4BD11 sample 148: - time = 2959979 + time = 2953500 flags = 1 data = length 20, hash 7BB4BD11 sample 149: - time = 2979979 + time = 2973500 flags = 1 data = length 539, hash D21DC823 sample 150: - time = 2999979 + time = 2993500 flags = 1 data = length 459, hash 274DD377 sample 151: - time = 3019979 + time = 3013500 flags = 1 data = length 339, hash 88CDC769 sample 152: - time = 3039979 + time = 3033500 flags = 1 data = length 339, hash 1B4B5109 sample 153: - time = 3059979 + time = 3053500 flags = 1 data = length 339, hash F446ED10 sample 154: - time = 3079979 + time = 3073500 flags = 1 data = length 339, hash FD28AB36 sample 155: - time = 3099979 + time = 3093500 flags = 1 data = length 338, hash 32CE018D sample 156: - time = 3119979 + time = 3113500 flags = 1 data = length 337, hash 1023482 sample 157: - time = 3139979 + time = 3133500 flags = 1 data = length 342, hash 274447BC sample 158: - time = 3159979 + time = 3153500 flags = 1 data = length 333, hash B7E5853E sample 159: - time = 3179979 + time = 3173500 flags = 1 data = length 333, hash 160EE196 sample 160: - time = 3199979 + time = 3193500 flags = 1 data = length 328, hash 70469877 sample 161: - time = 3219979 + time = 3213500 flags = 1 data = length 335, hash DF85F373 sample 162: - time = 3239979 + time = 3233500 flags = 1 data = length 331, hash B972FA14 sample 163: - time = 3259979 + time = 3253500 flags = 1 data = length 323, hash DAC964C0 sample 164: - time = 3279979 + time = 3273500 flags = 1 data = length 316, hash EDB3935D sample 165: - time = 3299979 + time = 3293500 flags = 1 data = length 303, hash E32BF857 sample 166: - time = 3319979 + time = 3313500 flags = 1 data = length 449, hash 1E734041 sample 167: - time = 3339979 + time = 3333500 flags = 1 data = length 20, hash 7BB4BD11 sample 168: - time = 3359979 + time = 3353500 flags = 1 data = length 20, hash 7BB4BD11 sample 169: - time = 3379979 + time = 3373500 flags = 1 data = length 20, hash 7BB4BD11 sample 170: - time = 3399979 + time = 3393500 flags = 1 data = length 20, hash 7BB4BD11 sample 171: - time = 3419979 + time = 3413500 flags = 1 data = length 20, hash 7BB4BD11 sample 172: - time = 3439979 + time = 3433500 flags = 1 data = length 20, hash 7BB4BD11 sample 173: - time = 3459979 + time = 3453500 flags = 1 data = length 20, hash 7BB4BD11 sample 174: - time = 3479979 + time = 3473500 flags = 1 data = length 20, hash 7BB4BD11 sample 175: - time = 3499979 + time = 3493500 flags = 1 data = length 20, hash 7BB4BD11 sample 176: - time = 3519979 + time = 3513500 flags = 1 data = length 20, hash 7BB4BD11 sample 177: - time = 3539979 + time = 3533500 flags = 1 data = length 20, hash 7BB4BD11 sample 178: - time = 3559979 + time = 3553500 flags = 1 data = length 20, hash 7BB4BD11 sample 179: - time = 3579979 + time = 3573500 flags = 1 data = length 20, hash 7BB4BD11 sample 180: - time = 3599979 + time = 3593500 flags = 1 data = length 20, hash 7BB4BD11 sample 181: - time = 3619979 + time = 3613500 flags = 1 data = length 20, hash 7BB4BD11 sample 182: - time = 3639979 + time = 3633500 flags = 1 data = length 20, hash 7BB4BD11 sample 183: - time = 3659979 + time = 3653500 flags = 1 data = length 20, hash 7BB4BD11 sample 184: - time = 3679979 + time = 3673500 flags = 1 data = length 20, hash 7BB4BD11 sample 185: - time = 3699979 + time = 3693500 flags = 1 data = length 20, hash 7BB4BD11 sample 186: - time = 3719979 + time = 3713500 flags = 1 data = length 20, hash 7BB4BD11 sample 187: - time = 3739979 + time = 3733500 flags = 1 data = length 20, hash 7BB4BD11 sample 188: - time = 3759979 + time = 3753500 flags = 1 data = length 20, hash 7BB4BD11 sample 189: - time = 3779979 + time = 3773500 flags = 1 data = length 20, hash 7BB4BD11 sample 190: - time = 3799979 + time = 3793500 flags = 1 data = length 20, hash 7BB4BD11 sample 191: - time = 3819979 + time = 3813500 flags = 1 data = length 20, hash 7BB4BD11 sample 192: - time = 3839979 + time = 3833500 flags = 1 data = length 20, hash 7BB4BD11 sample 193: - time = 3859979 + time = 3853500 flags = 1 data = length 20, hash 7BB4BD11 sample 194: - time = 3879979 + time = 3873500 flags = 1 data = length 20, hash 7BB4BD11 sample 195: - time = 3899979 + time = 3893500 flags = 1 data = length 20, hash 7BB4BD11 sample 196: - time = 3919979 + time = 3913500 flags = 1 data = length 20, hash 7BB4BD11 sample 197: - time = 3939979 + time = 3933500 flags = 1 data = length 20, hash 7BB4BD11 sample 198: - time = 3959979 + time = 3953500 flags = 1 data = length 20, hash 7BB4BD11 sample 199: - time = 3979979 + time = 3973500 flags = 1 data = length 555, hash 8315DA0F sample 200: - time = 3999979 + time = 3993500 flags = 1 data = length 338, hash 21E437FF sample 201: - time = 4019979 + time = 4013500 flags = 1 data = length 338, hash C1223E88 sample 202: - time = 4039979 + time = 4033500 flags = 1 data = length 337, hash C339D543 sample 203: - time = 4059979 + time = 4053500 flags = 1 data = length 338, hash 58A062A5 sample 204: - time = 4079979 + time = 4073500 flags = 1 data = length 333, hash 1914D9C0 sample 205: - time = 4099979 + time = 4093500 flags = 1 data = length 334, hash C124CB54 sample 206: - time = 4119979 + time = 4113500 flags = 1 data = length 331, hash 4564D29C sample 207: - time = 4139979 + time = 4133500 flags = 1 data = length 330, hash E684C274 sample 208: - time = 4159979 + time = 4153500 flags = 1 data = length 336, hash CE73DDBA sample 209: - time = 4179979 + time = 4173500 flags = 1 data = length 423, hash BE60203D sample 210: - time = 4199979 + time = 4193500 flags = 1 data = length 335, hash DF48C071 sample 211: - time = 4219979 + time = 4213500 flags = 1 data = length 332, hash 2BAB3174 sample 212: - time = 4239979 + time = 4233500 flags = 1 data = length 337, hash CD43B01E sample 213: - time = 4259979 + time = 4253500 flags = 1 data = length 334, hash 711B8C38 sample 214: - time = 4279979 + time = 4273500 flags = 1 data = length 413, hash 5BDD0A99 sample 215: - time = 4299979 + time = 4293500 flags = 1 data = length 339, hash B1AF5EAD sample 216: - time = 4319979 + time = 4313500 flags = 1 data = length 351, hash E72CE347 tracksEnded = true diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/audio_sample_entry_box_opus.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/audio_sample_entry_box_opus.dump index 1fbd46e74a3..41631beff3b 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/audio_sample_entry_box_opus.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/audio_sample_entry_box_opus.dump @@ -1,2 +1,2 @@ Opus (63 bytes): - Data = length 55, hash 75EDC5F7 + Data = length 55, hash 9556E587 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_6ch_8kHz_opus.ogg.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_6ch_8kHz_opus.ogg.dump index 4eb43f69150..08d1c956df0 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_6ch_8kHz_opus.ogg.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_6ch_8kHz_opus.ogg.dump @@ -20,7 +20,7 @@ track 0: sampleRate = 48000 metadata = entries=[Mp4Timestamp: creation time=100000000, modification time=500000000, timescale=10000] initializationData: - data = length 27, hash D13C96B8 + data = length 27, hash 9EE6F879 data = length 8, hash CA22068C data = length 8, hash 79C07075 sample 0: diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_6ch_8kHz_opus.ogg_fragmented.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_6ch_8kHz_opus.ogg_fragmented.dump index a726ee448df..85f39bc2182 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_6ch_8kHz_opus.ogg_fragmented.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_6ch_8kHz_opus.ogg_fragmented.dump @@ -14,7 +14,7 @@ track 0: sampleRate = 48000 metadata = entries=[Mp4Timestamp: creation time=100000000, modification time=500000000, timescale=10000] initializationData: - data = length 27, hash D13C96B8 + data = length 27, hash 9EE6F879 data = length 8, hash CA22068C data = length 8, hash 79C07075 sample 0: diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_2b_firstpts_10_sec.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_2b_firstpts_10_sec.mp4.dump index 63b6170fa63..2c5f5e3557c 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_2b_firstpts_10_sec.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_2b_firstpts_10_sec.mp4.dump @@ -43,11 +43,11 @@ track 0: data = length 1072, hash 1C517FCB sample 3: time = 33333 - flags = 0 + flags = 67108864 data = length 341, hash CEAA2DBB sample 4: time = 100000 - flags = 0 + flags = 67108864 data = length 721, hash 358E2671 sample 5: time = 166666 @@ -67,11 +67,11 @@ track 0: data = length 177, hash 763D378 sample 9: time = 233333 - flags = 0 + flags = 67108864 data = length 119, hash 3DB07F2 sample 10: time = 300000 - flags = 0 + flags = 67108864 data = length 116, hash 17427DB sample 11: time = 466666 @@ -83,11 +83,11 @@ track 0: data = length 257, hash 759A3120 sample 13: time = 366666 - flags = 0 + flags = 67108864 data = length 98, hash 7ECAA85C sample 14: time = 433333 - flags = 0 + flags = 67108864 data = length 128, hash 4A19D0E4 sample 15: time = 600000 @@ -99,11 +99,11 @@ track 0: data = length 1168, hash 414B6A20 sample 17: time = 500000 - flags = 0 + flags = 67108864 data = length 400, hash 5341224E sample 18: time = 566666 - flags = 0 + flags = 67108864 data = length 632, hash 8F1AF258 sample 19: time = 700000 @@ -115,7 +115,7 @@ track 0: data = length 638, hash BA291A36 sample 21: time = 666666 - flags = 0 + flags = 67108864 data = length 433, hash 2DE4D9CA sample 22: time = 733333 @@ -131,7 +131,7 @@ track 0: data = length 824, hash AE138727 sample 25: time = 800000 - flags = 0 + flags = 67108864 data = length 897, hash 96F4B8C9 sample 26: time = 866666 @@ -147,7 +147,7 @@ track 0: data = length 552, hash 8A1DC9BC sample 29: time = 933333 - flags = 0 + flags = 67108864 data = length 347, hash 54092001 sample 30: time = 1066666 @@ -159,7 +159,7 @@ track 0: data = length 417, hash 28831907 sample 32: time = 1033333 - flags = 0 + flags = 67108864 data = length 210, hash 1E356FE5 sample 33: time = 1166666 @@ -171,7 +171,7 @@ track 0: data = length 416, hash 3D730845 sample 35: time = 1133333 - flags = 0 + flags = 67108864 data = length 226, hash 4FAC0407 sample 36: time = 1300000 @@ -183,11 +183,11 @@ track 0: data = length 1335, hash 16B3AFB3 sample 38: time = 1200000 - flags = 0 + flags = 67108864 data = length 810, hash 6DD80172 sample 39: time = 1266666 - flags = 0 + flags = 67108864 data = length 333, hash 5344DC3A sample 40: time = 1400000 @@ -199,7 +199,7 @@ track 0: data = length 572, hash 4B05FB5F sample 42: time = 1366666 - flags = 0 + flags = 67108864 data = length 252, hash C0BBBE0 sample 43: time = 1500000 @@ -211,7 +211,7 @@ track 0: data = length 443, hash FE17394D sample 45: time = 1466666 - flags = 0 + flags = 67108864 data = length 234, hash C08793EE sample 46: time = 1633333 @@ -223,11 +223,11 @@ track 0: data = length 672, hash E07EAEAC sample 48: time = 1533333 - flags = 0 + flags = 67108864 data = length 409, hash 860C229E sample 49: time = 1600000 - flags = 0 + flags = 67108864 data = length 393, hash 593D091E sample 50: time = 1766666 @@ -239,11 +239,11 @@ track 0: data = length 758, hash 78AF296A sample 52: time = 1666666 - flags = 0 + flags = 67108864 data = length 262, hash 1BD840DE sample 53: time = 1733333 - flags = 0 + flags = 67108864 data = length 570, hash D720DDCE sample 54: time = 1900000 @@ -255,11 +255,11 @@ track 0: data = length 1512, hash 15AD5ED1 sample 56: time = 1800000 - flags = 0 + flags = 67108864 data = length 1211, hash 240D4D76 sample 57: time = 1866666 - flags = 0 + flags = 67108864 data = length 555, hash 7165F0E1 sample 58: time = 2000000 @@ -271,7 +271,7 @@ track 0: data = length 1324, hash 6630341C sample 60: time = 1966666 - flags = 0 + flags = 67108864 data = length 1113, hash 2CE419CB sample 61: time = 2100000 @@ -283,7 +283,7 @@ track 0: data = length 1658, hash 6FD30E25 sample 63: time = 2066666 - flags = 0 + flags = 67108864 data = length 711, hash 1BFD9623 sample 64: time = 2166666 @@ -291,7 +291,7 @@ track 0: data = length 2389, hash DA32F47A sample 65: time = 2133333 - flags = 0 + flags = 67108864 data = length 924, hash A52273EB sample 66: time = 2300000 @@ -303,11 +303,11 @@ track 0: data = length 1722, hash 21765422 sample 68: time = 2200000 - flags = 0 + flags = 67108864 data = length 1002, hash ECD9E400 sample 69: time = 2266666 - flags = 0 + flags = 67108864 data = length 483, hash 7D7026EF sample 70: time = 2366666 @@ -315,7 +315,7 @@ track 0: data = length 3148, hash 36BC693B sample 71: time = 2333333 - flags = 0 + flags = 67108864 data = length 751, hash 4128839C sample 72: time = 2466666 @@ -327,7 +327,7 @@ track 0: data = length 1262, hash 2E678B4A sample 74: time = 2433333 - flags = 0 + flags = 67108864 data = length 2051, hash 673F7005 sample 75: time = 2500000 @@ -343,11 +343,11 @@ track 0: data = length 1644, hash 6890A9E sample 78: time = 2533333 - flags = 0 + flags = 67108864 data = length 2066, hash 3E108A79 sample 79: time = 2600000 - flags = 0 + flags = 67108864 data = length 357, hash 9F3F46CA sample 80: time = 2733333 @@ -359,7 +359,7 @@ track 0: data = length 552, hash 9EB5AB6F sample 82: time = 2700000 - flags = 0 + flags = 67108864 data = length 267, hash 4DE266C7 sample 83: time = 2833333 @@ -371,7 +371,7 @@ track 0: data = length 442, hash 84A6016C sample 85: time = 2800000 - flags = 0 + flags = 67108864 data = length 246, hash 15E324CB sample 86: time = 2933333 @@ -383,7 +383,7 @@ track 0: data = length 456, hash AE0BFF63 sample 88: time = 2900000 - flags = 0 + flags = 67108864 data = length 329, hash A7C440BB sample 89: time = 3033333 @@ -395,7 +395,7 @@ track 0: data = length 644, hash 3971EF67 sample 91: time = 3000000 - flags = 0 + flags = 67108864 data = length 379, hash 957A7C64 sample 92: time = 3166666 @@ -407,11 +407,11 @@ track 0: data = length 571, hash 68AEF974 sample 94: time = 3066666 - flags = 0 + flags = 67108864 data = length 515, hash 4AF1201B sample 95: time = 3133333 - flags = 0 + flags = 67108864 data = length 314, hash 2FC434E9 sample 96: time = 3300000 @@ -423,11 +423,11 @@ track 0: data = length 954, hash 4C595D28 sample 98: time = 3200000 - flags = 0 + flags = 67108864 data = length 543, hash 254B4BFA sample 99: time = 3266666 - flags = 0 + flags = 67108864 data = length 288, hash A34C978F sample 100: time = 3433333 @@ -439,11 +439,11 @@ track 0: data = length 703, hash DAFBF3D0 sample 102: time = 3333333 - flags = 0 + flags = 67108864 data = length 455, hash A17E5A07 sample 103: time = 3400000 - flags = 0 + flags = 67108864 data = length 348, hash A6260498 sample 104: time = 3566666 @@ -455,11 +455,11 @@ track 0: data = length 492, hash 44DCDBBA sample 106: time = 3466666 - flags = 0 + flags = 67108864 data = length 349, hash 603A6F91 sample 107: time = 3533333 - flags = 0 + flags = 67108864 data = length 180, hash B6EA9F51 sample 108: time = 3700000 @@ -471,11 +471,11 @@ track 0: data = length 331, hash C02F3C83 sample 110: time = 3600000 - flags = 0 + flags = 67108864 data = length 167, hash 9824D4E8 sample 111: time = 3666666 - flags = 0 + flags = 67108864 data = length 194, hash 2DC7F1D7 sample 112: time = 3833333 @@ -487,11 +487,11 @@ track 0: data = length 484, hash B33D9C67 sample 114: time = 3733333 - flags = 0 + flags = 67108864 data = length 264, hash 8E6C96EF sample 115: time = 3800000 - flags = 0 + flags = 67108864 data = length 294, hash 36C8898C sample 116: time = 3966666 @@ -503,10 +503,10 @@ track 0: data = length 559, hash 91CA32EA sample 118: time = 3866666 - flags = 0 + flags = 67108864 data = length 197, hash AA87B8D8 sample 119: time = 3933333 - flags = 536870912 + flags = 603979776 data = length 406, hash 2DE88209 tracksEnded = true diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_2b_firstpts_10_sec.mp4_fragmented.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_2b_firstpts_10_sec.mp4_fragmented.dump index 350e9565cc0..b1d0833b712 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_2b_firstpts_10_sec.mp4_fragmented.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_2b_firstpts_10_sec.mp4_fragmented.dump @@ -23,483 +23,483 @@ track 0: data = length 33, hash E354B60D data = length 10, hash 7A0D0F2B sample 0: - time = 0 + time = 10000000 flags = 1 data = length 56222, hash 8B11262A sample 1: - time = 133333 + time = 10133333 flags = 0 data = length 2309, hash 8BD40493 sample 2: - time = 66666 + time = 10066666 flags = 0 data = length 1072, hash 1C517FCB sample 3: - time = 33333 - flags = 0 + time = 10033333 + flags = 67108864 data = length 341, hash CEAA2DBB sample 4: - time = 100000 - flags = 0 + time = 10100000 + flags = 67108864 data = length 721, hash 358E2671 sample 5: - time = 166666 + time = 10166666 flags = 0 data = length 792, hash 5F79CA7C sample 6: - time = 200000 + time = 10200000 flags = 0 data = length 34513, hash 99B0B4D7 sample 7: - time = 333333 + time = 10333333 flags = 0 data = length 617, hash 7299D29D sample 8: - time = 266666 + time = 10266666 flags = 0 data = length 177, hash 763D378 sample 9: - time = 233333 - flags = 0 + time = 10233333 + flags = 67108864 data = length 119, hash 3DB07F2 sample 10: - time = 300000 - flags = 0 + time = 10300000 + flags = 67108864 data = length 116, hash 17427DB sample 11: - time = 466666 + time = 10466666 flags = 0 data = length 563, hash CA9E382B sample 12: - time = 400000 + time = 10400000 flags = 0 data = length 257, hash 759A3120 sample 13: - time = 366666 - flags = 0 + time = 10366666 + flags = 67108864 data = length 98, hash 7ECAA85C sample 14: - time = 433333 - flags = 0 + time = 10433333 + flags = 67108864 data = length 128, hash 4A19D0E4 sample 15: - time = 600000 + time = 10600000 flags = 0 data = length 2293, hash 1E30DD74 sample 16: - time = 533333 + time = 10533333 flags = 0 data = length 1168, hash 414B6A20 sample 17: - time = 500000 - flags = 0 + time = 10500000 + flags = 67108864 data = length 400, hash 5341224E sample 18: - time = 566666 - flags = 0 + time = 10566666 + flags = 67108864 data = length 632, hash 8F1AF258 sample 19: - time = 700000 + time = 10700000 flags = 0 data = length 1777, hash 9C9ADBD sample 20: - time = 633333 + time = 10633333 flags = 0 data = length 638, hash BA291A36 sample 21: - time = 666666 - flags = 0 + time = 10666666 + flags = 67108864 data = length 433, hash 2DE4D9CA sample 22: - time = 733333 + time = 10733333 flags = 0 data = length 2423, hash 3DC4FE1B sample 23: - time = 833333 + time = 10833333 flags = 0 data = length 4702, hash D0EF6816 sample 24: - time = 766666 + time = 10766666 flags = 0 data = length 824, hash AE138727 sample 25: - time = 800000 - flags = 0 + time = 10800000 + flags = 67108864 data = length 897, hash 96F4B8C9 sample 26: - time = 866666 + time = 10866666 flags = 0 data = length 3600, hash 67877CF6 sample 27: - time = 966666 + time = 10966666 flags = 0 data = length 3641, hash B8CDE0D6 sample 28: - time = 900000 + time = 10900000 flags = 0 data = length 552, hash 8A1DC9BC sample 29: - time = 933333 - flags = 0 + time = 10933333 + flags = 67108864 data = length 347, hash 54092001 sample 30: - time = 1066666 + time = 11066666 flags = 0 data = length 2889, hash 652A194F sample 31: - time = 1000000 + time = 11000000 flags = 0 data = length 417, hash 28831907 sample 32: - time = 1033333 - flags = 0 + time = 11033333 + flags = 67108864 data = length 210, hash 1E356FE5 sample 33: - time = 1166666 + time = 11166666 flags = 0 data = length 2228, hash 710E8D3F sample 34: - time = 1100000 + time = 11100000 flags = 0 data = length 416, hash 3D730845 sample 35: - time = 1133333 - flags = 0 + time = 11133333 + flags = 67108864 data = length 226, hash 4FAC0407 sample 36: - time = 1300000 + time = 11300000 flags = 0 data = length 18728, hash BF70415D sample 37: - time = 1233333 + time = 11233333 flags = 0 data = length 1335, hash 16B3AFB3 sample 38: - time = 1200000 - flags = 0 + time = 11200000 + flags = 67108864 data = length 810, hash 6DD80172 sample 39: - time = 1266666 - flags = 0 + time = 11266666 + flags = 67108864 data = length 333, hash 5344DC3A sample 40: - time = 1400000 + time = 11400000 flags = 0 data = length 3476, hash 7642AE9C sample 41: - time = 1333333 + time = 11333333 flags = 0 data = length 572, hash 4B05FB5F sample 42: - time = 1366666 - flags = 0 + time = 11366666 + flags = 67108864 data = length 252, hash C0BBBE0 sample 43: - time = 1500000 + time = 11500000 flags = 0 data = length 3150, hash 5CF827A2 sample 44: - time = 1433333 + time = 11433333 flags = 0 data = length 443, hash FE17394D sample 45: - time = 1466666 - flags = 0 + time = 11466666 + flags = 67108864 data = length 234, hash C08793EE sample 46: - time = 1633333 + time = 11633333 flags = 0 data = length 3109, hash 91C22057 sample 47: - time = 1566666 + time = 11566666 flags = 0 data = length 672, hash E07EAEAC sample 48: - time = 1533333 - flags = 0 + time = 11533333 + flags = 67108864 data = length 409, hash 860C229E sample 49: - time = 1600000 - flags = 0 + time = 11600000 + flags = 67108864 data = length 393, hash 593D091E sample 50: - time = 1766666 + time = 11766666 flags = 0 data = length 4499, hash 768963B4 sample 51: - time = 1700000 + time = 11700000 flags = 0 data = length 758, hash 78AF296A sample 52: - time = 1666666 - flags = 0 + time = 11666666 + flags = 67108864 data = length 262, hash 1BD840DE sample 53: - time = 1733333 - flags = 0 + time = 11733333 + flags = 67108864 data = length 570, hash D720DDCE sample 54: - time = 1900000 + time = 11900000 flags = 0 data = length 4975, hash 2D0B435A sample 55: - time = 1833333 + time = 11833333 flags = 0 data = length 1512, hash 15AD5ED1 sample 56: - time = 1800000 - flags = 0 + time = 11800000 + flags = 67108864 data = length 1211, hash 240D4D76 sample 57: - time = 1866666 - flags = 0 + time = 11866666 + flags = 67108864 data = length 555, hash 7165F0E1 sample 58: - time = 2000000 + time = 12000000 flags = 0 data = length 6906, hash A76ED09E sample 59: - time = 1933333 + time = 11933333 flags = 0 data = length 1324, hash 6630341C sample 60: - time = 1966666 - flags = 0 + time = 11966666 + flags = 67108864 data = length 1113, hash 2CE419CB sample 61: - time = 2100000 + time = 12100000 flags = 0 data = length 6594, hash 4EBDF858 sample 62: - time = 2033333 + time = 12033333 flags = 0 data = length 1658, hash 6FD30E25 sample 63: - time = 2066666 - flags = 0 + time = 12066666 + flags = 67108864 data = length 711, hash 1BFD9623 sample 64: - time = 2166666 + time = 12166666 flags = 0 data = length 2389, hash DA32F47A sample 65: - time = 2133333 - flags = 0 + time = 12133333 + flags = 67108864 data = length 924, hash A52273EB sample 66: - time = 2300000 + time = 12300000 flags = 0 data = length 18090, hash F9B61BB4 sample 67: - time = 2233333 + time = 12233333 flags = 0 data = length 1722, hash 21765422 sample 68: - time = 2200000 - flags = 0 + time = 12200000 + flags = 67108864 data = length 1002, hash ECD9E400 sample 69: - time = 2266666 - flags = 0 + time = 12266666 + flags = 67108864 data = length 483, hash 7D7026EF sample 70: - time = 2366666 + time = 12366666 flags = 0 data = length 3148, hash 36BC693B sample 71: - time = 2333333 - flags = 0 + time = 12333333 + flags = 67108864 data = length 751, hash 4128839C sample 72: - time = 2466666 + time = 12466666 flags = 0 data = length 5143, hash 29EB5CBF sample 73: - time = 2400000 + time = 12400000 flags = 0 data = length 1262, hash 2E678B4A sample 74: - time = 2433333 - flags = 0 + time = 12433333 + flags = 67108864 data = length 2051, hash 673F7005 sample 75: - time = 2500000 + time = 12500000 flags = 0 data = length 5527, hash 63E719BB sample 76: - time = 2633333 + time = 12633333 flags = 0 data = length 9411, hash 7EB7F147 sample 77: - time = 2566666 + time = 12566666 flags = 0 data = length 1644, hash 6890A9E sample 78: - time = 2533333 - flags = 0 + time = 12533333 + flags = 67108864 data = length 2066, hash 3E108A79 sample 79: - time = 2600000 - flags = 0 + time = 12600000 + flags = 67108864 data = length 357, hash 9F3F46CA sample 80: - time = 2733333 + time = 12733333 flags = 0 data = length 3240, hash EAB2D17E sample 81: - time = 2666666 + time = 12666666 flags = 0 data = length 552, hash 9EB5AB6F sample 82: - time = 2700000 - flags = 0 + time = 12700000 + flags = 67108864 data = length 267, hash 4DE266C7 sample 83: - time = 2833333 + time = 12833333 flags = 0 data = length 2412, hash F7E0D2E3 sample 84: - time = 2766666 + time = 12766666 flags = 0 data = length 442, hash 84A6016C sample 85: - time = 2800000 - flags = 0 + time = 12800000 + flags = 67108864 data = length 246, hash 15E324CB sample 86: - time = 2933333 + time = 12933333 flags = 0 data = length 2859, hash A5F29E33 sample 87: - time = 2866666 + time = 12866666 flags = 0 data = length 456, hash AE0BFF63 sample 88: - time = 2900000 - flags = 0 + time = 12900000 + flags = 67108864 data = length 329, hash A7C440BB sample 89: - time = 3033333 + time = 13033333 flags = 0 data = length 3575, hash 854F101F sample 90: - time = 2966666 + time = 12966666 flags = 0 data = length 644, hash 3971EF67 sample 91: - time = 3000000 - flags = 0 + time = 13000000 + flags = 67108864 data = length 379, hash 957A7C64 sample 92: - time = 3166666 + time = 13166666 flags = 0 data = length 3171, hash E6980453 sample 93: - time = 3100000 + time = 13100000 flags = 0 data = length 571, hash 68AEF974 sample 94: - time = 3066666 - flags = 0 + time = 13066666 + flags = 67108864 data = length 515, hash 4AF1201B sample 95: - time = 3133333 - flags = 0 + time = 13133333 + flags = 67108864 data = length 314, hash 2FC434E9 sample 96: - time = 3300000 + time = 13300000 flags = 0 data = length 12330, hash 3163E200 sample 97: - time = 3233333 + time = 13233333 flags = 0 data = length 954, hash 4C595D28 sample 98: - time = 3200000 - flags = 0 + time = 13200000 + flags = 67108864 data = length 543, hash 254B4BFA sample 99: - time = 3266666 - flags = 0 + time = 13266666 + flags = 67108864 data = length 288, hash A34C978F sample 100: - time = 3433333 + time = 13433333 flags = 0 data = length 3475, hash 45B06B96 sample 101: - time = 3366666 + time = 13366666 flags = 0 data = length 703, hash DAFBF3D0 sample 102: - time = 3333333 - flags = 0 + time = 13333333 + flags = 67108864 data = length 455, hash A17E5A07 sample 103: - time = 3400000 - flags = 0 + time = 13400000 + flags = 67108864 data = length 348, hash A6260498 sample 104: - time = 3566666 + time = 13566666 flags = 0 data = length 2441, hash 5E47866A sample 105: - time = 3500000 + time = 13500000 flags = 0 data = length 492, hash 44DCDBBA sample 106: - time = 3466666 - flags = 0 + time = 13466666 + flags = 67108864 data = length 349, hash 603A6F91 sample 107: - time = 3533333 - flags = 0 + time = 13533333 + flags = 67108864 data = length 180, hash B6EA9F51 sample 108: - time = 3700000 + time = 13700000 flags = 0 data = length 1032, hash B8014F9A sample 109: - time = 3633333 + time = 13633333 flags = 0 data = length 331, hash C02F3C83 sample 110: - time = 3600000 - flags = 0 + time = 13600000 + flags = 67108864 data = length 167, hash 9824D4E8 sample 111: - time = 3666666 - flags = 0 + time = 13666666 + flags = 67108864 data = length 194, hash 2DC7F1D7 sample 112: - time = 3833333 + time = 13833333 flags = 0 data = length 1149, hash 3471EA3E sample 113: - time = 3766666 + time = 13766666 flags = 0 data = length 484, hash B33D9C67 sample 114: - time = 3733333 - flags = 0 + time = 13733333 + flags = 67108864 data = length 264, hash 8E6C96EF sample 115: - time = 3800000 - flags = 0 + time = 13800000 + flags = 67108864 data = length 294, hash 36C8898C sample 116: - time = 3966666 + time = 13966666 flags = 0 data = length 1023, hash AB618F21 sample 117: - time = 3900000 + time = 13900000 flags = 0 data = length 559, hash 91CA32EA sample 118: - time = 3866666 - flags = 0 + time = 13866666 + flags = 67108864 data = length 197, hash AA87B8D8 sample 119: - time = 3933333 - flags = 0 + time = 13933333 + flags = 67108864 data = length 406, hash 2DE88209 tracksEnded = true diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_non_reference_3b.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_non_reference_3b.mp4.dump index b8da8459a16..09c403b1ce0 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_non_reference_3b.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_non_reference_3b.mp4.dump @@ -39,15 +39,15 @@ track 0: data = length 3689, hash C17F9AAD sample 2: time = 33333 - flags = 0 + flags = 67108864 data = length 573, hash AE0E0278 sample 3: time = 66666 - flags = 0 + flags = 67108864 data = length 1219, hash D94D73C6 sample 4: time = 100000 - flags = 0 + flags = 67108864 data = length 1575, hash 84E86A1D sample 5: time = 266666 @@ -55,15 +55,15 @@ track 0: data = length 23470, hash F5757CD0 sample 6: time = 166666 - flags = 0 + flags = 67108864 data = length 974, hash E45D44A4 sample 7: time = 200000 - flags = 0 + flags = 67108864 data = length 129, hash 21B833C0 sample 8: time = 233333 - flags = 0 + flags = 67108864 data = length 109, hash 1DBB1436 sample 9: time = 400000 @@ -71,15 +71,15 @@ track 0: data = length 716, hash 75873FC1 sample 10: time = 300000 - flags = 0 + flags = 67108864 data = length 226, hash 289041D3 sample 11: time = 333333 - flags = 0 + flags = 67108864 data = length 237, hash 4C108F08 sample 12: time = 366666 - flags = 0 + flags = 67108864 data = length 131, hash 8F6A446D sample 13: time = 533333 @@ -87,15 +87,15 @@ track 0: data = length 2303, hash 54F5847D sample 14: time = 433333 - flags = 0 + flags = 67108864 data = length 185, hash 5109747F sample 15: time = 466666 - flags = 0 + flags = 67108864 data = length 262, hash 24BD065A sample 16: time = 500000 - flags = 0 + flags = 67108864 data = length 421, hash F9F1FE52 sample 17: time = 666666 @@ -103,15 +103,15 @@ track 0: data = length 4384, hash 62A87795 sample 18: time = 566666 - flags = 0 + flags = 67108864 data = length 2370, hash F0A5ED86 sample 19: time = 600000 - flags = 0 + flags = 67108864 data = length 2204, hash B7522E5E sample 20: time = 633333 - flags = 0 + flags = 67108864 data = length 1780, hash 4C8582ED sample 21: time = 800000 @@ -119,15 +119,15 @@ track 0: data = length 5879, hash FE4332DA sample 22: time = 700000 - flags = 0 + flags = 67108864 data = length 2204, hash 9AB285B2 sample 23: time = 733333 - flags = 0 + flags = 67108864 data = length 2877, hash 139BF1F4 sample 24: time = 766666 - flags = 0 + flags = 67108864 data = length 2982, hash E9ADDDFA sample 25: time = 933333 @@ -135,15 +135,15 @@ track 0: data = length 7925, hash 177E51B5 sample 26: time = 833333 - flags = 0 + flags = 67108864 data = length 3139, hash 10486834 sample 27: time = 866666 - flags = 0 + flags = 67108864 data = length 2843, hash 78AF658C sample 28: time = 900000 - flags = 0 + flags = 67108864 data = length 1252, hash 5933C8B8 sample 29: time = 966666 @@ -159,15 +159,15 @@ track 0: data = length 2572, hash 7D158CDD sample 32: time = 1033333 - flags = 0 + flags = 67108864 data = length 713, hash F3649370 sample 33: time = 1066666 - flags = 0 + flags = 67108864 data = length 789, hash DFDC8672 sample 34: time = 1100000 - flags = 0 + flags = 67108864 data = length 587, hash A095E207 sample 35: time = 1266666 @@ -175,15 +175,15 @@ track 0: data = length 9906, hash A8997416 sample 36: time = 1166666 - flags = 0 + flags = 67108864 data = length 703, hash 653E1AAF sample 37: time = 1200000 - flags = 0 + flags = 67108864 data = length 1925, hash AC147859 sample 38: time = 1233333 - flags = 0 + flags = 67108864 data = length 1128, hash 39EB8BCE sample 39: time = 1400000 @@ -191,15 +191,15 @@ track 0: data = length 3082, hash E011B328 sample 40: time = 1300000 - flags = 0 + flags = 67108864 data = length 775, hash 67AE4345 sample 41: time = 1333333 - flags = 0 + flags = 67108864 data = length 702, hash AF1F22E6 sample 42: time = 1366666 - flags = 0 + flags = 67108864 data = length 392, hash FE873AA sample 43: time = 1533333 @@ -207,15 +207,15 @@ track 0: data = length 3293, hash 7DA83640 sample 44: time = 1433333 - flags = 0 + flags = 67108864 data = length 721, hash 896D4F13 sample 45: time = 1466666 - flags = 0 + flags = 67108864 data = length 725, hash 95CBF946 sample 46: time = 1500000 - flags = 0 + flags = 67108864 data = length 776, hash 5EB11334 sample 47: time = 1666666 @@ -223,15 +223,15 @@ track 0: data = length 1956, hash 80B2B533 sample 48: time = 1566666 - flags = 0 + flags = 67108864 data = length 592, hash 7C7832BF sample 49: time = 1600000 - flags = 0 + flags = 67108864 data = length 731, hash 92AA1FD2 sample 50: time = 1633333 - flags = 0 + flags = 67108864 data = length 518, hash 18E84AFC sample 51: time = 1800000 @@ -239,15 +239,15 @@ track 0: data = length 5411, hash DF6B9A19 sample 52: time = 1700000 - flags = 0 + flags = 67108864 data = length 565, hash A18215E sample 53: time = 1733333 - flags = 0 + flags = 67108864 data = length 916, hash 4F4089C8 sample 54: time = 1766666 - flags = 0 + flags = 67108864 data = length 1651, hash 893D58B4 sample 55: time = 1933333 @@ -255,15 +255,15 @@ track 0: data = length 5170, hash 3D0964F5 sample 56: time = 1833333 - flags = 0 + flags = 67108864 data = length 1448, hash 3978CED5 sample 57: time = 1866666 - flags = 0 + flags = 67108864 data = length 1797, hash BB343C33 sample 58: time = 1900000 - flags = 0 + flags = 67108864 data = length 1650, hash BDB93BB sample 59: time = 1966666 @@ -279,15 +279,15 @@ track 0: data = length 7531, hash 1768FC1C sample 62: time = 2033333 - flags = 0 + flags = 67108864 data = length 2405, hash 82938115 sample 63: time = 2066666 - flags = 0 + flags = 67108864 data = length 2377, hash 5677E8DB sample 64: time = 2100000 - flags = 0 + flags = 67108864 data = length 2135, hash B0F9E72D sample 65: time = 2266666 @@ -295,15 +295,15 @@ track 0: data = length 12031, hash 4F954746 sample 66: time = 2166666 - flags = 0 + flags = 67108864 data = length 1790, hash 517CF8F5 sample 67: time = 2200000 - flags = 0 + flags = 67108864 data = length 2014, hash CC430DAF sample 68: time = 2233333 - flags = 0 + flags = 67108864 data = length 1192, hash 343BF233 sample 69: time = 2400000 @@ -311,15 +311,15 @@ track 0: data = length 5118, hash A4CA24FB sample 70: time = 2300000 - flags = 0 + flags = 67108864 data = length 1060, hash 4116AF37 sample 71: time = 2333333 - flags = 0 + flags = 67108864 data = length 1345, hash E94F739C sample 72: time = 2366666 - flags = 0 + flags = 67108864 data = length 1558, hash 2FB36B5A sample 73: time = 2533333 @@ -327,15 +327,15 @@ track 0: data = length 8004, hash 79ACB806 sample 74: time = 2433333 - flags = 0 + flags = 67108864 data = length 2858, hash EBE6B4A4 sample 75: time = 2466666 - flags = 0 + flags = 67108864 data = length 4289, hash FEBF0899 sample 76: time = 2500000 - flags = 0 + flags = 67108864 data = length 4678, hash 758EE78D sample 77: time = 2666666 @@ -343,15 +343,15 @@ track 0: data = length 6554, hash 6717D949 sample 78: time = 2566666 - flags = 0 + flags = 67108864 data = length 2376, hash F768DB52 sample 79: time = 2600000 - flags = 0 + flags = 67108864 data = length 1532, hash E205FE8A sample 80: time = 2633333 - flags = 0 + flags = 67108864 data = length 1366, hash 4202F62D sample 81: time = 2800000 @@ -359,15 +359,15 @@ track 0: data = length 3253, hash F406909B sample 82: time = 2700000 - flags = 0 + flags = 67108864 data = length 749, hash 68F6F14C sample 83: time = 2733333 - flags = 0 + flags = 67108864 data = length 946, hash 43364529 sample 84: time = 2766666 - flags = 0 + flags = 67108864 data = length 593, hash 6E23A558 sample 85: time = 2933333 @@ -375,15 +375,15 @@ track 0: data = length 2917, hash 1789092B sample 86: time = 2833333 - flags = 0 + flags = 67108864 data = length 644, hash 6C511C36 sample 87: time = 2866666 - flags = 0 + flags = 67108864 data = length 719, hash 6E2C34E3 sample 88: time = 2900000 - flags = 0 + flags = 67108864 data = length 594, hash F4C056C4 sample 89: time = 2966666 @@ -399,15 +399,15 @@ track 0: data = length 3349, hash C1D93B2C sample 92: time = 3033333 - flags = 0 + flags = 67108864 data = length 863, hash EC21EA13 sample 93: time = 3066666 - flags = 0 + flags = 67108864 data = length 807, hash C5533997 sample 94: time = 3100000 - flags = 0 + flags = 67108864 data = length 430, hash 918D1401 sample 95: time = 3266666 @@ -415,15 +415,15 @@ track 0: data = length 4883, hash 342581AD sample 96: time = 3166666 - flags = 0 + flags = 67108864 data = length 431, hash 84FF1E7C sample 97: time = 3200000 - flags = 0 + flags = 67108864 data = length 1272, hash C7833EE5 sample 98: time = 3233333 - flags = 0 + flags = 67108864 data = length 821, hash 2F5B3EC3 sample 99: time = 3400000 @@ -431,15 +431,15 @@ track 0: data = length 2214, hash 1F4632E2 sample 100: time = 3300000 - flags = 0 + flags = 67108864 data = length 530, hash AF4AC2BE sample 101: time = 3333333 - flags = 0 + flags = 67108864 data = length 681, hash 557F3F5F sample 102: time = 3366666 - flags = 0 + flags = 67108864 data = length 447, hash 8A6157D2 sample 103: time = 3533333 @@ -447,15 +447,15 @@ track 0: data = length 2523, hash 4D9C62B1 sample 104: time = 3433333 - flags = 0 + flags = 67108864 data = length 559, hash 7E751737 sample 105: time = 3466666 - flags = 0 + flags = 67108864 data = length 650, hash 66286CB2 sample 106: time = 3500000 - flags = 0 + flags = 67108864 data = length 603, hash 9948D320 sample 107: time = 3666666 @@ -463,15 +463,15 @@ track 0: data = length 1203, hash BEDA938F sample 108: time = 3566666 - flags = 0 + flags = 67108864 data = length 287, hash 97219C89 sample 109: time = 3600000 - flags = 0 + flags = 67108864 data = length 401, hash 524D4D75 sample 110: time = 3633333 - flags = 0 + flags = 67108864 data = length 403, hash C138F2A sample 111: time = 3800000 @@ -479,15 +479,15 @@ track 0: data = length 1328, hash F1B32991 sample 112: time = 3700000 - flags = 0 + flags = 67108864 data = length 453, hash 1FCC9636 sample 113: time = 3733333 - flags = 0 + flags = 67108864 data = length 422, hash 19573075 sample 114: time = 3766666 - flags = 0 + flags = 67108864 data = length 322, hash 6752D2AB sample 115: time = 3933333 @@ -495,15 +495,15 @@ track 0: data = length 1128, hash 3B04FE26 sample 116: time = 3833333 - flags = 0 + flags = 67108864 data = length 404, hash F0BF973E sample 117: time = 3866666 - flags = 0 + flags = 67108864 data = length 511, hash FB98417B sample 118: time = 3900000 - flags = 0 + flags = 67108864 data = length 458, hash 2440801E sample 119: time = 3966666 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_non_reference_3b.mp4_fragmented.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_non_reference_3b.mp4_fragmented.dump index eb00bb762c5..72c26bc9b4b 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_non_reference_3b.mp4_fragmented.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_non_reference_3b.mp4_fragmented.dump @@ -32,15 +32,15 @@ track 0: data = length 3689, hash C17F9AAD sample 2: time = 33333 - flags = 0 + flags = 67108864 data = length 573, hash AE0E0278 sample 3: time = 66666 - flags = 0 + flags = 67108864 data = length 1219, hash D94D73C6 sample 4: time = 100000 - flags = 0 + flags = 67108864 data = length 1575, hash 84E86A1D sample 5: time = 266666 @@ -48,15 +48,15 @@ track 0: data = length 23470, hash F5757CD0 sample 6: time = 166666 - flags = 0 + flags = 67108864 data = length 974, hash E45D44A4 sample 7: time = 200000 - flags = 0 + flags = 67108864 data = length 129, hash 21B833C0 sample 8: time = 233333 - flags = 0 + flags = 67108864 data = length 109, hash 1DBB1436 sample 9: time = 400000 @@ -64,15 +64,15 @@ track 0: data = length 716, hash 75873FC1 sample 10: time = 300000 - flags = 0 + flags = 67108864 data = length 226, hash 289041D3 sample 11: time = 333333 - flags = 0 + flags = 67108864 data = length 237, hash 4C108F08 sample 12: time = 366666 - flags = 0 + flags = 67108864 data = length 131, hash 8F6A446D sample 13: time = 533333 @@ -80,15 +80,15 @@ track 0: data = length 2303, hash 54F5847D sample 14: time = 433333 - flags = 0 + flags = 67108864 data = length 185, hash 5109747F sample 15: time = 466666 - flags = 0 + flags = 67108864 data = length 262, hash 24BD065A sample 16: time = 500000 - flags = 0 + flags = 67108864 data = length 421, hash F9F1FE52 sample 17: time = 666666 @@ -96,15 +96,15 @@ track 0: data = length 4384, hash 62A87795 sample 18: time = 566666 - flags = 0 + flags = 67108864 data = length 2370, hash F0A5ED86 sample 19: time = 600000 - flags = 0 + flags = 67108864 data = length 2204, hash B7522E5E sample 20: time = 633333 - flags = 0 + flags = 67108864 data = length 1780, hash 4C8582ED sample 21: time = 800000 @@ -112,15 +112,15 @@ track 0: data = length 5879, hash FE4332DA sample 22: time = 700000 - flags = 0 + flags = 67108864 data = length 2204, hash 9AB285B2 sample 23: time = 733333 - flags = 0 + flags = 67108864 data = length 2877, hash 139BF1F4 sample 24: time = 766666 - flags = 0 + flags = 67108864 data = length 2982, hash E9ADDDFA sample 25: time = 933333 @@ -128,15 +128,15 @@ track 0: data = length 7925, hash 177E51B5 sample 26: time = 833333 - flags = 0 + flags = 67108864 data = length 3139, hash 10486834 sample 27: time = 866666 - flags = 0 + flags = 67108864 data = length 2843, hash 78AF658C sample 28: time = 900000 - flags = 0 + flags = 67108864 data = length 1252, hash 5933C8B8 sample 29: time = 966666 @@ -152,15 +152,15 @@ track 0: data = length 2572, hash 7D158CDD sample 32: time = 1033333 - flags = 0 + flags = 67108864 data = length 713, hash F3649370 sample 33: time = 1066666 - flags = 0 + flags = 67108864 data = length 789, hash DFDC8672 sample 34: time = 1100000 - flags = 0 + flags = 67108864 data = length 587, hash A095E207 sample 35: time = 1266666 @@ -168,15 +168,15 @@ track 0: data = length 9906, hash A8997416 sample 36: time = 1166666 - flags = 0 + flags = 67108864 data = length 703, hash 653E1AAF sample 37: time = 1200000 - flags = 0 + flags = 67108864 data = length 1925, hash AC147859 sample 38: time = 1233333 - flags = 0 + flags = 67108864 data = length 1128, hash 39EB8BCE sample 39: time = 1400000 @@ -184,15 +184,15 @@ track 0: data = length 3082, hash E011B328 sample 40: time = 1300000 - flags = 0 + flags = 67108864 data = length 775, hash 67AE4345 sample 41: time = 1333333 - flags = 0 + flags = 67108864 data = length 702, hash AF1F22E6 sample 42: time = 1366666 - flags = 0 + flags = 67108864 data = length 392, hash FE873AA sample 43: time = 1533333 @@ -200,15 +200,15 @@ track 0: data = length 3293, hash 7DA83640 sample 44: time = 1433333 - flags = 0 + flags = 67108864 data = length 721, hash 896D4F13 sample 45: time = 1466666 - flags = 0 + flags = 67108864 data = length 725, hash 95CBF946 sample 46: time = 1500000 - flags = 0 + flags = 67108864 data = length 776, hash 5EB11334 sample 47: time = 1666666 @@ -216,15 +216,15 @@ track 0: data = length 1956, hash 80B2B533 sample 48: time = 1566666 - flags = 0 + flags = 67108864 data = length 592, hash 7C7832BF sample 49: time = 1600000 - flags = 0 + flags = 67108864 data = length 731, hash 92AA1FD2 sample 50: time = 1633333 - flags = 0 + flags = 67108864 data = length 518, hash 18E84AFC sample 51: time = 1800000 @@ -232,15 +232,15 @@ track 0: data = length 5411, hash DF6B9A19 sample 52: time = 1700000 - flags = 0 + flags = 67108864 data = length 565, hash A18215E sample 53: time = 1733333 - flags = 0 + flags = 67108864 data = length 916, hash 4F4089C8 sample 54: time = 1766666 - flags = 0 + flags = 67108864 data = length 1651, hash 893D58B4 sample 55: time = 1933333 @@ -248,15 +248,15 @@ track 0: data = length 5170, hash 3D0964F5 sample 56: time = 1833333 - flags = 0 + flags = 67108864 data = length 1448, hash 3978CED5 sample 57: time = 1866666 - flags = 0 + flags = 67108864 data = length 1797, hash BB343C33 sample 58: time = 1900000 - flags = 0 + flags = 67108864 data = length 1650, hash BDB93BB sample 59: time = 1966666 @@ -272,15 +272,15 @@ track 0: data = length 7531, hash 1768FC1C sample 62: time = 2033333 - flags = 0 + flags = 67108864 data = length 2405, hash 82938115 sample 63: time = 2066666 - flags = 0 + flags = 67108864 data = length 2377, hash 5677E8DB sample 64: time = 2100000 - flags = 0 + flags = 67108864 data = length 2135, hash B0F9E72D sample 65: time = 2266666 @@ -288,15 +288,15 @@ track 0: data = length 12031, hash 4F954746 sample 66: time = 2166666 - flags = 0 + flags = 67108864 data = length 1790, hash 517CF8F5 sample 67: time = 2200000 - flags = 0 + flags = 67108864 data = length 2014, hash CC430DAF sample 68: time = 2233333 - flags = 0 + flags = 67108864 data = length 1192, hash 343BF233 sample 69: time = 2400000 @@ -304,15 +304,15 @@ track 0: data = length 5118, hash A4CA24FB sample 70: time = 2300000 - flags = 0 + flags = 67108864 data = length 1060, hash 4116AF37 sample 71: time = 2333333 - flags = 0 + flags = 67108864 data = length 1345, hash E94F739C sample 72: time = 2366666 - flags = 0 + flags = 67108864 data = length 1558, hash 2FB36B5A sample 73: time = 2533333 @@ -320,15 +320,15 @@ track 0: data = length 8004, hash 79ACB806 sample 74: time = 2433333 - flags = 0 + flags = 67108864 data = length 2858, hash EBE6B4A4 sample 75: time = 2466666 - flags = 0 + flags = 67108864 data = length 4289, hash FEBF0899 sample 76: time = 2500000 - flags = 0 + flags = 67108864 data = length 4678, hash 758EE78D sample 77: time = 2666666 @@ -336,15 +336,15 @@ track 0: data = length 6554, hash 6717D949 sample 78: time = 2566666 - flags = 0 + flags = 67108864 data = length 2376, hash F768DB52 sample 79: time = 2600000 - flags = 0 + flags = 67108864 data = length 1532, hash E205FE8A sample 80: time = 2633333 - flags = 0 + flags = 67108864 data = length 1366, hash 4202F62D sample 81: time = 2800000 @@ -352,15 +352,15 @@ track 0: data = length 3253, hash F406909B sample 82: time = 2700000 - flags = 0 + flags = 67108864 data = length 749, hash 68F6F14C sample 83: time = 2733333 - flags = 0 + flags = 67108864 data = length 946, hash 43364529 sample 84: time = 2766666 - flags = 0 + flags = 67108864 data = length 593, hash 6E23A558 sample 85: time = 2933333 @@ -368,15 +368,15 @@ track 0: data = length 2917, hash 1789092B sample 86: time = 2833333 - flags = 0 + flags = 67108864 data = length 644, hash 6C511C36 sample 87: time = 2866666 - flags = 0 + flags = 67108864 data = length 719, hash 6E2C34E3 sample 88: time = 2900000 - flags = 0 + flags = 67108864 data = length 594, hash F4C056C4 sample 89: time = 2966666 @@ -392,15 +392,15 @@ track 0: data = length 3349, hash C1D93B2C sample 92: time = 3033333 - flags = 0 + flags = 67108864 data = length 863, hash EC21EA13 sample 93: time = 3066666 - flags = 0 + flags = 67108864 data = length 807, hash C5533997 sample 94: time = 3100000 - flags = 0 + flags = 67108864 data = length 430, hash 918D1401 sample 95: time = 3266666 @@ -408,15 +408,15 @@ track 0: data = length 4883, hash 342581AD sample 96: time = 3166666 - flags = 0 + flags = 67108864 data = length 431, hash 84FF1E7C sample 97: time = 3200000 - flags = 0 + flags = 67108864 data = length 1272, hash C7833EE5 sample 98: time = 3233333 - flags = 0 + flags = 67108864 data = length 821, hash 2F5B3EC3 sample 99: time = 3400000 @@ -424,15 +424,15 @@ track 0: data = length 2214, hash 1F4632E2 sample 100: time = 3300000 - flags = 0 + flags = 67108864 data = length 530, hash AF4AC2BE sample 101: time = 3333333 - flags = 0 + flags = 67108864 data = length 681, hash 557F3F5F sample 102: time = 3366666 - flags = 0 + flags = 67108864 data = length 447, hash 8A6157D2 sample 103: time = 3533333 @@ -440,15 +440,15 @@ track 0: data = length 2523, hash 4D9C62B1 sample 104: time = 3433333 - flags = 0 + flags = 67108864 data = length 559, hash 7E751737 sample 105: time = 3466666 - flags = 0 + flags = 67108864 data = length 650, hash 66286CB2 sample 106: time = 3500000 - flags = 0 + flags = 67108864 data = length 603, hash 9948D320 sample 107: time = 3666666 @@ -456,15 +456,15 @@ track 0: data = length 1203, hash BEDA938F sample 108: time = 3566666 - flags = 0 + flags = 67108864 data = length 287, hash 97219C89 sample 109: time = 3600000 - flags = 0 + flags = 67108864 data = length 401, hash 524D4D75 sample 110: time = 3633333 - flags = 0 + flags = 67108864 data = length 403, hash C138F2A sample 111: time = 3800000 @@ -472,15 +472,15 @@ track 0: data = length 1328, hash F1B32991 sample 112: time = 3700000 - flags = 0 + flags = 67108864 data = length 453, hash 1FCC9636 sample 113: time = 3733333 - flags = 0 + flags = 67108864 data = length 422, hash 19573075 sample 114: time = 3766666 - flags = 0 + flags = 67108864 data = length 322, hash 6752D2AB sample 115: time = 3933333 @@ -488,15 +488,15 @@ track 0: data = length 1128, hash 3B04FE26 sample 116: time = 3833333 - flags = 0 + flags = 67108864 data = length 404, hash F0BF973E sample 117: time = 3866666 - flags = 0 + flags = 67108864 data = length 511, hash FB98417B sample 118: time = 3900000 - flags = 0 + flags = 67108864 data = length 458, hash 2440801E sample 119: time = 3966666 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_pyramid_3b.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_pyramid_3b.mp4.dump index 2729e779073..de65bbd9062 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_pyramid_3b.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_pyramid_3b.mp4.dump @@ -43,11 +43,11 @@ track 0: data = length 1319, hash 7E6541DB sample 3: time = 33333 - flags = 0 + flags = 67108864 data = length 466, hash ECD93C68 sample 4: time = 100000 - flags = 0 + flags = 67108864 data = length 785, hash 3E824588 sample 5: time = 266666 @@ -59,11 +59,11 @@ track 0: data = length 169, hash B408CF0C sample 7: time = 166666 - flags = 0 + flags = 67108864 data = length 1008, hash 558016DF sample 8: time = 233333 - flags = 0 + flags = 67108864 data = length 60, hash A5220F0F sample 9: time = 400000 @@ -75,11 +75,11 @@ track 0: data = length 277, hash F27B77B0 sample 11: time = 300000 - flags = 0 + flags = 67108864 data = length 204, hash 451A7EF8 sample 12: time = 366666 - flags = 0 + flags = 67108864 data = length 100, hash 72CCF451 sample 13: time = 533333 @@ -91,11 +91,11 @@ track 0: data = length 313, hash 23FD6C99 sample 15: time = 433333 - flags = 0 + flags = 67108864 data = length 161, hash F9082751 sample 16: time = 500000 - flags = 0 + flags = 67108864 data = length 512, hash F0317F7B sample 17: time = 666666 @@ -107,11 +107,11 @@ track 0: data = length 2418, hash 3CA55886 sample 19: time = 566666 - flags = 0 + flags = 67108864 data = length 818, hash CF732FE9 sample 20: time = 633333 - flags = 0 + flags = 67108864 data = length 931, hash 9307BF9B sample 21: time = 800000 @@ -123,11 +123,11 @@ track 0: data = length 3241, hash C3722BD2 sample 23: time = 700000 - flags = 0 + flags = 67108864 data = length 1957, hash 54FE72A0 sample 24: time = 766666 - flags = 0 + flags = 67108864 data = length 943, hash 467A1DB1 sample 25: time = 933333 @@ -139,11 +139,11 @@ track 0: data = length 3313, hash B854BF3E sample 27: time = 833333 - flags = 0 + flags = 67108864 data = length 2729, hash FC827FEE sample 28: time = 900000 - flags = 0 + flags = 67108864 data = length 847, hash 739EFD75 sample 29: time = 966666 @@ -163,11 +163,11 @@ track 0: data = length 980, hash BD0ADFF1 sample 33: time = 1033333 - flags = 0 + flags = 67108864 data = length 343, hash 645A3973 sample 34: time = 1100000 - flags = 0 + flags = 67108864 data = length 476, hash 89BF903C sample 35: time = 1266666 @@ -179,11 +179,11 @@ track 0: data = length 2422, hash 863FCD89 sample 37: time = 1166666 - flags = 0 + flags = 67108864 data = length 718, hash E21DB793 sample 38: time = 1233333 - flags = 0 + flags = 67108864 data = length 475, hash 6E2B14C6 sample 39: time = 1400000 @@ -195,11 +195,11 @@ track 0: data = length 825, hash 978654C8 sample 41: time = 1300000 - flags = 0 + flags = 67108864 data = length 705, hash 28D04FCA sample 42: time = 1366666 - flags = 0 + flags = 67108864 data = length 335, hash D736EAC1 sample 43: time = 1533333 @@ -211,11 +211,11 @@ track 0: data = length 897, hash 6C76AFF1 sample 45: time = 1433333 - flags = 0 + flags = 67108864 data = length 499, hash DDBE39E sample 46: time = 1500000 - flags = 0 + flags = 67108864 data = length 565, hash D269272C sample 47: time = 1666666 @@ -227,11 +227,11 @@ track 0: data = length 871, hash 21A0191A sample 49: time = 1566666 - flags = 0 + flags = 67108864 data = length 363, hash 376C3CBE sample 50: time = 1633333 - flags = 0 + flags = 67108864 data = length 398, hash BA958C19 sample 51: time = 1800000 @@ -243,11 +243,11 @@ track 0: data = length 1124, hash D7ADA732 sample 53: time = 1700000 - flags = 0 + flags = 67108864 data = length 579, hash B6D6A34F sample 54: time = 1766666 - flags = 0 + flags = 67108864 data = length 1206, hash 1C5174A3 sample 55: time = 1933333 @@ -259,11 +259,11 @@ track 0: data = length 2045, hash 81261E50 sample 57: time = 1833333 - flags = 0 + flags = 67108864 data = length 915, hash FFEDB29E sample 58: time = 1900000 - flags = 0 + flags = 67108864 data = length 1513, hash C03809BF sample 59: time = 1966666 @@ -283,11 +283,11 @@ track 0: data = length 2740, hash 669E4DF1 sample 63: time = 2033333 - flags = 0 + flags = 67108864 data = length 1829, hash 66A7C400 sample 64: time = 2100000 - flags = 0 + flags = 67108864 data = length 1423, hash DD6288B3 sample 65: time = 2266666 @@ -299,11 +299,11 @@ track 0: data = length 2481, hash 9302A030 sample 67: time = 2166666 - flags = 0 + flags = 67108864 data = length 1770, hash 86E54271 sample 68: time = 2233333 - flags = 0 + flags = 67108864 data = length 636, hash B0BB7A9D sample 69: time = 2400000 @@ -315,11 +315,11 @@ track 0: data = length 1610, hash E9A0711D sample 71: time = 2300000 - flags = 0 + flags = 67108864 data = length 961, hash 494E3770 sample 72: time = 2366666 - flags = 0 + flags = 67108864 data = length 1402, hash 3D40A6D1 sample 73: time = 2533333 @@ -331,11 +331,11 @@ track 0: data = length 4620, hash 18719317 sample 75: time = 2433333 - flags = 0 + flags = 67108864 data = length 2507, hash 104771A0 sample 76: time = 2500000 - flags = 0 + flags = 67108864 data = length 3970, hash B52DD9E sample 77: time = 2666666 @@ -347,11 +347,11 @@ track 0: data = length 1824, hash 8BB687AC sample 79: time = 2566666 - flags = 0 + flags = 67108864 data = length 1851, hash 29CD7E58 sample 80: time = 2633333 - flags = 0 + flags = 67108864 data = length 731, hash 4E6BD69C sample 81: time = 2800000 @@ -363,11 +363,11 @@ track 0: data = length 1118, hash 77E855CA sample 83: time = 2700000 - flags = 0 + flags = 67108864 data = length 413, hash E83C433F sample 84: time = 2766666 - flags = 0 + flags = 67108864 data = length 514, hash 9ED8EF4B sample 85: time = 2933333 @@ -379,11 +379,11 @@ track 0: data = length 833, hash 213B7EAB sample 87: time = 2833333 - flags = 0 + flags = 67108864 data = length 574, hash 554020E9 sample 88: time = 2900000 - flags = 0 + flags = 67108864 data = length 431, hash 6C3E69F6 sample 89: time = 2966666 @@ -403,11 +403,11 @@ track 0: data = length 890, hash C384FE6A sample 93: time = 3033333 - flags = 0 + flags = 67108864 data = length 822, hash 2897A348 sample 94: time = 3100000 - flags = 0 + flags = 67108864 data = length 347, hash 968D3ED1 sample 95: time = 3266666 @@ -419,11 +419,11 @@ track 0: data = length 1714, hash 10B49A30 sample 97: time = 3166666 - flags = 0 + flags = 67108864 data = length 502, hash B0FDBCA sample 98: time = 3233333 - flags = 0 + flags = 67108864 data = length 378, hash AFCB47B5 sample 99: time = 3400000 @@ -435,11 +435,11 @@ track 0: data = length 785, hash 10C1C847 sample 101: time = 3300000 - flags = 0 + flags = 67108864 data = length 499, hash F11DD54 sample 102: time = 3366666 - flags = 0 + flags = 67108864 data = length 352, hash 32BD14FA sample 103: time = 3533333 @@ -451,11 +451,11 @@ track 0: data = length 732, hash 6BD8DF40 sample 105: time = 3433333 - flags = 0 + flags = 67108864 data = length 466, hash F2253523 sample 106: time = 3500000 - flags = 0 + flags = 67108864 data = length 519, hash 457EB20F sample 107: time = 3666666 @@ -467,11 +467,11 @@ track 0: data = length 483, hash 3C3E6CBA sample 109: time = 3566666 - flags = 0 + flags = 67108864 data = length 157, hash E3D5E025 sample 110: time = 3633333 - flags = 0 + flags = 67108864 data = length 237, hash A7217EC5 sample 111: time = 3800000 @@ -483,11 +483,11 @@ track 0: data = length 475, hash B9925752 sample 113: time = 3700000 - flags = 0 + flags = 67108864 data = length 392, hash 2709B2E0 sample 114: time = 3766666 - flags = 0 + flags = 67108864 data = length 258, hash 900651AF sample 115: time = 3933333 @@ -499,11 +499,11 @@ track 0: data = length 581, hash 79FAD374 sample 117: time = 3833333 - flags = 0 + flags = 67108864 data = length 285, hash B460CF8A sample 118: time = 3900000 - flags = 0 + flags = 67108864 data = length 277, hash A40F948C sample 119: time = 3966666 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_pyramid_3b.mp4_fragmented.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_pyramid_3b.mp4_fragmented.dump index 8efd9ec8d80..cc2fed24e59 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_pyramid_3b.mp4_fragmented.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/bbb_800x640_768kbps_30fps_avc_pyramid_3b.mp4_fragmented.dump @@ -36,11 +36,11 @@ track 0: data = length 1319, hash 7E6541DB sample 3: time = 33333 - flags = 0 + flags = 67108864 data = length 466, hash ECD93C68 sample 4: time = 100000 - flags = 0 + flags = 67108864 data = length 785, hash 3E824588 sample 5: time = 266666 @@ -52,11 +52,11 @@ track 0: data = length 169, hash B408CF0C sample 7: time = 166666 - flags = 0 + flags = 67108864 data = length 1008, hash 558016DF sample 8: time = 233333 - flags = 0 + flags = 67108864 data = length 60, hash A5220F0F sample 9: time = 400000 @@ -68,11 +68,11 @@ track 0: data = length 277, hash F27B77B0 sample 11: time = 300000 - flags = 0 + flags = 67108864 data = length 204, hash 451A7EF8 sample 12: time = 366666 - flags = 0 + flags = 67108864 data = length 100, hash 72CCF451 sample 13: time = 533333 @@ -84,11 +84,11 @@ track 0: data = length 313, hash 23FD6C99 sample 15: time = 433333 - flags = 0 + flags = 67108864 data = length 161, hash F9082751 sample 16: time = 500000 - flags = 0 + flags = 67108864 data = length 512, hash F0317F7B sample 17: time = 666666 @@ -100,11 +100,11 @@ track 0: data = length 2418, hash 3CA55886 sample 19: time = 566666 - flags = 0 + flags = 67108864 data = length 818, hash CF732FE9 sample 20: time = 633333 - flags = 0 + flags = 67108864 data = length 931, hash 9307BF9B sample 21: time = 800000 @@ -116,11 +116,11 @@ track 0: data = length 3241, hash C3722BD2 sample 23: time = 700000 - flags = 0 + flags = 67108864 data = length 1957, hash 54FE72A0 sample 24: time = 766666 - flags = 0 + flags = 67108864 data = length 943, hash 467A1DB1 sample 25: time = 933333 @@ -132,11 +132,11 @@ track 0: data = length 3313, hash B854BF3E sample 27: time = 833333 - flags = 0 + flags = 67108864 data = length 2729, hash FC827FEE sample 28: time = 900000 - flags = 0 + flags = 67108864 data = length 847, hash 739EFD75 sample 29: time = 966666 @@ -156,11 +156,11 @@ track 0: data = length 980, hash BD0ADFF1 sample 33: time = 1033333 - flags = 0 + flags = 67108864 data = length 343, hash 645A3973 sample 34: time = 1100000 - flags = 0 + flags = 67108864 data = length 476, hash 89BF903C sample 35: time = 1266666 @@ -172,11 +172,11 @@ track 0: data = length 2422, hash 863FCD89 sample 37: time = 1166666 - flags = 0 + flags = 67108864 data = length 718, hash E21DB793 sample 38: time = 1233333 - flags = 0 + flags = 67108864 data = length 475, hash 6E2B14C6 sample 39: time = 1400000 @@ -188,11 +188,11 @@ track 0: data = length 825, hash 978654C8 sample 41: time = 1300000 - flags = 0 + flags = 67108864 data = length 705, hash 28D04FCA sample 42: time = 1366666 - flags = 0 + flags = 67108864 data = length 335, hash D736EAC1 sample 43: time = 1533333 @@ -204,11 +204,11 @@ track 0: data = length 897, hash 6C76AFF1 sample 45: time = 1433333 - flags = 0 + flags = 67108864 data = length 499, hash DDBE39E sample 46: time = 1500000 - flags = 0 + flags = 67108864 data = length 565, hash D269272C sample 47: time = 1666666 @@ -220,11 +220,11 @@ track 0: data = length 871, hash 21A0191A sample 49: time = 1566666 - flags = 0 + flags = 67108864 data = length 363, hash 376C3CBE sample 50: time = 1633333 - flags = 0 + flags = 67108864 data = length 398, hash BA958C19 sample 51: time = 1800000 @@ -236,11 +236,11 @@ track 0: data = length 1124, hash D7ADA732 sample 53: time = 1700000 - flags = 0 + flags = 67108864 data = length 579, hash B6D6A34F sample 54: time = 1766666 - flags = 0 + flags = 67108864 data = length 1206, hash 1C5174A3 sample 55: time = 1933333 @@ -252,11 +252,11 @@ track 0: data = length 2045, hash 81261E50 sample 57: time = 1833333 - flags = 0 + flags = 67108864 data = length 915, hash FFEDB29E sample 58: time = 1900000 - flags = 0 + flags = 67108864 data = length 1513, hash C03809BF sample 59: time = 1966666 @@ -276,11 +276,11 @@ track 0: data = length 2740, hash 669E4DF1 sample 63: time = 2033333 - flags = 0 + flags = 67108864 data = length 1829, hash 66A7C400 sample 64: time = 2100000 - flags = 0 + flags = 67108864 data = length 1423, hash DD6288B3 sample 65: time = 2266666 @@ -292,11 +292,11 @@ track 0: data = length 2481, hash 9302A030 sample 67: time = 2166666 - flags = 0 + flags = 67108864 data = length 1770, hash 86E54271 sample 68: time = 2233333 - flags = 0 + flags = 67108864 data = length 636, hash B0BB7A9D sample 69: time = 2400000 @@ -308,11 +308,11 @@ track 0: data = length 1610, hash E9A0711D sample 71: time = 2300000 - flags = 0 + flags = 67108864 data = length 961, hash 494E3770 sample 72: time = 2366666 - flags = 0 + flags = 67108864 data = length 1402, hash 3D40A6D1 sample 73: time = 2533333 @@ -324,11 +324,11 @@ track 0: data = length 4620, hash 18719317 sample 75: time = 2433333 - flags = 0 + flags = 67108864 data = length 2507, hash 104771A0 sample 76: time = 2500000 - flags = 0 + flags = 67108864 data = length 3970, hash B52DD9E sample 77: time = 2666666 @@ -340,11 +340,11 @@ track 0: data = length 1824, hash 8BB687AC sample 79: time = 2566666 - flags = 0 + flags = 67108864 data = length 1851, hash 29CD7E58 sample 80: time = 2633333 - flags = 0 + flags = 67108864 data = length 731, hash 4E6BD69C sample 81: time = 2800000 @@ -356,11 +356,11 @@ track 0: data = length 1118, hash 77E855CA sample 83: time = 2700000 - flags = 0 + flags = 67108864 data = length 413, hash E83C433F sample 84: time = 2766666 - flags = 0 + flags = 67108864 data = length 514, hash 9ED8EF4B sample 85: time = 2933333 @@ -372,11 +372,11 @@ track 0: data = length 833, hash 213B7EAB sample 87: time = 2833333 - flags = 0 + flags = 67108864 data = length 574, hash 554020E9 sample 88: time = 2900000 - flags = 0 + flags = 67108864 data = length 431, hash 6C3E69F6 sample 89: time = 2966666 @@ -396,11 +396,11 @@ track 0: data = length 890, hash C384FE6A sample 93: time = 3033333 - flags = 0 + flags = 67108864 data = length 822, hash 2897A348 sample 94: time = 3100000 - flags = 0 + flags = 67108864 data = length 347, hash 968D3ED1 sample 95: time = 3266666 @@ -412,11 +412,11 @@ track 0: data = length 1714, hash 10B49A30 sample 97: time = 3166666 - flags = 0 + flags = 67108864 data = length 502, hash B0FDBCA sample 98: time = 3233333 - flags = 0 + flags = 67108864 data = length 378, hash AFCB47B5 sample 99: time = 3400000 @@ -428,11 +428,11 @@ track 0: data = length 785, hash 10C1C847 sample 101: time = 3300000 - flags = 0 + flags = 67108864 data = length 499, hash F11DD54 sample 102: time = 3366666 - flags = 0 + flags = 67108864 data = length 352, hash 32BD14FA sample 103: time = 3533333 @@ -444,11 +444,11 @@ track 0: data = length 732, hash 6BD8DF40 sample 105: time = 3433333 - flags = 0 + flags = 67108864 data = length 466, hash F2253523 sample 106: time = 3500000 - flags = 0 + flags = 67108864 data = length 519, hash 457EB20F sample 107: time = 3666666 @@ -460,11 +460,11 @@ track 0: data = length 483, hash 3C3E6CBA sample 109: time = 3566666 - flags = 0 + flags = 67108864 data = length 157, hash E3D5E025 sample 110: time = 3633333 - flags = 0 + flags = 67108864 data = length 237, hash A7217EC5 sample 111: time = 3800000 @@ -476,11 +476,11 @@ track 0: data = length 475, hash B9925752 sample 113: time = 3700000 - flags = 0 + flags = 67108864 data = length 392, hash 2709B2E0 sample 114: time = 3766666 - flags = 0 + flags = 67108864 data = length 258, hash 900651AF sample 115: time = 3933333 @@ -492,11 +492,11 @@ track 0: data = length 581, hash 79FAD374 sample 117: time = 3833333 - flags = 0 + flags = 67108864 data = length 285, hash B460CF8A sample 118: time = 3900000 - flags = 0 + flags = 67108864 data = length 277, hash A40F948C sample 119: time = 3966666 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/fragmented_mp4_with_metadata_track.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/fragmented_mp4_with_metadata_track.mp4.dump index 50fd27c8a3f..32a1f67e462 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/fragmented_mp4_with_metadata_track.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/fragmented_mp4_with_metadata_track.mp4.dump @@ -30,13 +30,17 @@ moov (537 bytes): mvex (40 bytes): trex (32 bytes): Data = length 24, hash C35D3183 -moof (136 bytes): +moof (156 bytes): mfhd (16 bytes): Data = length 8, hash 94446F02 - traf (112 bytes): + traf (132 bytes): tfhd (24 bytes): Data = length 16, hash D37153F6 + tfdt (20 bytes): + Data = length 12, hash F634ABE0 trun (80 bytes): - Data = length 72, hash 524AC512 + Data = length 72, hash E37B4F26 mdat (28 bytes): Data = length 20, hash B35D5DCB +mfra (67 bytes): + Data = length 59, hash 9E280F09 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/fragmented_mp4_with_unknown_track.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/fragmented_mp4_with_unknown_track.mp4.dump index 295f55c303c..7fe02ae3742 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/fragmented_mp4_with_unknown_track.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/fragmented_mp4_with_unknown_track.mp4.dump @@ -30,13 +30,17 @@ moov (511 bytes): mvex (40 bytes): trex (32 bytes): Data = length 24, hash C35D3183 -moof (136 bytes): +moof (156 bytes): mfhd (16 bytes): Data = length 8, hash 94446F02 - traf (112 bytes): + traf (132 bytes): tfhd (24 bytes): Data = length 16, hash D37153DC + tfdt (20 bytes): + Data = length 12, hash F634ABE0 trun (80 bytes): - Data = length 72, hash 9AA973FC + Data = length 72, hash 2BD9FE10 mdat (28 bytes): Data = length 20, hash B35D5DCB +mfra (67 bytes): + Data = length 59, hash 17B84963 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/h265_with_metadata_track.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/h265_with_metadata_track.mp4.dump index f0d2fa20a33..58050e509f5 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/h265_with_metadata_track.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/h265_with_metadata_track.mp4.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 100100 + duration = 100066 getPosition(0) = [[timeUs=0, position=400052]] getPosition(1) = [[timeUs=0, position=400052]] - getPosition(50050) = [[timeUs=0, position=400052]] - getPosition(100100) = [[timeUs=0, position=400052]] + getPosition(50033) = [[timeUs=0, position=400052]] + getPosition(100066) = [[timeUs=0, position=400052]] numberOfTracks = 3 track 0: total output bytes = 387 sample count = 3 - track duration = 100100 + track duration = 100066 format 0: averageBitrate = 30940 id = 1 @@ -32,7 +32,7 @@ track 0: track 1: total output bytes = 1726 sample count = 3 - track duration = 80700 + track duration = 80729 format 0: averageBitrate = 171041 peakBitrate = 192000 @@ -61,7 +61,7 @@ track 1: track 2: total output bytes = 678996 sample count = 3 - track duration = 100100 + track duration = 100066 format 0: averageBitrate = 54283853 id = 3 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/hdr10-720p.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/hdr10-720p.mp4.dump index 6a5ac23e42f..11e02f033ec 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/hdr10-720p.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/hdr10-720p.mp4.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 4236600 + duration = 4236588 getPosition(0) = [[timeUs=0, position=400052]] getPosition(1) = [[timeUs=0, position=400052], [timeUs=1002955, position=430177]] - getPosition(2118300) = [[timeUs=2003566, position=2562813], [timeUs=3003444, position=4426916]] - getPosition(4236600) = [[timeUs=4003277, position=6370233]] + getPosition(2118294) = [[timeUs=2003566, position=2562813], [timeUs=3003444, position=4426916]] + getPosition(4236588) = [[timeUs=4003277, position=6370233]] numberOfTracks = 2 track 0: total output bytes = 7944083 sample count = 127 - track duration = 4236600 + track duration = 4236588 format 0: averageBitrate = 15000907 id = 1 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/hdr10-720p.mp4_fragmented.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/hdr10-720p.mp4_fragmented.dump index 371f00e9af0..059f37e0918 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/hdr10-720p.mp4_fragmented.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/hdr10-720p.mp4_fragmented.dump @@ -550,783 +550,783 @@ track 1: initializationData: data = length 2, hash 560 sample 0: - time = 0 + time = 43000 flags = 1 data = length 682, hash 34C58E7C sample 1: - time = 21333 + time = 64333 flags = 1 data = length 846, hash 22FB31EF sample 2: - time = 42666 + time = 85666 flags = 1 data = length 757, hash F55DC63E sample 3: - time = 64000 + time = 107000 flags = 1 data = length 690, hash BD26CD74 sample 4: - time = 85333 + time = 128333 flags = 1 data = length 660, hash 5713CCCB sample 5: - time = 106666 + time = 149666 flags = 1 data = length 646, hash 74C2D10F sample 6: - time = 128000 + time = 171000 flags = 1 data = length 652, hash B49BACA1 sample 7: - time = 149333 + time = 192333 flags = 1 data = length 658, hash BF36BC46 sample 8: - time = 170666 + time = 213666 flags = 1 data = length 666, hash 69CBADD8 sample 9: - time = 192000 + time = 235000 flags = 1 data = length 668, hash 99204158 sample 10: - time = 213333 + time = 256333 flags = 1 data = length 671, hash AE181AC8 sample 11: - time = 234666 + time = 277666 flags = 1 data = length 678, hash C432ACF0 sample 12: - time = 256000 + time = 299000 flags = 1 data = length 682, hash C23E1F17 sample 13: - time = 277333 + time = 320333 flags = 1 data = length 685, hash B22AB49 sample 14: - time = 298666 + time = 341666 flags = 1 data = length 683, hash B073F447 sample 15: - time = 320000 + time = 363000 flags = 1 data = length 675, hash C7614E26 sample 16: - time = 341333 + time = 384333 flags = 1 data = length 679, hash 5B075ADF sample 17: - time = 362666 + time = 405666 flags = 1 data = length 687, hash E260C35B sample 18: - time = 384000 + time = 427000 flags = 1 data = length 687, hash 3C316334 sample 19: - time = 405333 + time = 448333 flags = 1 data = length 680, hash BCD8469 sample 20: - time = 426666 + time = 469666 flags = 1 data = length 681, hash CBD0BF0F sample 21: - time = 448000 + time = 491000 flags = 1 data = length 719, hash F7594265 sample 22: - time = 469333 + time = 512333 flags = 1 data = length 699, hash 4975812A sample 23: - time = 490666 + time = 533666 flags = 1 data = length 723, hash 11C7327E sample 24: - time = 512000 + time = 555000 flags = 1 data = length 765, hash DED05454 sample 25: - time = 533333 + time = 576333 flags = 1 data = length 785, hash 9E7C3FDD sample 26: - time = 554666 + time = 597666 flags = 1 data = length 728, hash B433E15E sample 27: - time = 576000 + time = 619000 flags = 1 data = length 708, hash ED0B3337 sample 28: - time = 597333 + time = 640333 flags = 1 data = length 642, hash 6B447435 sample 29: - time = 618666 + time = 661666 flags = 1 data = length 628, hash D59CB28F sample 30: - time = 640000 + time = 683000 flags = 1 data = length 594, hash BCC990B5 sample 31: - time = 661333 + time = 704333 flags = 1 data = length 622, hash C9AE9991 sample 32: - time = 682645 + time = 725645 flags = 1 data = length 633, hash 4B7D59B8 sample 33: - time = 703979 + time = 746979 flags = 1 data = length 661, hash A98CE814 sample 34: - time = 725312 + time = 768312 flags = 1 data = length 662, hash 9A3E0D79 sample 35: - time = 746645 + time = 789645 flags = 1 data = length 669, hash 2A0B67AC sample 36: - time = 767979 + time = 810979 flags = 1 data = length 691, hash C339C4EE sample 37: - time = 789312 + time = 832312 flags = 1 data = length 678, hash CF770B8C sample 38: - time = 810645 + time = 853645 flags = 1 data = length 678, hash 685F97BC sample 39: - time = 831979 + time = 874979 flags = 1 data = length 688, hash 7DC1FBD3 sample 40: - time = 853312 + time = 896312 flags = 1 data = length 684, hash F7D9FE89 sample 41: - time = 874645 + time = 917645 flags = 1 data = length 683, hash 5E3EA281 sample 42: - time = 895979 + time = 938979 flags = 1 data = length 675, hash F576AE6 sample 43: - time = 917312 + time = 960312 flags = 1 data = length 697, hash B0EBE204 sample 44: - time = 938645 + time = 981645 flags = 1 data = length 675, hash 3C928CCA sample 45: - time = 959979 + time = 1002979 flags = 1 data = length 680, hash 34650DF5 sample 46: - time = 981312 + time = 1024312 flags = 1 data = length 685, hash 564F62A sample 47: - time = 1002645 + time = 1045645 flags = 1 data = length 691, hash 71BBA88D sample 48: - time = 1023979 + time = 1066979 flags = 1 data = length 736, hash C8F0D575 sample 49: - time = 1045312 + time = 1088312 flags = 1 data = length 718, hash 2F13561A sample 50: - time = 1066645 + time = 1109645 flags = 1 data = length 692, hash CF153F84 sample 51: - time = 1087979 + time = 1130979 flags = 1 data = length 673, hash 30357B83 sample 52: - time = 1109312 + time = 1152312 flags = 1 data = length 690, hash 5FB65B72 sample 53: - time = 1130645 + time = 1173645 flags = 1 data = length 769, hash F5E1AAEA sample 54: - time = 1151979 + time = 1194979 flags = 1 data = length 733, hash 9327D738 sample 55: - time = 1173312 + time = 1216312 flags = 1 data = length 627, hash 203CFA24 sample 56: - time = 1194645 + time = 1237645 flags = 1 data = length 697, hash 1FCE39D0 sample 57: - time = 1215979 + time = 1258979 flags = 1 data = length 585, hash B53A076B sample 58: - time = 1237312 + time = 1280312 flags = 1 data = length 650, hash FDFCA752 sample 59: - time = 1258645 + time = 1301645 flags = 1 data = length 725, hash 4D4FA788 sample 60: - time = 1279979 + time = 1322979 flags = 1 data = length 788, hash 6D883F0B sample 61: - time = 1301312 + time = 1344312 flags = 1 data = length 649, hash 9125CC1A sample 62: - time = 1322645 + time = 1365645 flags = 1 data = length 616, hash C02AB7EA sample 63: - time = 1343979 + time = 1386979 flags = 1 data = length 624, hash D49000E1 sample 64: - time = 1365312 + time = 1408312 flags = 1 data = length 664, hash 482C9994 sample 65: - time = 1386645 + time = 1429645 flags = 1 data = length 656, hash A234172A sample 66: - time = 1407979 + time = 1450979 flags = 1 data = length 649, hash BCCAD04D sample 67: - time = 1429312 + time = 1472312 flags = 1 data = length 655, hash B961E395 sample 68: - time = 1450645 + time = 1493645 flags = 1 data = length 673, hash 5BD56013 sample 69: - time = 1471979 + time = 1514979 flags = 1 data = length 700, hash FE25D834 sample 70: - time = 1493312 + time = 1536312 flags = 1 data = length 668, hash 45203245 sample 71: - time = 1514645 + time = 1557645 flags = 1 data = length 672, hash F9269558 sample 72: - time = 1535979 + time = 1578979 flags = 1 data = length 682, hash C205B4DF sample 73: - time = 1557312 + time = 1600312 flags = 1 data = length 686, hash A4632474 sample 74: - time = 1578645 + time = 1621645 flags = 1 data = length 747, hash B2F3AA1D sample 75: - time = 1599979 + time = 1642979 flags = 1 data = length 711, hash B3A33D80 sample 76: - time = 1621312 + time = 1664312 flags = 1 data = length 652, hash 37A9B9BF sample 77: - time = 1642645 + time = 1685645 flags = 1 data = length 675, hash F6BE4CAC sample 78: - time = 1663979 + time = 1706979 flags = 1 data = length 672, hash 22A12DFC sample 79: - time = 1685312 + time = 1728312 flags = 1 data = length 674, hash E740F44 sample 80: - time = 1706645 + time = 1749645 flags = 1 data = length 680, hash A065804 sample 81: - time = 1727979 + time = 1770979 flags = 1 data = length 663, hash 805CE20 sample 82: - time = 1749312 + time = 1792312 flags = 1 data = length 688, hash C2E28B22 sample 83: - time = 1770645 + time = 1813645 flags = 1 data = length 672, hash BF738F27 sample 84: - time = 1791979 + time = 1834979 flags = 1 data = length 673, hash AFE85361 sample 85: - time = 1813312 + time = 1856312 flags = 1 data = length 679, hash C9D68F4F sample 86: - time = 1834645 + time = 1877645 flags = 1 data = length 676, hash 42C67933 sample 87: - time = 1855979 + time = 1898979 flags = 1 data = length 748, hash 16944018 sample 88: - time = 1877312 + time = 1920312 flags = 1 data = length 730, hash D592050C sample 89: - time = 1898645 + time = 1941645 flags = 1 data = length 785, hash DB11A4E8 sample 90: - time = 1919979 + time = 1962979 flags = 1 data = length 708, hash 445F4443 sample 91: - time = 1941312 + time = 1984312 flags = 1 data = length 630, hash BD57EF90 sample 92: - time = 1962645 + time = 2005645 flags = 1 data = length 621, hash FB977F1F sample 93: - time = 1983979 + time = 2026979 flags = 1 data = length 656, hash 53E25FBE sample 94: - time = 2005291 + time = 2048291 flags = 1 data = length 664, hash A9D0717 sample 95: - time = 2026625 + time = 2069625 flags = 1 data = length 672, hash 6F2663EA sample 96: - time = 2047958 + time = 2090958 flags = 1 data = length 677, hash 6EBB686B sample 97: - time = 2069291 + time = 2112291 flags = 1 data = length 679, hash BF29A1EC sample 98: - time = 2090625 + time = 2133625 flags = 1 data = length 683, hash 69F6750D sample 99: - time = 2111958 + time = 2154958 flags = 1 data = length 691, hash A79A804F sample 100: - time = 2133291 + time = 2176291 flags = 1 data = length 734, hash 31FB39E8 sample 101: - time = 2154625 + time = 2197625 flags = 1 data = length 657, hash F99E1ADC sample 102: - time = 2175958 + time = 2218958 flags = 1 data = length 659, hash FDC16724 sample 103: - time = 2197291 + time = 2240291 flags = 1 data = length 795, hash 23302539 sample 104: - time = 2218625 + time = 2261625 flags = 1 data = length 691, hash 5AA01A0 sample 105: - time = 2239958 + time = 2282958 flags = 1 data = length 640, hash A9A214AB sample 106: - time = 2261291 + time = 2304291 flags = 1 data = length 651, hash F3253A0E sample 107: - time = 2282625 + time = 2325625 flags = 1 data = length 652, hash 2D4DE02 sample 108: - time = 2303958 + time = 2346958 flags = 1 data = length 772, hash 16817D3A sample 109: - time = 2325291 + time = 2368291 flags = 1 data = length 756, hash 738E4C8D sample 110: - time = 2346625 + time = 2389625 flags = 1 data = length 781, hash 61372EAE sample 111: - time = 2367958 + time = 2410958 flags = 1 data = length 658, hash 83B5A955 sample 112: - time = 2389291 + time = 2432291 flags = 1 data = length 667, hash C3CF8AEF sample 113: - time = 2410625 + time = 2453625 flags = 1 data = length 768, hash C6534483 sample 114: - time = 2431958 + time = 2474958 flags = 1 data = length 688, hash 1C14B263 sample 115: - time = 2453291 + time = 2496291 flags = 1 data = length 599, hash 51CF483 sample 116: - time = 2474625 + time = 2517625 flags = 1 data = length 594, hash F290D460 sample 117: - time = 2495958 + time = 2538958 flags = 1 data = length 633, hash 262E26E6 sample 118: - time = 2517291 + time = 2560291 flags = 1 data = length 656, hash 9158E6A1 sample 119: - time = 2538625 + time = 2581625 flags = 1 data = length 668, hash 3AC6C8DF sample 120: - time = 2559958 + time = 2602958 flags = 1 data = length 667, hash DB111C93 sample 121: - time = 2581291 + time = 2624291 flags = 1 data = length 670, hash 5EA45C5E sample 122: - time = 2602625 + time = 2645625 flags = 1 data = length 663, hash 1CF1EC34 sample 123: - time = 2623958 + time = 2666958 flags = 1 data = length 673, hash 9609104 sample 124: - time = 2645291 + time = 2688291 flags = 1 data = length 704, hash D274E425 sample 125: - time = 2666625 + time = 2709625 flags = 1 data = length 681, hash 4D720ACE sample 126: - time = 2687958 + time = 2730958 flags = 1 data = length 682, hash C49E4619 sample 127: - time = 2709291 + time = 2752291 flags = 1 data = length 680, hash 1AB4733A sample 128: - time = 2730625 + time = 2773625 flags = 1 data = length 675, hash BA047E60 sample 129: - time = 2751958 + time = 2794958 flags = 1 data = length 688, hash 9679B8E9 sample 130: - time = 2773291 + time = 2816291 flags = 1 data = length 687, hash 57DBCD4 sample 131: - time = 2794625 + time = 2837625 flags = 1 data = length 680, hash 91BA9BF2 sample 132: - time = 2815958 + time = 2858958 flags = 1 data = length 757, hash 741D6330 sample 133: - time = 2837291 + time = 2880291 flags = 1 data = length 651, hash 60508D7D sample 134: - time = 2858625 + time = 2901625 flags = 1 data = length 679, hash 7A32FD22 sample 135: - time = 2879958 + time = 2922958 flags = 1 data = length 666, hash 98C3F963 sample 136: - time = 2901291 + time = 2944291 flags = 1 data = length 694, hash 59D9B67B sample 137: - time = 2922625 + time = 2965625 flags = 1 data = length 680, hash 6FA356DD sample 138: - time = 2943958 + time = 2986958 flags = 1 data = length 665, hash 3D7E32D9 sample 139: - time = 2965291 + time = 3008291 flags = 1 data = length 681, hash 2592B0DF sample 140: - time = 2986625 + time = 3029625 flags = 1 data = length 680, hash 2BA659D7 sample 141: - time = 3007958 + time = 3050958 flags = 1 data = length 667, hash 1E21B749 sample 142: - time = 3029291 + time = 3072291 flags = 1 data = length 683, hash 57E1A624 sample 143: - time = 3050625 + time = 3093625 flags = 1 data = length 673, hash B7216D34 sample 144: - time = 3071958 + time = 3114958 flags = 1 data = length 684, hash 2FDBEB3A sample 145: - time = 3093291 + time = 3136291 flags = 1 data = length 707, hash 1D528F18 sample 146: - time = 3114625 + time = 3157625 flags = 1 data = length 693, hash 24148721 sample 147: - time = 3135958 + time = 3178958 flags = 1 data = length 660, hash C89F9451 sample 148: - time = 3157291 + time = 3200291 flags = 1 data = length 679, hash 67C16179 sample 149: - time = 3178625 + time = 3221625 flags = 1 data = length 685, hash 6EF9DD57 sample 150: - time = 3199958 + time = 3242958 flags = 1 data = length 672, hash CFF4E296 sample 151: - time = 3221291 + time = 3264291 flags = 1 data = length 681, hash 994F630 sample 152: - time = 3242625 + time = 3285625 flags = 1 data = length 684, hash 3118D2E9 sample 153: - time = 3263958 + time = 3306958 flags = 1 data = length 677, hash 3628592F sample 154: - time = 3285291 + time = 3328291 flags = 1 data = length 689, hash 309E58A0 sample 155: - time = 3306625 + time = 3349625 flags = 1 data = length 677, hash D1F3255B sample 156: - time = 3327958 + time = 3370958 flags = 1 data = length 689, hash B3E864BA sample 157: - time = 3349270 + time = 3392270 flags = 1 data = length 680, hash 469FA2FF sample 158: - time = 3370604 + time = 3413604 flags = 1 data = length 688, hash DC9FC31B sample 159: - time = 3391937 + time = 3434937 flags = 1 data = length 675, hash E2396CC7 sample 160: - time = 3413270 + time = 3456270 flags = 1 data = length 738, hash C1B7A30A sample 161: - time = 3434604 + time = 3477604 flags = 1 data = length 723, hash CEABDA70 sample 162: - time = 3455937 + time = 3498937 flags = 1 data = length 698, hash 59E1B5D8 sample 163: - time = 3477270 + time = 3520270 flags = 1 data = length 671, hash 71CD7BFA sample 164: - time = 3498604 + time = 3541604 flags = 1 data = length 652, hash 45894636 sample 165: - time = 3519937 + time = 3562937 flags = 1 data = length 667, hash E98A528A sample 166: - time = 3541270 + time = 3584270 flags = 1 data = length 682, hash 8AA9E761 sample 167: - time = 3562604 + time = 3605604 flags = 1 data = length 670, hash 7D071859 sample 168: - time = 3583937 + time = 3626937 flags = 1 data = length 672, hash 4FA7BDBB sample 169: - time = 3605270 + time = 3648270 flags = 1 data = length 779, hash 85D8FF74 sample 170: - time = 3626604 + time = 3669604 flags = 1 data = length 699, hash CABC0AF6 sample 171: - time = 3647937 + time = 3690937 flags = 1 data = length 635, hash 35BD0FED sample 172: - time = 3669270 + time = 3712270 flags = 1 data = length 646, hash D4960FAC sample 173: - time = 3690604 + time = 3733604 flags = 1 data = length 669, hash 4DAC2897 sample 174: - time = 3711937 + time = 3754937 flags = 1 data = length 675, hash FD60998A sample 175: - time = 3733270 + time = 3776270 flags = 1 data = length 677, hash FED0180B sample 176: - time = 3754604 + time = 3797604 flags = 1 data = length 668, hash C6183862 sample 177: - time = 3775937 + time = 3818937 flags = 1 data = length 671, hash EBA9EF22 sample 178: - time = 3797270 + time = 3840270 flags = 1 data = length 668, hash CF88A2FF sample 179: - time = 3818604 + time = 3861604 flags = 1 data = length 727, hash A9311311 sample 180: - time = 3839937 + time = 3882937 flags = 1 data = length 701, hash C6351159 sample 181: - time = 3861270 + time = 3904270 flags = 1 data = length 847, hash 1864F774 sample 182: - time = 3882604 + time = 3925604 flags = 1 data = length 765, hash 616DD2EA sample 183: - time = 3903937 + time = 3946937 flags = 1 data = length 674, hash 671D4342 sample 184: - time = 3925270 + time = 3968270 flags = 1 data = length 723, hash 567566A2 sample 185: - time = 3946604 + time = 3989604 flags = 1 data = length 580, hash B38C9C63 sample 186: - time = 3967937 + time = 4010937 flags = 1 data = length 583, hash 5668BFCE sample 187: - time = 3989270 + time = 4032270 flags = 1 data = length 631, hash 7E86C98E sample 188: - time = 4010604 + time = 4053604 flags = 1 data = length 656, hash 95A41C9B sample 189: - time = 4031937 + time = 4074937 flags = 1 data = length 822, hash 2A045560 sample 190: - time = 4053270 + time = 4096270 flags = 1 data = length 643, hash 551E7C72 sample 191: - time = 4074604 + time = 4117604 flags = 1 data = length 617, hash 463482D9 sample 192: - time = 4095937 + time = 4138937 flags = 1 data = length 640, hash E714454F sample 193: - time = 4117270 + time = 4160270 flags = 1 data = length 646, hash 6DD5E81B sample 194: - time = 4138604 + time = 4181604 flags = 1 data = length 690, hash 407EC299 tracksEnded = true diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/hdr10-720p.mp4_fragmented_box_structure.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/hdr10-720p.mp4_fragmented_box_structure.dump index 90ff783072d..e7521a2fe80 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/hdr10-720p.mp4_fragmented_box_structure.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/hdr10-720p.mp4_fragmented_box_structure.dump @@ -58,33 +58,43 @@ moov (1209 bytes): Data = length 24, hash C35D3183 trex (32 bytes): Data = length 24, hash 14070F84 -moof (2924 bytes): +moof (2964 bytes): mfhd (16 bytes): Data = length 8, hash 94446F02 - traf (1132 bytes): + traf (1152 bytes): tfhd (24 bytes): Data = length 16, hash D37153D4 + tfdt (20 bytes): + Data = length 12, hash F634ABE0 trun (1100 bytes): - Data = length 1092, hash EB5519F9 - traf (1768 bytes): + Data = length 1092, hash 6DDDF022 + traf (1788 bytes): tfhd (24 bytes): Data = length 16, hash 67B5C2D5 + tfdt (20 bytes): + Data = length 12, hash F634ACE8 trun (1736 bytes): - Data = length 1728, hash C23EB3C1 + Data = length 1728, hash 8A81EFE9 mdat (5715098 bytes): Data = length 5715090, hash F05D5E74 -moof (1196 bytes): +moof (1236 bytes): mfhd (16 bytes): Data = length 8, hash 94446F03 - traf (496 bytes): + traf (516 bytes): tfhd (24 bytes): - Data = length 16, hash D372A231 + Data = length 16, hash D372A278 + tfdt (20 bytes): + Data = length 12, hash F634BE8A trun (464 bytes): - Data = length 456, hash 8DE68B27 - traf (676 bytes): + Data = length 456, hash 7A081F4F + traf (696 bytes): tfhd (24 bytes): - Data = length 16, hash 67B71132 + Data = length 16, hash 67B71179 + tfdt (20 bytes): + Data = length 12, hash F634BBAC trun (644 bytes): - Data = length 636, hash B5F6D197 + Data = length 636, hash 1F65A0BF mdat (2362210 bytes): Data = length 2362202, hash 99EB8832 +mfra (148 bytes): + Data = length 140, hash DDD500A8 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_auxiliary_tracks_in_axte.box.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_auxiliary_tracks_in_axte.box.dump index 144c3ea3693..2fa86c05eb8 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_auxiliary_tracks_in_axte.box.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_auxiliary_tracks_in_axte.box.dump @@ -1,6 +1,6 @@ ftyp (28 bytes): Data = length 20, hash EF896440 -moov (913 bytes): +moov (893 bytes): mvhd (108 bytes): Data = length 100, hash 2613A5C meta (191 bytes): @@ -10,40 +10,40 @@ moov (913 bytes): Data = length 70, hash C5147150 ilst (72 bytes): Data = length 64, hash 3B286CD9 - trak (606 bytes): + trak (586 bytes): tkhd (92 bytes): Data = length 84, hash 3D79758F - mdia (506 bytes): + mdia (486 bytes): mdhd (32 bytes): Data = length 24, hash 41542D81 hdlr (44 bytes): Data = length 36, hash A0852FF2 - minf (422 bytes): + minf (402 bytes): vmhd (20 bytes): Data = length 12, hash EE830681 dinf (36 bytes): Data = length 28, hash D535436B - stbl (358 bytes): + stbl (338 bytes): stsd (166 bytes): Data = length 158, hash 11532063 stts (24 bytes): Data = length 16, hash E534C287 - stsz (40 bytes): - Data = length 32, hash B3F09E + stsz (20 bytes): + Data = length 12, hash F1982ABE stsc (28 bytes): Data = length 20, hash 8F6E8285 co64 (56 bytes): Data = length 48, hash A4175473 stss (36 bytes): Data = length 28, hash 53024615 -free (399095 bytes): - Data = length 399087, hash A215F9DF +free (399115 bytes): + Data = length 399107, hash 859D545F mdat (296 bytes): Data = length 280, hash 8DCFD2E3 axte (400628 bytes): ftyp (28 bytes): Data = length 20, hash EF896440 - moov (1510 bytes): + moov (1470 bytes): mvhd (108 bytes): Data = length 100, hash 2613A5D meta (182 bytes): @@ -53,59 +53,59 @@ axte (400628 bytes): Data = length 72, hash 4E5C3894 ilst (61 bytes): Data = length 53, hash 75D49FBD - trak (606 bytes): + trak (586 bytes): tkhd (92 bytes): Data = length 84, hash 3D79758F - mdia (506 bytes): + mdia (486 bytes): mdhd (32 bytes): Data = length 24, hash 41542D81 hdlr (44 bytes): Data = length 36, hash A0852FF2 - minf (422 bytes): + minf (402 bytes): vmhd (20 bytes): Data = length 12, hash EE830681 dinf (36 bytes): Data = length 28, hash D535436B - stbl (358 bytes): + stbl (338 bytes): stsd (166 bytes): Data = length 158, hash 11532063 stts (24 bytes): Data = length 16, hash E534C287 - stsz (40 bytes): - Data = length 32, hash B3F09E + stsz (20 bytes): + Data = length 12, hash F1982ABE stsc (28 bytes): Data = length 20, hash 8F6E8285 co64 (56 bytes): Data = length 48, hash A4175473 stss (36 bytes): Data = length 28, hash 53024615 - trak (606 bytes): + trak (586 bytes): tkhd (92 bytes): Data = length 84, hash 2ECB0510 - mdia (506 bytes): + mdia (486 bytes): mdhd (32 bytes): Data = length 24, hash 41542D81 hdlr (44 bytes): Data = length 36, hash A0852FF2 - minf (422 bytes): + minf (402 bytes): vmhd (20 bytes): Data = length 12, hash EE830681 dinf (36 bytes): Data = length 28, hash D535436B - stbl (358 bytes): + stbl (338 bytes): stsd (166 bytes): Data = length 158, hash 11532063 stts (24 bytes): Data = length 16, hash E534C287 - stsz (40 bytes): - Data = length 32, hash B3F09E + stsz (20 bytes): + Data = length 12, hash F1982ABE stsc (28 bytes): Data = length 20, hash 8F6E8285 co64 (56 bytes): Data = length 48, hash 44DD22A5 stss (36 bytes): Data = length 28, hash 53024615 - free (398498 bytes): - Data = length 398490, hash B07980C1 + free (398538 bytes): + Data = length 398530, hash 1DB9EBC1 mdat (576 bytes): Data = length 560, hash 9E0D5FC5 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_auxiliary_tracks_without_axte.box.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_auxiliary_tracks_without_axte.box.dump index 0a16d740a7d..f8a191a7598 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_auxiliary_tracks_without_axte.box.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_auxiliary_tracks_without_axte.box.dump @@ -1,87 +1,87 @@ ftyp (28 bytes): Data = length 20, hash EF896440 -moov (1934 bytes): +moov (1874 bytes): mvhd (108 bytes): Data = length 100, hash 2613A5E - trak (606 bytes): + trak (586 bytes): tkhd (92 bytes): Data = length 84, hash 3D79758F - mdia (506 bytes): + mdia (486 bytes): mdhd (32 bytes): Data = length 24, hash 41542D81 hdlr (44 bytes): Data = length 36, hash A0852FF2 - minf (422 bytes): + minf (402 bytes): vmhd (20 bytes): Data = length 12, hash EE830681 dinf (36 bytes): Data = length 28, hash D535436B - stbl (358 bytes): + stbl (338 bytes): stsd (166 bytes): Data = length 158, hash 11532063 stts (24 bytes): Data = length 16, hash E534C287 - stsz (40 bytes): - Data = length 32, hash B3F09E + stsz (20 bytes): + Data = length 12, hash F1982ABE stsc (28 bytes): Data = length 20, hash 8F6E8285 co64 (56 bytes): Data = length 48, hash A4175473 stss (36 bytes): Data = length 28, hash 53024615 - trak (606 bytes): + trak (586 bytes): tkhd (92 bytes): Data = length 84, hash 2ECB0510 - mdia (506 bytes): + mdia (486 bytes): mdhd (32 bytes): Data = length 24, hash 41542D81 hdlr (44 bytes): Data = length 36, hash A0852FF2 - minf (422 bytes): + minf (402 bytes): vmhd (20 bytes): Data = length 12, hash EE830681 dinf (36 bytes): Data = length 28, hash D535436B - stbl (358 bytes): + stbl (338 bytes): stsd (166 bytes): Data = length 158, hash 11532063 stts (24 bytes): Data = length 16, hash E534C287 - stsz (40 bytes): - Data = length 32, hash B3F09E + stsz (20 bytes): + Data = length 12, hash F1982ABE stsc (28 bytes): Data = length 20, hash 8F6E8285 co64 (56 bytes): Data = length 48, hash 44DD22A5 stss (36 bytes): Data = length 28, hash 53024615 - trak (606 bytes): + trak (586 bytes): tkhd (92 bytes): Data = length 84, hash 201C9491 - mdia (506 bytes): + mdia (486 bytes): mdhd (32 bytes): Data = length 24, hash 41542D81 hdlr (44 bytes): Data = length 36, hash A0852FF2 - minf (422 bytes): + minf (402 bytes): vmhd (20 bytes): Data = length 12, hash EE830681 dinf (36 bytes): Data = length 28, hash D535436B - stbl (358 bytes): + stbl (338 bytes): stsd (166 bytes): Data = length 158, hash 11532063 stts (24 bytes): Data = length 16, hash E534C287 - stsz (40 bytes): - Data = length 32, hash B3F09E + stsz (20 bytes): + Data = length 12, hash F1982ABE stsc (28 bytes): Data = length 20, hash 8F6E8285 co64 (56 bytes): Data = length 48, hash 9F5A9CB8 stss (36 bytes): Data = length 28, hash 53024615 -free (398074 bytes): - Data = length 398066, hash B24945C1 +free (398134 bytes): + Data = length 398126, hash BDB9C641 mdat (856 bytes): Data = length 840, hash A66AA6A7 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_b_frame_large_pts.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_b_frame_large_pts.mp4.dump index 5b06d97ac51..95184a52bc6 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_b_frame_large_pts.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_b_frame_large_pts.mp4.dump @@ -9,7 +9,7 @@ numberOfTracks = 2 track 0: total output bytes = 224 sample count = 4 - track duration = 414900 + track duration = 414888 format 0: averageBitrate = 4319 id = 1 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_moov_at_the_end_and_free_box_at_start.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_moov_at_the_end_and_free_box_at_start.mp4.dump index 7dc40b0789c..bc5441c1a90 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_moov_at_the_end_and_free_box_at_start.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_moov_at_the_end_and_free_box_at_start.mp4.dump @@ -4,29 +4,29 @@ free (400008 bytes): Data = length 400000, hash BBC7B001 mdat (2800016 bytes): Data = length 2800000, hash 7A1CC21 -moov (872642 bytes): +moov (672642 bytes): mvhd (108 bytes): Data = length 100, hash AB629C6F - trak (872526 bytes): + trak (672526 bytes): tkhd (92 bytes): Data = length 84, hash 14345F22 - mdia (872426 bytes): + mdia (672426 bytes): mdhd (32 bytes): Data = length 24, hash 5864A5A4 hdlr (44 bytes): Data = length 36, hash A0852FF2 - minf (872342 bytes): + minf (672342 bytes): vmhd (20 bytes): Data = length 12, hash EE830681 dinf (36 bytes): Data = length 28, hash D535436B - stbl (872278 bytes): + stbl (672278 bytes): stsd (166 bytes): Data = length 158, hash 11532063 stts (72024 bytes): Data = length 72016, hash 282C954B - stsz (200020 bytes): - Data = length 200012, hash 719AE8EE + stsz (20 bytes): + Data = length 12, hash F19823A6 stsc (28 bytes): Data = length 20, hash 8F6E8285 co64 (400016 bytes): diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_moov_at_the_end_and_no_free_box.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_moov_at_the_end_and_no_free_box.mp4.dump index d250cfe84c5..6633b55188f 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_moov_at_the_end_and_no_free_box.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_moov_at_the_end_and_no_free_box.mp4.dump @@ -2,29 +2,29 @@ ftyp (28 bytes): Data = length 20, hash EF896440 mdat (56016 bytes): Data = length 56000, hash F535891 -moov (18082 bytes): +moov (14082 bytes): mvhd (108 bytes): Data = length 100, hash C71E9666 - trak (17966 bytes): + trak (13966 bytes): tkhd (92 bytes): Data = length 84, hash 8611BA99 - mdia (17866 bytes): + mdia (13866 bytes): mdhd (32 bytes): Data = length 24, hash 464870DB hdlr (44 bytes): Data = length 36, hash A0852FF2 - minf (17782 bytes): + minf (13782 bytes): vmhd (20 bytes): Data = length 12, hash EE830681 dinf (36 bytes): Data = length 28, hash D535436B - stbl (17718 bytes): + stbl (13718 bytes): stsd (166 bytes): Data = length 158, hash 11532063 stts (1464 bytes): Data = length 1456, hash 550AE6F8 - stsz (4020 bytes): - Data = length 4012, hash 537BD986 + stsz (20 bytes): + Data = length 12, hash F1982AFE stsc (28 bytes): Data = length 20, hash 8F6E8285 co64 (8016 bytes): diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_primary_tracks_when_auxiliary_track_samples_interleaved.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_primary_tracks_when_auxiliary_track_samples_interleaved.mp4.dump index 16708ed2e41..d561516931c 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_primary_tracks_when_auxiliary_track_samples_interleaved.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_primary_tracks_when_auxiliary_track_samples_interleaved.mp4.dump @@ -23,7 +23,7 @@ track 0: colorRange = 1 lumaBitdepth = 8 chromaBitdepth = 8 - metadata = entries=[mdta: key=auxiliary.tracks.length, value=1554, mdta: key=auxiliary.tracks.offset, value=400892, Mp4Timestamp: creation time=1000000, modification time=5000000, timescale=10000, ThumbnailMetadata: presentationTimeUs=0] + metadata = entries=[mdta: key=auxiliary.tracks.length, value=1514, mdta: key=auxiliary.tracks.offset, value=400892, Mp4Timestamp: creation time=1000000, modification time=5000000, timescale=10000, ThumbnailMetadata: presentationTimeUs=0] initializationData: data = length 28, hash 410B510 data = length 9, hash FBADD682 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_samples_and_metadata.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_samples_and_metadata.mp4.dump index 5d4fd1ad51c..99da546a8ef 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_samples_and_metadata.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_samples_and_metadata.mp4.dump @@ -1,6 +1,6 @@ ftyp (28 bytes): Data = length 20, hash EF896440 -moov (873 bytes): +moov (869 bytes): mvhd (108 bytes): Data = length 100, hash 2613A5C udta (38 bytes): @@ -12,26 +12,26 @@ moov (873 bytes): Data = length 57, hash E97C2148 ilst (71 bytes): Data = length 63, hash D81D174 - trak (542 bytes): + trak (538 bytes): tkhd (92 bytes): Data = length 84, hash 4ACEE52E - mdia (442 bytes): + mdia (438 bytes): mdhd (32 bytes): Data = length 24, hash 41542D81 hdlr (44 bytes): Data = length 36, hash A0852FF2 - minf (358 bytes): + minf (354 bytes): vmhd (20 bytes): Data = length 12, hash EE830681 dinf (36 bytes): Data = length 28, hash D535436B - stbl (294 bytes): + stbl (290 bytes): stsd (166 bytes): Data = length 158, hash 11532063 stts (24 bytes): Data = length 16, hash E4FC6483 - stsz (24 bytes): - Data = length 16, hash 50B7F5BA + stsz (20 bytes): + Data = length 12, hash F1982ABA stsc (28 bytes): Data = length 20, hash 8F6E8285 co64 (24 bytes): @@ -40,7 +40,7 @@ moov (873 bytes): Data = length 12, hash EE911E03 uuid (2853 bytes): Data = length 2845, hash 52AF0F9D -free (396282 bytes): - Data = length 396274, hash 946025C1 +free (396286 bytes): + Data = length 396278, hash 78655D41 mdat (72 bytes): Data = length 56, hash DB5662FB diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_some_free_space_after_ftyp.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_some_free_space_after_ftyp.mp4.dump index 08135b644b2..a6579eb1206 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_some_free_space_after_ftyp.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_some_free_space_after_ftyp.mp4.dump @@ -2,36 +2,36 @@ ftyp (28 bytes): Data = length 20, hash EF896440 free (500008 bytes): Data = length 500000, hash C6A19C01 -moov (658 bytes): +moov (654 bytes): mvhd (108 bytes): Data = length 100, hash 2613A5C - trak (542 bytes): + trak (538 bytes): tkhd (92 bytes): Data = length 84, hash 3D79758F - mdia (442 bytes): + mdia (438 bytes): mdhd (32 bytes): Data = length 24, hash 41542D81 hdlr (44 bytes): Data = length 36, hash A0852FF2 - minf (358 bytes): + minf (354 bytes): vmhd (20 bytes): Data = length 12, hash EE830681 dinf (36 bytes): Data = length 28, hash D535436B - stbl (294 bytes): + stbl (290 bytes): stsd (166 bytes): Data = length 158, hash 11532063 stts (24 bytes): Data = length 16, hash E4FC6483 - stsz (24 bytes): - Data = length 16, hash 50B7F5BA + stsz (20 bytes): + Data = length 12, hash F1982ABA stsc (28 bytes): Data = length 20, hash 8F6E8285 co64 (24 bytes): Data = length 16, hash E4EE7550 stss (20 bytes): Data = length 12, hash EE911E03 -free (399350 bytes): - Data = length 399342, hash 9C99AE41 +free (399354 bytes): + Data = length 399346, hash 3FA6A5C1 mdat (72 bytes): Data = length 56, hash DB5662FB diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_some_free_space_after_ftyp_and_streamable_output_disabled.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_some_free_space_after_ftyp_and_streamable_output_disabled.mp4.dump index e40194f8e2b..3a17339e248 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_some_free_space_after_ftyp_and_streamable_output_disabled.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_some_free_space_after_ftyp_and_streamable_output_disabled.mp4.dump @@ -4,29 +4,29 @@ free (508 bytes): Data = length 500, hash 96A2F981 mdat (72 bytes): Data = length 56, hash DB5662FB -moov (658 bytes): +moov (654 bytes): mvhd (108 bytes): Data = length 100, hash 2613A5C - trak (542 bytes): + trak (538 bytes): tkhd (92 bytes): Data = length 84, hash 3D79758F - mdia (442 bytes): + mdia (438 bytes): mdhd (32 bytes): Data = length 24, hash 41542D81 hdlr (44 bytes): Data = length 36, hash A0852FF2 - minf (358 bytes): + minf (354 bytes): vmhd (20 bytes): Data = length 12, hash EE830681 dinf (36 bytes): Data = length 28, hash D535436B - stbl (294 bytes): + stbl (290 bytes): stsd (166 bytes): Data = length 158, hash 11532063 stts (24 bytes): Data = length 16, hash E4FC6483 - stsz (24 bytes): - Data = length 16, hash 50B7F5BA + stsz (20 bytes): + Data = length 12, hash F1982ABA stsc (28 bytes): Data = length 20, hash 8F6E8285 co64 (24 bytes): diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_xmp.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_xmp.mp4.dump index d53f3030bee..4a7d24f6921 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_xmp.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_with_xmp.mp4.dump @@ -1,28 +1,28 @@ ftyp (28 bytes): Data = length 20, hash EF896440 -moov (658 bytes): +moov (654 bytes): mvhd (108 bytes): Data = length 100, hash 2613A5C - trak (542 bytes): + trak (538 bytes): tkhd (92 bytes): Data = length 84, hash 3D79758F - mdia (442 bytes): + mdia (438 bytes): mdhd (32 bytes): Data = length 24, hash 41542D81 hdlr (44 bytes): Data = length 36, hash A0852FF2 - minf (358 bytes): + minf (354 bytes): vmhd (20 bytes): Data = length 12, hash EE830681 dinf (36 bytes): Data = length 28, hash D535436B - stbl (294 bytes): + stbl (290 bytes): stsd (166 bytes): Data = length 158, hash 11532063 stts (24 bytes): Data = length 16, hash E4FC6483 - stsz (24 bytes): - Data = length 16, hash 50B7F5BA + stsz (20 bytes): + Data = length 12, hash F1982ABA stsc (28 bytes): Data = length 20, hash 8F6E8285 co64 (24 bytes): @@ -31,7 +31,7 @@ moov (658 bytes): Data = length 12, hash EE911E03 uuid (2853 bytes): Data = length 2845, hash 52AF0F9D -free (396497 bytes): - Data = length 396489, hash 429891F +free (396501 bytes): + Data = length 396493, hash 53F1E19F mdat (72 bytes): Data = length 56, hash DB5662FB diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_without_empty_track.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_without_empty_track.mp4.dump index 492835ab878..6e5ffe06c17 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_without_empty_track.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/mp4_without_empty_track.mp4.dump @@ -1,35 +1,35 @@ ftyp (28 bytes): Data = length 20, hash EF896440 -moov (682 bytes): +moov (674 bytes): mvhd (108 bytes): Data = length 100, hash 105FA889 - trak (566 bytes): + trak (558 bytes): tkhd (92 bytes): Data = length 84, hash AC22063C - mdia (466 bytes): + mdia (458 bytes): mdhd (32 bytes): Data = length 24, hash 88615B36 hdlr (44 bytes): Data = length 36, hash A0852FF2 - minf (382 bytes): + minf (374 bytes): vmhd (20 bytes): Data = length 12, hash EE830681 dinf (36 bytes): Data = length 28, hash D535436B - stbl (318 bytes): + stbl (310 bytes): stsd (166 bytes): Data = length 158, hash 11532063 stts (32 bytes): Data = length 24, hash 4A7D0E0E - stsz (28 bytes): - Data = length 20, hash 3836F7F3 + stsz (20 bytes): + Data = length 12, hash F1982ABB stsc (28 bytes): Data = length 20, hash 8F6E8285 co64 (32 bytes): Data = length 24, hash 6E04DBFB stss (24 bytes): Data = length 16, hash 7940D386 -free (399326 bytes): - Data = length 399318, hash 3D692141 +free (399334 bytes): + Data = length 399326, hash 3C185041 mdat (128 bytes): Data = length 112, hash 1AAF8FF5 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/partial_hdr10-720p.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/partial_hdr10-720p.mp4.dump index 1917254a3f4..ce5959cf058 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/partial_hdr10-720p.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/partial_hdr10-720p.mp4.dump @@ -9,7 +9,7 @@ numberOfTracks = 2 track 0: total output bytes = 5843480 sample count = 93 - track duration = 3103400 + track duration = 3103422 format 0: averageBitrate = 15063320 id = 1 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_audio_only_15s.mp4_fragmented_box_structure.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_audio_only_15s.mp4_fragmented_box_structure.dump index 29fd8ee2d4a..d9a469cb021 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_audio_only_15s.mp4_fragmented_box_structure.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_audio_only_15s.mp4_fragmented_box_structure.dump @@ -30,83 +30,101 @@ moov (570 bytes): mvex (40 bytes): trex (32 bytes): Data = length 24, hash C35D3183 -moof (1216 bytes): +moof (1236 bytes): mfhd (16 bytes): Data = length 8, hash 94446F02 - traf (1192 bytes): + traf (1212 bytes): tfhd (24 bytes): Data = length 16, hash D3715417 + tfdt (20 bytes): + Data = length 12, hash F634ABE0 trun (1160 bytes): - Data = length 1152, hash 1DBC5FAE + Data = length 1152, hash 13AD9DC2 mdat (595 bytes): Data = length 587, hash 5606AB27 -moof (1216 bytes): +moof (1236 bytes): mfhd (16 bytes): Data = length 8, hash 94446F03 - traf (1192 bytes): + traf (1212 bytes): tfhd (24 bytes): - Data = length 16, hash D3715503 + Data = length 16, hash D3715517 + tfdt (20 bytes): + Data = length 12, hash F634BE85 trun (1160 bytes): - Data = length 1152, hash D7BCFD1D + Data = length 1152, hash CDAE3B31 mdat (578 bytes): Data = length 570, hash 82739626 -moof (1216 bytes): +moof (1236 bytes): mfhd (16 bytes): Data = length 8, hash 94446F04 - traf (1192 bytes): + traf (1212 bytes): tfhd (24 bytes): - Data = length 16, hash D37155DE + Data = length 16, hash D3715506 + tfdt (20 bytes): + Data = length 12, hash F634B24A trun (1160 bytes): - Data = length 1152, hash 4B1C22AE + Data = length 1152, hash 410D60C2 mdat (595 bytes): Data = length 587, hash 5925807B -moof (1216 bytes): +moof (1236 bytes): mfhd (16 bytes): Data = length 8, hash 94446F05 - traf (1192 bytes): + traf (1212 bytes): tfhd (24 bytes): - Data = length 16, hash D37156CA + Data = length 16, hash D3715606 + tfdt (20 bytes): + Data = length 12, hash F634C8F0 trun (1160 bytes): - Data = length 1152, hash D7BCFD1D + Data = length 1152, hash CDAE3B31 mdat (578 bytes): Data = length 570, hash 82739626 -moof (1216 bytes): +moof (1236 bytes): mfhd (16 bytes): Data = length 8, hash 94446F06 - traf (1192 bytes): + traf (1212 bytes): tfhd (24 bytes): - Data = length 16, hash D37156A5 + Data = length 16, hash D37156F5 + tfdt (20 bytes): + Data = length 12, hash F634BC95 trun (1160 bytes): - Data = length 1152, hash D7BCFD1D + Data = length 1152, hash CDAE3B31 mdat (578 bytes): Data = length 570, hash 82739626 -moof (1216 bytes): +moof (1236 bytes): mfhd (16 bytes): Data = length 8, hash 94446F07 - traf (1192 bytes): + traf (1212 bytes): tfhd (24 bytes): - Data = length 16, hash D3715780 + Data = length 16, hash D37157E4 + tfdt (20 bytes): + Data = length 12, hash F634D31B trun (1160 bytes): - Data = length 1152, hash B425E72E + Data = length 1152, hash AA172542 mdat (595 bytes): Data = length 587, hash 310213B9 -moof (1216 bytes): +moof (1236 bytes): mfhd (16 bytes): Data = length 8, hash 94446F08 - traf (1192 bytes): + traf (1212 bytes): tfhd (24 bytes): - Data = length 16, hash D371586C + Data = length 16, hash D3715903 + tfdt (20 bytes): + Data = length 12, hash F634C700 trun (1160 bytes): - Data = length 1152, hash D7BCFD1D + Data = length 1152, hash CDAE3B31 mdat (578 bytes): Data = length 570, hash 82739626 -moof (556 bytes): +moof (576 bytes): mfhd (16 bytes): Data = length 8, hash 94446F09 - traf (532 bytes): + traf (552 bytes): tfhd (24 bytes): - Data = length 16, hash D3715947 + Data = length 16, hash D37159F2 + tfdt (20 bytes): + Data = length 12, hash F634DD66 trun (500 bytes): - Data = length 492, hash EC9CC954 + Data = length 492, hash 90AD1968 mdat (248 bytes): Data = length 240, hash BD3FF239 +mfra (200 bytes): + Data = length 192, hash 5C2CDE73 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_av1.mp4_fragmented.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_av1.mp4_fragmented.dump index 30ef867f362..852d6d01d7f 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_av1.mp4_fragmented.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_av1.mp4_fragmented.dump @@ -157,183 +157,183 @@ track 1: initializationData: data = length 5, hash 2B7623A sample 0: - time = 0 + time = 42993 flags = 1 data = length 21, hash 77102128 sample 1: - time = 23219 + time = 66213 flags = 1 data = length 17, hash BA75F5D4 sample 2: - time = 46439 + time = 89433 flags = 1 data = length 582, hash B5064B53 sample 3: - time = 69659 + time = 112653 flags = 1 data = length 218, hash 46000EEF sample 4: - time = 92879 + time = 135873 flags = 1 data = length 206, hash 7B12EC38 sample 5: - time = 116099 + time = 159092 flags = 1 data = length 215, hash C05E2F91 sample 6: - time = 139319 + time = 182312 flags = 1 data = length 217, hash 1E457BBF sample 7: - time = 162539 + time = 205532 flags = 1 data = length 195, hash DFD6F480 sample 8: - time = 185759 + time = 228752 flags = 1 data = length 198, hash 2BC702E sample 9: - time = 208979 + time = 251972 flags = 1 data = length 216, hash ED964B3D sample 10: - time = 232199 + time = 275192 flags = 1 data = length 204, hash DAF6FDC6 sample 11: - time = 255419 + time = 298412 flags = 1 data = length 205, hash D249FD76 sample 12: - time = 278639 + time = 321632 flags = 1 data = length 200, hash C8F844E4 sample 13: - time = 301859 + time = 344852 flags = 1 data = length 196, hash FDD0CA03 sample 14: - time = 325079 + time = 368072 flags = 1 data = length 196, hash E4E3A7B0 sample 15: - time = 348299 + time = 391292 flags = 1 data = length 207, hash 157773E3 sample 16: - time = 371519 + time = 414512 flags = 1 data = length 207, hash C9F46F0F sample 17: - time = 394739 + time = 437732 flags = 1 data = length 210, hash 127AC739 sample 18: - time = 417959 + time = 460952 flags = 1 data = length 217, hash B2649830 sample 19: - time = 441179 + time = 484172 flags = 1 data = length 188, hash 4D280759 sample 20: - time = 464399 + time = 507392 flags = 1 data = length 205, hash EAE6D6AD sample 21: - time = 487619 + time = 530612 flags = 1 data = length 226, hash BDD0EC44 sample 22: - time = 510839 + time = 553832 flags = 1 data = length 199, hash 60C719A2 sample 23: - time = 534058 + time = 577052 flags = 1 data = length 215, hash EDDE842F sample 24: - time = 557278 + time = 600272 flags = 1 data = length 201, hash D17187B sample 25: - time = 580498 + time = 623492 flags = 1 data = length 217, hash 58DD698C sample 26: - time = 603718 + time = 646712 flags = 1 data = length 202, hash 5168D405 sample 27: - time = 626938 + time = 669931 flags = 1 data = length 194, hash 7139AF8 sample 28: - time = 650158 + time = 693151 flags = 1 data = length 203, hash F775D9ED sample 29: - time = 673378 + time = 716371 flags = 1 data = length 200, hash 774C5045 sample 30: - time = 696598 + time = 739591 flags = 1 data = length 211, hash ED3C6FBC sample 31: - time = 719818 + time = 762811 flags = 1 data = length 205, hash FC4754A9 sample 32: - time = 743038 + time = 786031 flags = 1 data = length 216, hash 72F4AF29 sample 33: - time = 766258 + time = 809251 flags = 1 data = length 204, hash 1AF98D40 sample 34: - time = 789478 + time = 832471 flags = 1 data = length 200, hash E0004171 sample 35: - time = 812698 + time = 855691 flags = 1 data = length 215, hash B413079A sample 36: - time = 835918 + time = 878911 flags = 1 data = length 211, hash 107CEE52 sample 37: - time = 859138 + time = 902131 flags = 1 data = length 214, hash 1E588A0D sample 38: - time = 882358 + time = 925351 flags = 1 data = length 210, hash 84E5BBBD sample 39: - time = 905578 + time = 948571 flags = 1 data = length 211, hash 32D7ACAB sample 40: - time = 928798 + time = 971791 flags = 1 data = length 201, hash 1567F919 sample 41: - time = 952018 + time = 995011 flags = 1 data = length 196, hash 2F050463 sample 42: - time = 975238 + time = 1018231 flags = 1 data = length 215, hash 4BDD9C81 sample 43: - time = 998458 + time = 1041451 flags = 1 data = length 242, hash DD6FD967 sample 44: - time = 1021678 + time = 1064671 flags = 1 data = length 184, hash DAFC330D tracksEnded = true diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_batching_and_attempt_streamable_output_disabled_hdr10-720p.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_batching_and_attempt_streamable_output_disabled_hdr10-720p.mp4.dump index d1e6ba936d1..274c020ce79 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_batching_and_attempt_streamable_output_disabled_hdr10-720p.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_batching_and_attempt_streamable_output_disabled_hdr10-720p.mp4.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 4236600 + duration = 4236588 getPosition(0) = [[timeUs=0, position=44]] getPosition(1) = [[timeUs=0, position=44], [timeUs=1002955, position=30169]] - getPosition(2118300) = [[timeUs=2003566, position=2162805], [timeUs=3003444, position=4026908]] - getPosition(4236600) = [[timeUs=4003277, position=5970225]] + getPosition(2118294) = [[timeUs=2003566, position=2162805], [timeUs=3003444, position=4026908]] + getPosition(4236588) = [[timeUs=4003277, position=5970225]] numberOfTracks = 2 track 0: total output bytes = 7944083 sample count = 127 - track duration = 4236600 + track duration = 4236588 format 0: averageBitrate = 15000907 id = 1 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_batching_disabled_hdr10-720p.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_batching_disabled_hdr10-720p.mp4.dump index 6a5ac23e42f..11e02f033ec 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_batching_disabled_hdr10-720p.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_batching_disabled_hdr10-720p.mp4.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 4236600 + duration = 4236588 getPosition(0) = [[timeUs=0, position=400052]] getPosition(1) = [[timeUs=0, position=400052], [timeUs=1002955, position=430177]] - getPosition(2118300) = [[timeUs=2003566, position=2562813], [timeUs=3003444, position=4426916]] - getPosition(4236600) = [[timeUs=4003277, position=6370233]] + getPosition(2118294) = [[timeUs=2003566, position=2562813], [timeUs=3003444, position=4426916]] + getPosition(4236588) = [[timeUs=4003277, position=6370233]] numberOfTracks = 2 track 0: total output bytes = 7944083 sample count = 127 - track duration = 4236600 + track duration = 4236588 format 0: averageBitrate = 15000907 id = 1 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_eac3.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_eac3.mp4.dump new file mode 100644 index 00000000000..62bbb603d16 --- /dev/null +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_eac3.mp4.dump @@ -0,0 +1,242 @@ +seekMap: + isSeekable = true + duration = 1728000 + getPosition(0) = [[timeUs=0, position=400056]] + getPosition(1) = [[timeUs=1, position=400056]] + getPosition(864000) = [[timeUs=864000, position=508056]] + getPosition(1728000) = [[timeUs=1728000, position=612056]] +numberOfTracks = 1 +track 0: + total output bytes = 216000 + sample count = 54 + track duration = 1728000 + format 0: + averageBitrate = 1000000 + peakBitrate = 1000000 + id = 1 + containerMimeType = audio/mp4 + sampleMimeType = audio/eac3 + maxInputSize = 4030 + channelCount = 6 + sampleRate = 48000 + language = und + metadata = entries=[Mp4Timestamp: creation time=100000000, modification time=500000000, timescale=10000] + initializationData: + data = length 5, hash 38742EF + sample 0: + time = 0 + flags = 1 + data = length 4000, hash BAEAFB2A + sample 1: + time = 32000 + flags = 1 + data = length 4000, hash E3C5EBF0 + sample 2: + time = 64000 + flags = 1 + data = length 4000, hash 32E0F957 + sample 3: + time = 96000 + flags = 1 + data = length 4000, hash 5354CC5D + sample 4: + time = 128000 + flags = 1 + data = length 4000, hash FF834906 + sample 5: + time = 160000 + flags = 1 + data = length 4000, hash 6F571E61 + sample 6: + time = 192000 + flags = 1 + data = length 4000, hash 5C931F6B + sample 7: + time = 224000 + flags = 1 + data = length 4000, hash B1FB2E57 + sample 8: + time = 256000 + flags = 1 + data = length 4000, hash C71240EB + sample 9: + time = 288000 + flags = 1 + data = length 4000, hash C3E302EE + sample 10: + time = 320000 + flags = 1 + data = length 4000, hash 7994C27B + sample 11: + time = 352000 + flags = 1 + data = length 4000, hash 1ED4E6F3 + sample 12: + time = 384000 + flags = 1 + data = length 4000, hash 1D5E6AAC + sample 13: + time = 416000 + flags = 1 + data = length 4000, hash 30058F51 + sample 14: + time = 448000 + flags = 1 + data = length 4000, hash 15DD0E4A + sample 15: + time = 480000 + flags = 1 + data = length 4000, hash 37BE7C15 + sample 16: + time = 512000 + flags = 1 + data = length 4000, hash 7CFDD34B + sample 17: + time = 544000 + flags = 1 + data = length 4000, hash 27F20D29 + sample 18: + time = 576000 + flags = 1 + data = length 4000, hash 6F565894 + sample 19: + time = 608000 + flags = 1 + data = length 4000, hash A6F07C4A + sample 20: + time = 640000 + flags = 1 + data = length 4000, hash 3A0CA15C + sample 21: + time = 672000 + flags = 1 + data = length 4000, hash DB365414 + sample 22: + time = 704000 + flags = 1 + data = length 4000, hash 31E08469 + sample 23: + time = 736000 + flags = 1 + data = length 4000, hash 315F5C28 + sample 24: + time = 768000 + flags = 1 + data = length 4000, hash CC65DF80 + sample 25: + time = 800000 + flags = 1 + data = length 4000, hash 503FB64C + sample 26: + time = 832000 + flags = 1 + data = length 4000, hash 817CF735 + sample 27: + time = 864000 + flags = 1 + data = length 4000, hash 37391ADA + sample 28: + time = 896000 + flags = 1 + data = length 4000, hash 37391ADA + sample 29: + time = 928000 + flags = 1 + data = length 4000, hash 64DBF751 + sample 30: + time = 960000 + flags = 1 + data = length 4000, hash 81AE828E + sample 31: + time = 992000 + flags = 1 + data = length 4000, hash 767D6C98 + sample 32: + time = 1024000 + flags = 1 + data = length 4000, hash A5F6D4E + sample 33: + time = 1056000 + flags = 1 + data = length 4000, hash EABC6B0D + sample 34: + time = 1088000 + flags = 1 + data = length 4000, hash F47EF742 + sample 35: + time = 1120000 + flags = 1 + data = length 4000, hash 9B2549DA + sample 36: + time = 1152000 + flags = 1 + data = length 4000, hash A12733C9 + sample 37: + time = 1184000 + flags = 1 + data = length 4000, hash 95F62E99 + sample 38: + time = 1216000 + flags = 1 + data = length 4000, hash A4D858 + sample 39: + time = 1248000 + flags = 1 + data = length 4000, hash A4D858 + sample 40: + time = 1280000 + flags = 1 + data = length 4000, hash 22C1A129 + sample 41: + time = 1312000 + flags = 1 + data = length 4000, hash 2C51E4A1 + sample 42: + time = 1344000 + flags = 1 + data = length 4000, hash 3782E8BB + sample 43: + time = 1376000 + flags = 1 + data = length 4000, hash 2C51E4A1 + sample 44: + time = 1408000 + flags = 1 + data = length 4000, hash BDB3D129 + sample 45: + time = 1440000 + flags = 1 + data = length 4000, hash F642A55 + sample 46: + time = 1472000 + flags = 1 + data = length 4000, hash 32F259F4 + sample 47: + time = 1504000 + flags = 1 + data = length 4000, hash 4C987B7C + sample 48: + time = 1536000 + flags = 1 + data = length 4000, hash 57C98E1C + sample 49: + time = 1568000 + flags = 1 + data = length 4000, hash 4C987B7C + sample 50: + time = 1600000 + flags = 1 + data = length 4000, hash 4C987B7C + sample 51: + time = 1632000 + flags = 1 + data = length 4000, hash 4C987B7C + sample 52: + time = 1664000 + flags = 1 + data = length 4000, hash 4C987B7C + sample 53: + time = 1696000 + flags = 536870913 + data = length 4000, hash 4C987B7C +tracksEnded = true diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_eac3.mp4_fragmented.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_eac3.mp4_fragmented.dump new file mode 100644 index 00000000000..c057d89dcd2 --- /dev/null +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_eac3.mp4_fragmented.dump @@ -0,0 +1,236 @@ +seekMap: + isSeekable = false + duration = UNSET TIME + getPosition(0) = [[timeUs=0, position=573]] +numberOfTracks = 1 +track 0: + total output bytes = 216000 + sample count = 54 + format 0: + peakBitrate = 1000000 + id = 1 + containerMimeType = audio/mp4 + sampleMimeType = audio/eac3 + channelCount = 6 + sampleRate = 48000 + language = und + metadata = entries=[Mp4Timestamp: creation time=100000000, modification time=500000000, timescale=10000] + initializationData: + data = length 5, hash 38742EF + sample 0: + time = 0 + flags = 1 + data = length 4000, hash BAEAFB2A + sample 1: + time = 32000 + flags = 1 + data = length 4000, hash E3C5EBF0 + sample 2: + time = 64000 + flags = 1 + data = length 4000, hash 32E0F957 + sample 3: + time = 96000 + flags = 1 + data = length 4000, hash 5354CC5D + sample 4: + time = 128000 + flags = 1 + data = length 4000, hash FF834906 + sample 5: + time = 160000 + flags = 1 + data = length 4000, hash 6F571E61 + sample 6: + time = 192000 + flags = 1 + data = length 4000, hash 5C931F6B + sample 7: + time = 224000 + flags = 1 + data = length 4000, hash B1FB2E57 + sample 8: + time = 256000 + flags = 1 + data = length 4000, hash C71240EB + sample 9: + time = 288000 + flags = 1 + data = length 4000, hash C3E302EE + sample 10: + time = 320000 + flags = 1 + data = length 4000, hash 7994C27B + sample 11: + time = 352000 + flags = 1 + data = length 4000, hash 1ED4E6F3 + sample 12: + time = 384000 + flags = 1 + data = length 4000, hash 1D5E6AAC + sample 13: + time = 416000 + flags = 1 + data = length 4000, hash 30058F51 + sample 14: + time = 448000 + flags = 1 + data = length 4000, hash 15DD0E4A + sample 15: + time = 480000 + flags = 1 + data = length 4000, hash 37BE7C15 + sample 16: + time = 512000 + flags = 1 + data = length 4000, hash 7CFDD34B + sample 17: + time = 544000 + flags = 1 + data = length 4000, hash 27F20D29 + sample 18: + time = 576000 + flags = 1 + data = length 4000, hash 6F565894 + sample 19: + time = 608000 + flags = 1 + data = length 4000, hash A6F07C4A + sample 20: + time = 640000 + flags = 1 + data = length 4000, hash 3A0CA15C + sample 21: + time = 672000 + flags = 1 + data = length 4000, hash DB365414 + sample 22: + time = 704000 + flags = 1 + data = length 4000, hash 31E08469 + sample 23: + time = 736000 + flags = 1 + data = length 4000, hash 315F5C28 + sample 24: + time = 768000 + flags = 1 + data = length 4000, hash CC65DF80 + sample 25: + time = 800000 + flags = 1 + data = length 4000, hash 503FB64C + sample 26: + time = 832000 + flags = 1 + data = length 4000, hash 817CF735 + sample 27: + time = 864000 + flags = 1 + data = length 4000, hash 37391ADA + sample 28: + time = 896000 + flags = 1 + data = length 4000, hash 37391ADA + sample 29: + time = 928000 + flags = 1 + data = length 4000, hash 64DBF751 + sample 30: + time = 960000 + flags = 1 + data = length 4000, hash 81AE828E + sample 31: + time = 992000 + flags = 1 + data = length 4000, hash 767D6C98 + sample 32: + time = 1024000 + flags = 1 + data = length 4000, hash A5F6D4E + sample 33: + time = 1056000 + flags = 1 + data = length 4000, hash EABC6B0D + sample 34: + time = 1088000 + flags = 1 + data = length 4000, hash F47EF742 + sample 35: + time = 1120000 + flags = 1 + data = length 4000, hash 9B2549DA + sample 36: + time = 1152000 + flags = 1 + data = length 4000, hash A12733C9 + sample 37: + time = 1184000 + flags = 1 + data = length 4000, hash 95F62E99 + sample 38: + time = 1216000 + flags = 1 + data = length 4000, hash A4D858 + sample 39: + time = 1248000 + flags = 1 + data = length 4000, hash A4D858 + sample 40: + time = 1280000 + flags = 1 + data = length 4000, hash 22C1A129 + sample 41: + time = 1312000 + flags = 1 + data = length 4000, hash 2C51E4A1 + sample 42: + time = 1344000 + flags = 1 + data = length 4000, hash 3782E8BB + sample 43: + time = 1376000 + flags = 1 + data = length 4000, hash 2C51E4A1 + sample 44: + time = 1408000 + flags = 1 + data = length 4000, hash BDB3D129 + sample 45: + time = 1440000 + flags = 1 + data = length 4000, hash F642A55 + sample 46: + time = 1472000 + flags = 1 + data = length 4000, hash 32F259F4 + sample 47: + time = 1504000 + flags = 1 + data = length 4000, hash 4C987B7C + sample 48: + time = 1536000 + flags = 1 + data = length 4000, hash 57C98E1C + sample 49: + time = 1568000 + flags = 1 + data = length 4000, hash 4C987B7C + sample 50: + time = 1600000 + flags = 1 + data = length 4000, hash 4C987B7C + sample 51: + time = 1632000 + flags = 1 + data = length 4000, hash 4C987B7C + sample 52: + time = 1664000 + flags = 1 + data = length 4000, hash 4C987B7C + sample 53: + time = 1696000 + flags = 1 + data = length 4000, hash 4C987B7C +tracksEnded = true diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_eac3joc.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_eac3joc.mp4.dump new file mode 100644 index 00000000000..b908a876add --- /dev/null +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_eac3joc.mp4.dump @@ -0,0 +1,555 @@ +seekMap: + isSeekable = true + duration = 2048000 + getPosition(0) = [[timeUs=0, position=400056]] + getPosition(1) = [[timeUs=1, position=400056]] + getPosition(1024000) = [[timeUs=1024000, position=563896]] + getPosition(2048000) = [[timeUs=2048000, position=722616]] +numberOfTracks = 2 +track 0: + total output bytes = 163840 + sample count = 64 + track duration = 2048000 + format 0: + averageBitrate = 640000 + peakBitrate = 640000 + id = 1 + containerMimeType = audio/mp4 + sampleMimeType = audio/eac3-joc + maxInputSize = 2590 + channelCount = 6 + sampleRate = 48000 + language = und + metadata = entries=[Mp4Timestamp: creation time=100000000, modification time=500000000, timescale=10000] + initializationData: + data = length 7, hash 8BA78FD3 + sample 0: + time = 0 + flags = 1 + data = length 2560, hash 882594AD + sample 1: + time = 32000 + flags = 1 + data = length 2560, hash 41EC8B22 + sample 2: + time = 64000 + flags = 1 + data = length 2560, hash 67E6EFD4 + sample 3: + time = 96000 + flags = 1 + data = length 2560, hash A7E66AFD + sample 4: + time = 128000 + flags = 1 + data = length 2560, hash 3924116 + sample 5: + time = 160000 + flags = 1 + data = length 2560, hash 64DCE40B + sample 6: + time = 192000 + flags = 1 + data = length 2560, hash F2E0DA64 + sample 7: + time = 224000 + flags = 1 + data = length 2560, hash C156258B + sample 8: + time = 256000 + flags = 1 + data = length 2560, hash D8DBDCDE + sample 9: + time = 288000 + flags = 1 + data = length 2560, hash C11B2F25 + sample 10: + time = 320000 + flags = 1 + data = length 2560, hash B3C5612 + sample 11: + time = 352000 + flags = 1 + data = length 2560, hash A94B15D0 + sample 12: + time = 384000 + flags = 1 + data = length 2560, hash 12E4E306 + sample 13: + time = 416000 + flags = 1 + data = length 2560, hash 11CB959F + sample 14: + time = 448000 + flags = 1 + data = length 2560, hash B6433844 + sample 15: + time = 480000 + flags = 1 + data = length 2560, hash EA6DEB89 + sample 16: + time = 512000 + flags = 1 + data = length 2560, hash 6D65CBD9 + sample 17: + time = 544000 + flags = 1 + data = length 2560, hash A5D635C5 + sample 18: + time = 576000 + flags = 1 + data = length 2560, hash 992E36AB + sample 19: + time = 608000 + flags = 1 + data = length 2560, hash 1EC4E5AF + sample 20: + time = 640000 + flags = 1 + data = length 2560, hash DCFEB7D2 + sample 21: + time = 672000 + flags = 1 + data = length 2560, hash 45EFC639 + sample 22: + time = 704000 + flags = 1 + data = length 2560, hash F598673 + sample 23: + time = 736000 + flags = 1 + data = length 2560, hash 89E4E5EC + sample 24: + time = 768000 + flags = 1 + data = length 2560, hash FBE2532B + sample 25: + time = 800000 + flags = 1 + data = length 2560, hash 9CE5F83B + sample 26: + time = 832000 + flags = 1 + data = length 2560, hash 6ED49E2C + sample 27: + time = 864000 + flags = 1 + data = length 2560, hash BC52F8F3 + sample 28: + time = 896000 + flags = 1 + data = length 2560, hash 759203E2 + sample 29: + time = 928000 + flags = 1 + data = length 2560, hash D5D31AE9 + sample 30: + time = 960000 + flags = 1 + data = length 2560, hash 640A24ED + sample 31: + time = 992000 + flags = 1 + data = length 2560, hash 19B52B8B + sample 32: + time = 1024000 + flags = 1 + data = length 2560, hash 5DA977C3 + sample 33: + time = 1056000 + flags = 1 + data = length 2560, hash 982879DD + sample 34: + time = 1088000 + flags = 1 + data = length 2560, hash A7656B9C + sample 35: + time = 1120000 + flags = 1 + data = length 2560, hash 445CCC67 + sample 36: + time = 1152000 + flags = 1 + data = length 2560, hash ACD5CB5C + sample 37: + time = 1184000 + flags = 1 + data = length 2560, hash 175BBF26 + sample 38: + time = 1216000 + flags = 1 + data = length 2560, hash DBCBEB0 + sample 39: + time = 1248000 + flags = 1 + data = length 2560, hash DA39D991 + sample 40: + time = 1280000 + flags = 1 + data = length 2560, hash F08CC8E2 + sample 41: + time = 1312000 + flags = 1 + data = length 2560, hash 6B0842D7 + sample 42: + time = 1344000 + flags = 1 + data = length 2560, hash 9FE87594 + sample 43: + time = 1376000 + flags = 1 + data = length 2560, hash 8E62CE19 + sample 44: + time = 1408000 + flags = 1 + data = length 2560, hash 5FDC4084 + sample 45: + time = 1440000 + flags = 1 + data = length 2560, hash C32DAEE1 + sample 46: + time = 1472000 + flags = 1 + data = length 2560, hash BBEFB568 + sample 47: + time = 1504000 + flags = 1 + data = length 2560, hash 20504279 + sample 48: + time = 1536000 + flags = 1 + data = length 2560, hash 3B8192D2 + sample 49: + time = 1568000 + flags = 1 + data = length 2560, hash 4206B48 + sample 50: + time = 1600000 + flags = 1 + data = length 2560, hash B195AB53 + sample 51: + time = 1632000 + flags = 1 + data = length 2560, hash 3AA8E25F + sample 52: + time = 1664000 + flags = 1 + data = length 2560, hash BC227D7B + sample 53: + time = 1696000 + flags = 1 + data = length 2560, hash 6A34F7EA + sample 54: + time = 1728000 + flags = 1 + data = length 2560, hash F1E731C4 + sample 55: + time = 1760000 + flags = 1 + data = length 2560, hash 9CC406 + sample 56: + time = 1792000 + flags = 1 + data = length 2560, hash A1532233 + sample 57: + time = 1824000 + flags = 1 + data = length 2560, hash 98E49039 + sample 58: + time = 1856000 + flags = 1 + data = length 2560, hash 3F8B6DC0 + sample 59: + time = 1888000 + flags = 1 + data = length 2560, hash 4E7BF79F + sample 60: + time = 1920000 + flags = 1 + data = length 2560, hash 6DD6F2D7 + sample 61: + time = 1952000 + flags = 1 + data = length 2560, hash A05C0EC2 + sample 62: + time = 1984000 + flags = 1 + data = length 2560, hash 10C62F30 + sample 63: + time = 2016000 + flags = 536870913 + data = length 2560, hash EE4F848A +track 1: + total output bytes = 163840 + sample count = 64 + track duration = 2048000 + format 0: + averageBitrate = 640000 + peakBitrate = 640000 + id = 2 + containerMimeType = audio/mp4 + sampleMimeType = audio/eac3-joc + maxInputSize = 2590 + channelCount = 6 + sampleRate = 48000 + language = und + metadata = entries=[Mp4Timestamp: creation time=100000000, modification time=500000000, timescale=10000] + initializationData: + data = length 7, hash 8BA78FD3 + sample 0: + time = 0 + flags = 1 + data = length 2560, hash 882594AD + sample 1: + time = 32000 + flags = 1 + data = length 2560, hash 41EC8B22 + sample 2: + time = 64000 + flags = 1 + data = length 2560, hash 67E6EFD4 + sample 3: + time = 96000 + flags = 1 + data = length 2560, hash A7E66AFD + sample 4: + time = 128000 + flags = 1 + data = length 2560, hash 3924116 + sample 5: + time = 160000 + flags = 1 + data = length 2560, hash 64DCE40B + sample 6: + time = 192000 + flags = 1 + data = length 2560, hash F2E0DA64 + sample 7: + time = 224000 + flags = 1 + data = length 2560, hash C156258B + sample 8: + time = 256000 + flags = 1 + data = length 2560, hash D8DBDCDE + sample 9: + time = 288000 + flags = 1 + data = length 2560, hash C11B2F25 + sample 10: + time = 320000 + flags = 1 + data = length 2560, hash B3C5612 + sample 11: + time = 352000 + flags = 1 + data = length 2560, hash A94B15D0 + sample 12: + time = 384000 + flags = 1 + data = length 2560, hash 12E4E306 + sample 13: + time = 416000 + flags = 1 + data = length 2560, hash 11CB959F + sample 14: + time = 448000 + flags = 1 + data = length 2560, hash B6433844 + sample 15: + time = 480000 + flags = 1 + data = length 2560, hash EA6DEB89 + sample 16: + time = 512000 + flags = 1 + data = length 2560, hash 6D65CBD9 + sample 17: + time = 544000 + flags = 1 + data = length 2560, hash A5D635C5 + sample 18: + time = 576000 + flags = 1 + data = length 2560, hash 992E36AB + sample 19: + time = 608000 + flags = 1 + data = length 2560, hash 1EC4E5AF + sample 20: + time = 640000 + flags = 1 + data = length 2560, hash DCFEB7D2 + sample 21: + time = 672000 + flags = 1 + data = length 2560, hash 45EFC639 + sample 22: + time = 704000 + flags = 1 + data = length 2560, hash F598673 + sample 23: + time = 736000 + flags = 1 + data = length 2560, hash 89E4E5EC + sample 24: + time = 768000 + flags = 1 + data = length 2560, hash FBE2532B + sample 25: + time = 800000 + flags = 1 + data = length 2560, hash 9CE5F83B + sample 26: + time = 832000 + flags = 1 + data = length 2560, hash 6ED49E2C + sample 27: + time = 864000 + flags = 1 + data = length 2560, hash BC52F8F3 + sample 28: + time = 896000 + flags = 1 + data = length 2560, hash 759203E2 + sample 29: + time = 928000 + flags = 1 + data = length 2560, hash D5D31AE9 + sample 30: + time = 960000 + flags = 1 + data = length 2560, hash 640A24ED + sample 31: + time = 992000 + flags = 1 + data = length 2560, hash 19B52B8B + sample 32: + time = 1024000 + flags = 1 + data = length 2560, hash 5DA977C3 + sample 33: + time = 1056000 + flags = 1 + data = length 2560, hash 982879DD + sample 34: + time = 1088000 + flags = 1 + data = length 2560, hash A7656B9C + sample 35: + time = 1120000 + flags = 1 + data = length 2560, hash 445CCC67 + sample 36: + time = 1152000 + flags = 1 + data = length 2560, hash ACD5CB5C + sample 37: + time = 1184000 + flags = 1 + data = length 2560, hash 175BBF26 + sample 38: + time = 1216000 + flags = 1 + data = length 2560, hash DBCBEB0 + sample 39: + time = 1248000 + flags = 1 + data = length 2560, hash DA39D991 + sample 40: + time = 1280000 + flags = 1 + data = length 2560, hash F08CC8E2 + sample 41: + time = 1312000 + flags = 1 + data = length 2560, hash 6B0842D7 + sample 42: + time = 1344000 + flags = 1 + data = length 2560, hash 9FE87594 + sample 43: + time = 1376000 + flags = 1 + data = length 2560, hash 8E62CE19 + sample 44: + time = 1408000 + flags = 1 + data = length 2560, hash 5FDC4084 + sample 45: + time = 1440000 + flags = 1 + data = length 2560, hash C32DAEE1 + sample 46: + time = 1472000 + flags = 1 + data = length 2560, hash BBEFB568 + sample 47: + time = 1504000 + flags = 1 + data = length 2560, hash 20504279 + sample 48: + time = 1536000 + flags = 1 + data = length 2560, hash 3B8192D2 + sample 49: + time = 1568000 + flags = 1 + data = length 2560, hash 4206B48 + sample 50: + time = 1600000 + flags = 1 + data = length 2560, hash B195AB53 + sample 51: + time = 1632000 + flags = 1 + data = length 2560, hash 3AA8E25F + sample 52: + time = 1664000 + flags = 1 + data = length 2560, hash BC227D7B + sample 53: + time = 1696000 + flags = 1 + data = length 2560, hash 6A34F7EA + sample 54: + time = 1728000 + flags = 1 + data = length 2560, hash F1E731C4 + sample 55: + time = 1760000 + flags = 1 + data = length 2560, hash 9CC406 + sample 56: + time = 1792000 + flags = 1 + data = length 2560, hash A1532233 + sample 57: + time = 1824000 + flags = 1 + data = length 2560, hash 98E49039 + sample 58: + time = 1856000 + flags = 1 + data = length 2560, hash 3F8B6DC0 + sample 59: + time = 1888000 + flags = 1 + data = length 2560, hash 4E7BF79F + sample 60: + time = 1920000 + flags = 1 + data = length 2560, hash 6DD6F2D7 + sample 61: + time = 1952000 + flags = 1 + data = length 2560, hash A05C0EC2 + sample 62: + time = 1984000 + flags = 1 + data = length 2560, hash 10C62F30 + sample 63: + time = 2016000 + flags = 536870913 + data = length 2560, hash EE4F848A +tracksEnded = true diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_eac3joc.mp4_fragmented.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_eac3joc.mp4_fragmented.dump new file mode 100644 index 00000000000..8898b94460a --- /dev/null +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_eac3joc.mp4_fragmented.dump @@ -0,0 +1,546 @@ +seekMap: + isSeekable = false + duration = UNSET TIME + getPosition(0) = [[timeUs=0, position=994]] +numberOfTracks = 2 +track 0: + total output bytes = 163840 + sample count = 64 + format 0: + peakBitrate = 640000 + id = 1 + containerMimeType = audio/mp4 + sampleMimeType = audio/eac3-joc + channelCount = 6 + sampleRate = 48000 + language = und + metadata = entries=[Mp4Timestamp: creation time=100000000, modification time=500000000, timescale=10000] + initializationData: + data = length 7, hash 8BA78FD3 + sample 0: + time = 0 + flags = 1 + data = length 2560, hash 882594AD + sample 1: + time = 32000 + flags = 1 + data = length 2560, hash 41EC8B22 + sample 2: + time = 64000 + flags = 1 + data = length 2560, hash 67E6EFD4 + sample 3: + time = 96000 + flags = 1 + data = length 2560, hash A7E66AFD + sample 4: + time = 128000 + flags = 1 + data = length 2560, hash 3924116 + sample 5: + time = 160000 + flags = 1 + data = length 2560, hash 64DCE40B + sample 6: + time = 192000 + flags = 1 + data = length 2560, hash F2E0DA64 + sample 7: + time = 224000 + flags = 1 + data = length 2560, hash C156258B + sample 8: + time = 256000 + flags = 1 + data = length 2560, hash D8DBDCDE + sample 9: + time = 288000 + flags = 1 + data = length 2560, hash C11B2F25 + sample 10: + time = 320000 + flags = 1 + data = length 2560, hash B3C5612 + sample 11: + time = 352000 + flags = 1 + data = length 2560, hash A94B15D0 + sample 12: + time = 384000 + flags = 1 + data = length 2560, hash 12E4E306 + sample 13: + time = 416000 + flags = 1 + data = length 2560, hash 11CB959F + sample 14: + time = 448000 + flags = 1 + data = length 2560, hash B6433844 + sample 15: + time = 480000 + flags = 1 + data = length 2560, hash EA6DEB89 + sample 16: + time = 512000 + flags = 1 + data = length 2560, hash 6D65CBD9 + sample 17: + time = 544000 + flags = 1 + data = length 2560, hash A5D635C5 + sample 18: + time = 576000 + flags = 1 + data = length 2560, hash 992E36AB + sample 19: + time = 608000 + flags = 1 + data = length 2560, hash 1EC4E5AF + sample 20: + time = 640000 + flags = 1 + data = length 2560, hash DCFEB7D2 + sample 21: + time = 672000 + flags = 1 + data = length 2560, hash 45EFC639 + sample 22: + time = 704000 + flags = 1 + data = length 2560, hash F598673 + sample 23: + time = 736000 + flags = 1 + data = length 2560, hash 89E4E5EC + sample 24: + time = 768000 + flags = 1 + data = length 2560, hash FBE2532B + sample 25: + time = 800000 + flags = 1 + data = length 2560, hash 9CE5F83B + sample 26: + time = 832000 + flags = 1 + data = length 2560, hash 6ED49E2C + sample 27: + time = 864000 + flags = 1 + data = length 2560, hash BC52F8F3 + sample 28: + time = 896000 + flags = 1 + data = length 2560, hash 759203E2 + sample 29: + time = 928000 + flags = 1 + data = length 2560, hash D5D31AE9 + sample 30: + time = 960000 + flags = 1 + data = length 2560, hash 640A24ED + sample 31: + time = 992000 + flags = 1 + data = length 2560, hash 19B52B8B + sample 32: + time = 1024000 + flags = 1 + data = length 2560, hash 5DA977C3 + sample 33: + time = 1056000 + flags = 1 + data = length 2560, hash 982879DD + sample 34: + time = 1088000 + flags = 1 + data = length 2560, hash A7656B9C + sample 35: + time = 1120000 + flags = 1 + data = length 2560, hash 445CCC67 + sample 36: + time = 1152000 + flags = 1 + data = length 2560, hash ACD5CB5C + sample 37: + time = 1184000 + flags = 1 + data = length 2560, hash 175BBF26 + sample 38: + time = 1216000 + flags = 1 + data = length 2560, hash DBCBEB0 + sample 39: + time = 1248000 + flags = 1 + data = length 2560, hash DA39D991 + sample 40: + time = 1280000 + flags = 1 + data = length 2560, hash F08CC8E2 + sample 41: + time = 1312000 + flags = 1 + data = length 2560, hash 6B0842D7 + sample 42: + time = 1344000 + flags = 1 + data = length 2560, hash 9FE87594 + sample 43: + time = 1376000 + flags = 1 + data = length 2560, hash 8E62CE19 + sample 44: + time = 1408000 + flags = 1 + data = length 2560, hash 5FDC4084 + sample 45: + time = 1440000 + flags = 1 + data = length 2560, hash C32DAEE1 + sample 46: + time = 1472000 + flags = 1 + data = length 2560, hash BBEFB568 + sample 47: + time = 1504000 + flags = 1 + data = length 2560, hash 20504279 + sample 48: + time = 1536000 + flags = 1 + data = length 2560, hash 3B8192D2 + sample 49: + time = 1568000 + flags = 1 + data = length 2560, hash 4206B48 + sample 50: + time = 1600000 + flags = 1 + data = length 2560, hash B195AB53 + sample 51: + time = 1632000 + flags = 1 + data = length 2560, hash 3AA8E25F + sample 52: + time = 1664000 + flags = 1 + data = length 2560, hash BC227D7B + sample 53: + time = 1696000 + flags = 1 + data = length 2560, hash 6A34F7EA + sample 54: + time = 1728000 + flags = 1 + data = length 2560, hash F1E731C4 + sample 55: + time = 1760000 + flags = 1 + data = length 2560, hash 9CC406 + sample 56: + time = 1792000 + flags = 1 + data = length 2560, hash A1532233 + sample 57: + time = 1824000 + flags = 1 + data = length 2560, hash 98E49039 + sample 58: + time = 1856000 + flags = 1 + data = length 2560, hash 3F8B6DC0 + sample 59: + time = 1888000 + flags = 1 + data = length 2560, hash 4E7BF79F + sample 60: + time = 1920000 + flags = 1 + data = length 2560, hash 6DD6F2D7 + sample 61: + time = 1952000 + flags = 1 + data = length 2560, hash A05C0EC2 + sample 62: + time = 1984000 + flags = 1 + data = length 2560, hash 10C62F30 + sample 63: + time = 2016000 + flags = 1 + data = length 2560, hash EE4F848A +track 1: + total output bytes = 163840 + sample count = 64 + format 0: + peakBitrate = 640000 + id = 2 + containerMimeType = audio/mp4 + sampleMimeType = audio/eac3-joc + channelCount = 6 + sampleRate = 48000 + language = und + metadata = entries=[Mp4Timestamp: creation time=100000000, modification time=500000000, timescale=10000] + initializationData: + data = length 7, hash 8BA78FD3 + sample 0: + time = 0 + flags = 1 + data = length 2560, hash 882594AD + sample 1: + time = 32000 + flags = 1 + data = length 2560, hash 41EC8B22 + sample 2: + time = 64000 + flags = 1 + data = length 2560, hash 67E6EFD4 + sample 3: + time = 96000 + flags = 1 + data = length 2560, hash A7E66AFD + sample 4: + time = 128000 + flags = 1 + data = length 2560, hash 3924116 + sample 5: + time = 160000 + flags = 1 + data = length 2560, hash 64DCE40B + sample 6: + time = 192000 + flags = 1 + data = length 2560, hash F2E0DA64 + sample 7: + time = 224000 + flags = 1 + data = length 2560, hash C156258B + sample 8: + time = 256000 + flags = 1 + data = length 2560, hash D8DBDCDE + sample 9: + time = 288000 + flags = 1 + data = length 2560, hash C11B2F25 + sample 10: + time = 320000 + flags = 1 + data = length 2560, hash B3C5612 + sample 11: + time = 352000 + flags = 1 + data = length 2560, hash A94B15D0 + sample 12: + time = 384000 + flags = 1 + data = length 2560, hash 12E4E306 + sample 13: + time = 416000 + flags = 1 + data = length 2560, hash 11CB959F + sample 14: + time = 448000 + flags = 1 + data = length 2560, hash B6433844 + sample 15: + time = 480000 + flags = 1 + data = length 2560, hash EA6DEB89 + sample 16: + time = 512000 + flags = 1 + data = length 2560, hash 6D65CBD9 + sample 17: + time = 544000 + flags = 1 + data = length 2560, hash A5D635C5 + sample 18: + time = 576000 + flags = 1 + data = length 2560, hash 992E36AB + sample 19: + time = 608000 + flags = 1 + data = length 2560, hash 1EC4E5AF + sample 20: + time = 640000 + flags = 1 + data = length 2560, hash DCFEB7D2 + sample 21: + time = 672000 + flags = 1 + data = length 2560, hash 45EFC639 + sample 22: + time = 704000 + flags = 1 + data = length 2560, hash F598673 + sample 23: + time = 736000 + flags = 1 + data = length 2560, hash 89E4E5EC + sample 24: + time = 768000 + flags = 1 + data = length 2560, hash FBE2532B + sample 25: + time = 800000 + flags = 1 + data = length 2560, hash 9CE5F83B + sample 26: + time = 832000 + flags = 1 + data = length 2560, hash 6ED49E2C + sample 27: + time = 864000 + flags = 1 + data = length 2560, hash BC52F8F3 + sample 28: + time = 896000 + flags = 1 + data = length 2560, hash 759203E2 + sample 29: + time = 928000 + flags = 1 + data = length 2560, hash D5D31AE9 + sample 30: + time = 960000 + flags = 1 + data = length 2560, hash 640A24ED + sample 31: + time = 992000 + flags = 1 + data = length 2560, hash 19B52B8B + sample 32: + time = 1024000 + flags = 1 + data = length 2560, hash 5DA977C3 + sample 33: + time = 1056000 + flags = 1 + data = length 2560, hash 982879DD + sample 34: + time = 1088000 + flags = 1 + data = length 2560, hash A7656B9C + sample 35: + time = 1120000 + flags = 1 + data = length 2560, hash 445CCC67 + sample 36: + time = 1152000 + flags = 1 + data = length 2560, hash ACD5CB5C + sample 37: + time = 1184000 + flags = 1 + data = length 2560, hash 175BBF26 + sample 38: + time = 1216000 + flags = 1 + data = length 2560, hash DBCBEB0 + sample 39: + time = 1248000 + flags = 1 + data = length 2560, hash DA39D991 + sample 40: + time = 1280000 + flags = 1 + data = length 2560, hash F08CC8E2 + sample 41: + time = 1312000 + flags = 1 + data = length 2560, hash 6B0842D7 + sample 42: + time = 1344000 + flags = 1 + data = length 2560, hash 9FE87594 + sample 43: + time = 1376000 + flags = 1 + data = length 2560, hash 8E62CE19 + sample 44: + time = 1408000 + flags = 1 + data = length 2560, hash 5FDC4084 + sample 45: + time = 1440000 + flags = 1 + data = length 2560, hash C32DAEE1 + sample 46: + time = 1472000 + flags = 1 + data = length 2560, hash BBEFB568 + sample 47: + time = 1504000 + flags = 1 + data = length 2560, hash 20504279 + sample 48: + time = 1536000 + flags = 1 + data = length 2560, hash 3B8192D2 + sample 49: + time = 1568000 + flags = 1 + data = length 2560, hash 4206B48 + sample 50: + time = 1600000 + flags = 1 + data = length 2560, hash B195AB53 + sample 51: + time = 1632000 + flags = 1 + data = length 2560, hash 3AA8E25F + sample 52: + time = 1664000 + flags = 1 + data = length 2560, hash BC227D7B + sample 53: + time = 1696000 + flags = 1 + data = length 2560, hash 6A34F7EA + sample 54: + time = 1728000 + flags = 1 + data = length 2560, hash F1E731C4 + sample 55: + time = 1760000 + flags = 1 + data = length 2560, hash 9CC406 + sample 56: + time = 1792000 + flags = 1 + data = length 2560, hash A1532233 + sample 57: + time = 1824000 + flags = 1 + data = length 2560, hash 98E49039 + sample 58: + time = 1856000 + flags = 1 + data = length 2560, hash 3F8B6DC0 + sample 59: + time = 1888000 + flags = 1 + data = length 2560, hash 4E7BF79F + sample 60: + time = 1920000 + flags = 1 + data = length 2560, hash 6DD6F2D7 + sample 61: + time = 1952000 + flags = 1 + data = length 2560, hash A05C0EC2 + sample 62: + time = 1984000 + flags = 1 + data = length 2560, hash 10C62F30 + sample 63: + time = 2016000 + flags = 1 + data = length 2560, hash EE4F848A +tracksEnded = true diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_edit_list.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_edit_list.mp4.dump index 517f8615b62..8cc7a3493b8 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_edit_list.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_edit_list.mp4.dump @@ -1,10 +1,10 @@ seekMap: isSeekable = true duration = 2680000 - getPosition(0) = [[timeUs=-455000, position=400052], [timeUs=611666, position=411095]] - getPosition(1) = [[timeUs=-455000, position=400052], [timeUs=611666, position=411095]] - getPosition(1340000) = [[timeUs=611666, position=411095], [timeUs=1680000, position=429829]] - getPosition(2680000) = [[timeUs=1680000, position=429829]] + getPosition(0) = [[timeUs=-455000, position=400056], [timeUs=611666, position=411099]] + getPosition(1) = [[timeUs=-455000, position=400056], [timeUs=611666, position=411099]] + getPosition(1340000) = [[timeUs=611666, position=411099], [timeUs=1680000, position=429833]] + getPosition(2680000) = [[timeUs=1680000, position=429833]] numberOfTracks = 3 track 0: total output bytes = 3226787 @@ -753,7 +753,7 @@ track 1: track 2: total output bytes = 45765 sample count = 112 - track duration = 2600600 + track duration = 2600634 format 0: averageBitrate = 140781 peakBitrate = 192000 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_edit_list.mp4_fragmented.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_edit_list.mp4_fragmented.dump index 4ce874ebcfb..711c6e6dd25 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_edit_list.mp4_fragmented.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_edit_list.mp4_fragmented.dump @@ -1,7 +1,7 @@ seekMap: isSeekable = false duration = UNSET TIME - getPosition(0) = [[timeUs=0, position=1909]] + getPosition(0) = [[timeUs=0, position=1913]] numberOfTracks = 3 track 0: total output bytes = 3226787 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_no_bframes.mp4_fragmented.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_no_bframes.mp4_fragmented.dump index 2885178419e..8fd1cb6418d 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_no_bframes.mp4_fragmented.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_no_bframes.mp4_fragmented.dump @@ -19,183 +19,183 @@ track 0: initializationData: data = length 2, hash 5F7 sample 0: - time = 0 + time = 43990 flags = 1 data = length 23, hash 47DE9131 sample 1: - time = 23219 + time = 67210 flags = 1 data = length 6, hash 31EC5206 sample 2: - time = 46439 + time = 90430 flags = 1 data = length 148, hash 894A176B sample 3: - time = 69659 + time = 113650 flags = 1 data = length 189, hash CEF235A1 sample 4: - time = 92879 + time = 136870 flags = 1 data = length 205, hash BBF5F7B0 sample 5: - time = 116099 + time = 160090 flags = 1 data = length 210, hash F278B193 sample 6: - time = 139319 + time = 183310 flags = 1 data = length 210, hash 82DA1589 sample 7: - time = 162539 + time = 206530 flags = 1 data = length 207, hash 5BE231DF sample 8: - time = 185759 + time = 229750 flags = 1 data = length 225, hash 18819EE1 sample 9: - time = 208979 + time = 252970 flags = 1 data = length 215, hash CA7FA67B sample 10: - time = 232199 + time = 276190 flags = 1 data = length 211, hash 581A1C18 sample 11: - time = 255419 + time = 299410 flags = 1 data = length 216, hash ADB88187 sample 12: - time = 278639 + time = 322630 flags = 1 data = length 229, hash 2E8BA4DC sample 13: - time = 301859 + time = 345850 flags = 1 data = length 232, hash 22F0C510 sample 14: - time = 325079 + time = 369070 flags = 1 data = length 235, hash 867AD0DC sample 15: - time = 348299 + time = 392290 flags = 1 data = length 231, hash 84E823A8 sample 16: - time = 371519 + time = 415510 flags = 1 data = length 226, hash 1BEF3A95 sample 17: - time = 394739 + time = 438730 flags = 1 data = length 216, hash EAA345AE sample 18: - time = 417959 + time = 461950 flags = 1 data = length 229, hash 6957411F sample 19: - time = 441179 + time = 485170 flags = 1 data = length 219, hash 41275022 sample 20: - time = 464399 + time = 508390 flags = 1 data = length 241, hash 6495DF96 sample 21: - time = 487619 + time = 531609 flags = 1 data = length 228, hash 63D95906 sample 22: - time = 510839 + time = 554829 flags = 1 data = length 238, hash 34F676F9 sample 23: - time = 534058 + time = 578049 flags = 1 data = length 234, hash E5CBC045 sample 24: - time = 557278 + time = 601269 flags = 1 data = length 231, hash 5FC43661 sample 25: - time = 580498 + time = 624489 flags = 1 data = length 217, hash 682708ED sample 26: - time = 603718 + time = 647709 flags = 1 data = length 239, hash D43780FC sample 27: - time = 626938 + time = 670929 flags = 1 data = length 243, hash C5E17980 sample 28: - time = 650158 + time = 694149 flags = 1 data = length 231, hash AC5837BA sample 29: - time = 673378 + time = 717369 flags = 1 data = length 230, hash 169EE895 sample 30: - time = 696598 + time = 740589 flags = 1 data = length 238, hash C48FF3F1 sample 31: - time = 719818 + time = 763809 flags = 1 data = length 225, hash 531E4599 sample 32: - time = 743038 + time = 787029 flags = 1 data = length 232, hash CB3C6B8D sample 33: - time = 766258 + time = 810249 flags = 1 data = length 243, hash F8C94C7 sample 34: - time = 789478 + time = 833469 flags = 1 data = length 232, hash A646A7D0 sample 35: - time = 812698 + time = 856689 flags = 1 data = length 237, hash E8B787A5 sample 36: - time = 835918 + time = 879909 flags = 1 data = length 228, hash 3FA7A29F sample 37: - time = 859138 + time = 903129 flags = 1 data = length 235, hash B9B33B0A sample 38: - time = 882358 + time = 926349 flags = 1 data = length 264, hash 71A4869E sample 39: - time = 905578 + time = 949569 flags = 1 data = length 257, hash D049B54C sample 40: - time = 928798 + time = 972789 flags = 1 data = length 227, hash 66757231 sample 41: - time = 952018 + time = 996009 flags = 1 data = length 227, hash BD374F1B sample 42: - time = 975238 + time = 1019229 flags = 1 data = length 235, hash 999477F6 sample 43: - time = 998458 + time = 1042448 flags = 1 data = length 229, hash FFF98DF0 sample 44: - time = 1021678 + time = 1065668 flags = 1 data = length 6, hash 31B22286 track 1: diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_with_apvc.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_with_apvc.mp4.dump index ff08fa1aad7..d476edd4db8 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_with_apvc.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_with_apvc.mp4.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 4166600 + duration = 4166622 getPosition(0) = [[timeUs=0, position=400052]] getPosition(1) = [[timeUs=0, position=400052], [timeUs=33333, position=452429]] - getPosition(2083300) = [[timeUs=2066644, position=4412397], [timeUs=2099977, position=4482649]] - getPosition(4166600) = [[timeUs=4133288, position=8785560]] + getPosition(2083311) = [[timeUs=2066644, position=4412397], [timeUs=2099977, position=4482649]] + getPosition(4166622) = [[timeUs=4133288, position=8785560]] numberOfTracks = 1 track 0: total output bytes = 8456438 sample count = 125 - track duration = 4166600 + track duration = 4166622 format 0: averageBitrate = 16236535 id = 1 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_with_apvc.mp4_fragmented.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_with_apvc.mp4_fragmented.dump index 91fa6738d30..87382d5b489 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_with_apvc.mp4_fragmented.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_with_apvc.mp4_fragmented.dump @@ -268,255 +268,255 @@ track 0: flags = 1 data = length 70213, hash 98B3BE58 sample 62: - time = 2066644 + time = 2066633 flags = 1 data = length 70252, hash E1145A23 sample 63: - time = 2099977 + time = 2099966 flags = 1 data = length 70269, hash 347261C0 sample 64: - time = 2133311 + time = 2133300 flags = 1 data = length 70248, hash 63134CFF sample 65: - time = 2166644 + time = 2166633 flags = 1 data = length 70251, hash EDFC2BE9 sample 66: - time = 2199977 + time = 2199966 flags = 1 data = length 70288, hash BED92C89 sample 67: - time = 2233311 + time = 2233300 flags = 1 data = length 70310, hash 5406B02E sample 68: - time = 2266644 + time = 2266633 flags = 1 data = length 70742, hash 3D8DD8F sample 69: - time = 2299977 + time = 2299966 flags = 1 data = length 70729, hash 2120E912 sample 70: - time = 2333311 + time = 2333300 flags = 1 data = length 70792, hash 66750C58 sample 71: - time = 2366644 + time = 2366633 flags = 1 data = length 70735, hash 4362F81D sample 72: - time = 2399977 + time = 2399966 flags = 1 data = length 70709, hash 7602ABFF sample 73: - time = 2433311 + time = 2433300 flags = 1 data = length 70680, hash 4B2F5612 sample 74: - time = 2466644 + time = 2466633 flags = 1 data = length 70680, hash C5A0912C sample 75: - time = 2499977 + time = 2499966 flags = 1 data = length 70731, hash E06EE04 sample 76: - time = 2533311 + time = 2533300 flags = 1 data = length 70817, hash BEE7E9D4 sample 77: - time = 2566644 + time = 2566633 flags = 1 data = length 70727, hash CDCB8CDE sample 78: - time = 2599977 + time = 2599966 flags = 1 data = length 70660, hash 4F35C9FE sample 79: - time = 2633311 + time = 2633300 flags = 1 data = length 70646, hash CF60E125 sample 80: - time = 2666644 + time = 2666633 flags = 1 data = length 70572, hash 261E908C sample 81: - time = 2699977 + time = 2699966 flags = 1 data = length 70559, hash 904DA73A sample 82: - time = 2733311 + time = 2733300 flags = 1 data = length 70647, hash 7658C29 sample 83: - time = 2766644 + time = 2766633 flags = 1 data = length 70495, hash 483A64AF sample 84: - time = 2799966 + time = 2799955 flags = 1 data = length 70462, hash 51487309 sample 85: - time = 2833300 + time = 2833288 flags = 1 data = length 70710, hash 702CCEC5 sample 86: - time = 2866633 + time = 2866622 flags = 1 data = length 70673, hash D0BEB6EF sample 87: - time = 2899966 + time = 2899955 flags = 1 data = length 70655, hash 5CC59E74 sample 88: - time = 2933300 + time = 2933288 flags = 1 data = length 70581, hash DDF1BD6E sample 89: - time = 2966633 + time = 2966622 flags = 1 data = length 70609, hash 68B4B148 sample 90: - time = 2999966 + time = 2999955 flags = 1 data = length 70525, hash 30C40E7 sample 91: - time = 3033300 + time = 3033288 flags = 1 data = length 70481, hash D92EBDAD sample 92: - time = 3066633 + time = 3066622 flags = 1 data = length 70719, hash D989799F sample 93: - time = 3099966 + time = 3099955 flags = 1 data = length 70723, hash 54AC0C9E sample 94: - time = 3133300 + time = 3133288 flags = 1 data = length 70648, hash 1E2B3C17 sample 95: - time = 3166633 + time = 3166622 flags = 1 data = length 70604, hash 32415786 sample 96: - time = 3199966 + time = 3199955 flags = 1 data = length 70551, hash E6264E36 sample 97: - time = 3233300 + time = 3233288 flags = 1 data = length 70594, hash AAB4BFFA sample 98: - time = 3266633 + time = 3266622 flags = 1 data = length 70578, hash 10C3F52E sample 99: - time = 3299966 + time = 3299955 flags = 1 data = length 70501, hash 66DE5232 sample 100: - time = 3333300 + time = 3333288 flags = 1 data = length 70170, hash 1B6AAF6C sample 101: - time = 3366633 + time = 3366622 flags = 1 data = length 70229, hash 3C050C5F sample 102: - time = 3399966 + time = 3399955 flags = 1 data = length 70225, hash 815C3843 sample 103: - time = 3433300 + time = 3433288 flags = 1 data = length 70209, hash 926B6B46 sample 104: - time = 3466633 + time = 3466622 flags = 1 data = length 70217, hash 7C6BB0E sample 105: - time = 3499966 + time = 3499955 flags = 1 data = length 70245, hash 38AE5D4C sample 106: - time = 3533300 + time = 3533288 flags = 1 data = length 70230, hash 5D96C49C sample 107: - time = 3566633 + time = 3566622 flags = 1 data = length 70267, hash 4B1D3FB1 sample 108: - time = 3599966 + time = 3599955 flags = 1 data = length 70190, hash 56814D6D sample 109: - time = 3633300 + time = 3633288 flags = 1 data = length 70314, hash 8DE18BA4 sample 110: - time = 3666633 + time = 3666622 flags = 1 data = length 70333, hash F9958CFA sample 111: - time = 3699966 + time = 3699955 flags = 1 data = length 70349, hash 477CE6F1 sample 112: - time = 3733300 + time = 3733288 flags = 1 data = length 70451, hash 8B63B929 sample 113: - time = 3766633 + time = 3766622 flags = 1 data = length 70440, hash EE1D1A34 sample 114: - time = 3799966 + time = 3799955 flags = 1 data = length 70494, hash 278D4DFC sample 115: - time = 3833300 + time = 3833288 flags = 1 data = length 70561, hash 88B5A2D sample 116: - time = 3866633 + time = 3866622 flags = 1 data = length 70521, hash B37703F9 sample 117: - time = 3899955 + time = 3899944 flags = 1 data = length 70663, hash F4B7AA3D sample 118: - time = 3933288 + time = 3933277 flags = 1 data = length 70697, hash 3974C162 sample 119: - time = 3966622 + time = 3966611 flags = 1 data = length 70731, hash 482111DF sample 120: - time = 3999955 + time = 3999944 flags = 1 data = length 70818, hash 33C78B98 sample 121: - time = 4033288 + time = 4033277 flags = 1 data = length 70852, hash C7515D29 sample 122: - time = 4066622 + time = 4066611 flags = 1 data = length 70860, hash 35E3AE4 sample 123: - time = 4099955 + time = 4099944 flags = 1 data = length 70944, hash 7A711E1B sample 124: - time = 4133288 + time = 4133277 flags = 1 data = length 70930, hash 2CD9EE13 tracksEnded = true diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_with_it35_track.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_with_it35_track.mp4.dump new file mode 100644 index 00000000000..32651fbc484 --- /dev/null +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/sample_with_it35_track.mp4.dump @@ -0,0 +1,286 @@ +seekMap: + isSeekable = true + duration = 1002777 + getPosition(0) = [[timeUs=0, position=400052]] + getPosition(1) = [[timeUs=0, position=400052]] + getPosition(501388) = [[timeUs=0, position=400052]] + getPosition(1002777) = [[timeUs=0, position=400052]] +numberOfTracks = 2 +track 0: + total output bytes = 43683 + sample count = 30 + track duration = 1002777 + format 0: + averageBitrate = 348496 + id = 1 + containerMimeType = video/mp4 + sampleMimeType = video/av01 + codecs = av01.0.00M.10 + maxInputSize = 18085 + width = 480 + height = 270 + frameRate = 29.92 + colorInfo: + colorSpace = 6 + colorRange = 2 + colorTransfer = 6 + lumaBitdepth = 10 + chromaBitdepth = 10 + language = en + metadata = entries=[Mp4Timestamp: creation time=100000000, modification time=500000000, timescale=10000, ThumbnailMetadata: presentationTimeUs=0] + initializationData: + data = length 20, hash 27B68A02 + sample 0: + time = 0 + flags = 1 + data = length 18055, hash 4A4CEBD8 + sample 1: + time = 33422 + flags = 0 + data = length 10390, hash 1A97B887 + sample 2: + time = 66855 + flags = 0 + data = length 3, hash D5E0 + sample 3: + time = 100277 + flags = 0 + data = length 111, hash F52A3D2C + sample 4: + time = 133700 + flags = 0 + data = length 3, hash D5D0 + sample 5: + time = 167133 + flags = 0 + data = length 748, hash 7FA30B17 + sample 6: + time = 200555 + flags = 0 + data = length 3, hash D610 + sample 7: + time = 233977 + flags = 0 + data = length 94, hash D426B3A3 + sample 8: + time = 267411 + flags = 0 + data = length 3, hash D5C0 + sample 9: + time = 300833 + flags = 0 + data = length 1487, hash 1B89F2D3 + sample 10: + time = 334255 + flags = 0 + data = length 3, hash D5D0 + sample 11: + time = 367688 + flags = 0 + data = length 243, hash 83AC8EF9 + sample 12: + time = 401111 + flags = 0 + data = length 3, hash D600 + sample 13: + time = 434544 + flags = 0 + data = length 719, hash 3F9A2D2F + sample 14: + time = 467966 + flags = 0 + data = length 3, hash D5E0 + sample 15: + time = 501388 + flags = 0 + data = length 398, hash 29898A8D + sample 16: + time = 534822 + flags = 0 + data = length 200, hash 18678644 + sample 17: + time = 568244 + flags = 0 + data = length 3, hash D5B0 + sample 18: + time = 601666 + flags = 0 + data = length 8443, hash 90C5F9E + sample 19: + time = 635100 + flags = 0 + data = length 3, hash D5C0 + sample 20: + time = 668522 + flags = 0 + data = length 299, hash 3219B04 + sample 21: + time = 701944 + flags = 0 + data = length 222, hash FBB32D8 + sample 22: + time = 735377 + flags = 0 + data = length 3, hash D600 + sample 23: + time = 768800 + flags = 0 + data = length 645, hash 3FF6F2DF + sample 24: + time = 802233 + flags = 0 + data = length 3, hash D5E0 + sample 25: + time = 835655 + flags = 0 + data = length 251, hash 83CE17A6 + sample 26: + time = 869077 + flags = 0 + data = length 253, hash 34714BEA + sample 27: + time = 902511 + flags = 0 + data = length 3, hash D610 + sample 28: + time = 935933 + flags = 0 + data = length 482, hash 4D8E647E + sample 29: + time = 969355 + flags = 536870912 + data = length 607, hash 230AC585 +track 1: + total output bytes = 7350 + sample count = 30 + track duration = 1002777 + format 0: + averageBitrate = 58637 + id = 2 + containerMimeType = video/mp4 + sampleMimeType = application/x-itut-t35 + maxInputSize = 275 + metadata = entries=[Mp4Timestamp: creation time=100000000, modification time=500000000, timescale=10000] + initializationData: + data = length 5, hash FD925165 + sample 0: + time = 0 + flags = 1 + data = length 245, hash 9BFA0B + sample 1: + time = 33422 + flags = 1 + data = length 245, hash EA2065CD + sample 2: + time = 66855 + flags = 1 + data = length 245, hash F3A0DA5C + sample 3: + time = 100277 + flags = 1 + data = length 245, hash CF1BDEB2 + sample 4: + time = 133700 + flags = 1 + data = length 245, hash BA53650F + sample 5: + time = 167133 + flags = 1 + data = length 245, hash 88941B39 + sample 6: + time = 200555 + flags = 1 + data = length 245, hash 302A69B6 + sample 7: + time = 233977 + flags = 1 + data = length 245, hash 86A9B7CF + sample 8: + time = 267411 + flags = 1 + data = length 245, hash 6C8AF8DF + sample 9: + time = 300833 + flags = 1 + data = length 245, hash A8D264F + sample 10: + time = 334255 + flags = 1 + data = length 245, hash BF90E97B + sample 11: + time = 367688 + flags = 1 + data = length 245, hash 3A036FF7 + sample 12: + time = 401111 + flags = 1 + data = length 245, hash 4C76613D + sample 13: + time = 434544 + flags = 1 + data = length 245, hash 9C103E0D + sample 14: + time = 467966 + flags = 1 + data = length 245, hash 546611FD + sample 15: + time = 501388 + flags = 1 + data = length 245, hash 2EEAD77C + sample 16: + time = 534822 + flags = 1 + data = length 245, hash 45EE18FD + sample 17: + time = 568244 + flags = 1 + data = length 245, hash 6C4A5B60 + sample 18: + time = 601666 + flags = 1 + data = length 245, hash C55E7F4F + sample 19: + time = 635100 + flags = 1 + data = length 245, hash 8C43E61B + sample 20: + time = 668522 + flags = 1 + data = length 245, hash 88CB156F + sample 21: + time = 701944 + flags = 1 + data = length 245, hash 975E8975 + sample 22: + time = 735377 + flags = 1 + data = length 245, hash D7DAF392 + sample 23: + time = 768800 + flags = 1 + data = length 245, hash 72F8BC8A + sample 24: + time = 802233 + flags = 1 + data = length 245, hash 69F05BD3 + sample 25: + time = 835655 + flags = 1 + data = length 245, hash E12DFF0E + sample 26: + time = 869077 + flags = 1 + data = length 245, hash 5DD18D72 + sample 27: + time = 902511 + flags = 1 + data = length 245, hash 2B6AB923 + sample 28: + time = 935933 + flags = 1 + data = length 245, hash 68205A3F + sample 29: + time = 969355 + flags = 536870913 + data = length 245, hash 7D03F36A +tracksEnded = true diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/stsz_box_with_identically_sized_samples.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/stsz_box_with_identically_sized_samples.dump new file mode 100644 index 00000000000..49479c1a6fe --- /dev/null +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/stsz_box_with_identically_sized_samples.dump @@ -0,0 +1,2 @@ +stsz (20 bytes): + Data = length 12, hash E8AD4B1B diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/tref_box_with_multiple_track_reference.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/tref_box_with_multiple_track_reference.dump index 1885b4d1e58..65617dc51d3 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/tref_box_with_multiple_track_reference.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/tref_box_with_multiple_track_reference.dump @@ -1,2 +1,2 @@ tref (40 bytes): - Data = length 32, hash FEC3528B + Data = length 32, hash 3512300F diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/tref_box_with_one_track_reference.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/tref_box_with_one_track_reference.dump index 4ba977c7386..c91cc863461 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/tref_box_with_one_track_reference.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/tref_box_with_one_track_reference.dump @@ -1,2 +1,2 @@ tref (24 bytes): - Data = length 16, hash EA78AAEB + Data = length 16, hash 6F65E305 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/video_dovi_1920x1080_60fps_dvav_09.mp4.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/video_dovi_1920x1080_60fps_dvav_09.mp4.dump index 582b1a1bb3d..94f9063542d 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/video_dovi_1920x1080_60fps_dvav_09.mp4.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/video_dovi_1920x1080_60fps_dvav_09.mp4.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 1284200 - getPosition(0) = [[timeUs=0, position=400052]] - getPosition(1) = [[timeUs=0, position=400052]] - getPosition(642100) = [[timeUs=0, position=400052]] - getPosition(1284200) = [[timeUs=0, position=400052]] + duration = 1284188 + getPosition(0) = [[timeUs=0, position=400056]] + getPosition(1) = [[timeUs=0, position=400056]] + getPosition(642094) = [[timeUs=0, position=400056]] + getPosition(1284188) = [[timeUs=0, position=400056]] numberOfTracks = 2 track 0: total output bytes = 3324667 sample count = 77 - track duration = 1284200 + track duration = 1284188 format 0: averageBitrate = 20711404 id = 1 @@ -340,7 +340,7 @@ track 0: track 1: total output bytes = 3324667 sample count = 77 - track duration = 1284200 + track duration = 1284188 format 0: averageBitrate = 20711404 id = 2 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4/video_dovi_1920x1080_60fps_dvav_09.mp4_fragmented.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4/video_dovi_1920x1080_60fps_dvav_09.mp4_fragmented.dump index ed81e8895a7..846f51eb87d 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4/video_dovi_1920x1080_60fps_dvav_09.mp4_fragmented.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4/video_dovi_1920x1080_60fps_dvav_09.mp4_fragmented.dump @@ -1,7 +1,7 @@ seekMap: isSeekable = false duration = UNSET TIME - getPosition(0) = [[timeUs=0, position=1284]] + getPosition(0) = [[timeUs=0, position=1288]] numberOfTracks = 2 track 0: total output bytes = 3324667 diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4_with_t35_metadata.box.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4_with_t35_metadata.box.dump index 3e02f95f2ef..f0db0e4f11f 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4_with_t35_metadata.box.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4_with_t35_metadata.box.dump @@ -1,28 +1,28 @@ ftyp (28 bytes): Data = length 20, hash EF896440 -moov (2454 bytes): +moov (2434 bytes): mvhd (108 bytes): Data = length 100, hash 17D30507 - trak (433 bytes): + trak (413 bytes): tkhd (92 bytes): Data = length 84, hash 5C2D954E - mdia (333 bytes): + mdia (313 bytes): mdhd (32 bytes): Data = length 24, hash B4067EFA hdlr (43 bytes): Data = length 35, hash 3A96CDAD - minf (250 bytes): + minf (230 bytes): nmhd (12 bytes): Data = length 4, hash E1781 dinf (36 bytes): Data = length 28, hash D535436B - stbl (194 bytes): + stbl (174 bytes): stsd (38 bytes): Data = length 30, hash 7844C3 stts (24 bytes): Data = length 16, hash E534C6EC - stsz (40 bytes): - Data = length 32, hash AD2339A + stsz (20 bytes): + Data = length 12, hash EEBB648A stsc (28 bytes): Data = length 20, hash 8F6E8285 co64 (56 bytes): @@ -80,7 +80,7 @@ moov (2454 bytes): Data = length 20, hash 8F6E8285 co64 (376 bytes): Data = length 368, hash 24CFEB70 -free (397554 bytes): - Data = length 397546, hash A45BD6C1 +free (397574 bytes): + Data = length 397566, hash 4565EC41 mdat (78432 bytes): Data = length 78416, hash 50713FCB diff --git a/libraries/test_data/src/test/assets/muxerdumps/mp4_with_t35_metadata_track_referencing_video_track.box.dump b/libraries/test_data/src/test/assets/muxerdumps/mp4_with_t35_metadata_track_referencing_video_track.box.dump index 5d99312f6e1..6dd3a89623c 100644 --- a/libraries/test_data/src/test/assets/muxerdumps/mp4_with_t35_metadata_track_referencing_video_track.box.dump +++ b/libraries/test_data/src/test/assets/muxerdumps/mp4_with_t35_metadata_track_referencing_video_track.box.dump @@ -1,85 +1,85 @@ ftyp (28 bytes): Data = length 20, hash EF896440 -moov (1666 bytes): +moov (1606 bytes): mvhd (108 bytes): Data = length 100, hash 2613A5E - trak (606 bytes): + trak (586 bytes): tkhd (92 bytes): Data = length 84, hash 3D79758F - mdia (506 bytes): + mdia (486 bytes): mdhd (32 bytes): Data = length 24, hash 41542D81 hdlr (44 bytes): Data = length 36, hash A0852FF2 - minf (422 bytes): + minf (402 bytes): vmhd (20 bytes): Data = length 12, hash EE830681 dinf (36 bytes): Data = length 28, hash D535436B - stbl (358 bytes): + stbl (338 bytes): stsd (166 bytes): Data = length 158, hash 11532063 stts (24 bytes): Data = length 16, hash E534C287 - stsz (40 bytes): - Data = length 32, hash B3F09E + stsz (20 bytes): + Data = length 12, hash F1982ABE stsc (28 bytes): Data = length 20, hash 8F6E8285 co64 (56 bytes): Data = length 48, hash A4175473 stss (36 bytes): Data = length 28, hash 53024615 - trak (491 bytes): + trak (471 bytes): tkhd (92 bytes): Data = length 84, hash EDCABA59 - mdia (391 bytes): + mdia (371 bytes): mdhd (32 bytes): Data = length 24, hash 878AA0D3 hdlr (44 bytes): Data = length 36, hash 49FC755F - minf (307 bytes): + minf (287 bytes): smhd (16 bytes): Data = length 8, hash 94446F01 dinf (36 bytes): Data = length 28, hash D535436B - stbl (247 bytes): + stbl (227 bytes): stsd (91 bytes): Data = length 83, hash 12DB93AB stts (24 bytes): Data = length 16, hash E534C287 - stsz (40 bytes): - Data = length 32, hash 2D348599 + stsz (20 bytes): + Data = length 12, hash F18A133D stsc (28 bytes): Data = length 20, hash 8F6E8285 co64 (56 bytes): Data = length 48, hash 628BCC9B - trak (453 bytes): + trak (433 bytes): tkhd (92 bytes): Data = length 84, hash A551A7FB tref (20 bytes): - Data = length 12, hash 2B3ACB65 - mdia (333 bytes): + Data = length 12, hash 816C207F + mdia (313 bytes): mdhd (32 bytes): Data = length 24, hash 41542D81 hdlr (43 bytes): Data = length 35, hash 3A96CDAD - minf (250 bytes): + minf (230 bytes): nmhd (12 bytes): Data = length 4, hash E1781 dinf (36 bytes): Data = length 28, hash D535436B - stbl (194 bytes): + stbl (174 bytes): stsd (38 bytes): Data = length 30, hash 7844C3 stts (24 bytes): Data = length 16, hash E534C287 - stsz (40 bytes): - Data = length 32, hash 2D348599 + stsz (20 bytes): + Data = length 12, hash F18A133D stsc (28 bytes): Data = length 20, hash 8F6E8285 co64 (56 bytes): Data = length 48, hash 3A399895 -free (398342 bytes): - Data = length 398334, hash 28C78C41 +free (398402 bytes): + Data = length 398394, hash 97EB4CC1 mdat (846 bytes): Data = length 830, hash 5F02BE83 diff --git a/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_extend_after_load_finished_enabled.dump b/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_extend_after_load_finished_enabled.dump new file mode 100644 index 00000000000..f16a90437a2 --- /dev/null +++ b/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_extend_after_load_finished_enabled.dump @@ -0,0 +1,3497 @@ +extracted samples: + sample #0: + track = 0 + timeUs = 0 + flags = 1 + size = 5252 + sample #1: + track = 0 + timeUs = 160000 + size = 44 + sample #2: + track = 0 + timeUs = 80000 + size = 25 + sample #3: + track = 0 + timeUs = 40000 + flags = 67108864 + size = 18 + sample #4: + track = 1 + timeUs = 92879 + flags = 1 + size = 4 + sample #5: + track = 0 + timeUs = 120000 + flags = 67108864 + size = 18 + sample #6: + track = 1 + timeUs = 116099 + flags = 1 + size = 4 + sample #7: + track = 1 + timeUs = 139319 + flags = 1 + size = 4 + sample #8: + track = 0 + timeUs = 320000 + size = 32 + sample #9: + track = 1 + timeUs = 162539 + flags = 1 + size = 4 + sample #10: + track = 0 + timeUs = 240000 + size = 27 + sample #11: + track = 1 + timeUs = 185759 + flags = 1 + size = 4 + sample #12: + track = 1 + timeUs = 208979 + flags = 1 + size = 4 + sample #13: + track = 0 + timeUs = 200000 + flags = 67108864 + size = 18 + sample #14: + track = 1 + timeUs = 232199 + flags = 1 + size = 4 + sample #15: + track = 1 + timeUs = 255419 + flags = 1 + size = 4 + sample #16: + track = 0 + timeUs = 280000 + flags = 67108864 + size = 18 + sample #17: + track = 1 + timeUs = 278639 + flags = 1 + size = 4 + sample #18: + track = 1 + timeUs = 301859 + flags = 1 + size = 4 + sample #19: + track = 0 + timeUs = 480000 + size = 28 + sample #20: + track = 1 + timeUs = 325079 + flags = 1 + size = 4 + sample #21: + track = 0 + timeUs = 400000 + size = 27 + sample #22: + track = 1 + timeUs = 348299 + flags = 1 + size = 4 + sample #23: + track = 1 + timeUs = 371519 + flags = 1 + size = 4 + sample #24: + track = 0 + timeUs = 360000 + flags = 67108864 + size = 18 + sample #25: + track = 1 + timeUs = 394739 + flags = 1 + size = 4 + sample #26: + track = 1 + timeUs = 417959 + flags = 1 + size = 4 + sample #27: + track = 0 + timeUs = 440000 + flags = 67108864 + size = 18 + sample #28: + track = 1 + timeUs = 441179 + flags = 1 + size = 4 + sample #29: + track = 0 + timeUs = 640000 + size = 28 + sample #30: + track = 1 + timeUs = 464399 + flags = 1 + size = 4 + sample #31: + track = 1 + timeUs = 487619 + flags = 1 + size = 4 + sample #32: + track = 0 + timeUs = 560000 + size = 27 + sample #33: + track = 1 + timeUs = 510839 + flags = 1 + size = 4 + sample #34: + track = 1 + timeUs = 534058 + flags = 1 + size = 4 + sample #35: + track = 0 + timeUs = 520000 + flags = 67108864 + size = 18 + sample #36: + track = 1 + timeUs = 557278 + flags = 1 + size = 4 + sample #37: + track = 1 + timeUs = 580498 + flags = 1 + size = 4 + sample #38: + track = 0 + timeUs = 600000 + flags = 67108864 + size = 18 + sample #39: + track = 1 + timeUs = 603718 + flags = 1 + size = 4 + sample #40: + track = 0 + timeUs = 800000 + size = 28 + sample #41: + track = 1 + timeUs = 626938 + flags = 1 + size = 4 + sample #42: + track = 1 + timeUs = 650158 + flags = 1 + size = 4 + sample #43: + track = 0 + timeUs = 720000 + size = 27 + sample #44: + track = 1 + timeUs = 673378 + flags = 1 + size = 4 + sample #45: + track = 1 + timeUs = 696598 + flags = 1 + size = 4 + sample #46: + track = 0 + timeUs = 680000 + flags = 67108864 + size = 18 + sample #47: + track = 1 + timeUs = 719818 + flags = 1 + size = 4 + sample #48: + track = 1 + timeUs = 743038 + flags = 1 + size = 4 + sample #49: + track = 0 + timeUs = 760000 + flags = 67108864 + size = 18 + sample #50: + track = 1 + timeUs = 766258 + flags = 1 + size = 4 + sample #51: + track = 0 + timeUs = 960000 + size = 28 + sample #52: + track = 1 + timeUs = 789478 + flags = 1 + size = 4 + sample #53: + track = 1 + timeUs = 812698 + flags = 1 + size = 4 + sample #54: + track = 0 + timeUs = 880000 + size = 27 + sample #55: + track = 1 + timeUs = 835918 + flags = 1 + size = 4 + sample #56: + track = 1 + timeUs = 859138 + flags = 1 + size = 4 + sample #57: + track = 0 + timeUs = 840000 + flags = 67108864 + size = 18 + sample #58: + track = 1 + timeUs = 882358 + flags = 1 + size = 4 + sample #59: + track = 0 + timeUs = 920000 + flags = 67108864 + size = 18 + sample #60: + track = 1 + timeUs = 905578 + flags = 1 + size = 4 + sample #61: + track = 1 + timeUs = 928798 + flags = 1 + size = 4 + sample #62: + track = 0 + timeUs = 1120000 + size = 28 + sample #63: + track = 1 + timeUs = 952018 + flags = 1 + size = 4 + sample #64: + track = 1 + timeUs = 975238 + flags = 1 + size = 4 + sample #65: + track = 0 + timeUs = 1040000 + size = 27 + sample #66: + track = 1 + timeUs = 998458 + flags = 1 + size = 4 + sample #67: + track = 1 + timeUs = 1021678 + flags = 1 + size = 4 + sample #68: + track = 0 + timeUs = 1000000 + flags = 67108864 + size = 18 + sample #69: + track = 1 + timeUs = 1044897 + flags = 1 + size = 4 + sample #70: + track = 0 + timeUs = 1080000 + flags = 67108864 + size = 18 + sample #71: + track = 1 + timeUs = 1068117 + flags = 1 + size = 4 + sample #72: + track = 1 + timeUs = 1091337 + flags = 1 + size = 4 + sample #73: + track = 0 + timeUs = 1280000 + size = 28 + sample #74: + track = 1 + timeUs = 1114557 + flags = 1 + size = 4 + sample #75: + track = 1 + timeUs = 1137777 + flags = 1 + size = 4 + sample #76: + track = 0 + timeUs = 1200000 + size = 27 + sample #77: + track = 1 + timeUs = 1160997 + flags = 1 + size = 4 + sample #78: + track = 0 + timeUs = 1160000 + flags = 67108864 + size = 18 + sample #79: + track = 1 + timeUs = 1184217 + flags = 1 + size = 4 + sample #80: + track = 1 + timeUs = 1207437 + flags = 1 + size = 4 + sample #81: + track = 0 + timeUs = 1240000 + flags = 67108864 + size = 18 + sample #82: + track = 1 + timeUs = 1230657 + flags = 1 + size = 4 + sample #83: + track = 1 + timeUs = 1253877 + flags = 1 + size = 4 + sample #84: + track = 0 + timeUs = 1440000 + size = 28 + sample #85: + track = 1 + timeUs = 1277097 + flags = 1 + size = 4 + sample #86: + track = 1 + timeUs = 1300317 + flags = 1 + size = 4 + sample #87: + track = 0 + timeUs = 1360000 + size = 27 + sample #88: + track = 1 + timeUs = 1323537 + flags = 1 + size = 4 + sample #89: + track = 0 + timeUs = 1320000 + flags = 67108864 + size = 18 + sample #90: + track = 1 + timeUs = 1346757 + flags = 1 + size = 4 + sample #91: + track = 1 + timeUs = 1369977 + flags = 1 + size = 4 + sample #92: + track = 0 + timeUs = 1400000 + flags = 67108864 + size = 18 + sample #93: + track = 1 + timeUs = 1393197 + flags = 1 + size = 4 + sample #94: + track = 1 + timeUs = 1416417 + flags = 1 + size = 4 + sample #95: + track = 0 + timeUs = 1600000 + size = 28 + sample #96: + track = 1 + timeUs = 1439637 + flags = 1 + size = 4 + sample #97: + track = 1 + timeUs = 1462857 + flags = 1 + size = 4 + sample #98: + track = 0 + timeUs = 1520000 + size = 27 + sample #99: + track = 1 + timeUs = 1486077 + flags = 1 + size = 4 + sample #100: + track = 0 + timeUs = 1480000 + flags = 67108864 + size = 18 + sample #101: + track = 1 + timeUs = 1509297 + flags = 1 + size = 4 + sample #102: + track = 1 + timeUs = 1532517 + flags = 1 + size = 4 + sample #103: + track = 0 + timeUs = 1560000 + flags = 67108864 + size = 18 + sample #104: + track = 1 + timeUs = 1555736 + flags = 1 + size = 4 + sample #105: + track = 1 + timeUs = 1578956 + flags = 1 + size = 4 + sample #106: + track = 0 + timeUs = 1760000 + size = 28 + sample #107: + track = 1 + timeUs = 1602176 + flags = 1 + size = 4 + sample #108: + track = 0 + timeUs = 1680000 + size = 27 + sample #109: + track = 1 + timeUs = 1625396 + flags = 1 + size = 4 + sample #110: + track = 1 + timeUs = 1648616 + flags = 1 + size = 4 + sample #111: + track = 0 + timeUs = 1640000 + flags = 67108864 + size = 18 + sample #112: + track = 1 + timeUs = 1671836 + flags = 1 + size = 4 + sample #113: + track = 1 + timeUs = 1695056 + flags = 1 + size = 4 + sample #114: + track = 0 + timeUs = 1720000 + flags = 67108864 + size = 18 + sample #115: + track = 1 + timeUs = 1718276 + flags = 1 + size = 4 + sample #116: + track = 1 + timeUs = 1741496 + flags = 1 + size = 4 + sample #117: + track = 0 + timeUs = 1920000 + size = 28 + sample #118: + track = 1 + timeUs = 1764716 + flags = 1 + size = 4 + sample #119: + track = 0 + timeUs = 1840000 + size = 27 + sample #120: + track = 1 + timeUs = 1787936 + flags = 1 + size = 4 + sample #121: + track = 1 + timeUs = 1811156 + flags = 1 + size = 4 + sample #122: + track = 0 + timeUs = 1800000 + flags = 67108864 + size = 18 + sample #123: + track = 1 + timeUs = 1834376 + flags = 1 + size = 4 + sample #124: + track = 1 + timeUs = 1857596 + flags = 1 + size = 4 + sample #125: + track = 0 + timeUs = 1880000 + flags = 67108864 + size = 18 + sample #126: + track = 1 + timeUs = 1880816 + flags = 1 + size = 4 + sample #127: + track = 0 + timeUs = 2080000 + size = 28 + sample #128: + track = 1 + timeUs = 1904036 + flags = 1 + size = 4 + sample #129: + track = 1 + timeUs = 1927256 + flags = 1 + size = 4 + sample #130: + track = 0 + timeUs = 2000000 + size = 27 + sample #131: + track = 1 + timeUs = 1950476 + flags = 1 + size = 4 + sample #132: + track = 1 + timeUs = 1973696 + flags = 1 + size = 4 + sample #133: + track = 0 + timeUs = 1960000 + flags = 67108864 + size = 18 + sample #134: + track = 1 + timeUs = 1996916 + flags = 1 + size = 4 + sample #135: + track = 1 + timeUs = 2020136 + flags = 1 + size = 4 + sample #136: + track = 0 + timeUs = 2040000 + flags = 67108864 + size = 18 + sample #137: + track = 1 + timeUs = 2043356 + flags = 1 + size = 4 + sample #138: + track = 0 + timeUs = 2240000 + size = 28 + sample #139: + track = 1 + timeUs = 2066575 + flags = 1 + size = 4 + sample #140: + track = 1 + timeUs = 2089795 + flags = 1 + size = 4 + sample #141: + track = 0 + timeUs = 2160000 + size = 27 + sample #142: + track = 0 + timeUs = 0 + flags = 1 + size = 5252 + sample #143: + track = 0 + timeUs = 160000 + size = 44 + sample #144: + track = 1 + timeUs = 0 + flags = 1 + size = 21 + sample #145: + track = 0 + timeUs = 80000 + size = 25 + sample #146: + track = 1 + timeUs = 23219 + flags = 1 + size = 4 + sample #147: + track = 1 + timeUs = 46439 + flags = 1 + size = 4 + sample #148: + track = 0 + timeUs = 40000 + flags = 67108864 + size = 18 + sample #149: + track = 1 + timeUs = 69659 + flags = 1 + size = 4 + sample #150: + track = 1 + timeUs = 92879 + flags = 1 + size = 4 + sample #151: + track = 0 + timeUs = 120000 + flags = 67108864 + size = 18 + sample #152: + track = 1 + timeUs = 116099 + flags = 1 + size = 4 + sample #153: + track = 1 + timeUs = 139319 + flags = 1 + size = 4 + sample #154: + track = 0 + timeUs = 320000 + size = 32 + sample #155: + track = 1 + timeUs = 162539 + flags = 1 + size = 4 + sample #156: + track = 0 + timeUs = 240000 + size = 27 + sample #157: + track = 1 + timeUs = 185759 + flags = 1 + size = 4 + sample #158: + track = 1 + timeUs = 208979 + flags = 1 + size = 4 + sample #159: + track = 0 + timeUs = 200000 + flags = 67108864 + size = 18 + sample #160: + track = 1 + timeUs = 232199 + flags = 1 + size = 4 + sample #161: + track = 1 + timeUs = 255419 + flags = 1 + size = 4 + sample #162: + track = 0 + timeUs = 280000 + flags = 67108864 + size = 18 + sample #163: + track = 1 + timeUs = 278639 + flags = 1 + size = 4 + sample #164: + track = 1 + timeUs = 301859 + flags = 1 + size = 4 + sample #165: + track = 0 + timeUs = 480000 + size = 28 + sample #166: + track = 1 + timeUs = 325079 + flags = 1 + size = 4 + sample #167: + track = 0 + timeUs = 400000 + size = 27 + sample #168: + track = 1 + timeUs = 348299 + flags = 1 + size = 4 + sample #169: + track = 1 + timeUs = 371519 + flags = 1 + size = 4 + sample #170: + track = 0 + timeUs = 360000 + flags = 67108864 + size = 18 + sample #171: + track = 1 + timeUs = 394739 + flags = 1 + size = 4 + sample #172: + track = 1 + timeUs = 417959 + flags = 1 + size = 4 + sample #173: + track = 0 + timeUs = 440000 + flags = 67108864 + size = 18 + sample #174: + track = 1 + timeUs = 441179 + flags = 1 + size = 4 + sample #175: + track = 0 + timeUs = 640000 + size = 28 + sample #176: + track = 1 + timeUs = 464399 + flags = 1 + size = 4 + sample #177: + track = 1 + timeUs = 487619 + flags = 1 + size = 4 + sample #178: + track = 0 + timeUs = 560000 + size = 27 + sample #179: + track = 1 + timeUs = 510839 + flags = 1 + size = 4 + sample #180: + track = 1 + timeUs = 534058 + flags = 1 + size = 4 + sample #181: + track = 0 + timeUs = 520000 + flags = 67108864 + size = 18 + sample #182: + track = 1 + timeUs = 557278 + flags = 1 + size = 4 + sample #183: + track = 1 + timeUs = 580498 + flags = 1 + size = 4 + sample #184: + track = 0 + timeUs = 600000 + flags = 67108864 + size = 18 + sample #185: + track = 1 + timeUs = 603718 + flags = 1 + size = 4 + sample #186: + track = 0 + timeUs = 800000 + size = 28 + sample #187: + track = 1 + timeUs = 626938 + flags = 1 + size = 4 + sample #188: + track = 1 + timeUs = 650158 + flags = 1 + size = 4 + sample #189: + track = 0 + timeUs = 720000 + size = 27 + sample #190: + track = 1 + timeUs = 673378 + flags = 1 + size = 4 + sample #191: + track = 1 + timeUs = 696598 + flags = 1 + size = 4 + sample #192: + track = 0 + timeUs = 680000 + flags = 67108864 + size = 18 + sample #193: + track = 1 + timeUs = 719818 + flags = 1 + size = 4 + sample #194: + track = 1 + timeUs = 743038 + flags = 1 + size = 4 + sample #195: + track = 0 + timeUs = 760000 + flags = 67108864 + size = 18 + sample #196: + track = 1 + timeUs = 766258 + flags = 1 + size = 4 + sample #197: + track = 0 + timeUs = 960000 + size = 28 + sample #198: + track = 1 + timeUs = 789478 + flags = 1 + size = 4 + sample #199: + track = 1 + timeUs = 812698 + flags = 1 + size = 4 + sample #200: + track = 0 + timeUs = 880000 + size = 27 + sample #201: + track = 1 + timeUs = 835918 + flags = 1 + size = 4 + sample #202: + track = 1 + timeUs = 859138 + flags = 1 + size = 4 + sample #203: + track = 0 + timeUs = 840000 + flags = 67108864 + size = 18 + sample #204: + track = 1 + timeUs = 882358 + flags = 1 + size = 4 + sample #205: + track = 0 + timeUs = 920000 + flags = 67108864 + size = 18 + sample #206: + track = 1 + timeUs = 905578 + flags = 1 + size = 4 + sample #207: + track = 1 + timeUs = 928798 + flags = 1 + size = 4 + sample #208: + track = 0 + timeUs = 1120000 + size = 28 + sample #209: + track = 1 + timeUs = 952018 + flags = 1 + size = 4 + sample #210: + track = 1 + timeUs = 975238 + flags = 1 + size = 4 + sample #211: + track = 0 + timeUs = 1040000 + size = 27 + sample #212: + track = 1 + timeUs = 998458 + flags = 1 + size = 4 + sample #213: + track = 1 + timeUs = 1021678 + flags = 1 + size = 4 + sample #214: + track = 0 + timeUs = 1000000 + flags = 67108864 + size = 18 + sample #215: + track = 1 + timeUs = 1044897 + flags = 1 + size = 4 + sample #216: + track = 0 + timeUs = 1080000 + flags = 67108864 + size = 18 + sample #217: + track = 1 + timeUs = 1068117 + flags = 1 + size = 4 + sample #218: + track = 1 + timeUs = 1091337 + flags = 1 + size = 4 + sample #219: + track = 0 + timeUs = 1280000 + size = 28 + sample #220: + track = 1 + timeUs = 1114557 + flags = 1 + size = 4 + sample #221: + track = 1 + timeUs = 1137777 + flags = 1 + size = 4 + sample #222: + track = 0 + timeUs = 1200000 + size = 27 + sample #223: + track = 1 + timeUs = 1160997 + flags = 1 + size = 4 + sample #224: + track = 0 + timeUs = 1160000 + flags = 67108864 + size = 18 + sample #225: + track = 1 + timeUs = 1184217 + flags = 1 + size = 4 + sample #226: + track = 1 + timeUs = 1207437 + flags = 1 + size = 4 + sample #227: + track = 0 + timeUs = 1240000 + flags = 67108864 + size = 18 + sample #228: + track = 1 + timeUs = 1230657 + flags = 1 + size = 4 + sample #229: + track = 1 + timeUs = 1253877 + flags = 1 + size = 4 + sample #230: + track = 0 + timeUs = 1440000 + size = 28 + sample #231: + track = 1 + timeUs = 1277097 + flags = 1 + size = 4 + sample #232: + track = 1 + timeUs = 1300317 + flags = 1 + size = 4 + sample #233: + track = 0 + timeUs = 1360000 + size = 27 + sample #234: + track = 1 + timeUs = 1323537 + flags = 1 + size = 4 + sample #235: + track = 0 + timeUs = 1320000 + flags = 67108864 + size = 18 + sample #236: + track = 1 + timeUs = 1346757 + flags = 1 + size = 4 + sample #237: + track = 1 + timeUs = 1369977 + flags = 1 + size = 4 + sample #238: + track = 0 + timeUs = 1400000 + flags = 67108864 + size = 18 + sample #239: + track = 1 + timeUs = 1393197 + flags = 1 + size = 4 + sample #240: + track = 1 + timeUs = 1416417 + flags = 1 + size = 4 + sample #241: + track = 0 + timeUs = 1600000 + size = 28 + sample #242: + track = 1 + timeUs = 1439637 + flags = 1 + size = 4 + sample #243: + track = 1 + timeUs = 1462857 + flags = 1 + size = 4 + sample #244: + track = 0 + timeUs = 1520000 + size = 27 + sample #245: + track = 1 + timeUs = 1486077 + flags = 1 + size = 4 + sample #246: + track = 0 + timeUs = 1480000 + flags = 67108864 + size = 18 + sample #247: + track = 1 + timeUs = 1509297 + flags = 1 + size = 4 + sample #248: + track = 1 + timeUs = 1532517 + flags = 1 + size = 4 + sample #249: + track = 0 + timeUs = 1560000 + flags = 67108864 + size = 18 + sample #250: + track = 1 + timeUs = 1555736 + flags = 1 + size = 4 + sample #251: + track = 1 + timeUs = 1578956 + flags = 1 + size = 4 + sample #252: + track = 0 + timeUs = 1760000 + size = 28 + sample #253: + track = 1 + timeUs = 1602176 + flags = 1 + size = 4 + sample #254: + track = 0 + timeUs = 1680000 + size = 27 + sample #255: + track = 1 + timeUs = 1625396 + flags = 1 + size = 4 + sample #256: + track = 1 + timeUs = 1648616 + flags = 1 + size = 4 + sample #257: + track = 0 + timeUs = 1640000 + flags = 67108864 + size = 18 + sample #258: + track = 1 + timeUs = 1671836 + flags = 1 + size = 4 + sample #259: + track = 1 + timeUs = 1695056 + flags = 1 + size = 4 + sample #260: + track = 0 + timeUs = 1720000 + flags = 67108864 + size = 18 + sample #261: + track = 1 + timeUs = 1718276 + flags = 1 + size = 4 + sample #262: + track = 1 + timeUs = 1741496 + flags = 1 + size = 4 + sample #263: + track = 0 + timeUs = 1920000 + size = 28 + sample #264: + track = 1 + timeUs = 1764716 + flags = 1 + size = 4 + sample #265: + track = 0 + timeUs = 1840000 + size = 27 + sample #266: + track = 1 + timeUs = 1787936 + flags = 1 + size = 4 + sample #267: + track = 1 + timeUs = 1811156 + flags = 1 + size = 4 + sample #268: + track = 0 + timeUs = 1800000 + flags = 67108864 + size = 18 + sample #269: + track = 1 + timeUs = 1834376 + flags = 1 + size = 4 + sample #270: + track = 1 + timeUs = 1857596 + flags = 1 + size = 4 + sample #271: + track = 0 + timeUs = 1880000 + flags = 67108864 + size = 18 + sample #272: + track = 1 + timeUs = 1880816 + flags = 1 + size = 4 + sample #273: + track = 0 + timeUs = 2080000 + size = 28 + sample #274: + track = 1 + timeUs = 1904036 + flags = 1 + size = 4 + sample #275: + track = 1 + timeUs = 1927256 + flags = 1 + size = 4 + sample #276: + track = 0 + timeUs = 2000000 + size = 27 + sample #277: + track = 1 + timeUs = 1950476 + flags = 1 + size = 4 + sample #278: + track = 1 + timeUs = 1973696 + flags = 1 + size = 4 + sample #279: + track = 0 + timeUs = 1960000 + flags = 67108864 + size = 18 + sample #280: + track = 1 + timeUs = 1996916 + flags = 1 + size = 4 + sample #281: + track = 1 + timeUs = 2020136 + flags = 1 + size = 4 + sample #282: + track = 0 + timeUs = 2040000 + flags = 67108864 + size = 18 + sample #283: + track = 1 + timeUs = 2043356 + flags = 1 + size = 4 + sample #284: + track = 0 + timeUs = 2240000 + size = 28 + sample #285: + track = 1 + timeUs = 2066575 + flags = 1 + size = 4 + sample #286: + track = 1 + timeUs = 2089795 + flags = 1 + size = 4 + sample #287: + track = 0 + timeUs = 2160000 + size = 27 + sample #288: + track = 1 + timeUs = 2113015 + flags = 1 + size = 4 + sample #289: + track = 1 + timeUs = 2136235 + flags = 1 + size = 4 + sample #290: + track = 0 + timeUs = 2120000 + flags = 67108864 + size = 18 + sample #291: + track = 1 + timeUs = 2159455 + flags = 1 + size = 4 + sample #292: + track = 1 + timeUs = 2182675 + flags = 1 + size = 4 + sample #293: + track = 0 + timeUs = 2200000 + flags = 67108864 + size = 18 + sample #294: + track = 1 + timeUs = 2205895 + flags = 1 + size = 4 + sample #295: + track = 0 + timeUs = 2400000 + size = 28 + sample #296: + track = 1 + timeUs = 2229115 + flags = 1 + size = 4 + sample #297: + track = 1 + timeUs = 2252335 + flags = 1 + size = 4 + sample #298: + track = 0 + timeUs = 2320000 + size = 27 + sample #299: + track = 1 + timeUs = 2275555 + flags = 1 + size = 4 + sample #300: + track = 1 + timeUs = 2298775 + flags = 1 + size = 4 + sample #301: + track = 0 + timeUs = 2280000 + flags = 67108864 + size = 18 + sample #302: + track = 1 + timeUs = 2321995 + flags = 1 + size = 4 + sample #303: + track = 0 + timeUs = 2360000 + flags = 67108864 + size = 18 + sample #304: + track = 1 + timeUs = 2345215 + flags = 1 + size = 4 + sample #305: + track = 1 + timeUs = 2368435 + flags = 1 + size = 4 + sample #306: + track = 0 + timeUs = 2560000 + size = 28 + sample #307: + track = 1 + timeUs = 2391655 + flags = 1 + size = 4 + sample #308: + track = 1 + timeUs = 2414875 + flags = 1 + size = 4 + sample #309: + track = 0 + timeUs = 2480000 + size = 27 + sample #310: + track = 1 + timeUs = 2438095 + flags = 1 + size = 4 + sample #311: + track = 1 + timeUs = 2461315 + flags = 1 + size = 4 + sample #312: + track = 0 + timeUs = 2440000 + flags = 67108864 + size = 18 + sample #313: + track = 1 + timeUs = 2484535 + flags = 1 + size = 4 + sample #314: + track = 0 + timeUs = 2520000 + flags = 67108864 + size = 18 + sample #315: + track = 1 + timeUs = 2507755 + flags = 1 + size = 4 + sample #316: + track = 1 + timeUs = 2530975 + flags = 1 + size = 4 + sample #317: + track = 0 + timeUs = 2720000 + size = 28 + sample #318: + track = 1 + timeUs = 2554195 + flags = 1 + size = 4 + sample #319: + track = 1 + timeUs = 2577414 + flags = 1 + size = 4 + sample #320: + track = 0 + timeUs = 2640000 + size = 27 + sample #321: + track = 1 + timeUs = 2600634 + flags = 1 + size = 4 + sample #322: + track = 0 + timeUs = 2600000 + flags = 67108864 + size = 18 + sample #323: + track = 1 + timeUs = 2623854 + flags = 1 + size = 4 + sample #324: + track = 1 + timeUs = 2647074 + flags = 1 + size = 4 + sample #325: + track = 0 + timeUs = 2680000 + flags = 67108864 + size = 18 + sample #326: + track = 1 + timeUs = 2670294 + flags = 1 + size = 4 + sample #327: + track = 1 + timeUs = 2693514 + flags = 1 + size = 4 + sample #328: + track = 0 + timeUs = 2880000 + size = 28 + sample #329: + track = 1 + timeUs = 2716734 + flags = 1 + size = 4 + sample #330: + track = 1 + timeUs = 2739954 + flags = 1 + size = 4 + sample #331: + track = 0 + timeUs = 2800000 + size = 27 + sample #332: + track = 1 + timeUs = 2763174 + flags = 1 + size = 4 + sample #333: + track = 0 + timeUs = 2760000 + flags = 67108864 + size = 18 + sample #334: + track = 1 + timeUs = 2786394 + flags = 1 + size = 4 + sample #335: + track = 1 + timeUs = 2809614 + flags = 1 + size = 4 + sample #336: + track = 0 + timeUs = 2840000 + flags = 67108864 + size = 18 + sample #337: + track = 1 + timeUs = 2832834 + flags = 1 + size = 4 + sample #338: + track = 1 + timeUs = 2856054 + flags = 1 + size = 4 + sample #339: + track = 0 + timeUs = 3040000 + size = 28 + sample #340: + track = 1 + timeUs = 2879274 + flags = 1 + size = 4 + sample #341: + track = 1 + timeUs = 2902494 + flags = 1 + size = 4 + sample #342: + track = 0 + timeUs = 2960000 + size = 27 + sample #343: + track = 1 + timeUs = 2925714 + flags = 1 + size = 4 + sample #344: + track = 0 + timeUs = 2920000 + flags = 67108864 + size = 18 + sample #345: + track = 1 + timeUs = 2948934 + flags = 1 + size = 4 + sample #346: + track = 1 + timeUs = 2972154 + flags = 1 + size = 4 + sample #347: + track = 0 + timeUs = 3000000 + flags = 67108864 + size = 18 + sample #348: + track = 1 + timeUs = 2995374 + flags = 1 + size = 4 + sample #349: + track = 1 + timeUs = 3018594 + flags = 1 + size = 4 + sample #350: + track = 0 + timeUs = 3200000 + size = 28 + sample #351: + track = 1 + timeUs = 3041814 + flags = 1 + size = 4 + sample #352: + track = 0 + timeUs = 3120000 + size = 27 +MediaCodecAdapter (media3.audio.aac): + inputBuffers: + count = 126 + input buffer #0: + timeUs = 1000000116099 + contents = length 4, hash EE9DF + input buffer #1: + timeUs = 1000000139319 + contents = length 4, hash EE9DF + input buffer #2: + timeUs = 1000000162539 + contents = length 4, hash EE9DF + input buffer #3: + timeUs = 1000000185759 + contents = length 4, hash EE9DF + input buffer #4: + timeUs = 1000000208979 + contents = length 4, hash EE9DF + input buffer #5: + timeUs = 1000000232199 + contents = length 4, hash EE9DF + input buffer #6: + timeUs = 1000000255419 + contents = length 4, hash EE9DF + input buffer #7: + timeUs = 1000000278639 + contents = length 4, hash EE9DF + input buffer #8: + timeUs = 1000000301859 + contents = length 4, hash EE9DF + input buffer #9: + timeUs = 1000000325079 + contents = length 4, hash EE9DF + input buffer #10: + timeUs = 1000000348299 + contents = length 4, hash EE9DF + input buffer #11: + timeUs = 1000000371519 + contents = length 4, hash EE9DF + input buffer #12: + timeUs = 1000000394739 + contents = length 4, hash EE9DF + input buffer #13: + timeUs = 1000000417959 + contents = length 4, hash EE9DF + input buffer #14: + timeUs = 1000000441179 + contents = length 4, hash EE9DF + input buffer #15: + timeUs = 1000000464399 + contents = length 4, hash EE9DF + input buffer #16: + timeUs = 1000000487619 + contents = length 4, hash EE9DF + input buffer #17: + timeUs = 1000000510839 + contents = length 4, hash EE9DF + input buffer #18: + timeUs = 1000000534058 + contents = length 4, hash EE9DF + input buffer #19: + timeUs = 1000000557278 + contents = length 4, hash EE9DF + input buffer #20: + timeUs = 1000000580498 + contents = length 4, hash EE9DF + input buffer #21: + timeUs = 1000000603718 + contents = length 4, hash EE9DF + input buffer #22: + timeUs = 1000000626938 + contents = length 4, hash EE9DF + input buffer #23: + timeUs = 1000000650158 + contents = length 4, hash EE9DF + input buffer #24: + timeUs = 1000000673378 + contents = length 4, hash EE9DF + input buffer #25: + timeUs = 1000000696598 + contents = length 4, hash EE9DF + input buffer #26: + timeUs = 1000000719818 + contents = length 4, hash EE9DF + input buffer #27: + timeUs = 1000000743038 + contents = length 4, hash EE9DF + input buffer #28: + timeUs = 1000000766258 + contents = length 4, hash EE9DF + input buffer #29: + timeUs = 1000000789478 + contents = length 4, hash EE9DF + input buffer #30: + timeUs = 1000000812698 + contents = length 4, hash EE9DF + input buffer #31: + timeUs = 1000000835918 + contents = length 4, hash EE9DF + input buffer #32: + timeUs = 1000000859138 + contents = length 4, hash EE9DF + input buffer #33: + timeUs = 1000000882358 + contents = length 4, hash EE9DF + input buffer #34: + timeUs = 1000000905578 + contents = length 4, hash EE9DF + input buffer #35: + timeUs = 1000000928798 + contents = length 4, hash EE9DF + input buffer #36: + timeUs = 1000000952018 + contents = length 4, hash EE9DF + input buffer #37: + timeUs = 1000000975238 + contents = length 4, hash EE9DF + input buffer #38: + timeUs = 1000000998458 + contents = length 4, hash EE9DF + input buffer #39: + timeUs = 1000001021678 + contents = length 4, hash EE9DF + input buffer #40: + timeUs = 1000001044897 + contents = length 4, hash EE9DF + input buffer #41: + timeUs = 1000001068117 + contents = length 4, hash EE9DF + input buffer #42: + timeUs = 1000001091337 + contents = length 4, hash EE9DF + input buffer #43: + timeUs = 1000001114557 + contents = length 4, hash EE9DF + input buffer #44: + timeUs = 1000001137777 + contents = length 4, hash EE9DF + input buffer #45: + timeUs = 1000001160997 + contents = length 4, hash EE9DF + input buffer #46: + timeUs = 1000001184217 + contents = length 4, hash EE9DF + input buffer #47: + timeUs = 1000001207437 + contents = length 4, hash EE9DF + input buffer #48: + timeUs = 1000001230657 + contents = length 4, hash EE9DF + input buffer #49: + timeUs = 1000001253877 + contents = length 4, hash EE9DF + input buffer #50: + timeUs = 1000001277097 + contents = length 4, hash EE9DF + input buffer #51: + timeUs = 1000001300317 + contents = length 4, hash EE9DF + input buffer #52: + timeUs = 1000001323537 + contents = length 4, hash EE9DF + input buffer #53: + timeUs = 1000001346757 + contents = length 4, hash EE9DF + input buffer #54: + timeUs = 1000001369977 + contents = length 4, hash EE9DF + input buffer #55: + timeUs = 1000001393197 + contents = length 4, hash EE9DF + input buffer #56: + timeUs = 1000001416417 + contents = length 4, hash EE9DF + input buffer #57: + timeUs = 1000001439637 + contents = length 4, hash EE9DF + input buffer #58: + timeUs = 1000001462857 + contents = length 4, hash EE9DF + input buffer #59: + timeUs = 1000001486077 + contents = length 4, hash EE9DF + input buffer #60: + timeUs = 1000001509297 + contents = length 4, hash EE9DF + input buffer #61: + timeUs = 1000001532517 + contents = length 4, hash EE9DF + input buffer #62: + timeUs = 1000001555736 + contents = length 4, hash EE9DF + input buffer #63: + timeUs = 1000001578956 + contents = length 4, hash EE9DF + input buffer #64: + timeUs = 1000001602176 + contents = length 4, hash EE9DF + input buffer #65: + timeUs = 1000001625396 + contents = length 4, hash EE9DF + input buffer #66: + timeUs = 1000001648616 + contents = length 4, hash EE9DF + input buffer #67: + timeUs = 1000001671836 + contents = length 4, hash EE9DF + input buffer #68: + timeUs = 1000001695056 + contents = length 4, hash EE9DF + input buffer #69: + timeUs = 1000001718276 + contents = length 4, hash EE9DF + input buffer #70: + timeUs = 1000001741496 + contents = length 4, hash EE9DF + input buffer #71: + timeUs = 1000001764716 + contents = length 4, hash EE9DF + input buffer #72: + timeUs = 1000001787936 + contents = length 4, hash EE9DF + input buffer #73: + timeUs = 1000001811156 + contents = length 4, hash EE9DF + input buffer #74: + timeUs = 1000001834376 + contents = length 4, hash EE9DF + input buffer #75: + timeUs = 1000001857596 + contents = length 4, hash EE9DF + input buffer #76: + timeUs = 1000001880816 + contents = length 4, hash EE9DF + input buffer #77: + timeUs = 1000001904036 + contents = length 4, hash EE9DF + input buffer #78: + timeUs = 1000001927256 + contents = length 4, hash EE9DF + input buffer #79: + timeUs = 1000001950476 + contents = length 4, hash EE9DF + input buffer #80: + timeUs = 1000001973696 + contents = length 4, hash EE9DF + input buffer #81: + timeUs = 1000001996916 + contents = length 4, hash EE9DF + input buffer #82: + timeUs = 1000002020136 + contents = length 4, hash EE9DF + input buffer #83: + timeUs = 1000002043356 + contents = length 4, hash EE9DF + input buffer #84: + timeUs = 1000002066575 + contents = length 4, hash EE9DF + input buffer #85: + timeUs = 1000002089795 + contents = length 4, hash EE9DF + input buffer #86: + timeUs = 1000002113015 + contents = length 4, hash EE9DF + input buffer #87: + timeUs = 1000002136235 + contents = length 4, hash EE9DF + input buffer #88: + timeUs = 1000002159455 + contents = length 4, hash EE9DF + input buffer #89: + timeUs = 1000002182675 + contents = length 4, hash EE9DF + input buffer #90: + timeUs = 1000002205895 + contents = length 4, hash EE9DF + input buffer #91: + timeUs = 1000002229115 + contents = length 4, hash EE9DF + input buffer #92: + timeUs = 1000002252335 + contents = length 4, hash EE9DF + input buffer #93: + timeUs = 1000002275555 + contents = length 4, hash EE9DF + input buffer #94: + timeUs = 1000002298775 + contents = length 4, hash EE9DF + input buffer #95: + timeUs = 1000002321995 + contents = length 4, hash EE9DF + input buffer #96: + timeUs = 1000002345215 + contents = length 4, hash EE9DF + input buffer #97: + timeUs = 1000002368435 + contents = length 4, hash EE9DF + input buffer #98: + timeUs = 1000002391655 + contents = length 4, hash EE9DF + input buffer #99: + timeUs = 1000002414875 + contents = length 4, hash EE9DF + input buffer #100: + timeUs = 1000002438095 + contents = length 4, hash EE9DF + input buffer #101: + timeUs = 1000002461315 + contents = length 4, hash EE9DF + input buffer #102: + timeUs = 1000002484535 + contents = length 4, hash EE9DF + input buffer #103: + timeUs = 1000002507755 + contents = length 4, hash EE9DF + input buffer #104: + timeUs = 1000002530975 + contents = length 4, hash EE9DF + input buffer #105: + timeUs = 1000002554195 + contents = length 4, hash EE9DF + input buffer #106: + timeUs = 1000002577414 + contents = length 4, hash EE9DF + input buffer #107: + timeUs = 1000002600634 + contents = length 4, hash EE9DF + input buffer #108: + timeUs = 1000002623854 + contents = length 4, hash EE9DF + input buffer #109: + timeUs = 1000002647074 + contents = length 4, hash EE9DF + input buffer #110: + timeUs = 1000002670294 + contents = length 4, hash EE9DF + input buffer #111: + timeUs = 1000002693514 + contents = length 4, hash EE9DF + input buffer #112: + timeUs = 1000002716734 + contents = length 4, hash EE9DF + input buffer #113: + timeUs = 1000002739954 + contents = length 4, hash EE9DF + input buffer #114: + timeUs = 1000002763174 + contents = length 4, hash EE9DF + input buffer #115: + timeUs = 1000002786394 + contents = length 4, hash EE9DF + input buffer #116: + timeUs = 1000002809614 + contents = length 4, hash EE9DF + input buffer #117: + timeUs = 1000002832834 + contents = length 4, hash EE9DF + input buffer #118: + timeUs = 1000002856054 + contents = length 4, hash EE9DF + input buffer #119: + timeUs = 1000002879274 + contents = length 4, hash EE9DF + input buffer #120: + timeUs = 1000002902494 + contents = length 4, hash EE9DF + input buffer #121: + timeUs = 1000002925714 + contents = length 4, hash EE9DF + input buffer #122: + timeUs = 1000002948934 + contents = length 4, hash EE9DF + input buffer #123: + timeUs = 1000002972154 + contents = length 4, hash EE9DF + input buffer #124: + timeUs = 1000002995374 + contents = length 4, hash EE9DF + input buffer #125: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 125 + output buffer #0: + timeUs = 1000000116099 + size = 0 + rendered = false + output buffer #1: + timeUs = 1000000139319 + size = 0 + rendered = false + output buffer #2: + timeUs = 1000000162539 + size = 0 + rendered = false + output buffer #3: + timeUs = 1000000185759 + size = 0 + rendered = false + output buffer #4: + timeUs = 1000000208979 + size = 0 + rendered = false + output buffer #5: + timeUs = 1000000232199 + size = 0 + rendered = false + output buffer #6: + timeUs = 1000000255419 + size = 0 + rendered = false + output buffer #7: + timeUs = 1000000278639 + size = 0 + rendered = false + output buffer #8: + timeUs = 1000000301859 + size = 0 + rendered = false + output buffer #9: + timeUs = 1000000325079 + size = 0 + rendered = false + output buffer #10: + timeUs = 1000000348299 + size = 0 + rendered = false + output buffer #11: + timeUs = 1000000371519 + size = 0 + rendered = false + output buffer #12: + timeUs = 1000000394739 + size = 0 + rendered = false + output buffer #13: + timeUs = 1000000417959 + size = 0 + rendered = false + output buffer #14: + timeUs = 1000000441179 + size = 0 + rendered = false + output buffer #15: + timeUs = 1000000464399 + size = 0 + rendered = false + output buffer #16: + timeUs = 1000000487619 + size = 0 + rendered = false + output buffer #17: + timeUs = 1000000510839 + size = 0 + rendered = false + output buffer #18: + timeUs = 1000000534058 + size = 0 + rendered = false + output buffer #19: + timeUs = 1000000557278 + size = 0 + rendered = false + output buffer #20: + timeUs = 1000000580498 + size = 0 + rendered = false + output buffer #21: + timeUs = 1000000603718 + size = 0 + rendered = false + output buffer #22: + timeUs = 1000000626938 + size = 0 + rendered = false + output buffer #23: + timeUs = 1000000650158 + size = 0 + rendered = false + output buffer #24: + timeUs = 1000000673378 + size = 0 + rendered = false + output buffer #25: + timeUs = 1000000696598 + size = 0 + rendered = false + output buffer #26: + timeUs = 1000000719818 + size = 0 + rendered = false + output buffer #27: + timeUs = 1000000743038 + size = 0 + rendered = false + output buffer #28: + timeUs = 1000000766258 + size = 0 + rendered = false + output buffer #29: + timeUs = 1000000789478 + size = 0 + rendered = false + output buffer #30: + timeUs = 1000000812698 + size = 0 + rendered = false + output buffer #31: + timeUs = 1000000835918 + size = 0 + rendered = false + output buffer #32: + timeUs = 1000000859138 + size = 0 + rendered = false + output buffer #33: + timeUs = 1000000882358 + size = 0 + rendered = false + output buffer #34: + timeUs = 1000000905578 + size = 0 + rendered = false + output buffer #35: + timeUs = 1000000928798 + size = 0 + rendered = false + output buffer #36: + timeUs = 1000000952018 + size = 0 + rendered = false + output buffer #37: + timeUs = 1000000975238 + size = 0 + rendered = false + output buffer #38: + timeUs = 1000000998458 + size = 0 + rendered = false + output buffer #39: + timeUs = 1000001021678 + size = 0 + rendered = false + output buffer #40: + timeUs = 1000001044897 + size = 0 + rendered = false + output buffer #41: + timeUs = 1000001068117 + size = 0 + rendered = false + output buffer #42: + timeUs = 1000001091337 + size = 0 + rendered = false + output buffer #43: + timeUs = 1000001114557 + size = 0 + rendered = false + output buffer #44: + timeUs = 1000001137777 + size = 0 + rendered = false + output buffer #45: + timeUs = 1000001160997 + size = 0 + rendered = false + output buffer #46: + timeUs = 1000001184217 + size = 0 + rendered = false + output buffer #47: + timeUs = 1000001207437 + size = 0 + rendered = false + output buffer #48: + timeUs = 1000001230657 + size = 0 + rendered = false + output buffer #49: + timeUs = 1000001253877 + size = 0 + rendered = false + output buffer #50: + timeUs = 1000001277097 + size = 0 + rendered = false + output buffer #51: + timeUs = 1000001300317 + size = 0 + rendered = false + output buffer #52: + timeUs = 1000001323537 + size = 0 + rendered = false + output buffer #53: + timeUs = 1000001346757 + size = 0 + rendered = false + output buffer #54: + timeUs = 1000001369977 + size = 0 + rendered = false + output buffer #55: + timeUs = 1000001393197 + size = 0 + rendered = false + output buffer #56: + timeUs = 1000001416417 + size = 0 + rendered = false + output buffer #57: + timeUs = 1000001439637 + size = 0 + rendered = false + output buffer #58: + timeUs = 1000001462857 + size = 0 + rendered = false + output buffer #59: + timeUs = 1000001486077 + size = 0 + rendered = false + output buffer #60: + timeUs = 1000001509297 + size = 0 + rendered = false + output buffer #61: + timeUs = 1000001532517 + size = 0 + rendered = false + output buffer #62: + timeUs = 1000001555736 + size = 0 + rendered = false + output buffer #63: + timeUs = 1000001578956 + size = 0 + rendered = false + output buffer #64: + timeUs = 1000001602176 + size = 0 + rendered = false + output buffer #65: + timeUs = 1000001625396 + size = 0 + rendered = false + output buffer #66: + timeUs = 1000001648616 + size = 0 + rendered = false + output buffer #67: + timeUs = 1000001671836 + size = 0 + rendered = false + output buffer #68: + timeUs = 1000001695056 + size = 0 + rendered = false + output buffer #69: + timeUs = 1000001718276 + size = 0 + rendered = false + output buffer #70: + timeUs = 1000001741496 + size = 0 + rendered = false + output buffer #71: + timeUs = 1000001764716 + size = 0 + rendered = false + output buffer #72: + timeUs = 1000001787936 + size = 0 + rendered = false + output buffer #73: + timeUs = 1000001811156 + size = 0 + rendered = false + output buffer #74: + timeUs = 1000001834376 + size = 0 + rendered = false + output buffer #75: + timeUs = 1000001857596 + size = 0 + rendered = false + output buffer #76: + timeUs = 1000001880816 + size = 0 + rendered = false + output buffer #77: + timeUs = 1000001904036 + size = 0 + rendered = false + output buffer #78: + timeUs = 1000001927256 + size = 0 + rendered = false + output buffer #79: + timeUs = 1000001950476 + size = 0 + rendered = false + output buffer #80: + timeUs = 1000001973696 + size = 0 + rendered = false + output buffer #81: + timeUs = 1000001996916 + size = 0 + rendered = false + output buffer #82: + timeUs = 1000002020136 + size = 0 + rendered = false + output buffer #83: + timeUs = 1000002043356 + size = 0 + rendered = false + output buffer #84: + timeUs = 1000002066575 + size = 0 + rendered = false + output buffer #85: + timeUs = 1000002089795 + size = 0 + rendered = false + output buffer #86: + timeUs = 1000002113015 + size = 0 + rendered = false + output buffer #87: + timeUs = 1000002136235 + size = 0 + rendered = false + output buffer #88: + timeUs = 1000002159455 + size = 0 + rendered = false + output buffer #89: + timeUs = 1000002182675 + size = 0 + rendered = false + output buffer #90: + timeUs = 1000002205895 + size = 0 + rendered = false + output buffer #91: + timeUs = 1000002229115 + size = 0 + rendered = false + output buffer #92: + timeUs = 1000002252335 + size = 0 + rendered = false + output buffer #93: + timeUs = 1000002275555 + size = 0 + rendered = false + output buffer #94: + timeUs = 1000002298775 + size = 0 + rendered = false + output buffer #95: + timeUs = 1000002321995 + size = 0 + rendered = false + output buffer #96: + timeUs = 1000002345215 + size = 0 + rendered = false + output buffer #97: + timeUs = 1000002368435 + size = 0 + rendered = false + output buffer #98: + timeUs = 1000002391655 + size = 0 + rendered = false + output buffer #99: + timeUs = 1000002414875 + size = 0 + rendered = false + output buffer #100: + timeUs = 1000002438095 + size = 0 + rendered = false + output buffer #101: + timeUs = 1000002461315 + size = 0 + rendered = false + output buffer #102: + timeUs = 1000002484535 + size = 0 + rendered = false + output buffer #103: + timeUs = 1000002507755 + size = 0 + rendered = false + output buffer #104: + timeUs = 1000002530975 + size = 0 + rendered = false + output buffer #105: + timeUs = 1000002554195 + size = 0 + rendered = false + output buffer #106: + timeUs = 1000002577414 + size = 0 + rendered = false + output buffer #107: + timeUs = 1000002600634 + size = 0 + rendered = false + output buffer #108: + timeUs = 1000002623854 + size = 0 + rendered = false + output buffer #109: + timeUs = 1000002647074 + size = 0 + rendered = false + output buffer #110: + timeUs = 1000002670294 + size = 0 + rendered = false + output buffer #111: + timeUs = 1000002693514 + size = 0 + rendered = false + output buffer #112: + timeUs = 1000002716734 + size = 0 + rendered = false + output buffer #113: + timeUs = 1000002739954 + size = 0 + rendered = false + output buffer #114: + timeUs = 1000002763174 + size = 0 + rendered = false + output buffer #115: + timeUs = 1000002786394 + size = 0 + rendered = false + output buffer #116: + timeUs = 1000002809614 + size = 0 + rendered = false + output buffer #117: + timeUs = 1000002832834 + size = 0 + rendered = false + output buffer #118: + timeUs = 1000002856054 + size = 0 + rendered = false + output buffer #119: + timeUs = 1000002879274 + size = 0 + rendered = false + output buffer #120: + timeUs = 1000002902494 + size = 0 + rendered = false + output buffer #121: + timeUs = 1000002925714 + size = 0 + rendered = false + output buffer #122: + timeUs = 1000002948934 + size = 0 + rendered = false + output buffer #123: + timeUs = 1000002972154 + size = 0 + rendered = false + output buffer #124: + timeUs = 1000002995374 + size = 0 + rendered = false +MediaCodecAdapter (media3.video.avc): + inputBuffers: + count = 76 + input buffer #0: + timeUs = 1000000000000 + contents = length 5252, hash 13893A4C + input buffer #1: + timeUs = 1000000160000 + contents = length 44, hash A05B3BEA + input buffer #2: + timeUs = 1000000080000 + contents = length 25, hash 441EC4FE + input buffer #3: + timeUs = 1000000120000 + contents = length 18, hash B64DA059 + input buffer #4: + timeUs = 1000000320000 + contents = length 32, hash 5BFD9538 + input buffer #5: + timeUs = 1000000240000 + contents = length 27, hash 705869A5 + input buffer #6: + timeUs = 1000000200000 + contents = length 18, hash D163DF61 + input buffer #7: + timeUs = 1000000280000 + contents = length 18, hash FD82E95 + input buffer #8: + timeUs = 1000000480000 + contents = length 28, hash 44A16E72 + input buffer #9: + timeUs = 1000000400000 + contents = length 27, hash FFA326ED + input buffer #10: + timeUs = 1000000360000 + contents = length 18, hash DC93CC9D + input buffer #11: + timeUs = 1000000440000 + contents = length 18, hash 1B081BD1 + input buffer #12: + timeUs = 1000000640000 + contents = length 28, hash 2700AF + input buffer #13: + timeUs = 1000000560000 + contents = length 27, hash DECA17D0 + input buffer #14: + timeUs = 1000000520000 + contents = length 18, hash D646C05A + input buffer #15: + timeUs = 1000000600000 + contents = length 18, hash 14BB0F8E + input buffer #16: + timeUs = 1000000800000 + contents = length 28, hash 5DE2C2B + input buffer #17: + timeUs = 1000000720000 + contents = length 27, hash 376B4E14 + input buffer #18: + timeUs = 1000000680000 + contents = length 18, hash E176AD96 + input buffer #19: + timeUs = 1000000760000 + contents = length 18, hash 1FEAFCCA + input buffer #20: + timeUs = 1000000960000 + contents = length 28, hash C163BE68 + input buffer #21: + timeUs = 1000000880000 + contents = length 27, hash F7170357 + input buffer #22: + timeUs = 1000000840000 + contents = length 18, hash 3B013BD2 + input buffer #23: + timeUs = 1000000920000 + contents = length 18, hash 79758B06 + input buffer #24: + timeUs = 1000001120000 + contents = length 28, hash F72EB1A3 + input buffer #25: + timeUs = 1000001040000 + contents = length 27, hash 4FB8399C + input buffer #26: + timeUs = 1000001000000 + contents = length 18, hash 4631290E + input buffer #27: + timeUs = 1000001080000 + contents = length 18, hash 84A57842 + input buffer #28: + timeUs = 1000001280000 + contents = length 28, hash E1FCF000 + input buffer #29: + timeUs = 1000001200000 + contents = length 27, hash 21D3B540 + input buffer #30: + timeUs = 1000001160000 + contents = length 18, hash 62DE0FC9 + input buffer #31: + timeUs = 1000001240000 + contents = length 18, hash A1525EFD + input buffer #32: + timeUs = 1000001440000 + contents = length 28, hash 5350E8FA + input buffer #33: + timeUs = 1000001360000 + contents = length 27, hash E7563B65 + input buffer #34: + timeUs = 1000001320000 + contents = length 18, hash 77D95125 + input buffer #35: + timeUs = 1000001400000 + contents = length 18, hash B64DA059 + input buffer #36: + timeUs = 1000001600000 + contents = length 28, hash ED67B37 + input buffer #37: + timeUs = 1000001520000 + contents = length 27, hash A701F0A9 + input buffer #38: + timeUs = 1000001480000 + contents = length 18, hash D163DF61 + input buffer #39: + timeUs = 1000001560000 + contents = length 18, hash FD82E95 + input buffer #40: + timeUs = 1000001760000 + contents = length 28, hash 44A16E72 + input buffer #41: + timeUs = 1000001680000 + contents = length 27, hash FFA326ED + input buffer #42: + timeUs = 1000001640000 + contents = length 18, hash DC93CC9D + input buffer #43: + timeUs = 1000001720000 + contents = length 18, hash 1B081BD1 + input buffer #44: + timeUs = 1000001920000 + contents = length 28, hash 2700AF + input buffer #45: + timeUs = 1000001840000 + contents = length 27, hash DECA17D0 + input buffer #46: + timeUs = 1000001800000 + contents = length 18, hash D646C05A + input buffer #47: + timeUs = 1000001880000 + contents = length 18, hash 14BB0F8E + input buffer #48: + timeUs = 1000002080000 + contents = length 28, hash 5DE2C2B + input buffer #49: + timeUs = 1000002000000 + contents = length 27, hash 376B4E14 + input buffer #50: + timeUs = 1000001960000 + contents = length 18, hash E176AD96 + input buffer #51: + timeUs = 1000002040000 + contents = length 18, hash 1FEAFCCA + input buffer #52: + timeUs = 1000002240000 + contents = length 28, hash C163BE68 + input buffer #53: + timeUs = 1000002160000 + contents = length 27, hash F7170357 + input buffer #54: + timeUs = 1000002120000 + contents = length 18, hash 3B013BD2 + input buffer #55: + timeUs = 1000002200000 + contents = length 18, hash 79758B06 + input buffer #56: + timeUs = 1000002400000 + contents = length 28, hash F72EB1A3 + input buffer #57: + timeUs = 1000002320000 + contents = length 27, hash 4FB8399C + input buffer #58: + timeUs = 1000002280000 + contents = length 18, hash 4631290E + input buffer #59: + timeUs = 1000002360000 + contents = length 18, hash 84A57842 + input buffer #60: + timeUs = 1000002560000 + contents = length 28, hash E1FCF000 + input buffer #61: + timeUs = 1000002480000 + contents = length 27, hash 21D3B540 + input buffer #62: + timeUs = 1000002440000 + contents = length 18, hash 62DE0FC9 + input buffer #63: + timeUs = 1000002520000 + contents = length 18, hash A1525EFD + input buffer #64: + timeUs = 1000002720000 + contents = length 28, hash 5350E8FA + input buffer #65: + timeUs = 1000002640000 + contents = length 27, hash E7563B65 + input buffer #66: + timeUs = 1000002600000 + contents = length 18, hash 77D95125 + input buffer #67: + timeUs = 1000002680000 + contents = length 18, hash B64DA059 + input buffer #68: + timeUs = 1000002880000 + contents = length 28, hash ED67B37 + input buffer #69: + timeUs = 1000002800000 + contents = length 27, hash A701F0A9 + input buffer #70: + timeUs = 1000002760000 + contents = length 18, hash D163DF61 + input buffer #71: + timeUs = 1000002840000 + contents = length 18, hash FD82E95 + input buffer #72: + timeUs = 1000003040000 + contents = length 28, hash 44A16E72 + input buffer #73: + timeUs = 1000002960000 + contents = length 27, hash FFA326ED + input buffer #74: + timeUs = 1000002920000 + contents = length 18, hash DC93CC9D + input buffer #75: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 75 + output buffer #0: + timeUs = 1000000000000 + size = 5252 + rendered = false + output buffer #1: + timeUs = 1000000160000 + size = 44 + rendered = true + output buffer #2: + timeUs = 1000000080000 + size = 25 + rendered = false + output buffer #3: + timeUs = 1000000120000 + size = 18 + rendered = true + output buffer #4: + timeUs = 1000000320000 + size = 32 + rendered = true + output buffer #5: + timeUs = 1000000240000 + size = 27 + rendered = true + output buffer #6: + timeUs = 1000000200000 + size = 18 + rendered = true + output buffer #7: + timeUs = 1000000280000 + size = 18 + rendered = true + output buffer #8: + timeUs = 1000000480000 + size = 28 + rendered = true + output buffer #9: + timeUs = 1000000400000 + size = 27 + rendered = true + output buffer #10: + timeUs = 1000000360000 + size = 18 + rendered = true + output buffer #11: + timeUs = 1000000440000 + size = 18 + rendered = true + output buffer #12: + timeUs = 1000000640000 + size = 28 + rendered = true + output buffer #13: + timeUs = 1000000560000 + size = 27 + rendered = true + output buffer #14: + timeUs = 1000000520000 + size = 18 + rendered = true + output buffer #15: + timeUs = 1000000600000 + size = 18 + rendered = true + output buffer #16: + timeUs = 1000000800000 + size = 28 + rendered = true + output buffer #17: + timeUs = 1000000720000 + size = 27 + rendered = true + output buffer #18: + timeUs = 1000000680000 + size = 18 + rendered = true + output buffer #19: + timeUs = 1000000760000 + size = 18 + rendered = true + output buffer #20: + timeUs = 1000000960000 + size = 28 + rendered = true + output buffer #21: + timeUs = 1000000880000 + size = 27 + rendered = true + output buffer #22: + timeUs = 1000000840000 + size = 18 + rendered = true + output buffer #23: + timeUs = 1000000920000 + size = 18 + rendered = true + output buffer #24: + timeUs = 1000001120000 + size = 28 + rendered = true + output buffer #25: + timeUs = 1000001040000 + size = 27 + rendered = true + output buffer #26: + timeUs = 1000001000000 + size = 18 + rendered = true + output buffer #27: + timeUs = 1000001080000 + size = 18 + rendered = true + output buffer #28: + timeUs = 1000001280000 + size = 28 + rendered = true + output buffer #29: + timeUs = 1000001200000 + size = 27 + rendered = true + output buffer #30: + timeUs = 1000001160000 + size = 18 + rendered = true + output buffer #31: + timeUs = 1000001240000 + size = 18 + rendered = true + output buffer #32: + timeUs = 1000001440000 + size = 28 + rendered = true + output buffer #33: + timeUs = 1000001360000 + size = 27 + rendered = true + output buffer #34: + timeUs = 1000001320000 + size = 18 + rendered = true + output buffer #35: + timeUs = 1000001400000 + size = 18 + rendered = true + output buffer #36: + timeUs = 1000001600000 + size = 28 + rendered = true + output buffer #37: + timeUs = 1000001520000 + size = 27 + rendered = true + output buffer #38: + timeUs = 1000001480000 + size = 18 + rendered = true + output buffer #39: + timeUs = 1000001560000 + size = 18 + rendered = true + output buffer #40: + timeUs = 1000001760000 + size = 28 + rendered = true + output buffer #41: + timeUs = 1000001680000 + size = 27 + rendered = true + output buffer #42: + timeUs = 1000001640000 + size = 18 + rendered = true + output buffer #43: + timeUs = 1000001720000 + size = 18 + rendered = true + output buffer #44: + timeUs = 1000001920000 + size = 28 + rendered = true + output buffer #45: + timeUs = 1000001840000 + size = 27 + rendered = true + output buffer #46: + timeUs = 1000001800000 + size = 18 + rendered = true + output buffer #47: + timeUs = 1000001880000 + size = 18 + rendered = true + output buffer #48: + timeUs = 1000002080000 + size = 28 + rendered = true + output buffer #49: + timeUs = 1000002000000 + size = 27 + rendered = true + output buffer #50: + timeUs = 1000001960000 + size = 18 + rendered = true + output buffer #51: + timeUs = 1000002040000 + size = 18 + rendered = true + output buffer #52: + timeUs = 1000002240000 + size = 28 + rendered = true + output buffer #53: + timeUs = 1000002160000 + size = 27 + rendered = true + output buffer #54: + timeUs = 1000002120000 + size = 18 + rendered = true + output buffer #55: + timeUs = 1000002200000 + size = 18 + rendered = true + output buffer #56: + timeUs = 1000002400000 + size = 28 + rendered = true + output buffer #57: + timeUs = 1000002320000 + size = 27 + rendered = true + output buffer #58: + timeUs = 1000002280000 + size = 18 + rendered = true + output buffer #59: + timeUs = 1000002360000 + size = 18 + rendered = true + output buffer #60: + timeUs = 1000002560000 + size = 28 + rendered = true + output buffer #61: + timeUs = 1000002480000 + size = 27 + rendered = true + output buffer #62: + timeUs = 1000002440000 + size = 18 + rendered = true + output buffer #63: + timeUs = 1000002520000 + size = 18 + rendered = true + output buffer #64: + timeUs = 1000002720000 + size = 28 + rendered = true + output buffer #65: + timeUs = 1000002640000 + size = 27 + rendered = true + output buffer #66: + timeUs = 1000002600000 + size = 18 + rendered = true + output buffer #67: + timeUs = 1000002680000 + size = 18 + rendered = true + output buffer #68: + timeUs = 1000002880000 + size = 28 + rendered = true + output buffer #69: + timeUs = 1000002800000 + size = 27 + rendered = true + output buffer #70: + timeUs = 1000002760000 + size = 18 + rendered = true + output buffer #71: + timeUs = 1000002840000 + size = 18 + rendered = true + output buffer #72: + timeUs = 1000003040000 + size = 28 + rendered = false + output buffer #73: + timeUs = 1000002960000 + size = 27 + rendered = true + output buffer #74: + timeUs = 1000002920000 + size = 18 + rendered = true +AudioSink: + buffer count = 125 + config: + pcmEncoding = pcm-16 + channelCount = 1 + sampleRate = 44100 + buffer #0: + time = 1000000116099 + data = empty + buffer #1: + time = 1000000139319 + data = empty + buffer #2: + time = 1000000162539 + data = empty + buffer #3: + time = 1000000185759 + data = empty + buffer #4: + time = 1000000208979 + data = empty + buffer #5: + time = 1000000232199 + data = empty + buffer #6: + time = 1000000255419 + data = empty + buffer #7: + time = 1000000278639 + data = empty + buffer #8: + time = 1000000301859 + data = empty + buffer #9: + time = 1000000325079 + data = empty + buffer #10: + time = 1000000348299 + data = empty + buffer #11: + time = 1000000371519 + data = empty + buffer #12: + time = 1000000394739 + data = empty + buffer #13: + time = 1000000417959 + data = empty + buffer #14: + time = 1000000441179 + data = empty + buffer #15: + time = 1000000464399 + data = empty + buffer #16: + time = 1000000487619 + data = empty + buffer #17: + time = 1000000510839 + data = empty + buffer #18: + time = 1000000534058 + data = empty + buffer #19: + time = 1000000557278 + data = empty + buffer #20: + time = 1000000580498 + data = empty + buffer #21: + time = 1000000603718 + data = empty + buffer #22: + time = 1000000626938 + data = empty + buffer #23: + time = 1000000650158 + data = empty + buffer #24: + time = 1000000673378 + data = empty + buffer #25: + time = 1000000696598 + data = empty + buffer #26: + time = 1000000719818 + data = empty + buffer #27: + time = 1000000743038 + data = empty + buffer #28: + time = 1000000766258 + data = empty + buffer #29: + time = 1000000789478 + data = empty + buffer #30: + time = 1000000812698 + data = empty + buffer #31: + time = 1000000835918 + data = empty + buffer #32: + time = 1000000859138 + data = empty + buffer #33: + time = 1000000882358 + data = empty + buffer #34: + time = 1000000905578 + data = empty + buffer #35: + time = 1000000928798 + data = empty + buffer #36: + time = 1000000952018 + data = empty + buffer #37: + time = 1000000975238 + data = empty + buffer #38: + time = 1000000998458 + data = empty + buffer #39: + time = 1000001021678 + data = empty + buffer #40: + time = 1000001044897 + data = empty + buffer #41: + time = 1000001068117 + data = empty + buffer #42: + time = 1000001091337 + data = empty + buffer #43: + time = 1000001114557 + data = empty + buffer #44: + time = 1000001137777 + data = empty + buffer #45: + time = 1000001160997 + data = empty + buffer #46: + time = 1000001184217 + data = empty + buffer #47: + time = 1000001207437 + data = empty + buffer #48: + time = 1000001230657 + data = empty + buffer #49: + time = 1000001253877 + data = empty + buffer #50: + time = 1000001277097 + data = empty + buffer #51: + time = 1000001300317 + data = empty + buffer #52: + time = 1000001323537 + data = empty + buffer #53: + time = 1000001346757 + data = empty + buffer #54: + time = 1000001369977 + data = empty + buffer #55: + time = 1000001393197 + data = empty + buffer #56: + time = 1000001416417 + data = empty + buffer #57: + time = 1000001439637 + data = empty + buffer #58: + time = 1000001462857 + data = empty + buffer #59: + time = 1000001486077 + data = empty + buffer #60: + time = 1000001509297 + data = empty + buffer #61: + time = 1000001532517 + data = empty + buffer #62: + time = 1000001555736 + data = empty + buffer #63: + time = 1000001578956 + data = empty + buffer #64: + time = 1000001602176 + data = empty + buffer #65: + time = 1000001625396 + data = empty + buffer #66: + time = 1000001648616 + data = empty + buffer #67: + time = 1000001671836 + data = empty + buffer #68: + time = 1000001695056 + data = empty + buffer #69: + time = 1000001718276 + data = empty + buffer #70: + time = 1000001741496 + data = empty + buffer #71: + time = 1000001764716 + data = empty + buffer #72: + time = 1000001787936 + data = empty + buffer #73: + time = 1000001811156 + data = empty + buffer #74: + time = 1000001834376 + data = empty + buffer #75: + time = 1000001857596 + data = empty + buffer #76: + time = 1000001880816 + data = empty + buffer #77: + time = 1000001904036 + data = empty + buffer #78: + time = 1000001927256 + data = empty + buffer #79: + time = 1000001950476 + data = empty + buffer #80: + time = 1000001973696 + data = empty + buffer #81: + time = 1000001996916 + data = empty + buffer #82: + time = 1000002020136 + data = empty + buffer #83: + time = 1000002043356 + data = empty + buffer #84: + time = 1000002066575 + data = empty + buffer #85: + time = 1000002089795 + data = empty + buffer #86: + time = 1000002113015 + data = empty + buffer #87: + time = 1000002136235 + data = empty + buffer #88: + time = 1000002159455 + data = empty + buffer #89: + time = 1000002182675 + data = empty + buffer #90: + time = 1000002205895 + data = empty + buffer #91: + time = 1000002229115 + data = empty + buffer #92: + time = 1000002252335 + data = empty + buffer #93: + time = 1000002275555 + data = empty + buffer #94: + time = 1000002298775 + data = empty + buffer #95: + time = 1000002321995 + data = empty + buffer #96: + time = 1000002345215 + data = empty + buffer #97: + time = 1000002368435 + data = empty + buffer #98: + time = 1000002391655 + data = empty + buffer #99: + time = 1000002414875 + data = empty + buffer #100: + time = 1000002438095 + data = empty + buffer #101: + time = 1000002461315 + data = empty + buffer #102: + time = 1000002484535 + data = empty + buffer #103: + time = 1000002507755 + data = empty + buffer #104: + time = 1000002530975 + data = empty + buffer #105: + time = 1000002554195 + data = empty + buffer #106: + time = 1000002577414 + data = empty + buffer #107: + time = 1000002600634 + data = empty + buffer #108: + time = 1000002623854 + data = empty + buffer #109: + time = 1000002647074 + data = empty + buffer #110: + time = 1000002670294 + data = empty + buffer #111: + time = 1000002693514 + data = empty + buffer #112: + time = 1000002716734 + data = empty + buffer #113: + time = 1000002739954 + data = empty + buffer #114: + time = 1000002763174 + data = empty + buffer #115: + time = 1000002786394 + data = empty + buffer #116: + time = 1000002809614 + data = empty + buffer #117: + time = 1000002832834 + data = empty + buffer #118: + time = 1000002856054 + data = empty + buffer #119: + time = 1000002879274 + data = empty + buffer #120: + time = 1000002902494 + data = empty + buffer #121: + time = 1000002925714 + data = empty + buffer #122: + time = 1000002948934 + data = empty + buffer #123: + time = 1000002972154 + data = empty + buffer #124: + time = 1000002995374 + data = empty diff --git a/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_extend_before_load_finished_enabled.dump b/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_extend_before_load_finished_enabled.dump new file mode 100644 index 00000000000..4665cdbc8d8 --- /dev/null +++ b/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_extend_before_load_finished_enabled.dump @@ -0,0 +1,2795 @@ +extracted samples: + sample #0: + track = 0 + timeUs = 0 + flags = 1 + size = 5252 + sample #1: + track = 0 + timeUs = 160000 + size = 44 + sample #2: + track = 0 + timeUs = 80000 + size = 25 + sample #3: + track = 0 + timeUs = 40000 + flags = 67108864 + size = 18 + sample #4: + track = 1 + timeUs = 92879 + flags = 1 + size = 4 + sample #5: + track = 0 + timeUs = 120000 + flags = 67108864 + size = 18 + sample #6: + track = 1 + timeUs = 116099 + flags = 1 + size = 4 + sample #7: + track = 1 + timeUs = 139319 + flags = 1 + size = 4 + sample #8: + track = 0 + timeUs = 320000 + size = 32 + sample #9: + track = 1 + timeUs = 162539 + flags = 1 + size = 4 + sample #10: + track = 0 + timeUs = 240000 + size = 27 + sample #11: + track = 1 + timeUs = 185759 + flags = 1 + size = 4 + sample #12: + track = 1 + timeUs = 208979 + flags = 1 + size = 4 + sample #13: + track = 0 + timeUs = 200000 + flags = 67108864 + size = 18 + sample #14: + track = 1 + timeUs = 232199 + flags = 1 + size = 4 + sample #15: + track = 1 + timeUs = 255419 + flags = 1 + size = 4 + sample #16: + track = 0 + timeUs = 280000 + flags = 67108864 + size = 18 + sample #17: + track = 1 + timeUs = 278639 + flags = 1 + size = 4 + sample #18: + track = 1 + timeUs = 301859 + flags = 1 + size = 4 + sample #19: + track = 0 + timeUs = 480000 + size = 28 + sample #20: + track = 1 + timeUs = 325079 + flags = 1 + size = 4 + sample #21: + track = 0 + timeUs = 400000 + size = 27 + sample #22: + track = 1 + timeUs = 348299 + flags = 1 + size = 4 + sample #23: + track = 1 + timeUs = 371519 + flags = 1 + size = 4 + sample #24: + track = 0 + timeUs = 360000 + flags = 67108864 + size = 18 + sample #25: + track = 1 + timeUs = 394739 + flags = 1 + size = 4 + sample #26: + track = 1 + timeUs = 417959 + flags = 1 + size = 4 + sample #27: + track = 0 + timeUs = 440000 + flags = 67108864 + size = 18 + sample #28: + track = 1 + timeUs = 441179 + flags = 1 + size = 4 + sample #29: + track = 0 + timeUs = 640000 + size = 28 + sample #30: + track = 1 + timeUs = 464399 + flags = 1 + size = 4 + sample #31: + track = 1 + timeUs = 487619 + flags = 1 + size = 4 + sample #32: + track = 0 + timeUs = 560000 + size = 27 + sample #33: + track = 1 + timeUs = 510839 + flags = 1 + size = 4 + sample #34: + track = 1 + timeUs = 534058 + flags = 1 + size = 4 + sample #35: + track = 0 + timeUs = 520000 + flags = 67108864 + size = 18 + sample #36: + track = 1 + timeUs = 557278 + flags = 1 + size = 4 + sample #37: + track = 1 + timeUs = 580498 + flags = 1 + size = 4 + sample #38: + track = 0 + timeUs = 600000 + flags = 67108864 + size = 18 + sample #39: + track = 1 + timeUs = 603718 + flags = 1 + size = 4 + sample #40: + track = 0 + timeUs = 800000 + size = 28 + sample #41: + track = 1 + timeUs = 626938 + flags = 1 + size = 4 + sample #42: + track = 1 + timeUs = 650158 + flags = 1 + size = 4 + sample #43: + track = 0 + timeUs = 720000 + size = 27 + sample #44: + track = 1 + timeUs = 673378 + flags = 1 + size = 4 + sample #45: + track = 1 + timeUs = 696598 + flags = 1 + size = 4 + sample #46: + track = 0 + timeUs = 680000 + flags = 67108864 + size = 18 + sample #47: + track = 1 + timeUs = 719818 + flags = 1 + size = 4 + sample #48: + track = 1 + timeUs = 743038 + flags = 1 + size = 4 + sample #49: + track = 0 + timeUs = 760000 + flags = 67108864 + size = 18 + sample #50: + track = 1 + timeUs = 766258 + flags = 1 + size = 4 + sample #51: + track = 0 + timeUs = 960000 + size = 28 + sample #52: + track = 1 + timeUs = 789478 + flags = 1 + size = 4 + sample #53: + track = 1 + timeUs = 812698 + flags = 1 + size = 4 + sample #54: + track = 0 + timeUs = 880000 + size = 27 + sample #55: + track = 1 + timeUs = 835918 + flags = 1 + size = 4 + sample #56: + track = 1 + timeUs = 859138 + flags = 1 + size = 4 + sample #57: + track = 0 + timeUs = 840000 + flags = 67108864 + size = 18 + sample #58: + track = 1 + timeUs = 882358 + flags = 1 + size = 4 + sample #59: + track = 0 + timeUs = 920000 + flags = 67108864 + size = 18 + sample #60: + track = 1 + timeUs = 905578 + flags = 1 + size = 4 + sample #61: + track = 1 + timeUs = 928798 + flags = 1 + size = 4 + sample #62: + track = 0 + timeUs = 1120000 + size = 28 + sample #63: + track = 1 + timeUs = 952018 + flags = 1 + size = 4 + sample #64: + track = 1 + timeUs = 975238 + flags = 1 + size = 4 + sample #65: + track = 0 + timeUs = 1040000 + size = 27 + sample #66: + track = 1 + timeUs = 998458 + flags = 1 + size = 4 + sample #67: + track = 1 + timeUs = 1021678 + flags = 1 + size = 4 + sample #68: + track = 0 + timeUs = 1000000 + flags = 67108864 + size = 18 + sample #69: + track = 1 + timeUs = 1044897 + flags = 1 + size = 4 + sample #70: + track = 0 + timeUs = 1080000 + flags = 67108864 + size = 18 + sample #71: + track = 1 + timeUs = 1068117 + flags = 1 + size = 4 + sample #72: + track = 1 + timeUs = 1091337 + flags = 1 + size = 4 + sample #73: + track = 0 + timeUs = 1280000 + size = 28 + sample #74: + track = 1 + timeUs = 1114557 + flags = 1 + size = 4 + sample #75: + track = 1 + timeUs = 1137777 + flags = 1 + size = 4 + sample #76: + track = 0 + timeUs = 1200000 + size = 27 + sample #77: + track = 1 + timeUs = 1160997 + flags = 1 + size = 4 + sample #78: + track = 0 + timeUs = 1160000 + flags = 67108864 + size = 18 + sample #79: + track = 1 + timeUs = 1184217 + flags = 1 + size = 4 + sample #80: + track = 1 + timeUs = 1207437 + flags = 1 + size = 4 + sample #81: + track = 0 + timeUs = 1240000 + flags = 67108864 + size = 18 + sample #82: + track = 1 + timeUs = 1230657 + flags = 1 + size = 4 + sample #83: + track = 1 + timeUs = 1253877 + flags = 1 + size = 4 + sample #84: + track = 0 + timeUs = 1440000 + size = 28 + sample #85: + track = 1 + timeUs = 1277097 + flags = 1 + size = 4 + sample #86: + track = 1 + timeUs = 1300317 + flags = 1 + size = 4 + sample #87: + track = 0 + timeUs = 1360000 + size = 27 + sample #88: + track = 1 + timeUs = 1323537 + flags = 1 + size = 4 + sample #89: + track = 0 + timeUs = 1320000 + flags = 67108864 + size = 18 + sample #90: + track = 1 + timeUs = 1346757 + flags = 1 + size = 4 + sample #91: + track = 1 + timeUs = 1369977 + flags = 1 + size = 4 + sample #92: + track = 0 + timeUs = 1400000 + flags = 67108864 + size = 18 + sample #93: + track = 1 + timeUs = 1393197 + flags = 1 + size = 4 + sample #94: + track = 1 + timeUs = 1416417 + flags = 1 + size = 4 + sample #95: + track = 0 + timeUs = 1600000 + size = 28 + sample #96: + track = 1 + timeUs = 1439637 + flags = 1 + size = 4 + sample #97: + track = 1 + timeUs = 1462857 + flags = 1 + size = 4 + sample #98: + track = 0 + timeUs = 1520000 + size = 27 + sample #99: + track = 1 + timeUs = 1486077 + flags = 1 + size = 4 + sample #100: + track = 0 + timeUs = 1480000 + flags = 67108864 + size = 18 + sample #101: + track = 1 + timeUs = 1509297 + flags = 1 + size = 4 + sample #102: + track = 1 + timeUs = 1532517 + flags = 1 + size = 4 + sample #103: + track = 0 + timeUs = 1560000 + flags = 67108864 + size = 18 + sample #104: + track = 1 + timeUs = 1555736 + flags = 1 + size = 4 + sample #105: + track = 1 + timeUs = 1578956 + flags = 1 + size = 4 + sample #106: + track = 0 + timeUs = 1760000 + size = 28 + sample #107: + track = 1 + timeUs = 1602176 + flags = 1 + size = 4 + sample #108: + track = 0 + timeUs = 1680000 + size = 27 + sample #109: + track = 1 + timeUs = 1625396 + flags = 1 + size = 4 + sample #110: + track = 1 + timeUs = 1648616 + flags = 1 + size = 4 + sample #111: + track = 0 + timeUs = 1640000 + flags = 67108864 + size = 18 + sample #112: + track = 1 + timeUs = 1671836 + flags = 1 + size = 4 + sample #113: + track = 1 + timeUs = 1695056 + flags = 1 + size = 4 + sample #114: + track = 0 + timeUs = 1720000 + flags = 67108864 + size = 18 + sample #115: + track = 1 + timeUs = 1718276 + flags = 1 + size = 4 + sample #116: + track = 1 + timeUs = 1741496 + flags = 1 + size = 4 + sample #117: + track = 0 + timeUs = 1920000 + size = 28 + sample #118: + track = 1 + timeUs = 1764716 + flags = 1 + size = 4 + sample #119: + track = 0 + timeUs = 1840000 + size = 27 + sample #120: + track = 1 + timeUs = 1787936 + flags = 1 + size = 4 + sample #121: + track = 1 + timeUs = 1811156 + flags = 1 + size = 4 + sample #122: + track = 0 + timeUs = 1800000 + flags = 67108864 + size = 18 + sample #123: + track = 1 + timeUs = 1834376 + flags = 1 + size = 4 + sample #124: + track = 1 + timeUs = 1857596 + flags = 1 + size = 4 + sample #125: + track = 0 + timeUs = 1880000 + flags = 67108864 + size = 18 + sample #126: + track = 1 + timeUs = 1880816 + flags = 1 + size = 4 + sample #127: + track = 0 + timeUs = 2080000 + size = 28 + sample #128: + track = 1 + timeUs = 1904036 + flags = 1 + size = 4 + sample #129: + track = 1 + timeUs = 1927256 + flags = 1 + size = 4 + sample #130: + track = 0 + timeUs = 2000000 + size = 27 + sample #131: + track = 1 + timeUs = 1950476 + flags = 1 + size = 4 + sample #132: + track = 1 + timeUs = 1973696 + flags = 1 + size = 4 + sample #133: + track = 0 + timeUs = 1960000 + flags = 67108864 + size = 18 + sample #134: + track = 1 + timeUs = 1996916 + flags = 1 + size = 4 + sample #135: + track = 1 + timeUs = 2020136 + flags = 1 + size = 4 + sample #136: + track = 0 + timeUs = 2040000 + flags = 67108864 + size = 18 + sample #137: + track = 1 + timeUs = 2043356 + flags = 1 + size = 4 + sample #138: + track = 0 + timeUs = 2240000 + size = 28 + sample #139: + track = 1 + timeUs = 2066575 + flags = 1 + size = 4 + sample #140: + track = 1 + timeUs = 2089795 + flags = 1 + size = 4 + sample #141: + track = 0 + timeUs = 2160000 + size = 27 + sample #142: + track = 1 + timeUs = 2113015 + flags = 1 + size = 4 + sample #143: + track = 1 + timeUs = 2136235 + flags = 1 + size = 4 + sample #144: + track = 0 + timeUs = 2120000 + flags = 67108864 + size = 18 + sample #145: + track = 1 + timeUs = 2159455 + flags = 1 + size = 4 + sample #146: + track = 1 + timeUs = 2182675 + flags = 1 + size = 4 + sample #147: + track = 0 + timeUs = 2200000 + flags = 67108864 + size = 18 + sample #148: + track = 1 + timeUs = 2205895 + flags = 1 + size = 4 + sample #149: + track = 0 + timeUs = 2400000 + size = 28 + sample #150: + track = 1 + timeUs = 2229115 + flags = 1 + size = 4 + sample #151: + track = 1 + timeUs = 2252335 + flags = 1 + size = 4 + sample #152: + track = 0 + timeUs = 2320000 + size = 27 + sample #153: + track = 1 + timeUs = 2275555 + flags = 1 + size = 4 + sample #154: + track = 1 + timeUs = 2298775 + flags = 1 + size = 4 + sample #155: + track = 0 + timeUs = 2280000 + flags = 67108864 + size = 18 + sample #156: + track = 1 + timeUs = 2321995 + flags = 1 + size = 4 + sample #157: + track = 0 + timeUs = 2360000 + flags = 67108864 + size = 18 + sample #158: + track = 1 + timeUs = 2345215 + flags = 1 + size = 4 + sample #159: + track = 1 + timeUs = 2368435 + flags = 1 + size = 4 + sample #160: + track = 0 + timeUs = 2560000 + size = 28 + sample #161: + track = 1 + timeUs = 2391655 + flags = 1 + size = 4 + sample #162: + track = 1 + timeUs = 2414875 + flags = 1 + size = 4 + sample #163: + track = 0 + timeUs = 2480000 + size = 27 + sample #164: + track = 1 + timeUs = 2438095 + flags = 1 + size = 4 + sample #165: + track = 1 + timeUs = 2461315 + flags = 1 + size = 4 + sample #166: + track = 0 + timeUs = 2440000 + flags = 67108864 + size = 18 + sample #167: + track = 1 + timeUs = 2484535 + flags = 1 + size = 4 + sample #168: + track = 0 + timeUs = 2520000 + flags = 67108864 + size = 18 + sample #169: + track = 1 + timeUs = 2507755 + flags = 1 + size = 4 + sample #170: + track = 1 + timeUs = 2530975 + flags = 1 + size = 4 + sample #171: + track = 0 + timeUs = 2720000 + size = 28 + sample #172: + track = 1 + timeUs = 2554195 + flags = 1 + size = 4 + sample #173: + track = 1 + timeUs = 2577414 + flags = 1 + size = 4 + sample #174: + track = 0 + timeUs = 2640000 + size = 27 + sample #175: + track = 1 + timeUs = 2600634 + flags = 1 + size = 4 + sample #176: + track = 0 + timeUs = 2600000 + flags = 67108864 + size = 18 + sample #177: + track = 1 + timeUs = 2623854 + flags = 1 + size = 4 + sample #178: + track = 1 + timeUs = 2647074 + flags = 1 + size = 4 + sample #179: + track = 0 + timeUs = 2680000 + flags = 67108864 + size = 18 + sample #180: + track = 1 + timeUs = 2670294 + flags = 1 + size = 4 + sample #181: + track = 1 + timeUs = 2693514 + flags = 1 + size = 4 + sample #182: + track = 0 + timeUs = 2880000 + size = 28 + sample #183: + track = 1 + timeUs = 2716734 + flags = 1 + size = 4 + sample #184: + track = 1 + timeUs = 2739954 + flags = 1 + size = 4 + sample #185: + track = 0 + timeUs = 2800000 + size = 27 + sample #186: + track = 1 + timeUs = 2763174 + flags = 1 + size = 4 + sample #187: + track = 0 + timeUs = 2760000 + flags = 67108864 + size = 18 + sample #188: + track = 1 + timeUs = 2786394 + flags = 1 + size = 4 + sample #189: + track = 1 + timeUs = 2809614 + flags = 1 + size = 4 + sample #190: + track = 0 + timeUs = 2840000 + flags = 67108864 + size = 18 + sample #191: + track = 1 + timeUs = 2832834 + flags = 1 + size = 4 + sample #192: + track = 1 + timeUs = 2856054 + flags = 1 + size = 4 + sample #193: + track = 0 + timeUs = 3040000 + size = 28 + sample #194: + track = 1 + timeUs = 2879274 + flags = 1 + size = 4 + sample #195: + track = 1 + timeUs = 2902494 + flags = 1 + size = 4 + sample #196: + track = 0 + timeUs = 2960000 + size = 27 + sample #197: + track = 1 + timeUs = 2925714 + flags = 1 + size = 4 + sample #198: + track = 0 + timeUs = 2920000 + flags = 67108864 + size = 18 + sample #199: + track = 1 + timeUs = 2948934 + flags = 1 + size = 4 + sample #200: + track = 1 + timeUs = 2972154 + flags = 1 + size = 4 + sample #201: + track = 0 + timeUs = 3000000 + flags = 67108864 + size = 18 + sample #202: + track = 1 + timeUs = 2995374 + flags = 1 + size = 4 + sample #203: + track = 1 + timeUs = 3018594 + flags = 1 + size = 4 + sample #204: + track = 0 + timeUs = 3200000 + size = 28 + sample #205: + track = 1 + timeUs = 3041814 + flags = 1 + size = 4 + sample #206: + track = 0 + timeUs = 3120000 + size = 27 +MediaCodecAdapter (media3.audio.aac): + inputBuffers: + count = 126 + input buffer #0: + timeUs = 1000000116099 + contents = length 4, hash EE9DF + input buffer #1: + timeUs = 1000000139319 + contents = length 4, hash EE9DF + input buffer #2: + timeUs = 1000000162539 + contents = length 4, hash EE9DF + input buffer #3: + timeUs = 1000000185759 + contents = length 4, hash EE9DF + input buffer #4: + timeUs = 1000000208979 + contents = length 4, hash EE9DF + input buffer #5: + timeUs = 1000000232199 + contents = length 4, hash EE9DF + input buffer #6: + timeUs = 1000000255419 + contents = length 4, hash EE9DF + input buffer #7: + timeUs = 1000000278639 + contents = length 4, hash EE9DF + input buffer #8: + timeUs = 1000000301859 + contents = length 4, hash EE9DF + input buffer #9: + timeUs = 1000000325079 + contents = length 4, hash EE9DF + input buffer #10: + timeUs = 1000000348299 + contents = length 4, hash EE9DF + input buffer #11: + timeUs = 1000000371519 + contents = length 4, hash EE9DF + input buffer #12: + timeUs = 1000000394739 + contents = length 4, hash EE9DF + input buffer #13: + timeUs = 1000000417959 + contents = length 4, hash EE9DF + input buffer #14: + timeUs = 1000000441179 + contents = length 4, hash EE9DF + input buffer #15: + timeUs = 1000000464399 + contents = length 4, hash EE9DF + input buffer #16: + timeUs = 1000000487619 + contents = length 4, hash EE9DF + input buffer #17: + timeUs = 1000000510839 + contents = length 4, hash EE9DF + input buffer #18: + timeUs = 1000000534058 + contents = length 4, hash EE9DF + input buffer #19: + timeUs = 1000000557278 + contents = length 4, hash EE9DF + input buffer #20: + timeUs = 1000000580498 + contents = length 4, hash EE9DF + input buffer #21: + timeUs = 1000000603718 + contents = length 4, hash EE9DF + input buffer #22: + timeUs = 1000000626938 + contents = length 4, hash EE9DF + input buffer #23: + timeUs = 1000000650158 + contents = length 4, hash EE9DF + input buffer #24: + timeUs = 1000000673378 + contents = length 4, hash EE9DF + input buffer #25: + timeUs = 1000000696598 + contents = length 4, hash EE9DF + input buffer #26: + timeUs = 1000000719818 + contents = length 4, hash EE9DF + input buffer #27: + timeUs = 1000000743038 + contents = length 4, hash EE9DF + input buffer #28: + timeUs = 1000000766258 + contents = length 4, hash EE9DF + input buffer #29: + timeUs = 1000000789478 + contents = length 4, hash EE9DF + input buffer #30: + timeUs = 1000000812698 + contents = length 4, hash EE9DF + input buffer #31: + timeUs = 1000000835918 + contents = length 4, hash EE9DF + input buffer #32: + timeUs = 1000000859138 + contents = length 4, hash EE9DF + input buffer #33: + timeUs = 1000000882358 + contents = length 4, hash EE9DF + input buffer #34: + timeUs = 1000000905578 + contents = length 4, hash EE9DF + input buffer #35: + timeUs = 1000000928798 + contents = length 4, hash EE9DF + input buffer #36: + timeUs = 1000000952018 + contents = length 4, hash EE9DF + input buffer #37: + timeUs = 1000000975238 + contents = length 4, hash EE9DF + input buffer #38: + timeUs = 1000000998458 + contents = length 4, hash EE9DF + input buffer #39: + timeUs = 1000001021678 + contents = length 4, hash EE9DF + input buffer #40: + timeUs = 1000001044897 + contents = length 4, hash EE9DF + input buffer #41: + timeUs = 1000001068117 + contents = length 4, hash EE9DF + input buffer #42: + timeUs = 1000001091337 + contents = length 4, hash EE9DF + input buffer #43: + timeUs = 1000001114557 + contents = length 4, hash EE9DF + input buffer #44: + timeUs = 1000001137777 + contents = length 4, hash EE9DF + input buffer #45: + timeUs = 1000001160997 + contents = length 4, hash EE9DF + input buffer #46: + timeUs = 1000001184217 + contents = length 4, hash EE9DF + input buffer #47: + timeUs = 1000001207437 + contents = length 4, hash EE9DF + input buffer #48: + timeUs = 1000001230657 + contents = length 4, hash EE9DF + input buffer #49: + timeUs = 1000001253877 + contents = length 4, hash EE9DF + input buffer #50: + timeUs = 1000001277097 + contents = length 4, hash EE9DF + input buffer #51: + timeUs = 1000001300317 + contents = length 4, hash EE9DF + input buffer #52: + timeUs = 1000001323537 + contents = length 4, hash EE9DF + input buffer #53: + timeUs = 1000001346757 + contents = length 4, hash EE9DF + input buffer #54: + timeUs = 1000001369977 + contents = length 4, hash EE9DF + input buffer #55: + timeUs = 1000001393197 + contents = length 4, hash EE9DF + input buffer #56: + timeUs = 1000001416417 + contents = length 4, hash EE9DF + input buffer #57: + timeUs = 1000001439637 + contents = length 4, hash EE9DF + input buffer #58: + timeUs = 1000001462857 + contents = length 4, hash EE9DF + input buffer #59: + timeUs = 1000001486077 + contents = length 4, hash EE9DF + input buffer #60: + timeUs = 1000001509297 + contents = length 4, hash EE9DF + input buffer #61: + timeUs = 1000001532517 + contents = length 4, hash EE9DF + input buffer #62: + timeUs = 1000001555736 + contents = length 4, hash EE9DF + input buffer #63: + timeUs = 1000001578956 + contents = length 4, hash EE9DF + input buffer #64: + timeUs = 1000001602176 + contents = length 4, hash EE9DF + input buffer #65: + timeUs = 1000001625396 + contents = length 4, hash EE9DF + input buffer #66: + timeUs = 1000001648616 + contents = length 4, hash EE9DF + input buffer #67: + timeUs = 1000001671836 + contents = length 4, hash EE9DF + input buffer #68: + timeUs = 1000001695056 + contents = length 4, hash EE9DF + input buffer #69: + timeUs = 1000001718276 + contents = length 4, hash EE9DF + input buffer #70: + timeUs = 1000001741496 + contents = length 4, hash EE9DF + input buffer #71: + timeUs = 1000001764716 + contents = length 4, hash EE9DF + input buffer #72: + timeUs = 1000001787936 + contents = length 4, hash EE9DF + input buffer #73: + timeUs = 1000001811156 + contents = length 4, hash EE9DF + input buffer #74: + timeUs = 1000001834376 + contents = length 4, hash EE9DF + input buffer #75: + timeUs = 1000001857596 + contents = length 4, hash EE9DF + input buffer #76: + timeUs = 1000001880816 + contents = length 4, hash EE9DF + input buffer #77: + timeUs = 1000001904036 + contents = length 4, hash EE9DF + input buffer #78: + timeUs = 1000001927256 + contents = length 4, hash EE9DF + input buffer #79: + timeUs = 1000001950476 + contents = length 4, hash EE9DF + input buffer #80: + timeUs = 1000001973696 + contents = length 4, hash EE9DF + input buffer #81: + timeUs = 1000001996916 + contents = length 4, hash EE9DF + input buffer #82: + timeUs = 1000002020136 + contents = length 4, hash EE9DF + input buffer #83: + timeUs = 1000002043356 + contents = length 4, hash EE9DF + input buffer #84: + timeUs = 1000002066575 + contents = length 4, hash EE9DF + input buffer #85: + timeUs = 1000002089795 + contents = length 4, hash EE9DF + input buffer #86: + timeUs = 1000002113015 + contents = length 4, hash EE9DF + input buffer #87: + timeUs = 1000002136235 + contents = length 4, hash EE9DF + input buffer #88: + timeUs = 1000002159455 + contents = length 4, hash EE9DF + input buffer #89: + timeUs = 1000002182675 + contents = length 4, hash EE9DF + input buffer #90: + timeUs = 1000002205895 + contents = length 4, hash EE9DF + input buffer #91: + timeUs = 1000002229115 + contents = length 4, hash EE9DF + input buffer #92: + timeUs = 1000002252335 + contents = length 4, hash EE9DF + input buffer #93: + timeUs = 1000002275555 + contents = length 4, hash EE9DF + input buffer #94: + timeUs = 1000002298775 + contents = length 4, hash EE9DF + input buffer #95: + timeUs = 1000002321995 + contents = length 4, hash EE9DF + input buffer #96: + timeUs = 1000002345215 + contents = length 4, hash EE9DF + input buffer #97: + timeUs = 1000002368435 + contents = length 4, hash EE9DF + input buffer #98: + timeUs = 1000002391655 + contents = length 4, hash EE9DF + input buffer #99: + timeUs = 1000002414875 + contents = length 4, hash EE9DF + input buffer #100: + timeUs = 1000002438095 + contents = length 4, hash EE9DF + input buffer #101: + timeUs = 1000002461315 + contents = length 4, hash EE9DF + input buffer #102: + timeUs = 1000002484535 + contents = length 4, hash EE9DF + input buffer #103: + timeUs = 1000002507755 + contents = length 4, hash EE9DF + input buffer #104: + timeUs = 1000002530975 + contents = length 4, hash EE9DF + input buffer #105: + timeUs = 1000002554195 + contents = length 4, hash EE9DF + input buffer #106: + timeUs = 1000002577414 + contents = length 4, hash EE9DF + input buffer #107: + timeUs = 1000002600634 + contents = length 4, hash EE9DF + input buffer #108: + timeUs = 1000002623854 + contents = length 4, hash EE9DF + input buffer #109: + timeUs = 1000002647074 + contents = length 4, hash EE9DF + input buffer #110: + timeUs = 1000002670294 + contents = length 4, hash EE9DF + input buffer #111: + timeUs = 1000002693514 + contents = length 4, hash EE9DF + input buffer #112: + timeUs = 1000002716734 + contents = length 4, hash EE9DF + input buffer #113: + timeUs = 1000002739954 + contents = length 4, hash EE9DF + input buffer #114: + timeUs = 1000002763174 + contents = length 4, hash EE9DF + input buffer #115: + timeUs = 1000002786394 + contents = length 4, hash EE9DF + input buffer #116: + timeUs = 1000002809614 + contents = length 4, hash EE9DF + input buffer #117: + timeUs = 1000002832834 + contents = length 4, hash EE9DF + input buffer #118: + timeUs = 1000002856054 + contents = length 4, hash EE9DF + input buffer #119: + timeUs = 1000002879274 + contents = length 4, hash EE9DF + input buffer #120: + timeUs = 1000002902494 + contents = length 4, hash EE9DF + input buffer #121: + timeUs = 1000002925714 + contents = length 4, hash EE9DF + input buffer #122: + timeUs = 1000002948934 + contents = length 4, hash EE9DF + input buffer #123: + timeUs = 1000002972154 + contents = length 4, hash EE9DF + input buffer #124: + timeUs = 1000002995374 + contents = length 4, hash EE9DF + input buffer #125: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 125 + output buffer #0: + timeUs = 1000000116099 + size = 0 + rendered = false + output buffer #1: + timeUs = 1000000139319 + size = 0 + rendered = false + output buffer #2: + timeUs = 1000000162539 + size = 0 + rendered = false + output buffer #3: + timeUs = 1000000185759 + size = 0 + rendered = false + output buffer #4: + timeUs = 1000000208979 + size = 0 + rendered = false + output buffer #5: + timeUs = 1000000232199 + size = 0 + rendered = false + output buffer #6: + timeUs = 1000000255419 + size = 0 + rendered = false + output buffer #7: + timeUs = 1000000278639 + size = 0 + rendered = false + output buffer #8: + timeUs = 1000000301859 + size = 0 + rendered = false + output buffer #9: + timeUs = 1000000325079 + size = 0 + rendered = false + output buffer #10: + timeUs = 1000000348299 + size = 0 + rendered = false + output buffer #11: + timeUs = 1000000371519 + size = 0 + rendered = false + output buffer #12: + timeUs = 1000000394739 + size = 0 + rendered = false + output buffer #13: + timeUs = 1000000417959 + size = 0 + rendered = false + output buffer #14: + timeUs = 1000000441179 + size = 0 + rendered = false + output buffer #15: + timeUs = 1000000464399 + size = 0 + rendered = false + output buffer #16: + timeUs = 1000000487619 + size = 0 + rendered = false + output buffer #17: + timeUs = 1000000510839 + size = 0 + rendered = false + output buffer #18: + timeUs = 1000000534058 + size = 0 + rendered = false + output buffer #19: + timeUs = 1000000557278 + size = 0 + rendered = false + output buffer #20: + timeUs = 1000000580498 + size = 0 + rendered = false + output buffer #21: + timeUs = 1000000603718 + size = 0 + rendered = false + output buffer #22: + timeUs = 1000000626938 + size = 0 + rendered = false + output buffer #23: + timeUs = 1000000650158 + size = 0 + rendered = false + output buffer #24: + timeUs = 1000000673378 + size = 0 + rendered = false + output buffer #25: + timeUs = 1000000696598 + size = 0 + rendered = false + output buffer #26: + timeUs = 1000000719818 + size = 0 + rendered = false + output buffer #27: + timeUs = 1000000743038 + size = 0 + rendered = false + output buffer #28: + timeUs = 1000000766258 + size = 0 + rendered = false + output buffer #29: + timeUs = 1000000789478 + size = 0 + rendered = false + output buffer #30: + timeUs = 1000000812698 + size = 0 + rendered = false + output buffer #31: + timeUs = 1000000835918 + size = 0 + rendered = false + output buffer #32: + timeUs = 1000000859138 + size = 0 + rendered = false + output buffer #33: + timeUs = 1000000882358 + size = 0 + rendered = false + output buffer #34: + timeUs = 1000000905578 + size = 0 + rendered = false + output buffer #35: + timeUs = 1000000928798 + size = 0 + rendered = false + output buffer #36: + timeUs = 1000000952018 + size = 0 + rendered = false + output buffer #37: + timeUs = 1000000975238 + size = 0 + rendered = false + output buffer #38: + timeUs = 1000000998458 + size = 0 + rendered = false + output buffer #39: + timeUs = 1000001021678 + size = 0 + rendered = false + output buffer #40: + timeUs = 1000001044897 + size = 0 + rendered = false + output buffer #41: + timeUs = 1000001068117 + size = 0 + rendered = false + output buffer #42: + timeUs = 1000001091337 + size = 0 + rendered = false + output buffer #43: + timeUs = 1000001114557 + size = 0 + rendered = false + output buffer #44: + timeUs = 1000001137777 + size = 0 + rendered = false + output buffer #45: + timeUs = 1000001160997 + size = 0 + rendered = false + output buffer #46: + timeUs = 1000001184217 + size = 0 + rendered = false + output buffer #47: + timeUs = 1000001207437 + size = 0 + rendered = false + output buffer #48: + timeUs = 1000001230657 + size = 0 + rendered = false + output buffer #49: + timeUs = 1000001253877 + size = 0 + rendered = false + output buffer #50: + timeUs = 1000001277097 + size = 0 + rendered = false + output buffer #51: + timeUs = 1000001300317 + size = 0 + rendered = false + output buffer #52: + timeUs = 1000001323537 + size = 0 + rendered = false + output buffer #53: + timeUs = 1000001346757 + size = 0 + rendered = false + output buffer #54: + timeUs = 1000001369977 + size = 0 + rendered = false + output buffer #55: + timeUs = 1000001393197 + size = 0 + rendered = false + output buffer #56: + timeUs = 1000001416417 + size = 0 + rendered = false + output buffer #57: + timeUs = 1000001439637 + size = 0 + rendered = false + output buffer #58: + timeUs = 1000001462857 + size = 0 + rendered = false + output buffer #59: + timeUs = 1000001486077 + size = 0 + rendered = false + output buffer #60: + timeUs = 1000001509297 + size = 0 + rendered = false + output buffer #61: + timeUs = 1000001532517 + size = 0 + rendered = false + output buffer #62: + timeUs = 1000001555736 + size = 0 + rendered = false + output buffer #63: + timeUs = 1000001578956 + size = 0 + rendered = false + output buffer #64: + timeUs = 1000001602176 + size = 0 + rendered = false + output buffer #65: + timeUs = 1000001625396 + size = 0 + rendered = false + output buffer #66: + timeUs = 1000001648616 + size = 0 + rendered = false + output buffer #67: + timeUs = 1000001671836 + size = 0 + rendered = false + output buffer #68: + timeUs = 1000001695056 + size = 0 + rendered = false + output buffer #69: + timeUs = 1000001718276 + size = 0 + rendered = false + output buffer #70: + timeUs = 1000001741496 + size = 0 + rendered = false + output buffer #71: + timeUs = 1000001764716 + size = 0 + rendered = false + output buffer #72: + timeUs = 1000001787936 + size = 0 + rendered = false + output buffer #73: + timeUs = 1000001811156 + size = 0 + rendered = false + output buffer #74: + timeUs = 1000001834376 + size = 0 + rendered = false + output buffer #75: + timeUs = 1000001857596 + size = 0 + rendered = false + output buffer #76: + timeUs = 1000001880816 + size = 0 + rendered = false + output buffer #77: + timeUs = 1000001904036 + size = 0 + rendered = false + output buffer #78: + timeUs = 1000001927256 + size = 0 + rendered = false + output buffer #79: + timeUs = 1000001950476 + size = 0 + rendered = false + output buffer #80: + timeUs = 1000001973696 + size = 0 + rendered = false + output buffer #81: + timeUs = 1000001996916 + size = 0 + rendered = false + output buffer #82: + timeUs = 1000002020136 + size = 0 + rendered = false + output buffer #83: + timeUs = 1000002043356 + size = 0 + rendered = false + output buffer #84: + timeUs = 1000002066575 + size = 0 + rendered = false + output buffer #85: + timeUs = 1000002089795 + size = 0 + rendered = false + output buffer #86: + timeUs = 1000002113015 + size = 0 + rendered = false + output buffer #87: + timeUs = 1000002136235 + size = 0 + rendered = false + output buffer #88: + timeUs = 1000002159455 + size = 0 + rendered = false + output buffer #89: + timeUs = 1000002182675 + size = 0 + rendered = false + output buffer #90: + timeUs = 1000002205895 + size = 0 + rendered = false + output buffer #91: + timeUs = 1000002229115 + size = 0 + rendered = false + output buffer #92: + timeUs = 1000002252335 + size = 0 + rendered = false + output buffer #93: + timeUs = 1000002275555 + size = 0 + rendered = false + output buffer #94: + timeUs = 1000002298775 + size = 0 + rendered = false + output buffer #95: + timeUs = 1000002321995 + size = 0 + rendered = false + output buffer #96: + timeUs = 1000002345215 + size = 0 + rendered = false + output buffer #97: + timeUs = 1000002368435 + size = 0 + rendered = false + output buffer #98: + timeUs = 1000002391655 + size = 0 + rendered = false + output buffer #99: + timeUs = 1000002414875 + size = 0 + rendered = false + output buffer #100: + timeUs = 1000002438095 + size = 0 + rendered = false + output buffer #101: + timeUs = 1000002461315 + size = 0 + rendered = false + output buffer #102: + timeUs = 1000002484535 + size = 0 + rendered = false + output buffer #103: + timeUs = 1000002507755 + size = 0 + rendered = false + output buffer #104: + timeUs = 1000002530975 + size = 0 + rendered = false + output buffer #105: + timeUs = 1000002554195 + size = 0 + rendered = false + output buffer #106: + timeUs = 1000002577414 + size = 0 + rendered = false + output buffer #107: + timeUs = 1000002600634 + size = 0 + rendered = false + output buffer #108: + timeUs = 1000002623854 + size = 0 + rendered = false + output buffer #109: + timeUs = 1000002647074 + size = 0 + rendered = false + output buffer #110: + timeUs = 1000002670294 + size = 0 + rendered = false + output buffer #111: + timeUs = 1000002693514 + size = 0 + rendered = false + output buffer #112: + timeUs = 1000002716734 + size = 0 + rendered = false + output buffer #113: + timeUs = 1000002739954 + size = 0 + rendered = false + output buffer #114: + timeUs = 1000002763174 + size = 0 + rendered = false + output buffer #115: + timeUs = 1000002786394 + size = 0 + rendered = false + output buffer #116: + timeUs = 1000002809614 + size = 0 + rendered = false + output buffer #117: + timeUs = 1000002832834 + size = 0 + rendered = false + output buffer #118: + timeUs = 1000002856054 + size = 0 + rendered = false + output buffer #119: + timeUs = 1000002879274 + size = 0 + rendered = false + output buffer #120: + timeUs = 1000002902494 + size = 0 + rendered = false + output buffer #121: + timeUs = 1000002925714 + size = 0 + rendered = false + output buffer #122: + timeUs = 1000002948934 + size = 0 + rendered = false + output buffer #123: + timeUs = 1000002972154 + size = 0 + rendered = false + output buffer #124: + timeUs = 1000002995374 + size = 0 + rendered = false +MediaCodecAdapter (media3.video.avc): + inputBuffers: + count = 76 + input buffer #0: + timeUs = 1000000000000 + contents = length 5252, hash 13893A4C + input buffer #1: + timeUs = 1000000160000 + contents = length 44, hash A05B3BEA + input buffer #2: + timeUs = 1000000080000 + contents = length 25, hash 441EC4FE + input buffer #3: + timeUs = 1000000120000 + contents = length 18, hash B64DA059 + input buffer #4: + timeUs = 1000000320000 + contents = length 32, hash 5BFD9538 + input buffer #5: + timeUs = 1000000240000 + contents = length 27, hash 705869A5 + input buffer #6: + timeUs = 1000000200000 + contents = length 18, hash D163DF61 + input buffer #7: + timeUs = 1000000280000 + contents = length 18, hash FD82E95 + input buffer #8: + timeUs = 1000000480000 + contents = length 28, hash 44A16E72 + input buffer #9: + timeUs = 1000000400000 + contents = length 27, hash FFA326ED + input buffer #10: + timeUs = 1000000360000 + contents = length 18, hash DC93CC9D + input buffer #11: + timeUs = 1000000440000 + contents = length 18, hash 1B081BD1 + input buffer #12: + timeUs = 1000000640000 + contents = length 28, hash 2700AF + input buffer #13: + timeUs = 1000000560000 + contents = length 27, hash DECA17D0 + input buffer #14: + timeUs = 1000000520000 + contents = length 18, hash D646C05A + input buffer #15: + timeUs = 1000000600000 + contents = length 18, hash 14BB0F8E + input buffer #16: + timeUs = 1000000800000 + contents = length 28, hash 5DE2C2B + input buffer #17: + timeUs = 1000000720000 + contents = length 27, hash 376B4E14 + input buffer #18: + timeUs = 1000000680000 + contents = length 18, hash E176AD96 + input buffer #19: + timeUs = 1000000760000 + contents = length 18, hash 1FEAFCCA + input buffer #20: + timeUs = 1000000960000 + contents = length 28, hash C163BE68 + input buffer #21: + timeUs = 1000000880000 + contents = length 27, hash F7170357 + input buffer #22: + timeUs = 1000000840000 + contents = length 18, hash 3B013BD2 + input buffer #23: + timeUs = 1000000920000 + contents = length 18, hash 79758B06 + input buffer #24: + timeUs = 1000001120000 + contents = length 28, hash F72EB1A3 + input buffer #25: + timeUs = 1000001040000 + contents = length 27, hash 4FB8399C + input buffer #26: + timeUs = 1000001000000 + contents = length 18, hash 4631290E + input buffer #27: + timeUs = 1000001080000 + contents = length 18, hash 84A57842 + input buffer #28: + timeUs = 1000001280000 + contents = length 28, hash E1FCF000 + input buffer #29: + timeUs = 1000001200000 + contents = length 27, hash 21D3B540 + input buffer #30: + timeUs = 1000001160000 + contents = length 18, hash 62DE0FC9 + input buffer #31: + timeUs = 1000001240000 + contents = length 18, hash A1525EFD + input buffer #32: + timeUs = 1000001440000 + contents = length 28, hash 5350E8FA + input buffer #33: + timeUs = 1000001360000 + contents = length 27, hash E7563B65 + input buffer #34: + timeUs = 1000001320000 + contents = length 18, hash 77D95125 + input buffer #35: + timeUs = 1000001400000 + contents = length 18, hash B64DA059 + input buffer #36: + timeUs = 1000001600000 + contents = length 28, hash ED67B37 + input buffer #37: + timeUs = 1000001520000 + contents = length 27, hash A701F0A9 + input buffer #38: + timeUs = 1000001480000 + contents = length 18, hash D163DF61 + input buffer #39: + timeUs = 1000001560000 + contents = length 18, hash FD82E95 + input buffer #40: + timeUs = 1000001760000 + contents = length 28, hash 44A16E72 + input buffer #41: + timeUs = 1000001680000 + contents = length 27, hash FFA326ED + input buffer #42: + timeUs = 1000001640000 + contents = length 18, hash DC93CC9D + input buffer #43: + timeUs = 1000001720000 + contents = length 18, hash 1B081BD1 + input buffer #44: + timeUs = 1000001920000 + contents = length 28, hash 2700AF + input buffer #45: + timeUs = 1000001840000 + contents = length 27, hash DECA17D0 + input buffer #46: + timeUs = 1000001800000 + contents = length 18, hash D646C05A + input buffer #47: + timeUs = 1000001880000 + contents = length 18, hash 14BB0F8E + input buffer #48: + timeUs = 1000002080000 + contents = length 28, hash 5DE2C2B + input buffer #49: + timeUs = 1000002000000 + contents = length 27, hash 376B4E14 + input buffer #50: + timeUs = 1000001960000 + contents = length 18, hash E176AD96 + input buffer #51: + timeUs = 1000002040000 + contents = length 18, hash 1FEAFCCA + input buffer #52: + timeUs = 1000002240000 + contents = length 28, hash C163BE68 + input buffer #53: + timeUs = 1000002160000 + contents = length 27, hash F7170357 + input buffer #54: + timeUs = 1000002120000 + contents = length 18, hash 3B013BD2 + input buffer #55: + timeUs = 1000002200000 + contents = length 18, hash 79758B06 + input buffer #56: + timeUs = 1000002400000 + contents = length 28, hash F72EB1A3 + input buffer #57: + timeUs = 1000002320000 + contents = length 27, hash 4FB8399C + input buffer #58: + timeUs = 1000002280000 + contents = length 18, hash 4631290E + input buffer #59: + timeUs = 1000002360000 + contents = length 18, hash 84A57842 + input buffer #60: + timeUs = 1000002560000 + contents = length 28, hash E1FCF000 + input buffer #61: + timeUs = 1000002480000 + contents = length 27, hash 21D3B540 + input buffer #62: + timeUs = 1000002440000 + contents = length 18, hash 62DE0FC9 + input buffer #63: + timeUs = 1000002520000 + contents = length 18, hash A1525EFD + input buffer #64: + timeUs = 1000002720000 + contents = length 28, hash 5350E8FA + input buffer #65: + timeUs = 1000002640000 + contents = length 27, hash E7563B65 + input buffer #66: + timeUs = 1000002600000 + contents = length 18, hash 77D95125 + input buffer #67: + timeUs = 1000002680000 + contents = length 18, hash B64DA059 + input buffer #68: + timeUs = 1000002880000 + contents = length 28, hash ED67B37 + input buffer #69: + timeUs = 1000002800000 + contents = length 27, hash A701F0A9 + input buffer #70: + timeUs = 1000002760000 + contents = length 18, hash D163DF61 + input buffer #71: + timeUs = 1000002840000 + contents = length 18, hash FD82E95 + input buffer #72: + timeUs = 1000003040000 + contents = length 28, hash 44A16E72 + input buffer #73: + timeUs = 1000002960000 + contents = length 27, hash FFA326ED + input buffer #74: + timeUs = 1000002920000 + contents = length 18, hash DC93CC9D + input buffer #75: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 75 + output buffer #0: + timeUs = 1000000000000 + size = 5252 + rendered = false + output buffer #1: + timeUs = 1000000160000 + size = 44 + rendered = true + output buffer #2: + timeUs = 1000000080000 + size = 25 + rendered = false + output buffer #3: + timeUs = 1000000120000 + size = 18 + rendered = true + output buffer #4: + timeUs = 1000000320000 + size = 32 + rendered = true + output buffer #5: + timeUs = 1000000240000 + size = 27 + rendered = true + output buffer #6: + timeUs = 1000000200000 + size = 18 + rendered = true + output buffer #7: + timeUs = 1000000280000 + size = 18 + rendered = true + output buffer #8: + timeUs = 1000000480000 + size = 28 + rendered = true + output buffer #9: + timeUs = 1000000400000 + size = 27 + rendered = true + output buffer #10: + timeUs = 1000000360000 + size = 18 + rendered = true + output buffer #11: + timeUs = 1000000440000 + size = 18 + rendered = true + output buffer #12: + timeUs = 1000000640000 + size = 28 + rendered = true + output buffer #13: + timeUs = 1000000560000 + size = 27 + rendered = true + output buffer #14: + timeUs = 1000000520000 + size = 18 + rendered = true + output buffer #15: + timeUs = 1000000600000 + size = 18 + rendered = true + output buffer #16: + timeUs = 1000000800000 + size = 28 + rendered = true + output buffer #17: + timeUs = 1000000720000 + size = 27 + rendered = true + output buffer #18: + timeUs = 1000000680000 + size = 18 + rendered = true + output buffer #19: + timeUs = 1000000760000 + size = 18 + rendered = true + output buffer #20: + timeUs = 1000000960000 + size = 28 + rendered = true + output buffer #21: + timeUs = 1000000880000 + size = 27 + rendered = true + output buffer #22: + timeUs = 1000000840000 + size = 18 + rendered = true + output buffer #23: + timeUs = 1000000920000 + size = 18 + rendered = true + output buffer #24: + timeUs = 1000001120000 + size = 28 + rendered = true + output buffer #25: + timeUs = 1000001040000 + size = 27 + rendered = true + output buffer #26: + timeUs = 1000001000000 + size = 18 + rendered = true + output buffer #27: + timeUs = 1000001080000 + size = 18 + rendered = true + output buffer #28: + timeUs = 1000001280000 + size = 28 + rendered = true + output buffer #29: + timeUs = 1000001200000 + size = 27 + rendered = true + output buffer #30: + timeUs = 1000001160000 + size = 18 + rendered = true + output buffer #31: + timeUs = 1000001240000 + size = 18 + rendered = true + output buffer #32: + timeUs = 1000001440000 + size = 28 + rendered = true + output buffer #33: + timeUs = 1000001360000 + size = 27 + rendered = true + output buffer #34: + timeUs = 1000001320000 + size = 18 + rendered = true + output buffer #35: + timeUs = 1000001400000 + size = 18 + rendered = true + output buffer #36: + timeUs = 1000001600000 + size = 28 + rendered = true + output buffer #37: + timeUs = 1000001520000 + size = 27 + rendered = true + output buffer #38: + timeUs = 1000001480000 + size = 18 + rendered = true + output buffer #39: + timeUs = 1000001560000 + size = 18 + rendered = true + output buffer #40: + timeUs = 1000001760000 + size = 28 + rendered = true + output buffer #41: + timeUs = 1000001680000 + size = 27 + rendered = true + output buffer #42: + timeUs = 1000001640000 + size = 18 + rendered = true + output buffer #43: + timeUs = 1000001720000 + size = 18 + rendered = true + output buffer #44: + timeUs = 1000001920000 + size = 28 + rendered = true + output buffer #45: + timeUs = 1000001840000 + size = 27 + rendered = true + output buffer #46: + timeUs = 1000001800000 + size = 18 + rendered = true + output buffer #47: + timeUs = 1000001880000 + size = 18 + rendered = true + output buffer #48: + timeUs = 1000002080000 + size = 28 + rendered = true + output buffer #49: + timeUs = 1000002000000 + size = 27 + rendered = true + output buffer #50: + timeUs = 1000001960000 + size = 18 + rendered = true + output buffer #51: + timeUs = 1000002040000 + size = 18 + rendered = true + output buffer #52: + timeUs = 1000002240000 + size = 28 + rendered = true + output buffer #53: + timeUs = 1000002160000 + size = 27 + rendered = true + output buffer #54: + timeUs = 1000002120000 + size = 18 + rendered = true + output buffer #55: + timeUs = 1000002200000 + size = 18 + rendered = true + output buffer #56: + timeUs = 1000002400000 + size = 28 + rendered = true + output buffer #57: + timeUs = 1000002320000 + size = 27 + rendered = true + output buffer #58: + timeUs = 1000002280000 + size = 18 + rendered = true + output buffer #59: + timeUs = 1000002360000 + size = 18 + rendered = true + output buffer #60: + timeUs = 1000002560000 + size = 28 + rendered = true + output buffer #61: + timeUs = 1000002480000 + size = 27 + rendered = true + output buffer #62: + timeUs = 1000002440000 + size = 18 + rendered = true + output buffer #63: + timeUs = 1000002520000 + size = 18 + rendered = true + output buffer #64: + timeUs = 1000002720000 + size = 28 + rendered = true + output buffer #65: + timeUs = 1000002640000 + size = 27 + rendered = true + output buffer #66: + timeUs = 1000002600000 + size = 18 + rendered = true + output buffer #67: + timeUs = 1000002680000 + size = 18 + rendered = true + output buffer #68: + timeUs = 1000002880000 + size = 28 + rendered = true + output buffer #69: + timeUs = 1000002800000 + size = 27 + rendered = true + output buffer #70: + timeUs = 1000002760000 + size = 18 + rendered = true + output buffer #71: + timeUs = 1000002840000 + size = 18 + rendered = true + output buffer #72: + timeUs = 1000003040000 + size = 28 + rendered = false + output buffer #73: + timeUs = 1000002960000 + size = 27 + rendered = true + output buffer #74: + timeUs = 1000002920000 + size = 18 + rendered = true +AudioSink: + buffer count = 125 + config: + pcmEncoding = pcm-16 + channelCount = 1 + sampleRate = 44100 + buffer #0: + time = 1000000116099 + data = empty + buffer #1: + time = 1000000139319 + data = empty + buffer #2: + time = 1000000162539 + data = empty + buffer #3: + time = 1000000185759 + data = empty + buffer #4: + time = 1000000208979 + data = empty + buffer #5: + time = 1000000232199 + data = empty + buffer #6: + time = 1000000255419 + data = empty + buffer #7: + time = 1000000278639 + data = empty + buffer #8: + time = 1000000301859 + data = empty + buffer #9: + time = 1000000325079 + data = empty + buffer #10: + time = 1000000348299 + data = empty + buffer #11: + time = 1000000371519 + data = empty + buffer #12: + time = 1000000394739 + data = empty + buffer #13: + time = 1000000417959 + data = empty + buffer #14: + time = 1000000441179 + data = empty + buffer #15: + time = 1000000464399 + data = empty + buffer #16: + time = 1000000487619 + data = empty + buffer #17: + time = 1000000510839 + data = empty + buffer #18: + time = 1000000534058 + data = empty + buffer #19: + time = 1000000557278 + data = empty + buffer #20: + time = 1000000580498 + data = empty + buffer #21: + time = 1000000603718 + data = empty + buffer #22: + time = 1000000626938 + data = empty + buffer #23: + time = 1000000650158 + data = empty + buffer #24: + time = 1000000673378 + data = empty + buffer #25: + time = 1000000696598 + data = empty + buffer #26: + time = 1000000719818 + data = empty + buffer #27: + time = 1000000743038 + data = empty + buffer #28: + time = 1000000766258 + data = empty + buffer #29: + time = 1000000789478 + data = empty + buffer #30: + time = 1000000812698 + data = empty + buffer #31: + time = 1000000835918 + data = empty + buffer #32: + time = 1000000859138 + data = empty + buffer #33: + time = 1000000882358 + data = empty + buffer #34: + time = 1000000905578 + data = empty + buffer #35: + time = 1000000928798 + data = empty + buffer #36: + time = 1000000952018 + data = empty + buffer #37: + time = 1000000975238 + data = empty + buffer #38: + time = 1000000998458 + data = empty + buffer #39: + time = 1000001021678 + data = empty + buffer #40: + time = 1000001044897 + data = empty + buffer #41: + time = 1000001068117 + data = empty + buffer #42: + time = 1000001091337 + data = empty + buffer #43: + time = 1000001114557 + data = empty + buffer #44: + time = 1000001137777 + data = empty + buffer #45: + time = 1000001160997 + data = empty + buffer #46: + time = 1000001184217 + data = empty + buffer #47: + time = 1000001207437 + data = empty + buffer #48: + time = 1000001230657 + data = empty + buffer #49: + time = 1000001253877 + data = empty + buffer #50: + time = 1000001277097 + data = empty + buffer #51: + time = 1000001300317 + data = empty + buffer #52: + time = 1000001323537 + data = empty + buffer #53: + time = 1000001346757 + data = empty + buffer #54: + time = 1000001369977 + data = empty + buffer #55: + time = 1000001393197 + data = empty + buffer #56: + time = 1000001416417 + data = empty + buffer #57: + time = 1000001439637 + data = empty + buffer #58: + time = 1000001462857 + data = empty + buffer #59: + time = 1000001486077 + data = empty + buffer #60: + time = 1000001509297 + data = empty + buffer #61: + time = 1000001532517 + data = empty + buffer #62: + time = 1000001555736 + data = empty + buffer #63: + time = 1000001578956 + data = empty + buffer #64: + time = 1000001602176 + data = empty + buffer #65: + time = 1000001625396 + data = empty + buffer #66: + time = 1000001648616 + data = empty + buffer #67: + time = 1000001671836 + data = empty + buffer #68: + time = 1000001695056 + data = empty + buffer #69: + time = 1000001718276 + data = empty + buffer #70: + time = 1000001741496 + data = empty + buffer #71: + time = 1000001764716 + data = empty + buffer #72: + time = 1000001787936 + data = empty + buffer #73: + time = 1000001811156 + data = empty + buffer #74: + time = 1000001834376 + data = empty + buffer #75: + time = 1000001857596 + data = empty + buffer #76: + time = 1000001880816 + data = empty + buffer #77: + time = 1000001904036 + data = empty + buffer #78: + time = 1000001927256 + data = empty + buffer #79: + time = 1000001950476 + data = empty + buffer #80: + time = 1000001973696 + data = empty + buffer #81: + time = 1000001996916 + data = empty + buffer #82: + time = 1000002020136 + data = empty + buffer #83: + time = 1000002043356 + data = empty + buffer #84: + time = 1000002066575 + data = empty + buffer #85: + time = 1000002089795 + data = empty + buffer #86: + time = 1000002113015 + data = empty + buffer #87: + time = 1000002136235 + data = empty + buffer #88: + time = 1000002159455 + data = empty + buffer #89: + time = 1000002182675 + data = empty + buffer #90: + time = 1000002205895 + data = empty + buffer #91: + time = 1000002229115 + data = empty + buffer #92: + time = 1000002252335 + data = empty + buffer #93: + time = 1000002275555 + data = empty + buffer #94: + time = 1000002298775 + data = empty + buffer #95: + time = 1000002321995 + data = empty + buffer #96: + time = 1000002345215 + data = empty + buffer #97: + time = 1000002368435 + data = empty + buffer #98: + time = 1000002391655 + data = empty + buffer #99: + time = 1000002414875 + data = empty + buffer #100: + time = 1000002438095 + data = empty + buffer #101: + time = 1000002461315 + data = empty + buffer #102: + time = 1000002484535 + data = empty + buffer #103: + time = 1000002507755 + data = empty + buffer #104: + time = 1000002530975 + data = empty + buffer #105: + time = 1000002554195 + data = empty + buffer #106: + time = 1000002577414 + data = empty + buffer #107: + time = 1000002600634 + data = empty + buffer #108: + time = 1000002623854 + data = empty + buffer #109: + time = 1000002647074 + data = empty + buffer #110: + time = 1000002670294 + data = empty + buffer #111: + time = 1000002693514 + data = empty + buffer #112: + time = 1000002716734 + data = empty + buffer #113: + time = 1000002739954 + data = empty + buffer #114: + time = 1000002763174 + data = empty + buffer #115: + time = 1000002786394 + data = empty + buffer #116: + time = 1000002809614 + data = empty + buffer #117: + time = 1000002832834 + data = empty + buffer #118: + time = 1000002856054 + data = empty + buffer #119: + time = 1000002879274 + data = empty + buffer #120: + time = 1000002902494 + data = empty + buffer #121: + time = 1000002925714 + data = empty + buffer #122: + time = 1000002948934 + data = empty + buffer #123: + time = 1000002972154 + data = empty + buffer #124: + time = 1000002995374 + data = empty diff --git a/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_extend_to_end_of_source_after_load_finished_enabled.dump b/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_extend_to_end_of_source_after_load_finished_enabled.dump new file mode 100644 index 00000000000..96c5e30c4af --- /dev/null +++ b/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_extend_to_end_of_source_after_load_finished_enabled.dump @@ -0,0 +1,5343 @@ +extracted samples: + sample #0: + track = 0 + timeUs = 0 + flags = 1 + size = 5252 + sample #1: + track = 0 + timeUs = 160000 + size = 44 + sample #2: + track = 0 + timeUs = 80000 + size = 25 + sample #3: + track = 0 + timeUs = 40000 + flags = 67108864 + size = 18 + sample #4: + track = 1 + timeUs = 92879 + flags = 1 + size = 4 + sample #5: + track = 0 + timeUs = 120000 + flags = 67108864 + size = 18 + sample #6: + track = 1 + timeUs = 116099 + flags = 1 + size = 4 + sample #7: + track = 1 + timeUs = 139319 + flags = 1 + size = 4 + sample #8: + track = 0 + timeUs = 320000 + size = 32 + sample #9: + track = 1 + timeUs = 162539 + flags = 1 + size = 4 + sample #10: + track = 0 + timeUs = 240000 + size = 27 + sample #11: + track = 1 + timeUs = 185759 + flags = 1 + size = 4 + sample #12: + track = 1 + timeUs = 208979 + flags = 1 + size = 4 + sample #13: + track = 0 + timeUs = 200000 + flags = 67108864 + size = 18 + sample #14: + track = 1 + timeUs = 232199 + flags = 1 + size = 4 + sample #15: + track = 1 + timeUs = 255419 + flags = 1 + size = 4 + sample #16: + track = 0 + timeUs = 280000 + flags = 67108864 + size = 18 + sample #17: + track = 1 + timeUs = 278639 + flags = 1 + size = 4 + sample #18: + track = 1 + timeUs = 301859 + flags = 1 + size = 4 + sample #19: + track = 0 + timeUs = 480000 + size = 28 + sample #20: + track = 1 + timeUs = 325079 + flags = 1 + size = 4 + sample #21: + track = 0 + timeUs = 400000 + size = 27 + sample #22: + track = 1 + timeUs = 348299 + flags = 1 + size = 4 + sample #23: + track = 1 + timeUs = 371519 + flags = 1 + size = 4 + sample #24: + track = 0 + timeUs = 360000 + flags = 67108864 + size = 18 + sample #25: + track = 1 + timeUs = 394739 + flags = 1 + size = 4 + sample #26: + track = 1 + timeUs = 417959 + flags = 1 + size = 4 + sample #27: + track = 0 + timeUs = 440000 + flags = 67108864 + size = 18 + sample #28: + track = 1 + timeUs = 441179 + flags = 1 + size = 4 + sample #29: + track = 0 + timeUs = 640000 + size = 28 + sample #30: + track = 1 + timeUs = 464399 + flags = 1 + size = 4 + sample #31: + track = 1 + timeUs = 487619 + flags = 1 + size = 4 + sample #32: + track = 0 + timeUs = 560000 + size = 27 + sample #33: + track = 1 + timeUs = 510839 + flags = 1 + size = 4 + sample #34: + track = 1 + timeUs = 534058 + flags = 1 + size = 4 + sample #35: + track = 0 + timeUs = 520000 + flags = 67108864 + size = 18 + sample #36: + track = 1 + timeUs = 557278 + flags = 1 + size = 4 + sample #37: + track = 1 + timeUs = 580498 + flags = 1 + size = 4 + sample #38: + track = 0 + timeUs = 600000 + flags = 67108864 + size = 18 + sample #39: + track = 1 + timeUs = 603718 + flags = 1 + size = 4 + sample #40: + track = 0 + timeUs = 800000 + size = 28 + sample #41: + track = 1 + timeUs = 626938 + flags = 1 + size = 4 + sample #42: + track = 1 + timeUs = 650158 + flags = 1 + size = 4 + sample #43: + track = 0 + timeUs = 720000 + size = 27 + sample #44: + track = 1 + timeUs = 673378 + flags = 1 + size = 4 + sample #45: + track = 1 + timeUs = 696598 + flags = 1 + size = 4 + sample #46: + track = 0 + timeUs = 680000 + flags = 67108864 + size = 18 + sample #47: + track = 1 + timeUs = 719818 + flags = 1 + size = 4 + sample #48: + track = 1 + timeUs = 743038 + flags = 1 + size = 4 + sample #49: + track = 0 + timeUs = 760000 + flags = 67108864 + size = 18 + sample #50: + track = 1 + timeUs = 766258 + flags = 1 + size = 4 + sample #51: + track = 0 + timeUs = 960000 + size = 28 + sample #52: + track = 1 + timeUs = 789478 + flags = 1 + size = 4 + sample #53: + track = 1 + timeUs = 812698 + flags = 1 + size = 4 + sample #54: + track = 0 + timeUs = 880000 + size = 27 + sample #55: + track = 1 + timeUs = 835918 + flags = 1 + size = 4 + sample #56: + track = 1 + timeUs = 859138 + flags = 1 + size = 4 + sample #57: + track = 0 + timeUs = 840000 + flags = 67108864 + size = 18 + sample #58: + track = 1 + timeUs = 882358 + flags = 1 + size = 4 + sample #59: + track = 0 + timeUs = 920000 + flags = 67108864 + size = 18 + sample #60: + track = 1 + timeUs = 905578 + flags = 1 + size = 4 + sample #61: + track = 1 + timeUs = 928798 + flags = 1 + size = 4 + sample #62: + track = 0 + timeUs = 1120000 + size = 28 + sample #63: + track = 1 + timeUs = 952018 + flags = 1 + size = 4 + sample #64: + track = 1 + timeUs = 975238 + flags = 1 + size = 4 + sample #65: + track = 0 + timeUs = 1040000 + size = 27 + sample #66: + track = 1 + timeUs = 998458 + flags = 1 + size = 4 + sample #67: + track = 1 + timeUs = 1021678 + flags = 1 + size = 4 + sample #68: + track = 0 + timeUs = 1000000 + flags = 67108864 + size = 18 + sample #69: + track = 1 + timeUs = 1044897 + flags = 1 + size = 4 + sample #70: + track = 0 + timeUs = 1080000 + flags = 67108864 + size = 18 + sample #71: + track = 1 + timeUs = 1068117 + flags = 1 + size = 4 + sample #72: + track = 1 + timeUs = 1091337 + flags = 1 + size = 4 + sample #73: + track = 0 + timeUs = 1280000 + size = 28 + sample #74: + track = 1 + timeUs = 1114557 + flags = 1 + size = 4 + sample #75: + track = 1 + timeUs = 1137777 + flags = 1 + size = 4 + sample #76: + track = 0 + timeUs = 1200000 + size = 27 + sample #77: + track = 1 + timeUs = 1160997 + flags = 1 + size = 4 + sample #78: + track = 0 + timeUs = 1160000 + flags = 67108864 + size = 18 + sample #79: + track = 1 + timeUs = 1184217 + flags = 1 + size = 4 + sample #80: + track = 1 + timeUs = 1207437 + flags = 1 + size = 4 + sample #81: + track = 0 + timeUs = 1240000 + flags = 67108864 + size = 18 + sample #82: + track = 1 + timeUs = 1230657 + flags = 1 + size = 4 + sample #83: + track = 1 + timeUs = 1253877 + flags = 1 + size = 4 + sample #84: + track = 0 + timeUs = 1440000 + size = 28 + sample #85: + track = 1 + timeUs = 1277097 + flags = 1 + size = 4 + sample #86: + track = 1 + timeUs = 1300317 + flags = 1 + size = 4 + sample #87: + track = 0 + timeUs = 1360000 + size = 27 + sample #88: + track = 1 + timeUs = 1323537 + flags = 1 + size = 4 + sample #89: + track = 0 + timeUs = 1320000 + flags = 67108864 + size = 18 + sample #90: + track = 1 + timeUs = 1346757 + flags = 1 + size = 4 + sample #91: + track = 1 + timeUs = 1369977 + flags = 1 + size = 4 + sample #92: + track = 0 + timeUs = 1400000 + flags = 67108864 + size = 18 + sample #93: + track = 1 + timeUs = 1393197 + flags = 1 + size = 4 + sample #94: + track = 1 + timeUs = 1416417 + flags = 1 + size = 4 + sample #95: + track = 0 + timeUs = 1600000 + size = 28 + sample #96: + track = 1 + timeUs = 1439637 + flags = 1 + size = 4 + sample #97: + track = 1 + timeUs = 1462857 + flags = 1 + size = 4 + sample #98: + track = 0 + timeUs = 1520000 + size = 27 + sample #99: + track = 1 + timeUs = 1486077 + flags = 1 + size = 4 + sample #100: + track = 0 + timeUs = 1480000 + flags = 67108864 + size = 18 + sample #101: + track = 1 + timeUs = 1509297 + flags = 1 + size = 4 + sample #102: + track = 1 + timeUs = 1532517 + flags = 1 + size = 4 + sample #103: + track = 0 + timeUs = 1560000 + flags = 67108864 + size = 18 + sample #104: + track = 1 + timeUs = 1555736 + flags = 1 + size = 4 + sample #105: + track = 1 + timeUs = 1578956 + flags = 1 + size = 4 + sample #106: + track = 0 + timeUs = 1760000 + size = 28 + sample #107: + track = 1 + timeUs = 1602176 + flags = 1 + size = 4 + sample #108: + track = 0 + timeUs = 1680000 + size = 27 + sample #109: + track = 1 + timeUs = 1625396 + flags = 1 + size = 4 + sample #110: + track = 1 + timeUs = 1648616 + flags = 1 + size = 4 + sample #111: + track = 0 + timeUs = 1640000 + flags = 67108864 + size = 18 + sample #112: + track = 1 + timeUs = 1671836 + flags = 1 + size = 4 + sample #113: + track = 1 + timeUs = 1695056 + flags = 1 + size = 4 + sample #114: + track = 0 + timeUs = 1720000 + flags = 67108864 + size = 18 + sample #115: + track = 1 + timeUs = 1718276 + flags = 1 + size = 4 + sample #116: + track = 1 + timeUs = 1741496 + flags = 1 + size = 4 + sample #117: + track = 0 + timeUs = 1920000 + size = 28 + sample #118: + track = 1 + timeUs = 1764716 + flags = 1 + size = 4 + sample #119: + track = 0 + timeUs = 1840000 + size = 27 + sample #120: + track = 1 + timeUs = 1787936 + flags = 1 + size = 4 + sample #121: + track = 1 + timeUs = 1811156 + flags = 1 + size = 4 + sample #122: + track = 0 + timeUs = 1800000 + flags = 67108864 + size = 18 + sample #123: + track = 1 + timeUs = 1834376 + flags = 1 + size = 4 + sample #124: + track = 1 + timeUs = 1857596 + flags = 1 + size = 4 + sample #125: + track = 0 + timeUs = 1880000 + flags = 67108864 + size = 18 + sample #126: + track = 1 + timeUs = 1880816 + flags = 1 + size = 4 + sample #127: + track = 0 + timeUs = 2080000 + size = 28 + sample #128: + track = 1 + timeUs = 1904036 + flags = 1 + size = 4 + sample #129: + track = 1 + timeUs = 1927256 + flags = 1 + size = 4 + sample #130: + track = 0 + timeUs = 2000000 + size = 27 + sample #131: + track = 1 + timeUs = 1950476 + flags = 1 + size = 4 + sample #132: + track = 1 + timeUs = 1973696 + flags = 1 + size = 4 + sample #133: + track = 0 + timeUs = 1960000 + flags = 67108864 + size = 18 + sample #134: + track = 1 + timeUs = 1996916 + flags = 1 + size = 4 + sample #135: + track = 1 + timeUs = 2020136 + flags = 1 + size = 4 + sample #136: + track = 0 + timeUs = 2040000 + flags = 67108864 + size = 18 + sample #137: + track = 1 + timeUs = 2043356 + flags = 1 + size = 4 + sample #138: + track = 0 + timeUs = 2240000 + size = 28 + sample #139: + track = 1 + timeUs = 2066575 + flags = 1 + size = 4 + sample #140: + track = 1 + timeUs = 2089795 + flags = 1 + size = 4 + sample #141: + track = 0 + timeUs = 2160000 + size = 27 + sample #142: + track = 0 + timeUs = 0 + flags = 1 + size = 5252 + sample #143: + track = 0 + timeUs = 160000 + size = 44 + sample #144: + track = 1 + timeUs = 0 + flags = 1 + size = 21 + sample #145: + track = 0 + timeUs = 80000 + size = 25 + sample #146: + track = 1 + timeUs = 23219 + flags = 1 + size = 4 + sample #147: + track = 1 + timeUs = 46439 + flags = 1 + size = 4 + sample #148: + track = 0 + timeUs = 40000 + flags = 67108864 + size = 18 + sample #149: + track = 1 + timeUs = 69659 + flags = 1 + size = 4 + sample #150: + track = 1 + timeUs = 92879 + flags = 1 + size = 4 + sample #151: + track = 0 + timeUs = 120000 + flags = 67108864 + size = 18 + sample #152: + track = 1 + timeUs = 116099 + flags = 1 + size = 4 + sample #153: + track = 1 + timeUs = 139319 + flags = 1 + size = 4 + sample #154: + track = 0 + timeUs = 320000 + size = 32 + sample #155: + track = 1 + timeUs = 162539 + flags = 1 + size = 4 + sample #156: + track = 0 + timeUs = 240000 + size = 27 + sample #157: + track = 1 + timeUs = 185759 + flags = 1 + size = 4 + sample #158: + track = 1 + timeUs = 208979 + flags = 1 + size = 4 + sample #159: + track = 0 + timeUs = 200000 + flags = 67108864 + size = 18 + sample #160: + track = 1 + timeUs = 232199 + flags = 1 + size = 4 + sample #161: + track = 1 + timeUs = 255419 + flags = 1 + size = 4 + sample #162: + track = 0 + timeUs = 280000 + flags = 67108864 + size = 18 + sample #163: + track = 1 + timeUs = 278639 + flags = 1 + size = 4 + sample #164: + track = 1 + timeUs = 301859 + flags = 1 + size = 4 + sample #165: + track = 0 + timeUs = 480000 + size = 28 + sample #166: + track = 1 + timeUs = 325079 + flags = 1 + size = 4 + sample #167: + track = 0 + timeUs = 400000 + size = 27 + sample #168: + track = 1 + timeUs = 348299 + flags = 1 + size = 4 + sample #169: + track = 1 + timeUs = 371519 + flags = 1 + size = 4 + sample #170: + track = 0 + timeUs = 360000 + flags = 67108864 + size = 18 + sample #171: + track = 1 + timeUs = 394739 + flags = 1 + size = 4 + sample #172: + track = 1 + timeUs = 417959 + flags = 1 + size = 4 + sample #173: + track = 0 + timeUs = 440000 + flags = 67108864 + size = 18 + sample #174: + track = 1 + timeUs = 441179 + flags = 1 + size = 4 + sample #175: + track = 0 + timeUs = 640000 + size = 28 + sample #176: + track = 1 + timeUs = 464399 + flags = 1 + size = 4 + sample #177: + track = 1 + timeUs = 487619 + flags = 1 + size = 4 + sample #178: + track = 0 + timeUs = 560000 + size = 27 + sample #179: + track = 1 + timeUs = 510839 + flags = 1 + size = 4 + sample #180: + track = 1 + timeUs = 534058 + flags = 1 + size = 4 + sample #181: + track = 0 + timeUs = 520000 + flags = 67108864 + size = 18 + sample #182: + track = 1 + timeUs = 557278 + flags = 1 + size = 4 + sample #183: + track = 1 + timeUs = 580498 + flags = 1 + size = 4 + sample #184: + track = 0 + timeUs = 600000 + flags = 67108864 + size = 18 + sample #185: + track = 1 + timeUs = 603718 + flags = 1 + size = 4 + sample #186: + track = 0 + timeUs = 800000 + size = 28 + sample #187: + track = 1 + timeUs = 626938 + flags = 1 + size = 4 + sample #188: + track = 1 + timeUs = 650158 + flags = 1 + size = 4 + sample #189: + track = 0 + timeUs = 720000 + size = 27 + sample #190: + track = 1 + timeUs = 673378 + flags = 1 + size = 4 + sample #191: + track = 1 + timeUs = 696598 + flags = 1 + size = 4 + sample #192: + track = 0 + timeUs = 680000 + flags = 67108864 + size = 18 + sample #193: + track = 1 + timeUs = 719818 + flags = 1 + size = 4 + sample #194: + track = 1 + timeUs = 743038 + flags = 1 + size = 4 + sample #195: + track = 0 + timeUs = 760000 + flags = 67108864 + size = 18 + sample #196: + track = 1 + timeUs = 766258 + flags = 1 + size = 4 + sample #197: + track = 0 + timeUs = 960000 + size = 28 + sample #198: + track = 1 + timeUs = 789478 + flags = 1 + size = 4 + sample #199: + track = 1 + timeUs = 812698 + flags = 1 + size = 4 + sample #200: + track = 0 + timeUs = 880000 + size = 27 + sample #201: + track = 1 + timeUs = 835918 + flags = 1 + size = 4 + sample #202: + track = 1 + timeUs = 859138 + flags = 1 + size = 4 + sample #203: + track = 0 + timeUs = 840000 + flags = 67108864 + size = 18 + sample #204: + track = 1 + timeUs = 882358 + flags = 1 + size = 4 + sample #205: + track = 0 + timeUs = 920000 + flags = 67108864 + size = 18 + sample #206: + track = 1 + timeUs = 905578 + flags = 1 + size = 4 + sample #207: + track = 1 + timeUs = 928798 + flags = 1 + size = 4 + sample #208: + track = 0 + timeUs = 1120000 + size = 28 + sample #209: + track = 1 + timeUs = 952018 + flags = 1 + size = 4 + sample #210: + track = 1 + timeUs = 975238 + flags = 1 + size = 4 + sample #211: + track = 0 + timeUs = 1040000 + size = 27 + sample #212: + track = 1 + timeUs = 998458 + flags = 1 + size = 4 + sample #213: + track = 1 + timeUs = 1021678 + flags = 1 + size = 4 + sample #214: + track = 0 + timeUs = 1000000 + flags = 67108864 + size = 18 + sample #215: + track = 1 + timeUs = 1044897 + flags = 1 + size = 4 + sample #216: + track = 0 + timeUs = 1080000 + flags = 67108864 + size = 18 + sample #217: + track = 1 + timeUs = 1068117 + flags = 1 + size = 4 + sample #218: + track = 1 + timeUs = 1091337 + flags = 1 + size = 4 + sample #219: + track = 0 + timeUs = 1280000 + size = 28 + sample #220: + track = 1 + timeUs = 1114557 + flags = 1 + size = 4 + sample #221: + track = 1 + timeUs = 1137777 + flags = 1 + size = 4 + sample #222: + track = 0 + timeUs = 1200000 + size = 27 + sample #223: + track = 1 + timeUs = 1160997 + flags = 1 + size = 4 + sample #224: + track = 0 + timeUs = 1160000 + flags = 67108864 + size = 18 + sample #225: + track = 1 + timeUs = 1184217 + flags = 1 + size = 4 + sample #226: + track = 1 + timeUs = 1207437 + flags = 1 + size = 4 + sample #227: + track = 0 + timeUs = 1240000 + flags = 67108864 + size = 18 + sample #228: + track = 1 + timeUs = 1230657 + flags = 1 + size = 4 + sample #229: + track = 1 + timeUs = 1253877 + flags = 1 + size = 4 + sample #230: + track = 0 + timeUs = 1440000 + size = 28 + sample #231: + track = 1 + timeUs = 1277097 + flags = 1 + size = 4 + sample #232: + track = 1 + timeUs = 1300317 + flags = 1 + size = 4 + sample #233: + track = 0 + timeUs = 1360000 + size = 27 + sample #234: + track = 1 + timeUs = 1323537 + flags = 1 + size = 4 + sample #235: + track = 0 + timeUs = 1320000 + flags = 67108864 + size = 18 + sample #236: + track = 1 + timeUs = 1346757 + flags = 1 + size = 4 + sample #237: + track = 1 + timeUs = 1369977 + flags = 1 + size = 4 + sample #238: + track = 0 + timeUs = 1400000 + flags = 67108864 + size = 18 + sample #239: + track = 1 + timeUs = 1393197 + flags = 1 + size = 4 + sample #240: + track = 1 + timeUs = 1416417 + flags = 1 + size = 4 + sample #241: + track = 0 + timeUs = 1600000 + size = 28 + sample #242: + track = 1 + timeUs = 1439637 + flags = 1 + size = 4 + sample #243: + track = 1 + timeUs = 1462857 + flags = 1 + size = 4 + sample #244: + track = 0 + timeUs = 1520000 + size = 27 + sample #245: + track = 1 + timeUs = 1486077 + flags = 1 + size = 4 + sample #246: + track = 0 + timeUs = 1480000 + flags = 67108864 + size = 18 + sample #247: + track = 1 + timeUs = 1509297 + flags = 1 + size = 4 + sample #248: + track = 1 + timeUs = 1532517 + flags = 1 + size = 4 + sample #249: + track = 0 + timeUs = 1560000 + flags = 67108864 + size = 18 + sample #250: + track = 1 + timeUs = 1555736 + flags = 1 + size = 4 + sample #251: + track = 1 + timeUs = 1578956 + flags = 1 + size = 4 + sample #252: + track = 0 + timeUs = 1760000 + size = 28 + sample #253: + track = 1 + timeUs = 1602176 + flags = 1 + size = 4 + sample #254: + track = 0 + timeUs = 1680000 + size = 27 + sample #255: + track = 1 + timeUs = 1625396 + flags = 1 + size = 4 + sample #256: + track = 1 + timeUs = 1648616 + flags = 1 + size = 4 + sample #257: + track = 0 + timeUs = 1640000 + flags = 67108864 + size = 18 + sample #258: + track = 1 + timeUs = 1671836 + flags = 1 + size = 4 + sample #259: + track = 1 + timeUs = 1695056 + flags = 1 + size = 4 + sample #260: + track = 0 + timeUs = 1720000 + flags = 67108864 + size = 18 + sample #261: + track = 1 + timeUs = 1718276 + flags = 1 + size = 4 + sample #262: + track = 1 + timeUs = 1741496 + flags = 1 + size = 4 + sample #263: + track = 0 + timeUs = 1920000 + size = 28 + sample #264: + track = 1 + timeUs = 1764716 + flags = 1 + size = 4 + sample #265: + track = 0 + timeUs = 1840000 + size = 27 + sample #266: + track = 1 + timeUs = 1787936 + flags = 1 + size = 4 + sample #267: + track = 1 + timeUs = 1811156 + flags = 1 + size = 4 + sample #268: + track = 0 + timeUs = 1800000 + flags = 67108864 + size = 18 + sample #269: + track = 1 + timeUs = 1834376 + flags = 1 + size = 4 + sample #270: + track = 1 + timeUs = 1857596 + flags = 1 + size = 4 + sample #271: + track = 0 + timeUs = 1880000 + flags = 67108864 + size = 18 + sample #272: + track = 1 + timeUs = 1880816 + flags = 1 + size = 4 + sample #273: + track = 0 + timeUs = 2080000 + size = 28 + sample #274: + track = 1 + timeUs = 1904036 + flags = 1 + size = 4 + sample #275: + track = 1 + timeUs = 1927256 + flags = 1 + size = 4 + sample #276: + track = 0 + timeUs = 2000000 + size = 27 + sample #277: + track = 1 + timeUs = 1950476 + flags = 1 + size = 4 + sample #278: + track = 1 + timeUs = 1973696 + flags = 1 + size = 4 + sample #279: + track = 0 + timeUs = 1960000 + flags = 67108864 + size = 18 + sample #280: + track = 1 + timeUs = 1996916 + flags = 1 + size = 4 + sample #281: + track = 1 + timeUs = 2020136 + flags = 1 + size = 4 + sample #282: + track = 0 + timeUs = 2040000 + flags = 67108864 + size = 18 + sample #283: + track = 1 + timeUs = 2043356 + flags = 1 + size = 4 + sample #284: + track = 0 + timeUs = 2240000 + size = 28 + sample #285: + track = 1 + timeUs = 2066575 + flags = 1 + size = 4 + sample #286: + track = 1 + timeUs = 2089795 + flags = 1 + size = 4 + sample #287: + track = 0 + timeUs = 2160000 + size = 27 + sample #288: + track = 1 + timeUs = 2113015 + flags = 1 + size = 4 + sample #289: + track = 1 + timeUs = 2136235 + flags = 1 + size = 4 + sample #290: + track = 0 + timeUs = 2120000 + flags = 67108864 + size = 18 + sample #291: + track = 1 + timeUs = 2159455 + flags = 1 + size = 4 + sample #292: + track = 1 + timeUs = 2182675 + flags = 1 + size = 4 + sample #293: + track = 0 + timeUs = 2200000 + flags = 67108864 + size = 18 + sample #294: + track = 1 + timeUs = 2205895 + flags = 1 + size = 4 + sample #295: + track = 0 + timeUs = 2400000 + size = 28 + sample #296: + track = 1 + timeUs = 2229115 + flags = 1 + size = 4 + sample #297: + track = 1 + timeUs = 2252335 + flags = 1 + size = 4 + sample #298: + track = 0 + timeUs = 2320000 + size = 27 + sample #299: + track = 1 + timeUs = 2275555 + flags = 1 + size = 4 + sample #300: + track = 1 + timeUs = 2298775 + flags = 1 + size = 4 + sample #301: + track = 0 + timeUs = 2280000 + flags = 67108864 + size = 18 + sample #302: + track = 1 + timeUs = 2321995 + flags = 1 + size = 4 + sample #303: + track = 0 + timeUs = 2360000 + flags = 67108864 + size = 18 + sample #304: + track = 1 + timeUs = 2345215 + flags = 1 + size = 4 + sample #305: + track = 1 + timeUs = 2368435 + flags = 1 + size = 4 + sample #306: + track = 0 + timeUs = 2560000 + size = 28 + sample #307: + track = 1 + timeUs = 2391655 + flags = 1 + size = 4 + sample #308: + track = 1 + timeUs = 2414875 + flags = 1 + size = 4 + sample #309: + track = 0 + timeUs = 2480000 + size = 27 + sample #310: + track = 1 + timeUs = 2438095 + flags = 1 + size = 4 + sample #311: + track = 1 + timeUs = 2461315 + flags = 1 + size = 4 + sample #312: + track = 0 + timeUs = 2440000 + flags = 67108864 + size = 18 + sample #313: + track = 1 + timeUs = 2484535 + flags = 1 + size = 4 + sample #314: + track = 0 + timeUs = 2520000 + flags = 67108864 + size = 18 + sample #315: + track = 1 + timeUs = 2507755 + flags = 1 + size = 4 + sample #316: + track = 1 + timeUs = 2530975 + flags = 1 + size = 4 + sample #317: + track = 0 + timeUs = 2720000 + size = 28 + sample #318: + track = 1 + timeUs = 2554195 + flags = 1 + size = 4 + sample #319: + track = 1 + timeUs = 2577414 + flags = 1 + size = 4 + sample #320: + track = 0 + timeUs = 2640000 + size = 27 + sample #321: + track = 1 + timeUs = 2600634 + flags = 1 + size = 4 + sample #322: + track = 0 + timeUs = 2600000 + flags = 67108864 + size = 18 + sample #323: + track = 1 + timeUs = 2623854 + flags = 1 + size = 4 + sample #324: + track = 1 + timeUs = 2647074 + flags = 1 + size = 4 + sample #325: + track = 0 + timeUs = 2680000 + flags = 67108864 + size = 18 + sample #326: + track = 1 + timeUs = 2670294 + flags = 1 + size = 4 + sample #327: + track = 1 + timeUs = 2693514 + flags = 1 + size = 4 + sample #328: + track = 0 + timeUs = 2880000 + size = 28 + sample #329: + track = 1 + timeUs = 2716734 + flags = 1 + size = 4 + sample #330: + track = 1 + timeUs = 2739954 + flags = 1 + size = 4 + sample #331: + track = 0 + timeUs = 2800000 + size = 27 + sample #332: + track = 1 + timeUs = 2763174 + flags = 1 + size = 4 + sample #333: + track = 0 + timeUs = 2760000 + flags = 67108864 + size = 18 + sample #334: + track = 1 + timeUs = 2786394 + flags = 1 + size = 4 + sample #335: + track = 1 + timeUs = 2809614 + flags = 1 + size = 4 + sample #336: + track = 0 + timeUs = 2840000 + flags = 67108864 + size = 18 + sample #337: + track = 1 + timeUs = 2832834 + flags = 1 + size = 4 + sample #338: + track = 1 + timeUs = 2856054 + flags = 1 + size = 4 + sample #339: + track = 0 + timeUs = 3040000 + size = 28 + sample #340: + track = 1 + timeUs = 2879274 + flags = 1 + size = 4 + sample #341: + track = 1 + timeUs = 2902494 + flags = 1 + size = 4 + sample #342: + track = 0 + timeUs = 2960000 + size = 27 + sample #343: + track = 1 + timeUs = 2925714 + flags = 1 + size = 4 + sample #344: + track = 0 + timeUs = 2920000 + flags = 67108864 + size = 18 + sample #345: + track = 1 + timeUs = 2948934 + flags = 1 + size = 4 + sample #346: + track = 1 + timeUs = 2972154 + flags = 1 + size = 4 + sample #347: + track = 0 + timeUs = 3000000 + flags = 67108864 + size = 18 + sample #348: + track = 1 + timeUs = 2995374 + flags = 1 + size = 4 + sample #349: + track = 1 + timeUs = 3018594 + flags = 1 + size = 4 + sample #350: + track = 0 + timeUs = 3200000 + size = 28 + sample #351: + track = 1 + timeUs = 3041814 + flags = 1 + size = 4 + sample #352: + track = 0 + timeUs = 3120000 + size = 27 + sample #353: + track = 1 + timeUs = 3065034 + flags = 1 + size = 4 + sample #354: + track = 1 + timeUs = 3088253 + flags = 1 + size = 4 + sample #355: + track = 0 + timeUs = 3080000 + flags = 67108864 + size = 18 + sample #356: + track = 1 + timeUs = 3111473 + flags = 1 + size = 4 + sample #357: + track = 1 + timeUs = 3134693 + flags = 1 + size = 4 + sample #358: + track = 0 + timeUs = 3160000 + flags = 67108864 + size = 18 + sample #359: + track = 1 + timeUs = 3157913 + flags = 1 + size = 4 + sample #360: + track = 1 + timeUs = 3181133 + flags = 1 + size = 4 + sample #361: + track = 0 + timeUs = 3360000 + size = 28 + sample #362: + track = 1 + timeUs = 3204353 + flags = 1 + size = 4 + sample #363: + track = 0 + timeUs = 3280000 + size = 27 + sample #364: + track = 1 + timeUs = 3227573 + flags = 1 + size = 4 + sample #365: + track = 1 + timeUs = 3250793 + flags = 1 + size = 4 + sample #366: + track = 0 + timeUs = 3240000 + flags = 67108864 + size = 18 + sample #367: + track = 1 + timeUs = 3274013 + flags = 1 + size = 4 + sample #368: + track = 1 + timeUs = 3297233 + flags = 1 + size = 4 + sample #369: + track = 0 + timeUs = 3320000 + flags = 67108864 + size = 18 + sample #370: + track = 1 + timeUs = 3320453 + flags = 1 + size = 4 + sample #371: + track = 0 + timeUs = 3520000 + size = 28 + sample #372: + track = 1 + timeUs = 3343673 + flags = 1 + size = 4 + sample #373: + track = 1 + timeUs = 3366893 + flags = 1 + size = 4 + sample #374: + track = 0 + timeUs = 3440000 + size = 27 + sample #375: + track = 1 + timeUs = 3390113 + flags = 1 + size = 4 + sample #376: + track = 1 + timeUs = 3413333 + flags = 1 + size = 4 + sample #377: + track = 0 + timeUs = 3400000 + flags = 67108864 + size = 18 + sample #378: + track = 1 + timeUs = 3436553 + flags = 1 + size = 4 + sample #379: + track = 1 + timeUs = 3459773 + flags = 1 + size = 4 + sample #380: + track = 0 + timeUs = 3480000 + flags = 67108864 + size = 18 + sample #381: + track = 1 + timeUs = 3482993 + flags = 1 + size = 4 + sample #382: + track = 0 + timeUs = 3680000 + size = 28 + sample #383: + track = 1 + timeUs = 3506213 + flags = 1 + size = 4 + sample #384: + track = 1 + timeUs = 3529433 + flags = 1 + size = 4 + sample #385: + track = 0 + timeUs = 3600000 + size = 27 + sample #386: + track = 1 + timeUs = 3552653 + flags = 1 + size = 4 + sample #387: + track = 1 + timeUs = 3575873 + flags = 1 + size = 4 + sample #388: + track = 0 + timeUs = 3560000 + flags = 67108864 + size = 18 + sample #389: + track = 1 + timeUs = 3599092 + flags = 1 + size = 4 + sample #390: + track = 1 + timeUs = 3622312 + flags = 1 + size = 4 + sample #391: + track = 0 + timeUs = 3640000 + flags = 67108864 + size = 18 + sample #392: + track = 1 + timeUs = 3645532 + flags = 1 + size = 4 + sample #393: + track = 0 + timeUs = 3840000 + size = 33 + sample #394: + track = 1 + timeUs = 3668752 + flags = 1 + size = 4 + sample #395: + track = 1 + timeUs = 3691972 + flags = 1 + size = 4 + sample #396: + track = 0 + timeUs = 3760000 + size = 27 + sample #397: + track = 1 + timeUs = 3715192 + flags = 1 + size = 4 + sample #398: + track = 1 + timeUs = 3738412 + flags = 1 + size = 4 + sample #399: + track = 0 + timeUs = 3720000 + flags = 67108864 + size = 18 + sample #400: + track = 1 + timeUs = 3761632 + flags = 1 + size = 4 + sample #401: + track = 0 + timeUs = 3800000 + flags = 67108864 + size = 18 + sample #402: + track = 1 + timeUs = 3784852 + flags = 1 + size = 4 + sample #403: + track = 1 + timeUs = 3808072 + flags = 1 + size = 4 + sample #404: + track = 0 + timeUs = 4000000 + size = 33 + sample #405: + track = 1 + timeUs = 3831292 + flags = 1 + size = 4 + sample #406: + track = 1 + timeUs = 3854512 + flags = 1 + size = 4 + sample #407: + track = 0 + timeUs = 3920000 + size = 27 + sample #408: + track = 1 + timeUs = 3877732 + flags = 1 + size = 4 + sample #409: + track = 1 + timeUs = 3900952 + flags = 1 + size = 4 + sample #410: + track = 0 + timeUs = 3880000 + flags = 67108864 + size = 18 + sample #411: + track = 1 + timeUs = 3924172 + flags = 1 + size = 4 + sample #412: + track = 0 + timeUs = 3960000 + flags = 67108864 + size = 18 + sample #413: + track = 1 + timeUs = 3947392 + flags = 1 + size = 4 + sample #414: + track = 1 + timeUs = 3970612 + flags = 1 + size = 4 + sample #415: + track = 0 + timeUs = 4160000 + size = 28 + sample #416: + track = 1 + timeUs = 3993832 + flags = 1 + size = 4 + sample #417: + track = 1 + timeUs = 4017052 + flags = 1 + size = 4 + sample #418: + track = 0 + timeUs = 4080000 + size = 27 + sample #419: + track = 1 + timeUs = 4040272 + flags = 1 + size = 4 + sample #420: + track = 0 + timeUs = 4040000 + flags = 67108864 + size = 18 + sample #421: + track = 1 + timeUs = 4063492 + flags = 1 + size = 4 + sample #422: + track = 1 + timeUs = 4086712 + flags = 1 + size = 4 + sample #423: + track = 0 + timeUs = 4120000 + flags = 67108864 + size = 18 + sample #424: + track = 1 + timeUs = 4109931 + flags = 1 + size = 4 + sample #425: + track = 1 + timeUs = 4133151 + flags = 1 + size = 4 + sample #426: + track = 0 + timeUs = 4320000 + size = 28 + sample #427: + track = 1 + timeUs = 4156371 + flags = 1 + size = 4 + sample #428: + track = 1 + timeUs = 4179591 + flags = 1 + size = 4 + sample #429: + track = 0 + timeUs = 4240000 + size = 27 + sample #430: + track = 1 + timeUs = 4202811 + flags = 1 + size = 4 + sample #431: + track = 0 + timeUs = 4200000 + flags = 67108864 + size = 18 + sample #432: + track = 1 + timeUs = 4226031 + flags = 1 + size = 4 + sample #433: + track = 1 + timeUs = 4249251 + flags = 1 + size = 4 + sample #434: + track = 0 + timeUs = 4280000 + flags = 67108864 + size = 18 + sample #435: + track = 1 + timeUs = 4272471 + flags = 1 + size = 4 + sample #436: + track = 1 + timeUs = 4295691 + flags = 1 + size = 4 + sample #437: + track = 0 + timeUs = 4480000 + size = 28 + sample #438: + track = 1 + timeUs = 4318911 + flags = 1 + size = 4 + sample #439: + track = 1 + timeUs = 4342131 + flags = 1 + size = 4 + sample #440: + track = 0 + timeUs = 4400000 + size = 27 + sample #441: + track = 1 + timeUs = 4365351 + flags = 1 + size = 4 + sample #442: + track = 0 + timeUs = 4360000 + flags = 67108864 + size = 18 + sample #443: + track = 1 + timeUs = 4388571 + flags = 1 + size = 4 + sample #444: + track = 1 + timeUs = 4411791 + flags = 1 + size = 4 + sample #445: + track = 0 + timeUs = 4440000 + flags = 67108864 + size = 18 + sample #446: + track = 1 + timeUs = 4435011 + flags = 1 + size = 4 + sample #447: + track = 1 + timeUs = 4458231 + flags = 1 + size = 4 + sample #448: + track = 0 + timeUs = 4640000 + size = 27 + sample #449: + track = 1 + timeUs = 4481451 + flags = 1 + size = 4 + sample #450: + track = 0 + timeUs = 4560000 + size = 27 + sample #451: + track = 1 + timeUs = 4504671 + flags = 1 + size = 4 + sample #452: + track = 1 + timeUs = 4527891 + flags = 1 + size = 4 + sample #453: + track = 0 + timeUs = 4520000 + flags = 67108864 + size = 18 + sample #454: + track = 1 + timeUs = 4551111 + flags = 1 + size = 4 + sample #455: + track = 1 + timeUs = 4574331 + flags = 1 + size = 4 + sample #456: + track = 0 + timeUs = 4600000 + flags = 67108864 + size = 18 + sample #457: + track = 1 + timeUs = 4597551 + flags = 1 + size = 4 + sample #458: + track = 1 + timeUs = 4620770 + flags = 1 + size = 4 + sample #459: + track = 0 + timeUs = 4800000 + size = 26 + sample #460: + track = 1 + timeUs = 4643990 + flags = 1 + size = 4 + sample #461: + track = 0 + timeUs = 4720000 + size = 27 + sample #462: + track = 1 + timeUs = 4667210 + flags = 1 + size = 4 + sample #463: + track = 1 + timeUs = 4690430 + flags = 1 + size = 4 + sample #464: + track = 0 + timeUs = 4680000 + flags = 67108864 + size = 18 + sample #465: + track = 1 + timeUs = 4713650 + flags = 1 + size = 4 + sample #466: + track = 1 + timeUs = 4736870 + flags = 1 + size = 4 + sample #467: + track = 0 + timeUs = 4760000 + flags = 67108864 + size = 18 + sample #468: + track = 1 + timeUs = 4760090 + flags = 1 + size = 4 + sample #469: + track = 0 + timeUs = 4960000 + size = 26 + sample #470: + track = 1 + timeUs = 4783310 + flags = 1 + size = 4 + sample #471: + track = 1 + timeUs = 4806530 + flags = 1 + size = 4 + sample #472: + track = 0 + timeUs = 4880000 + size = 27 + sample #473: + track = 1 + timeUs = 4829750 + flags = 1 + size = 4 + sample #474: + track = 1 + timeUs = 4852970 + flags = 1 + size = 4 + sample #475: + track = 0 + timeUs = 4840000 + flags = 67108864 + size = 18 + sample #476: + track = 1 + timeUs = 4876190 + flags = 1 + size = 4 + sample #477: + track = 1 + timeUs = 4899410 + flags = 1 + size = 4 + sample #478: + track = 0 + timeUs = 4920000 + flags = 603979776 + size = 18 + sample #479: + track = 1 + timeUs = 4922630 + flags = 1 + size = 4 + sample #480: + track = 1 + timeUs = 4945850 + flags = 1 + size = 4 + sample #481: + track = 1 + timeUs = 4969070 + flags = 1 + size = 4 + sample #482: + track = 1 + timeUs = 4992290 + flags = 1 + size = 4 + sample #483: + track = 1 + timeUs = 5015510 + flags = 536870913 + size = 4 +MediaCodecAdapter (media3.audio.aac): + inputBuffers: + count = 213 + input buffer #0: + timeUs = 1000000116099 + contents = length 4, hash EE9DF + input buffer #1: + timeUs = 1000000139319 + contents = length 4, hash EE9DF + input buffer #2: + timeUs = 1000000162539 + contents = length 4, hash EE9DF + input buffer #3: + timeUs = 1000000185759 + contents = length 4, hash EE9DF + input buffer #4: + timeUs = 1000000208979 + contents = length 4, hash EE9DF + input buffer #5: + timeUs = 1000000232199 + contents = length 4, hash EE9DF + input buffer #6: + timeUs = 1000000255419 + contents = length 4, hash EE9DF + input buffer #7: + timeUs = 1000000278639 + contents = length 4, hash EE9DF + input buffer #8: + timeUs = 1000000301859 + contents = length 4, hash EE9DF + input buffer #9: + timeUs = 1000000325079 + contents = length 4, hash EE9DF + input buffer #10: + timeUs = 1000000348299 + contents = length 4, hash EE9DF + input buffer #11: + timeUs = 1000000371519 + contents = length 4, hash EE9DF + input buffer #12: + timeUs = 1000000394739 + contents = length 4, hash EE9DF + input buffer #13: + timeUs = 1000000417959 + contents = length 4, hash EE9DF + input buffer #14: + timeUs = 1000000441179 + contents = length 4, hash EE9DF + input buffer #15: + timeUs = 1000000464399 + contents = length 4, hash EE9DF + input buffer #16: + timeUs = 1000000487619 + contents = length 4, hash EE9DF + input buffer #17: + timeUs = 1000000510839 + contents = length 4, hash EE9DF + input buffer #18: + timeUs = 1000000534058 + contents = length 4, hash EE9DF + input buffer #19: + timeUs = 1000000557278 + contents = length 4, hash EE9DF + input buffer #20: + timeUs = 1000000580498 + contents = length 4, hash EE9DF + input buffer #21: + timeUs = 1000000603718 + contents = length 4, hash EE9DF + input buffer #22: + timeUs = 1000000626938 + contents = length 4, hash EE9DF + input buffer #23: + timeUs = 1000000650158 + contents = length 4, hash EE9DF + input buffer #24: + timeUs = 1000000673378 + contents = length 4, hash EE9DF + input buffer #25: + timeUs = 1000000696598 + contents = length 4, hash EE9DF + input buffer #26: + timeUs = 1000000719818 + contents = length 4, hash EE9DF + input buffer #27: + timeUs = 1000000743038 + contents = length 4, hash EE9DF + input buffer #28: + timeUs = 1000000766258 + contents = length 4, hash EE9DF + input buffer #29: + timeUs = 1000000789478 + contents = length 4, hash EE9DF + input buffer #30: + timeUs = 1000000812698 + contents = length 4, hash EE9DF + input buffer #31: + timeUs = 1000000835918 + contents = length 4, hash EE9DF + input buffer #32: + timeUs = 1000000859138 + contents = length 4, hash EE9DF + input buffer #33: + timeUs = 1000000882358 + contents = length 4, hash EE9DF + input buffer #34: + timeUs = 1000000905578 + contents = length 4, hash EE9DF + input buffer #35: + timeUs = 1000000928798 + contents = length 4, hash EE9DF + input buffer #36: + timeUs = 1000000952018 + contents = length 4, hash EE9DF + input buffer #37: + timeUs = 1000000975238 + contents = length 4, hash EE9DF + input buffer #38: + timeUs = 1000000998458 + contents = length 4, hash EE9DF + input buffer #39: + timeUs = 1000001021678 + contents = length 4, hash EE9DF + input buffer #40: + timeUs = 1000001044897 + contents = length 4, hash EE9DF + input buffer #41: + timeUs = 1000001068117 + contents = length 4, hash EE9DF + input buffer #42: + timeUs = 1000001091337 + contents = length 4, hash EE9DF + input buffer #43: + timeUs = 1000001114557 + contents = length 4, hash EE9DF + input buffer #44: + timeUs = 1000001137777 + contents = length 4, hash EE9DF + input buffer #45: + timeUs = 1000001160997 + contents = length 4, hash EE9DF + input buffer #46: + timeUs = 1000001184217 + contents = length 4, hash EE9DF + input buffer #47: + timeUs = 1000001207437 + contents = length 4, hash EE9DF + input buffer #48: + timeUs = 1000001230657 + contents = length 4, hash EE9DF + input buffer #49: + timeUs = 1000001253877 + contents = length 4, hash EE9DF + input buffer #50: + timeUs = 1000001277097 + contents = length 4, hash EE9DF + input buffer #51: + timeUs = 1000001300317 + contents = length 4, hash EE9DF + input buffer #52: + timeUs = 1000001323537 + contents = length 4, hash EE9DF + input buffer #53: + timeUs = 1000001346757 + contents = length 4, hash EE9DF + input buffer #54: + timeUs = 1000001369977 + contents = length 4, hash EE9DF + input buffer #55: + timeUs = 1000001393197 + contents = length 4, hash EE9DF + input buffer #56: + timeUs = 1000001416417 + contents = length 4, hash EE9DF + input buffer #57: + timeUs = 1000001439637 + contents = length 4, hash EE9DF + input buffer #58: + timeUs = 1000001462857 + contents = length 4, hash EE9DF + input buffer #59: + timeUs = 1000001486077 + contents = length 4, hash EE9DF + input buffer #60: + timeUs = 1000001509297 + contents = length 4, hash EE9DF + input buffer #61: + timeUs = 1000001532517 + contents = length 4, hash EE9DF + input buffer #62: + timeUs = 1000001555736 + contents = length 4, hash EE9DF + input buffer #63: + timeUs = 1000001578956 + contents = length 4, hash EE9DF + input buffer #64: + timeUs = 1000001602176 + contents = length 4, hash EE9DF + input buffer #65: + timeUs = 1000001625396 + contents = length 4, hash EE9DF + input buffer #66: + timeUs = 1000001648616 + contents = length 4, hash EE9DF + input buffer #67: + timeUs = 1000001671836 + contents = length 4, hash EE9DF + input buffer #68: + timeUs = 1000001695056 + contents = length 4, hash EE9DF + input buffer #69: + timeUs = 1000001718276 + contents = length 4, hash EE9DF + input buffer #70: + timeUs = 1000001741496 + contents = length 4, hash EE9DF + input buffer #71: + timeUs = 1000001764716 + contents = length 4, hash EE9DF + input buffer #72: + timeUs = 1000001787936 + contents = length 4, hash EE9DF + input buffer #73: + timeUs = 1000001811156 + contents = length 4, hash EE9DF + input buffer #74: + timeUs = 1000001834376 + contents = length 4, hash EE9DF + input buffer #75: + timeUs = 1000001857596 + contents = length 4, hash EE9DF + input buffer #76: + timeUs = 1000001880816 + contents = length 4, hash EE9DF + input buffer #77: + timeUs = 1000001904036 + contents = length 4, hash EE9DF + input buffer #78: + timeUs = 1000001927256 + contents = length 4, hash EE9DF + input buffer #79: + timeUs = 1000001950476 + contents = length 4, hash EE9DF + input buffer #80: + timeUs = 1000001973696 + contents = length 4, hash EE9DF + input buffer #81: + timeUs = 1000001996916 + contents = length 4, hash EE9DF + input buffer #82: + timeUs = 1000002020136 + contents = length 4, hash EE9DF + input buffer #83: + timeUs = 1000002043356 + contents = length 4, hash EE9DF + input buffer #84: + timeUs = 1000002066575 + contents = length 4, hash EE9DF + input buffer #85: + timeUs = 1000002089795 + contents = length 4, hash EE9DF + input buffer #86: + timeUs = 1000002113015 + contents = length 4, hash EE9DF + input buffer #87: + timeUs = 1000002136235 + contents = length 4, hash EE9DF + input buffer #88: + timeUs = 1000002159455 + contents = length 4, hash EE9DF + input buffer #89: + timeUs = 1000002182675 + contents = length 4, hash EE9DF + input buffer #90: + timeUs = 1000002205895 + contents = length 4, hash EE9DF + input buffer #91: + timeUs = 1000002229115 + contents = length 4, hash EE9DF + input buffer #92: + timeUs = 1000002252335 + contents = length 4, hash EE9DF + input buffer #93: + timeUs = 1000002275555 + contents = length 4, hash EE9DF + input buffer #94: + timeUs = 1000002298775 + contents = length 4, hash EE9DF + input buffer #95: + timeUs = 1000002321995 + contents = length 4, hash EE9DF + input buffer #96: + timeUs = 1000002345215 + contents = length 4, hash EE9DF + input buffer #97: + timeUs = 1000002368435 + contents = length 4, hash EE9DF + input buffer #98: + timeUs = 1000002391655 + contents = length 4, hash EE9DF + input buffer #99: + timeUs = 1000002414875 + contents = length 4, hash EE9DF + input buffer #100: + timeUs = 1000002438095 + contents = length 4, hash EE9DF + input buffer #101: + timeUs = 1000002461315 + contents = length 4, hash EE9DF + input buffer #102: + timeUs = 1000002484535 + contents = length 4, hash EE9DF + input buffer #103: + timeUs = 1000002507755 + contents = length 4, hash EE9DF + input buffer #104: + timeUs = 1000002530975 + contents = length 4, hash EE9DF + input buffer #105: + timeUs = 1000002554195 + contents = length 4, hash EE9DF + input buffer #106: + timeUs = 1000002577414 + contents = length 4, hash EE9DF + input buffer #107: + timeUs = 1000002600634 + contents = length 4, hash EE9DF + input buffer #108: + timeUs = 1000002623854 + contents = length 4, hash EE9DF + input buffer #109: + timeUs = 1000002647074 + contents = length 4, hash EE9DF + input buffer #110: + timeUs = 1000002670294 + contents = length 4, hash EE9DF + input buffer #111: + timeUs = 1000002693514 + contents = length 4, hash EE9DF + input buffer #112: + timeUs = 1000002716734 + contents = length 4, hash EE9DF + input buffer #113: + timeUs = 1000002739954 + contents = length 4, hash EE9DF + input buffer #114: + timeUs = 1000002763174 + contents = length 4, hash EE9DF + input buffer #115: + timeUs = 1000002786394 + contents = length 4, hash EE9DF + input buffer #116: + timeUs = 1000002809614 + contents = length 4, hash EE9DF + input buffer #117: + timeUs = 1000002832834 + contents = length 4, hash EE9DF + input buffer #118: + timeUs = 1000002856054 + contents = length 4, hash EE9DF + input buffer #119: + timeUs = 1000002879274 + contents = length 4, hash EE9DF + input buffer #120: + timeUs = 1000002902494 + contents = length 4, hash EE9DF + input buffer #121: + timeUs = 1000002925714 + contents = length 4, hash EE9DF + input buffer #122: + timeUs = 1000002948934 + contents = length 4, hash EE9DF + input buffer #123: + timeUs = 1000002972154 + contents = length 4, hash EE9DF + input buffer #124: + timeUs = 1000002995374 + contents = length 4, hash EE9DF + input buffer #125: + timeUs = 1000003018594 + contents = length 4, hash EE9DF + input buffer #126: + timeUs = 1000003041814 + contents = length 4, hash EE9DF + input buffer #127: + timeUs = 1000003065034 + contents = length 4, hash EE9DF + input buffer #128: + timeUs = 1000003088253 + contents = length 4, hash EE9DF + input buffer #129: + timeUs = 1000003111473 + contents = length 4, hash EE9DF + input buffer #130: + timeUs = 1000003134693 + contents = length 4, hash EE9DF + input buffer #131: + timeUs = 1000003157913 + contents = length 4, hash EE9DF + input buffer #132: + timeUs = 1000003181133 + contents = length 4, hash EE9DF + input buffer #133: + timeUs = 1000003204353 + contents = length 4, hash EE9DF + input buffer #134: + timeUs = 1000003227573 + contents = length 4, hash EE9DF + input buffer #135: + timeUs = 1000003250793 + contents = length 4, hash EE9DF + input buffer #136: + timeUs = 1000003274013 + contents = length 4, hash EE9DF + input buffer #137: + timeUs = 1000003297233 + contents = length 4, hash EE9DF + input buffer #138: + timeUs = 1000003320453 + contents = length 4, hash EE9DF + input buffer #139: + timeUs = 1000003343673 + contents = length 4, hash EE9DF + input buffer #140: + timeUs = 1000003366893 + contents = length 4, hash EE9DF + input buffer #141: + timeUs = 1000003390113 + contents = length 4, hash EE9DF + input buffer #142: + timeUs = 1000003413333 + contents = length 4, hash EE9DF + input buffer #143: + timeUs = 1000003436553 + contents = length 4, hash EE9DF + input buffer #144: + timeUs = 1000003459773 + contents = length 4, hash EE9DF + input buffer #145: + timeUs = 1000003482993 + contents = length 4, hash EE9DF + input buffer #146: + timeUs = 1000003506213 + contents = length 4, hash EE9DF + input buffer #147: + timeUs = 1000003529433 + contents = length 4, hash EE9DF + input buffer #148: + timeUs = 1000003552653 + contents = length 4, hash EE9DF + input buffer #149: + timeUs = 1000003575873 + contents = length 4, hash EE9DF + input buffer #150: + timeUs = 1000003599092 + contents = length 4, hash EE9DF + input buffer #151: + timeUs = 1000003622312 + contents = length 4, hash EE9DF + input buffer #152: + timeUs = 1000003645532 + contents = length 4, hash EE9DF + input buffer #153: + timeUs = 1000003668752 + contents = length 4, hash EE9DF + input buffer #154: + timeUs = 1000003691972 + contents = length 4, hash EE9DF + input buffer #155: + timeUs = 1000003715192 + contents = length 4, hash EE9DF + input buffer #156: + timeUs = 1000003738412 + contents = length 4, hash EE9DF + input buffer #157: + timeUs = 1000003761632 + contents = length 4, hash EE9DF + input buffer #158: + timeUs = 1000003784852 + contents = length 4, hash EE9DF + input buffer #159: + timeUs = 1000003808072 + contents = length 4, hash EE9DF + input buffer #160: + timeUs = 1000003831292 + contents = length 4, hash EE9DF + input buffer #161: + timeUs = 1000003854512 + contents = length 4, hash EE9DF + input buffer #162: + timeUs = 1000003877732 + contents = length 4, hash EE9DF + input buffer #163: + timeUs = 1000003900952 + contents = length 4, hash EE9DF + input buffer #164: + timeUs = 1000003924172 + contents = length 4, hash EE9DF + input buffer #165: + timeUs = 1000003947392 + contents = length 4, hash EE9DF + input buffer #166: + timeUs = 1000003970612 + contents = length 4, hash EE9DF + input buffer #167: + timeUs = 1000003993832 + contents = length 4, hash EE9DF + input buffer #168: + timeUs = 1000004017052 + contents = length 4, hash EE9DF + input buffer #169: + timeUs = 1000004040272 + contents = length 4, hash EE9DF + input buffer #170: + timeUs = 1000004063492 + contents = length 4, hash EE9DF + input buffer #171: + timeUs = 1000004086712 + contents = length 4, hash EE9DF + input buffer #172: + timeUs = 1000004109931 + contents = length 4, hash EE9DF + input buffer #173: + timeUs = 1000004133151 + contents = length 4, hash EE9DF + input buffer #174: + timeUs = 1000004156371 + contents = length 4, hash EE9DF + input buffer #175: + timeUs = 1000004179591 + contents = length 4, hash EE9DF + input buffer #176: + timeUs = 1000004202811 + contents = length 4, hash EE9DF + input buffer #177: + timeUs = 1000004226031 + contents = length 4, hash EE9DF + input buffer #178: + timeUs = 1000004249251 + contents = length 4, hash EE9DF + input buffer #179: + timeUs = 1000004272471 + contents = length 4, hash EE9DF + input buffer #180: + timeUs = 1000004295691 + contents = length 4, hash EE9DF + input buffer #181: + timeUs = 1000004318911 + contents = length 4, hash EE9DF + input buffer #182: + timeUs = 1000004342131 + contents = length 4, hash EE9DF + input buffer #183: + timeUs = 1000004365351 + contents = length 4, hash EE9DF + input buffer #184: + timeUs = 1000004388571 + contents = length 4, hash EE9DF + input buffer #185: + timeUs = 1000004411791 + contents = length 4, hash EE9DF + input buffer #186: + timeUs = 1000004435011 + contents = length 4, hash EE9DF + input buffer #187: + timeUs = 1000004458231 + contents = length 4, hash EE9DF + input buffer #188: + timeUs = 1000004481451 + contents = length 4, hash EE9DF + input buffer #189: + timeUs = 1000004504671 + contents = length 4, hash EE9DF + input buffer #190: + timeUs = 1000004527891 + contents = length 4, hash EE9DF + input buffer #191: + timeUs = 1000004551111 + contents = length 4, hash EE9DF + input buffer #192: + timeUs = 1000004574331 + contents = length 4, hash EE9DF + input buffer #193: + timeUs = 1000004597551 + contents = length 4, hash EE9DF + input buffer #194: + timeUs = 1000004620770 + contents = length 4, hash EE9DF + input buffer #195: + timeUs = 1000004643990 + contents = length 4, hash EE9DF + input buffer #196: + timeUs = 1000004667210 + contents = length 4, hash EE9DF + input buffer #197: + timeUs = 1000004690430 + contents = length 4, hash EE9DF + input buffer #198: + timeUs = 1000004713650 + contents = length 4, hash EE9DF + input buffer #199: + timeUs = 1000004736870 + contents = length 4, hash EE9DF + input buffer #200: + timeUs = 1000004760090 + contents = length 4, hash EE9DF + input buffer #201: + timeUs = 1000004783310 + contents = length 4, hash EE9DF + input buffer #202: + timeUs = 1000004806530 + contents = length 4, hash EE9DF + input buffer #203: + timeUs = 1000004829750 + contents = length 4, hash EE9DF + input buffer #204: + timeUs = 1000004852970 + contents = length 4, hash EE9DF + input buffer #205: + timeUs = 1000004876190 + contents = length 4, hash EE9DF + input buffer #206: + timeUs = 1000004899410 + contents = length 4, hash EE9DF + input buffer #207: + timeUs = 1000004922630 + contents = length 4, hash EE9DF + input buffer #208: + timeUs = 1000004945850 + contents = length 4, hash EE9DF + input buffer #209: + timeUs = 1000004969070 + contents = length 4, hash EE9DF + input buffer #210: + timeUs = 1000004992290 + contents = length 4, hash EE9DF + input buffer #211: + timeUs = 1000005015510 + contents = length 4, hash EE9DF + input buffer #212: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 212 + output buffer #0: + timeUs = 1000000116099 + size = 0 + rendered = false + output buffer #1: + timeUs = 1000000139319 + size = 0 + rendered = false + output buffer #2: + timeUs = 1000000162539 + size = 0 + rendered = false + output buffer #3: + timeUs = 1000000185759 + size = 0 + rendered = false + output buffer #4: + timeUs = 1000000208979 + size = 0 + rendered = false + output buffer #5: + timeUs = 1000000232199 + size = 0 + rendered = false + output buffer #6: + timeUs = 1000000255419 + size = 0 + rendered = false + output buffer #7: + timeUs = 1000000278639 + size = 0 + rendered = false + output buffer #8: + timeUs = 1000000301859 + size = 0 + rendered = false + output buffer #9: + timeUs = 1000000325079 + size = 0 + rendered = false + output buffer #10: + timeUs = 1000000348299 + size = 0 + rendered = false + output buffer #11: + timeUs = 1000000371519 + size = 0 + rendered = false + output buffer #12: + timeUs = 1000000394739 + size = 0 + rendered = false + output buffer #13: + timeUs = 1000000417959 + size = 0 + rendered = false + output buffer #14: + timeUs = 1000000441179 + size = 0 + rendered = false + output buffer #15: + timeUs = 1000000464399 + size = 0 + rendered = false + output buffer #16: + timeUs = 1000000487619 + size = 0 + rendered = false + output buffer #17: + timeUs = 1000000510839 + size = 0 + rendered = false + output buffer #18: + timeUs = 1000000534058 + size = 0 + rendered = false + output buffer #19: + timeUs = 1000000557278 + size = 0 + rendered = false + output buffer #20: + timeUs = 1000000580498 + size = 0 + rendered = false + output buffer #21: + timeUs = 1000000603718 + size = 0 + rendered = false + output buffer #22: + timeUs = 1000000626938 + size = 0 + rendered = false + output buffer #23: + timeUs = 1000000650158 + size = 0 + rendered = false + output buffer #24: + timeUs = 1000000673378 + size = 0 + rendered = false + output buffer #25: + timeUs = 1000000696598 + size = 0 + rendered = false + output buffer #26: + timeUs = 1000000719818 + size = 0 + rendered = false + output buffer #27: + timeUs = 1000000743038 + size = 0 + rendered = false + output buffer #28: + timeUs = 1000000766258 + size = 0 + rendered = false + output buffer #29: + timeUs = 1000000789478 + size = 0 + rendered = false + output buffer #30: + timeUs = 1000000812698 + size = 0 + rendered = false + output buffer #31: + timeUs = 1000000835918 + size = 0 + rendered = false + output buffer #32: + timeUs = 1000000859138 + size = 0 + rendered = false + output buffer #33: + timeUs = 1000000882358 + size = 0 + rendered = false + output buffer #34: + timeUs = 1000000905578 + size = 0 + rendered = false + output buffer #35: + timeUs = 1000000928798 + size = 0 + rendered = false + output buffer #36: + timeUs = 1000000952018 + size = 0 + rendered = false + output buffer #37: + timeUs = 1000000975238 + size = 0 + rendered = false + output buffer #38: + timeUs = 1000000998458 + size = 0 + rendered = false + output buffer #39: + timeUs = 1000001021678 + size = 0 + rendered = false + output buffer #40: + timeUs = 1000001044897 + size = 0 + rendered = false + output buffer #41: + timeUs = 1000001068117 + size = 0 + rendered = false + output buffer #42: + timeUs = 1000001091337 + size = 0 + rendered = false + output buffer #43: + timeUs = 1000001114557 + size = 0 + rendered = false + output buffer #44: + timeUs = 1000001137777 + size = 0 + rendered = false + output buffer #45: + timeUs = 1000001160997 + size = 0 + rendered = false + output buffer #46: + timeUs = 1000001184217 + size = 0 + rendered = false + output buffer #47: + timeUs = 1000001207437 + size = 0 + rendered = false + output buffer #48: + timeUs = 1000001230657 + size = 0 + rendered = false + output buffer #49: + timeUs = 1000001253877 + size = 0 + rendered = false + output buffer #50: + timeUs = 1000001277097 + size = 0 + rendered = false + output buffer #51: + timeUs = 1000001300317 + size = 0 + rendered = false + output buffer #52: + timeUs = 1000001323537 + size = 0 + rendered = false + output buffer #53: + timeUs = 1000001346757 + size = 0 + rendered = false + output buffer #54: + timeUs = 1000001369977 + size = 0 + rendered = false + output buffer #55: + timeUs = 1000001393197 + size = 0 + rendered = false + output buffer #56: + timeUs = 1000001416417 + size = 0 + rendered = false + output buffer #57: + timeUs = 1000001439637 + size = 0 + rendered = false + output buffer #58: + timeUs = 1000001462857 + size = 0 + rendered = false + output buffer #59: + timeUs = 1000001486077 + size = 0 + rendered = false + output buffer #60: + timeUs = 1000001509297 + size = 0 + rendered = false + output buffer #61: + timeUs = 1000001532517 + size = 0 + rendered = false + output buffer #62: + timeUs = 1000001555736 + size = 0 + rendered = false + output buffer #63: + timeUs = 1000001578956 + size = 0 + rendered = false + output buffer #64: + timeUs = 1000001602176 + size = 0 + rendered = false + output buffer #65: + timeUs = 1000001625396 + size = 0 + rendered = false + output buffer #66: + timeUs = 1000001648616 + size = 0 + rendered = false + output buffer #67: + timeUs = 1000001671836 + size = 0 + rendered = false + output buffer #68: + timeUs = 1000001695056 + size = 0 + rendered = false + output buffer #69: + timeUs = 1000001718276 + size = 0 + rendered = false + output buffer #70: + timeUs = 1000001741496 + size = 0 + rendered = false + output buffer #71: + timeUs = 1000001764716 + size = 0 + rendered = false + output buffer #72: + timeUs = 1000001787936 + size = 0 + rendered = false + output buffer #73: + timeUs = 1000001811156 + size = 0 + rendered = false + output buffer #74: + timeUs = 1000001834376 + size = 0 + rendered = false + output buffer #75: + timeUs = 1000001857596 + size = 0 + rendered = false + output buffer #76: + timeUs = 1000001880816 + size = 0 + rendered = false + output buffer #77: + timeUs = 1000001904036 + size = 0 + rendered = false + output buffer #78: + timeUs = 1000001927256 + size = 0 + rendered = false + output buffer #79: + timeUs = 1000001950476 + size = 0 + rendered = false + output buffer #80: + timeUs = 1000001973696 + size = 0 + rendered = false + output buffer #81: + timeUs = 1000001996916 + size = 0 + rendered = false + output buffer #82: + timeUs = 1000002020136 + size = 0 + rendered = false + output buffer #83: + timeUs = 1000002043356 + size = 0 + rendered = false + output buffer #84: + timeUs = 1000002066575 + size = 0 + rendered = false + output buffer #85: + timeUs = 1000002089795 + size = 0 + rendered = false + output buffer #86: + timeUs = 1000002113015 + size = 0 + rendered = false + output buffer #87: + timeUs = 1000002136235 + size = 0 + rendered = false + output buffer #88: + timeUs = 1000002159455 + size = 0 + rendered = false + output buffer #89: + timeUs = 1000002182675 + size = 0 + rendered = false + output buffer #90: + timeUs = 1000002205895 + size = 0 + rendered = false + output buffer #91: + timeUs = 1000002229115 + size = 0 + rendered = false + output buffer #92: + timeUs = 1000002252335 + size = 0 + rendered = false + output buffer #93: + timeUs = 1000002275555 + size = 0 + rendered = false + output buffer #94: + timeUs = 1000002298775 + size = 0 + rendered = false + output buffer #95: + timeUs = 1000002321995 + size = 0 + rendered = false + output buffer #96: + timeUs = 1000002345215 + size = 0 + rendered = false + output buffer #97: + timeUs = 1000002368435 + size = 0 + rendered = false + output buffer #98: + timeUs = 1000002391655 + size = 0 + rendered = false + output buffer #99: + timeUs = 1000002414875 + size = 0 + rendered = false + output buffer #100: + timeUs = 1000002438095 + size = 0 + rendered = false + output buffer #101: + timeUs = 1000002461315 + size = 0 + rendered = false + output buffer #102: + timeUs = 1000002484535 + size = 0 + rendered = false + output buffer #103: + timeUs = 1000002507755 + size = 0 + rendered = false + output buffer #104: + timeUs = 1000002530975 + size = 0 + rendered = false + output buffer #105: + timeUs = 1000002554195 + size = 0 + rendered = false + output buffer #106: + timeUs = 1000002577414 + size = 0 + rendered = false + output buffer #107: + timeUs = 1000002600634 + size = 0 + rendered = false + output buffer #108: + timeUs = 1000002623854 + size = 0 + rendered = false + output buffer #109: + timeUs = 1000002647074 + size = 0 + rendered = false + output buffer #110: + timeUs = 1000002670294 + size = 0 + rendered = false + output buffer #111: + timeUs = 1000002693514 + size = 0 + rendered = false + output buffer #112: + timeUs = 1000002716734 + size = 0 + rendered = false + output buffer #113: + timeUs = 1000002739954 + size = 0 + rendered = false + output buffer #114: + timeUs = 1000002763174 + size = 0 + rendered = false + output buffer #115: + timeUs = 1000002786394 + size = 0 + rendered = false + output buffer #116: + timeUs = 1000002809614 + size = 0 + rendered = false + output buffer #117: + timeUs = 1000002832834 + size = 0 + rendered = false + output buffer #118: + timeUs = 1000002856054 + size = 0 + rendered = false + output buffer #119: + timeUs = 1000002879274 + size = 0 + rendered = false + output buffer #120: + timeUs = 1000002902494 + size = 0 + rendered = false + output buffer #121: + timeUs = 1000002925714 + size = 0 + rendered = false + output buffer #122: + timeUs = 1000002948934 + size = 0 + rendered = false + output buffer #123: + timeUs = 1000002972154 + size = 0 + rendered = false + output buffer #124: + timeUs = 1000002995374 + size = 0 + rendered = false + output buffer #125: + timeUs = 1000003018594 + size = 0 + rendered = false + output buffer #126: + timeUs = 1000003041814 + size = 0 + rendered = false + output buffer #127: + timeUs = 1000003065034 + size = 0 + rendered = false + output buffer #128: + timeUs = 1000003088253 + size = 0 + rendered = false + output buffer #129: + timeUs = 1000003111473 + size = 0 + rendered = false + output buffer #130: + timeUs = 1000003134693 + size = 0 + rendered = false + output buffer #131: + timeUs = 1000003157913 + size = 0 + rendered = false + output buffer #132: + timeUs = 1000003181133 + size = 0 + rendered = false + output buffer #133: + timeUs = 1000003204353 + size = 0 + rendered = false + output buffer #134: + timeUs = 1000003227573 + size = 0 + rendered = false + output buffer #135: + timeUs = 1000003250793 + size = 0 + rendered = false + output buffer #136: + timeUs = 1000003274013 + size = 0 + rendered = false + output buffer #137: + timeUs = 1000003297233 + size = 0 + rendered = false + output buffer #138: + timeUs = 1000003320453 + size = 0 + rendered = false + output buffer #139: + timeUs = 1000003343673 + size = 0 + rendered = false + output buffer #140: + timeUs = 1000003366893 + size = 0 + rendered = false + output buffer #141: + timeUs = 1000003390113 + size = 0 + rendered = false + output buffer #142: + timeUs = 1000003413333 + size = 0 + rendered = false + output buffer #143: + timeUs = 1000003436553 + size = 0 + rendered = false + output buffer #144: + timeUs = 1000003459773 + size = 0 + rendered = false + output buffer #145: + timeUs = 1000003482993 + size = 0 + rendered = false + output buffer #146: + timeUs = 1000003506213 + size = 0 + rendered = false + output buffer #147: + timeUs = 1000003529433 + size = 0 + rendered = false + output buffer #148: + timeUs = 1000003552653 + size = 0 + rendered = false + output buffer #149: + timeUs = 1000003575873 + size = 0 + rendered = false + output buffer #150: + timeUs = 1000003599092 + size = 0 + rendered = false + output buffer #151: + timeUs = 1000003622312 + size = 0 + rendered = false + output buffer #152: + timeUs = 1000003645532 + size = 0 + rendered = false + output buffer #153: + timeUs = 1000003668752 + size = 0 + rendered = false + output buffer #154: + timeUs = 1000003691972 + size = 0 + rendered = false + output buffer #155: + timeUs = 1000003715192 + size = 0 + rendered = false + output buffer #156: + timeUs = 1000003738412 + size = 0 + rendered = false + output buffer #157: + timeUs = 1000003761632 + size = 0 + rendered = false + output buffer #158: + timeUs = 1000003784852 + size = 0 + rendered = false + output buffer #159: + timeUs = 1000003808072 + size = 0 + rendered = false + output buffer #160: + timeUs = 1000003831292 + size = 0 + rendered = false + output buffer #161: + timeUs = 1000003854512 + size = 0 + rendered = false + output buffer #162: + timeUs = 1000003877732 + size = 0 + rendered = false + output buffer #163: + timeUs = 1000003900952 + size = 0 + rendered = false + output buffer #164: + timeUs = 1000003924172 + size = 0 + rendered = false + output buffer #165: + timeUs = 1000003947392 + size = 0 + rendered = false + output buffer #166: + timeUs = 1000003970612 + size = 0 + rendered = false + output buffer #167: + timeUs = 1000003993832 + size = 0 + rendered = false + output buffer #168: + timeUs = 1000004017052 + size = 0 + rendered = false + output buffer #169: + timeUs = 1000004040272 + size = 0 + rendered = false + output buffer #170: + timeUs = 1000004063492 + size = 0 + rendered = false + output buffer #171: + timeUs = 1000004086712 + size = 0 + rendered = false + output buffer #172: + timeUs = 1000004109931 + size = 0 + rendered = false + output buffer #173: + timeUs = 1000004133151 + size = 0 + rendered = false + output buffer #174: + timeUs = 1000004156371 + size = 0 + rendered = false + output buffer #175: + timeUs = 1000004179591 + size = 0 + rendered = false + output buffer #176: + timeUs = 1000004202811 + size = 0 + rendered = false + output buffer #177: + timeUs = 1000004226031 + size = 0 + rendered = false + output buffer #178: + timeUs = 1000004249251 + size = 0 + rendered = false + output buffer #179: + timeUs = 1000004272471 + size = 0 + rendered = false + output buffer #180: + timeUs = 1000004295691 + size = 0 + rendered = false + output buffer #181: + timeUs = 1000004318911 + size = 0 + rendered = false + output buffer #182: + timeUs = 1000004342131 + size = 0 + rendered = false + output buffer #183: + timeUs = 1000004365351 + size = 0 + rendered = false + output buffer #184: + timeUs = 1000004388571 + size = 0 + rendered = false + output buffer #185: + timeUs = 1000004411791 + size = 0 + rendered = false + output buffer #186: + timeUs = 1000004435011 + size = 0 + rendered = false + output buffer #187: + timeUs = 1000004458231 + size = 0 + rendered = false + output buffer #188: + timeUs = 1000004481451 + size = 0 + rendered = false + output buffer #189: + timeUs = 1000004504671 + size = 0 + rendered = false + output buffer #190: + timeUs = 1000004527891 + size = 0 + rendered = false + output buffer #191: + timeUs = 1000004551111 + size = 0 + rendered = false + output buffer #192: + timeUs = 1000004574331 + size = 0 + rendered = false + output buffer #193: + timeUs = 1000004597551 + size = 0 + rendered = false + output buffer #194: + timeUs = 1000004620770 + size = 0 + rendered = false + output buffer #195: + timeUs = 1000004643990 + size = 0 + rendered = false + output buffer #196: + timeUs = 1000004667210 + size = 0 + rendered = false + output buffer #197: + timeUs = 1000004690430 + size = 0 + rendered = false + output buffer #198: + timeUs = 1000004713650 + size = 0 + rendered = false + output buffer #199: + timeUs = 1000004736870 + size = 0 + rendered = false + output buffer #200: + timeUs = 1000004760090 + size = 0 + rendered = false + output buffer #201: + timeUs = 1000004783310 + size = 0 + rendered = false + output buffer #202: + timeUs = 1000004806530 + size = 0 + rendered = false + output buffer #203: + timeUs = 1000004829750 + size = 0 + rendered = false + output buffer #204: + timeUs = 1000004852970 + size = 0 + rendered = false + output buffer #205: + timeUs = 1000004876190 + size = 0 + rendered = false + output buffer #206: + timeUs = 1000004899410 + size = 0 + rendered = false + output buffer #207: + timeUs = 1000004922630 + size = 0 + rendered = false + output buffer #208: + timeUs = 1000004945850 + size = 0 + rendered = false + output buffer #209: + timeUs = 1000004969070 + size = 0 + rendered = false + output buffer #210: + timeUs = 1000004992290 + size = 0 + rendered = false + output buffer #211: + timeUs = 1000005015510 + size = 0 + rendered = false +MediaCodecAdapter (media3.video.avc): + inputBuffers: + count = 125 + input buffer #0: + timeUs = 1000000000000 + contents = length 5252, hash 13893A4C + input buffer #1: + timeUs = 1000000160000 + contents = length 44, hash A05B3BEA + input buffer #2: + timeUs = 1000000080000 + contents = length 25, hash 441EC4FE + input buffer #3: + timeUs = 1000000120000 + contents = length 18, hash B64DA059 + input buffer #4: + timeUs = 1000000320000 + contents = length 32, hash 5BFD9538 + input buffer #5: + timeUs = 1000000240000 + contents = length 27, hash 705869A5 + input buffer #6: + timeUs = 1000000200000 + contents = length 18, hash D163DF61 + input buffer #7: + timeUs = 1000000280000 + contents = length 18, hash FD82E95 + input buffer #8: + timeUs = 1000000480000 + contents = length 28, hash 44A16E72 + input buffer #9: + timeUs = 1000000400000 + contents = length 27, hash FFA326ED + input buffer #10: + timeUs = 1000000360000 + contents = length 18, hash DC93CC9D + input buffer #11: + timeUs = 1000000440000 + contents = length 18, hash 1B081BD1 + input buffer #12: + timeUs = 1000000640000 + contents = length 28, hash 2700AF + input buffer #13: + timeUs = 1000000560000 + contents = length 27, hash DECA17D0 + input buffer #14: + timeUs = 1000000520000 + contents = length 18, hash D646C05A + input buffer #15: + timeUs = 1000000600000 + contents = length 18, hash 14BB0F8E + input buffer #16: + timeUs = 1000000800000 + contents = length 28, hash 5DE2C2B + input buffer #17: + timeUs = 1000000720000 + contents = length 27, hash 376B4E14 + input buffer #18: + timeUs = 1000000680000 + contents = length 18, hash E176AD96 + input buffer #19: + timeUs = 1000000760000 + contents = length 18, hash 1FEAFCCA + input buffer #20: + timeUs = 1000000960000 + contents = length 28, hash C163BE68 + input buffer #21: + timeUs = 1000000880000 + contents = length 27, hash F7170357 + input buffer #22: + timeUs = 1000000840000 + contents = length 18, hash 3B013BD2 + input buffer #23: + timeUs = 1000000920000 + contents = length 18, hash 79758B06 + input buffer #24: + timeUs = 1000001120000 + contents = length 28, hash F72EB1A3 + input buffer #25: + timeUs = 1000001040000 + contents = length 27, hash 4FB8399C + input buffer #26: + timeUs = 1000001000000 + contents = length 18, hash 4631290E + input buffer #27: + timeUs = 1000001080000 + contents = length 18, hash 84A57842 + input buffer #28: + timeUs = 1000001280000 + contents = length 28, hash E1FCF000 + input buffer #29: + timeUs = 1000001200000 + contents = length 27, hash 21D3B540 + input buffer #30: + timeUs = 1000001160000 + contents = length 18, hash 62DE0FC9 + input buffer #31: + timeUs = 1000001240000 + contents = length 18, hash A1525EFD + input buffer #32: + timeUs = 1000001440000 + contents = length 28, hash 5350E8FA + input buffer #33: + timeUs = 1000001360000 + contents = length 27, hash E7563B65 + input buffer #34: + timeUs = 1000001320000 + contents = length 18, hash 77D95125 + input buffer #35: + timeUs = 1000001400000 + contents = length 18, hash B64DA059 + input buffer #36: + timeUs = 1000001600000 + contents = length 28, hash ED67B37 + input buffer #37: + timeUs = 1000001520000 + contents = length 27, hash A701F0A9 + input buffer #38: + timeUs = 1000001480000 + contents = length 18, hash D163DF61 + input buffer #39: + timeUs = 1000001560000 + contents = length 18, hash FD82E95 + input buffer #40: + timeUs = 1000001760000 + contents = length 28, hash 44A16E72 + input buffer #41: + timeUs = 1000001680000 + contents = length 27, hash FFA326ED + input buffer #42: + timeUs = 1000001640000 + contents = length 18, hash DC93CC9D + input buffer #43: + timeUs = 1000001720000 + contents = length 18, hash 1B081BD1 + input buffer #44: + timeUs = 1000001920000 + contents = length 28, hash 2700AF + input buffer #45: + timeUs = 1000001840000 + contents = length 27, hash DECA17D0 + input buffer #46: + timeUs = 1000001800000 + contents = length 18, hash D646C05A + input buffer #47: + timeUs = 1000001880000 + contents = length 18, hash 14BB0F8E + input buffer #48: + timeUs = 1000002080000 + contents = length 28, hash 5DE2C2B + input buffer #49: + timeUs = 1000002000000 + contents = length 27, hash 376B4E14 + input buffer #50: + timeUs = 1000001960000 + contents = length 18, hash E176AD96 + input buffer #51: + timeUs = 1000002040000 + contents = length 18, hash 1FEAFCCA + input buffer #52: + timeUs = 1000002240000 + contents = length 28, hash C163BE68 + input buffer #53: + timeUs = 1000002160000 + contents = length 27, hash F7170357 + input buffer #54: + timeUs = 1000002120000 + contents = length 18, hash 3B013BD2 + input buffer #55: + timeUs = 1000002200000 + contents = length 18, hash 79758B06 + input buffer #56: + timeUs = 1000002400000 + contents = length 28, hash F72EB1A3 + input buffer #57: + timeUs = 1000002320000 + contents = length 27, hash 4FB8399C + input buffer #58: + timeUs = 1000002280000 + contents = length 18, hash 4631290E + input buffer #59: + timeUs = 1000002360000 + contents = length 18, hash 84A57842 + input buffer #60: + timeUs = 1000002560000 + contents = length 28, hash E1FCF000 + input buffer #61: + timeUs = 1000002480000 + contents = length 27, hash 21D3B540 + input buffer #62: + timeUs = 1000002440000 + contents = length 18, hash 62DE0FC9 + input buffer #63: + timeUs = 1000002520000 + contents = length 18, hash A1525EFD + input buffer #64: + timeUs = 1000002720000 + contents = length 28, hash 5350E8FA + input buffer #65: + timeUs = 1000002640000 + contents = length 27, hash E7563B65 + input buffer #66: + timeUs = 1000002600000 + contents = length 18, hash 77D95125 + input buffer #67: + timeUs = 1000002680000 + contents = length 18, hash B64DA059 + input buffer #68: + timeUs = 1000002880000 + contents = length 28, hash ED67B37 + input buffer #69: + timeUs = 1000002800000 + contents = length 27, hash A701F0A9 + input buffer #70: + timeUs = 1000002760000 + contents = length 18, hash D163DF61 + input buffer #71: + timeUs = 1000002840000 + contents = length 18, hash FD82E95 + input buffer #72: + timeUs = 1000003040000 + contents = length 28, hash 44A16E72 + input buffer #73: + timeUs = 1000002960000 + contents = length 27, hash FFA326ED + input buffer #74: + timeUs = 1000002920000 + contents = length 18, hash DC93CC9D + input buffer #75: + timeUs = 1000003000000 + contents = length 18, hash 1B081BD1 + input buffer #76: + timeUs = 1000003200000 + contents = length 28, hash 2700AF + input buffer #77: + timeUs = 1000003120000 + contents = length 27, hash DECA17D0 + input buffer #78: + timeUs = 1000003080000 + contents = length 18, hash D646C05A + input buffer #79: + timeUs = 1000003160000 + contents = length 18, hash 14BB0F8E + input buffer #80: + timeUs = 1000003360000 + contents = length 28, hash 5DE2C2B + input buffer #81: + timeUs = 1000003280000 + contents = length 27, hash 376B4E14 + input buffer #82: + timeUs = 1000003240000 + contents = length 18, hash E176AD96 + input buffer #83: + timeUs = 1000003320000 + contents = length 18, hash 1FEAFCCA + input buffer #84: + timeUs = 1000003520000 + contents = length 28, hash C163BE68 + input buffer #85: + timeUs = 1000003440000 + contents = length 27, hash F7170357 + input buffer #86: + timeUs = 1000003400000 + contents = length 18, hash 3B013BD2 + input buffer #87: + timeUs = 1000003480000 + contents = length 18, hash 79758B06 + input buffer #88: + timeUs = 1000003680000 + contents = length 28, hash F72EB1A3 + input buffer #89: + timeUs = 1000003600000 + contents = length 27, hash 4FB8399C + input buffer #90: + timeUs = 1000003560000 + contents = length 18, hash 4631290E + input buffer #91: + timeUs = 1000003640000 + contents = length 18, hash 84A57842 + input buffer #92: + timeUs = 1000003840000 + contents = length 33, hash AF5CF49E + input buffer #93: + timeUs = 1000003760000 + contents = length 27, hash 21D3B540 + input buffer #94: + timeUs = 1000003720000 + contents = length 18, hash 62DE0FC9 + input buffer #95: + timeUs = 1000003800000 + contents = length 18, hash A1525EFD + input buffer #96: + timeUs = 1000004000000 + contents = length 33, hash F4C6DE46 + input buffer #97: + timeUs = 1000003920000 + contents = length 27, hash E7563B65 + input buffer #98: + timeUs = 1000003880000 + contents = length 18, hash 77D95125 + input buffer #99: + timeUs = 1000003960000 + contents = length 18, hash B64DA059 + input buffer #100: + timeUs = 1000004160000 + contents = length 28, hash ED67B37 + input buffer #101: + timeUs = 1000004080000 + contents = length 27, hash A701F0A9 + input buffer #102: + timeUs = 1000004040000 + contents = length 18, hash D163DF61 + input buffer #103: + timeUs = 1000004120000 + contents = length 18, hash FD82E95 + input buffer #104: + timeUs = 1000004320000 + contents = length 28, hash 44A16E72 + input buffer #105: + timeUs = 1000004240000 + contents = length 27, hash FFA326ED + input buffer #106: + timeUs = 1000004200000 + contents = length 18, hash DC93CC9D + input buffer #107: + timeUs = 1000004280000 + contents = length 18, hash 1B081BD1 + input buffer #108: + timeUs = 1000004480000 + contents = length 28, hash 2700AF + input buffer #109: + timeUs = 1000004400000 + contents = length 27, hash DECA17D0 + input buffer #110: + timeUs = 1000004360000 + contents = length 18, hash D646C05A + input buffer #111: + timeUs = 1000004440000 + contents = length 18, hash 14BB0F8E + input buffer #112: + timeUs = 1000004640000 + contents = length 27, hash 5292D9E + input buffer #113: + timeUs = 1000004560000 + contents = length 27, hash 376B4E14 + input buffer #114: + timeUs = 1000004520000 + contents = length 18, hash E176AD96 + input buffer #115: + timeUs = 1000004600000 + contents = length 18, hash 1FEAFCCA + input buffer #116: + timeUs = 1000004800000 + contents = length 26, hash B0CAA4C9 + input buffer #117: + timeUs = 1000004720000 + contents = length 27, hash F7170357 + input buffer #118: + timeUs = 1000004680000 + contents = length 18, hash 3B013BD2 + input buffer #119: + timeUs = 1000004760000 + contents = length 18, hash 79758B06 + input buffer #120: + timeUs = 1000004960000 + contents = length 26, hash C63A1445 + input buffer #121: + timeUs = 1000004880000 + contents = length 27, hash 4FB8399C + input buffer #122: + timeUs = 1000004840000 + contents = length 18, hash 4631290E + input buffer #123: + timeUs = 1000004920000 + contents = length 18, hash 84A57842 + input buffer #124: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 124 + output buffer #0: + timeUs = 1000000000000 + size = 5252 + rendered = false + output buffer #1: + timeUs = 1000000160000 + size = 44 + rendered = true + output buffer #2: + timeUs = 1000000080000 + size = 25 + rendered = false + output buffer #3: + timeUs = 1000000120000 + size = 18 + rendered = true + output buffer #4: + timeUs = 1000000320000 + size = 32 + rendered = true + output buffer #5: + timeUs = 1000000240000 + size = 27 + rendered = true + output buffer #6: + timeUs = 1000000200000 + size = 18 + rendered = true + output buffer #7: + timeUs = 1000000280000 + size = 18 + rendered = true + output buffer #8: + timeUs = 1000000480000 + size = 28 + rendered = true + output buffer #9: + timeUs = 1000000400000 + size = 27 + rendered = true + output buffer #10: + timeUs = 1000000360000 + size = 18 + rendered = true + output buffer #11: + timeUs = 1000000440000 + size = 18 + rendered = true + output buffer #12: + timeUs = 1000000640000 + size = 28 + rendered = true + output buffer #13: + timeUs = 1000000560000 + size = 27 + rendered = true + output buffer #14: + timeUs = 1000000520000 + size = 18 + rendered = true + output buffer #15: + timeUs = 1000000600000 + size = 18 + rendered = true + output buffer #16: + timeUs = 1000000800000 + size = 28 + rendered = true + output buffer #17: + timeUs = 1000000720000 + size = 27 + rendered = true + output buffer #18: + timeUs = 1000000680000 + size = 18 + rendered = true + output buffer #19: + timeUs = 1000000760000 + size = 18 + rendered = true + output buffer #20: + timeUs = 1000000960000 + size = 28 + rendered = true + output buffer #21: + timeUs = 1000000880000 + size = 27 + rendered = true + output buffer #22: + timeUs = 1000000840000 + size = 18 + rendered = true + output buffer #23: + timeUs = 1000000920000 + size = 18 + rendered = true + output buffer #24: + timeUs = 1000001120000 + size = 28 + rendered = true + output buffer #25: + timeUs = 1000001040000 + size = 27 + rendered = true + output buffer #26: + timeUs = 1000001000000 + size = 18 + rendered = true + output buffer #27: + timeUs = 1000001080000 + size = 18 + rendered = true + output buffer #28: + timeUs = 1000001280000 + size = 28 + rendered = true + output buffer #29: + timeUs = 1000001200000 + size = 27 + rendered = true + output buffer #30: + timeUs = 1000001160000 + size = 18 + rendered = true + output buffer #31: + timeUs = 1000001240000 + size = 18 + rendered = true + output buffer #32: + timeUs = 1000001440000 + size = 28 + rendered = true + output buffer #33: + timeUs = 1000001360000 + size = 27 + rendered = true + output buffer #34: + timeUs = 1000001320000 + size = 18 + rendered = true + output buffer #35: + timeUs = 1000001400000 + size = 18 + rendered = true + output buffer #36: + timeUs = 1000001600000 + size = 28 + rendered = true + output buffer #37: + timeUs = 1000001520000 + size = 27 + rendered = true + output buffer #38: + timeUs = 1000001480000 + size = 18 + rendered = true + output buffer #39: + timeUs = 1000001560000 + size = 18 + rendered = true + output buffer #40: + timeUs = 1000001760000 + size = 28 + rendered = true + output buffer #41: + timeUs = 1000001680000 + size = 27 + rendered = true + output buffer #42: + timeUs = 1000001640000 + size = 18 + rendered = true + output buffer #43: + timeUs = 1000001720000 + size = 18 + rendered = true + output buffer #44: + timeUs = 1000001920000 + size = 28 + rendered = true + output buffer #45: + timeUs = 1000001840000 + size = 27 + rendered = true + output buffer #46: + timeUs = 1000001800000 + size = 18 + rendered = true + output buffer #47: + timeUs = 1000001880000 + size = 18 + rendered = true + output buffer #48: + timeUs = 1000002080000 + size = 28 + rendered = true + output buffer #49: + timeUs = 1000002000000 + size = 27 + rendered = true + output buffer #50: + timeUs = 1000001960000 + size = 18 + rendered = true + output buffer #51: + timeUs = 1000002040000 + size = 18 + rendered = true + output buffer #52: + timeUs = 1000002240000 + size = 28 + rendered = true + output buffer #53: + timeUs = 1000002160000 + size = 27 + rendered = true + output buffer #54: + timeUs = 1000002120000 + size = 18 + rendered = true + output buffer #55: + timeUs = 1000002200000 + size = 18 + rendered = true + output buffer #56: + timeUs = 1000002400000 + size = 28 + rendered = true + output buffer #57: + timeUs = 1000002320000 + size = 27 + rendered = true + output buffer #58: + timeUs = 1000002280000 + size = 18 + rendered = true + output buffer #59: + timeUs = 1000002360000 + size = 18 + rendered = true + output buffer #60: + timeUs = 1000002560000 + size = 28 + rendered = true + output buffer #61: + timeUs = 1000002480000 + size = 27 + rendered = true + output buffer #62: + timeUs = 1000002440000 + size = 18 + rendered = true + output buffer #63: + timeUs = 1000002520000 + size = 18 + rendered = true + output buffer #64: + timeUs = 1000002720000 + size = 28 + rendered = true + output buffer #65: + timeUs = 1000002640000 + size = 27 + rendered = true + output buffer #66: + timeUs = 1000002600000 + size = 18 + rendered = true + output buffer #67: + timeUs = 1000002680000 + size = 18 + rendered = true + output buffer #68: + timeUs = 1000002880000 + size = 28 + rendered = true + output buffer #69: + timeUs = 1000002800000 + size = 27 + rendered = true + output buffer #70: + timeUs = 1000002760000 + size = 18 + rendered = true + output buffer #71: + timeUs = 1000002840000 + size = 18 + rendered = true + output buffer #72: + timeUs = 1000003040000 + size = 28 + rendered = true + output buffer #73: + timeUs = 1000002960000 + size = 27 + rendered = true + output buffer #74: + timeUs = 1000002920000 + size = 18 + rendered = true + output buffer #75: + timeUs = 1000003000000 + size = 18 + rendered = true + output buffer #76: + timeUs = 1000003200000 + size = 28 + rendered = true + output buffer #77: + timeUs = 1000003120000 + size = 27 + rendered = true + output buffer #78: + timeUs = 1000003080000 + size = 18 + rendered = true + output buffer #79: + timeUs = 1000003160000 + size = 18 + rendered = true + output buffer #80: + timeUs = 1000003360000 + size = 28 + rendered = true + output buffer #81: + timeUs = 1000003280000 + size = 27 + rendered = true + output buffer #82: + timeUs = 1000003240000 + size = 18 + rendered = true + output buffer #83: + timeUs = 1000003320000 + size = 18 + rendered = true + output buffer #84: + timeUs = 1000003520000 + size = 28 + rendered = true + output buffer #85: + timeUs = 1000003440000 + size = 27 + rendered = true + output buffer #86: + timeUs = 1000003400000 + size = 18 + rendered = true + output buffer #87: + timeUs = 1000003480000 + size = 18 + rendered = true + output buffer #88: + timeUs = 1000003680000 + size = 28 + rendered = true + output buffer #89: + timeUs = 1000003600000 + size = 27 + rendered = true + output buffer #90: + timeUs = 1000003560000 + size = 18 + rendered = true + output buffer #91: + timeUs = 1000003640000 + size = 18 + rendered = true + output buffer #92: + timeUs = 1000003840000 + size = 33 + rendered = true + output buffer #93: + timeUs = 1000003760000 + size = 27 + rendered = true + output buffer #94: + timeUs = 1000003720000 + size = 18 + rendered = true + output buffer #95: + timeUs = 1000003800000 + size = 18 + rendered = true + output buffer #96: + timeUs = 1000004000000 + size = 33 + rendered = true + output buffer #97: + timeUs = 1000003920000 + size = 27 + rendered = true + output buffer #98: + timeUs = 1000003880000 + size = 18 + rendered = true + output buffer #99: + timeUs = 1000003960000 + size = 18 + rendered = true + output buffer #100: + timeUs = 1000004160000 + size = 28 + rendered = true + output buffer #101: + timeUs = 1000004080000 + size = 27 + rendered = true + output buffer #102: + timeUs = 1000004040000 + size = 18 + rendered = true + output buffer #103: + timeUs = 1000004120000 + size = 18 + rendered = true + output buffer #104: + timeUs = 1000004320000 + size = 28 + rendered = true + output buffer #105: + timeUs = 1000004240000 + size = 27 + rendered = true + output buffer #106: + timeUs = 1000004200000 + size = 18 + rendered = true + output buffer #107: + timeUs = 1000004280000 + size = 18 + rendered = true + output buffer #108: + timeUs = 1000004480000 + size = 28 + rendered = true + output buffer #109: + timeUs = 1000004400000 + size = 27 + rendered = true + output buffer #110: + timeUs = 1000004360000 + size = 18 + rendered = true + output buffer #111: + timeUs = 1000004440000 + size = 18 + rendered = true + output buffer #112: + timeUs = 1000004640000 + size = 27 + rendered = true + output buffer #113: + timeUs = 1000004560000 + size = 27 + rendered = true + output buffer #114: + timeUs = 1000004520000 + size = 18 + rendered = true + output buffer #115: + timeUs = 1000004600000 + size = 18 + rendered = true + output buffer #116: + timeUs = 1000004800000 + size = 26 + rendered = true + output buffer #117: + timeUs = 1000004720000 + size = 27 + rendered = true + output buffer #118: + timeUs = 1000004680000 + size = 18 + rendered = true + output buffer #119: + timeUs = 1000004760000 + size = 18 + rendered = true + output buffer #120: + timeUs = 1000004960000 + size = 26 + rendered = true + output buffer #121: + timeUs = 1000004880000 + size = 27 + rendered = true + output buffer #122: + timeUs = 1000004840000 + size = 18 + rendered = true + output buffer #123: + timeUs = 1000004920000 + size = 18 + rendered = true +AudioSink: + buffer count = 212 + config: + pcmEncoding = pcm-16 + channelCount = 1 + sampleRate = 44100 + buffer #0: + time = 1000000116099 + data = empty + buffer #1: + time = 1000000139319 + data = empty + buffer #2: + time = 1000000162539 + data = empty + buffer #3: + time = 1000000185759 + data = empty + buffer #4: + time = 1000000208979 + data = empty + buffer #5: + time = 1000000232199 + data = empty + buffer #6: + time = 1000000255419 + data = empty + buffer #7: + time = 1000000278639 + data = empty + buffer #8: + time = 1000000301859 + data = empty + buffer #9: + time = 1000000325079 + data = empty + buffer #10: + time = 1000000348299 + data = empty + buffer #11: + time = 1000000371519 + data = empty + buffer #12: + time = 1000000394739 + data = empty + buffer #13: + time = 1000000417959 + data = empty + buffer #14: + time = 1000000441179 + data = empty + buffer #15: + time = 1000000464399 + data = empty + buffer #16: + time = 1000000487619 + data = empty + buffer #17: + time = 1000000510839 + data = empty + buffer #18: + time = 1000000534058 + data = empty + buffer #19: + time = 1000000557278 + data = empty + buffer #20: + time = 1000000580498 + data = empty + buffer #21: + time = 1000000603718 + data = empty + buffer #22: + time = 1000000626938 + data = empty + buffer #23: + time = 1000000650158 + data = empty + buffer #24: + time = 1000000673378 + data = empty + buffer #25: + time = 1000000696598 + data = empty + buffer #26: + time = 1000000719818 + data = empty + buffer #27: + time = 1000000743038 + data = empty + buffer #28: + time = 1000000766258 + data = empty + buffer #29: + time = 1000000789478 + data = empty + buffer #30: + time = 1000000812698 + data = empty + buffer #31: + time = 1000000835918 + data = empty + buffer #32: + time = 1000000859138 + data = empty + buffer #33: + time = 1000000882358 + data = empty + buffer #34: + time = 1000000905578 + data = empty + buffer #35: + time = 1000000928798 + data = empty + buffer #36: + time = 1000000952018 + data = empty + buffer #37: + time = 1000000975238 + data = empty + buffer #38: + time = 1000000998458 + data = empty + buffer #39: + time = 1000001021678 + data = empty + buffer #40: + time = 1000001044897 + data = empty + buffer #41: + time = 1000001068117 + data = empty + buffer #42: + time = 1000001091337 + data = empty + buffer #43: + time = 1000001114557 + data = empty + buffer #44: + time = 1000001137777 + data = empty + buffer #45: + time = 1000001160997 + data = empty + buffer #46: + time = 1000001184217 + data = empty + buffer #47: + time = 1000001207437 + data = empty + buffer #48: + time = 1000001230657 + data = empty + buffer #49: + time = 1000001253877 + data = empty + buffer #50: + time = 1000001277097 + data = empty + buffer #51: + time = 1000001300317 + data = empty + buffer #52: + time = 1000001323537 + data = empty + buffer #53: + time = 1000001346757 + data = empty + buffer #54: + time = 1000001369977 + data = empty + buffer #55: + time = 1000001393197 + data = empty + buffer #56: + time = 1000001416417 + data = empty + buffer #57: + time = 1000001439637 + data = empty + buffer #58: + time = 1000001462857 + data = empty + buffer #59: + time = 1000001486077 + data = empty + buffer #60: + time = 1000001509297 + data = empty + buffer #61: + time = 1000001532517 + data = empty + buffer #62: + time = 1000001555736 + data = empty + buffer #63: + time = 1000001578956 + data = empty + buffer #64: + time = 1000001602176 + data = empty + buffer #65: + time = 1000001625396 + data = empty + buffer #66: + time = 1000001648616 + data = empty + buffer #67: + time = 1000001671836 + data = empty + buffer #68: + time = 1000001695056 + data = empty + buffer #69: + time = 1000001718276 + data = empty + buffer #70: + time = 1000001741496 + data = empty + buffer #71: + time = 1000001764716 + data = empty + buffer #72: + time = 1000001787936 + data = empty + buffer #73: + time = 1000001811156 + data = empty + buffer #74: + time = 1000001834376 + data = empty + buffer #75: + time = 1000001857596 + data = empty + buffer #76: + time = 1000001880816 + data = empty + buffer #77: + time = 1000001904036 + data = empty + buffer #78: + time = 1000001927256 + data = empty + buffer #79: + time = 1000001950476 + data = empty + buffer #80: + time = 1000001973696 + data = empty + buffer #81: + time = 1000001996916 + data = empty + buffer #82: + time = 1000002020136 + data = empty + buffer #83: + time = 1000002043356 + data = empty + buffer #84: + time = 1000002066575 + data = empty + buffer #85: + time = 1000002089795 + data = empty + buffer #86: + time = 1000002113015 + data = empty + buffer #87: + time = 1000002136235 + data = empty + buffer #88: + time = 1000002159455 + data = empty + buffer #89: + time = 1000002182675 + data = empty + buffer #90: + time = 1000002205895 + data = empty + buffer #91: + time = 1000002229115 + data = empty + buffer #92: + time = 1000002252335 + data = empty + buffer #93: + time = 1000002275555 + data = empty + buffer #94: + time = 1000002298775 + data = empty + buffer #95: + time = 1000002321995 + data = empty + buffer #96: + time = 1000002345215 + data = empty + buffer #97: + time = 1000002368435 + data = empty + buffer #98: + time = 1000002391655 + data = empty + buffer #99: + time = 1000002414875 + data = empty + buffer #100: + time = 1000002438095 + data = empty + buffer #101: + time = 1000002461315 + data = empty + buffer #102: + time = 1000002484535 + data = empty + buffer #103: + time = 1000002507755 + data = empty + buffer #104: + time = 1000002530975 + data = empty + buffer #105: + time = 1000002554195 + data = empty + buffer #106: + time = 1000002577414 + data = empty + buffer #107: + time = 1000002600634 + data = empty + buffer #108: + time = 1000002623854 + data = empty + buffer #109: + time = 1000002647074 + data = empty + buffer #110: + time = 1000002670294 + data = empty + buffer #111: + time = 1000002693514 + data = empty + buffer #112: + time = 1000002716734 + data = empty + buffer #113: + time = 1000002739954 + data = empty + buffer #114: + time = 1000002763174 + data = empty + buffer #115: + time = 1000002786394 + data = empty + buffer #116: + time = 1000002809614 + data = empty + buffer #117: + time = 1000002832834 + data = empty + buffer #118: + time = 1000002856054 + data = empty + buffer #119: + time = 1000002879274 + data = empty + buffer #120: + time = 1000002902494 + data = empty + buffer #121: + time = 1000002925714 + data = empty + buffer #122: + time = 1000002948934 + data = empty + buffer #123: + time = 1000002972154 + data = empty + buffer #124: + time = 1000002995374 + data = empty + buffer #125: + time = 1000003018594 + data = empty + buffer #126: + time = 1000003041814 + data = empty + buffer #127: + time = 1000003065034 + data = empty + buffer #128: + time = 1000003088253 + data = empty + buffer #129: + time = 1000003111473 + data = empty + buffer #130: + time = 1000003134693 + data = empty + buffer #131: + time = 1000003157913 + data = empty + buffer #132: + time = 1000003181133 + data = empty + buffer #133: + time = 1000003204353 + data = empty + buffer #134: + time = 1000003227573 + data = empty + buffer #135: + time = 1000003250793 + data = empty + buffer #136: + time = 1000003274013 + data = empty + buffer #137: + time = 1000003297233 + data = empty + buffer #138: + time = 1000003320453 + data = empty + buffer #139: + time = 1000003343673 + data = empty + buffer #140: + time = 1000003366893 + data = empty + buffer #141: + time = 1000003390113 + data = empty + buffer #142: + time = 1000003413333 + data = empty + buffer #143: + time = 1000003436553 + data = empty + buffer #144: + time = 1000003459773 + data = empty + buffer #145: + time = 1000003482993 + data = empty + buffer #146: + time = 1000003506213 + data = empty + buffer #147: + time = 1000003529433 + data = empty + buffer #148: + time = 1000003552653 + data = empty + buffer #149: + time = 1000003575873 + data = empty + buffer #150: + time = 1000003599092 + data = empty + buffer #151: + time = 1000003622312 + data = empty + buffer #152: + time = 1000003645532 + data = empty + buffer #153: + time = 1000003668752 + data = empty + buffer #154: + time = 1000003691972 + data = empty + buffer #155: + time = 1000003715192 + data = empty + buffer #156: + time = 1000003738412 + data = empty + buffer #157: + time = 1000003761632 + data = empty + buffer #158: + time = 1000003784852 + data = empty + buffer #159: + time = 1000003808072 + data = empty + buffer #160: + time = 1000003831292 + data = empty + buffer #161: + time = 1000003854512 + data = empty + buffer #162: + time = 1000003877732 + data = empty + buffer #163: + time = 1000003900952 + data = empty + buffer #164: + time = 1000003924172 + data = empty + buffer #165: + time = 1000003947392 + data = empty + buffer #166: + time = 1000003970612 + data = empty + buffer #167: + time = 1000003993832 + data = empty + buffer #168: + time = 1000004017052 + data = empty + buffer #169: + time = 1000004040272 + data = empty + buffer #170: + time = 1000004063492 + data = empty + buffer #171: + time = 1000004086712 + data = empty + buffer #172: + time = 1000004109931 + data = empty + buffer #173: + time = 1000004133151 + data = empty + buffer #174: + time = 1000004156371 + data = empty + buffer #175: + time = 1000004179591 + data = empty + buffer #176: + time = 1000004202811 + data = empty + buffer #177: + time = 1000004226031 + data = empty + buffer #178: + time = 1000004249251 + data = empty + buffer #179: + time = 1000004272471 + data = empty + buffer #180: + time = 1000004295691 + data = empty + buffer #181: + time = 1000004318911 + data = empty + buffer #182: + time = 1000004342131 + data = empty + buffer #183: + time = 1000004365351 + data = empty + buffer #184: + time = 1000004388571 + data = empty + buffer #185: + time = 1000004411791 + data = empty + buffer #186: + time = 1000004435011 + data = empty + buffer #187: + time = 1000004458231 + data = empty + buffer #188: + time = 1000004481451 + data = empty + buffer #189: + time = 1000004504671 + data = empty + buffer #190: + time = 1000004527891 + data = empty + buffer #191: + time = 1000004551111 + data = empty + buffer #192: + time = 1000004574331 + data = empty + buffer #193: + time = 1000004597551 + data = empty + buffer #194: + time = 1000004620770 + data = empty + buffer #195: + time = 1000004643990 + data = empty + buffer #196: + time = 1000004667210 + data = empty + buffer #197: + time = 1000004690430 + data = empty + buffer #198: + time = 1000004713650 + data = empty + buffer #199: + time = 1000004736870 + data = empty + buffer #200: + time = 1000004760090 + data = empty + buffer #201: + time = 1000004783310 + data = empty + buffer #202: + time = 1000004806530 + data = empty + buffer #203: + time = 1000004829750 + data = empty + buffer #204: + time = 1000004852970 + data = empty + buffer #205: + time = 1000004876190 + data = empty + buffer #206: + time = 1000004899410 + data = empty + buffer #207: + time = 1000004922630 + data = empty + buffer #208: + time = 1000004945850 + data = empty + buffer #209: + time = 1000004969070 + data = empty + buffer #210: + time = 1000004992290 + data = empty + buffer #211: + time = 1000005015510 + data = empty diff --git a/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_reduce_after_load_finished_enabled.dump b/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_reduce_after_load_finished_enabled.dump new file mode 100644 index 00000000000..96dfb364600 --- /dev/null +++ b/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_reduce_after_load_finished_enabled.dump @@ -0,0 +1,2197 @@ +extracted samples: + sample #0: + track = 0 + timeUs = 0 + flags = 1 + size = 5252 + sample #1: + track = 0 + timeUs = 160000 + size = 44 + sample #2: + track = 0 + timeUs = 80000 + size = 25 + sample #3: + track = 0 + timeUs = 40000 + flags = 67108864 + size = 18 + sample #4: + track = 1 + timeUs = 92879 + flags = 1 + size = 4 + sample #5: + track = 0 + timeUs = 120000 + flags = 67108864 + size = 18 + sample #6: + track = 1 + timeUs = 116099 + flags = 1 + size = 4 + sample #7: + track = 1 + timeUs = 139319 + flags = 1 + size = 4 + sample #8: + track = 0 + timeUs = 320000 + size = 32 + sample #9: + track = 1 + timeUs = 162539 + flags = 1 + size = 4 + sample #10: + track = 0 + timeUs = 240000 + size = 27 + sample #11: + track = 1 + timeUs = 185759 + flags = 1 + size = 4 + sample #12: + track = 1 + timeUs = 208979 + flags = 1 + size = 4 + sample #13: + track = 0 + timeUs = 200000 + flags = 67108864 + size = 18 + sample #14: + track = 1 + timeUs = 232199 + flags = 1 + size = 4 + sample #15: + track = 1 + timeUs = 255419 + flags = 1 + size = 4 + sample #16: + track = 0 + timeUs = 280000 + flags = 67108864 + size = 18 + sample #17: + track = 1 + timeUs = 278639 + flags = 1 + size = 4 + sample #18: + track = 1 + timeUs = 301859 + flags = 1 + size = 4 + sample #19: + track = 0 + timeUs = 480000 + size = 28 + sample #20: + track = 1 + timeUs = 325079 + flags = 1 + size = 4 + sample #21: + track = 0 + timeUs = 400000 + size = 27 + sample #22: + track = 1 + timeUs = 348299 + flags = 1 + size = 4 + sample #23: + track = 1 + timeUs = 371519 + flags = 1 + size = 4 + sample #24: + track = 0 + timeUs = 360000 + flags = 67108864 + size = 18 + sample #25: + track = 1 + timeUs = 394739 + flags = 1 + size = 4 + sample #26: + track = 1 + timeUs = 417959 + flags = 1 + size = 4 + sample #27: + track = 0 + timeUs = 440000 + flags = 67108864 + size = 18 + sample #28: + track = 1 + timeUs = 441179 + flags = 1 + size = 4 + sample #29: + track = 0 + timeUs = 640000 + size = 28 + sample #30: + track = 1 + timeUs = 464399 + flags = 1 + size = 4 + sample #31: + track = 1 + timeUs = 487619 + flags = 1 + size = 4 + sample #32: + track = 0 + timeUs = 560000 + size = 27 + sample #33: + track = 1 + timeUs = 510839 + flags = 1 + size = 4 + sample #34: + track = 1 + timeUs = 534058 + flags = 1 + size = 4 + sample #35: + track = 0 + timeUs = 520000 + flags = 67108864 + size = 18 + sample #36: + track = 1 + timeUs = 557278 + flags = 1 + size = 4 + sample #37: + track = 1 + timeUs = 580498 + flags = 1 + size = 4 + sample #38: + track = 0 + timeUs = 600000 + flags = 67108864 + size = 18 + sample #39: + track = 1 + timeUs = 603718 + flags = 1 + size = 4 + sample #40: + track = 0 + timeUs = 800000 + size = 28 + sample #41: + track = 1 + timeUs = 626938 + flags = 1 + size = 4 + sample #42: + track = 1 + timeUs = 650158 + flags = 1 + size = 4 + sample #43: + track = 0 + timeUs = 720000 + size = 27 + sample #44: + track = 1 + timeUs = 673378 + flags = 1 + size = 4 + sample #45: + track = 1 + timeUs = 696598 + flags = 1 + size = 4 + sample #46: + track = 0 + timeUs = 680000 + flags = 67108864 + size = 18 + sample #47: + track = 1 + timeUs = 719818 + flags = 1 + size = 4 + sample #48: + track = 1 + timeUs = 743038 + flags = 1 + size = 4 + sample #49: + track = 0 + timeUs = 760000 + flags = 67108864 + size = 18 + sample #50: + track = 1 + timeUs = 766258 + flags = 1 + size = 4 + sample #51: + track = 0 + timeUs = 960000 + size = 28 + sample #52: + track = 1 + timeUs = 789478 + flags = 1 + size = 4 + sample #53: + track = 1 + timeUs = 812698 + flags = 1 + size = 4 + sample #54: + track = 0 + timeUs = 880000 + size = 27 + sample #55: + track = 1 + timeUs = 835918 + flags = 1 + size = 4 + sample #56: + track = 1 + timeUs = 859138 + flags = 1 + size = 4 + sample #57: + track = 0 + timeUs = 840000 + flags = 67108864 + size = 18 + sample #58: + track = 1 + timeUs = 882358 + flags = 1 + size = 4 + sample #59: + track = 0 + timeUs = 920000 + flags = 67108864 + size = 18 + sample #60: + track = 1 + timeUs = 905578 + flags = 1 + size = 4 + sample #61: + track = 1 + timeUs = 928798 + flags = 1 + size = 4 + sample #62: + track = 0 + timeUs = 1120000 + size = 28 + sample #63: + track = 1 + timeUs = 952018 + flags = 1 + size = 4 + sample #64: + track = 1 + timeUs = 975238 + flags = 1 + size = 4 + sample #65: + track = 0 + timeUs = 1040000 + size = 27 + sample #66: + track = 1 + timeUs = 998458 + flags = 1 + size = 4 + sample #67: + track = 1 + timeUs = 1021678 + flags = 1 + size = 4 + sample #68: + track = 0 + timeUs = 1000000 + flags = 67108864 + size = 18 + sample #69: + track = 1 + timeUs = 1044897 + flags = 1 + size = 4 + sample #70: + track = 0 + timeUs = 1080000 + flags = 67108864 + size = 18 + sample #71: + track = 1 + timeUs = 1068117 + flags = 1 + size = 4 + sample #72: + track = 1 + timeUs = 1091337 + flags = 1 + size = 4 + sample #73: + track = 0 + timeUs = 1280000 + size = 28 + sample #74: + track = 1 + timeUs = 1114557 + flags = 1 + size = 4 + sample #75: + track = 1 + timeUs = 1137777 + flags = 1 + size = 4 + sample #76: + track = 0 + timeUs = 1200000 + size = 27 + sample #77: + track = 1 + timeUs = 1160997 + flags = 1 + size = 4 + sample #78: + track = 0 + timeUs = 1160000 + flags = 67108864 + size = 18 + sample #79: + track = 1 + timeUs = 1184217 + flags = 1 + size = 4 + sample #80: + track = 1 + timeUs = 1207437 + flags = 1 + size = 4 + sample #81: + track = 0 + timeUs = 1240000 + flags = 67108864 + size = 18 + sample #82: + track = 1 + timeUs = 1230657 + flags = 1 + size = 4 + sample #83: + track = 1 + timeUs = 1253877 + flags = 1 + size = 4 + sample #84: + track = 0 + timeUs = 1440000 + size = 28 + sample #85: + track = 1 + timeUs = 1277097 + flags = 1 + size = 4 + sample #86: + track = 1 + timeUs = 1300317 + flags = 1 + size = 4 + sample #87: + track = 0 + timeUs = 1360000 + size = 27 + sample #88: + track = 1 + timeUs = 1323537 + flags = 1 + size = 4 + sample #89: + track = 0 + timeUs = 1320000 + flags = 67108864 + size = 18 + sample #90: + track = 1 + timeUs = 1346757 + flags = 1 + size = 4 + sample #91: + track = 1 + timeUs = 1369977 + flags = 1 + size = 4 + sample #92: + track = 0 + timeUs = 1400000 + flags = 67108864 + size = 18 + sample #93: + track = 1 + timeUs = 1393197 + flags = 1 + size = 4 + sample #94: + track = 1 + timeUs = 1416417 + flags = 1 + size = 4 + sample #95: + track = 0 + timeUs = 1600000 + size = 28 + sample #96: + track = 1 + timeUs = 1439637 + flags = 1 + size = 4 + sample #97: + track = 1 + timeUs = 1462857 + flags = 1 + size = 4 + sample #98: + track = 0 + timeUs = 1520000 + size = 27 + sample #99: + track = 1 + timeUs = 1486077 + flags = 1 + size = 4 + sample #100: + track = 0 + timeUs = 1480000 + flags = 67108864 + size = 18 + sample #101: + track = 1 + timeUs = 1509297 + flags = 1 + size = 4 + sample #102: + track = 1 + timeUs = 1532517 + flags = 1 + size = 4 + sample #103: + track = 0 + timeUs = 1560000 + flags = 67108864 + size = 18 + sample #104: + track = 1 + timeUs = 1555736 + flags = 1 + size = 4 + sample #105: + track = 1 + timeUs = 1578956 + flags = 1 + size = 4 + sample #106: + track = 0 + timeUs = 1760000 + size = 28 + sample #107: + track = 1 + timeUs = 1602176 + flags = 1 + size = 4 + sample #108: + track = 0 + timeUs = 1680000 + size = 27 + sample #109: + track = 1 + timeUs = 1625396 + flags = 1 + size = 4 + sample #110: + track = 1 + timeUs = 1648616 + flags = 1 + size = 4 + sample #111: + track = 0 + timeUs = 1640000 + flags = 67108864 + size = 18 + sample #112: + track = 1 + timeUs = 1671836 + flags = 1 + size = 4 + sample #113: + track = 1 + timeUs = 1695056 + flags = 1 + size = 4 + sample #114: + track = 0 + timeUs = 1720000 + flags = 67108864 + size = 18 + sample #115: + track = 1 + timeUs = 1718276 + flags = 1 + size = 4 + sample #116: + track = 1 + timeUs = 1741496 + flags = 1 + size = 4 + sample #117: + track = 0 + timeUs = 1920000 + size = 28 + sample #118: + track = 1 + timeUs = 1764716 + flags = 1 + size = 4 + sample #119: + track = 0 + timeUs = 1840000 + size = 27 + sample #120: + track = 1 + timeUs = 1787936 + flags = 1 + size = 4 + sample #121: + track = 1 + timeUs = 1811156 + flags = 1 + size = 4 + sample #122: + track = 0 + timeUs = 1800000 + flags = 67108864 + size = 18 + sample #123: + track = 1 + timeUs = 1834376 + flags = 1 + size = 4 + sample #124: + track = 1 + timeUs = 1857596 + flags = 1 + size = 4 + sample #125: + track = 0 + timeUs = 1880000 + flags = 67108864 + size = 18 + sample #126: + track = 1 + timeUs = 1880816 + flags = 1 + size = 4 + sample #127: + track = 0 + timeUs = 2080000 + size = 28 + sample #128: + track = 1 + timeUs = 1904036 + flags = 1 + size = 4 + sample #129: + track = 1 + timeUs = 1927256 + flags = 1 + size = 4 + sample #130: + track = 0 + timeUs = 2000000 + size = 27 + sample #131: + track = 1 + timeUs = 1950476 + flags = 1 + size = 4 + sample #132: + track = 1 + timeUs = 1973696 + flags = 1 + size = 4 + sample #133: + track = 0 + timeUs = 1960000 + flags = 67108864 + size = 18 + sample #134: + track = 1 + timeUs = 1996916 + flags = 1 + size = 4 + sample #135: + track = 1 + timeUs = 2020136 + flags = 1 + size = 4 + sample #136: + track = 0 + timeUs = 2040000 + flags = 67108864 + size = 18 + sample #137: + track = 1 + timeUs = 2043356 + flags = 1 + size = 4 + sample #138: + track = 0 + timeUs = 2240000 + size = 28 + sample #139: + track = 1 + timeUs = 2066575 + flags = 1 + size = 4 + sample #140: + track = 1 + timeUs = 2089795 + flags = 1 + size = 4 + sample #141: + track = 0 + timeUs = 2160000 + size = 27 + sample #142: + track = 1 + timeUs = 2113015 + flags = 1 + size = 4 + sample #143: + track = 1 + timeUs = 2136235 + flags = 1 + size = 4 + sample #144: + track = 0 + timeUs = 2120000 + flags = 67108864 + size = 18 + sample #145: + track = 1 + timeUs = 2159455 + flags = 1 + size = 4 + sample #146: + track = 1 + timeUs = 2182675 + flags = 1 + size = 4 + sample #147: + track = 0 + timeUs = 2200000 + flags = 67108864 + size = 18 + sample #148: + track = 1 + timeUs = 2205895 + flags = 1 + size = 4 + sample #149: + track = 0 + timeUs = 2400000 + size = 28 + sample #150: + track = 1 + timeUs = 2229115 + flags = 1 + size = 4 + sample #151: + track = 1 + timeUs = 2252335 + flags = 1 + size = 4 + sample #152: + track = 0 + timeUs = 2320000 + size = 27 + sample #153: + track = 1 + timeUs = 2275555 + flags = 1 + size = 4 + sample #154: + track = 1 + timeUs = 2298775 + flags = 1 + size = 4 + sample #155: + track = 0 + timeUs = 2280000 + flags = 67108864 + size = 18 + sample #156: + track = 1 + timeUs = 2321995 + flags = 1 + size = 4 + sample #157: + track = 0 + timeUs = 2360000 + flags = 67108864 + size = 18 + sample #158: + track = 1 + timeUs = 2345215 + flags = 1 + size = 4 + sample #159: + track = 1 + timeUs = 2368435 + flags = 1 + size = 4 + sample #160: + track = 0 + timeUs = 2560000 + size = 28 + sample #161: + track = 1 + timeUs = 2391655 + flags = 1 + size = 4 + sample #162: + track = 1 + timeUs = 2414875 + flags = 1 + size = 4 + sample #163: + track = 0 + timeUs = 2480000 + size = 27 + sample #164: + track = 1 + timeUs = 2438095 + flags = 1 + size = 4 + sample #165: + track = 1 + timeUs = 2461315 + flags = 1 + size = 4 + sample #166: + track = 0 + timeUs = 2440000 + flags = 67108864 + size = 18 + sample #167: + track = 1 + timeUs = 2484535 + flags = 1 + size = 4 + sample #168: + track = 0 + timeUs = 2520000 + flags = 67108864 + size = 18 + sample #169: + track = 1 + timeUs = 2507755 + flags = 1 + size = 4 + sample #170: + track = 1 + timeUs = 2530975 + flags = 1 + size = 4 + sample #171: + track = 0 + timeUs = 2720000 + size = 28 + sample #172: + track = 1 + timeUs = 2554195 + flags = 1 + size = 4 + sample #173: + track = 1 + timeUs = 2577414 + flags = 1 + size = 4 + sample #174: + track = 0 + timeUs = 2640000 + size = 27 + sample #175: + track = 1 + timeUs = 2600634 + flags = 1 + size = 4 + sample #176: + track = 0 + timeUs = 2600000 + flags = 67108864 + size = 18 + sample #177: + track = 1 + timeUs = 2623854 + flags = 1 + size = 4 + sample #178: + track = 1 + timeUs = 2647074 + flags = 1 + size = 4 + sample #179: + track = 0 + timeUs = 2680000 + flags = 67108864 + size = 18 + sample #180: + track = 1 + timeUs = 2670294 + flags = 1 + size = 4 + sample #181: + track = 1 + timeUs = 2693514 + flags = 1 + size = 4 + sample #182: + track = 0 + timeUs = 2880000 + size = 28 + sample #183: + track = 1 + timeUs = 2716734 + flags = 1 + size = 4 + sample #184: + track = 1 + timeUs = 2739954 + flags = 1 + size = 4 + sample #185: + track = 0 + timeUs = 2800000 + size = 27 + sample #186: + track = 1 + timeUs = 2763174 + flags = 1 + size = 4 + sample #187: + track = 0 + timeUs = 2760000 + flags = 67108864 + size = 18 + sample #188: + track = 1 + timeUs = 2786394 + flags = 1 + size = 4 + sample #189: + track = 1 + timeUs = 2809614 + flags = 1 + size = 4 + sample #190: + track = 0 + timeUs = 2840000 + flags = 67108864 + size = 18 + sample #191: + track = 1 + timeUs = 2832834 + flags = 1 + size = 4 + sample #192: + track = 1 + timeUs = 2856054 + flags = 1 + size = 4 + sample #193: + track = 0 + timeUs = 3040000 + size = 28 + sample #194: + track = 1 + timeUs = 2879274 + flags = 1 + size = 4 + sample #195: + track = 1 + timeUs = 2902494 + flags = 1 + size = 4 + sample #196: + track = 0 + timeUs = 2960000 + size = 27 + sample #197: + track = 1 + timeUs = 2925714 + flags = 1 + size = 4 + sample #198: + track = 0 + timeUs = 2920000 + flags = 67108864 + size = 18 + sample #199: + track = 1 + timeUs = 2948934 + flags = 1 + size = 4 + sample #200: + track = 1 + timeUs = 2972154 + flags = 1 + size = 4 + sample #201: + track = 0 + timeUs = 3000000 + flags = 67108864 + size = 18 + sample #202: + track = 1 + timeUs = 2995374 + flags = 1 + size = 4 + sample #203: + track = 1 + timeUs = 3018594 + flags = 1 + size = 4 + sample #204: + track = 0 + timeUs = 3200000 + size = 28 + sample #205: + track = 1 + timeUs = 3041814 + flags = 1 + size = 4 + sample #206: + track = 0 + timeUs = 3120000 + size = 27 +MediaCodecAdapter (media3.audio.aac): + inputBuffers: + count = 83 + input buffer #0: + timeUs = 1000000116099 + contents = length 4, hash EE9DF + input buffer #1: + timeUs = 1000000139319 + contents = length 4, hash EE9DF + input buffer #2: + timeUs = 1000000162539 + contents = length 4, hash EE9DF + input buffer #3: + timeUs = 1000000185759 + contents = length 4, hash EE9DF + input buffer #4: + timeUs = 1000000208979 + contents = length 4, hash EE9DF + input buffer #5: + timeUs = 1000000232199 + contents = length 4, hash EE9DF + input buffer #6: + timeUs = 1000000255419 + contents = length 4, hash EE9DF + input buffer #7: + timeUs = 1000000278639 + contents = length 4, hash EE9DF + input buffer #8: + timeUs = 1000000301859 + contents = length 4, hash EE9DF + input buffer #9: + timeUs = 1000000325079 + contents = length 4, hash EE9DF + input buffer #10: + timeUs = 1000000348299 + contents = length 4, hash EE9DF + input buffer #11: + timeUs = 1000000371519 + contents = length 4, hash EE9DF + input buffer #12: + timeUs = 1000000394739 + contents = length 4, hash EE9DF + input buffer #13: + timeUs = 1000000417959 + contents = length 4, hash EE9DF + input buffer #14: + timeUs = 1000000441179 + contents = length 4, hash EE9DF + input buffer #15: + timeUs = 1000000464399 + contents = length 4, hash EE9DF + input buffer #16: + timeUs = 1000000487619 + contents = length 4, hash EE9DF + input buffer #17: + timeUs = 1000000510839 + contents = length 4, hash EE9DF + input buffer #18: + timeUs = 1000000534058 + contents = length 4, hash EE9DF + input buffer #19: + timeUs = 1000000557278 + contents = length 4, hash EE9DF + input buffer #20: + timeUs = 1000000580498 + contents = length 4, hash EE9DF + input buffer #21: + timeUs = 1000000603718 + contents = length 4, hash EE9DF + input buffer #22: + timeUs = 1000000626938 + contents = length 4, hash EE9DF + input buffer #23: + timeUs = 1000000650158 + contents = length 4, hash EE9DF + input buffer #24: + timeUs = 1000000673378 + contents = length 4, hash EE9DF + input buffer #25: + timeUs = 1000000696598 + contents = length 4, hash EE9DF + input buffer #26: + timeUs = 1000000719818 + contents = length 4, hash EE9DF + input buffer #27: + timeUs = 1000000743038 + contents = length 4, hash EE9DF + input buffer #28: + timeUs = 1000000766258 + contents = length 4, hash EE9DF + input buffer #29: + timeUs = 1000000789478 + contents = length 4, hash EE9DF + input buffer #30: + timeUs = 1000000812698 + contents = length 4, hash EE9DF + input buffer #31: + timeUs = 1000000835918 + contents = length 4, hash EE9DF + input buffer #32: + timeUs = 1000000859138 + contents = length 4, hash EE9DF + input buffer #33: + timeUs = 1000000882358 + contents = length 4, hash EE9DF + input buffer #34: + timeUs = 1000000905578 + contents = length 4, hash EE9DF + input buffer #35: + timeUs = 1000000928798 + contents = length 4, hash EE9DF + input buffer #36: + timeUs = 1000000952018 + contents = length 4, hash EE9DF + input buffer #37: + timeUs = 1000000975238 + contents = length 4, hash EE9DF + input buffer #38: + timeUs = 1000000998458 + contents = length 4, hash EE9DF + input buffer #39: + timeUs = 1000001021678 + contents = length 4, hash EE9DF + input buffer #40: + timeUs = 1000001044897 + contents = length 4, hash EE9DF + input buffer #41: + timeUs = 1000001068117 + contents = length 4, hash EE9DF + input buffer #42: + timeUs = 1000001091337 + contents = length 4, hash EE9DF + input buffer #43: + timeUs = 1000001114557 + contents = length 4, hash EE9DF + input buffer #44: + timeUs = 1000001137777 + contents = length 4, hash EE9DF + input buffer #45: + timeUs = 1000001160997 + contents = length 4, hash EE9DF + input buffer #46: + timeUs = 1000001184217 + contents = length 4, hash EE9DF + input buffer #47: + timeUs = 1000001207437 + contents = length 4, hash EE9DF + input buffer #48: + timeUs = 1000001230657 + contents = length 4, hash EE9DF + input buffer #49: + timeUs = 1000001253877 + contents = length 4, hash EE9DF + input buffer #50: + timeUs = 1000001277097 + contents = length 4, hash EE9DF + input buffer #51: + timeUs = 1000001300317 + contents = length 4, hash EE9DF + input buffer #52: + timeUs = 1000001323537 + contents = length 4, hash EE9DF + input buffer #53: + timeUs = 1000001346757 + contents = length 4, hash EE9DF + input buffer #54: + timeUs = 1000001369977 + contents = length 4, hash EE9DF + input buffer #55: + timeUs = 1000001393197 + contents = length 4, hash EE9DF + input buffer #56: + timeUs = 1000001416417 + contents = length 4, hash EE9DF + input buffer #57: + timeUs = 1000001439637 + contents = length 4, hash EE9DF + input buffer #58: + timeUs = 1000001462857 + contents = length 4, hash EE9DF + input buffer #59: + timeUs = 1000001486077 + contents = length 4, hash EE9DF + input buffer #60: + timeUs = 1000001509297 + contents = length 4, hash EE9DF + input buffer #61: + timeUs = 1000001532517 + contents = length 4, hash EE9DF + input buffer #62: + timeUs = 1000001555736 + contents = length 4, hash EE9DF + input buffer #63: + timeUs = 1000001578956 + contents = length 4, hash EE9DF + input buffer #64: + timeUs = 1000001602176 + contents = length 4, hash EE9DF + input buffer #65: + timeUs = 1000001625396 + contents = length 4, hash EE9DF + input buffer #66: + timeUs = 1000001648616 + contents = length 4, hash EE9DF + input buffer #67: + timeUs = 1000001671836 + contents = length 4, hash EE9DF + input buffer #68: + timeUs = 1000001695056 + contents = length 4, hash EE9DF + input buffer #69: + timeUs = 1000001718276 + contents = length 4, hash EE9DF + input buffer #70: + timeUs = 1000001741496 + contents = length 4, hash EE9DF + input buffer #71: + timeUs = 1000001764716 + contents = length 4, hash EE9DF + input buffer #72: + timeUs = 1000001787936 + contents = length 4, hash EE9DF + input buffer #73: + timeUs = 1000001811156 + contents = length 4, hash EE9DF + input buffer #74: + timeUs = 1000001834376 + contents = length 4, hash EE9DF + input buffer #75: + timeUs = 1000001857596 + contents = length 4, hash EE9DF + input buffer #76: + timeUs = 1000001880816 + contents = length 4, hash EE9DF + input buffer #77: + timeUs = 1000001904036 + contents = length 4, hash EE9DF + input buffer #78: + timeUs = 1000001927256 + contents = length 4, hash EE9DF + input buffer #79: + timeUs = 1000001950476 + contents = length 4, hash EE9DF + input buffer #80: + timeUs = 1000001973696 + contents = length 4, hash EE9DF + input buffer #81: + timeUs = 1000001996916 + contents = length 4, hash EE9DF + input buffer #82: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 82 + output buffer #0: + timeUs = 1000000116099 + size = 0 + rendered = false + output buffer #1: + timeUs = 1000000139319 + size = 0 + rendered = false + output buffer #2: + timeUs = 1000000162539 + size = 0 + rendered = false + output buffer #3: + timeUs = 1000000185759 + size = 0 + rendered = false + output buffer #4: + timeUs = 1000000208979 + size = 0 + rendered = false + output buffer #5: + timeUs = 1000000232199 + size = 0 + rendered = false + output buffer #6: + timeUs = 1000000255419 + size = 0 + rendered = false + output buffer #7: + timeUs = 1000000278639 + size = 0 + rendered = false + output buffer #8: + timeUs = 1000000301859 + size = 0 + rendered = false + output buffer #9: + timeUs = 1000000325079 + size = 0 + rendered = false + output buffer #10: + timeUs = 1000000348299 + size = 0 + rendered = false + output buffer #11: + timeUs = 1000000371519 + size = 0 + rendered = false + output buffer #12: + timeUs = 1000000394739 + size = 0 + rendered = false + output buffer #13: + timeUs = 1000000417959 + size = 0 + rendered = false + output buffer #14: + timeUs = 1000000441179 + size = 0 + rendered = false + output buffer #15: + timeUs = 1000000464399 + size = 0 + rendered = false + output buffer #16: + timeUs = 1000000487619 + size = 0 + rendered = false + output buffer #17: + timeUs = 1000000510839 + size = 0 + rendered = false + output buffer #18: + timeUs = 1000000534058 + size = 0 + rendered = false + output buffer #19: + timeUs = 1000000557278 + size = 0 + rendered = false + output buffer #20: + timeUs = 1000000580498 + size = 0 + rendered = false + output buffer #21: + timeUs = 1000000603718 + size = 0 + rendered = false + output buffer #22: + timeUs = 1000000626938 + size = 0 + rendered = false + output buffer #23: + timeUs = 1000000650158 + size = 0 + rendered = false + output buffer #24: + timeUs = 1000000673378 + size = 0 + rendered = false + output buffer #25: + timeUs = 1000000696598 + size = 0 + rendered = false + output buffer #26: + timeUs = 1000000719818 + size = 0 + rendered = false + output buffer #27: + timeUs = 1000000743038 + size = 0 + rendered = false + output buffer #28: + timeUs = 1000000766258 + size = 0 + rendered = false + output buffer #29: + timeUs = 1000000789478 + size = 0 + rendered = false + output buffer #30: + timeUs = 1000000812698 + size = 0 + rendered = false + output buffer #31: + timeUs = 1000000835918 + size = 0 + rendered = false + output buffer #32: + timeUs = 1000000859138 + size = 0 + rendered = false + output buffer #33: + timeUs = 1000000882358 + size = 0 + rendered = false + output buffer #34: + timeUs = 1000000905578 + size = 0 + rendered = false + output buffer #35: + timeUs = 1000000928798 + size = 0 + rendered = false + output buffer #36: + timeUs = 1000000952018 + size = 0 + rendered = false + output buffer #37: + timeUs = 1000000975238 + size = 0 + rendered = false + output buffer #38: + timeUs = 1000000998458 + size = 0 + rendered = false + output buffer #39: + timeUs = 1000001021678 + size = 0 + rendered = false + output buffer #40: + timeUs = 1000001044897 + size = 0 + rendered = false + output buffer #41: + timeUs = 1000001068117 + size = 0 + rendered = false + output buffer #42: + timeUs = 1000001091337 + size = 0 + rendered = false + output buffer #43: + timeUs = 1000001114557 + size = 0 + rendered = false + output buffer #44: + timeUs = 1000001137777 + size = 0 + rendered = false + output buffer #45: + timeUs = 1000001160997 + size = 0 + rendered = false + output buffer #46: + timeUs = 1000001184217 + size = 0 + rendered = false + output buffer #47: + timeUs = 1000001207437 + size = 0 + rendered = false + output buffer #48: + timeUs = 1000001230657 + size = 0 + rendered = false + output buffer #49: + timeUs = 1000001253877 + size = 0 + rendered = false + output buffer #50: + timeUs = 1000001277097 + size = 0 + rendered = false + output buffer #51: + timeUs = 1000001300317 + size = 0 + rendered = false + output buffer #52: + timeUs = 1000001323537 + size = 0 + rendered = false + output buffer #53: + timeUs = 1000001346757 + size = 0 + rendered = false + output buffer #54: + timeUs = 1000001369977 + size = 0 + rendered = false + output buffer #55: + timeUs = 1000001393197 + size = 0 + rendered = false + output buffer #56: + timeUs = 1000001416417 + size = 0 + rendered = false + output buffer #57: + timeUs = 1000001439637 + size = 0 + rendered = false + output buffer #58: + timeUs = 1000001462857 + size = 0 + rendered = false + output buffer #59: + timeUs = 1000001486077 + size = 0 + rendered = false + output buffer #60: + timeUs = 1000001509297 + size = 0 + rendered = false + output buffer #61: + timeUs = 1000001532517 + size = 0 + rendered = false + output buffer #62: + timeUs = 1000001555736 + size = 0 + rendered = false + output buffer #63: + timeUs = 1000001578956 + size = 0 + rendered = false + output buffer #64: + timeUs = 1000001602176 + size = 0 + rendered = false + output buffer #65: + timeUs = 1000001625396 + size = 0 + rendered = false + output buffer #66: + timeUs = 1000001648616 + size = 0 + rendered = false + output buffer #67: + timeUs = 1000001671836 + size = 0 + rendered = false + output buffer #68: + timeUs = 1000001695056 + size = 0 + rendered = false + output buffer #69: + timeUs = 1000001718276 + size = 0 + rendered = false + output buffer #70: + timeUs = 1000001741496 + size = 0 + rendered = false + output buffer #71: + timeUs = 1000001764716 + size = 0 + rendered = false + output buffer #72: + timeUs = 1000001787936 + size = 0 + rendered = false + output buffer #73: + timeUs = 1000001811156 + size = 0 + rendered = false + output buffer #74: + timeUs = 1000001834376 + size = 0 + rendered = false + output buffer #75: + timeUs = 1000001857596 + size = 0 + rendered = false + output buffer #76: + timeUs = 1000001880816 + size = 0 + rendered = false + output buffer #77: + timeUs = 1000001904036 + size = 0 + rendered = false + output buffer #78: + timeUs = 1000001927256 + size = 0 + rendered = false + output buffer #79: + timeUs = 1000001950476 + size = 0 + rendered = false + output buffer #80: + timeUs = 1000001973696 + size = 0 + rendered = false + output buffer #81: + timeUs = 1000001996916 + size = 0 + rendered = false +MediaCodecAdapter (media3.video.avc): + inputBuffers: + count = 52 + input buffer #0: + timeUs = 1000000000000 + contents = length 5252, hash 13893A4C + input buffer #1: + timeUs = 1000000160000 + contents = length 44, hash A05B3BEA + input buffer #2: + timeUs = 1000000080000 + contents = length 25, hash 441EC4FE + input buffer #3: + timeUs = 1000000120000 + contents = length 18, hash B64DA059 + input buffer #4: + timeUs = 1000000320000 + contents = length 32, hash 5BFD9538 + input buffer #5: + timeUs = 1000000240000 + contents = length 27, hash 705869A5 + input buffer #6: + timeUs = 1000000200000 + contents = length 18, hash D163DF61 + input buffer #7: + timeUs = 1000000280000 + contents = length 18, hash FD82E95 + input buffer #8: + timeUs = 1000000480000 + contents = length 28, hash 44A16E72 + input buffer #9: + timeUs = 1000000400000 + contents = length 27, hash FFA326ED + input buffer #10: + timeUs = 1000000360000 + contents = length 18, hash DC93CC9D + input buffer #11: + timeUs = 1000000440000 + contents = length 18, hash 1B081BD1 + input buffer #12: + timeUs = 1000000640000 + contents = length 28, hash 2700AF + input buffer #13: + timeUs = 1000000560000 + contents = length 27, hash DECA17D0 + input buffer #14: + timeUs = 1000000520000 + contents = length 18, hash D646C05A + input buffer #15: + timeUs = 1000000600000 + contents = length 18, hash 14BB0F8E + input buffer #16: + timeUs = 1000000800000 + contents = length 28, hash 5DE2C2B + input buffer #17: + timeUs = 1000000720000 + contents = length 27, hash 376B4E14 + input buffer #18: + timeUs = 1000000680000 + contents = length 18, hash E176AD96 + input buffer #19: + timeUs = 1000000760000 + contents = length 18, hash 1FEAFCCA + input buffer #20: + timeUs = 1000000960000 + contents = length 28, hash C163BE68 + input buffer #21: + timeUs = 1000000880000 + contents = length 27, hash F7170357 + input buffer #22: + timeUs = 1000000840000 + contents = length 18, hash 3B013BD2 + input buffer #23: + timeUs = 1000000920000 + contents = length 18, hash 79758B06 + input buffer #24: + timeUs = 1000001120000 + contents = length 28, hash F72EB1A3 + input buffer #25: + timeUs = 1000001040000 + contents = length 27, hash 4FB8399C + input buffer #26: + timeUs = 1000001000000 + contents = length 18, hash 4631290E + input buffer #27: + timeUs = 1000001080000 + contents = length 18, hash 84A57842 + input buffer #28: + timeUs = 1000001280000 + contents = length 28, hash E1FCF000 + input buffer #29: + timeUs = 1000001200000 + contents = length 27, hash 21D3B540 + input buffer #30: + timeUs = 1000001160000 + contents = length 18, hash 62DE0FC9 + input buffer #31: + timeUs = 1000001240000 + contents = length 18, hash A1525EFD + input buffer #32: + timeUs = 1000001440000 + contents = length 28, hash 5350E8FA + input buffer #33: + timeUs = 1000001360000 + contents = length 27, hash E7563B65 + input buffer #34: + timeUs = 1000001320000 + contents = length 18, hash 77D95125 + input buffer #35: + timeUs = 1000001400000 + contents = length 18, hash B64DA059 + input buffer #36: + timeUs = 1000001600000 + contents = length 28, hash ED67B37 + input buffer #37: + timeUs = 1000001520000 + contents = length 27, hash A701F0A9 + input buffer #38: + timeUs = 1000001480000 + contents = length 18, hash D163DF61 + input buffer #39: + timeUs = 1000001560000 + contents = length 18, hash FD82E95 + input buffer #40: + timeUs = 1000001760000 + contents = length 28, hash 44A16E72 + input buffer #41: + timeUs = 1000001680000 + contents = length 27, hash FFA326ED + input buffer #42: + timeUs = 1000001640000 + contents = length 18, hash DC93CC9D + input buffer #43: + timeUs = 1000001720000 + contents = length 18, hash 1B081BD1 + input buffer #44: + timeUs = 1000001920000 + contents = length 28, hash 2700AF + input buffer #45: + timeUs = 1000001840000 + contents = length 27, hash DECA17D0 + input buffer #46: + timeUs = 1000001800000 + contents = length 18, hash D646C05A + input buffer #47: + timeUs = 1000001880000 + contents = length 18, hash 14BB0F8E + input buffer #48: + timeUs = 1000002080000 + contents = length 28, hash 5DE2C2B + input buffer #49: + timeUs = 1000002000000 + contents = length 27, hash 376B4E14 + input buffer #50: + timeUs = 1000001960000 + contents = length 18, hash E176AD96 + input buffer #51: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 51 + output buffer #0: + timeUs = 1000000000000 + size = 5252 + rendered = false + output buffer #1: + timeUs = 1000000160000 + size = 44 + rendered = true + output buffer #2: + timeUs = 1000000080000 + size = 25 + rendered = false + output buffer #3: + timeUs = 1000000120000 + size = 18 + rendered = true + output buffer #4: + timeUs = 1000000320000 + size = 32 + rendered = true + output buffer #5: + timeUs = 1000000240000 + size = 27 + rendered = true + output buffer #6: + timeUs = 1000000200000 + size = 18 + rendered = true + output buffer #7: + timeUs = 1000000280000 + size = 18 + rendered = true + output buffer #8: + timeUs = 1000000480000 + size = 28 + rendered = true + output buffer #9: + timeUs = 1000000400000 + size = 27 + rendered = true + output buffer #10: + timeUs = 1000000360000 + size = 18 + rendered = true + output buffer #11: + timeUs = 1000000440000 + size = 18 + rendered = true + output buffer #12: + timeUs = 1000000640000 + size = 28 + rendered = true + output buffer #13: + timeUs = 1000000560000 + size = 27 + rendered = true + output buffer #14: + timeUs = 1000000520000 + size = 18 + rendered = true + output buffer #15: + timeUs = 1000000600000 + size = 18 + rendered = true + output buffer #16: + timeUs = 1000000800000 + size = 28 + rendered = true + output buffer #17: + timeUs = 1000000720000 + size = 27 + rendered = true + output buffer #18: + timeUs = 1000000680000 + size = 18 + rendered = true + output buffer #19: + timeUs = 1000000760000 + size = 18 + rendered = true + output buffer #20: + timeUs = 1000000960000 + size = 28 + rendered = true + output buffer #21: + timeUs = 1000000880000 + size = 27 + rendered = true + output buffer #22: + timeUs = 1000000840000 + size = 18 + rendered = true + output buffer #23: + timeUs = 1000000920000 + size = 18 + rendered = true + output buffer #24: + timeUs = 1000001120000 + size = 28 + rendered = true + output buffer #25: + timeUs = 1000001040000 + size = 27 + rendered = true + output buffer #26: + timeUs = 1000001000000 + size = 18 + rendered = true + output buffer #27: + timeUs = 1000001080000 + size = 18 + rendered = true + output buffer #28: + timeUs = 1000001280000 + size = 28 + rendered = true + output buffer #29: + timeUs = 1000001200000 + size = 27 + rendered = true + output buffer #30: + timeUs = 1000001160000 + size = 18 + rendered = true + output buffer #31: + timeUs = 1000001240000 + size = 18 + rendered = true + output buffer #32: + timeUs = 1000001440000 + size = 28 + rendered = true + output buffer #33: + timeUs = 1000001360000 + size = 27 + rendered = true + output buffer #34: + timeUs = 1000001320000 + size = 18 + rendered = true + output buffer #35: + timeUs = 1000001400000 + size = 18 + rendered = true + output buffer #36: + timeUs = 1000001600000 + size = 28 + rendered = true + output buffer #37: + timeUs = 1000001520000 + size = 27 + rendered = true + output buffer #38: + timeUs = 1000001480000 + size = 18 + rendered = true + output buffer #39: + timeUs = 1000001560000 + size = 18 + rendered = true + output buffer #40: + timeUs = 1000001760000 + size = 28 + rendered = true + output buffer #41: + timeUs = 1000001680000 + size = 27 + rendered = true + output buffer #42: + timeUs = 1000001640000 + size = 18 + rendered = true + output buffer #43: + timeUs = 1000001720000 + size = 18 + rendered = true + output buffer #44: + timeUs = 1000001920000 + size = 28 + rendered = true + output buffer #45: + timeUs = 1000001840000 + size = 27 + rendered = true + output buffer #46: + timeUs = 1000001800000 + size = 18 + rendered = true + output buffer #47: + timeUs = 1000001880000 + size = 18 + rendered = true + output buffer #48: + timeUs = 1000002080000 + size = 28 + rendered = false + output buffer #49: + timeUs = 1000002000000 + size = 27 + rendered = false + output buffer #50: + timeUs = 1000001960000 + size = 18 + rendered = true +AudioSink: + buffer count = 82 + config: + pcmEncoding = pcm-16 + channelCount = 1 + sampleRate = 44100 + buffer #0: + time = 1000000116099 + data = empty + buffer #1: + time = 1000000139319 + data = empty + buffer #2: + time = 1000000162539 + data = empty + buffer #3: + time = 1000000185759 + data = empty + buffer #4: + time = 1000000208979 + data = empty + buffer #5: + time = 1000000232199 + data = empty + buffer #6: + time = 1000000255419 + data = empty + buffer #7: + time = 1000000278639 + data = empty + buffer #8: + time = 1000000301859 + data = empty + buffer #9: + time = 1000000325079 + data = empty + buffer #10: + time = 1000000348299 + data = empty + buffer #11: + time = 1000000371519 + data = empty + buffer #12: + time = 1000000394739 + data = empty + buffer #13: + time = 1000000417959 + data = empty + buffer #14: + time = 1000000441179 + data = empty + buffer #15: + time = 1000000464399 + data = empty + buffer #16: + time = 1000000487619 + data = empty + buffer #17: + time = 1000000510839 + data = empty + buffer #18: + time = 1000000534058 + data = empty + buffer #19: + time = 1000000557278 + data = empty + buffer #20: + time = 1000000580498 + data = empty + buffer #21: + time = 1000000603718 + data = empty + buffer #22: + time = 1000000626938 + data = empty + buffer #23: + time = 1000000650158 + data = empty + buffer #24: + time = 1000000673378 + data = empty + buffer #25: + time = 1000000696598 + data = empty + buffer #26: + time = 1000000719818 + data = empty + buffer #27: + time = 1000000743038 + data = empty + buffer #28: + time = 1000000766258 + data = empty + buffer #29: + time = 1000000789478 + data = empty + buffer #30: + time = 1000000812698 + data = empty + buffer #31: + time = 1000000835918 + data = empty + buffer #32: + time = 1000000859138 + data = empty + buffer #33: + time = 1000000882358 + data = empty + buffer #34: + time = 1000000905578 + data = empty + buffer #35: + time = 1000000928798 + data = empty + buffer #36: + time = 1000000952018 + data = empty + buffer #37: + time = 1000000975238 + data = empty + buffer #38: + time = 1000000998458 + data = empty + buffer #39: + time = 1000001021678 + data = empty + buffer #40: + time = 1000001044897 + data = empty + buffer #41: + time = 1000001068117 + data = empty + buffer #42: + time = 1000001091337 + data = empty + buffer #43: + time = 1000001114557 + data = empty + buffer #44: + time = 1000001137777 + data = empty + buffer #45: + time = 1000001160997 + data = empty + buffer #46: + time = 1000001184217 + data = empty + buffer #47: + time = 1000001207437 + data = empty + buffer #48: + time = 1000001230657 + data = empty + buffer #49: + time = 1000001253877 + data = empty + buffer #50: + time = 1000001277097 + data = empty + buffer #51: + time = 1000001300317 + data = empty + buffer #52: + time = 1000001323537 + data = empty + buffer #53: + time = 1000001346757 + data = empty + buffer #54: + time = 1000001369977 + data = empty + buffer #55: + time = 1000001393197 + data = empty + buffer #56: + time = 1000001416417 + data = empty + buffer #57: + time = 1000001439637 + data = empty + buffer #58: + time = 1000001462857 + data = empty + buffer #59: + time = 1000001486077 + data = empty + buffer #60: + time = 1000001509297 + data = empty + buffer #61: + time = 1000001532517 + data = empty + buffer #62: + time = 1000001555736 + data = empty + buffer #63: + time = 1000001578956 + data = empty + buffer #64: + time = 1000001602176 + data = empty + buffer #65: + time = 1000001625396 + data = empty + buffer #66: + time = 1000001648616 + data = empty + buffer #67: + time = 1000001671836 + data = empty + buffer #68: + time = 1000001695056 + data = empty + buffer #69: + time = 1000001718276 + data = empty + buffer #70: + time = 1000001741496 + data = empty + buffer #71: + time = 1000001764716 + data = empty + buffer #72: + time = 1000001787936 + data = empty + buffer #73: + time = 1000001811156 + data = empty + buffer #74: + time = 1000001834376 + data = empty + buffer #75: + time = 1000001857596 + data = empty + buffer #76: + time = 1000001880816 + data = empty + buffer #77: + time = 1000001904036 + data = empty + buffer #78: + time = 1000001927256 + data = empty + buffer #79: + time = 1000001950476 + data = empty + buffer #80: + time = 1000001973696 + data = empty + buffer #81: + time = 1000001996916 + data = empty diff --git a/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_reduce_before_load_finished_to_already_loaded_value_enabled.dump b/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_reduce_before_load_finished_to_already_loaded_value_enabled.dump new file mode 100644 index 00000000000..bab8fd3fa3c --- /dev/null +++ b/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_reduce_before_load_finished_to_already_loaded_value_enabled.dump @@ -0,0 +1,1976 @@ +extracted samples: + sample #0: + track = 0 + timeUs = 0 + flags = 1 + size = 5252 + sample #1: + track = 0 + timeUs = 160000 + size = 44 + sample #2: + track = 0 + timeUs = 80000 + size = 25 + sample #3: + track = 0 + timeUs = 40000 + flags = 67108864 + size = 18 + sample #4: + track = 1 + timeUs = 92879 + flags = 1 + size = 4 + sample #5: + track = 0 + timeUs = 120000 + flags = 67108864 + size = 18 + sample #6: + track = 1 + timeUs = 116099 + flags = 1 + size = 4 + sample #7: + track = 1 + timeUs = 139319 + flags = 1 + size = 4 + sample #8: + track = 0 + timeUs = 320000 + size = 32 + sample #9: + track = 1 + timeUs = 162539 + flags = 1 + size = 4 + sample #10: + track = 0 + timeUs = 240000 + size = 27 + sample #11: + track = 1 + timeUs = 185759 + flags = 1 + size = 4 + sample #12: + track = 1 + timeUs = 208979 + flags = 1 + size = 4 + sample #13: + track = 0 + timeUs = 200000 + flags = 67108864 + size = 18 + sample #14: + track = 1 + timeUs = 232199 + flags = 1 + size = 4 + sample #15: + track = 1 + timeUs = 255419 + flags = 1 + size = 4 + sample #16: + track = 0 + timeUs = 280000 + flags = 67108864 + size = 18 + sample #17: + track = 1 + timeUs = 278639 + flags = 1 + size = 4 + sample #18: + track = 1 + timeUs = 301859 + flags = 1 + size = 4 + sample #19: + track = 0 + timeUs = 480000 + size = 28 + sample #20: + track = 1 + timeUs = 325079 + flags = 1 + size = 4 + sample #21: + track = 0 + timeUs = 400000 + size = 27 + sample #22: + track = 1 + timeUs = 348299 + flags = 1 + size = 4 + sample #23: + track = 1 + timeUs = 371519 + flags = 1 + size = 4 + sample #24: + track = 0 + timeUs = 360000 + flags = 67108864 + size = 18 + sample #25: + track = 1 + timeUs = 394739 + flags = 1 + size = 4 + sample #26: + track = 1 + timeUs = 417959 + flags = 1 + size = 4 + sample #27: + track = 0 + timeUs = 440000 + flags = 67108864 + size = 18 + sample #28: + track = 1 + timeUs = 441179 + flags = 1 + size = 4 + sample #29: + track = 0 + timeUs = 640000 + size = 28 + sample #30: + track = 1 + timeUs = 464399 + flags = 1 + size = 4 + sample #31: + track = 1 + timeUs = 487619 + flags = 1 + size = 4 + sample #32: + track = 0 + timeUs = 560000 + size = 27 + sample #33: + track = 1 + timeUs = 510839 + flags = 1 + size = 4 + sample #34: + track = 1 + timeUs = 534058 + flags = 1 + size = 4 + sample #35: + track = 0 + timeUs = 520000 + flags = 67108864 + size = 18 + sample #36: + track = 1 + timeUs = 557278 + flags = 1 + size = 4 + sample #37: + track = 1 + timeUs = 580498 + flags = 1 + size = 4 + sample #38: + track = 0 + timeUs = 600000 + flags = 67108864 + size = 18 + sample #39: + track = 1 + timeUs = 603718 + flags = 1 + size = 4 + sample #40: + track = 0 + timeUs = 800000 + size = 28 + sample #41: + track = 1 + timeUs = 626938 + flags = 1 + size = 4 + sample #42: + track = 1 + timeUs = 650158 + flags = 1 + size = 4 + sample #43: + track = 0 + timeUs = 720000 + size = 27 + sample #44: + track = 1 + timeUs = 673378 + flags = 1 + size = 4 + sample #45: + track = 1 + timeUs = 696598 + flags = 1 + size = 4 + sample #46: + track = 0 + timeUs = 680000 + flags = 67108864 + size = 18 + sample #47: + track = 1 + timeUs = 719818 + flags = 1 + size = 4 + sample #48: + track = 1 + timeUs = 743038 + flags = 1 + size = 4 + sample #49: + track = 0 + timeUs = 760000 + flags = 67108864 + size = 18 + sample #50: + track = 1 + timeUs = 766258 + flags = 1 + size = 4 + sample #51: + track = 0 + timeUs = 960000 + size = 28 + sample #52: + track = 1 + timeUs = 789478 + flags = 1 + size = 4 + sample #53: + track = 1 + timeUs = 812698 + flags = 1 + size = 4 + sample #54: + track = 0 + timeUs = 880000 + size = 27 + sample #55: + track = 1 + timeUs = 835918 + flags = 1 + size = 4 + sample #56: + track = 1 + timeUs = 859138 + flags = 1 + size = 4 + sample #57: + track = 0 + timeUs = 840000 + flags = 67108864 + size = 18 + sample #58: + track = 1 + timeUs = 882358 + flags = 1 + size = 4 + sample #59: + track = 0 + timeUs = 920000 + flags = 67108864 + size = 18 + sample #60: + track = 1 + timeUs = 905578 + flags = 1 + size = 4 + sample #61: + track = 1 + timeUs = 928798 + flags = 1 + size = 4 + sample #62: + track = 0 + timeUs = 1120000 + size = 28 + sample #63: + track = 1 + timeUs = 952018 + flags = 1 + size = 4 + sample #64: + track = 1 + timeUs = 975238 + flags = 1 + size = 4 + sample #65: + track = 0 + timeUs = 1040000 + size = 27 + sample #66: + track = 1 + timeUs = 998458 + flags = 1 + size = 4 + sample #67: + track = 1 + timeUs = 1021678 + flags = 1 + size = 4 + sample #68: + track = 0 + timeUs = 1000000 + flags = 67108864 + size = 18 + sample #69: + track = 1 + timeUs = 1044897 + flags = 1 + size = 4 + sample #70: + track = 0 + timeUs = 1080000 + flags = 67108864 + size = 18 + sample #71: + track = 1 + timeUs = 1068117 + flags = 1 + size = 4 + sample #72: + track = 1 + timeUs = 1091337 + flags = 1 + size = 4 + sample #73: + track = 0 + timeUs = 1280000 + size = 28 + sample #74: + track = 1 + timeUs = 1114557 + flags = 1 + size = 4 + sample #75: + track = 1 + timeUs = 1137777 + flags = 1 + size = 4 + sample #76: + track = 0 + timeUs = 1200000 + size = 27 + sample #77: + track = 1 + timeUs = 1160997 + flags = 1 + size = 4 + sample #78: + track = 0 + timeUs = 1160000 + flags = 67108864 + size = 18 + sample #79: + track = 1 + timeUs = 1184217 + flags = 1 + size = 4 + sample #80: + track = 1 + timeUs = 1207437 + flags = 1 + size = 4 + sample #81: + track = 0 + timeUs = 1240000 + flags = 67108864 + size = 18 + sample #82: + track = 1 + timeUs = 1230657 + flags = 1 + size = 4 + sample #83: + track = 1 + timeUs = 1253877 + flags = 1 + size = 4 + sample #84: + track = 0 + timeUs = 1440000 + size = 28 + sample #85: + track = 1 + timeUs = 1277097 + flags = 1 + size = 4 + sample #86: + track = 1 + timeUs = 1300317 + flags = 1 + size = 4 + sample #87: + track = 0 + timeUs = 1360000 + size = 27 + sample #88: + track = 1 + timeUs = 1323537 + flags = 1 + size = 4 + sample #89: + track = 0 + timeUs = 1320000 + flags = 67108864 + size = 18 + sample #90: + track = 1 + timeUs = 1346757 + flags = 1 + size = 4 + sample #91: + track = 1 + timeUs = 1369977 + flags = 1 + size = 4 + sample #92: + track = 0 + timeUs = 1400000 + flags = 67108864 + size = 18 + sample #93: + track = 1 + timeUs = 1393197 + flags = 1 + size = 4 + sample #94: + track = 1 + timeUs = 1416417 + flags = 1 + size = 4 + sample #95: + track = 0 + timeUs = 1600000 + size = 28 + sample #96: + track = 1 + timeUs = 1439637 + flags = 1 + size = 4 + sample #97: + track = 1 + timeUs = 1462857 + flags = 1 + size = 4 + sample #98: + track = 0 + timeUs = 1520000 + size = 27 + sample #99: + track = 1 + timeUs = 1486077 + flags = 1 + size = 4 + sample #100: + track = 0 + timeUs = 1480000 + flags = 67108864 + size = 18 + sample #101: + track = 1 + timeUs = 1509297 + flags = 1 + size = 4 + sample #102: + track = 1 + timeUs = 1532517 + flags = 1 + size = 4 + sample #103: + track = 0 + timeUs = 1560000 + flags = 67108864 + size = 18 + sample #104: + track = 1 + timeUs = 1555736 + flags = 1 + size = 4 + sample #105: + track = 1 + timeUs = 1578956 + flags = 1 + size = 4 + sample #106: + track = 0 + timeUs = 1760000 + size = 28 + sample #107: + track = 1 + timeUs = 1602176 + flags = 1 + size = 4 + sample #108: + track = 0 + timeUs = 1680000 + size = 27 + sample #109: + track = 1 + timeUs = 1625396 + flags = 1 + size = 4 + sample #110: + track = 1 + timeUs = 1648616 + flags = 1 + size = 4 + sample #111: + track = 0 + timeUs = 1640000 + flags = 67108864 + size = 18 + sample #112: + track = 1 + timeUs = 1671836 + flags = 1 + size = 4 + sample #113: + track = 1 + timeUs = 1695056 + flags = 1 + size = 4 + sample #114: + track = 0 + timeUs = 1720000 + flags = 67108864 + size = 18 + sample #115: + track = 1 + timeUs = 1718276 + flags = 1 + size = 4 + sample #116: + track = 1 + timeUs = 1741496 + flags = 1 + size = 4 + sample #117: + track = 0 + timeUs = 1920000 + size = 28 + sample #118: + track = 1 + timeUs = 1764716 + flags = 1 + size = 4 + sample #119: + track = 0 + timeUs = 1840000 + size = 27 + sample #120: + track = 1 + timeUs = 1787936 + flags = 1 + size = 4 + sample #121: + track = 1 + timeUs = 1811156 + flags = 1 + size = 4 + sample #122: + track = 0 + timeUs = 1800000 + flags = 67108864 + size = 18 + sample #123: + track = 1 + timeUs = 1834376 + flags = 1 + size = 4 + sample #124: + track = 1 + timeUs = 1857596 + flags = 1 + size = 4 + sample #125: + track = 0 + timeUs = 1880000 + flags = 67108864 + size = 18 + sample #126: + track = 1 + timeUs = 1880816 + flags = 1 + size = 4 + sample #127: + track = 0 + timeUs = 2080000 + size = 28 + sample #128: + track = 1 + timeUs = 1904036 + flags = 1 + size = 4 + sample #129: + track = 1 + timeUs = 1927256 + flags = 1 + size = 4 + sample #130: + track = 0 + timeUs = 2000000 + size = 27 + sample #131: + track = 1 + timeUs = 1950476 + flags = 1 + size = 4 + sample #132: + track = 1 + timeUs = 1973696 + flags = 1 + size = 4 + sample #133: + track = 0 + timeUs = 1960000 + flags = 67108864 + size = 18 + sample #134: + track = 1 + timeUs = 1996916 + flags = 1 + size = 4 + sample #135: + track = 1 + timeUs = 2020136 + flags = 1 + size = 4 + sample #136: + track = 0 + timeUs = 2040000 + flags = 67108864 + size = 18 + sample #137: + track = 1 + timeUs = 2043356 + flags = 1 + size = 4 + sample #138: + track = 0 + timeUs = 2240000 + size = 28 + sample #139: + track = 1 + timeUs = 2066575 + flags = 1 + size = 4 + sample #140: + track = 1 + timeUs = 2089795 + flags = 1 + size = 4 + sample #141: + track = 0 + timeUs = 2160000 + size = 27 + sample #142: + track = 1 + timeUs = 2113015 + flags = 1 + size = 4 + sample #143: + track = 1 + timeUs = 2136235 + flags = 1 + size = 4 + sample #144: + track = 0 + timeUs = 2120000 + flags = 67108864 + size = 18 + sample #145: + track = 1 + timeUs = 2159455 + flags = 1 + size = 4 + sample #146: + track = 1 + timeUs = 2182675 + flags = 1 + size = 4 + sample #147: + track = 0 + timeUs = 2200000 + flags = 67108864 + size = 18 + sample #148: + track = 1 + timeUs = 2205895 + flags = 1 + size = 4 + sample #149: + track = 0 + timeUs = 2400000 + size = 28 + sample #150: + track = 1 + timeUs = 2229115 + flags = 1 + size = 4 + sample #151: + track = 1 + timeUs = 2252335 + flags = 1 + size = 4 + sample #152: + track = 0 + timeUs = 2320000 + size = 27 + sample #153: + track = 1 + timeUs = 2275555 + flags = 1 + size = 4 + sample #154: + track = 1 + timeUs = 2298775 + flags = 1 + size = 4 + sample #155: + track = 0 + timeUs = 2280000 + flags = 67108864 + size = 18 + sample #156: + track = 1 + timeUs = 2321995 + flags = 1 + size = 4 + sample #157: + track = 0 + timeUs = 2360000 + flags = 67108864 + size = 18 + sample #158: + track = 1 + timeUs = 2345215 + flags = 1 + size = 4 + sample #159: + track = 1 + timeUs = 2368435 + flags = 1 + size = 4 + sample #160: + track = 0 + timeUs = 2560000 + size = 28 +MediaCodecAdapter (media3.audio.aac): + inputBuffers: + count = 83 + input buffer #0: + timeUs = 1000000116099 + contents = length 4, hash EE9DF + input buffer #1: + timeUs = 1000000139319 + contents = length 4, hash EE9DF + input buffer #2: + timeUs = 1000000162539 + contents = length 4, hash EE9DF + input buffer #3: + timeUs = 1000000185759 + contents = length 4, hash EE9DF + input buffer #4: + timeUs = 1000000208979 + contents = length 4, hash EE9DF + input buffer #5: + timeUs = 1000000232199 + contents = length 4, hash EE9DF + input buffer #6: + timeUs = 1000000255419 + contents = length 4, hash EE9DF + input buffer #7: + timeUs = 1000000278639 + contents = length 4, hash EE9DF + input buffer #8: + timeUs = 1000000301859 + contents = length 4, hash EE9DF + input buffer #9: + timeUs = 1000000325079 + contents = length 4, hash EE9DF + input buffer #10: + timeUs = 1000000348299 + contents = length 4, hash EE9DF + input buffer #11: + timeUs = 1000000371519 + contents = length 4, hash EE9DF + input buffer #12: + timeUs = 1000000394739 + contents = length 4, hash EE9DF + input buffer #13: + timeUs = 1000000417959 + contents = length 4, hash EE9DF + input buffer #14: + timeUs = 1000000441179 + contents = length 4, hash EE9DF + input buffer #15: + timeUs = 1000000464399 + contents = length 4, hash EE9DF + input buffer #16: + timeUs = 1000000487619 + contents = length 4, hash EE9DF + input buffer #17: + timeUs = 1000000510839 + contents = length 4, hash EE9DF + input buffer #18: + timeUs = 1000000534058 + contents = length 4, hash EE9DF + input buffer #19: + timeUs = 1000000557278 + contents = length 4, hash EE9DF + input buffer #20: + timeUs = 1000000580498 + contents = length 4, hash EE9DF + input buffer #21: + timeUs = 1000000603718 + contents = length 4, hash EE9DF + input buffer #22: + timeUs = 1000000626938 + contents = length 4, hash EE9DF + input buffer #23: + timeUs = 1000000650158 + contents = length 4, hash EE9DF + input buffer #24: + timeUs = 1000000673378 + contents = length 4, hash EE9DF + input buffer #25: + timeUs = 1000000696598 + contents = length 4, hash EE9DF + input buffer #26: + timeUs = 1000000719818 + contents = length 4, hash EE9DF + input buffer #27: + timeUs = 1000000743038 + contents = length 4, hash EE9DF + input buffer #28: + timeUs = 1000000766258 + contents = length 4, hash EE9DF + input buffer #29: + timeUs = 1000000789478 + contents = length 4, hash EE9DF + input buffer #30: + timeUs = 1000000812698 + contents = length 4, hash EE9DF + input buffer #31: + timeUs = 1000000835918 + contents = length 4, hash EE9DF + input buffer #32: + timeUs = 1000000859138 + contents = length 4, hash EE9DF + input buffer #33: + timeUs = 1000000882358 + contents = length 4, hash EE9DF + input buffer #34: + timeUs = 1000000905578 + contents = length 4, hash EE9DF + input buffer #35: + timeUs = 1000000928798 + contents = length 4, hash EE9DF + input buffer #36: + timeUs = 1000000952018 + contents = length 4, hash EE9DF + input buffer #37: + timeUs = 1000000975238 + contents = length 4, hash EE9DF + input buffer #38: + timeUs = 1000000998458 + contents = length 4, hash EE9DF + input buffer #39: + timeUs = 1000001021678 + contents = length 4, hash EE9DF + input buffer #40: + timeUs = 1000001044897 + contents = length 4, hash EE9DF + input buffer #41: + timeUs = 1000001068117 + contents = length 4, hash EE9DF + input buffer #42: + timeUs = 1000001091337 + contents = length 4, hash EE9DF + input buffer #43: + timeUs = 1000001114557 + contents = length 4, hash EE9DF + input buffer #44: + timeUs = 1000001137777 + contents = length 4, hash EE9DF + input buffer #45: + timeUs = 1000001160997 + contents = length 4, hash EE9DF + input buffer #46: + timeUs = 1000001184217 + contents = length 4, hash EE9DF + input buffer #47: + timeUs = 1000001207437 + contents = length 4, hash EE9DF + input buffer #48: + timeUs = 1000001230657 + contents = length 4, hash EE9DF + input buffer #49: + timeUs = 1000001253877 + contents = length 4, hash EE9DF + input buffer #50: + timeUs = 1000001277097 + contents = length 4, hash EE9DF + input buffer #51: + timeUs = 1000001300317 + contents = length 4, hash EE9DF + input buffer #52: + timeUs = 1000001323537 + contents = length 4, hash EE9DF + input buffer #53: + timeUs = 1000001346757 + contents = length 4, hash EE9DF + input buffer #54: + timeUs = 1000001369977 + contents = length 4, hash EE9DF + input buffer #55: + timeUs = 1000001393197 + contents = length 4, hash EE9DF + input buffer #56: + timeUs = 1000001416417 + contents = length 4, hash EE9DF + input buffer #57: + timeUs = 1000001439637 + contents = length 4, hash EE9DF + input buffer #58: + timeUs = 1000001462857 + contents = length 4, hash EE9DF + input buffer #59: + timeUs = 1000001486077 + contents = length 4, hash EE9DF + input buffer #60: + timeUs = 1000001509297 + contents = length 4, hash EE9DF + input buffer #61: + timeUs = 1000001532517 + contents = length 4, hash EE9DF + input buffer #62: + timeUs = 1000001555736 + contents = length 4, hash EE9DF + input buffer #63: + timeUs = 1000001578956 + contents = length 4, hash EE9DF + input buffer #64: + timeUs = 1000001602176 + contents = length 4, hash EE9DF + input buffer #65: + timeUs = 1000001625396 + contents = length 4, hash EE9DF + input buffer #66: + timeUs = 1000001648616 + contents = length 4, hash EE9DF + input buffer #67: + timeUs = 1000001671836 + contents = length 4, hash EE9DF + input buffer #68: + timeUs = 1000001695056 + contents = length 4, hash EE9DF + input buffer #69: + timeUs = 1000001718276 + contents = length 4, hash EE9DF + input buffer #70: + timeUs = 1000001741496 + contents = length 4, hash EE9DF + input buffer #71: + timeUs = 1000001764716 + contents = length 4, hash EE9DF + input buffer #72: + timeUs = 1000001787936 + contents = length 4, hash EE9DF + input buffer #73: + timeUs = 1000001811156 + contents = length 4, hash EE9DF + input buffer #74: + timeUs = 1000001834376 + contents = length 4, hash EE9DF + input buffer #75: + timeUs = 1000001857596 + contents = length 4, hash EE9DF + input buffer #76: + timeUs = 1000001880816 + contents = length 4, hash EE9DF + input buffer #77: + timeUs = 1000001904036 + contents = length 4, hash EE9DF + input buffer #78: + timeUs = 1000001927256 + contents = length 4, hash EE9DF + input buffer #79: + timeUs = 1000001950476 + contents = length 4, hash EE9DF + input buffer #80: + timeUs = 1000001973696 + contents = length 4, hash EE9DF + input buffer #81: + timeUs = 1000001996916 + contents = length 4, hash EE9DF + input buffer #82: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 82 + output buffer #0: + timeUs = 1000000116099 + size = 0 + rendered = false + output buffer #1: + timeUs = 1000000139319 + size = 0 + rendered = false + output buffer #2: + timeUs = 1000000162539 + size = 0 + rendered = false + output buffer #3: + timeUs = 1000000185759 + size = 0 + rendered = false + output buffer #4: + timeUs = 1000000208979 + size = 0 + rendered = false + output buffer #5: + timeUs = 1000000232199 + size = 0 + rendered = false + output buffer #6: + timeUs = 1000000255419 + size = 0 + rendered = false + output buffer #7: + timeUs = 1000000278639 + size = 0 + rendered = false + output buffer #8: + timeUs = 1000000301859 + size = 0 + rendered = false + output buffer #9: + timeUs = 1000000325079 + size = 0 + rendered = false + output buffer #10: + timeUs = 1000000348299 + size = 0 + rendered = false + output buffer #11: + timeUs = 1000000371519 + size = 0 + rendered = false + output buffer #12: + timeUs = 1000000394739 + size = 0 + rendered = false + output buffer #13: + timeUs = 1000000417959 + size = 0 + rendered = false + output buffer #14: + timeUs = 1000000441179 + size = 0 + rendered = false + output buffer #15: + timeUs = 1000000464399 + size = 0 + rendered = false + output buffer #16: + timeUs = 1000000487619 + size = 0 + rendered = false + output buffer #17: + timeUs = 1000000510839 + size = 0 + rendered = false + output buffer #18: + timeUs = 1000000534058 + size = 0 + rendered = false + output buffer #19: + timeUs = 1000000557278 + size = 0 + rendered = false + output buffer #20: + timeUs = 1000000580498 + size = 0 + rendered = false + output buffer #21: + timeUs = 1000000603718 + size = 0 + rendered = false + output buffer #22: + timeUs = 1000000626938 + size = 0 + rendered = false + output buffer #23: + timeUs = 1000000650158 + size = 0 + rendered = false + output buffer #24: + timeUs = 1000000673378 + size = 0 + rendered = false + output buffer #25: + timeUs = 1000000696598 + size = 0 + rendered = false + output buffer #26: + timeUs = 1000000719818 + size = 0 + rendered = false + output buffer #27: + timeUs = 1000000743038 + size = 0 + rendered = false + output buffer #28: + timeUs = 1000000766258 + size = 0 + rendered = false + output buffer #29: + timeUs = 1000000789478 + size = 0 + rendered = false + output buffer #30: + timeUs = 1000000812698 + size = 0 + rendered = false + output buffer #31: + timeUs = 1000000835918 + size = 0 + rendered = false + output buffer #32: + timeUs = 1000000859138 + size = 0 + rendered = false + output buffer #33: + timeUs = 1000000882358 + size = 0 + rendered = false + output buffer #34: + timeUs = 1000000905578 + size = 0 + rendered = false + output buffer #35: + timeUs = 1000000928798 + size = 0 + rendered = false + output buffer #36: + timeUs = 1000000952018 + size = 0 + rendered = false + output buffer #37: + timeUs = 1000000975238 + size = 0 + rendered = false + output buffer #38: + timeUs = 1000000998458 + size = 0 + rendered = false + output buffer #39: + timeUs = 1000001021678 + size = 0 + rendered = false + output buffer #40: + timeUs = 1000001044897 + size = 0 + rendered = false + output buffer #41: + timeUs = 1000001068117 + size = 0 + rendered = false + output buffer #42: + timeUs = 1000001091337 + size = 0 + rendered = false + output buffer #43: + timeUs = 1000001114557 + size = 0 + rendered = false + output buffer #44: + timeUs = 1000001137777 + size = 0 + rendered = false + output buffer #45: + timeUs = 1000001160997 + size = 0 + rendered = false + output buffer #46: + timeUs = 1000001184217 + size = 0 + rendered = false + output buffer #47: + timeUs = 1000001207437 + size = 0 + rendered = false + output buffer #48: + timeUs = 1000001230657 + size = 0 + rendered = false + output buffer #49: + timeUs = 1000001253877 + size = 0 + rendered = false + output buffer #50: + timeUs = 1000001277097 + size = 0 + rendered = false + output buffer #51: + timeUs = 1000001300317 + size = 0 + rendered = false + output buffer #52: + timeUs = 1000001323537 + size = 0 + rendered = false + output buffer #53: + timeUs = 1000001346757 + size = 0 + rendered = false + output buffer #54: + timeUs = 1000001369977 + size = 0 + rendered = false + output buffer #55: + timeUs = 1000001393197 + size = 0 + rendered = false + output buffer #56: + timeUs = 1000001416417 + size = 0 + rendered = false + output buffer #57: + timeUs = 1000001439637 + size = 0 + rendered = false + output buffer #58: + timeUs = 1000001462857 + size = 0 + rendered = false + output buffer #59: + timeUs = 1000001486077 + size = 0 + rendered = false + output buffer #60: + timeUs = 1000001509297 + size = 0 + rendered = false + output buffer #61: + timeUs = 1000001532517 + size = 0 + rendered = false + output buffer #62: + timeUs = 1000001555736 + size = 0 + rendered = false + output buffer #63: + timeUs = 1000001578956 + size = 0 + rendered = false + output buffer #64: + timeUs = 1000001602176 + size = 0 + rendered = false + output buffer #65: + timeUs = 1000001625396 + size = 0 + rendered = false + output buffer #66: + timeUs = 1000001648616 + size = 0 + rendered = false + output buffer #67: + timeUs = 1000001671836 + size = 0 + rendered = false + output buffer #68: + timeUs = 1000001695056 + size = 0 + rendered = false + output buffer #69: + timeUs = 1000001718276 + size = 0 + rendered = false + output buffer #70: + timeUs = 1000001741496 + size = 0 + rendered = false + output buffer #71: + timeUs = 1000001764716 + size = 0 + rendered = false + output buffer #72: + timeUs = 1000001787936 + size = 0 + rendered = false + output buffer #73: + timeUs = 1000001811156 + size = 0 + rendered = false + output buffer #74: + timeUs = 1000001834376 + size = 0 + rendered = false + output buffer #75: + timeUs = 1000001857596 + size = 0 + rendered = false + output buffer #76: + timeUs = 1000001880816 + size = 0 + rendered = false + output buffer #77: + timeUs = 1000001904036 + size = 0 + rendered = false + output buffer #78: + timeUs = 1000001927256 + size = 0 + rendered = false + output buffer #79: + timeUs = 1000001950476 + size = 0 + rendered = false + output buffer #80: + timeUs = 1000001973696 + size = 0 + rendered = false + output buffer #81: + timeUs = 1000001996916 + size = 0 + rendered = false +MediaCodecAdapter (media3.video.avc): + inputBuffers: + count = 52 + input buffer #0: + timeUs = 1000000000000 + contents = length 5252, hash 13893A4C + input buffer #1: + timeUs = 1000000160000 + contents = length 44, hash A05B3BEA + input buffer #2: + timeUs = 1000000080000 + contents = length 25, hash 441EC4FE + input buffer #3: + timeUs = 1000000120000 + contents = length 18, hash B64DA059 + input buffer #4: + timeUs = 1000000320000 + contents = length 32, hash 5BFD9538 + input buffer #5: + timeUs = 1000000240000 + contents = length 27, hash 705869A5 + input buffer #6: + timeUs = 1000000200000 + contents = length 18, hash D163DF61 + input buffer #7: + timeUs = 1000000280000 + contents = length 18, hash FD82E95 + input buffer #8: + timeUs = 1000000480000 + contents = length 28, hash 44A16E72 + input buffer #9: + timeUs = 1000000400000 + contents = length 27, hash FFA326ED + input buffer #10: + timeUs = 1000000360000 + contents = length 18, hash DC93CC9D + input buffer #11: + timeUs = 1000000440000 + contents = length 18, hash 1B081BD1 + input buffer #12: + timeUs = 1000000640000 + contents = length 28, hash 2700AF + input buffer #13: + timeUs = 1000000560000 + contents = length 27, hash DECA17D0 + input buffer #14: + timeUs = 1000000520000 + contents = length 18, hash D646C05A + input buffer #15: + timeUs = 1000000600000 + contents = length 18, hash 14BB0F8E + input buffer #16: + timeUs = 1000000800000 + contents = length 28, hash 5DE2C2B + input buffer #17: + timeUs = 1000000720000 + contents = length 27, hash 376B4E14 + input buffer #18: + timeUs = 1000000680000 + contents = length 18, hash E176AD96 + input buffer #19: + timeUs = 1000000760000 + contents = length 18, hash 1FEAFCCA + input buffer #20: + timeUs = 1000000960000 + contents = length 28, hash C163BE68 + input buffer #21: + timeUs = 1000000880000 + contents = length 27, hash F7170357 + input buffer #22: + timeUs = 1000000840000 + contents = length 18, hash 3B013BD2 + input buffer #23: + timeUs = 1000000920000 + contents = length 18, hash 79758B06 + input buffer #24: + timeUs = 1000001120000 + contents = length 28, hash F72EB1A3 + input buffer #25: + timeUs = 1000001040000 + contents = length 27, hash 4FB8399C + input buffer #26: + timeUs = 1000001000000 + contents = length 18, hash 4631290E + input buffer #27: + timeUs = 1000001080000 + contents = length 18, hash 84A57842 + input buffer #28: + timeUs = 1000001280000 + contents = length 28, hash E1FCF000 + input buffer #29: + timeUs = 1000001200000 + contents = length 27, hash 21D3B540 + input buffer #30: + timeUs = 1000001160000 + contents = length 18, hash 62DE0FC9 + input buffer #31: + timeUs = 1000001240000 + contents = length 18, hash A1525EFD + input buffer #32: + timeUs = 1000001440000 + contents = length 28, hash 5350E8FA + input buffer #33: + timeUs = 1000001360000 + contents = length 27, hash E7563B65 + input buffer #34: + timeUs = 1000001320000 + contents = length 18, hash 77D95125 + input buffer #35: + timeUs = 1000001400000 + contents = length 18, hash B64DA059 + input buffer #36: + timeUs = 1000001600000 + contents = length 28, hash ED67B37 + input buffer #37: + timeUs = 1000001520000 + contents = length 27, hash A701F0A9 + input buffer #38: + timeUs = 1000001480000 + contents = length 18, hash D163DF61 + input buffer #39: + timeUs = 1000001560000 + contents = length 18, hash FD82E95 + input buffer #40: + timeUs = 1000001760000 + contents = length 28, hash 44A16E72 + input buffer #41: + timeUs = 1000001680000 + contents = length 27, hash FFA326ED + input buffer #42: + timeUs = 1000001640000 + contents = length 18, hash DC93CC9D + input buffer #43: + timeUs = 1000001720000 + contents = length 18, hash 1B081BD1 + input buffer #44: + timeUs = 1000001920000 + contents = length 28, hash 2700AF + input buffer #45: + timeUs = 1000001840000 + contents = length 27, hash DECA17D0 + input buffer #46: + timeUs = 1000001800000 + contents = length 18, hash D646C05A + input buffer #47: + timeUs = 1000001880000 + contents = length 18, hash 14BB0F8E + input buffer #48: + timeUs = 1000002080000 + contents = length 28, hash 5DE2C2B + input buffer #49: + timeUs = 1000002000000 + contents = length 27, hash 376B4E14 + input buffer #50: + timeUs = 1000001960000 + contents = length 18, hash E176AD96 + input buffer #51: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 51 + output buffer #0: + timeUs = 1000000000000 + size = 5252 + rendered = false + output buffer #1: + timeUs = 1000000160000 + size = 44 + rendered = true + output buffer #2: + timeUs = 1000000080000 + size = 25 + rendered = false + output buffer #3: + timeUs = 1000000120000 + size = 18 + rendered = true + output buffer #4: + timeUs = 1000000320000 + size = 32 + rendered = true + output buffer #5: + timeUs = 1000000240000 + size = 27 + rendered = true + output buffer #6: + timeUs = 1000000200000 + size = 18 + rendered = true + output buffer #7: + timeUs = 1000000280000 + size = 18 + rendered = true + output buffer #8: + timeUs = 1000000480000 + size = 28 + rendered = true + output buffer #9: + timeUs = 1000000400000 + size = 27 + rendered = true + output buffer #10: + timeUs = 1000000360000 + size = 18 + rendered = true + output buffer #11: + timeUs = 1000000440000 + size = 18 + rendered = true + output buffer #12: + timeUs = 1000000640000 + size = 28 + rendered = true + output buffer #13: + timeUs = 1000000560000 + size = 27 + rendered = true + output buffer #14: + timeUs = 1000000520000 + size = 18 + rendered = true + output buffer #15: + timeUs = 1000000600000 + size = 18 + rendered = true + output buffer #16: + timeUs = 1000000800000 + size = 28 + rendered = true + output buffer #17: + timeUs = 1000000720000 + size = 27 + rendered = true + output buffer #18: + timeUs = 1000000680000 + size = 18 + rendered = true + output buffer #19: + timeUs = 1000000760000 + size = 18 + rendered = true + output buffer #20: + timeUs = 1000000960000 + size = 28 + rendered = true + output buffer #21: + timeUs = 1000000880000 + size = 27 + rendered = true + output buffer #22: + timeUs = 1000000840000 + size = 18 + rendered = true + output buffer #23: + timeUs = 1000000920000 + size = 18 + rendered = true + output buffer #24: + timeUs = 1000001120000 + size = 28 + rendered = true + output buffer #25: + timeUs = 1000001040000 + size = 27 + rendered = true + output buffer #26: + timeUs = 1000001000000 + size = 18 + rendered = true + output buffer #27: + timeUs = 1000001080000 + size = 18 + rendered = true + output buffer #28: + timeUs = 1000001280000 + size = 28 + rendered = true + output buffer #29: + timeUs = 1000001200000 + size = 27 + rendered = true + output buffer #30: + timeUs = 1000001160000 + size = 18 + rendered = true + output buffer #31: + timeUs = 1000001240000 + size = 18 + rendered = true + output buffer #32: + timeUs = 1000001440000 + size = 28 + rendered = true + output buffer #33: + timeUs = 1000001360000 + size = 27 + rendered = true + output buffer #34: + timeUs = 1000001320000 + size = 18 + rendered = true + output buffer #35: + timeUs = 1000001400000 + size = 18 + rendered = true + output buffer #36: + timeUs = 1000001600000 + size = 28 + rendered = true + output buffer #37: + timeUs = 1000001520000 + size = 27 + rendered = true + output buffer #38: + timeUs = 1000001480000 + size = 18 + rendered = true + output buffer #39: + timeUs = 1000001560000 + size = 18 + rendered = true + output buffer #40: + timeUs = 1000001760000 + size = 28 + rendered = true + output buffer #41: + timeUs = 1000001680000 + size = 27 + rendered = true + output buffer #42: + timeUs = 1000001640000 + size = 18 + rendered = true + output buffer #43: + timeUs = 1000001720000 + size = 18 + rendered = true + output buffer #44: + timeUs = 1000001920000 + size = 28 + rendered = true + output buffer #45: + timeUs = 1000001840000 + size = 27 + rendered = true + output buffer #46: + timeUs = 1000001800000 + size = 18 + rendered = true + output buffer #47: + timeUs = 1000001880000 + size = 18 + rendered = true + output buffer #48: + timeUs = 1000002080000 + size = 28 + rendered = false + output buffer #49: + timeUs = 1000002000000 + size = 27 + rendered = false + output buffer #50: + timeUs = 1000001960000 + size = 18 + rendered = true +AudioSink: + buffer count = 82 + config: + pcmEncoding = pcm-16 + channelCount = 1 + sampleRate = 44100 + buffer #0: + time = 1000000116099 + data = empty + buffer #1: + time = 1000000139319 + data = empty + buffer #2: + time = 1000000162539 + data = empty + buffer #3: + time = 1000000185759 + data = empty + buffer #4: + time = 1000000208979 + data = empty + buffer #5: + time = 1000000232199 + data = empty + buffer #6: + time = 1000000255419 + data = empty + buffer #7: + time = 1000000278639 + data = empty + buffer #8: + time = 1000000301859 + data = empty + buffer #9: + time = 1000000325079 + data = empty + buffer #10: + time = 1000000348299 + data = empty + buffer #11: + time = 1000000371519 + data = empty + buffer #12: + time = 1000000394739 + data = empty + buffer #13: + time = 1000000417959 + data = empty + buffer #14: + time = 1000000441179 + data = empty + buffer #15: + time = 1000000464399 + data = empty + buffer #16: + time = 1000000487619 + data = empty + buffer #17: + time = 1000000510839 + data = empty + buffer #18: + time = 1000000534058 + data = empty + buffer #19: + time = 1000000557278 + data = empty + buffer #20: + time = 1000000580498 + data = empty + buffer #21: + time = 1000000603718 + data = empty + buffer #22: + time = 1000000626938 + data = empty + buffer #23: + time = 1000000650158 + data = empty + buffer #24: + time = 1000000673378 + data = empty + buffer #25: + time = 1000000696598 + data = empty + buffer #26: + time = 1000000719818 + data = empty + buffer #27: + time = 1000000743038 + data = empty + buffer #28: + time = 1000000766258 + data = empty + buffer #29: + time = 1000000789478 + data = empty + buffer #30: + time = 1000000812698 + data = empty + buffer #31: + time = 1000000835918 + data = empty + buffer #32: + time = 1000000859138 + data = empty + buffer #33: + time = 1000000882358 + data = empty + buffer #34: + time = 1000000905578 + data = empty + buffer #35: + time = 1000000928798 + data = empty + buffer #36: + time = 1000000952018 + data = empty + buffer #37: + time = 1000000975238 + data = empty + buffer #38: + time = 1000000998458 + data = empty + buffer #39: + time = 1000001021678 + data = empty + buffer #40: + time = 1000001044897 + data = empty + buffer #41: + time = 1000001068117 + data = empty + buffer #42: + time = 1000001091337 + data = empty + buffer #43: + time = 1000001114557 + data = empty + buffer #44: + time = 1000001137777 + data = empty + buffer #45: + time = 1000001160997 + data = empty + buffer #46: + time = 1000001184217 + data = empty + buffer #47: + time = 1000001207437 + data = empty + buffer #48: + time = 1000001230657 + data = empty + buffer #49: + time = 1000001253877 + data = empty + buffer #50: + time = 1000001277097 + data = empty + buffer #51: + time = 1000001300317 + data = empty + buffer #52: + time = 1000001323537 + data = empty + buffer #53: + time = 1000001346757 + data = empty + buffer #54: + time = 1000001369977 + data = empty + buffer #55: + time = 1000001393197 + data = empty + buffer #56: + time = 1000001416417 + data = empty + buffer #57: + time = 1000001439637 + data = empty + buffer #58: + time = 1000001462857 + data = empty + buffer #59: + time = 1000001486077 + data = empty + buffer #60: + time = 1000001509297 + data = empty + buffer #61: + time = 1000001532517 + data = empty + buffer #62: + time = 1000001555736 + data = empty + buffer #63: + time = 1000001578956 + data = empty + buffer #64: + time = 1000001602176 + data = empty + buffer #65: + time = 1000001625396 + data = empty + buffer #66: + time = 1000001648616 + data = empty + buffer #67: + time = 1000001671836 + data = empty + buffer #68: + time = 1000001695056 + data = empty + buffer #69: + time = 1000001718276 + data = empty + buffer #70: + time = 1000001741496 + data = empty + buffer #71: + time = 1000001764716 + data = empty + buffer #72: + time = 1000001787936 + data = empty + buffer #73: + time = 1000001811156 + data = empty + buffer #74: + time = 1000001834376 + data = empty + buffer #75: + time = 1000001857596 + data = empty + buffer #76: + time = 1000001880816 + data = empty + buffer #77: + time = 1000001904036 + data = empty + buffer #78: + time = 1000001927256 + data = empty + buffer #79: + time = 1000001950476 + data = empty + buffer #80: + time = 1000001973696 + data = empty + buffer #81: + time = 1000001996916 + data = empty diff --git a/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_reduce_before_load_finished_to_not_loaded_value_enabled.dump b/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_reduce_before_load_finished_to_not_loaded_value_enabled.dump new file mode 100644 index 00000000000..3ca36f27aa4 --- /dev/null +++ b/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_reduce_before_load_finished_to_not_loaded_value_enabled.dump @@ -0,0 +1,1884 @@ +extracted samples: + sample #0: + track = 0 + timeUs = 0 + flags = 1 + size = 5252 + sample #1: + track = 0 + timeUs = 160000 + size = 44 + sample #2: + track = 0 + timeUs = 80000 + size = 25 + sample #3: + track = 0 + timeUs = 40000 + flags = 67108864 + size = 18 + sample #4: + track = 1 + timeUs = 92879 + flags = 1 + size = 4 + sample #5: + track = 0 + timeUs = 120000 + flags = 67108864 + size = 18 + sample #6: + track = 1 + timeUs = 116099 + flags = 1 + size = 4 + sample #7: + track = 1 + timeUs = 139319 + flags = 1 + size = 4 + sample #8: + track = 0 + timeUs = 320000 + size = 32 + sample #9: + track = 1 + timeUs = 162539 + flags = 1 + size = 4 + sample #10: + track = 0 + timeUs = 240000 + size = 27 + sample #11: + track = 1 + timeUs = 185759 + flags = 1 + size = 4 + sample #12: + track = 1 + timeUs = 208979 + flags = 1 + size = 4 + sample #13: + track = 0 + timeUs = 200000 + flags = 67108864 + size = 18 + sample #14: + track = 1 + timeUs = 232199 + flags = 1 + size = 4 + sample #15: + track = 1 + timeUs = 255419 + flags = 1 + size = 4 + sample #16: + track = 0 + timeUs = 280000 + flags = 67108864 + size = 18 + sample #17: + track = 1 + timeUs = 278639 + flags = 1 + size = 4 + sample #18: + track = 1 + timeUs = 301859 + flags = 1 + size = 4 + sample #19: + track = 0 + timeUs = 480000 + size = 28 + sample #20: + track = 1 + timeUs = 325079 + flags = 1 + size = 4 + sample #21: + track = 0 + timeUs = 400000 + size = 27 + sample #22: + track = 1 + timeUs = 348299 + flags = 1 + size = 4 + sample #23: + track = 1 + timeUs = 371519 + flags = 1 + size = 4 + sample #24: + track = 0 + timeUs = 360000 + flags = 67108864 + size = 18 + sample #25: + track = 1 + timeUs = 394739 + flags = 1 + size = 4 + sample #26: + track = 1 + timeUs = 417959 + flags = 1 + size = 4 + sample #27: + track = 0 + timeUs = 440000 + flags = 67108864 + size = 18 + sample #28: + track = 1 + timeUs = 441179 + flags = 1 + size = 4 + sample #29: + track = 0 + timeUs = 640000 + size = 28 + sample #30: + track = 1 + timeUs = 464399 + flags = 1 + size = 4 + sample #31: + track = 1 + timeUs = 487619 + flags = 1 + size = 4 + sample #32: + track = 0 + timeUs = 560000 + size = 27 + sample #33: + track = 1 + timeUs = 510839 + flags = 1 + size = 4 + sample #34: + track = 1 + timeUs = 534058 + flags = 1 + size = 4 + sample #35: + track = 0 + timeUs = 520000 + flags = 67108864 + size = 18 + sample #36: + track = 1 + timeUs = 557278 + flags = 1 + size = 4 + sample #37: + track = 1 + timeUs = 580498 + flags = 1 + size = 4 + sample #38: + track = 0 + timeUs = 600000 + flags = 67108864 + size = 18 + sample #39: + track = 1 + timeUs = 603718 + flags = 1 + size = 4 + sample #40: + track = 0 + timeUs = 800000 + size = 28 + sample #41: + track = 1 + timeUs = 626938 + flags = 1 + size = 4 + sample #42: + track = 1 + timeUs = 650158 + flags = 1 + size = 4 + sample #43: + track = 0 + timeUs = 720000 + size = 27 + sample #44: + track = 1 + timeUs = 673378 + flags = 1 + size = 4 + sample #45: + track = 1 + timeUs = 696598 + flags = 1 + size = 4 + sample #46: + track = 0 + timeUs = 680000 + flags = 67108864 + size = 18 + sample #47: + track = 1 + timeUs = 719818 + flags = 1 + size = 4 + sample #48: + track = 1 + timeUs = 743038 + flags = 1 + size = 4 + sample #49: + track = 0 + timeUs = 760000 + flags = 67108864 + size = 18 + sample #50: + track = 1 + timeUs = 766258 + flags = 1 + size = 4 + sample #51: + track = 0 + timeUs = 960000 + size = 28 + sample #52: + track = 1 + timeUs = 789478 + flags = 1 + size = 4 + sample #53: + track = 1 + timeUs = 812698 + flags = 1 + size = 4 + sample #54: + track = 0 + timeUs = 880000 + size = 27 + sample #55: + track = 1 + timeUs = 835918 + flags = 1 + size = 4 + sample #56: + track = 1 + timeUs = 859138 + flags = 1 + size = 4 + sample #57: + track = 0 + timeUs = 840000 + flags = 67108864 + size = 18 + sample #58: + track = 1 + timeUs = 882358 + flags = 1 + size = 4 + sample #59: + track = 0 + timeUs = 920000 + flags = 67108864 + size = 18 + sample #60: + track = 1 + timeUs = 905578 + flags = 1 + size = 4 + sample #61: + track = 1 + timeUs = 928798 + flags = 1 + size = 4 + sample #62: + track = 0 + timeUs = 1120000 + size = 28 + sample #63: + track = 1 + timeUs = 952018 + flags = 1 + size = 4 + sample #64: + track = 1 + timeUs = 975238 + flags = 1 + size = 4 + sample #65: + track = 0 + timeUs = 1040000 + size = 27 + sample #66: + track = 1 + timeUs = 998458 + flags = 1 + size = 4 + sample #67: + track = 1 + timeUs = 1021678 + flags = 1 + size = 4 + sample #68: + track = 0 + timeUs = 1000000 + flags = 67108864 + size = 18 + sample #69: + track = 1 + timeUs = 1044897 + flags = 1 + size = 4 + sample #70: + track = 0 + timeUs = 1080000 + flags = 67108864 + size = 18 + sample #71: + track = 1 + timeUs = 1068117 + flags = 1 + size = 4 + sample #72: + track = 1 + timeUs = 1091337 + flags = 1 + size = 4 + sample #73: + track = 0 + timeUs = 1280000 + size = 28 + sample #74: + track = 1 + timeUs = 1114557 + flags = 1 + size = 4 + sample #75: + track = 1 + timeUs = 1137777 + flags = 1 + size = 4 + sample #76: + track = 0 + timeUs = 1200000 + size = 27 + sample #77: + track = 1 + timeUs = 1160997 + flags = 1 + size = 4 + sample #78: + track = 0 + timeUs = 1160000 + flags = 67108864 + size = 18 + sample #79: + track = 1 + timeUs = 1184217 + flags = 1 + size = 4 + sample #80: + track = 1 + timeUs = 1207437 + flags = 1 + size = 4 + sample #81: + track = 0 + timeUs = 1240000 + flags = 67108864 + size = 18 + sample #82: + track = 1 + timeUs = 1230657 + flags = 1 + size = 4 + sample #83: + track = 1 + timeUs = 1253877 + flags = 1 + size = 4 + sample #84: + track = 0 + timeUs = 1440000 + size = 28 + sample #85: + track = 1 + timeUs = 1277097 + flags = 1 + size = 4 + sample #86: + track = 1 + timeUs = 1300317 + flags = 1 + size = 4 + sample #87: + track = 0 + timeUs = 1360000 + size = 27 + sample #88: + track = 1 + timeUs = 1323537 + flags = 1 + size = 4 + sample #89: + track = 0 + timeUs = 1320000 + flags = 67108864 + size = 18 + sample #90: + track = 1 + timeUs = 1346757 + flags = 1 + size = 4 + sample #91: + track = 1 + timeUs = 1369977 + flags = 1 + size = 4 + sample #92: + track = 0 + timeUs = 1400000 + flags = 67108864 + size = 18 + sample #93: + track = 1 + timeUs = 1393197 + flags = 1 + size = 4 + sample #94: + track = 1 + timeUs = 1416417 + flags = 1 + size = 4 + sample #95: + track = 0 + timeUs = 1600000 + size = 28 + sample #96: + track = 1 + timeUs = 1439637 + flags = 1 + size = 4 + sample #97: + track = 1 + timeUs = 1462857 + flags = 1 + size = 4 + sample #98: + track = 0 + timeUs = 1520000 + size = 27 + sample #99: + track = 1 + timeUs = 1486077 + flags = 1 + size = 4 + sample #100: + track = 0 + timeUs = 1480000 + flags = 67108864 + size = 18 + sample #101: + track = 1 + timeUs = 1509297 + flags = 1 + size = 4 + sample #102: + track = 1 + timeUs = 1532517 + flags = 1 + size = 4 + sample #103: + track = 0 + timeUs = 1560000 + flags = 67108864 + size = 18 + sample #104: + track = 1 + timeUs = 1555736 + flags = 1 + size = 4 + sample #105: + track = 1 + timeUs = 1578956 + flags = 1 + size = 4 + sample #106: + track = 0 + timeUs = 1760000 + size = 28 + sample #107: + track = 1 + timeUs = 1602176 + flags = 1 + size = 4 + sample #108: + track = 0 + timeUs = 1680000 + size = 27 + sample #109: + track = 1 + timeUs = 1625396 + flags = 1 + size = 4 + sample #110: + track = 1 + timeUs = 1648616 + flags = 1 + size = 4 + sample #111: + track = 0 + timeUs = 1640000 + flags = 67108864 + size = 18 + sample #112: + track = 1 + timeUs = 1671836 + flags = 1 + size = 4 + sample #113: + track = 1 + timeUs = 1695056 + flags = 1 + size = 4 + sample #114: + track = 0 + timeUs = 1720000 + flags = 67108864 + size = 18 + sample #115: + track = 1 + timeUs = 1718276 + flags = 1 + size = 4 + sample #116: + track = 1 + timeUs = 1741496 + flags = 1 + size = 4 + sample #117: + track = 0 + timeUs = 1920000 + size = 28 + sample #118: + track = 1 + timeUs = 1764716 + flags = 1 + size = 4 + sample #119: + track = 0 + timeUs = 1840000 + size = 27 + sample #120: + track = 1 + timeUs = 1787936 + flags = 1 + size = 4 + sample #121: + track = 1 + timeUs = 1811156 + flags = 1 + size = 4 + sample #122: + track = 0 + timeUs = 1800000 + flags = 67108864 + size = 18 + sample #123: + track = 1 + timeUs = 1834376 + flags = 1 + size = 4 + sample #124: + track = 1 + timeUs = 1857596 + flags = 1 + size = 4 + sample #125: + track = 0 + timeUs = 1880000 + flags = 67108864 + size = 18 + sample #126: + track = 1 + timeUs = 1880816 + flags = 1 + size = 4 + sample #127: + track = 0 + timeUs = 2080000 + size = 28 + sample #128: + track = 1 + timeUs = 1904036 + flags = 1 + size = 4 + sample #129: + track = 1 + timeUs = 1927256 + flags = 1 + size = 4 + sample #130: + track = 0 + timeUs = 2000000 + size = 27 + sample #131: + track = 1 + timeUs = 1950476 + flags = 1 + size = 4 + sample #132: + track = 1 + timeUs = 1973696 + flags = 1 + size = 4 + sample #133: + track = 0 + timeUs = 1960000 + flags = 67108864 + size = 18 + sample #134: + track = 1 + timeUs = 1996916 + flags = 1 + size = 4 + sample #135: + track = 1 + timeUs = 2020136 + flags = 1 + size = 4 + sample #136: + track = 0 + timeUs = 2040000 + flags = 67108864 + size = 18 + sample #137: + track = 1 + timeUs = 2043356 + flags = 1 + size = 4 + sample #138: + track = 0 + timeUs = 2240000 + size = 28 + sample #139: + track = 1 + timeUs = 2066575 + flags = 1 + size = 4 + sample #140: + track = 1 + timeUs = 2089795 + flags = 1 + size = 4 + sample #141: + track = 0 + timeUs = 2160000 + size = 27 +MediaCodecAdapter (media3.audio.aac): + inputBuffers: + count = 83 + input buffer #0: + timeUs = 1000000116099 + contents = length 4, hash EE9DF + input buffer #1: + timeUs = 1000000139319 + contents = length 4, hash EE9DF + input buffer #2: + timeUs = 1000000162539 + contents = length 4, hash EE9DF + input buffer #3: + timeUs = 1000000185759 + contents = length 4, hash EE9DF + input buffer #4: + timeUs = 1000000208979 + contents = length 4, hash EE9DF + input buffer #5: + timeUs = 1000000232199 + contents = length 4, hash EE9DF + input buffer #6: + timeUs = 1000000255419 + contents = length 4, hash EE9DF + input buffer #7: + timeUs = 1000000278639 + contents = length 4, hash EE9DF + input buffer #8: + timeUs = 1000000301859 + contents = length 4, hash EE9DF + input buffer #9: + timeUs = 1000000325079 + contents = length 4, hash EE9DF + input buffer #10: + timeUs = 1000000348299 + contents = length 4, hash EE9DF + input buffer #11: + timeUs = 1000000371519 + contents = length 4, hash EE9DF + input buffer #12: + timeUs = 1000000394739 + contents = length 4, hash EE9DF + input buffer #13: + timeUs = 1000000417959 + contents = length 4, hash EE9DF + input buffer #14: + timeUs = 1000000441179 + contents = length 4, hash EE9DF + input buffer #15: + timeUs = 1000000464399 + contents = length 4, hash EE9DF + input buffer #16: + timeUs = 1000000487619 + contents = length 4, hash EE9DF + input buffer #17: + timeUs = 1000000510839 + contents = length 4, hash EE9DF + input buffer #18: + timeUs = 1000000534058 + contents = length 4, hash EE9DF + input buffer #19: + timeUs = 1000000557278 + contents = length 4, hash EE9DF + input buffer #20: + timeUs = 1000000580498 + contents = length 4, hash EE9DF + input buffer #21: + timeUs = 1000000603718 + contents = length 4, hash EE9DF + input buffer #22: + timeUs = 1000000626938 + contents = length 4, hash EE9DF + input buffer #23: + timeUs = 1000000650158 + contents = length 4, hash EE9DF + input buffer #24: + timeUs = 1000000673378 + contents = length 4, hash EE9DF + input buffer #25: + timeUs = 1000000696598 + contents = length 4, hash EE9DF + input buffer #26: + timeUs = 1000000719818 + contents = length 4, hash EE9DF + input buffer #27: + timeUs = 1000000743038 + contents = length 4, hash EE9DF + input buffer #28: + timeUs = 1000000766258 + contents = length 4, hash EE9DF + input buffer #29: + timeUs = 1000000789478 + contents = length 4, hash EE9DF + input buffer #30: + timeUs = 1000000812698 + contents = length 4, hash EE9DF + input buffer #31: + timeUs = 1000000835918 + contents = length 4, hash EE9DF + input buffer #32: + timeUs = 1000000859138 + contents = length 4, hash EE9DF + input buffer #33: + timeUs = 1000000882358 + contents = length 4, hash EE9DF + input buffer #34: + timeUs = 1000000905578 + contents = length 4, hash EE9DF + input buffer #35: + timeUs = 1000000928798 + contents = length 4, hash EE9DF + input buffer #36: + timeUs = 1000000952018 + contents = length 4, hash EE9DF + input buffer #37: + timeUs = 1000000975238 + contents = length 4, hash EE9DF + input buffer #38: + timeUs = 1000000998458 + contents = length 4, hash EE9DF + input buffer #39: + timeUs = 1000001021678 + contents = length 4, hash EE9DF + input buffer #40: + timeUs = 1000001044897 + contents = length 4, hash EE9DF + input buffer #41: + timeUs = 1000001068117 + contents = length 4, hash EE9DF + input buffer #42: + timeUs = 1000001091337 + contents = length 4, hash EE9DF + input buffer #43: + timeUs = 1000001114557 + contents = length 4, hash EE9DF + input buffer #44: + timeUs = 1000001137777 + contents = length 4, hash EE9DF + input buffer #45: + timeUs = 1000001160997 + contents = length 4, hash EE9DF + input buffer #46: + timeUs = 1000001184217 + contents = length 4, hash EE9DF + input buffer #47: + timeUs = 1000001207437 + contents = length 4, hash EE9DF + input buffer #48: + timeUs = 1000001230657 + contents = length 4, hash EE9DF + input buffer #49: + timeUs = 1000001253877 + contents = length 4, hash EE9DF + input buffer #50: + timeUs = 1000001277097 + contents = length 4, hash EE9DF + input buffer #51: + timeUs = 1000001300317 + contents = length 4, hash EE9DF + input buffer #52: + timeUs = 1000001323537 + contents = length 4, hash EE9DF + input buffer #53: + timeUs = 1000001346757 + contents = length 4, hash EE9DF + input buffer #54: + timeUs = 1000001369977 + contents = length 4, hash EE9DF + input buffer #55: + timeUs = 1000001393197 + contents = length 4, hash EE9DF + input buffer #56: + timeUs = 1000001416417 + contents = length 4, hash EE9DF + input buffer #57: + timeUs = 1000001439637 + contents = length 4, hash EE9DF + input buffer #58: + timeUs = 1000001462857 + contents = length 4, hash EE9DF + input buffer #59: + timeUs = 1000001486077 + contents = length 4, hash EE9DF + input buffer #60: + timeUs = 1000001509297 + contents = length 4, hash EE9DF + input buffer #61: + timeUs = 1000001532517 + contents = length 4, hash EE9DF + input buffer #62: + timeUs = 1000001555736 + contents = length 4, hash EE9DF + input buffer #63: + timeUs = 1000001578956 + contents = length 4, hash EE9DF + input buffer #64: + timeUs = 1000001602176 + contents = length 4, hash EE9DF + input buffer #65: + timeUs = 1000001625396 + contents = length 4, hash EE9DF + input buffer #66: + timeUs = 1000001648616 + contents = length 4, hash EE9DF + input buffer #67: + timeUs = 1000001671836 + contents = length 4, hash EE9DF + input buffer #68: + timeUs = 1000001695056 + contents = length 4, hash EE9DF + input buffer #69: + timeUs = 1000001718276 + contents = length 4, hash EE9DF + input buffer #70: + timeUs = 1000001741496 + contents = length 4, hash EE9DF + input buffer #71: + timeUs = 1000001764716 + contents = length 4, hash EE9DF + input buffer #72: + timeUs = 1000001787936 + contents = length 4, hash EE9DF + input buffer #73: + timeUs = 1000001811156 + contents = length 4, hash EE9DF + input buffer #74: + timeUs = 1000001834376 + contents = length 4, hash EE9DF + input buffer #75: + timeUs = 1000001857596 + contents = length 4, hash EE9DF + input buffer #76: + timeUs = 1000001880816 + contents = length 4, hash EE9DF + input buffer #77: + timeUs = 1000001904036 + contents = length 4, hash EE9DF + input buffer #78: + timeUs = 1000001927256 + contents = length 4, hash EE9DF + input buffer #79: + timeUs = 1000001950476 + contents = length 4, hash EE9DF + input buffer #80: + timeUs = 1000001973696 + contents = length 4, hash EE9DF + input buffer #81: + timeUs = 1000001996916 + contents = length 4, hash EE9DF + input buffer #82: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 82 + output buffer #0: + timeUs = 1000000116099 + size = 0 + rendered = false + output buffer #1: + timeUs = 1000000139319 + size = 0 + rendered = false + output buffer #2: + timeUs = 1000000162539 + size = 0 + rendered = false + output buffer #3: + timeUs = 1000000185759 + size = 0 + rendered = false + output buffer #4: + timeUs = 1000000208979 + size = 0 + rendered = false + output buffer #5: + timeUs = 1000000232199 + size = 0 + rendered = false + output buffer #6: + timeUs = 1000000255419 + size = 0 + rendered = false + output buffer #7: + timeUs = 1000000278639 + size = 0 + rendered = false + output buffer #8: + timeUs = 1000000301859 + size = 0 + rendered = false + output buffer #9: + timeUs = 1000000325079 + size = 0 + rendered = false + output buffer #10: + timeUs = 1000000348299 + size = 0 + rendered = false + output buffer #11: + timeUs = 1000000371519 + size = 0 + rendered = false + output buffer #12: + timeUs = 1000000394739 + size = 0 + rendered = false + output buffer #13: + timeUs = 1000000417959 + size = 0 + rendered = false + output buffer #14: + timeUs = 1000000441179 + size = 0 + rendered = false + output buffer #15: + timeUs = 1000000464399 + size = 0 + rendered = false + output buffer #16: + timeUs = 1000000487619 + size = 0 + rendered = false + output buffer #17: + timeUs = 1000000510839 + size = 0 + rendered = false + output buffer #18: + timeUs = 1000000534058 + size = 0 + rendered = false + output buffer #19: + timeUs = 1000000557278 + size = 0 + rendered = false + output buffer #20: + timeUs = 1000000580498 + size = 0 + rendered = false + output buffer #21: + timeUs = 1000000603718 + size = 0 + rendered = false + output buffer #22: + timeUs = 1000000626938 + size = 0 + rendered = false + output buffer #23: + timeUs = 1000000650158 + size = 0 + rendered = false + output buffer #24: + timeUs = 1000000673378 + size = 0 + rendered = false + output buffer #25: + timeUs = 1000000696598 + size = 0 + rendered = false + output buffer #26: + timeUs = 1000000719818 + size = 0 + rendered = false + output buffer #27: + timeUs = 1000000743038 + size = 0 + rendered = false + output buffer #28: + timeUs = 1000000766258 + size = 0 + rendered = false + output buffer #29: + timeUs = 1000000789478 + size = 0 + rendered = false + output buffer #30: + timeUs = 1000000812698 + size = 0 + rendered = false + output buffer #31: + timeUs = 1000000835918 + size = 0 + rendered = false + output buffer #32: + timeUs = 1000000859138 + size = 0 + rendered = false + output buffer #33: + timeUs = 1000000882358 + size = 0 + rendered = false + output buffer #34: + timeUs = 1000000905578 + size = 0 + rendered = false + output buffer #35: + timeUs = 1000000928798 + size = 0 + rendered = false + output buffer #36: + timeUs = 1000000952018 + size = 0 + rendered = false + output buffer #37: + timeUs = 1000000975238 + size = 0 + rendered = false + output buffer #38: + timeUs = 1000000998458 + size = 0 + rendered = false + output buffer #39: + timeUs = 1000001021678 + size = 0 + rendered = false + output buffer #40: + timeUs = 1000001044897 + size = 0 + rendered = false + output buffer #41: + timeUs = 1000001068117 + size = 0 + rendered = false + output buffer #42: + timeUs = 1000001091337 + size = 0 + rendered = false + output buffer #43: + timeUs = 1000001114557 + size = 0 + rendered = false + output buffer #44: + timeUs = 1000001137777 + size = 0 + rendered = false + output buffer #45: + timeUs = 1000001160997 + size = 0 + rendered = false + output buffer #46: + timeUs = 1000001184217 + size = 0 + rendered = false + output buffer #47: + timeUs = 1000001207437 + size = 0 + rendered = false + output buffer #48: + timeUs = 1000001230657 + size = 0 + rendered = false + output buffer #49: + timeUs = 1000001253877 + size = 0 + rendered = false + output buffer #50: + timeUs = 1000001277097 + size = 0 + rendered = false + output buffer #51: + timeUs = 1000001300317 + size = 0 + rendered = false + output buffer #52: + timeUs = 1000001323537 + size = 0 + rendered = false + output buffer #53: + timeUs = 1000001346757 + size = 0 + rendered = false + output buffer #54: + timeUs = 1000001369977 + size = 0 + rendered = false + output buffer #55: + timeUs = 1000001393197 + size = 0 + rendered = false + output buffer #56: + timeUs = 1000001416417 + size = 0 + rendered = false + output buffer #57: + timeUs = 1000001439637 + size = 0 + rendered = false + output buffer #58: + timeUs = 1000001462857 + size = 0 + rendered = false + output buffer #59: + timeUs = 1000001486077 + size = 0 + rendered = false + output buffer #60: + timeUs = 1000001509297 + size = 0 + rendered = false + output buffer #61: + timeUs = 1000001532517 + size = 0 + rendered = false + output buffer #62: + timeUs = 1000001555736 + size = 0 + rendered = false + output buffer #63: + timeUs = 1000001578956 + size = 0 + rendered = false + output buffer #64: + timeUs = 1000001602176 + size = 0 + rendered = false + output buffer #65: + timeUs = 1000001625396 + size = 0 + rendered = false + output buffer #66: + timeUs = 1000001648616 + size = 0 + rendered = false + output buffer #67: + timeUs = 1000001671836 + size = 0 + rendered = false + output buffer #68: + timeUs = 1000001695056 + size = 0 + rendered = false + output buffer #69: + timeUs = 1000001718276 + size = 0 + rendered = false + output buffer #70: + timeUs = 1000001741496 + size = 0 + rendered = false + output buffer #71: + timeUs = 1000001764716 + size = 0 + rendered = false + output buffer #72: + timeUs = 1000001787936 + size = 0 + rendered = false + output buffer #73: + timeUs = 1000001811156 + size = 0 + rendered = false + output buffer #74: + timeUs = 1000001834376 + size = 0 + rendered = false + output buffer #75: + timeUs = 1000001857596 + size = 0 + rendered = false + output buffer #76: + timeUs = 1000001880816 + size = 0 + rendered = false + output buffer #77: + timeUs = 1000001904036 + size = 0 + rendered = false + output buffer #78: + timeUs = 1000001927256 + size = 0 + rendered = false + output buffer #79: + timeUs = 1000001950476 + size = 0 + rendered = false + output buffer #80: + timeUs = 1000001973696 + size = 0 + rendered = false + output buffer #81: + timeUs = 1000001996916 + size = 0 + rendered = false +MediaCodecAdapter (media3.video.avc): + inputBuffers: + count = 52 + input buffer #0: + timeUs = 1000000000000 + contents = length 5252, hash 13893A4C + input buffer #1: + timeUs = 1000000160000 + contents = length 44, hash A05B3BEA + input buffer #2: + timeUs = 1000000080000 + contents = length 25, hash 441EC4FE + input buffer #3: + timeUs = 1000000120000 + contents = length 18, hash B64DA059 + input buffer #4: + timeUs = 1000000320000 + contents = length 32, hash 5BFD9538 + input buffer #5: + timeUs = 1000000240000 + contents = length 27, hash 705869A5 + input buffer #6: + timeUs = 1000000200000 + contents = length 18, hash D163DF61 + input buffer #7: + timeUs = 1000000280000 + contents = length 18, hash FD82E95 + input buffer #8: + timeUs = 1000000480000 + contents = length 28, hash 44A16E72 + input buffer #9: + timeUs = 1000000400000 + contents = length 27, hash FFA326ED + input buffer #10: + timeUs = 1000000360000 + contents = length 18, hash DC93CC9D + input buffer #11: + timeUs = 1000000440000 + contents = length 18, hash 1B081BD1 + input buffer #12: + timeUs = 1000000640000 + contents = length 28, hash 2700AF + input buffer #13: + timeUs = 1000000560000 + contents = length 27, hash DECA17D0 + input buffer #14: + timeUs = 1000000520000 + contents = length 18, hash D646C05A + input buffer #15: + timeUs = 1000000600000 + contents = length 18, hash 14BB0F8E + input buffer #16: + timeUs = 1000000800000 + contents = length 28, hash 5DE2C2B + input buffer #17: + timeUs = 1000000720000 + contents = length 27, hash 376B4E14 + input buffer #18: + timeUs = 1000000680000 + contents = length 18, hash E176AD96 + input buffer #19: + timeUs = 1000000760000 + contents = length 18, hash 1FEAFCCA + input buffer #20: + timeUs = 1000000960000 + contents = length 28, hash C163BE68 + input buffer #21: + timeUs = 1000000880000 + contents = length 27, hash F7170357 + input buffer #22: + timeUs = 1000000840000 + contents = length 18, hash 3B013BD2 + input buffer #23: + timeUs = 1000000920000 + contents = length 18, hash 79758B06 + input buffer #24: + timeUs = 1000001120000 + contents = length 28, hash F72EB1A3 + input buffer #25: + timeUs = 1000001040000 + contents = length 27, hash 4FB8399C + input buffer #26: + timeUs = 1000001000000 + contents = length 18, hash 4631290E + input buffer #27: + timeUs = 1000001080000 + contents = length 18, hash 84A57842 + input buffer #28: + timeUs = 1000001280000 + contents = length 28, hash E1FCF000 + input buffer #29: + timeUs = 1000001200000 + contents = length 27, hash 21D3B540 + input buffer #30: + timeUs = 1000001160000 + contents = length 18, hash 62DE0FC9 + input buffer #31: + timeUs = 1000001240000 + contents = length 18, hash A1525EFD + input buffer #32: + timeUs = 1000001440000 + contents = length 28, hash 5350E8FA + input buffer #33: + timeUs = 1000001360000 + contents = length 27, hash E7563B65 + input buffer #34: + timeUs = 1000001320000 + contents = length 18, hash 77D95125 + input buffer #35: + timeUs = 1000001400000 + contents = length 18, hash B64DA059 + input buffer #36: + timeUs = 1000001600000 + contents = length 28, hash ED67B37 + input buffer #37: + timeUs = 1000001520000 + contents = length 27, hash A701F0A9 + input buffer #38: + timeUs = 1000001480000 + contents = length 18, hash D163DF61 + input buffer #39: + timeUs = 1000001560000 + contents = length 18, hash FD82E95 + input buffer #40: + timeUs = 1000001760000 + contents = length 28, hash 44A16E72 + input buffer #41: + timeUs = 1000001680000 + contents = length 27, hash FFA326ED + input buffer #42: + timeUs = 1000001640000 + contents = length 18, hash DC93CC9D + input buffer #43: + timeUs = 1000001720000 + contents = length 18, hash 1B081BD1 + input buffer #44: + timeUs = 1000001920000 + contents = length 28, hash 2700AF + input buffer #45: + timeUs = 1000001840000 + contents = length 27, hash DECA17D0 + input buffer #46: + timeUs = 1000001800000 + contents = length 18, hash D646C05A + input buffer #47: + timeUs = 1000001880000 + contents = length 18, hash 14BB0F8E + input buffer #48: + timeUs = 1000002080000 + contents = length 28, hash 5DE2C2B + input buffer #49: + timeUs = 1000002000000 + contents = length 27, hash 376B4E14 + input buffer #50: + timeUs = 1000001960000 + contents = length 18, hash E176AD96 + input buffer #51: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 51 + output buffer #0: + timeUs = 1000000000000 + size = 5252 + rendered = false + output buffer #1: + timeUs = 1000000160000 + size = 44 + rendered = true + output buffer #2: + timeUs = 1000000080000 + size = 25 + rendered = false + output buffer #3: + timeUs = 1000000120000 + size = 18 + rendered = true + output buffer #4: + timeUs = 1000000320000 + size = 32 + rendered = true + output buffer #5: + timeUs = 1000000240000 + size = 27 + rendered = true + output buffer #6: + timeUs = 1000000200000 + size = 18 + rendered = true + output buffer #7: + timeUs = 1000000280000 + size = 18 + rendered = true + output buffer #8: + timeUs = 1000000480000 + size = 28 + rendered = true + output buffer #9: + timeUs = 1000000400000 + size = 27 + rendered = true + output buffer #10: + timeUs = 1000000360000 + size = 18 + rendered = true + output buffer #11: + timeUs = 1000000440000 + size = 18 + rendered = true + output buffer #12: + timeUs = 1000000640000 + size = 28 + rendered = true + output buffer #13: + timeUs = 1000000560000 + size = 27 + rendered = true + output buffer #14: + timeUs = 1000000520000 + size = 18 + rendered = true + output buffer #15: + timeUs = 1000000600000 + size = 18 + rendered = true + output buffer #16: + timeUs = 1000000800000 + size = 28 + rendered = true + output buffer #17: + timeUs = 1000000720000 + size = 27 + rendered = true + output buffer #18: + timeUs = 1000000680000 + size = 18 + rendered = true + output buffer #19: + timeUs = 1000000760000 + size = 18 + rendered = true + output buffer #20: + timeUs = 1000000960000 + size = 28 + rendered = true + output buffer #21: + timeUs = 1000000880000 + size = 27 + rendered = true + output buffer #22: + timeUs = 1000000840000 + size = 18 + rendered = true + output buffer #23: + timeUs = 1000000920000 + size = 18 + rendered = true + output buffer #24: + timeUs = 1000001120000 + size = 28 + rendered = true + output buffer #25: + timeUs = 1000001040000 + size = 27 + rendered = true + output buffer #26: + timeUs = 1000001000000 + size = 18 + rendered = true + output buffer #27: + timeUs = 1000001080000 + size = 18 + rendered = true + output buffer #28: + timeUs = 1000001280000 + size = 28 + rendered = true + output buffer #29: + timeUs = 1000001200000 + size = 27 + rendered = true + output buffer #30: + timeUs = 1000001160000 + size = 18 + rendered = true + output buffer #31: + timeUs = 1000001240000 + size = 18 + rendered = true + output buffer #32: + timeUs = 1000001440000 + size = 28 + rendered = true + output buffer #33: + timeUs = 1000001360000 + size = 27 + rendered = true + output buffer #34: + timeUs = 1000001320000 + size = 18 + rendered = true + output buffer #35: + timeUs = 1000001400000 + size = 18 + rendered = true + output buffer #36: + timeUs = 1000001600000 + size = 28 + rendered = true + output buffer #37: + timeUs = 1000001520000 + size = 27 + rendered = true + output buffer #38: + timeUs = 1000001480000 + size = 18 + rendered = true + output buffer #39: + timeUs = 1000001560000 + size = 18 + rendered = true + output buffer #40: + timeUs = 1000001760000 + size = 28 + rendered = true + output buffer #41: + timeUs = 1000001680000 + size = 27 + rendered = true + output buffer #42: + timeUs = 1000001640000 + size = 18 + rendered = true + output buffer #43: + timeUs = 1000001720000 + size = 18 + rendered = true + output buffer #44: + timeUs = 1000001920000 + size = 28 + rendered = true + output buffer #45: + timeUs = 1000001840000 + size = 27 + rendered = true + output buffer #46: + timeUs = 1000001800000 + size = 18 + rendered = true + output buffer #47: + timeUs = 1000001880000 + size = 18 + rendered = true + output buffer #48: + timeUs = 1000002080000 + size = 28 + rendered = false + output buffer #49: + timeUs = 1000002000000 + size = 27 + rendered = false + output buffer #50: + timeUs = 1000001960000 + size = 18 + rendered = true +AudioSink: + buffer count = 82 + config: + pcmEncoding = pcm-16 + channelCount = 1 + sampleRate = 44100 + buffer #0: + time = 1000000116099 + data = empty + buffer #1: + time = 1000000139319 + data = empty + buffer #2: + time = 1000000162539 + data = empty + buffer #3: + time = 1000000185759 + data = empty + buffer #4: + time = 1000000208979 + data = empty + buffer #5: + time = 1000000232199 + data = empty + buffer #6: + time = 1000000255419 + data = empty + buffer #7: + time = 1000000278639 + data = empty + buffer #8: + time = 1000000301859 + data = empty + buffer #9: + time = 1000000325079 + data = empty + buffer #10: + time = 1000000348299 + data = empty + buffer #11: + time = 1000000371519 + data = empty + buffer #12: + time = 1000000394739 + data = empty + buffer #13: + time = 1000000417959 + data = empty + buffer #14: + time = 1000000441179 + data = empty + buffer #15: + time = 1000000464399 + data = empty + buffer #16: + time = 1000000487619 + data = empty + buffer #17: + time = 1000000510839 + data = empty + buffer #18: + time = 1000000534058 + data = empty + buffer #19: + time = 1000000557278 + data = empty + buffer #20: + time = 1000000580498 + data = empty + buffer #21: + time = 1000000603718 + data = empty + buffer #22: + time = 1000000626938 + data = empty + buffer #23: + time = 1000000650158 + data = empty + buffer #24: + time = 1000000673378 + data = empty + buffer #25: + time = 1000000696598 + data = empty + buffer #26: + time = 1000000719818 + data = empty + buffer #27: + time = 1000000743038 + data = empty + buffer #28: + time = 1000000766258 + data = empty + buffer #29: + time = 1000000789478 + data = empty + buffer #30: + time = 1000000812698 + data = empty + buffer #31: + time = 1000000835918 + data = empty + buffer #32: + time = 1000000859138 + data = empty + buffer #33: + time = 1000000882358 + data = empty + buffer #34: + time = 1000000905578 + data = empty + buffer #35: + time = 1000000928798 + data = empty + buffer #36: + time = 1000000952018 + data = empty + buffer #37: + time = 1000000975238 + data = empty + buffer #38: + time = 1000000998458 + data = empty + buffer #39: + time = 1000001021678 + data = empty + buffer #40: + time = 1000001044897 + data = empty + buffer #41: + time = 1000001068117 + data = empty + buffer #42: + time = 1000001091337 + data = empty + buffer #43: + time = 1000001114557 + data = empty + buffer #44: + time = 1000001137777 + data = empty + buffer #45: + time = 1000001160997 + data = empty + buffer #46: + time = 1000001184217 + data = empty + buffer #47: + time = 1000001207437 + data = empty + buffer #48: + time = 1000001230657 + data = empty + buffer #49: + time = 1000001253877 + data = empty + buffer #50: + time = 1000001277097 + data = empty + buffer #51: + time = 1000001300317 + data = empty + buffer #52: + time = 1000001323537 + data = empty + buffer #53: + time = 1000001346757 + data = empty + buffer #54: + time = 1000001369977 + data = empty + buffer #55: + time = 1000001393197 + data = empty + buffer #56: + time = 1000001416417 + data = empty + buffer #57: + time = 1000001439637 + data = empty + buffer #58: + time = 1000001462857 + data = empty + buffer #59: + time = 1000001486077 + data = empty + buffer #60: + time = 1000001509297 + data = empty + buffer #61: + time = 1000001532517 + data = empty + buffer #62: + time = 1000001555736 + data = empty + buffer #63: + time = 1000001578956 + data = empty + buffer #64: + time = 1000001602176 + data = empty + buffer #65: + time = 1000001625396 + data = empty + buffer #66: + time = 1000001648616 + data = empty + buffer #67: + time = 1000001671836 + data = empty + buffer #68: + time = 1000001695056 + data = empty + buffer #69: + time = 1000001718276 + data = empty + buffer #70: + time = 1000001741496 + data = empty + buffer #71: + time = 1000001764716 + data = empty + buffer #72: + time = 1000001787936 + data = empty + buffer #73: + time = 1000001811156 + data = empty + buffer #74: + time = 1000001834376 + data = empty + buffer #75: + time = 1000001857596 + data = empty + buffer #76: + time = 1000001880816 + data = empty + buffer #77: + time = 1000001904036 + data = empty + buffer #78: + time = 1000001927256 + data = empty + buffer #79: + time = 1000001950476 + data = empty + buffer #80: + time = 1000001973696 + data = empty + buffer #81: + time = 1000001996916 + data = empty diff --git a/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_reduce_from_end_of_source_after_load_finished_enabled.dump b/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_reduce_from_end_of_source_after_load_finished_enabled.dump new file mode 100644 index 00000000000..91d65843aa1 --- /dev/null +++ b/libraries/test_data/src/test/assets/playbackdumps/clipping/replace_reduce_from_end_of_source_after_load_finished_enabled.dump @@ -0,0 +1,2830 @@ +extracted samples: + sample #0: + track = 0 + timeUs = 0 + flags = 1 + size = 5252 + sample #1: + track = 0 + timeUs = 160000 + size = 44 + sample #2: + track = 0 + timeUs = 80000 + size = 25 + sample #3: + track = 0 + timeUs = 40000 + flags = 67108864 + size = 18 + sample #4: + track = 1 + timeUs = 92879 + flags = 1 + size = 4 + sample #5: + track = 0 + timeUs = 120000 + flags = 67108864 + size = 18 + sample #6: + track = 1 + timeUs = 116099 + flags = 1 + size = 4 + sample #7: + track = 1 + timeUs = 139319 + flags = 1 + size = 4 + sample #8: + track = 0 + timeUs = 320000 + size = 32 + sample #9: + track = 1 + timeUs = 162539 + flags = 1 + size = 4 + sample #10: + track = 0 + timeUs = 240000 + size = 27 + sample #11: + track = 1 + timeUs = 185759 + flags = 1 + size = 4 + sample #12: + track = 1 + timeUs = 208979 + flags = 1 + size = 4 + sample #13: + track = 0 + timeUs = 200000 + flags = 67108864 + size = 18 + sample #14: + track = 1 + timeUs = 232199 + flags = 1 + size = 4 + sample #15: + track = 1 + timeUs = 255419 + flags = 1 + size = 4 + sample #16: + track = 0 + timeUs = 280000 + flags = 67108864 + size = 18 + sample #17: + track = 1 + timeUs = 278639 + flags = 1 + size = 4 + sample #18: + track = 1 + timeUs = 301859 + flags = 1 + size = 4 + sample #19: + track = 0 + timeUs = 480000 + size = 28 + sample #20: + track = 1 + timeUs = 325079 + flags = 1 + size = 4 + sample #21: + track = 0 + timeUs = 400000 + size = 27 + sample #22: + track = 1 + timeUs = 348299 + flags = 1 + size = 4 + sample #23: + track = 1 + timeUs = 371519 + flags = 1 + size = 4 + sample #24: + track = 0 + timeUs = 360000 + flags = 67108864 + size = 18 + sample #25: + track = 1 + timeUs = 394739 + flags = 1 + size = 4 + sample #26: + track = 1 + timeUs = 417959 + flags = 1 + size = 4 + sample #27: + track = 0 + timeUs = 440000 + flags = 67108864 + size = 18 + sample #28: + track = 1 + timeUs = 441179 + flags = 1 + size = 4 + sample #29: + track = 0 + timeUs = 640000 + size = 28 + sample #30: + track = 1 + timeUs = 464399 + flags = 1 + size = 4 + sample #31: + track = 1 + timeUs = 487619 + flags = 1 + size = 4 + sample #32: + track = 0 + timeUs = 560000 + size = 27 + sample #33: + track = 1 + timeUs = 510839 + flags = 1 + size = 4 + sample #34: + track = 1 + timeUs = 534058 + flags = 1 + size = 4 + sample #35: + track = 0 + timeUs = 520000 + flags = 67108864 + size = 18 + sample #36: + track = 1 + timeUs = 557278 + flags = 1 + size = 4 + sample #37: + track = 1 + timeUs = 580498 + flags = 1 + size = 4 + sample #38: + track = 0 + timeUs = 600000 + flags = 67108864 + size = 18 + sample #39: + track = 1 + timeUs = 603718 + flags = 1 + size = 4 + sample #40: + track = 0 + timeUs = 800000 + size = 28 + sample #41: + track = 1 + timeUs = 626938 + flags = 1 + size = 4 + sample #42: + track = 1 + timeUs = 650158 + flags = 1 + size = 4 + sample #43: + track = 0 + timeUs = 720000 + size = 27 + sample #44: + track = 1 + timeUs = 673378 + flags = 1 + size = 4 + sample #45: + track = 1 + timeUs = 696598 + flags = 1 + size = 4 + sample #46: + track = 0 + timeUs = 680000 + flags = 67108864 + size = 18 + sample #47: + track = 1 + timeUs = 719818 + flags = 1 + size = 4 + sample #48: + track = 1 + timeUs = 743038 + flags = 1 + size = 4 + sample #49: + track = 0 + timeUs = 760000 + flags = 67108864 + size = 18 + sample #50: + track = 1 + timeUs = 766258 + flags = 1 + size = 4 + sample #51: + track = 0 + timeUs = 960000 + size = 28 + sample #52: + track = 1 + timeUs = 789478 + flags = 1 + size = 4 + sample #53: + track = 1 + timeUs = 812698 + flags = 1 + size = 4 + sample #54: + track = 0 + timeUs = 880000 + size = 27 + sample #55: + track = 1 + timeUs = 835918 + flags = 1 + size = 4 + sample #56: + track = 1 + timeUs = 859138 + flags = 1 + size = 4 + sample #57: + track = 0 + timeUs = 840000 + flags = 67108864 + size = 18 + sample #58: + track = 1 + timeUs = 882358 + flags = 1 + size = 4 + sample #59: + track = 0 + timeUs = 920000 + flags = 67108864 + size = 18 + sample #60: + track = 1 + timeUs = 905578 + flags = 1 + size = 4 + sample #61: + track = 1 + timeUs = 928798 + flags = 1 + size = 4 + sample #62: + track = 0 + timeUs = 1120000 + size = 28 + sample #63: + track = 1 + timeUs = 952018 + flags = 1 + size = 4 + sample #64: + track = 1 + timeUs = 975238 + flags = 1 + size = 4 + sample #65: + track = 0 + timeUs = 1040000 + size = 27 + sample #66: + track = 1 + timeUs = 998458 + flags = 1 + size = 4 + sample #67: + track = 1 + timeUs = 1021678 + flags = 1 + size = 4 + sample #68: + track = 0 + timeUs = 1000000 + flags = 67108864 + size = 18 + sample #69: + track = 1 + timeUs = 1044897 + flags = 1 + size = 4 + sample #70: + track = 0 + timeUs = 1080000 + flags = 67108864 + size = 18 + sample #71: + track = 1 + timeUs = 1068117 + flags = 1 + size = 4 + sample #72: + track = 1 + timeUs = 1091337 + flags = 1 + size = 4 + sample #73: + track = 0 + timeUs = 1280000 + size = 28 + sample #74: + track = 1 + timeUs = 1114557 + flags = 1 + size = 4 + sample #75: + track = 1 + timeUs = 1137777 + flags = 1 + size = 4 + sample #76: + track = 0 + timeUs = 1200000 + size = 27 + sample #77: + track = 1 + timeUs = 1160997 + flags = 1 + size = 4 + sample #78: + track = 0 + timeUs = 1160000 + flags = 67108864 + size = 18 + sample #79: + track = 1 + timeUs = 1184217 + flags = 1 + size = 4 + sample #80: + track = 1 + timeUs = 1207437 + flags = 1 + size = 4 + sample #81: + track = 0 + timeUs = 1240000 + flags = 67108864 + size = 18 + sample #82: + track = 1 + timeUs = 1230657 + flags = 1 + size = 4 + sample #83: + track = 1 + timeUs = 1253877 + flags = 1 + size = 4 + sample #84: + track = 0 + timeUs = 1440000 + size = 28 + sample #85: + track = 1 + timeUs = 1277097 + flags = 1 + size = 4 + sample #86: + track = 1 + timeUs = 1300317 + flags = 1 + size = 4 + sample #87: + track = 0 + timeUs = 1360000 + size = 27 + sample #88: + track = 1 + timeUs = 1323537 + flags = 1 + size = 4 + sample #89: + track = 0 + timeUs = 1320000 + flags = 67108864 + size = 18 + sample #90: + track = 1 + timeUs = 1346757 + flags = 1 + size = 4 + sample #91: + track = 1 + timeUs = 1369977 + flags = 1 + size = 4 + sample #92: + track = 0 + timeUs = 1400000 + flags = 67108864 + size = 18 + sample #93: + track = 1 + timeUs = 1393197 + flags = 1 + size = 4 + sample #94: + track = 1 + timeUs = 1416417 + flags = 1 + size = 4 + sample #95: + track = 0 + timeUs = 1600000 + size = 28 + sample #96: + track = 1 + timeUs = 1439637 + flags = 1 + size = 4 + sample #97: + track = 1 + timeUs = 1462857 + flags = 1 + size = 4 + sample #98: + track = 0 + timeUs = 1520000 + size = 27 + sample #99: + track = 1 + timeUs = 1486077 + flags = 1 + size = 4 + sample #100: + track = 0 + timeUs = 1480000 + flags = 67108864 + size = 18 + sample #101: + track = 1 + timeUs = 1509297 + flags = 1 + size = 4 + sample #102: + track = 1 + timeUs = 1532517 + flags = 1 + size = 4 + sample #103: + track = 0 + timeUs = 1560000 + flags = 67108864 + size = 18 + sample #104: + track = 1 + timeUs = 1555736 + flags = 1 + size = 4 + sample #105: + track = 1 + timeUs = 1578956 + flags = 1 + size = 4 + sample #106: + track = 0 + timeUs = 1760000 + size = 28 + sample #107: + track = 1 + timeUs = 1602176 + flags = 1 + size = 4 + sample #108: + track = 0 + timeUs = 1680000 + size = 27 + sample #109: + track = 1 + timeUs = 1625396 + flags = 1 + size = 4 + sample #110: + track = 1 + timeUs = 1648616 + flags = 1 + size = 4 + sample #111: + track = 0 + timeUs = 1640000 + flags = 67108864 + size = 18 + sample #112: + track = 1 + timeUs = 1671836 + flags = 1 + size = 4 + sample #113: + track = 1 + timeUs = 1695056 + flags = 1 + size = 4 + sample #114: + track = 0 + timeUs = 1720000 + flags = 67108864 + size = 18 + sample #115: + track = 1 + timeUs = 1718276 + flags = 1 + size = 4 + sample #116: + track = 1 + timeUs = 1741496 + flags = 1 + size = 4 + sample #117: + track = 0 + timeUs = 1920000 + size = 28 + sample #118: + track = 1 + timeUs = 1764716 + flags = 1 + size = 4 + sample #119: + track = 0 + timeUs = 1840000 + size = 27 + sample #120: + track = 1 + timeUs = 1787936 + flags = 1 + size = 4 + sample #121: + track = 1 + timeUs = 1811156 + flags = 1 + size = 4 + sample #122: + track = 0 + timeUs = 1800000 + flags = 67108864 + size = 18 + sample #123: + track = 1 + timeUs = 1834376 + flags = 1 + size = 4 + sample #124: + track = 1 + timeUs = 1857596 + flags = 1 + size = 4 + sample #125: + track = 0 + timeUs = 1880000 + flags = 67108864 + size = 18 + sample #126: + track = 1 + timeUs = 1880816 + flags = 1 + size = 4 + sample #127: + track = 0 + timeUs = 2080000 + size = 28 + sample #128: + track = 1 + timeUs = 1904036 + flags = 1 + size = 4 + sample #129: + track = 1 + timeUs = 1927256 + flags = 1 + size = 4 + sample #130: + track = 0 + timeUs = 2000000 + size = 27 + sample #131: + track = 1 + timeUs = 1950476 + flags = 1 + size = 4 + sample #132: + track = 1 + timeUs = 1973696 + flags = 1 + size = 4 + sample #133: + track = 0 + timeUs = 1960000 + flags = 67108864 + size = 18 + sample #134: + track = 1 + timeUs = 1996916 + flags = 1 + size = 4 + sample #135: + track = 1 + timeUs = 2020136 + flags = 1 + size = 4 + sample #136: + track = 0 + timeUs = 2040000 + flags = 67108864 + size = 18 + sample #137: + track = 1 + timeUs = 2043356 + flags = 1 + size = 4 + sample #138: + track = 0 + timeUs = 2240000 + size = 28 + sample #139: + track = 1 + timeUs = 2066575 + flags = 1 + size = 4 + sample #140: + track = 1 + timeUs = 2089795 + flags = 1 + size = 4 + sample #141: + track = 0 + timeUs = 2160000 + size = 27 + sample #142: + track = 1 + timeUs = 2113015 + flags = 1 + size = 4 + sample #143: + track = 1 + timeUs = 2136235 + flags = 1 + size = 4 + sample #144: + track = 0 + timeUs = 2120000 + flags = 67108864 + size = 18 + sample #145: + track = 1 + timeUs = 2159455 + flags = 1 + size = 4 + sample #146: + track = 1 + timeUs = 2182675 + flags = 1 + size = 4 + sample #147: + track = 0 + timeUs = 2200000 + flags = 67108864 + size = 18 + sample #148: + track = 1 + timeUs = 2205895 + flags = 1 + size = 4 + sample #149: + track = 0 + timeUs = 2400000 + size = 28 + sample #150: + track = 1 + timeUs = 2229115 + flags = 1 + size = 4 + sample #151: + track = 1 + timeUs = 2252335 + flags = 1 + size = 4 + sample #152: + track = 0 + timeUs = 2320000 + size = 27 + sample #153: + track = 1 + timeUs = 2275555 + flags = 1 + size = 4 + sample #154: + track = 1 + timeUs = 2298775 + flags = 1 + size = 4 + sample #155: + track = 0 + timeUs = 2280000 + flags = 67108864 + size = 18 + sample #156: + track = 1 + timeUs = 2321995 + flags = 1 + size = 4 + sample #157: + track = 0 + timeUs = 2360000 + flags = 67108864 + size = 18 + sample #158: + track = 1 + timeUs = 2345215 + flags = 1 + size = 4 + sample #159: + track = 1 + timeUs = 2368435 + flags = 1 + size = 4 + sample #160: + track = 0 + timeUs = 2560000 + size = 28 + sample #161: + track = 1 + timeUs = 2391655 + flags = 1 + size = 4 + sample #162: + track = 1 + timeUs = 2414875 + flags = 1 + size = 4 + sample #163: + track = 0 + timeUs = 2480000 + size = 27 + sample #164: + track = 1 + timeUs = 2438095 + flags = 1 + size = 4 + sample #165: + track = 1 + timeUs = 2461315 + flags = 1 + size = 4 + sample #166: + track = 0 + timeUs = 2440000 + flags = 67108864 + size = 18 + sample #167: + track = 1 + timeUs = 2484535 + flags = 1 + size = 4 + sample #168: + track = 0 + timeUs = 2520000 + flags = 67108864 + size = 18 + sample #169: + track = 1 + timeUs = 2507755 + flags = 1 + size = 4 + sample #170: + track = 1 + timeUs = 2530975 + flags = 1 + size = 4 + sample #171: + track = 0 + timeUs = 2720000 + size = 28 + sample #172: + track = 1 + timeUs = 2554195 + flags = 1 + size = 4 + sample #173: + track = 1 + timeUs = 2577414 + flags = 1 + size = 4 + sample #174: + track = 0 + timeUs = 2640000 + size = 27 + sample #175: + track = 1 + timeUs = 2600634 + flags = 1 + size = 4 + sample #176: + track = 0 + timeUs = 2600000 + flags = 67108864 + size = 18 + sample #177: + track = 1 + timeUs = 2623854 + flags = 1 + size = 4 + sample #178: + track = 1 + timeUs = 2647074 + flags = 1 + size = 4 + sample #179: + track = 0 + timeUs = 2680000 + flags = 67108864 + size = 18 + sample #180: + track = 1 + timeUs = 2670294 + flags = 1 + size = 4 + sample #181: + track = 1 + timeUs = 2693514 + flags = 1 + size = 4 + sample #182: + track = 0 + timeUs = 2880000 + size = 28 + sample #183: + track = 1 + timeUs = 2716734 + flags = 1 + size = 4 + sample #184: + track = 1 + timeUs = 2739954 + flags = 1 + size = 4 + sample #185: + track = 0 + timeUs = 2800000 + size = 27 + sample #186: + track = 1 + timeUs = 2763174 + flags = 1 + size = 4 + sample #187: + track = 0 + timeUs = 2760000 + flags = 67108864 + size = 18 + sample #188: + track = 1 + timeUs = 2786394 + flags = 1 + size = 4 + sample #189: + track = 1 + timeUs = 2809614 + flags = 1 + size = 4 + sample #190: + track = 0 + timeUs = 2840000 + flags = 67108864 + size = 18 + sample #191: + track = 1 + timeUs = 2832834 + flags = 1 + size = 4 + sample #192: + track = 1 + timeUs = 2856054 + flags = 1 + size = 4 + sample #193: + track = 0 + timeUs = 3040000 + size = 28 + sample #194: + track = 1 + timeUs = 2879274 + flags = 1 + size = 4 + sample #195: + track = 1 + timeUs = 2902494 + flags = 1 + size = 4 + sample #196: + track = 0 + timeUs = 2960000 + size = 27 + sample #197: + track = 1 + timeUs = 2925714 + flags = 1 + size = 4 + sample #198: + track = 0 + timeUs = 2920000 + flags = 67108864 + size = 18 + sample #199: + track = 1 + timeUs = 2948934 + flags = 1 + size = 4 + sample #200: + track = 1 + timeUs = 2972154 + flags = 1 + size = 4 + sample #201: + track = 0 + timeUs = 3000000 + flags = 67108864 + size = 18 + sample #202: + track = 1 + timeUs = 2995374 + flags = 1 + size = 4 + sample #203: + track = 1 + timeUs = 3018594 + flags = 1 + size = 4 + sample #204: + track = 0 + timeUs = 3200000 + size = 28 + sample #205: + track = 1 + timeUs = 3041814 + flags = 1 + size = 4 + sample #206: + track = 0 + timeUs = 3120000 + size = 27 + sample #207: + track = 1 + timeUs = 3065034 + flags = 1 + size = 4 + sample #208: + track = 1 + timeUs = 3088253 + flags = 1 + size = 4 + sample #209: + track = 0 + timeUs = 3080000 + flags = 67108864 + size = 18 + sample #210: + track = 1 + timeUs = 3111473 + flags = 1 + size = 4 + sample #211: + track = 1 + timeUs = 3134693 + flags = 1 + size = 4 + sample #212: + track = 0 + timeUs = 3160000 + flags = 67108864 + size = 18 + sample #213: + track = 1 + timeUs = 3157913 + flags = 1 + size = 4 + sample #214: + track = 1 + timeUs = 3181133 + flags = 1 + size = 4 + sample #215: + track = 0 + timeUs = 3360000 + size = 28 + sample #216: + track = 1 + timeUs = 3204353 + flags = 1 + size = 4 + sample #217: + track = 0 + timeUs = 3280000 + size = 27 + sample #218: + track = 1 + timeUs = 3227573 + flags = 1 + size = 4 + sample #219: + track = 1 + timeUs = 3250793 + flags = 1 + size = 4 + sample #220: + track = 0 + timeUs = 3240000 + flags = 67108864 + size = 18 + sample #221: + track = 1 + timeUs = 3274013 + flags = 1 + size = 4 + sample #222: + track = 1 + timeUs = 3297233 + flags = 1 + size = 4 + sample #223: + track = 0 + timeUs = 3320000 + flags = 67108864 + size = 18 + sample #224: + track = 1 + timeUs = 3320453 + flags = 1 + size = 4 + sample #225: + track = 0 + timeUs = 3520000 + size = 28 + sample #226: + track = 1 + timeUs = 3343673 + flags = 1 + size = 4 + sample #227: + track = 1 + timeUs = 3366893 + flags = 1 + size = 4 + sample #228: + track = 0 + timeUs = 3440000 + size = 27 + sample #229: + track = 1 + timeUs = 3390113 + flags = 1 + size = 4 + sample #230: + track = 1 + timeUs = 3413333 + flags = 1 + size = 4 + sample #231: + track = 0 + timeUs = 3400000 + flags = 67108864 + size = 18 + sample #232: + track = 1 + timeUs = 3436553 + flags = 1 + size = 4 + sample #233: + track = 1 + timeUs = 3459773 + flags = 1 + size = 4 + sample #234: + track = 0 + timeUs = 3480000 + flags = 67108864 + size = 18 + sample #235: + track = 1 + timeUs = 3482993 + flags = 1 + size = 4 + sample #236: + track = 0 + timeUs = 3680000 + size = 28 + sample #237: + track = 1 + timeUs = 3506213 + flags = 1 + size = 4 + sample #238: + track = 1 + timeUs = 3529433 + flags = 1 + size = 4 + sample #239: + track = 0 + timeUs = 3600000 + size = 27 + sample #240: + track = 1 + timeUs = 3552653 + flags = 1 + size = 4 + sample #241: + track = 1 + timeUs = 3575873 + flags = 1 + size = 4 + sample #242: + track = 0 + timeUs = 3560000 + flags = 67108864 + size = 18 + sample #243: + track = 1 + timeUs = 3599092 + flags = 1 + size = 4 + sample #244: + track = 1 + timeUs = 3622312 + flags = 1 + size = 4 + sample #245: + track = 0 + timeUs = 3640000 + flags = 67108864 + size = 18 + sample #246: + track = 1 + timeUs = 3645532 + flags = 1 + size = 4 + sample #247: + track = 0 + timeUs = 3840000 + size = 33 + sample #248: + track = 1 + timeUs = 3668752 + flags = 1 + size = 4 + sample #249: + track = 1 + timeUs = 3691972 + flags = 1 + size = 4 + sample #250: + track = 0 + timeUs = 3760000 + size = 27 + sample #251: + track = 1 + timeUs = 3715192 + flags = 1 + size = 4 + sample #252: + track = 1 + timeUs = 3738412 + flags = 1 + size = 4 + sample #253: + track = 0 + timeUs = 3720000 + flags = 67108864 + size = 18 + sample #254: + track = 1 + timeUs = 3761632 + flags = 1 + size = 4 + sample #255: + track = 0 + timeUs = 3800000 + flags = 67108864 + size = 18 + sample #256: + track = 1 + timeUs = 3784852 + flags = 1 + size = 4 + sample #257: + track = 1 + timeUs = 3808072 + flags = 1 + size = 4 + sample #258: + track = 0 + timeUs = 4000000 + size = 33 + sample #259: + track = 1 + timeUs = 3831292 + flags = 1 + size = 4 + sample #260: + track = 1 + timeUs = 3854512 + flags = 1 + size = 4 + sample #261: + track = 0 + timeUs = 3920000 + size = 27 + sample #262: + track = 1 + timeUs = 3877732 + flags = 1 + size = 4 + sample #263: + track = 1 + timeUs = 3900952 + flags = 1 + size = 4 + sample #264: + track = 0 + timeUs = 3880000 + flags = 67108864 + size = 18 + sample #265: + track = 1 + timeUs = 3924172 + flags = 1 + size = 4 + sample #266: + track = 0 + timeUs = 3960000 + flags = 67108864 + size = 18 + sample #267: + track = 1 + timeUs = 3947392 + flags = 1 + size = 4 + sample #268: + track = 1 + timeUs = 3970612 + flags = 1 + size = 4 + sample #269: + track = 0 + timeUs = 4160000 + size = 28 + sample #270: + track = 1 + timeUs = 3993832 + flags = 1 + size = 4 + sample #271: + track = 1 + timeUs = 4017052 + flags = 1 + size = 4 + sample #272: + track = 0 + timeUs = 4080000 + size = 27 + sample #273: + track = 1 + timeUs = 4040272 + flags = 1 + size = 4 + sample #274: + track = 0 + timeUs = 4040000 + flags = 67108864 + size = 18 + sample #275: + track = 1 + timeUs = 4063492 + flags = 1 + size = 4 + sample #276: + track = 1 + timeUs = 4086712 + flags = 1 + size = 4 + sample #277: + track = 0 + timeUs = 4120000 + flags = 67108864 + size = 18 + sample #278: + track = 1 + timeUs = 4109931 + flags = 1 + size = 4 + sample #279: + track = 1 + timeUs = 4133151 + flags = 1 + size = 4 + sample #280: + track = 0 + timeUs = 4320000 + size = 28 + sample #281: + track = 1 + timeUs = 4156371 + flags = 1 + size = 4 + sample #282: + track = 1 + timeUs = 4179591 + flags = 1 + size = 4 + sample #283: + track = 0 + timeUs = 4240000 + size = 27 + sample #284: + track = 1 + timeUs = 4202811 + flags = 1 + size = 4 + sample #285: + track = 0 + timeUs = 4200000 + flags = 67108864 + size = 18 + sample #286: + track = 1 + timeUs = 4226031 + flags = 1 + size = 4 + sample #287: + track = 1 + timeUs = 4249251 + flags = 1 + size = 4 + sample #288: + track = 0 + timeUs = 4280000 + flags = 67108864 + size = 18 + sample #289: + track = 1 + timeUs = 4272471 + flags = 1 + size = 4 + sample #290: + track = 1 + timeUs = 4295691 + flags = 1 + size = 4 + sample #291: + track = 0 + timeUs = 4480000 + size = 28 + sample #292: + track = 1 + timeUs = 4318911 + flags = 1 + size = 4 + sample #293: + track = 1 + timeUs = 4342131 + flags = 1 + size = 4 + sample #294: + track = 0 + timeUs = 4400000 + size = 27 + sample #295: + track = 1 + timeUs = 4365351 + flags = 1 + size = 4 + sample #296: + track = 0 + timeUs = 4360000 + flags = 67108864 + size = 18 + sample #297: + track = 1 + timeUs = 4388571 + flags = 1 + size = 4 + sample #298: + track = 1 + timeUs = 4411791 + flags = 1 + size = 4 + sample #299: + track = 0 + timeUs = 4440000 + flags = 67108864 + size = 18 + sample #300: + track = 1 + timeUs = 4435011 + flags = 1 + size = 4 + sample #301: + track = 1 + timeUs = 4458231 + flags = 1 + size = 4 + sample #302: + track = 0 + timeUs = 4640000 + size = 27 + sample #303: + track = 1 + timeUs = 4481451 + flags = 1 + size = 4 + sample #304: + track = 0 + timeUs = 4560000 + size = 27 + sample #305: + track = 1 + timeUs = 4504671 + flags = 1 + size = 4 + sample #306: + track = 1 + timeUs = 4527891 + flags = 1 + size = 4 + sample #307: + track = 0 + timeUs = 4520000 + flags = 67108864 + size = 18 + sample #308: + track = 1 + timeUs = 4551111 + flags = 1 + size = 4 + sample #309: + track = 1 + timeUs = 4574331 + flags = 1 + size = 4 + sample #310: + track = 0 + timeUs = 4600000 + flags = 67108864 + size = 18 + sample #311: + track = 1 + timeUs = 4597551 + flags = 1 + size = 4 + sample #312: + track = 1 + timeUs = 4620770 + flags = 1 + size = 4 + sample #313: + track = 0 + timeUs = 4800000 + size = 26 + sample #314: + track = 1 + timeUs = 4643990 + flags = 1 + size = 4 + sample #315: + track = 0 + timeUs = 4720000 + size = 27 + sample #316: + track = 1 + timeUs = 4667210 + flags = 1 + size = 4 + sample #317: + track = 1 + timeUs = 4690430 + flags = 1 + size = 4 + sample #318: + track = 0 + timeUs = 4680000 + flags = 67108864 + size = 18 + sample #319: + track = 1 + timeUs = 4713650 + flags = 1 + size = 4 + sample #320: + track = 1 + timeUs = 4736870 + flags = 1 + size = 4 + sample #321: + track = 0 + timeUs = 4760000 + flags = 67108864 + size = 18 + sample #322: + track = 1 + timeUs = 4760090 + flags = 1 + size = 4 + sample #323: + track = 0 + timeUs = 4960000 + size = 26 + sample #324: + track = 1 + timeUs = 4783310 + flags = 1 + size = 4 + sample #325: + track = 1 + timeUs = 4806530 + flags = 1 + size = 4 + sample #326: + track = 0 + timeUs = 4880000 + size = 27 + sample #327: + track = 1 + timeUs = 4829750 + flags = 1 + size = 4 + sample #328: + track = 1 + timeUs = 4852970 + flags = 1 + size = 4 + sample #329: + track = 0 + timeUs = 4840000 + flags = 67108864 + size = 18 + sample #330: + track = 1 + timeUs = 4876190 + flags = 1 + size = 4 + sample #331: + track = 1 + timeUs = 4899410 + flags = 1 + size = 4 + sample #332: + track = 0 + timeUs = 4920000 + flags = 603979776 + size = 18 + sample #333: + track = 1 + timeUs = 4922630 + flags = 1 + size = 4 + sample #334: + track = 1 + timeUs = 4945850 + flags = 1 + size = 4 + sample #335: + track = 1 + timeUs = 4969070 + flags = 1 + size = 4 + sample #336: + track = 1 + timeUs = 4992290 + flags = 1 + size = 4 + sample #337: + track = 1 + timeUs = 5015510 + flags = 536870913 + size = 4 +MediaCodecAdapter (media3.audio.aac): + inputBuffers: + count = 83 + input buffer #0: + timeUs = 1000000116099 + contents = length 4, hash EE9DF + input buffer #1: + timeUs = 1000000139319 + contents = length 4, hash EE9DF + input buffer #2: + timeUs = 1000000162539 + contents = length 4, hash EE9DF + input buffer #3: + timeUs = 1000000185759 + contents = length 4, hash EE9DF + input buffer #4: + timeUs = 1000000208979 + contents = length 4, hash EE9DF + input buffer #5: + timeUs = 1000000232199 + contents = length 4, hash EE9DF + input buffer #6: + timeUs = 1000000255419 + contents = length 4, hash EE9DF + input buffer #7: + timeUs = 1000000278639 + contents = length 4, hash EE9DF + input buffer #8: + timeUs = 1000000301859 + contents = length 4, hash EE9DF + input buffer #9: + timeUs = 1000000325079 + contents = length 4, hash EE9DF + input buffer #10: + timeUs = 1000000348299 + contents = length 4, hash EE9DF + input buffer #11: + timeUs = 1000000371519 + contents = length 4, hash EE9DF + input buffer #12: + timeUs = 1000000394739 + contents = length 4, hash EE9DF + input buffer #13: + timeUs = 1000000417959 + contents = length 4, hash EE9DF + input buffer #14: + timeUs = 1000000441179 + contents = length 4, hash EE9DF + input buffer #15: + timeUs = 1000000464399 + contents = length 4, hash EE9DF + input buffer #16: + timeUs = 1000000487619 + contents = length 4, hash EE9DF + input buffer #17: + timeUs = 1000000510839 + contents = length 4, hash EE9DF + input buffer #18: + timeUs = 1000000534058 + contents = length 4, hash EE9DF + input buffer #19: + timeUs = 1000000557278 + contents = length 4, hash EE9DF + input buffer #20: + timeUs = 1000000580498 + contents = length 4, hash EE9DF + input buffer #21: + timeUs = 1000000603718 + contents = length 4, hash EE9DF + input buffer #22: + timeUs = 1000000626938 + contents = length 4, hash EE9DF + input buffer #23: + timeUs = 1000000650158 + contents = length 4, hash EE9DF + input buffer #24: + timeUs = 1000000673378 + contents = length 4, hash EE9DF + input buffer #25: + timeUs = 1000000696598 + contents = length 4, hash EE9DF + input buffer #26: + timeUs = 1000000719818 + contents = length 4, hash EE9DF + input buffer #27: + timeUs = 1000000743038 + contents = length 4, hash EE9DF + input buffer #28: + timeUs = 1000000766258 + contents = length 4, hash EE9DF + input buffer #29: + timeUs = 1000000789478 + contents = length 4, hash EE9DF + input buffer #30: + timeUs = 1000000812698 + contents = length 4, hash EE9DF + input buffer #31: + timeUs = 1000000835918 + contents = length 4, hash EE9DF + input buffer #32: + timeUs = 1000000859138 + contents = length 4, hash EE9DF + input buffer #33: + timeUs = 1000000882358 + contents = length 4, hash EE9DF + input buffer #34: + timeUs = 1000000905578 + contents = length 4, hash EE9DF + input buffer #35: + timeUs = 1000000928798 + contents = length 4, hash EE9DF + input buffer #36: + timeUs = 1000000952018 + contents = length 4, hash EE9DF + input buffer #37: + timeUs = 1000000975238 + contents = length 4, hash EE9DF + input buffer #38: + timeUs = 1000000998458 + contents = length 4, hash EE9DF + input buffer #39: + timeUs = 1000001021678 + contents = length 4, hash EE9DF + input buffer #40: + timeUs = 1000001044897 + contents = length 4, hash EE9DF + input buffer #41: + timeUs = 1000001068117 + contents = length 4, hash EE9DF + input buffer #42: + timeUs = 1000001091337 + contents = length 4, hash EE9DF + input buffer #43: + timeUs = 1000001114557 + contents = length 4, hash EE9DF + input buffer #44: + timeUs = 1000001137777 + contents = length 4, hash EE9DF + input buffer #45: + timeUs = 1000001160997 + contents = length 4, hash EE9DF + input buffer #46: + timeUs = 1000001184217 + contents = length 4, hash EE9DF + input buffer #47: + timeUs = 1000001207437 + contents = length 4, hash EE9DF + input buffer #48: + timeUs = 1000001230657 + contents = length 4, hash EE9DF + input buffer #49: + timeUs = 1000001253877 + contents = length 4, hash EE9DF + input buffer #50: + timeUs = 1000001277097 + contents = length 4, hash EE9DF + input buffer #51: + timeUs = 1000001300317 + contents = length 4, hash EE9DF + input buffer #52: + timeUs = 1000001323537 + contents = length 4, hash EE9DF + input buffer #53: + timeUs = 1000001346757 + contents = length 4, hash EE9DF + input buffer #54: + timeUs = 1000001369977 + contents = length 4, hash EE9DF + input buffer #55: + timeUs = 1000001393197 + contents = length 4, hash EE9DF + input buffer #56: + timeUs = 1000001416417 + contents = length 4, hash EE9DF + input buffer #57: + timeUs = 1000001439637 + contents = length 4, hash EE9DF + input buffer #58: + timeUs = 1000001462857 + contents = length 4, hash EE9DF + input buffer #59: + timeUs = 1000001486077 + contents = length 4, hash EE9DF + input buffer #60: + timeUs = 1000001509297 + contents = length 4, hash EE9DF + input buffer #61: + timeUs = 1000001532517 + contents = length 4, hash EE9DF + input buffer #62: + timeUs = 1000001555736 + contents = length 4, hash EE9DF + input buffer #63: + timeUs = 1000001578956 + contents = length 4, hash EE9DF + input buffer #64: + timeUs = 1000001602176 + contents = length 4, hash EE9DF + input buffer #65: + timeUs = 1000001625396 + contents = length 4, hash EE9DF + input buffer #66: + timeUs = 1000001648616 + contents = length 4, hash EE9DF + input buffer #67: + timeUs = 1000001671836 + contents = length 4, hash EE9DF + input buffer #68: + timeUs = 1000001695056 + contents = length 4, hash EE9DF + input buffer #69: + timeUs = 1000001718276 + contents = length 4, hash EE9DF + input buffer #70: + timeUs = 1000001741496 + contents = length 4, hash EE9DF + input buffer #71: + timeUs = 1000001764716 + contents = length 4, hash EE9DF + input buffer #72: + timeUs = 1000001787936 + contents = length 4, hash EE9DF + input buffer #73: + timeUs = 1000001811156 + contents = length 4, hash EE9DF + input buffer #74: + timeUs = 1000001834376 + contents = length 4, hash EE9DF + input buffer #75: + timeUs = 1000001857596 + contents = length 4, hash EE9DF + input buffer #76: + timeUs = 1000001880816 + contents = length 4, hash EE9DF + input buffer #77: + timeUs = 1000001904036 + contents = length 4, hash EE9DF + input buffer #78: + timeUs = 1000001927256 + contents = length 4, hash EE9DF + input buffer #79: + timeUs = 1000001950476 + contents = length 4, hash EE9DF + input buffer #80: + timeUs = 1000001973696 + contents = length 4, hash EE9DF + input buffer #81: + timeUs = 1000001996916 + contents = length 4, hash EE9DF + input buffer #82: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 82 + output buffer #0: + timeUs = 1000000116099 + size = 0 + rendered = false + output buffer #1: + timeUs = 1000000139319 + size = 0 + rendered = false + output buffer #2: + timeUs = 1000000162539 + size = 0 + rendered = false + output buffer #3: + timeUs = 1000000185759 + size = 0 + rendered = false + output buffer #4: + timeUs = 1000000208979 + size = 0 + rendered = false + output buffer #5: + timeUs = 1000000232199 + size = 0 + rendered = false + output buffer #6: + timeUs = 1000000255419 + size = 0 + rendered = false + output buffer #7: + timeUs = 1000000278639 + size = 0 + rendered = false + output buffer #8: + timeUs = 1000000301859 + size = 0 + rendered = false + output buffer #9: + timeUs = 1000000325079 + size = 0 + rendered = false + output buffer #10: + timeUs = 1000000348299 + size = 0 + rendered = false + output buffer #11: + timeUs = 1000000371519 + size = 0 + rendered = false + output buffer #12: + timeUs = 1000000394739 + size = 0 + rendered = false + output buffer #13: + timeUs = 1000000417959 + size = 0 + rendered = false + output buffer #14: + timeUs = 1000000441179 + size = 0 + rendered = false + output buffer #15: + timeUs = 1000000464399 + size = 0 + rendered = false + output buffer #16: + timeUs = 1000000487619 + size = 0 + rendered = false + output buffer #17: + timeUs = 1000000510839 + size = 0 + rendered = false + output buffer #18: + timeUs = 1000000534058 + size = 0 + rendered = false + output buffer #19: + timeUs = 1000000557278 + size = 0 + rendered = false + output buffer #20: + timeUs = 1000000580498 + size = 0 + rendered = false + output buffer #21: + timeUs = 1000000603718 + size = 0 + rendered = false + output buffer #22: + timeUs = 1000000626938 + size = 0 + rendered = false + output buffer #23: + timeUs = 1000000650158 + size = 0 + rendered = false + output buffer #24: + timeUs = 1000000673378 + size = 0 + rendered = false + output buffer #25: + timeUs = 1000000696598 + size = 0 + rendered = false + output buffer #26: + timeUs = 1000000719818 + size = 0 + rendered = false + output buffer #27: + timeUs = 1000000743038 + size = 0 + rendered = false + output buffer #28: + timeUs = 1000000766258 + size = 0 + rendered = false + output buffer #29: + timeUs = 1000000789478 + size = 0 + rendered = false + output buffer #30: + timeUs = 1000000812698 + size = 0 + rendered = false + output buffer #31: + timeUs = 1000000835918 + size = 0 + rendered = false + output buffer #32: + timeUs = 1000000859138 + size = 0 + rendered = false + output buffer #33: + timeUs = 1000000882358 + size = 0 + rendered = false + output buffer #34: + timeUs = 1000000905578 + size = 0 + rendered = false + output buffer #35: + timeUs = 1000000928798 + size = 0 + rendered = false + output buffer #36: + timeUs = 1000000952018 + size = 0 + rendered = false + output buffer #37: + timeUs = 1000000975238 + size = 0 + rendered = false + output buffer #38: + timeUs = 1000000998458 + size = 0 + rendered = false + output buffer #39: + timeUs = 1000001021678 + size = 0 + rendered = false + output buffer #40: + timeUs = 1000001044897 + size = 0 + rendered = false + output buffer #41: + timeUs = 1000001068117 + size = 0 + rendered = false + output buffer #42: + timeUs = 1000001091337 + size = 0 + rendered = false + output buffer #43: + timeUs = 1000001114557 + size = 0 + rendered = false + output buffer #44: + timeUs = 1000001137777 + size = 0 + rendered = false + output buffer #45: + timeUs = 1000001160997 + size = 0 + rendered = false + output buffer #46: + timeUs = 1000001184217 + size = 0 + rendered = false + output buffer #47: + timeUs = 1000001207437 + size = 0 + rendered = false + output buffer #48: + timeUs = 1000001230657 + size = 0 + rendered = false + output buffer #49: + timeUs = 1000001253877 + size = 0 + rendered = false + output buffer #50: + timeUs = 1000001277097 + size = 0 + rendered = false + output buffer #51: + timeUs = 1000001300317 + size = 0 + rendered = false + output buffer #52: + timeUs = 1000001323537 + size = 0 + rendered = false + output buffer #53: + timeUs = 1000001346757 + size = 0 + rendered = false + output buffer #54: + timeUs = 1000001369977 + size = 0 + rendered = false + output buffer #55: + timeUs = 1000001393197 + size = 0 + rendered = false + output buffer #56: + timeUs = 1000001416417 + size = 0 + rendered = false + output buffer #57: + timeUs = 1000001439637 + size = 0 + rendered = false + output buffer #58: + timeUs = 1000001462857 + size = 0 + rendered = false + output buffer #59: + timeUs = 1000001486077 + size = 0 + rendered = false + output buffer #60: + timeUs = 1000001509297 + size = 0 + rendered = false + output buffer #61: + timeUs = 1000001532517 + size = 0 + rendered = false + output buffer #62: + timeUs = 1000001555736 + size = 0 + rendered = false + output buffer #63: + timeUs = 1000001578956 + size = 0 + rendered = false + output buffer #64: + timeUs = 1000001602176 + size = 0 + rendered = false + output buffer #65: + timeUs = 1000001625396 + size = 0 + rendered = false + output buffer #66: + timeUs = 1000001648616 + size = 0 + rendered = false + output buffer #67: + timeUs = 1000001671836 + size = 0 + rendered = false + output buffer #68: + timeUs = 1000001695056 + size = 0 + rendered = false + output buffer #69: + timeUs = 1000001718276 + size = 0 + rendered = false + output buffer #70: + timeUs = 1000001741496 + size = 0 + rendered = false + output buffer #71: + timeUs = 1000001764716 + size = 0 + rendered = false + output buffer #72: + timeUs = 1000001787936 + size = 0 + rendered = false + output buffer #73: + timeUs = 1000001811156 + size = 0 + rendered = false + output buffer #74: + timeUs = 1000001834376 + size = 0 + rendered = false + output buffer #75: + timeUs = 1000001857596 + size = 0 + rendered = false + output buffer #76: + timeUs = 1000001880816 + size = 0 + rendered = false + output buffer #77: + timeUs = 1000001904036 + size = 0 + rendered = false + output buffer #78: + timeUs = 1000001927256 + size = 0 + rendered = false + output buffer #79: + timeUs = 1000001950476 + size = 0 + rendered = false + output buffer #80: + timeUs = 1000001973696 + size = 0 + rendered = false + output buffer #81: + timeUs = 1000001996916 + size = 0 + rendered = false +MediaCodecAdapter (media3.video.avc): + inputBuffers: + count = 52 + input buffer #0: + timeUs = 1000000000000 + contents = length 5252, hash 13893A4C + input buffer #1: + timeUs = 1000000160000 + contents = length 44, hash A05B3BEA + input buffer #2: + timeUs = 1000000080000 + contents = length 25, hash 441EC4FE + input buffer #3: + timeUs = 1000000120000 + contents = length 18, hash B64DA059 + input buffer #4: + timeUs = 1000000320000 + contents = length 32, hash 5BFD9538 + input buffer #5: + timeUs = 1000000240000 + contents = length 27, hash 705869A5 + input buffer #6: + timeUs = 1000000200000 + contents = length 18, hash D163DF61 + input buffer #7: + timeUs = 1000000280000 + contents = length 18, hash FD82E95 + input buffer #8: + timeUs = 1000000480000 + contents = length 28, hash 44A16E72 + input buffer #9: + timeUs = 1000000400000 + contents = length 27, hash FFA326ED + input buffer #10: + timeUs = 1000000360000 + contents = length 18, hash DC93CC9D + input buffer #11: + timeUs = 1000000440000 + contents = length 18, hash 1B081BD1 + input buffer #12: + timeUs = 1000000640000 + contents = length 28, hash 2700AF + input buffer #13: + timeUs = 1000000560000 + contents = length 27, hash DECA17D0 + input buffer #14: + timeUs = 1000000520000 + contents = length 18, hash D646C05A + input buffer #15: + timeUs = 1000000600000 + contents = length 18, hash 14BB0F8E + input buffer #16: + timeUs = 1000000800000 + contents = length 28, hash 5DE2C2B + input buffer #17: + timeUs = 1000000720000 + contents = length 27, hash 376B4E14 + input buffer #18: + timeUs = 1000000680000 + contents = length 18, hash E176AD96 + input buffer #19: + timeUs = 1000000760000 + contents = length 18, hash 1FEAFCCA + input buffer #20: + timeUs = 1000000960000 + contents = length 28, hash C163BE68 + input buffer #21: + timeUs = 1000000880000 + contents = length 27, hash F7170357 + input buffer #22: + timeUs = 1000000840000 + contents = length 18, hash 3B013BD2 + input buffer #23: + timeUs = 1000000920000 + contents = length 18, hash 79758B06 + input buffer #24: + timeUs = 1000001120000 + contents = length 28, hash F72EB1A3 + input buffer #25: + timeUs = 1000001040000 + contents = length 27, hash 4FB8399C + input buffer #26: + timeUs = 1000001000000 + contents = length 18, hash 4631290E + input buffer #27: + timeUs = 1000001080000 + contents = length 18, hash 84A57842 + input buffer #28: + timeUs = 1000001280000 + contents = length 28, hash E1FCF000 + input buffer #29: + timeUs = 1000001200000 + contents = length 27, hash 21D3B540 + input buffer #30: + timeUs = 1000001160000 + contents = length 18, hash 62DE0FC9 + input buffer #31: + timeUs = 1000001240000 + contents = length 18, hash A1525EFD + input buffer #32: + timeUs = 1000001440000 + contents = length 28, hash 5350E8FA + input buffer #33: + timeUs = 1000001360000 + contents = length 27, hash E7563B65 + input buffer #34: + timeUs = 1000001320000 + contents = length 18, hash 77D95125 + input buffer #35: + timeUs = 1000001400000 + contents = length 18, hash B64DA059 + input buffer #36: + timeUs = 1000001600000 + contents = length 28, hash ED67B37 + input buffer #37: + timeUs = 1000001520000 + contents = length 27, hash A701F0A9 + input buffer #38: + timeUs = 1000001480000 + contents = length 18, hash D163DF61 + input buffer #39: + timeUs = 1000001560000 + contents = length 18, hash FD82E95 + input buffer #40: + timeUs = 1000001760000 + contents = length 28, hash 44A16E72 + input buffer #41: + timeUs = 1000001680000 + contents = length 27, hash FFA326ED + input buffer #42: + timeUs = 1000001640000 + contents = length 18, hash DC93CC9D + input buffer #43: + timeUs = 1000001720000 + contents = length 18, hash 1B081BD1 + input buffer #44: + timeUs = 1000001920000 + contents = length 28, hash 2700AF + input buffer #45: + timeUs = 1000001840000 + contents = length 27, hash DECA17D0 + input buffer #46: + timeUs = 1000001800000 + contents = length 18, hash D646C05A + input buffer #47: + timeUs = 1000001880000 + contents = length 18, hash 14BB0F8E + input buffer #48: + timeUs = 1000002080000 + contents = length 28, hash 5DE2C2B + input buffer #49: + timeUs = 1000002000000 + contents = length 27, hash 376B4E14 + input buffer #50: + timeUs = 1000001960000 + contents = length 18, hash E176AD96 + input buffer #51: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 51 + output buffer #0: + timeUs = 1000000000000 + size = 5252 + rendered = false + output buffer #1: + timeUs = 1000000160000 + size = 44 + rendered = true + output buffer #2: + timeUs = 1000000080000 + size = 25 + rendered = false + output buffer #3: + timeUs = 1000000120000 + size = 18 + rendered = true + output buffer #4: + timeUs = 1000000320000 + size = 32 + rendered = true + output buffer #5: + timeUs = 1000000240000 + size = 27 + rendered = true + output buffer #6: + timeUs = 1000000200000 + size = 18 + rendered = true + output buffer #7: + timeUs = 1000000280000 + size = 18 + rendered = true + output buffer #8: + timeUs = 1000000480000 + size = 28 + rendered = true + output buffer #9: + timeUs = 1000000400000 + size = 27 + rendered = true + output buffer #10: + timeUs = 1000000360000 + size = 18 + rendered = true + output buffer #11: + timeUs = 1000000440000 + size = 18 + rendered = true + output buffer #12: + timeUs = 1000000640000 + size = 28 + rendered = true + output buffer #13: + timeUs = 1000000560000 + size = 27 + rendered = true + output buffer #14: + timeUs = 1000000520000 + size = 18 + rendered = true + output buffer #15: + timeUs = 1000000600000 + size = 18 + rendered = true + output buffer #16: + timeUs = 1000000800000 + size = 28 + rendered = true + output buffer #17: + timeUs = 1000000720000 + size = 27 + rendered = true + output buffer #18: + timeUs = 1000000680000 + size = 18 + rendered = true + output buffer #19: + timeUs = 1000000760000 + size = 18 + rendered = true + output buffer #20: + timeUs = 1000000960000 + size = 28 + rendered = true + output buffer #21: + timeUs = 1000000880000 + size = 27 + rendered = true + output buffer #22: + timeUs = 1000000840000 + size = 18 + rendered = true + output buffer #23: + timeUs = 1000000920000 + size = 18 + rendered = true + output buffer #24: + timeUs = 1000001120000 + size = 28 + rendered = true + output buffer #25: + timeUs = 1000001040000 + size = 27 + rendered = true + output buffer #26: + timeUs = 1000001000000 + size = 18 + rendered = true + output buffer #27: + timeUs = 1000001080000 + size = 18 + rendered = true + output buffer #28: + timeUs = 1000001280000 + size = 28 + rendered = true + output buffer #29: + timeUs = 1000001200000 + size = 27 + rendered = true + output buffer #30: + timeUs = 1000001160000 + size = 18 + rendered = true + output buffer #31: + timeUs = 1000001240000 + size = 18 + rendered = true + output buffer #32: + timeUs = 1000001440000 + size = 28 + rendered = true + output buffer #33: + timeUs = 1000001360000 + size = 27 + rendered = true + output buffer #34: + timeUs = 1000001320000 + size = 18 + rendered = true + output buffer #35: + timeUs = 1000001400000 + size = 18 + rendered = true + output buffer #36: + timeUs = 1000001600000 + size = 28 + rendered = true + output buffer #37: + timeUs = 1000001520000 + size = 27 + rendered = true + output buffer #38: + timeUs = 1000001480000 + size = 18 + rendered = true + output buffer #39: + timeUs = 1000001560000 + size = 18 + rendered = true + output buffer #40: + timeUs = 1000001760000 + size = 28 + rendered = true + output buffer #41: + timeUs = 1000001680000 + size = 27 + rendered = true + output buffer #42: + timeUs = 1000001640000 + size = 18 + rendered = true + output buffer #43: + timeUs = 1000001720000 + size = 18 + rendered = true + output buffer #44: + timeUs = 1000001920000 + size = 28 + rendered = true + output buffer #45: + timeUs = 1000001840000 + size = 27 + rendered = true + output buffer #46: + timeUs = 1000001800000 + size = 18 + rendered = true + output buffer #47: + timeUs = 1000001880000 + size = 18 + rendered = true + output buffer #48: + timeUs = 1000002080000 + size = 28 + rendered = false + output buffer #49: + timeUs = 1000002000000 + size = 27 + rendered = false + output buffer #50: + timeUs = 1000001960000 + size = 18 + rendered = true +AudioSink: + buffer count = 82 + config: + pcmEncoding = pcm-16 + channelCount = 1 + sampleRate = 44100 + buffer #0: + time = 1000000116099 + data = empty + buffer #1: + time = 1000000139319 + data = empty + buffer #2: + time = 1000000162539 + data = empty + buffer #3: + time = 1000000185759 + data = empty + buffer #4: + time = 1000000208979 + data = empty + buffer #5: + time = 1000000232199 + data = empty + buffer #6: + time = 1000000255419 + data = empty + buffer #7: + time = 1000000278639 + data = empty + buffer #8: + time = 1000000301859 + data = empty + buffer #9: + time = 1000000325079 + data = empty + buffer #10: + time = 1000000348299 + data = empty + buffer #11: + time = 1000000371519 + data = empty + buffer #12: + time = 1000000394739 + data = empty + buffer #13: + time = 1000000417959 + data = empty + buffer #14: + time = 1000000441179 + data = empty + buffer #15: + time = 1000000464399 + data = empty + buffer #16: + time = 1000000487619 + data = empty + buffer #17: + time = 1000000510839 + data = empty + buffer #18: + time = 1000000534058 + data = empty + buffer #19: + time = 1000000557278 + data = empty + buffer #20: + time = 1000000580498 + data = empty + buffer #21: + time = 1000000603718 + data = empty + buffer #22: + time = 1000000626938 + data = empty + buffer #23: + time = 1000000650158 + data = empty + buffer #24: + time = 1000000673378 + data = empty + buffer #25: + time = 1000000696598 + data = empty + buffer #26: + time = 1000000719818 + data = empty + buffer #27: + time = 1000000743038 + data = empty + buffer #28: + time = 1000000766258 + data = empty + buffer #29: + time = 1000000789478 + data = empty + buffer #30: + time = 1000000812698 + data = empty + buffer #31: + time = 1000000835918 + data = empty + buffer #32: + time = 1000000859138 + data = empty + buffer #33: + time = 1000000882358 + data = empty + buffer #34: + time = 1000000905578 + data = empty + buffer #35: + time = 1000000928798 + data = empty + buffer #36: + time = 1000000952018 + data = empty + buffer #37: + time = 1000000975238 + data = empty + buffer #38: + time = 1000000998458 + data = empty + buffer #39: + time = 1000001021678 + data = empty + buffer #40: + time = 1000001044897 + data = empty + buffer #41: + time = 1000001068117 + data = empty + buffer #42: + time = 1000001091337 + data = empty + buffer #43: + time = 1000001114557 + data = empty + buffer #44: + time = 1000001137777 + data = empty + buffer #45: + time = 1000001160997 + data = empty + buffer #46: + time = 1000001184217 + data = empty + buffer #47: + time = 1000001207437 + data = empty + buffer #48: + time = 1000001230657 + data = empty + buffer #49: + time = 1000001253877 + data = empty + buffer #50: + time = 1000001277097 + data = empty + buffer #51: + time = 1000001300317 + data = empty + buffer #52: + time = 1000001323537 + data = empty + buffer #53: + time = 1000001346757 + data = empty + buffer #54: + time = 1000001369977 + data = empty + buffer #55: + time = 1000001393197 + data = empty + buffer #56: + time = 1000001416417 + data = empty + buffer #57: + time = 1000001439637 + data = empty + buffer #58: + time = 1000001462857 + data = empty + buffer #59: + time = 1000001486077 + data = empty + buffer #60: + time = 1000001509297 + data = empty + buffer #61: + time = 1000001532517 + data = empty + buffer #62: + time = 1000001555736 + data = empty + buffer #63: + time = 1000001578956 + data = empty + buffer #64: + time = 1000001602176 + data = empty + buffer #65: + time = 1000001625396 + data = empty + buffer #66: + time = 1000001648616 + data = empty + buffer #67: + time = 1000001671836 + data = empty + buffer #68: + time = 1000001695056 + data = empty + buffer #69: + time = 1000001718276 + data = empty + buffer #70: + time = 1000001741496 + data = empty + buffer #71: + time = 1000001764716 + data = empty + buffer #72: + time = 1000001787936 + data = empty + buffer #73: + time = 1000001811156 + data = empty + buffer #74: + time = 1000001834376 + data = empty + buffer #75: + time = 1000001857596 + data = empty + buffer #76: + time = 1000001880816 + data = empty + buffer #77: + time = 1000001904036 + data = empty + buffer #78: + time = 1000001927256 + data = empty + buffer #79: + time = 1000001950476 + data = empty + buffer #80: + time = 1000001973696 + data = empty + buffer #81: + time = 1000001996916 + data = empty diff --git a/libraries/test_data/src/test/assets/playbackdumps/prewarmingRenderer/multipleVideoImagePlaylist-withSecondaryRenderers.dump b/libraries/test_data/src/test/assets/playbackdumps/prewarmingRenderer/multipleVideoImagePlaylist-withSecondaryRenderers.dump new file mode 100644 index 00000000000..8be873cf4ff --- /dev/null +++ b/libraries/test_data/src/test/assets/playbackdumps/prewarmingRenderer/multipleVideoImagePlaylist-withSecondaryRenderers.dump @@ -0,0 +1,2052 @@ +MediaCodecAdapter (media3.audio.aac): + inputBuffers: + count = 92 + input buffer #0: + timeUs = 1000000044000 + contents = length 23, hash 47DE9131 + input buffer #1: + timeUs = 1000000067219 + contents = length 6, hash 31EC5206 + input buffer #2: + timeUs = 1000000090439 + contents = length 148, hash 894A176B + input buffer #3: + timeUs = 1000000113659 + contents = length 189, hash CEF235A1 + input buffer #4: + timeUs = 1000000136879 + contents = length 205, hash BBF5F7B0 + input buffer #5: + timeUs = 1000000160099 + contents = length 210, hash F278B193 + input buffer #6: + timeUs = 1000000183319 + contents = length 210, hash 82DA1589 + input buffer #7: + timeUs = 1000000206539 + contents = length 207, hash 5BE231DF + input buffer #8: + timeUs = 1000000229759 + contents = length 225, hash 18819EE1 + input buffer #9: + timeUs = 1000000252979 + contents = length 215, hash CA7FA67B + input buffer #10: + timeUs = 1000000276199 + contents = length 211, hash 581A1C18 + input buffer #11: + timeUs = 1000000299419 + contents = length 216, hash ADB88187 + input buffer #12: + timeUs = 1000000322639 + contents = length 229, hash 2E8BA4DC + input buffer #13: + timeUs = 1000000345859 + contents = length 232, hash 22F0C510 + input buffer #14: + timeUs = 1000000369079 + contents = length 235, hash 867AD0DC + input buffer #15: + timeUs = 1000000392299 + contents = length 231, hash 84E823A8 + input buffer #16: + timeUs = 1000000415519 + contents = length 226, hash 1BEF3A95 + input buffer #17: + timeUs = 1000000438739 + contents = length 216, hash EAA345AE + input buffer #18: + timeUs = 1000000461959 + contents = length 229, hash 6957411F + input buffer #19: + timeUs = 1000000485179 + contents = length 219, hash 41275022 + input buffer #20: + timeUs = 1000000508399 + contents = length 241, hash 6495DF96 + input buffer #21: + timeUs = 1000000531619 + contents = length 228, hash 63D95906 + input buffer #22: + timeUs = 1000000554839 + contents = length 238, hash 34F676F9 + input buffer #23: + timeUs = 1000000578058 + contents = length 234, hash E5CBC045 + input buffer #24: + timeUs = 1000000601278 + contents = length 231, hash 5FC43661 + input buffer #25: + timeUs = 1000000624498 + contents = length 217, hash 682708ED + input buffer #26: + timeUs = 1000000647718 + contents = length 239, hash D43780FC + input buffer #27: + timeUs = 1000000670938 + contents = length 243, hash C5E17980 + input buffer #28: + timeUs = 1000000694158 + contents = length 231, hash AC5837BA + input buffer #29: + timeUs = 1000000717378 + contents = length 230, hash 169EE895 + input buffer #30: + timeUs = 1000000740598 + contents = length 238, hash C48FF3F1 + input buffer #31: + timeUs = 1000000763818 + contents = length 225, hash 531E4599 + input buffer #32: + timeUs = 1000000787038 + contents = length 232, hash CB3C6B8D + input buffer #33: + timeUs = 1000000810258 + contents = length 243, hash F8C94C7 + input buffer #34: + timeUs = 1000000833478 + contents = length 232, hash A646A7D0 + input buffer #35: + timeUs = 1000000856698 + contents = length 237, hash E8B787A5 + input buffer #36: + timeUs = 1000000879918 + contents = length 228, hash 3FA7A29F + input buffer #37: + timeUs = 1000000903138 + contents = length 235, hash B9B33B0A + input buffer #38: + timeUs = 1000000926358 + contents = length 264, hash 71A4869E + input buffer #39: + timeUs = 1000000949578 + contents = length 257, hash D049B54C + input buffer #40: + timeUs = 1000000972798 + contents = length 227, hash 66757231 + input buffer #41: + timeUs = 1000000996018 + contents = length 227, hash BD374F1B + input buffer #42: + timeUs = 1000001019238 + contents = length 235, hash 999477F6 + input buffer #43: + timeUs = 1000001042458 + contents = length 229, hash FFF98DF0 + input buffer #44: + timeUs = 1000001065678 + contents = length 6, hash 31B22286 + input buffer #45: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + input buffer #46: + timeUs = 1000001068000 + contents = length 23, hash 47DE9131 + input buffer #47: + timeUs = 1000001091219 + contents = length 6, hash 31EC5206 + input buffer #48: + timeUs = 1000001114439 + contents = length 148, hash 894A176B + input buffer #49: + timeUs = 1000001137659 + contents = length 189, hash CEF235A1 + input buffer #50: + timeUs = 1000001160879 + contents = length 205, hash BBF5F7B0 + input buffer #51: + timeUs = 1000001184099 + contents = length 210, hash F278B193 + input buffer #52: + timeUs = 1000001207319 + contents = length 210, hash 82DA1589 + input buffer #53: + timeUs = 1000001230539 + contents = length 207, hash 5BE231DF + input buffer #54: + timeUs = 1000001253759 + contents = length 225, hash 18819EE1 + input buffer #55: + timeUs = 1000001276979 + contents = length 215, hash CA7FA67B + input buffer #56: + timeUs = 1000001300199 + contents = length 211, hash 581A1C18 + input buffer #57: + timeUs = 1000001323419 + contents = length 216, hash ADB88187 + input buffer #58: + timeUs = 1000001346639 + contents = length 229, hash 2E8BA4DC + input buffer #59: + timeUs = 1000001369859 + contents = length 232, hash 22F0C510 + input buffer #60: + timeUs = 1000001393079 + contents = length 235, hash 867AD0DC + input buffer #61: + timeUs = 1000001416299 + contents = length 231, hash 84E823A8 + input buffer #62: + timeUs = 1000001439519 + contents = length 226, hash 1BEF3A95 + input buffer #63: + timeUs = 1000001462739 + contents = length 216, hash EAA345AE + input buffer #64: + timeUs = 1000001485959 + contents = length 229, hash 6957411F + input buffer #65: + timeUs = 1000001509179 + contents = length 219, hash 41275022 + input buffer #66: + timeUs = 1000001532399 + contents = length 241, hash 6495DF96 + input buffer #67: + timeUs = 1000001555619 + contents = length 228, hash 63D95906 + input buffer #68: + timeUs = 1000001578839 + contents = length 238, hash 34F676F9 + input buffer #69: + timeUs = 1000001602058 + contents = length 234, hash E5CBC045 + input buffer #70: + timeUs = 1000001625278 + contents = length 231, hash 5FC43661 + input buffer #71: + timeUs = 1000001648498 + contents = length 217, hash 682708ED + input buffer #72: + timeUs = 1000001671718 + contents = length 239, hash D43780FC + input buffer #73: + timeUs = 1000001694938 + contents = length 243, hash C5E17980 + input buffer #74: + timeUs = 1000001718158 + contents = length 231, hash AC5837BA + input buffer #75: + timeUs = 1000001741378 + contents = length 230, hash 169EE895 + input buffer #76: + timeUs = 1000001764598 + contents = length 238, hash C48FF3F1 + input buffer #77: + timeUs = 1000001787818 + contents = length 225, hash 531E4599 + input buffer #78: + timeUs = 1000001811038 + contents = length 232, hash CB3C6B8D + input buffer #79: + timeUs = 1000001834258 + contents = length 243, hash F8C94C7 + input buffer #80: + timeUs = 1000001857478 + contents = length 232, hash A646A7D0 + input buffer #81: + timeUs = 1000001880698 + contents = length 237, hash E8B787A5 + input buffer #82: + timeUs = 1000001903918 + contents = length 228, hash 3FA7A29F + input buffer #83: + timeUs = 1000001927138 + contents = length 235, hash B9B33B0A + input buffer #84: + timeUs = 1000001950358 + contents = length 264, hash 71A4869E + input buffer #85: + timeUs = 1000001973578 + contents = length 257, hash D049B54C + input buffer #86: + timeUs = 1000001996798 + contents = length 227, hash 66757231 + input buffer #87: + timeUs = 1000002020018 + contents = length 227, hash BD374F1B + input buffer #88: + timeUs = 1000002043238 + contents = length 235, hash 999477F6 + input buffer #89: + timeUs = 1000002066458 + contents = length 229, hash FFF98DF0 + input buffer #90: + timeUs = 1000002089678 + contents = length 6, hash 31B22286 + input buffer #91: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 90 + output buffer #0: + timeUs = 1000000044000 + size = 0 + rendered = false + output buffer #1: + timeUs = 1000000067219 + size = 0 + rendered = false + output buffer #2: + timeUs = 1000000090439 + size = 0 + rendered = false + output buffer #3: + timeUs = 1000000113659 + size = 0 + rendered = false + output buffer #4: + timeUs = 1000000136879 + size = 0 + rendered = false + output buffer #5: + timeUs = 1000000160099 + size = 0 + rendered = false + output buffer #6: + timeUs = 1000000183319 + size = 0 + rendered = false + output buffer #7: + timeUs = 1000000206539 + size = 0 + rendered = false + output buffer #8: + timeUs = 1000000229759 + size = 0 + rendered = false + output buffer #9: + timeUs = 1000000252979 + size = 0 + rendered = false + output buffer #10: + timeUs = 1000000276199 + size = 0 + rendered = false + output buffer #11: + timeUs = 1000000299419 + size = 0 + rendered = false + output buffer #12: + timeUs = 1000000322639 + size = 0 + rendered = false + output buffer #13: + timeUs = 1000000345859 + size = 0 + rendered = false + output buffer #14: + timeUs = 1000000369079 + size = 0 + rendered = false + output buffer #15: + timeUs = 1000000392299 + size = 0 + rendered = false + output buffer #16: + timeUs = 1000000415519 + size = 0 + rendered = false + output buffer #17: + timeUs = 1000000438739 + size = 0 + rendered = false + output buffer #18: + timeUs = 1000000461959 + size = 0 + rendered = false + output buffer #19: + timeUs = 1000000485179 + size = 0 + rendered = false + output buffer #20: + timeUs = 1000000508399 + size = 0 + rendered = false + output buffer #21: + timeUs = 1000000531619 + size = 0 + rendered = false + output buffer #22: + timeUs = 1000000554839 + size = 0 + rendered = false + output buffer #23: + timeUs = 1000000578058 + size = 0 + rendered = false + output buffer #24: + timeUs = 1000000601278 + size = 0 + rendered = false + output buffer #25: + timeUs = 1000000624498 + size = 0 + rendered = false + output buffer #26: + timeUs = 1000000647718 + size = 0 + rendered = false + output buffer #27: + timeUs = 1000000670938 + size = 0 + rendered = false + output buffer #28: + timeUs = 1000000694158 + size = 0 + rendered = false + output buffer #29: + timeUs = 1000000717378 + size = 0 + rendered = false + output buffer #30: + timeUs = 1000000740598 + size = 0 + rendered = false + output buffer #31: + timeUs = 1000000763818 + size = 0 + rendered = false + output buffer #32: + timeUs = 1000000787038 + size = 0 + rendered = false + output buffer #33: + timeUs = 1000000810258 + size = 0 + rendered = false + output buffer #34: + timeUs = 1000000833478 + size = 0 + rendered = false + output buffer #35: + timeUs = 1000000856698 + size = 0 + rendered = false + output buffer #36: + timeUs = 1000000879918 + size = 0 + rendered = false + output buffer #37: + timeUs = 1000000903138 + size = 0 + rendered = false + output buffer #38: + timeUs = 1000000926358 + size = 0 + rendered = false + output buffer #39: + timeUs = 1000000949578 + size = 0 + rendered = false + output buffer #40: + timeUs = 1000000972798 + size = 0 + rendered = false + output buffer #41: + timeUs = 1000000996018 + size = 0 + rendered = false + output buffer #42: + timeUs = 1000001019238 + size = 0 + rendered = false + output buffer #43: + timeUs = 1000001042458 + size = 0 + rendered = false + output buffer #44: + timeUs = 1000001065678 + size = 0 + rendered = false + output buffer #45: + timeUs = 1000001068000 + size = 0 + rendered = false + output buffer #46: + timeUs = 1000001091219 + size = 0 + rendered = false + output buffer #47: + timeUs = 1000001114439 + size = 0 + rendered = false + output buffer #48: + timeUs = 1000001137659 + size = 0 + rendered = false + output buffer #49: + timeUs = 1000001160879 + size = 0 + rendered = false + output buffer #50: + timeUs = 1000001184099 + size = 0 + rendered = false + output buffer #51: + timeUs = 1000001207319 + size = 0 + rendered = false + output buffer #52: + timeUs = 1000001230539 + size = 0 + rendered = false + output buffer #53: + timeUs = 1000001253759 + size = 0 + rendered = false + output buffer #54: + timeUs = 1000001276979 + size = 0 + rendered = false + output buffer #55: + timeUs = 1000001300199 + size = 0 + rendered = false + output buffer #56: + timeUs = 1000001323419 + size = 0 + rendered = false + output buffer #57: + timeUs = 1000001346639 + size = 0 + rendered = false + output buffer #58: + timeUs = 1000001369859 + size = 0 + rendered = false + output buffer #59: + timeUs = 1000001393079 + size = 0 + rendered = false + output buffer #60: + timeUs = 1000001416299 + size = 0 + rendered = false + output buffer #61: + timeUs = 1000001439519 + size = 0 + rendered = false + output buffer #62: + timeUs = 1000001462739 + size = 0 + rendered = false + output buffer #63: + timeUs = 1000001485959 + size = 0 + rendered = false + output buffer #64: + timeUs = 1000001509179 + size = 0 + rendered = false + output buffer #65: + timeUs = 1000001532399 + size = 0 + rendered = false + output buffer #66: + timeUs = 1000001555619 + size = 0 + rendered = false + output buffer #67: + timeUs = 1000001578839 + size = 0 + rendered = false + output buffer #68: + timeUs = 1000001602058 + size = 0 + rendered = false + output buffer #69: + timeUs = 1000001625278 + size = 0 + rendered = false + output buffer #70: + timeUs = 1000001648498 + size = 0 + rendered = false + output buffer #71: + timeUs = 1000001671718 + size = 0 + rendered = false + output buffer #72: + timeUs = 1000001694938 + size = 0 + rendered = false + output buffer #73: + timeUs = 1000001718158 + size = 0 + rendered = false + output buffer #74: + timeUs = 1000001741378 + size = 0 + rendered = false + output buffer #75: + timeUs = 1000001764598 + size = 0 + rendered = false + output buffer #76: + timeUs = 1000001787818 + size = 0 + rendered = false + output buffer #77: + timeUs = 1000001811038 + size = 0 + rendered = false + output buffer #78: + timeUs = 1000001834258 + size = 0 + rendered = false + output buffer #79: + timeUs = 1000001857478 + size = 0 + rendered = false + output buffer #80: + timeUs = 1000001880698 + size = 0 + rendered = false + output buffer #81: + timeUs = 1000001903918 + size = 0 + rendered = false + output buffer #82: + timeUs = 1000001927138 + size = 0 + rendered = false + output buffer #83: + timeUs = 1000001950358 + size = 0 + rendered = false + output buffer #84: + timeUs = 1000001973578 + size = 0 + rendered = false + output buffer #85: + timeUs = 1000001996798 + size = 0 + rendered = false + output buffer #86: + timeUs = 1000002020018 + size = 0 + rendered = false + output buffer #87: + timeUs = 1000002043238 + size = 0 + rendered = false + output buffer #88: + timeUs = 1000002066458 + size = 0 + rendered = false + output buffer #89: + timeUs = 1000002089678 + size = 0 + rendered = false +MediaCodecAdapter (media3.audio.aac): + inputBuffers: + count = 46 + input buffer #0: + timeUs = 1000004092000 + contents = length 23, hash 47DE9131 + input buffer #1: + timeUs = 1000004115219 + contents = length 6, hash 31EC5206 + input buffer #2: + timeUs = 1000004138439 + contents = length 148, hash 894A176B + input buffer #3: + timeUs = 1000004161659 + contents = length 189, hash CEF235A1 + input buffer #4: + timeUs = 1000004184879 + contents = length 205, hash BBF5F7B0 + input buffer #5: + timeUs = 1000004208099 + contents = length 210, hash F278B193 + input buffer #6: + timeUs = 1000004231319 + contents = length 210, hash 82DA1589 + input buffer #7: + timeUs = 1000004254539 + contents = length 207, hash 5BE231DF + input buffer #8: + timeUs = 1000004277759 + contents = length 225, hash 18819EE1 + input buffer #9: + timeUs = 1000004300979 + contents = length 215, hash CA7FA67B + input buffer #10: + timeUs = 1000004324199 + contents = length 211, hash 581A1C18 + input buffer #11: + timeUs = 1000004347419 + contents = length 216, hash ADB88187 + input buffer #12: + timeUs = 1000004370639 + contents = length 229, hash 2E8BA4DC + input buffer #13: + timeUs = 1000004393859 + contents = length 232, hash 22F0C510 + input buffer #14: + timeUs = 1000004417079 + contents = length 235, hash 867AD0DC + input buffer #15: + timeUs = 1000004440299 + contents = length 231, hash 84E823A8 + input buffer #16: + timeUs = 1000004463519 + contents = length 226, hash 1BEF3A95 + input buffer #17: + timeUs = 1000004486739 + contents = length 216, hash EAA345AE + input buffer #18: + timeUs = 1000004509959 + contents = length 229, hash 6957411F + input buffer #19: + timeUs = 1000004533179 + contents = length 219, hash 41275022 + input buffer #20: + timeUs = 1000004556399 + contents = length 241, hash 6495DF96 + input buffer #21: + timeUs = 1000004579619 + contents = length 228, hash 63D95906 + input buffer #22: + timeUs = 1000004602839 + contents = length 238, hash 34F676F9 + input buffer #23: + timeUs = 1000004626058 + contents = length 234, hash E5CBC045 + input buffer #24: + timeUs = 1000004649278 + contents = length 231, hash 5FC43661 + input buffer #25: + timeUs = 1000004672498 + contents = length 217, hash 682708ED + input buffer #26: + timeUs = 1000004695718 + contents = length 239, hash D43780FC + input buffer #27: + timeUs = 1000004718938 + contents = length 243, hash C5E17980 + input buffer #28: + timeUs = 1000004742158 + contents = length 231, hash AC5837BA + input buffer #29: + timeUs = 1000004765378 + contents = length 230, hash 169EE895 + input buffer #30: + timeUs = 1000004788598 + contents = length 238, hash C48FF3F1 + input buffer #31: + timeUs = 1000004811818 + contents = length 225, hash 531E4599 + input buffer #32: + timeUs = 1000004835038 + contents = length 232, hash CB3C6B8D + input buffer #33: + timeUs = 1000004858258 + contents = length 243, hash F8C94C7 + input buffer #34: + timeUs = 1000004881478 + contents = length 232, hash A646A7D0 + input buffer #35: + timeUs = 1000004904698 + contents = length 237, hash E8B787A5 + input buffer #36: + timeUs = 1000004927918 + contents = length 228, hash 3FA7A29F + input buffer #37: + timeUs = 1000004951138 + contents = length 235, hash B9B33B0A + input buffer #38: + timeUs = 1000004974358 + contents = length 264, hash 71A4869E + input buffer #39: + timeUs = 1000004997578 + contents = length 257, hash D049B54C + input buffer #40: + timeUs = 1000005020798 + contents = length 227, hash 66757231 + input buffer #41: + timeUs = 1000005044018 + contents = length 227, hash BD374F1B + input buffer #42: + timeUs = 1000005067238 + contents = length 235, hash 999477F6 + input buffer #43: + timeUs = 1000005090458 + contents = length 229, hash FFF98DF0 + input buffer #44: + timeUs = 1000005113678 + contents = length 6, hash 31B22286 + input buffer #45: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 45 + output buffer #0: + timeUs = 1000004092000 + size = 0 + rendered = false + output buffer #1: + timeUs = 1000004115219 + size = 0 + rendered = false + output buffer #2: + timeUs = 1000004138439 + size = 0 + rendered = false + output buffer #3: + timeUs = 1000004161659 + size = 0 + rendered = false + output buffer #4: + timeUs = 1000004184879 + size = 0 + rendered = false + output buffer #5: + timeUs = 1000004208099 + size = 0 + rendered = false + output buffer #6: + timeUs = 1000004231319 + size = 0 + rendered = false + output buffer #7: + timeUs = 1000004254539 + size = 0 + rendered = false + output buffer #8: + timeUs = 1000004277759 + size = 0 + rendered = false + output buffer #9: + timeUs = 1000004300979 + size = 0 + rendered = false + output buffer #10: + timeUs = 1000004324199 + size = 0 + rendered = false + output buffer #11: + timeUs = 1000004347419 + size = 0 + rendered = false + output buffer #12: + timeUs = 1000004370639 + size = 0 + rendered = false + output buffer #13: + timeUs = 1000004393859 + size = 0 + rendered = false + output buffer #14: + timeUs = 1000004417079 + size = 0 + rendered = false + output buffer #15: + timeUs = 1000004440299 + size = 0 + rendered = false + output buffer #16: + timeUs = 1000004463519 + size = 0 + rendered = false + output buffer #17: + timeUs = 1000004486739 + size = 0 + rendered = false + output buffer #18: + timeUs = 1000004509959 + size = 0 + rendered = false + output buffer #19: + timeUs = 1000004533179 + size = 0 + rendered = false + output buffer #20: + timeUs = 1000004556399 + size = 0 + rendered = false + output buffer #21: + timeUs = 1000004579619 + size = 0 + rendered = false + output buffer #22: + timeUs = 1000004602839 + size = 0 + rendered = false + output buffer #23: + timeUs = 1000004626058 + size = 0 + rendered = false + output buffer #24: + timeUs = 1000004649278 + size = 0 + rendered = false + output buffer #25: + timeUs = 1000004672498 + size = 0 + rendered = false + output buffer #26: + timeUs = 1000004695718 + size = 0 + rendered = false + output buffer #27: + timeUs = 1000004718938 + size = 0 + rendered = false + output buffer #28: + timeUs = 1000004742158 + size = 0 + rendered = false + output buffer #29: + timeUs = 1000004765378 + size = 0 + rendered = false + output buffer #30: + timeUs = 1000004788598 + size = 0 + rendered = false + output buffer #31: + timeUs = 1000004811818 + size = 0 + rendered = false + output buffer #32: + timeUs = 1000004835038 + size = 0 + rendered = false + output buffer #33: + timeUs = 1000004858258 + size = 0 + rendered = false + output buffer #34: + timeUs = 1000004881478 + size = 0 + rendered = false + output buffer #35: + timeUs = 1000004904698 + size = 0 + rendered = false + output buffer #36: + timeUs = 1000004927918 + size = 0 + rendered = false + output buffer #37: + timeUs = 1000004951138 + size = 0 + rendered = false + output buffer #38: + timeUs = 1000004974358 + size = 0 + rendered = false + output buffer #39: + timeUs = 1000004997578 + size = 0 + rendered = false + output buffer #40: + timeUs = 1000005020798 + size = 0 + rendered = false + output buffer #41: + timeUs = 1000005044018 + size = 0 + rendered = false + output buffer #42: + timeUs = 1000005067238 + size = 0 + rendered = false + output buffer #43: + timeUs = 1000005090458 + size = 0 + rendered = false + output buffer #44: + timeUs = 1000005113678 + size = 0 + rendered = false +MediaCodecAdapter (media3.video.avc): + inputBuffers: + count = 31 + input buffer #0: + timeUs = 1000000000000 + contents = length 36692, hash D216076E + input buffer #1: + timeUs = 1000000066733 + contents = length 5312, hash D45D3CA0 + input buffer #2: + timeUs = 1000000033366 + contents = length 599, hash 1BE7812D + input buffer #3: + timeUs = 1000000200200 + contents = length 7735, hash 4490F110 + input buffer #4: + timeUs = 1000000133466 + contents = length 987, hash 560B5036 + input buffer #5: + timeUs = 1000000100100 + contents = length 673, hash ED7CD8C7 + input buffer #6: + timeUs = 1000000166833 + contents = length 523, hash 3020DF50 + input buffer #7: + timeUs = 1000000333666 + contents = length 6061, hash 736C72B2 + input buffer #8: + timeUs = 1000000266933 + contents = length 992, hash FE132F23 + input buffer #9: + timeUs = 1000000233566 + contents = length 623, hash 5B2C1816 + input buffer #10: + timeUs = 1000000300300 + contents = length 421, hash 742E69C1 + input buffer #11: + timeUs = 1000000433766 + contents = length 4899, hash F72F86A1 + input buffer #12: + timeUs = 1000000400400 + contents = length 568, hash 519A8E50 + input buffer #13: + timeUs = 1000000367033 + contents = length 620, hash 3990AA39 + input buffer #14: + timeUs = 1000000567233 + contents = length 5450, hash F06EC4AA + input buffer #15: + timeUs = 1000000500500 + contents = length 1051, hash 92DFA63A + input buffer #16: + timeUs = 1000000467133 + contents = length 874, hash 69587FB4 + input buffer #17: + timeUs = 1000000533866 + contents = length 781, hash 36BE495B + input buffer #18: + timeUs = 1000000700700 + contents = length 4725, hash AC0C8CD3 + input buffer #19: + timeUs = 1000000633966 + contents = length 1022, hash 5D8BFF34 + input buffer #20: + timeUs = 1000000600600 + contents = length 790, hash 99413A99 + input buffer #21: + timeUs = 1000000667333 + contents = length 610, hash 5E129290 + input buffer #22: + timeUs = 1000000834166 + contents = length 2751, hash 769974CB + input buffer #23: + timeUs = 1000000767433 + contents = length 745, hash B78A477A + input buffer #24: + timeUs = 1000000734066 + contents = length 621, hash CF741E7A + input buffer #25: + timeUs = 1000000800800 + contents = length 505, hash 1DB4894E + input buffer #26: + timeUs = 1000000967633 + contents = length 1268, hash C15348DC + input buffer #27: + timeUs = 1000000900900 + contents = length 880, hash C2DE85D0 + input buffer #28: + timeUs = 1000000867533 + contents = length 530, hash C98BC6A8 + input buffer #29: + timeUs = 1000000934266 + contents = length 568, hash 4FE5C8EA + input buffer #30: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 30 + output buffer #0: + timeUs = 1000000000000 + size = 36692 + rendered = true + output buffer #1: + timeUs = 1000000066733 + size = 5312 + rendered = true + output buffer #2: + timeUs = 1000000033366 + size = 599 + rendered = true + output buffer #3: + timeUs = 1000000200200 + size = 7735 + rendered = true + output buffer #4: + timeUs = 1000000133466 + size = 987 + rendered = true + output buffer #5: + timeUs = 1000000100100 + size = 673 + rendered = true + output buffer #6: + timeUs = 1000000166833 + size = 523 + rendered = true + output buffer #7: + timeUs = 1000000333666 + size = 6061 + rendered = true + output buffer #8: + timeUs = 1000000266933 + size = 992 + rendered = true + output buffer #9: + timeUs = 1000000233566 + size = 623 + rendered = true + output buffer #10: + timeUs = 1000000300300 + size = 421 + rendered = true + output buffer #11: + timeUs = 1000000433766 + size = 4899 + rendered = true + output buffer #12: + timeUs = 1000000400400 + size = 568 + rendered = true + output buffer #13: + timeUs = 1000000367033 + size = 620 + rendered = true + output buffer #14: + timeUs = 1000000567233 + size = 5450 + rendered = true + output buffer #15: + timeUs = 1000000500500 + size = 1051 + rendered = true + output buffer #16: + timeUs = 1000000467133 + size = 874 + rendered = true + output buffer #17: + timeUs = 1000000533866 + size = 781 + rendered = true + output buffer #18: + timeUs = 1000000700700 + size = 4725 + rendered = true + output buffer #19: + timeUs = 1000000633966 + size = 1022 + rendered = true + output buffer #20: + timeUs = 1000000600600 + size = 790 + rendered = true + output buffer #21: + timeUs = 1000000667333 + size = 610 + rendered = true + output buffer #22: + timeUs = 1000000834166 + size = 2751 + rendered = true + output buffer #23: + timeUs = 1000000767433 + size = 745 + rendered = true + output buffer #24: + timeUs = 1000000734066 + size = 621 + rendered = true + output buffer #25: + timeUs = 1000000800800 + size = 505 + rendered = true + output buffer #26: + timeUs = 1000000967633 + size = 1268 + rendered = true + output buffer #27: + timeUs = 1000000900900 + size = 880 + rendered = true + output buffer #28: + timeUs = 1000000867533 + size = 530 + rendered = true + output buffer #29: + timeUs = 1000000934266 + size = 568 + rendered = true +MediaCodecAdapter (media3.video.avc): + inputBuffers: + count = 31 + input buffer #0: + timeUs = 1000001024000 + contents = length 36692, hash D216076E + input buffer #1: + timeUs = 1000001090733 + contents = length 5312, hash D45D3CA0 + input buffer #2: + timeUs = 1000001057366 + contents = length 599, hash 1BE7812D + input buffer #3: + timeUs = 1000001224200 + contents = length 7735, hash 4490F110 + input buffer #4: + timeUs = 1000001157466 + contents = length 987, hash 560B5036 + input buffer #5: + timeUs = 1000001124100 + contents = length 673, hash ED7CD8C7 + input buffer #6: + timeUs = 1000001190833 + contents = length 523, hash 3020DF50 + input buffer #7: + timeUs = 1000001357666 + contents = length 6061, hash 736C72B2 + input buffer #8: + timeUs = 1000001290933 + contents = length 992, hash FE132F23 + input buffer #9: + timeUs = 1000001257566 + contents = length 623, hash 5B2C1816 + input buffer #10: + timeUs = 1000001324300 + contents = length 421, hash 742E69C1 + input buffer #11: + timeUs = 1000001457766 + contents = length 4899, hash F72F86A1 + input buffer #12: + timeUs = 1000001424400 + contents = length 568, hash 519A8E50 + input buffer #13: + timeUs = 1000001391033 + contents = length 620, hash 3990AA39 + input buffer #14: + timeUs = 1000001591233 + contents = length 5450, hash F06EC4AA + input buffer #15: + timeUs = 1000001524500 + contents = length 1051, hash 92DFA63A + input buffer #16: + timeUs = 1000001491133 + contents = length 874, hash 69587FB4 + input buffer #17: + timeUs = 1000001557866 + contents = length 781, hash 36BE495B + input buffer #18: + timeUs = 1000001724700 + contents = length 4725, hash AC0C8CD3 + input buffer #19: + timeUs = 1000001657966 + contents = length 1022, hash 5D8BFF34 + input buffer #20: + timeUs = 1000001624600 + contents = length 790, hash 99413A99 + input buffer #21: + timeUs = 1000001691333 + contents = length 610, hash 5E129290 + input buffer #22: + timeUs = 1000001858166 + contents = length 2751, hash 769974CB + input buffer #23: + timeUs = 1000001791433 + contents = length 745, hash B78A477A + input buffer #24: + timeUs = 1000001758066 + contents = length 621, hash CF741E7A + input buffer #25: + timeUs = 1000001824800 + contents = length 505, hash 1DB4894E + input buffer #26: + timeUs = 1000001991633 + contents = length 1268, hash C15348DC + input buffer #27: + timeUs = 1000001924900 + contents = length 880, hash C2DE85D0 + input buffer #28: + timeUs = 1000001891533 + contents = length 530, hash C98BC6A8 + input buffer #29: + timeUs = 1000001958266 + contents = length 568, hash 4FE5C8EA + input buffer #30: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 30 + output buffer #0: + timeUs = 1000001024000 + size = 36692 + rendered = true + output buffer #1: + timeUs = 1000001090733 + size = 5312 + rendered = true + output buffer #2: + timeUs = 1000001057366 + size = 599 + rendered = true + output buffer #3: + timeUs = 1000001224200 + size = 7735 + rendered = true + output buffer #4: + timeUs = 1000001157466 + size = 987 + rendered = true + output buffer #5: + timeUs = 1000001124100 + size = 673 + rendered = true + output buffer #6: + timeUs = 1000001190833 + size = 523 + rendered = true + output buffer #7: + timeUs = 1000001357666 + size = 6061 + rendered = true + output buffer #8: + timeUs = 1000001290933 + size = 992 + rendered = true + output buffer #9: + timeUs = 1000001257566 + size = 623 + rendered = true + output buffer #10: + timeUs = 1000001324300 + size = 421 + rendered = true + output buffer #11: + timeUs = 1000001457766 + size = 4899 + rendered = true + output buffer #12: + timeUs = 1000001424400 + size = 568 + rendered = true + output buffer #13: + timeUs = 1000001391033 + size = 620 + rendered = true + output buffer #14: + timeUs = 1000001591233 + size = 5450 + rendered = true + output buffer #15: + timeUs = 1000001524500 + size = 1051 + rendered = true + output buffer #16: + timeUs = 1000001491133 + size = 874 + rendered = true + output buffer #17: + timeUs = 1000001557866 + size = 781 + rendered = true + output buffer #18: + timeUs = 1000001724700 + size = 4725 + rendered = true + output buffer #19: + timeUs = 1000001657966 + size = 1022 + rendered = true + output buffer #20: + timeUs = 1000001624600 + size = 790 + rendered = true + output buffer #21: + timeUs = 1000001691333 + size = 610 + rendered = true + output buffer #22: + timeUs = 1000001858166 + size = 2751 + rendered = true + output buffer #23: + timeUs = 1000001791433 + size = 745 + rendered = true + output buffer #24: + timeUs = 1000001758066 + size = 621 + rendered = true + output buffer #25: + timeUs = 1000001824800 + size = 505 + rendered = true + output buffer #26: + timeUs = 1000001991633 + size = 1268 + rendered = true + output buffer #27: + timeUs = 1000001924900 + size = 880 + rendered = true + output buffer #28: + timeUs = 1000001891533 + size = 530 + rendered = true + output buffer #29: + timeUs = 1000001958266 + size = 568 + rendered = true +MediaCodecAdapter (media3.video.avc): + inputBuffers: + count = 31 + input buffer #0: + timeUs = 1000004048000 + contents = length 36692, hash D216076E + input buffer #1: + timeUs = 1000004114733 + contents = length 5312, hash D45D3CA0 + input buffer #2: + timeUs = 1000004081366 + contents = length 599, hash 1BE7812D + input buffer #3: + timeUs = 1000004248200 + contents = length 7735, hash 4490F110 + input buffer #4: + timeUs = 1000004181466 + contents = length 987, hash 560B5036 + input buffer #5: + timeUs = 1000004148100 + contents = length 673, hash ED7CD8C7 + input buffer #6: + timeUs = 1000004214833 + contents = length 523, hash 3020DF50 + input buffer #7: + timeUs = 1000004381666 + contents = length 6061, hash 736C72B2 + input buffer #8: + timeUs = 1000004314933 + contents = length 992, hash FE132F23 + input buffer #9: + timeUs = 1000004281566 + contents = length 623, hash 5B2C1816 + input buffer #10: + timeUs = 1000004348300 + contents = length 421, hash 742E69C1 + input buffer #11: + timeUs = 1000004481766 + contents = length 4899, hash F72F86A1 + input buffer #12: + timeUs = 1000004448400 + contents = length 568, hash 519A8E50 + input buffer #13: + timeUs = 1000004415033 + contents = length 620, hash 3990AA39 + input buffer #14: + timeUs = 1000004615233 + contents = length 5450, hash F06EC4AA + input buffer #15: + timeUs = 1000004548500 + contents = length 1051, hash 92DFA63A + input buffer #16: + timeUs = 1000004515133 + contents = length 874, hash 69587FB4 + input buffer #17: + timeUs = 1000004581866 + contents = length 781, hash 36BE495B + input buffer #18: + timeUs = 1000004748700 + contents = length 4725, hash AC0C8CD3 + input buffer #19: + timeUs = 1000004681966 + contents = length 1022, hash 5D8BFF34 + input buffer #20: + timeUs = 1000004648600 + contents = length 790, hash 99413A99 + input buffer #21: + timeUs = 1000004715333 + contents = length 610, hash 5E129290 + input buffer #22: + timeUs = 1000004882166 + contents = length 2751, hash 769974CB + input buffer #23: + timeUs = 1000004815433 + contents = length 745, hash B78A477A + input buffer #24: + timeUs = 1000004782066 + contents = length 621, hash CF741E7A + input buffer #25: + timeUs = 1000004848800 + contents = length 505, hash 1DB4894E + input buffer #26: + timeUs = 1000005015633 + contents = length 1268, hash C15348DC + input buffer #27: + timeUs = 1000004948900 + contents = length 880, hash C2DE85D0 + input buffer #28: + timeUs = 1000004915533 + contents = length 530, hash C98BC6A8 + input buffer #29: + timeUs = 1000004982266 + contents = length 568, hash 4FE5C8EA + input buffer #30: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 30 + output buffer #0: + timeUs = 1000004048000 + size = 36692 + rendered = true + output buffer #1: + timeUs = 1000004114733 + size = 5312 + rendered = true + output buffer #2: + timeUs = 1000004081366 + size = 599 + rendered = true + output buffer #3: + timeUs = 1000004248200 + size = 7735 + rendered = true + output buffer #4: + timeUs = 1000004181466 + size = 987 + rendered = true + output buffer #5: + timeUs = 1000004148100 + size = 673 + rendered = true + output buffer #6: + timeUs = 1000004214833 + size = 523 + rendered = true + output buffer #7: + timeUs = 1000004381666 + size = 6061 + rendered = true + output buffer #8: + timeUs = 1000004314933 + size = 992 + rendered = true + output buffer #9: + timeUs = 1000004281566 + size = 623 + rendered = true + output buffer #10: + timeUs = 1000004348300 + size = 421 + rendered = true + output buffer #11: + timeUs = 1000004481766 + size = 4899 + rendered = true + output buffer #12: + timeUs = 1000004448400 + size = 568 + rendered = true + output buffer #13: + timeUs = 1000004415033 + size = 620 + rendered = true + output buffer #14: + timeUs = 1000004615233 + size = 5450 + rendered = true + output buffer #15: + timeUs = 1000004548500 + size = 1051 + rendered = true + output buffer #16: + timeUs = 1000004515133 + size = 874 + rendered = true + output buffer #17: + timeUs = 1000004581866 + size = 781 + rendered = true + output buffer #18: + timeUs = 1000004748700 + size = 4725 + rendered = true + output buffer #19: + timeUs = 1000004681966 + size = 1022 + rendered = true + output buffer #20: + timeUs = 1000004648600 + size = 790 + rendered = true + output buffer #21: + timeUs = 1000004715333 + size = 610 + rendered = true + output buffer #22: + timeUs = 1000004882166 + size = 2751 + rendered = true + output buffer #23: + timeUs = 1000004815433 + size = 745 + rendered = true + output buffer #24: + timeUs = 1000004782066 + size = 621 + rendered = true + output buffer #25: + timeUs = 1000004848800 + size = 505 + rendered = true + output buffer #26: + timeUs = 1000005015633 + size = 1268 + rendered = true + output buffer #27: + timeUs = 1000004948900 + size = 880 + rendered = true + output buffer #28: + timeUs = 1000004915533 + size = 530 + rendered = true + output buffer #29: + timeUs = 1000004982266 + size = 568 + rendered = true +AudioSink: + buffer count = 135 + config: + pcmEncoding = pcm-16 + channelCount = 1 + sampleRate = 44100 + buffer #0: + time = 1000000044000 + data = empty + buffer #1: + time = 1000000067219 + data = empty + buffer #2: + time = 1000000090439 + data = empty + buffer #3: + time = 1000000113659 + data = empty + buffer #4: + time = 1000000136879 + data = empty + buffer #5: + time = 1000000160099 + data = empty + buffer #6: + time = 1000000183319 + data = empty + buffer #7: + time = 1000000206539 + data = empty + buffer #8: + time = 1000000229759 + data = empty + buffer #9: + time = 1000000252979 + data = empty + buffer #10: + time = 1000000276199 + data = empty + buffer #11: + time = 1000000299419 + data = empty + buffer #12: + time = 1000000322639 + data = empty + buffer #13: + time = 1000000345859 + data = empty + buffer #14: + time = 1000000369079 + data = empty + buffer #15: + time = 1000000392299 + data = empty + buffer #16: + time = 1000000415519 + data = empty + buffer #17: + time = 1000000438739 + data = empty + buffer #18: + time = 1000000461959 + data = empty + buffer #19: + time = 1000000485179 + data = empty + buffer #20: + time = 1000000508399 + data = empty + buffer #21: + time = 1000000531619 + data = empty + buffer #22: + time = 1000000554839 + data = empty + buffer #23: + time = 1000000578058 + data = empty + buffer #24: + time = 1000000601278 + data = empty + buffer #25: + time = 1000000624498 + data = empty + buffer #26: + time = 1000000647718 + data = empty + buffer #27: + time = 1000000670938 + data = empty + buffer #28: + time = 1000000694158 + data = empty + buffer #29: + time = 1000000717378 + data = empty + buffer #30: + time = 1000000740598 + data = empty + buffer #31: + time = 1000000763818 + data = empty + buffer #32: + time = 1000000787038 + data = empty + buffer #33: + time = 1000000810258 + data = empty + buffer #34: + time = 1000000833478 + data = empty + buffer #35: + time = 1000000856698 + data = empty + buffer #36: + time = 1000000879918 + data = empty + buffer #37: + time = 1000000903138 + data = empty + buffer #38: + time = 1000000926358 + data = empty + buffer #39: + time = 1000000949578 + data = empty + buffer #40: + time = 1000000972798 + data = empty + buffer #41: + time = 1000000996018 + data = empty + buffer #42: + time = 1000001019238 + data = empty + buffer #43: + time = 1000001042458 + data = empty + buffer #44: + time = 1000001065678 + data = empty + discontinuity: + config: + pcmEncoding = pcm-16 + channelCount = 1 + sampleRate = 44100 + buffer #45: + time = 1000001068000 + data = empty + buffer #46: + time = 1000001091219 + data = empty + buffer #47: + time = 1000001114439 + data = empty + buffer #48: + time = 1000001137659 + data = empty + buffer #49: + time = 1000001160879 + data = empty + buffer #50: + time = 1000001184099 + data = empty + buffer #51: + time = 1000001207319 + data = empty + buffer #52: + time = 1000001230539 + data = empty + buffer #53: + time = 1000001253759 + data = empty + buffer #54: + time = 1000001276979 + data = empty + buffer #55: + time = 1000001300199 + data = empty + buffer #56: + time = 1000001323419 + data = empty + buffer #57: + time = 1000001346639 + data = empty + buffer #58: + time = 1000001369859 + data = empty + buffer #59: + time = 1000001393079 + data = empty + buffer #60: + time = 1000001416299 + data = empty + buffer #61: + time = 1000001439519 + data = empty + buffer #62: + time = 1000001462739 + data = empty + buffer #63: + time = 1000001485959 + data = empty + buffer #64: + time = 1000001509179 + data = empty + buffer #65: + time = 1000001532399 + data = empty + buffer #66: + time = 1000001555619 + data = empty + buffer #67: + time = 1000001578839 + data = empty + buffer #68: + time = 1000001602058 + data = empty + buffer #69: + time = 1000001625278 + data = empty + buffer #70: + time = 1000001648498 + data = empty + buffer #71: + time = 1000001671718 + data = empty + buffer #72: + time = 1000001694938 + data = empty + buffer #73: + time = 1000001718158 + data = empty + buffer #74: + time = 1000001741378 + data = empty + buffer #75: + time = 1000001764598 + data = empty + buffer #76: + time = 1000001787818 + data = empty + buffer #77: + time = 1000001811038 + data = empty + buffer #78: + time = 1000001834258 + data = empty + buffer #79: + time = 1000001857478 + data = empty + buffer #80: + time = 1000001880698 + data = empty + buffer #81: + time = 1000001903918 + data = empty + buffer #82: + time = 1000001927138 + data = empty + buffer #83: + time = 1000001950358 + data = empty + buffer #84: + time = 1000001973578 + data = empty + buffer #85: + time = 1000001996798 + data = empty + buffer #86: + time = 1000002020018 + data = empty + buffer #87: + time = 1000002043238 + data = empty + buffer #88: + time = 1000002066458 + data = empty + buffer #89: + time = 1000002089678 + data = empty + config: + pcmEncoding = pcm-16 + channelCount = 1 + sampleRate = 44100 + buffer #90: + time = 1000004092000 + data = empty + buffer #91: + time = 1000004115219 + data = empty + buffer #92: + time = 1000004138439 + data = empty + buffer #93: + time = 1000004161659 + data = empty + buffer #94: + time = 1000004184879 + data = empty + buffer #95: + time = 1000004208099 + data = empty + buffer #96: + time = 1000004231319 + data = empty + buffer #97: + time = 1000004254539 + data = empty + buffer #98: + time = 1000004277759 + data = empty + buffer #99: + time = 1000004300979 + data = empty + buffer #100: + time = 1000004324199 + data = empty + buffer #101: + time = 1000004347419 + data = empty + buffer #102: + time = 1000004370639 + data = empty + buffer #103: + time = 1000004393859 + data = empty + buffer #104: + time = 1000004417079 + data = empty + buffer #105: + time = 1000004440299 + data = empty + buffer #106: + time = 1000004463519 + data = empty + buffer #107: + time = 1000004486739 + data = empty + buffer #108: + time = 1000004509959 + data = empty + buffer #109: + time = 1000004533179 + data = empty + buffer #110: + time = 1000004556399 + data = empty + buffer #111: + time = 1000004579619 + data = empty + buffer #112: + time = 1000004602839 + data = empty + buffer #113: + time = 1000004626058 + data = empty + buffer #114: + time = 1000004649278 + data = empty + buffer #115: + time = 1000004672498 + data = empty + buffer #116: + time = 1000004695718 + data = empty + buffer #117: + time = 1000004718938 + data = empty + buffer #118: + time = 1000004742158 + data = empty + buffer #119: + time = 1000004765378 + data = empty + buffer #120: + time = 1000004788598 + data = empty + buffer #121: + time = 1000004811818 + data = empty + buffer #122: + time = 1000004835038 + data = empty + buffer #123: + time = 1000004858258 + data = empty + buffer #124: + time = 1000004881478 + data = empty + buffer #125: + time = 1000004904698 + data = empty + buffer #126: + time = 1000004927918 + data = empty + buffer #127: + time = 1000004951138 + data = empty + buffer #128: + time = 1000004974358 + data = empty + buffer #129: + time = 1000004997578 + data = empty + buffer #130: + time = 1000005020798 + data = empty + buffer #131: + time = 1000005044018 + data = empty + buffer #132: + time = 1000005067238 + data = empty + buffer #133: + time = 1000005090458 + data = empty + buffer #134: + time = 1000005113678 + data = empty +ImageOutput: + rendered image count = 2 + image output #1: + presentationTimeUs = 0 + bitmap hash = -1041353260 + image output #2: + presentationTimeUs = 0 + bitmap hash = -1041353260 diff --git a/libraries/test_data/src/test/assets/playbackdumps/prewarmingRenderer/videoVideoImagePlaylist-seekToStart-withSecondaryRenderers.dump b/libraries/test_data/src/test/assets/playbackdumps/prewarmingRenderer/videoVideoImagePlaylist-seekToStart-withSecondaryRenderers.dump new file mode 100644 index 00000000000..1b22823a2e2 --- /dev/null +++ b/libraries/test_data/src/test/assets/playbackdumps/prewarmingRenderer/videoVideoImagePlaylist-seekToStart-withSecondaryRenderers.dump @@ -0,0 +1,2730 @@ +MediaCodecAdapter (media3.audio.aac): + inputBuffers: + count = 92 + input buffer #0: + timeUs = 1000000044000 + contents = length 23, hash 47DE9131 + input buffer #1: + timeUs = 1000000067219 + contents = length 6, hash 31EC5206 + input buffer #2: + timeUs = 1000000090439 + contents = length 148, hash 894A176B + input buffer #3: + timeUs = 1000000113659 + contents = length 189, hash CEF235A1 + input buffer #4: + timeUs = 1000000136879 + contents = length 205, hash BBF5F7B0 + input buffer #5: + timeUs = 1000000160099 + contents = length 210, hash F278B193 + input buffer #6: + timeUs = 1000000183319 + contents = length 210, hash 82DA1589 + input buffer #7: + timeUs = 1000000206539 + contents = length 207, hash 5BE231DF + input buffer #8: + timeUs = 1000000229759 + contents = length 225, hash 18819EE1 + input buffer #9: + timeUs = 1000000252979 + contents = length 215, hash CA7FA67B + input buffer #10: + timeUs = 1000000276199 + contents = length 211, hash 581A1C18 + input buffer #11: + timeUs = 1000000299419 + contents = length 216, hash ADB88187 + input buffer #12: + timeUs = 1000000322639 + contents = length 229, hash 2E8BA4DC + input buffer #13: + timeUs = 1000000345859 + contents = length 232, hash 22F0C510 + input buffer #14: + timeUs = 1000000369079 + contents = length 235, hash 867AD0DC + input buffer #15: + timeUs = 1000000392299 + contents = length 231, hash 84E823A8 + input buffer #16: + timeUs = 1000000415519 + contents = length 226, hash 1BEF3A95 + input buffer #17: + timeUs = 1000000438739 + contents = length 216, hash EAA345AE + input buffer #18: + timeUs = 1000000461959 + contents = length 229, hash 6957411F + input buffer #19: + timeUs = 1000000485179 + contents = length 219, hash 41275022 + input buffer #20: + timeUs = 1000000508399 + contents = length 241, hash 6495DF96 + input buffer #21: + timeUs = 1000000531619 + contents = length 228, hash 63D95906 + input buffer #22: + timeUs = 1000000554839 + contents = length 238, hash 34F676F9 + input buffer #23: + timeUs = 1000000578058 + contents = length 234, hash E5CBC045 + input buffer #24: + timeUs = 1000000601278 + contents = length 231, hash 5FC43661 + input buffer #25: + timeUs = 1000000624498 + contents = length 217, hash 682708ED + input buffer #26: + timeUs = 1000000647718 + contents = length 239, hash D43780FC + input buffer #27: + timeUs = 1000000670938 + contents = length 243, hash C5E17980 + input buffer #28: + timeUs = 1000000694158 + contents = length 231, hash AC5837BA + input buffer #29: + timeUs = 1000000717378 + contents = length 230, hash 169EE895 + input buffer #30: + timeUs = 1000000740598 + contents = length 238, hash C48FF3F1 + input buffer #31: + timeUs = 1000000763818 + contents = length 225, hash 531E4599 + input buffer #32: + timeUs = 1000000787038 + contents = length 232, hash CB3C6B8D + input buffer #33: + timeUs = 1000000810258 + contents = length 243, hash F8C94C7 + input buffer #34: + timeUs = 1000000833478 + contents = length 232, hash A646A7D0 + input buffer #35: + timeUs = 1000000856698 + contents = length 237, hash E8B787A5 + input buffer #36: + timeUs = 1000000879918 + contents = length 228, hash 3FA7A29F + input buffer #37: + timeUs = 1000000903138 + contents = length 235, hash B9B33B0A + input buffer #38: + timeUs = 1000000926358 + contents = length 264, hash 71A4869E + input buffer #39: + timeUs = 1000000949578 + contents = length 257, hash D049B54C + input buffer #40: + timeUs = 1000000972798 + contents = length 227, hash 66757231 + input buffer #41: + timeUs = 1000000996018 + contents = length 227, hash BD374F1B + input buffer #42: + timeUs = 1000001019238 + contents = length 235, hash 999477F6 + input buffer #43: + timeUs = 1000001042458 + contents = length 229, hash FFF98DF0 + input buffer #44: + timeUs = 1000001065678 + contents = length 6, hash 31B22286 + input buffer #45: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + input buffer #46: + timeUs = 1000001068000 + contents = length 23, hash 47DE9131 + input buffer #47: + timeUs = 1000001091219 + contents = length 6, hash 31EC5206 + input buffer #48: + timeUs = 1000001114439 + contents = length 148, hash 894A176B + input buffer #49: + timeUs = 1000001137659 + contents = length 189, hash CEF235A1 + input buffer #50: + timeUs = 1000001160879 + contents = length 205, hash BBF5F7B0 + input buffer #51: + timeUs = 1000001184099 + contents = length 210, hash F278B193 + input buffer #52: + timeUs = 1000001207319 + contents = length 210, hash 82DA1589 + input buffer #53: + timeUs = 1000001230539 + contents = length 207, hash 5BE231DF + input buffer #54: + timeUs = 1000001253759 + contents = length 225, hash 18819EE1 + input buffer #55: + timeUs = 1000001276979 + contents = length 215, hash CA7FA67B + input buffer #56: + timeUs = 1000001300199 + contents = length 211, hash 581A1C18 + input buffer #57: + timeUs = 1000001323419 + contents = length 216, hash ADB88187 + input buffer #58: + timeUs = 1000001346639 + contents = length 229, hash 2E8BA4DC + input buffer #59: + timeUs = 1000001369859 + contents = length 232, hash 22F0C510 + input buffer #60: + timeUs = 1000001393079 + contents = length 235, hash 867AD0DC + input buffer #61: + timeUs = 1000001416299 + contents = length 231, hash 84E823A8 + input buffer #62: + timeUs = 1000001439519 + contents = length 226, hash 1BEF3A95 + input buffer #63: + timeUs = 1000001462739 + contents = length 216, hash EAA345AE + input buffer #64: + timeUs = 1000001485959 + contents = length 229, hash 6957411F + input buffer #65: + timeUs = 1000001509179 + contents = length 219, hash 41275022 + input buffer #66: + timeUs = 1000001532399 + contents = length 241, hash 6495DF96 + input buffer #67: + timeUs = 1000001555619 + contents = length 228, hash 63D95906 + input buffer #68: + timeUs = 1000001578839 + contents = length 238, hash 34F676F9 + input buffer #69: + timeUs = 1000001602058 + contents = length 234, hash E5CBC045 + input buffer #70: + timeUs = 1000001625278 + contents = length 231, hash 5FC43661 + input buffer #71: + timeUs = 1000001648498 + contents = length 217, hash 682708ED + input buffer #72: + timeUs = 1000001671718 + contents = length 239, hash D43780FC + input buffer #73: + timeUs = 1000001694938 + contents = length 243, hash C5E17980 + input buffer #74: + timeUs = 1000001718158 + contents = length 231, hash AC5837BA + input buffer #75: + timeUs = 1000001741378 + contents = length 230, hash 169EE895 + input buffer #76: + timeUs = 1000001764598 + contents = length 238, hash C48FF3F1 + input buffer #77: + timeUs = 1000001787818 + contents = length 225, hash 531E4599 + input buffer #78: + timeUs = 1000001811038 + contents = length 232, hash CB3C6B8D + input buffer #79: + timeUs = 1000001834258 + contents = length 243, hash F8C94C7 + input buffer #80: + timeUs = 1000001857478 + contents = length 232, hash A646A7D0 + input buffer #81: + timeUs = 1000001880698 + contents = length 237, hash E8B787A5 + input buffer #82: + timeUs = 1000001903918 + contents = length 228, hash 3FA7A29F + input buffer #83: + timeUs = 1000001927138 + contents = length 235, hash B9B33B0A + input buffer #84: + timeUs = 1000001950358 + contents = length 264, hash 71A4869E + input buffer #85: + timeUs = 1000001973578 + contents = length 257, hash D049B54C + input buffer #86: + timeUs = 1000001996798 + contents = length 227, hash 66757231 + input buffer #87: + timeUs = 1000002020018 + contents = length 227, hash BD374F1B + input buffer #88: + timeUs = 1000002043238 + contents = length 235, hash 999477F6 + input buffer #89: + timeUs = 1000002066458 + contents = length 229, hash FFF98DF0 + input buffer #90: + timeUs = 1000002089678 + contents = length 6, hash 31B22286 + input buffer #91: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 90 + output buffer #0: + timeUs = 1000000044000 + size = 0 + rendered = false + output buffer #1: + timeUs = 1000000067219 + size = 0 + rendered = false + output buffer #2: + timeUs = 1000000090439 + size = 0 + rendered = false + output buffer #3: + timeUs = 1000000113659 + size = 0 + rendered = false + output buffer #4: + timeUs = 1000000136879 + size = 0 + rendered = false + output buffer #5: + timeUs = 1000000160099 + size = 0 + rendered = false + output buffer #6: + timeUs = 1000000183319 + size = 0 + rendered = false + output buffer #7: + timeUs = 1000000206539 + size = 0 + rendered = false + output buffer #8: + timeUs = 1000000229759 + size = 0 + rendered = false + output buffer #9: + timeUs = 1000000252979 + size = 0 + rendered = false + output buffer #10: + timeUs = 1000000276199 + size = 0 + rendered = false + output buffer #11: + timeUs = 1000000299419 + size = 0 + rendered = false + output buffer #12: + timeUs = 1000000322639 + size = 0 + rendered = false + output buffer #13: + timeUs = 1000000345859 + size = 0 + rendered = false + output buffer #14: + timeUs = 1000000369079 + size = 0 + rendered = false + output buffer #15: + timeUs = 1000000392299 + size = 0 + rendered = false + output buffer #16: + timeUs = 1000000415519 + size = 0 + rendered = false + output buffer #17: + timeUs = 1000000438739 + size = 0 + rendered = false + output buffer #18: + timeUs = 1000000461959 + size = 0 + rendered = false + output buffer #19: + timeUs = 1000000485179 + size = 0 + rendered = false + output buffer #20: + timeUs = 1000000508399 + size = 0 + rendered = false + output buffer #21: + timeUs = 1000000531619 + size = 0 + rendered = false + output buffer #22: + timeUs = 1000000554839 + size = 0 + rendered = false + output buffer #23: + timeUs = 1000000578058 + size = 0 + rendered = false + output buffer #24: + timeUs = 1000000601278 + size = 0 + rendered = false + output buffer #25: + timeUs = 1000000624498 + size = 0 + rendered = false + output buffer #26: + timeUs = 1000000647718 + size = 0 + rendered = false + output buffer #27: + timeUs = 1000000670938 + size = 0 + rendered = false + output buffer #28: + timeUs = 1000000694158 + size = 0 + rendered = false + output buffer #29: + timeUs = 1000000717378 + size = 0 + rendered = false + output buffer #30: + timeUs = 1000000740598 + size = 0 + rendered = false + output buffer #31: + timeUs = 1000000763818 + size = 0 + rendered = false + output buffer #32: + timeUs = 1000000787038 + size = 0 + rendered = false + output buffer #33: + timeUs = 1000000810258 + size = 0 + rendered = false + output buffer #34: + timeUs = 1000000833478 + size = 0 + rendered = false + output buffer #35: + timeUs = 1000000856698 + size = 0 + rendered = false + output buffer #36: + timeUs = 1000000879918 + size = 0 + rendered = false + output buffer #37: + timeUs = 1000000903138 + size = 0 + rendered = false + output buffer #38: + timeUs = 1000000926358 + size = 0 + rendered = false + output buffer #39: + timeUs = 1000000949578 + size = 0 + rendered = false + output buffer #40: + timeUs = 1000000972798 + size = 0 + rendered = false + output buffer #41: + timeUs = 1000000996018 + size = 0 + rendered = false + output buffer #42: + timeUs = 1000001019238 + size = 0 + rendered = false + output buffer #43: + timeUs = 1000001042458 + size = 0 + rendered = false + output buffer #44: + timeUs = 1000001065678 + size = 0 + rendered = false + output buffer #45: + timeUs = 1000001068000 + size = 0 + rendered = false + output buffer #46: + timeUs = 1000001091219 + size = 0 + rendered = false + output buffer #47: + timeUs = 1000001114439 + size = 0 + rendered = false + output buffer #48: + timeUs = 1000001137659 + size = 0 + rendered = false + output buffer #49: + timeUs = 1000001160879 + size = 0 + rendered = false + output buffer #50: + timeUs = 1000001184099 + size = 0 + rendered = false + output buffer #51: + timeUs = 1000001207319 + size = 0 + rendered = false + output buffer #52: + timeUs = 1000001230539 + size = 0 + rendered = false + output buffer #53: + timeUs = 1000001253759 + size = 0 + rendered = false + output buffer #54: + timeUs = 1000001276979 + size = 0 + rendered = false + output buffer #55: + timeUs = 1000001300199 + size = 0 + rendered = false + output buffer #56: + timeUs = 1000001323419 + size = 0 + rendered = false + output buffer #57: + timeUs = 1000001346639 + size = 0 + rendered = false + output buffer #58: + timeUs = 1000001369859 + size = 0 + rendered = false + output buffer #59: + timeUs = 1000001393079 + size = 0 + rendered = false + output buffer #60: + timeUs = 1000001416299 + size = 0 + rendered = false + output buffer #61: + timeUs = 1000001439519 + size = 0 + rendered = false + output buffer #62: + timeUs = 1000001462739 + size = 0 + rendered = false + output buffer #63: + timeUs = 1000001485959 + size = 0 + rendered = false + output buffer #64: + timeUs = 1000001509179 + size = 0 + rendered = false + output buffer #65: + timeUs = 1000001532399 + size = 0 + rendered = false + output buffer #66: + timeUs = 1000001555619 + size = 0 + rendered = false + output buffer #67: + timeUs = 1000001578839 + size = 0 + rendered = false + output buffer #68: + timeUs = 1000001602058 + size = 0 + rendered = false + output buffer #69: + timeUs = 1000001625278 + size = 0 + rendered = false + output buffer #70: + timeUs = 1000001648498 + size = 0 + rendered = false + output buffer #71: + timeUs = 1000001671718 + size = 0 + rendered = false + output buffer #72: + timeUs = 1000001694938 + size = 0 + rendered = false + output buffer #73: + timeUs = 1000001718158 + size = 0 + rendered = false + output buffer #74: + timeUs = 1000001741378 + size = 0 + rendered = false + output buffer #75: + timeUs = 1000001764598 + size = 0 + rendered = false + output buffer #76: + timeUs = 1000001787818 + size = 0 + rendered = false + output buffer #77: + timeUs = 1000001811038 + size = 0 + rendered = false + output buffer #78: + timeUs = 1000001834258 + size = 0 + rendered = false + output buffer #79: + timeUs = 1000001857478 + size = 0 + rendered = false + output buffer #80: + timeUs = 1000001880698 + size = 0 + rendered = false + output buffer #81: + timeUs = 1000001903918 + size = 0 + rendered = false + output buffer #82: + timeUs = 1000001927138 + size = 0 + rendered = false + output buffer #83: + timeUs = 1000001950358 + size = 0 + rendered = false + output buffer #84: + timeUs = 1000001973578 + size = 0 + rendered = false + output buffer #85: + timeUs = 1000001996798 + size = 0 + rendered = false + output buffer #86: + timeUs = 1000002020018 + size = 0 + rendered = false + output buffer #87: + timeUs = 1000002043238 + size = 0 + rendered = false + output buffer #88: + timeUs = 1000002066458 + size = 0 + rendered = false + output buffer #89: + timeUs = 1000002089678 + size = 0 + rendered = false +MediaCodecAdapter (media3.audio.aac): + inputBuffers: + count = 92 + input buffer #0: + timeUs = 1000000044000 + contents = length 23, hash 47DE9131 + input buffer #1: + timeUs = 1000000067219 + contents = length 6, hash 31EC5206 + input buffer #2: + timeUs = 1000000090439 + contents = length 148, hash 894A176B + input buffer #3: + timeUs = 1000000113659 + contents = length 189, hash CEF235A1 + input buffer #4: + timeUs = 1000000136879 + contents = length 205, hash BBF5F7B0 + input buffer #5: + timeUs = 1000000160099 + contents = length 210, hash F278B193 + input buffer #6: + timeUs = 1000000183319 + contents = length 210, hash 82DA1589 + input buffer #7: + timeUs = 1000000206539 + contents = length 207, hash 5BE231DF + input buffer #8: + timeUs = 1000000229759 + contents = length 225, hash 18819EE1 + input buffer #9: + timeUs = 1000000252979 + contents = length 215, hash CA7FA67B + input buffer #10: + timeUs = 1000000276199 + contents = length 211, hash 581A1C18 + input buffer #11: + timeUs = 1000000299419 + contents = length 216, hash ADB88187 + input buffer #12: + timeUs = 1000000322639 + contents = length 229, hash 2E8BA4DC + input buffer #13: + timeUs = 1000000345859 + contents = length 232, hash 22F0C510 + input buffer #14: + timeUs = 1000000369079 + contents = length 235, hash 867AD0DC + input buffer #15: + timeUs = 1000000392299 + contents = length 231, hash 84E823A8 + input buffer #16: + timeUs = 1000000415519 + contents = length 226, hash 1BEF3A95 + input buffer #17: + timeUs = 1000000438739 + contents = length 216, hash EAA345AE + input buffer #18: + timeUs = 1000000461959 + contents = length 229, hash 6957411F + input buffer #19: + timeUs = 1000000485179 + contents = length 219, hash 41275022 + input buffer #20: + timeUs = 1000000508399 + contents = length 241, hash 6495DF96 + input buffer #21: + timeUs = 1000000531619 + contents = length 228, hash 63D95906 + input buffer #22: + timeUs = 1000000554839 + contents = length 238, hash 34F676F9 + input buffer #23: + timeUs = 1000000578058 + contents = length 234, hash E5CBC045 + input buffer #24: + timeUs = 1000000601278 + contents = length 231, hash 5FC43661 + input buffer #25: + timeUs = 1000000624498 + contents = length 217, hash 682708ED + input buffer #26: + timeUs = 1000000647718 + contents = length 239, hash D43780FC + input buffer #27: + timeUs = 1000000670938 + contents = length 243, hash C5E17980 + input buffer #28: + timeUs = 1000000694158 + contents = length 231, hash AC5837BA + input buffer #29: + timeUs = 1000000717378 + contents = length 230, hash 169EE895 + input buffer #30: + timeUs = 1000000740598 + contents = length 238, hash C48FF3F1 + input buffer #31: + timeUs = 1000000763818 + contents = length 225, hash 531E4599 + input buffer #32: + timeUs = 1000000787038 + contents = length 232, hash CB3C6B8D + input buffer #33: + timeUs = 1000000810258 + contents = length 243, hash F8C94C7 + input buffer #34: + timeUs = 1000000833478 + contents = length 232, hash A646A7D0 + input buffer #35: + timeUs = 1000000856698 + contents = length 237, hash E8B787A5 + input buffer #36: + timeUs = 1000000879918 + contents = length 228, hash 3FA7A29F + input buffer #37: + timeUs = 1000000903138 + contents = length 235, hash B9B33B0A + input buffer #38: + timeUs = 1000000926358 + contents = length 264, hash 71A4869E + input buffer #39: + timeUs = 1000000949578 + contents = length 257, hash D049B54C + input buffer #40: + timeUs = 1000000972798 + contents = length 227, hash 66757231 + input buffer #41: + timeUs = 1000000996018 + contents = length 227, hash BD374F1B + input buffer #42: + timeUs = 1000001019238 + contents = length 235, hash 999477F6 + input buffer #43: + timeUs = 1000001042458 + contents = length 229, hash FFF98DF0 + input buffer #44: + timeUs = 1000001065678 + contents = length 6, hash 31B22286 + input buffer #45: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + input buffer #46: + timeUs = 1000001068000 + contents = length 23, hash 47DE9131 + input buffer #47: + timeUs = 1000001091219 + contents = length 6, hash 31EC5206 + input buffer #48: + timeUs = 1000001114439 + contents = length 148, hash 894A176B + input buffer #49: + timeUs = 1000001137659 + contents = length 189, hash CEF235A1 + input buffer #50: + timeUs = 1000001160879 + contents = length 205, hash BBF5F7B0 + input buffer #51: + timeUs = 1000001184099 + contents = length 210, hash F278B193 + input buffer #52: + timeUs = 1000001207319 + contents = length 210, hash 82DA1589 + input buffer #53: + timeUs = 1000001230539 + contents = length 207, hash 5BE231DF + input buffer #54: + timeUs = 1000001253759 + contents = length 225, hash 18819EE1 + input buffer #55: + timeUs = 1000001276979 + contents = length 215, hash CA7FA67B + input buffer #56: + timeUs = 1000001300199 + contents = length 211, hash 581A1C18 + input buffer #57: + timeUs = 1000001323419 + contents = length 216, hash ADB88187 + input buffer #58: + timeUs = 1000001346639 + contents = length 229, hash 2E8BA4DC + input buffer #59: + timeUs = 1000001369859 + contents = length 232, hash 22F0C510 + input buffer #60: + timeUs = 1000001393079 + contents = length 235, hash 867AD0DC + input buffer #61: + timeUs = 1000001416299 + contents = length 231, hash 84E823A8 + input buffer #62: + timeUs = 1000001439519 + contents = length 226, hash 1BEF3A95 + input buffer #63: + timeUs = 1000001462739 + contents = length 216, hash EAA345AE + input buffer #64: + timeUs = 1000001485959 + contents = length 229, hash 6957411F + input buffer #65: + timeUs = 1000001509179 + contents = length 219, hash 41275022 + input buffer #66: + timeUs = 1000001532399 + contents = length 241, hash 6495DF96 + input buffer #67: + timeUs = 1000001555619 + contents = length 228, hash 63D95906 + input buffer #68: + timeUs = 1000001578839 + contents = length 238, hash 34F676F9 + input buffer #69: + timeUs = 1000001602058 + contents = length 234, hash E5CBC045 + input buffer #70: + timeUs = 1000001625278 + contents = length 231, hash 5FC43661 + input buffer #71: + timeUs = 1000001648498 + contents = length 217, hash 682708ED + input buffer #72: + timeUs = 1000001671718 + contents = length 239, hash D43780FC + input buffer #73: + timeUs = 1000001694938 + contents = length 243, hash C5E17980 + input buffer #74: + timeUs = 1000001718158 + contents = length 231, hash AC5837BA + input buffer #75: + timeUs = 1000001741378 + contents = length 230, hash 169EE895 + input buffer #76: + timeUs = 1000001764598 + contents = length 238, hash C48FF3F1 + input buffer #77: + timeUs = 1000001787818 + contents = length 225, hash 531E4599 + input buffer #78: + timeUs = 1000001811038 + contents = length 232, hash CB3C6B8D + input buffer #79: + timeUs = 1000001834258 + contents = length 243, hash F8C94C7 + input buffer #80: + timeUs = 1000001857478 + contents = length 232, hash A646A7D0 + input buffer #81: + timeUs = 1000001880698 + contents = length 237, hash E8B787A5 + input buffer #82: + timeUs = 1000001903918 + contents = length 228, hash 3FA7A29F + input buffer #83: + timeUs = 1000001927138 + contents = length 235, hash B9B33B0A + input buffer #84: + timeUs = 1000001950358 + contents = length 264, hash 71A4869E + input buffer #85: + timeUs = 1000001973578 + contents = length 257, hash D049B54C + input buffer #86: + timeUs = 1000001996798 + contents = length 227, hash 66757231 + input buffer #87: + timeUs = 1000002020018 + contents = length 227, hash BD374F1B + input buffer #88: + timeUs = 1000002043238 + contents = length 235, hash 999477F6 + input buffer #89: + timeUs = 1000002066458 + contents = length 229, hash FFF98DF0 + input buffer #90: + timeUs = 1000002089678 + contents = length 6, hash 31B22286 + input buffer #91: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 90 + output buffer #0: + timeUs = 1000000044000 + size = 0 + rendered = false + output buffer #1: + timeUs = 1000000067219 + size = 0 + rendered = false + output buffer #2: + timeUs = 1000000090439 + size = 0 + rendered = false + output buffer #3: + timeUs = 1000000113659 + size = 0 + rendered = false + output buffer #4: + timeUs = 1000000136879 + size = 0 + rendered = false + output buffer #5: + timeUs = 1000000160099 + size = 0 + rendered = false + output buffer #6: + timeUs = 1000000183319 + size = 0 + rendered = false + output buffer #7: + timeUs = 1000000206539 + size = 0 + rendered = false + output buffer #8: + timeUs = 1000000229759 + size = 0 + rendered = false + output buffer #9: + timeUs = 1000000252979 + size = 0 + rendered = false + output buffer #10: + timeUs = 1000000276199 + size = 0 + rendered = false + output buffer #11: + timeUs = 1000000299419 + size = 0 + rendered = false + output buffer #12: + timeUs = 1000000322639 + size = 0 + rendered = false + output buffer #13: + timeUs = 1000000345859 + size = 0 + rendered = false + output buffer #14: + timeUs = 1000000369079 + size = 0 + rendered = false + output buffer #15: + timeUs = 1000000392299 + size = 0 + rendered = false + output buffer #16: + timeUs = 1000000415519 + size = 0 + rendered = false + output buffer #17: + timeUs = 1000000438739 + size = 0 + rendered = false + output buffer #18: + timeUs = 1000000461959 + size = 0 + rendered = false + output buffer #19: + timeUs = 1000000485179 + size = 0 + rendered = false + output buffer #20: + timeUs = 1000000508399 + size = 0 + rendered = false + output buffer #21: + timeUs = 1000000531619 + size = 0 + rendered = false + output buffer #22: + timeUs = 1000000554839 + size = 0 + rendered = false + output buffer #23: + timeUs = 1000000578058 + size = 0 + rendered = false + output buffer #24: + timeUs = 1000000601278 + size = 0 + rendered = false + output buffer #25: + timeUs = 1000000624498 + size = 0 + rendered = false + output buffer #26: + timeUs = 1000000647718 + size = 0 + rendered = false + output buffer #27: + timeUs = 1000000670938 + size = 0 + rendered = false + output buffer #28: + timeUs = 1000000694158 + size = 0 + rendered = false + output buffer #29: + timeUs = 1000000717378 + size = 0 + rendered = false + output buffer #30: + timeUs = 1000000740598 + size = 0 + rendered = false + output buffer #31: + timeUs = 1000000763818 + size = 0 + rendered = false + output buffer #32: + timeUs = 1000000787038 + size = 0 + rendered = false + output buffer #33: + timeUs = 1000000810258 + size = 0 + rendered = false + output buffer #34: + timeUs = 1000000833478 + size = 0 + rendered = false + output buffer #35: + timeUs = 1000000856698 + size = 0 + rendered = false + output buffer #36: + timeUs = 1000000879918 + size = 0 + rendered = false + output buffer #37: + timeUs = 1000000903138 + size = 0 + rendered = false + output buffer #38: + timeUs = 1000000926358 + size = 0 + rendered = false + output buffer #39: + timeUs = 1000000949578 + size = 0 + rendered = false + output buffer #40: + timeUs = 1000000972798 + size = 0 + rendered = false + output buffer #41: + timeUs = 1000000996018 + size = 0 + rendered = false + output buffer #42: + timeUs = 1000001019238 + size = 0 + rendered = false + output buffer #43: + timeUs = 1000001042458 + size = 0 + rendered = false + output buffer #44: + timeUs = 1000001065678 + size = 0 + rendered = false + output buffer #45: + timeUs = 1000001068000 + size = 0 + rendered = false + output buffer #46: + timeUs = 1000001091219 + size = 0 + rendered = false + output buffer #47: + timeUs = 1000001114439 + size = 0 + rendered = false + output buffer #48: + timeUs = 1000001137659 + size = 0 + rendered = false + output buffer #49: + timeUs = 1000001160879 + size = 0 + rendered = false + output buffer #50: + timeUs = 1000001184099 + size = 0 + rendered = false + output buffer #51: + timeUs = 1000001207319 + size = 0 + rendered = false + output buffer #52: + timeUs = 1000001230539 + size = 0 + rendered = false + output buffer #53: + timeUs = 1000001253759 + size = 0 + rendered = false + output buffer #54: + timeUs = 1000001276979 + size = 0 + rendered = false + output buffer #55: + timeUs = 1000001300199 + size = 0 + rendered = false + output buffer #56: + timeUs = 1000001323419 + size = 0 + rendered = false + output buffer #57: + timeUs = 1000001346639 + size = 0 + rendered = false + output buffer #58: + timeUs = 1000001369859 + size = 0 + rendered = false + output buffer #59: + timeUs = 1000001393079 + size = 0 + rendered = false + output buffer #60: + timeUs = 1000001416299 + size = 0 + rendered = false + output buffer #61: + timeUs = 1000001439519 + size = 0 + rendered = false + output buffer #62: + timeUs = 1000001462739 + size = 0 + rendered = false + output buffer #63: + timeUs = 1000001485959 + size = 0 + rendered = false + output buffer #64: + timeUs = 1000001509179 + size = 0 + rendered = false + output buffer #65: + timeUs = 1000001532399 + size = 0 + rendered = false + output buffer #66: + timeUs = 1000001555619 + size = 0 + rendered = false + output buffer #67: + timeUs = 1000001578839 + size = 0 + rendered = false + output buffer #68: + timeUs = 1000001602058 + size = 0 + rendered = false + output buffer #69: + timeUs = 1000001625278 + size = 0 + rendered = false + output buffer #70: + timeUs = 1000001648498 + size = 0 + rendered = false + output buffer #71: + timeUs = 1000001671718 + size = 0 + rendered = false + output buffer #72: + timeUs = 1000001694938 + size = 0 + rendered = false + output buffer #73: + timeUs = 1000001718158 + size = 0 + rendered = false + output buffer #74: + timeUs = 1000001741378 + size = 0 + rendered = false + output buffer #75: + timeUs = 1000001764598 + size = 0 + rendered = false + output buffer #76: + timeUs = 1000001787818 + size = 0 + rendered = false + output buffer #77: + timeUs = 1000001811038 + size = 0 + rendered = false + output buffer #78: + timeUs = 1000001834258 + size = 0 + rendered = false + output buffer #79: + timeUs = 1000001857478 + size = 0 + rendered = false + output buffer #80: + timeUs = 1000001880698 + size = 0 + rendered = false + output buffer #81: + timeUs = 1000001903918 + size = 0 + rendered = false + output buffer #82: + timeUs = 1000001927138 + size = 0 + rendered = false + output buffer #83: + timeUs = 1000001950358 + size = 0 + rendered = false + output buffer #84: + timeUs = 1000001973578 + size = 0 + rendered = false + output buffer #85: + timeUs = 1000001996798 + size = 0 + rendered = false + output buffer #86: + timeUs = 1000002020018 + size = 0 + rendered = false + output buffer #87: + timeUs = 1000002043238 + size = 0 + rendered = false + output buffer #88: + timeUs = 1000002066458 + size = 0 + rendered = false + output buffer #89: + timeUs = 1000002089678 + size = 0 + rendered = false +MediaCodecAdapter (media3.video.avc): + inputBuffers: + count = 31 + input buffer #0: + timeUs = 1000000000000 + contents = length 36692, hash D216076E + input buffer #1: + timeUs = 1000000066733 + contents = length 5312, hash D45D3CA0 + input buffer #2: + timeUs = 1000000033366 + contents = length 599, hash 1BE7812D + input buffer #3: + timeUs = 1000000200200 + contents = length 7735, hash 4490F110 + input buffer #4: + timeUs = 1000000133466 + contents = length 987, hash 560B5036 + input buffer #5: + timeUs = 1000000100100 + contents = length 673, hash ED7CD8C7 + input buffer #6: + timeUs = 1000000166833 + contents = length 523, hash 3020DF50 + input buffer #7: + timeUs = 1000000333666 + contents = length 6061, hash 736C72B2 + input buffer #8: + timeUs = 1000000266933 + contents = length 992, hash FE132F23 + input buffer #9: + timeUs = 1000000233566 + contents = length 623, hash 5B2C1816 + input buffer #10: + timeUs = 1000000300300 + contents = length 421, hash 742E69C1 + input buffer #11: + timeUs = 1000000433766 + contents = length 4899, hash F72F86A1 + input buffer #12: + timeUs = 1000000400400 + contents = length 568, hash 519A8E50 + input buffer #13: + timeUs = 1000000367033 + contents = length 620, hash 3990AA39 + input buffer #14: + timeUs = 1000000567233 + contents = length 5450, hash F06EC4AA + input buffer #15: + timeUs = 1000000500500 + contents = length 1051, hash 92DFA63A + input buffer #16: + timeUs = 1000000467133 + contents = length 874, hash 69587FB4 + input buffer #17: + timeUs = 1000000533866 + contents = length 781, hash 36BE495B + input buffer #18: + timeUs = 1000000700700 + contents = length 4725, hash AC0C8CD3 + input buffer #19: + timeUs = 1000000633966 + contents = length 1022, hash 5D8BFF34 + input buffer #20: + timeUs = 1000000600600 + contents = length 790, hash 99413A99 + input buffer #21: + timeUs = 1000000667333 + contents = length 610, hash 5E129290 + input buffer #22: + timeUs = 1000000834166 + contents = length 2751, hash 769974CB + input buffer #23: + timeUs = 1000000767433 + contents = length 745, hash B78A477A + input buffer #24: + timeUs = 1000000734066 + contents = length 621, hash CF741E7A + input buffer #25: + timeUs = 1000000800800 + contents = length 505, hash 1DB4894E + input buffer #26: + timeUs = 1000000967633 + contents = length 1268, hash C15348DC + input buffer #27: + timeUs = 1000000900900 + contents = length 880, hash C2DE85D0 + input buffer #28: + timeUs = 1000000867533 + contents = length 530, hash C98BC6A8 + input buffer #29: + timeUs = 1000000934266 + contents = length 568, hash 4FE5C8EA + input buffer #30: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 30 + output buffer #0: + timeUs = 1000000000000 + size = 36692 + rendered = true + output buffer #1: + timeUs = 1000000066733 + size = 5312 + rendered = true + output buffer #2: + timeUs = 1000000033366 + size = 599 + rendered = true + output buffer #3: + timeUs = 1000000200200 + size = 7735 + rendered = true + output buffer #4: + timeUs = 1000000133466 + size = 987 + rendered = true + output buffer #5: + timeUs = 1000000100100 + size = 673 + rendered = true + output buffer #6: + timeUs = 1000000166833 + size = 523 + rendered = true + output buffer #7: + timeUs = 1000000333666 + size = 6061 + rendered = true + output buffer #8: + timeUs = 1000000266933 + size = 992 + rendered = true + output buffer #9: + timeUs = 1000000233566 + size = 623 + rendered = true + output buffer #10: + timeUs = 1000000300300 + size = 421 + rendered = true + output buffer #11: + timeUs = 1000000433766 + size = 4899 + rendered = true + output buffer #12: + timeUs = 1000000400400 + size = 568 + rendered = true + output buffer #13: + timeUs = 1000000367033 + size = 620 + rendered = true + output buffer #14: + timeUs = 1000000567233 + size = 5450 + rendered = true + output buffer #15: + timeUs = 1000000500500 + size = 1051 + rendered = true + output buffer #16: + timeUs = 1000000467133 + size = 874 + rendered = true + output buffer #17: + timeUs = 1000000533866 + size = 781 + rendered = true + output buffer #18: + timeUs = 1000000700700 + size = 4725 + rendered = true + output buffer #19: + timeUs = 1000000633966 + size = 1022 + rendered = true + output buffer #20: + timeUs = 1000000600600 + size = 790 + rendered = true + output buffer #21: + timeUs = 1000000667333 + size = 610 + rendered = true + output buffer #22: + timeUs = 1000000834166 + size = 2751 + rendered = true + output buffer #23: + timeUs = 1000000767433 + size = 745 + rendered = true + output buffer #24: + timeUs = 1000000734066 + size = 621 + rendered = true + output buffer #25: + timeUs = 1000000800800 + size = 505 + rendered = true + output buffer #26: + timeUs = 1000000967633 + size = 1268 + rendered = true + output buffer #27: + timeUs = 1000000900900 + size = 880 + rendered = true + output buffer #28: + timeUs = 1000000867533 + size = 530 + rendered = true + output buffer #29: + timeUs = 1000000934266 + size = 568 + rendered = true +MediaCodecAdapter (media3.video.avc): + inputBuffers: + count = 31 + input buffer #0: + timeUs = 1000001024000 + contents = length 36692, hash D216076E + input buffer #1: + timeUs = 1000001090733 + contents = length 5312, hash D45D3CA0 + input buffer #2: + timeUs = 1000001057366 + contents = length 599, hash 1BE7812D + input buffer #3: + timeUs = 1000001224200 + contents = length 7735, hash 4490F110 + input buffer #4: + timeUs = 1000001157466 + contents = length 987, hash 560B5036 + input buffer #5: + timeUs = 1000001124100 + contents = length 673, hash ED7CD8C7 + input buffer #6: + timeUs = 1000001190833 + contents = length 523, hash 3020DF50 + input buffer #7: + timeUs = 1000001357666 + contents = length 6061, hash 736C72B2 + input buffer #8: + timeUs = 1000001290933 + contents = length 992, hash FE132F23 + input buffer #9: + timeUs = 1000001257566 + contents = length 623, hash 5B2C1816 + input buffer #10: + timeUs = 1000001324300 + contents = length 421, hash 742E69C1 + input buffer #11: + timeUs = 1000001457766 + contents = length 4899, hash F72F86A1 + input buffer #12: + timeUs = 1000001424400 + contents = length 568, hash 519A8E50 + input buffer #13: + timeUs = 1000001391033 + contents = length 620, hash 3990AA39 + input buffer #14: + timeUs = 1000001591233 + contents = length 5450, hash F06EC4AA + input buffer #15: + timeUs = 1000001524500 + contents = length 1051, hash 92DFA63A + input buffer #16: + timeUs = 1000001491133 + contents = length 874, hash 69587FB4 + input buffer #17: + timeUs = 1000001557866 + contents = length 781, hash 36BE495B + input buffer #18: + timeUs = 1000001724700 + contents = length 4725, hash AC0C8CD3 + input buffer #19: + timeUs = 1000001657966 + contents = length 1022, hash 5D8BFF34 + input buffer #20: + timeUs = 1000001624600 + contents = length 790, hash 99413A99 + input buffer #21: + timeUs = 1000001691333 + contents = length 610, hash 5E129290 + input buffer #22: + timeUs = 1000001858166 + contents = length 2751, hash 769974CB + input buffer #23: + timeUs = 1000001791433 + contents = length 745, hash B78A477A + input buffer #24: + timeUs = 1000001758066 + contents = length 621, hash CF741E7A + input buffer #25: + timeUs = 1000001824800 + contents = length 505, hash 1DB4894E + input buffer #26: + timeUs = 1000001991633 + contents = length 1268, hash C15348DC + input buffer #27: + timeUs = 1000001924900 + contents = length 880, hash C2DE85D0 + input buffer #28: + timeUs = 1000001891533 + contents = length 530, hash C98BC6A8 + input buffer #29: + timeUs = 1000001958266 + contents = length 568, hash 4FE5C8EA + input buffer #30: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 30 + output buffer #0: + timeUs = 1000001024000 + size = 36692 + rendered = true + output buffer #1: + timeUs = 1000001090733 + size = 5312 + rendered = true + output buffer #2: + timeUs = 1000001057366 + size = 599 + rendered = true + output buffer #3: + timeUs = 1000001224200 + size = 7735 + rendered = true + output buffer #4: + timeUs = 1000001157466 + size = 987 + rendered = true + output buffer #5: + timeUs = 1000001124100 + size = 673 + rendered = true + output buffer #6: + timeUs = 1000001190833 + size = 523 + rendered = true + output buffer #7: + timeUs = 1000001357666 + size = 6061 + rendered = true + output buffer #8: + timeUs = 1000001290933 + size = 992 + rendered = true + output buffer #9: + timeUs = 1000001257566 + size = 623 + rendered = true + output buffer #10: + timeUs = 1000001324300 + size = 421 + rendered = true + output buffer #11: + timeUs = 1000001457766 + size = 4899 + rendered = true + output buffer #12: + timeUs = 1000001424400 + size = 568 + rendered = true + output buffer #13: + timeUs = 1000001391033 + size = 620 + rendered = true + output buffer #14: + timeUs = 1000001591233 + size = 5450 + rendered = true + output buffer #15: + timeUs = 1000001524500 + size = 1051 + rendered = true + output buffer #16: + timeUs = 1000001491133 + size = 874 + rendered = true + output buffer #17: + timeUs = 1000001557866 + size = 781 + rendered = true + output buffer #18: + timeUs = 1000001724700 + size = 4725 + rendered = true + output buffer #19: + timeUs = 1000001657966 + size = 1022 + rendered = true + output buffer #20: + timeUs = 1000001624600 + size = 790 + rendered = true + output buffer #21: + timeUs = 1000001691333 + size = 610 + rendered = true + output buffer #22: + timeUs = 1000001858166 + size = 2751 + rendered = true + output buffer #23: + timeUs = 1000001791433 + size = 745 + rendered = true + output buffer #24: + timeUs = 1000001758066 + size = 621 + rendered = true + output buffer #25: + timeUs = 1000001824800 + size = 505 + rendered = true + output buffer #26: + timeUs = 1000001991633 + size = 1268 + rendered = true + output buffer #27: + timeUs = 1000001924900 + size = 880 + rendered = true + output buffer #28: + timeUs = 1000001891533 + size = 530 + rendered = true + output buffer #29: + timeUs = 1000001958266 + size = 568 + rendered = true +MediaCodecAdapter (media3.video.avc): + inputBuffers: + count = 31 + input buffer #0: + timeUs = 1000000000000 + contents = length 36692, hash D216076E + input buffer #1: + timeUs = 1000000066733 + contents = length 5312, hash D45D3CA0 + input buffer #2: + timeUs = 1000000033366 + contents = length 599, hash 1BE7812D + input buffer #3: + timeUs = 1000000200200 + contents = length 7735, hash 4490F110 + input buffer #4: + timeUs = 1000000133466 + contents = length 987, hash 560B5036 + input buffer #5: + timeUs = 1000000100100 + contents = length 673, hash ED7CD8C7 + input buffer #6: + timeUs = 1000000166833 + contents = length 523, hash 3020DF50 + input buffer #7: + timeUs = 1000000333666 + contents = length 6061, hash 736C72B2 + input buffer #8: + timeUs = 1000000266933 + contents = length 992, hash FE132F23 + input buffer #9: + timeUs = 1000000233566 + contents = length 623, hash 5B2C1816 + input buffer #10: + timeUs = 1000000300300 + contents = length 421, hash 742E69C1 + input buffer #11: + timeUs = 1000000433766 + contents = length 4899, hash F72F86A1 + input buffer #12: + timeUs = 1000000400400 + contents = length 568, hash 519A8E50 + input buffer #13: + timeUs = 1000000367033 + contents = length 620, hash 3990AA39 + input buffer #14: + timeUs = 1000000567233 + contents = length 5450, hash F06EC4AA + input buffer #15: + timeUs = 1000000500500 + contents = length 1051, hash 92DFA63A + input buffer #16: + timeUs = 1000000467133 + contents = length 874, hash 69587FB4 + input buffer #17: + timeUs = 1000000533866 + contents = length 781, hash 36BE495B + input buffer #18: + timeUs = 1000000700700 + contents = length 4725, hash AC0C8CD3 + input buffer #19: + timeUs = 1000000633966 + contents = length 1022, hash 5D8BFF34 + input buffer #20: + timeUs = 1000000600600 + contents = length 790, hash 99413A99 + input buffer #21: + timeUs = 1000000667333 + contents = length 610, hash 5E129290 + input buffer #22: + timeUs = 1000000834166 + contents = length 2751, hash 769974CB + input buffer #23: + timeUs = 1000000767433 + contents = length 745, hash B78A477A + input buffer #24: + timeUs = 1000000734066 + contents = length 621, hash CF741E7A + input buffer #25: + timeUs = 1000000800800 + contents = length 505, hash 1DB4894E + input buffer #26: + timeUs = 1000000967633 + contents = length 1268, hash C15348DC + input buffer #27: + timeUs = 1000000900900 + contents = length 880, hash C2DE85D0 + input buffer #28: + timeUs = 1000000867533 + contents = length 530, hash C98BC6A8 + input buffer #29: + timeUs = 1000000934266 + contents = length 568, hash 4FE5C8EA + input buffer #30: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 30 + output buffer #0: + timeUs = 1000000000000 + size = 36692 + rendered = true + output buffer #1: + timeUs = 1000000066733 + size = 5312 + rendered = true + output buffer #2: + timeUs = 1000000033366 + size = 599 + rendered = true + output buffer #3: + timeUs = 1000000200200 + size = 7735 + rendered = true + output buffer #4: + timeUs = 1000000133466 + size = 987 + rendered = true + output buffer #5: + timeUs = 1000000100100 + size = 673 + rendered = true + output buffer #6: + timeUs = 1000000166833 + size = 523 + rendered = true + output buffer #7: + timeUs = 1000000333666 + size = 6061 + rendered = true + output buffer #8: + timeUs = 1000000266933 + size = 992 + rendered = true + output buffer #9: + timeUs = 1000000233566 + size = 623 + rendered = true + output buffer #10: + timeUs = 1000000300300 + size = 421 + rendered = true + output buffer #11: + timeUs = 1000000433766 + size = 4899 + rendered = true + output buffer #12: + timeUs = 1000000400400 + size = 568 + rendered = true + output buffer #13: + timeUs = 1000000367033 + size = 620 + rendered = true + output buffer #14: + timeUs = 1000000567233 + size = 5450 + rendered = true + output buffer #15: + timeUs = 1000000500500 + size = 1051 + rendered = true + output buffer #16: + timeUs = 1000000467133 + size = 874 + rendered = true + output buffer #17: + timeUs = 1000000533866 + size = 781 + rendered = true + output buffer #18: + timeUs = 1000000700700 + size = 4725 + rendered = true + output buffer #19: + timeUs = 1000000633966 + size = 1022 + rendered = true + output buffer #20: + timeUs = 1000000600600 + size = 790 + rendered = true + output buffer #21: + timeUs = 1000000667333 + size = 610 + rendered = true + output buffer #22: + timeUs = 1000000834166 + size = 2751 + rendered = true + output buffer #23: + timeUs = 1000000767433 + size = 745 + rendered = true + output buffer #24: + timeUs = 1000000734066 + size = 621 + rendered = true + output buffer #25: + timeUs = 1000000800800 + size = 505 + rendered = true + output buffer #26: + timeUs = 1000000967633 + size = 1268 + rendered = true + output buffer #27: + timeUs = 1000000900900 + size = 880 + rendered = true + output buffer #28: + timeUs = 1000000867533 + size = 530 + rendered = true + output buffer #29: + timeUs = 1000000934266 + size = 568 + rendered = true +MediaCodecAdapter (media3.video.avc): + inputBuffers: + count = 31 + input buffer #0: + timeUs = 1000001024000 + contents = length 36692, hash D216076E + input buffer #1: + timeUs = 1000001090733 + contents = length 5312, hash D45D3CA0 + input buffer #2: + timeUs = 1000001057366 + contents = length 599, hash 1BE7812D + input buffer #3: + timeUs = 1000001224200 + contents = length 7735, hash 4490F110 + input buffer #4: + timeUs = 1000001157466 + contents = length 987, hash 560B5036 + input buffer #5: + timeUs = 1000001124100 + contents = length 673, hash ED7CD8C7 + input buffer #6: + timeUs = 1000001190833 + contents = length 523, hash 3020DF50 + input buffer #7: + timeUs = 1000001357666 + contents = length 6061, hash 736C72B2 + input buffer #8: + timeUs = 1000001290933 + contents = length 992, hash FE132F23 + input buffer #9: + timeUs = 1000001257566 + contents = length 623, hash 5B2C1816 + input buffer #10: + timeUs = 1000001324300 + contents = length 421, hash 742E69C1 + input buffer #11: + timeUs = 1000001457766 + contents = length 4899, hash F72F86A1 + input buffer #12: + timeUs = 1000001424400 + contents = length 568, hash 519A8E50 + input buffer #13: + timeUs = 1000001391033 + contents = length 620, hash 3990AA39 + input buffer #14: + timeUs = 1000001591233 + contents = length 5450, hash F06EC4AA + input buffer #15: + timeUs = 1000001524500 + contents = length 1051, hash 92DFA63A + input buffer #16: + timeUs = 1000001491133 + contents = length 874, hash 69587FB4 + input buffer #17: + timeUs = 1000001557866 + contents = length 781, hash 36BE495B + input buffer #18: + timeUs = 1000001724700 + contents = length 4725, hash AC0C8CD3 + input buffer #19: + timeUs = 1000001657966 + contents = length 1022, hash 5D8BFF34 + input buffer #20: + timeUs = 1000001624600 + contents = length 790, hash 99413A99 + input buffer #21: + timeUs = 1000001691333 + contents = length 610, hash 5E129290 + input buffer #22: + timeUs = 1000001858166 + contents = length 2751, hash 769974CB + input buffer #23: + timeUs = 1000001791433 + contents = length 745, hash B78A477A + input buffer #24: + timeUs = 1000001758066 + contents = length 621, hash CF741E7A + input buffer #25: + timeUs = 1000001824800 + contents = length 505, hash 1DB4894E + input buffer #26: + timeUs = 1000001991633 + contents = length 1268, hash C15348DC + input buffer #27: + timeUs = 1000001924900 + contents = length 880, hash C2DE85D0 + input buffer #28: + timeUs = 1000001891533 + contents = length 530, hash C98BC6A8 + input buffer #29: + timeUs = 1000001958266 + contents = length 568, hash 4FE5C8EA + input buffer #30: + timeUs = 0 + flags = 4 + contents = length 0, hash 1 + outputBuffers: + count = 30 + output buffer #0: + timeUs = 1000001024000 + size = 36692 + rendered = true + output buffer #1: + timeUs = 1000001090733 + size = 5312 + rendered = true + output buffer #2: + timeUs = 1000001057366 + size = 599 + rendered = true + output buffer #3: + timeUs = 1000001224200 + size = 7735 + rendered = true + output buffer #4: + timeUs = 1000001157466 + size = 987 + rendered = true + output buffer #5: + timeUs = 1000001124100 + size = 673 + rendered = true + output buffer #6: + timeUs = 1000001190833 + size = 523 + rendered = true + output buffer #7: + timeUs = 1000001357666 + size = 6061 + rendered = true + output buffer #8: + timeUs = 1000001290933 + size = 992 + rendered = true + output buffer #9: + timeUs = 1000001257566 + size = 623 + rendered = true + output buffer #10: + timeUs = 1000001324300 + size = 421 + rendered = true + output buffer #11: + timeUs = 1000001457766 + size = 4899 + rendered = true + output buffer #12: + timeUs = 1000001424400 + size = 568 + rendered = true + output buffer #13: + timeUs = 1000001391033 + size = 620 + rendered = true + output buffer #14: + timeUs = 1000001591233 + size = 5450 + rendered = true + output buffer #15: + timeUs = 1000001524500 + size = 1051 + rendered = true + output buffer #16: + timeUs = 1000001491133 + size = 874 + rendered = true + output buffer #17: + timeUs = 1000001557866 + size = 781 + rendered = true + output buffer #18: + timeUs = 1000001724700 + size = 4725 + rendered = true + output buffer #19: + timeUs = 1000001657966 + size = 1022 + rendered = true + output buffer #20: + timeUs = 1000001624600 + size = 790 + rendered = true + output buffer #21: + timeUs = 1000001691333 + size = 610 + rendered = true + output buffer #22: + timeUs = 1000001858166 + size = 2751 + rendered = true + output buffer #23: + timeUs = 1000001791433 + size = 745 + rendered = true + output buffer #24: + timeUs = 1000001758066 + size = 621 + rendered = true + output buffer #25: + timeUs = 1000001824800 + size = 505 + rendered = true + output buffer #26: + timeUs = 1000001991633 + size = 1268 + rendered = true + output buffer #27: + timeUs = 1000001924900 + size = 880 + rendered = true + output buffer #28: + timeUs = 1000001891533 + size = 530 + rendered = true + output buffer #29: + timeUs = 1000001958266 + size = 568 + rendered = true +AudioSink: + buffer count = 180 + config: + pcmEncoding = pcm-16 + channelCount = 1 + sampleRate = 44100 + buffer #0: + time = 1000000044000 + data = empty + buffer #1: + time = 1000000067219 + data = empty + buffer #2: + time = 1000000090439 + data = empty + buffer #3: + time = 1000000113659 + data = empty + buffer #4: + time = 1000000136879 + data = empty + buffer #5: + time = 1000000160099 + data = empty + buffer #6: + time = 1000000183319 + data = empty + buffer #7: + time = 1000000206539 + data = empty + buffer #8: + time = 1000000229759 + data = empty + buffer #9: + time = 1000000252979 + data = empty + buffer #10: + time = 1000000276199 + data = empty + buffer #11: + time = 1000000299419 + data = empty + buffer #12: + time = 1000000322639 + data = empty + buffer #13: + time = 1000000345859 + data = empty + buffer #14: + time = 1000000369079 + data = empty + buffer #15: + time = 1000000392299 + data = empty + buffer #16: + time = 1000000415519 + data = empty + buffer #17: + time = 1000000438739 + data = empty + buffer #18: + time = 1000000461959 + data = empty + buffer #19: + time = 1000000485179 + data = empty + buffer #20: + time = 1000000508399 + data = empty + buffer #21: + time = 1000000531619 + data = empty + buffer #22: + time = 1000000554839 + data = empty + buffer #23: + time = 1000000578058 + data = empty + buffer #24: + time = 1000000601278 + data = empty + buffer #25: + time = 1000000624498 + data = empty + buffer #26: + time = 1000000647718 + data = empty + buffer #27: + time = 1000000670938 + data = empty + buffer #28: + time = 1000000694158 + data = empty + buffer #29: + time = 1000000717378 + data = empty + buffer #30: + time = 1000000740598 + data = empty + buffer #31: + time = 1000000763818 + data = empty + buffer #32: + time = 1000000787038 + data = empty + buffer #33: + time = 1000000810258 + data = empty + buffer #34: + time = 1000000833478 + data = empty + buffer #35: + time = 1000000856698 + data = empty + buffer #36: + time = 1000000879918 + data = empty + buffer #37: + time = 1000000903138 + data = empty + buffer #38: + time = 1000000926358 + data = empty + buffer #39: + time = 1000000949578 + data = empty + buffer #40: + time = 1000000972798 + data = empty + buffer #41: + time = 1000000996018 + data = empty + buffer #42: + time = 1000001019238 + data = empty + buffer #43: + time = 1000001042458 + data = empty + buffer #44: + time = 1000001065678 + data = empty + discontinuity: + config: + pcmEncoding = pcm-16 + channelCount = 1 + sampleRate = 44100 + buffer #45: + time = 1000001068000 + data = empty + buffer #46: + time = 1000001091219 + data = empty + buffer #47: + time = 1000001114439 + data = empty + buffer #48: + time = 1000001137659 + data = empty + buffer #49: + time = 1000001160879 + data = empty + buffer #50: + time = 1000001184099 + data = empty + buffer #51: + time = 1000001207319 + data = empty + buffer #52: + time = 1000001230539 + data = empty + buffer #53: + time = 1000001253759 + data = empty + buffer #54: + time = 1000001276979 + data = empty + buffer #55: + time = 1000001300199 + data = empty + buffer #56: + time = 1000001323419 + data = empty + buffer #57: + time = 1000001346639 + data = empty + buffer #58: + time = 1000001369859 + data = empty + buffer #59: + time = 1000001393079 + data = empty + buffer #60: + time = 1000001416299 + data = empty + buffer #61: + time = 1000001439519 + data = empty + buffer #62: + time = 1000001462739 + data = empty + buffer #63: + time = 1000001485959 + data = empty + buffer #64: + time = 1000001509179 + data = empty + buffer #65: + time = 1000001532399 + data = empty + buffer #66: + time = 1000001555619 + data = empty + buffer #67: + time = 1000001578839 + data = empty + buffer #68: + time = 1000001602058 + data = empty + buffer #69: + time = 1000001625278 + data = empty + buffer #70: + time = 1000001648498 + data = empty + buffer #71: + time = 1000001671718 + data = empty + buffer #72: + time = 1000001694938 + data = empty + buffer #73: + time = 1000001718158 + data = empty + buffer #74: + time = 1000001741378 + data = empty + buffer #75: + time = 1000001764598 + data = empty + buffer #76: + time = 1000001787818 + data = empty + buffer #77: + time = 1000001811038 + data = empty + buffer #78: + time = 1000001834258 + data = empty + buffer #79: + time = 1000001857478 + data = empty + buffer #80: + time = 1000001880698 + data = empty + buffer #81: + time = 1000001903918 + data = empty + buffer #82: + time = 1000001927138 + data = empty + buffer #83: + time = 1000001950358 + data = empty + buffer #84: + time = 1000001973578 + data = empty + buffer #85: + time = 1000001996798 + data = empty + buffer #86: + time = 1000002020018 + data = empty + buffer #87: + time = 1000002043238 + data = empty + buffer #88: + time = 1000002066458 + data = empty + buffer #89: + time = 1000002089678 + data = empty + config: + pcmEncoding = pcm-16 + channelCount = 1 + sampleRate = 44100 + buffer #90: + time = 1000000044000 + data = empty + buffer #91: + time = 1000000067219 + data = empty + buffer #92: + time = 1000000090439 + data = empty + buffer #93: + time = 1000000113659 + data = empty + buffer #94: + time = 1000000136879 + data = empty + buffer #95: + time = 1000000160099 + data = empty + buffer #96: + time = 1000000183319 + data = empty + buffer #97: + time = 1000000206539 + data = empty + buffer #98: + time = 1000000229759 + data = empty + buffer #99: + time = 1000000252979 + data = empty + buffer #100: + time = 1000000276199 + data = empty + buffer #101: + time = 1000000299419 + data = empty + buffer #102: + time = 1000000322639 + data = empty + buffer #103: + time = 1000000345859 + data = empty + buffer #104: + time = 1000000369079 + data = empty + buffer #105: + time = 1000000392299 + data = empty + buffer #106: + time = 1000000415519 + data = empty + buffer #107: + time = 1000000438739 + data = empty + buffer #108: + time = 1000000461959 + data = empty + buffer #109: + time = 1000000485179 + data = empty + buffer #110: + time = 1000000508399 + data = empty + buffer #111: + time = 1000000531619 + data = empty + buffer #112: + time = 1000000554839 + data = empty + buffer #113: + time = 1000000578058 + data = empty + buffer #114: + time = 1000000601278 + data = empty + buffer #115: + time = 1000000624498 + data = empty + buffer #116: + time = 1000000647718 + data = empty + buffer #117: + time = 1000000670938 + data = empty + buffer #118: + time = 1000000694158 + data = empty + buffer #119: + time = 1000000717378 + data = empty + buffer #120: + time = 1000000740598 + data = empty + buffer #121: + time = 1000000763818 + data = empty + buffer #122: + time = 1000000787038 + data = empty + buffer #123: + time = 1000000810258 + data = empty + buffer #124: + time = 1000000833478 + data = empty + buffer #125: + time = 1000000856698 + data = empty + buffer #126: + time = 1000000879918 + data = empty + buffer #127: + time = 1000000903138 + data = empty + buffer #128: + time = 1000000926358 + data = empty + buffer #129: + time = 1000000949578 + data = empty + buffer #130: + time = 1000000972798 + data = empty + buffer #131: + time = 1000000996018 + data = empty + buffer #132: + time = 1000001019238 + data = empty + buffer #133: + time = 1000001042458 + data = empty + buffer #134: + time = 1000001065678 + data = empty + discontinuity: + config: + pcmEncoding = pcm-16 + channelCount = 1 + sampleRate = 44100 + buffer #135: + time = 1000001068000 + data = empty + buffer #136: + time = 1000001091219 + data = empty + buffer #137: + time = 1000001114439 + data = empty + buffer #138: + time = 1000001137659 + data = empty + buffer #139: + time = 1000001160879 + data = empty + buffer #140: + time = 1000001184099 + data = empty + buffer #141: + time = 1000001207319 + data = empty + buffer #142: + time = 1000001230539 + data = empty + buffer #143: + time = 1000001253759 + data = empty + buffer #144: + time = 1000001276979 + data = empty + buffer #145: + time = 1000001300199 + data = empty + buffer #146: + time = 1000001323419 + data = empty + buffer #147: + time = 1000001346639 + data = empty + buffer #148: + time = 1000001369859 + data = empty + buffer #149: + time = 1000001393079 + data = empty + buffer #150: + time = 1000001416299 + data = empty + buffer #151: + time = 1000001439519 + data = empty + buffer #152: + time = 1000001462739 + data = empty + buffer #153: + time = 1000001485959 + data = empty + buffer #154: + time = 1000001509179 + data = empty + buffer #155: + time = 1000001532399 + data = empty + buffer #156: + time = 1000001555619 + data = empty + buffer #157: + time = 1000001578839 + data = empty + buffer #158: + time = 1000001602058 + data = empty + buffer #159: + time = 1000001625278 + data = empty + buffer #160: + time = 1000001648498 + data = empty + buffer #161: + time = 1000001671718 + data = empty + buffer #162: + time = 1000001694938 + data = empty + buffer #163: + time = 1000001718158 + data = empty + buffer #164: + time = 1000001741378 + data = empty + buffer #165: + time = 1000001764598 + data = empty + buffer #166: + time = 1000001787818 + data = empty + buffer #167: + time = 1000001811038 + data = empty + buffer #168: + time = 1000001834258 + data = empty + buffer #169: + time = 1000001857478 + data = empty + buffer #170: + time = 1000001880698 + data = empty + buffer #171: + time = 1000001903918 + data = empty + buffer #172: + time = 1000001927138 + data = empty + buffer #173: + time = 1000001950358 + data = empty + buffer #174: + time = 1000001973578 + data = empty + buffer #175: + time = 1000001996798 + data = empty + buffer #176: + time = 1000002020018 + data = empty + buffer #177: + time = 1000002043238 + data = empty + buffer #178: + time = 1000002066458 + data = empty + buffer #179: + time = 1000002089678 + data = empty +ImageOutput: + rendered image count = 2 + image output #1: + presentationTimeUs = 0 + bitmap hash = -1041353260 + image output #2: + presentationTimeUs = 0 + bitmap hash = -1041353260 diff --git a/libraries/test_data/src/test/assets/playbackdumps/ts/sample_scte35.ts.dump b/libraries/test_data/src/test/assets/playbackdumps/ts/sample_scte35.ts.dump index 9d13278b553..99d0f718ea5 100644 --- a/libraries/test_data/src/test/assets/playbackdumps/ts/sample_scte35.ts.dump +++ b/libraries/test_data/src/test/assets/playbackdumps/ts/sample_scte35.ts.dump @@ -86,10 +86,10 @@ AudioSink: MetadataOutput: Metadata[0]: presentationTimeUs = 33366 - entry[0] = SCTE-35 SpliceInsertCommand { programSplicePts=200000, programSplicePlaybackPositionUs= 954439399111 } + entry[0] = SCTE-35 SpliceInsertCommand { programSplicePts=200000, programSplicePlaybackPositionUs= 1000000822222 } Metadata[1]: presentationTimeUs = 33366 - entry[0] = SCTE-35 SpliceInsertCommand { programSplicePts=200000, programSplicePlaybackPositionUs= 954439399111 } + entry[0] = SCTE-35 SpliceInsertCommand { programSplicePts=200000, programSplicePlaybackPositionUs= 1000000822222 } Metadata[2]: presentationTimeUs = 33366 - entry[0] = SCTE-35 SpliceInsertCommand { programSplicePts=300000, programSplicePlaybackPositionUs= 954440510222 } + entry[0] = SCTE-35 SpliceInsertCommand { programSplicePts=300000, programSplicePlaybackPositionUs= 1000001933333 } diff --git a/libraries/test_data/src/test/assets/test-generated-goldens/CompositionPlayerPacketConsumerSurfaceViewPixelTest/compositionPlayer_withPacketConsumer_rendersFirstFrameAndReturnsBitmap.png b/libraries/test_data/src/test/assets/test-generated-goldens/CompositionPlayerFrameProcessorSurfaceViewPixelTest/compositionPlayer_withFrameProcessor_rendersFirstFrameAndReturnsBitmap.png similarity index 100% rename from libraries/test_data/src/test/assets/test-generated-goldens/CompositionPlayerPacketConsumerSurfaceViewPixelTest/compositionPlayer_withPacketConsumer_rendersFirstFrameAndReturnsBitmap.png rename to libraries/test_data/src/test/assets/test-generated-goldens/CompositionPlayerFrameProcessorSurfaceViewPixelTest/compositionPlayer_withFrameProcessor_rendersFirstFrameAndReturnsBitmap.png diff --git a/libraries/test_data/src/test/assets/test-generated-goldens/DefaultVideoFrameProcessorPixelTest/toneMap_pqRegisterInput.png b/libraries/test_data/src/test/assets/test-generated-goldens/DefaultVideoFrameProcessorPixelTest/toneMap_pqRegisterInput.png new file mode 100644 index 00000000000..9377f4626fa Binary files /dev/null and b/libraries/test_data/src/test/assets/test-generated-goldens/DefaultVideoFrameProcessorPixelTest/toneMap_pqRegisterInput.png differ diff --git a/libraries/test_data/src/test/assets/test-generated-goldens/HardwareBufferToGlTextureConverterTest/convert_withRgba8888HardwareBufferAndSdrToHdrConversion_outputsCorrectGlTexture.png b/libraries/test_data/src/test/assets/test-generated-goldens/HardwareBufferToGlTextureConverterTest/convert_withRgba8888HardwareBufferAndSdrToHdrConversion_outputsCorrectGlTexture.png new file mode 100644 index 00000000000..a8c75576b69 Binary files /dev/null and b/libraries/test_data/src/test/assets/test-generated-goldens/HardwareBufferToGlTextureConverterTest/convert_withRgba8888HardwareBufferAndSdrToHdrConversion_outputsCorrectGlTexture.png differ diff --git a/libraries/test_data/src/test/assets/test-generated-goldens/HardwareBufferToGlTextureConverterTest/convert_withYuv420Bt601HardwareBuffer_convertsToBt709GlTexture.png b/libraries/test_data/src/test/assets/test-generated-goldens/HardwareBufferToGlTextureConverterTest/convert_withYuv420Bt601HardwareBuffer_convertsToBt709GlTexture.png new file mode 100644 index 00000000000..955550f3eff Binary files /dev/null and b/libraries/test_data/src/test/assets/test-generated-goldens/HardwareBufferToGlTextureConverterTest/convert_withYuv420Bt601HardwareBuffer_convertsToBt709GlTexture.png differ diff --git a/libraries/test_data/src/test/assets/test-generated-goldens/sample_mp4_first_frame/electrical_colors/hlg_1000nit_strip_tonemap_with_gamut_compression.png b/libraries/test_data/src/test/assets/test-generated-goldens/sample_mp4_first_frame/electrical_colors/hlg_1000nit_strip_tonemap_with_gamut_compression.png new file mode 100644 index 00000000000..6833655da78 Binary files /dev/null and b/libraries/test_data/src/test/assets/test-generated-goldens/sample_mp4_first_frame/electrical_colors/hlg_1000nit_strip_tonemap_with_gamut_compression.png differ diff --git a/libraries/test_data/src/test/assets/test-generated-goldens/transformer_sequence_effect_test/export_image_samplesFromTextureCorrectly_0.png b/libraries/test_data/src/test/assets/test-generated-goldens/transformer_sequence_effect_test/export_image_samplesFromTextureCorrectly_0.png deleted file mode 100644 index 92b339b5f86..00000000000 Binary files a/libraries/test_data/src/test/assets/test-generated-goldens/transformer_sequence_effect_test/export_image_samplesFromTextureCorrectly_0.png and /dev/null differ diff --git a/libraries/test_data/src/test/assets/test-generated-goldens/transformer_sequence_effect_test/export_image_samplesFromTextureCorrectly_1.png b/libraries/test_data/src/test/assets/test-generated-goldens/transformer_sequence_effect_test/export_image_samplesFromTextureCorrectly_1.png deleted file mode 100644 index 92b339b5f86..00000000000 Binary files a/libraries/test_data/src/test/assets/test-generated-goldens/transformer_sequence_effect_test/export_image_samplesFromTextureCorrectly_1.png and /dev/null differ diff --git a/libraries/test_data/src/test/assets/test-generated-goldens/transformer_sequence_effect_test/export_withCompositionPresentationAndWithPerMediaItemEffectsLessVideo_0.png b/libraries/test_data/src/test/assets/test-generated-goldens/transformer_sequence_effect_test/export_withCompositionPresentationAndWithPerMediaItemEffectsLessVideo_0.png new file mode 100644 index 00000000000..e9a232e3dbb Binary files /dev/null and b/libraries/test_data/src/test/assets/test-generated-goldens/transformer_sequence_effect_test/export_withCompositionPresentationAndWithPerMediaItemEffectsLessVideo_0.png differ diff --git a/libraries/test_data/src/test/assets/test-generated-goldens/transformer_sequence_effect_test/export_withCompositionPresentationAndWithPerMediaItemEffectsLessVideo_1.png b/libraries/test_data/src/test/assets/test-generated-goldens/transformer_sequence_effect_test/export_withCompositionPresentationAndWithPerMediaItemEffectsLessVideo_1.png new file mode 100644 index 00000000000..2f40f32d116 Binary files /dev/null and b/libraries/test_data/src/test/assets/test-generated-goldens/transformer_sequence_effect_test/export_withCompositionPresentationAndWithPerMediaItemEffectsLessVideo_1.png differ diff --git a/libraries/test_data/src/test/assets/test-generated-goldens/transformer_sequence_effect_test/export_withCompositionPresentationAndWithPerMediaItemEffectsLessVideo_2.png b/libraries/test_data/src/test/assets/test-generated-goldens/transformer_sequence_effect_test/export_withCompositionPresentationAndWithPerMediaItemEffectsLessVideo_2.png new file mode 100644 index 00000000000..d3110a6c414 Binary files /dev/null and b/libraries/test_data/src/test/assets/test-generated-goldens/transformer_sequence_effect_test/export_withCompositionPresentationAndWithPerMediaItemEffectsLessVideo_2.png differ diff --git a/libraries/test_data/src/test/assets/transformerdumps/composition/seq-gap_1000ms+seq-sample.wav.dump b/libraries/test_data/src/test/assets/transformerdumps/composition/seq-gap_1000ms+seq-sample.wav.dump index 6d8eb58a13a..7f8d5649c5f 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/composition/seq-gap_1000ms+seq-sample.wav.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/composition/seq-gap_1000ms+seq-sample.wav.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/composition/seq-gap_100ms-sample_rf64.wav+seq-sample.wav.dump b/libraries/test_data/src/test/assets/transformerdumps/composition/seq-gap_100ms-sample_rf64.wav+seq-sample.wav.dump index 2d66aeb06fb..0166983b9b6 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/composition/seq-gap_100ms-sample_rf64.wav+seq-sample.wav.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/composition/seq-gap_100ms-sample_rf64.wav+seq-sample.wav.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sample.wav+seq-gap_1000ms.dump b/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sample.wav+seq-gap_1000ms.dump index 47829fc3cd9..abdd8ae247a 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sample.wav+seq-gap_1000ms.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sample.wav+seq-gap_1000ms.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sample.wav+seq-sample.wav_clipped100msTo400ms-gap_700ms.dump b/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sample.wav+seq-sample.wav_clipped100msTo400ms-gap_700ms.dump index 7be29fff8e5..6992b3c8277 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sample.wav+seq-sample.wav_clipped100msTo400ms-gap_700ms.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sample.wav+seq-sample.wav_clipped100msTo400ms-gap_700ms.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sample.wav+seq-sample.wav_clipped_100ms_to_400ms.dump b/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sample.wav+seq-sample.wav_clipped_100ms_to_400ms.dump index 7be29fff8e5..6992b3c8277 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sample.wav+seq-sample.wav_clipped_100ms_to_400ms.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sample.wav+seq-sample.wav_clipped_100ms_to_400ms.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sowt-with-video.mov+seq-gap_200ms-sample_rf64.wav.dump b/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sowt-with-video.mov+seq-gap_200ms-sample_rf64.wav.dump index b77bdd1fc9d..eab849411e2 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sowt-with-video.mov+seq-gap_200ms-sample_rf64.wav.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sowt-with-video.mov+seq-gap_200ms-sample_rf64.wav.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sowt-with-video.mov+seq-sample_rf64.wav-gap_100ms.dump b/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sowt-with-video.mov+seq-sample_rf64.wav-gap_100ms.dump index 9ef852b99fe..c7dd7ce3caa 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sowt-with-video.mov+seq-sample_rf64.wav-gap_100ms.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sowt-with-video.mov+seq-sample_rf64.wav-gap_100ms.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sowt-with-video.mov+seq-sample_rf64.wav_clipped0msTo300ms-gap_200ms-sample_rf64.wav_clipped300msTo800ms.dump b/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sowt-with-video.mov+seq-sample_rf64.wav_clipped0msTo300ms-gap_200ms-sample_rf64.wav_clipped300msTo800ms.dump index a0959305731..724034d87fa 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sowt-with-video.mov+seq-sample_rf64.wav_clipped0msTo300ms-gap_200ms-sample_rf64.wav_clipped300msTo800ms.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/composition/seq-sowt-with-video.mov+seq-sample_rf64.wav_clipped0msTo300ms-gap_200ms-sample_rf64.wav_clipped300msTo800ms.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/gap/500ms.dump b/libraries/test_data/src/test/assets/transformerdumps/gap/500ms.dump index 5eb3da1fb29..f30d64b1e4a 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/gap/500ms.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/gap/500ms.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/bbb_6ch_8kHz_opus.ogg/transmuxed_with_inappmuxer.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/bbb_6ch_8kHz_opus.ogg/transmuxed_with_inappmuxer.dump index e1648d1a9df..3ee427ef182 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/bbb_6ch_8kHz_opus.ogg/transmuxed_with_inappmuxer.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/bbb_6ch_8kHz_opus.ogg/transmuxed_with_inappmuxer.dump @@ -20,7 +20,7 @@ track 0: sampleRate = 48000 metadata = entries=[Mp4Timestamp: creation time=3000000000, modification time=4000000000, timescale=10000] initializationData: - data = length 27, hash D13C96B8 + data = length 27, hash 9EE6F879 data = length 8, hash CA22068C data = length 8, hash 79C07075 sample 0: diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/dolbyVision-hdr.MOV/transmuxed_with_inappmuxer.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/dolbyVision-hdr.MOV/transmuxed_with_inappmuxer.dump index 161fe773e28..1a86314c1f2 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/dolbyVision-hdr.MOV/transmuxed_with_inappmuxer.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/dolbyVision-hdr.MOV/transmuxed_with_inappmuxer.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 166700 - getPosition(0) = [[timeUs=0, position=400052]] - getPosition(1) = [[timeUs=0, position=400052]] - getPosition(83350) = [[timeUs=0, position=400052]] - getPosition(166700) = [[timeUs=0, position=400052]] + duration = 166666 + getPosition(0) = [[timeUs=0, position=400056]] + getPosition(1) = [[timeUs=0, position=400056]] + getPosition(83333) = [[timeUs=0, position=400056]] + getPosition(166666) = [[timeUs=0, position=400056]] numberOfTracks = 2 track 0: total output bytes = 56045 sample count = 5 - track duration = 166700 + track duration = 166666 format 0: averageBitrate = 2690171 id = 1 diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/h265_with_metadata_track.mp4/transmuxed_with_inappmuxer.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/h265_with_metadata_track.mp4/transmuxed_with_inappmuxer.dump index 16f9fcef48e..b248d9b64da 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/h265_with_metadata_track.mp4/transmuxed_with_inappmuxer.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/h265_with_metadata_track.mp4/transmuxed_with_inappmuxer.dump @@ -1,15 +1,15 @@ seekMap: isSeekable = true - duration = 100100 + duration = 100066 getPosition(0) = [[timeUs=0, position=400052]] getPosition(1) = [[timeUs=0, position=400052]] - getPosition(50050) = [[timeUs=0, position=400052]] - getPosition(100100) = [[timeUs=0, position=400052]] + getPosition(50033) = [[timeUs=0, position=400052]] + getPosition(100066) = [[timeUs=0, position=400052]] numberOfTracks = 2 track 0: total output bytes = 678996 sample count = 3 - track duration = 100100 + track duration = 100066 format 0: averageBitrate = 54283853 id = 1 @@ -49,7 +49,7 @@ track 0: track 1: total output bytes = 1726 sample count = 3 - track duration = 80700 + track duration = 80729 format 0: averageBitrate = 171041 peakBitrate = 192000 diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/sample.mp4/silence.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/sample.mp4/silence.dump index 93158b0d612..7798dea06d4 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/sample.mp4/silence.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/sample.mp4/silence.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/sample.mp4/silence_48000hz.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/sample.mp4/silence_48000hz.dump index 8395d05f75b..0dc072b5caa 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/sample.mp4/silence_48000hz.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/sample.mp4/silence_48000hz.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/sample_18byte_nclx_colr.mp4/silence.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/sample_18byte_nclx_colr.mp4/silence.dump index 04a9a10eb36..a0008fec769 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/sample_18byte_nclx_colr.mp4/silence.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/sample_18byte_nclx_colr.mp4/silence.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/sample_twos_pcm.mp4/toInt16.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/sample_twos_pcm.mp4/toInt16.dump index c9e92357987..600e9c8c227 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/sample_twos_pcm.mp4/toInt16.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/sample_twos_pcm.mp4/toInt16.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, xyz: latitude=40.68, longitude=-74.5, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/highPitch_silenceHighPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/highPitch_silenceHighPitch.dump index b91a06375f0..ca2c6c9d02a 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/highPitch_silenceHighPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/highPitch_silenceHighPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/mixed_sample_rf64.wav.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/mixed_sample_rf64.wav.dump index 9ef852b99fe..c7dd7ce3caa 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/mixed_sample_rf64.wav.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/mixed_sample_rf64.wav.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/original_silence.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/original_silence.dump index 6461f8e8249..842941cc96c 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/original_silence.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/original_silence.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/sequence_repeated3Times.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/sequence_repeated3Times.dump index b8831722280..7062113501d 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/sequence_repeated3Times.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/sequence_repeated3Times.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/sequence_repeated3Times_mixed_loopingAudiosowt-with-video.mov.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/sequence_repeated3Times_mixed_loopingAudiosowt-with-video.mov.dump index 67d7dae5e78..e22c72cccbd 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/sequence_repeated3Times_mixed_loopingAudiosowt-with-video.mov.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/sequence_repeated3Times_mixed_loopingAudiosowt-with-video.mov.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silence.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silence.dump index 25ea48b538f..4cbfeb2ecf1 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silence.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silence.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silenceFromEffect.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silenceFromEffect.dump index ac89f65b4dd..7f0271092e2 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silenceFromEffect.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silenceFromEffect.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silenceHighPitch_highPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silenceHighPitch_highPitch.dump index a2b2df49589..2e4bf864042 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silenceHighPitch_highPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silenceHighPitch_highPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silenceHighPitch_silence.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silenceHighPitch_silence.dump index cae0032e9e0..906f0394d63 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silenceHighPitch_silence.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silenceHighPitch_silence.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silenceHighPitch_silenceHighPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silenceHighPitch_silenceHighPitch.dump index fc5c1d4507f..c509c44a32f 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silenceHighPitch_silenceHighPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silenceHighPitch_silenceHighPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silence_original.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silence_original.dump index 79458b960ff..e24bce2dd86 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silence_original.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silence_original.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silence_silence.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silence_silence.dump index ae2247e8622..a94d2b48810 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silence_silence.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silence_silence.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silence_silenceHighPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silence_silenceHighPitch.dump index cae0032e9e0..906f0394d63 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silence_silenceHighPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/mp4/sowt-with-video.mov/silence_silenceHighPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/gap_200ms/gap_500ms/sample.wav.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/gap_200ms/gap_500ms/sample.wav.dump index b3b990b3b6d..e96d3fc995e 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/gap_200ms/gap_500ms/sample.wav.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/gap_200ms/gap_500ms/sample.wav.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/gap_500ms/sample.wav/gap_200ms/sample.wav.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/gap_500ms/sample.wav/gap_200ms/sample.wav.dump index f6431f3b888..7346693b97b 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/gap_500ms/sample.wav/gap_200ms/sample.wav.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/gap_500ms/sample.wav/gap_200ms/sample.wav.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/gap_200ms/sample.wav/gap_500ms.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/gap_200ms/sample.wav/gap_500ms.dump index 76b186e24f1..d6291b98b21 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/gap_200ms/sample.wav/gap_500ms.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/gap_200ms/sample.wav/gap_500ms.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/gap_500ms/gap_200ms.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/gap_500ms/gap_200ms.dump index 525c7738c5a..7b35a5f4665 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/gap_500ms/gap_200ms.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/gap_500ms/gap_200ms.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/gap_500ms/gap_200ms/sample.wav.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/gap_500ms/gap_200ms/sample.wav.dump index 386df4cbe80..9524389110f 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/gap_500ms/gap_200ms/sample.wav.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/gap_500ms/gap_200ms/sample.wav.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sample.wav_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sample.wav_halfPitch.dump index f11126c578d..59227c81b3d 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sample.wav_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sample.wav_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo.dump index a7da68ce2fe..9f724c617ff 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sample.wav_halfPitch/sowt-with-video.mov_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sample.wav_halfPitch/sowt-with-video.mov_removeVideo.dump index 1d4b5332db7..1fb5f418b54 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sample.wav_halfPitch/sowt-with-video.mov_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sample.wav_halfPitch/sowt-with-video.mov_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_halfPitch_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_halfPitch_removeVideo.dump index 829aed594f8..46c1627ce25 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_halfPitch_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_halfPitch_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch.dump index 964b05dc155..0a181876f74 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo.dump index 4d7a9b18cee..e4c8d8a65f3 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_removeVideo.dump index e06cc0338e6..1ad5e1216da 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_removeVideo/sample.wav_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_removeVideo/sample.wav_halfPitch.dump index bab9ed663f9..ad72f6d0478 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_removeVideo/sample.wav_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_removeVideo/sample.wav_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo.dump index e6d1054d3f4..a22b4e328b7 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch.dump index 3da97ea694d..6d9e9c78ac1 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sample.wav.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sample.wav.dump index 2c246fc0463..0ef3ec8ef67 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sample.wav.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sample.wav.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sample.wav/sowt-with-video.mov_halfPitch_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sample.wav/sowt-with-video.mov_halfPitch_removeVideo.dump index c91afa2ad5f..7de8c74c67f 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sample.wav/sowt-with-video.mov_halfPitch_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sample.wav/sowt-with-video.mov_halfPitch_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sample.wav/sowt-with-video.mov_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sample.wav/sowt-with-video.mov_removeVideo.dump index 75ec01674c1..e593623c3cc 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sample.wav/sowt-with-video.mov_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sample.wav/sowt-with-video.mov_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo.dump index 9be71002e0f..1be3aed9fb7 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo/sample.wav.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo/sample.wav.dump index 38afe9a7915..4d15f6a9421 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo/sample.wav.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo/sample.wav.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo.dump index c31ff9d80b4..96e7623df8c 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_removeVideo.dump index f6acc947e47..3fb7a48dd4b 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_removeVideo/sample.wav.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_removeVideo/sample.wav.dump index 2fc8690adcb..681bb2b7fb2 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_removeVideo/sample.wav.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_removeVideo/sample.wav.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo.dump index feaf100387a..87d319bfc3d 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sample.wav_halfPitch/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_halfPitch.dump index ebd499d3a88..cc4599e53e8 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence.dump index 0b6ac02efd4..e3a08270c01 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch.dump index 0b6ac02efd4..e3a08270c01 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence.dump index 6461f8e8249..842941cc96c 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch.dump index 821634ede0f..51042ea1484 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch.dump index a843a769ba7..fd1564786fe 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch.dump index 6461f8e8249..842941cc96c 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch.dump index 821634ede0f..51042ea1484 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence.dump index a843a769ba7..fd1564786fe 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch.dump index 7afc1131047..4ad8a3278e0 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov.dump index 54b8642cea6..aa2493253c7 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov/sowt-with-video.mov_silence.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov/sowt-with-video.mov_silence.dump index 25bcf2567f9..0ec0a95b018 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov/sowt-with-video.mov_silence.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov/sowt-with-video.mov_silence.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch.dump index 25bcf2567f9..0ec0a95b018 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence.dump index a59f87ac636..cc363d3cee0 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence/sowt-with-video.mov.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence/sowt-with-video.mov.dump index 46c18432e68..09f0eaebdf8 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence/sowt-with-video.mov.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence/sowt-with-video.mov.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch.dump index 60fa9a3d6b3..e6c2cd3d5b6 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch.dump index a59f87ac636..cc363d3cee0 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov.dump index 46c18432e68..09f0eaebdf8 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence.dump index 60fa9a3d6b3..e6c2cd3d5b6 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo.dump index cea44009d85..6461fd3aa25 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav.dump index 17d16c61405..9113bcec8e8 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav/sample.wav_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav/sample.wav_halfPitch.dump index 5e5528d2673..bd286c3a9b1 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav/sample.wav_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav/sample.wav_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav/sowt-with-video.mov_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav/sowt-with-video.mov_removeVideo.dump index 4429e91c31d..086be135af3 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav/sowt-with-video.mov_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav/sowt-with-video.mov_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch.dump index 4d75e952fc8..8a964cf990a 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch/sample.wav.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch/sample.wav.dump index a720b611fa1..a1c6edf7f41 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch/sample.wav.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch/sample.wav.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch/sowt-with-video.mov_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch/sowt-with-video.mov_removeVideo.dump index 3a1a8495908..436ba06eb43 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch/sowt-with-video.mov_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch/sowt-with-video.mov_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo.dump index ce6b9680640..5e110021037 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo/sample.wav.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo/sample.wav.dump index 9f0558cffc5..4eb9c4f11ea 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo/sample.wav.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo/sample.wav.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo/sample.wav_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo/sample.wav_halfPitch.dump index defb1e7e3ca..fe4b6bca0c6 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo/sample.wav_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_halfPitch_removeVideo/sowt-with-video.mov_removeVideo/sample.wav_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav.dump index c4ecc897871..54e45195e99 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav/sample.wav_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav/sample.wav_halfPitch.dump index 7a0acce398a..a4cfa4a6309 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav/sample.wav_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav/sample.wav_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav/sowt-with-video.mov_halfPitch_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav/sowt-with-video.mov_halfPitch_removeVideo.dump index 4892d0c582c..7d7457cdded 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav/sowt-with-video.mov_halfPitch_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav/sowt-with-video.mov_halfPitch_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav_halfPitch.dump index 8cb767cb1bb..2ccb3e45a45 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav_halfPitch/sample.wav.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav_halfPitch/sample.wav.dump index 9b076722279..72597e10360 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav_halfPitch/sample.wav.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav_halfPitch/sample.wav.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo.dump index a56557283df..bc3c387502b 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sample.wav_halfPitch/sowt-with-video.mov_halfPitch_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo.dump index 1f8cc183bea..876969fe153 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo/sample.wav.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo/sample.wav.dump index 100dedf206d..3ceaac4e29a 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo/sample.wav.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo/sample.wav.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch.dump index 23612d43491..e82957e82bb 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_removeVideo/sowt-with-video.mov_halfPitch_removeVideo/sample.wav_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 metadata = entries=[Mp4AlternateGroup: 1, Mp4Timestamp: creation time=0, modification time=0, timescale=1000] diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence.dump index 25ea48b538f..4cbfeb2ecf1 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov.dump index 79458b960ff..e24bce2dd86 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov/sowt-with-video.mov_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov/sowt-with-video.mov_halfPitch.dump index 15d131d388c..78349a00f08 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov/sowt-with-video.mov_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov/sowt-with-video.mov_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch.dump index 347f0654a1b..6c2c6155ed5 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov/sowt-with-video.mov_silence_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch.dump index e39e64670ff..523ba4b83e1 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch/sowt-with-video.mov.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch/sowt-with-video.mov.dump index 62efd1d3630..9a437bcece9 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch/sowt-with-video.mov.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch/sowt-with-video.mov.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch.dump index 5c473378cd5..6fc8879f5ae 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch.dump index ae2247e8622..a94d2b48810 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov.dump index 6ac1ebcbc16..efc1de816ed 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch.dump index 433feb9b961..9f689da8039 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch.dump index 25ea48b538f..4cbfeb2ecf1 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov.dump index 79458b960ff..e24bce2dd86 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov/sowt-with-video.mov_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov/sowt-with-video.mov_halfPitch.dump index 15d131d388c..78349a00f08 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov/sowt-with-video.mov_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov/sowt-with-video.mov_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov/sowt-with-video.mov_silence.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov/sowt-with-video.mov_silence.dump index 347f0654a1b..6c2c6155ed5 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov/sowt-with-video.mov_silence.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov/sowt-with-video.mov_silence.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch.dump index e39e64670ff..523ba4b83e1 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch/sowt-with-video.mov.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch/sowt-with-video.mov.dump index 62efd1d3630..9a437bcece9 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch/sowt-with-video.mov.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch/sowt-with-video.mov.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence.dump index 5c473378cd5..6fc8879f5ae 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_halfPitch/sowt-with-video.mov_silence.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence.dump index ae2247e8622..a94d2b48810 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence/sowt-with-video.mov.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence/sowt-with-video.mov.dump index 6ac1ebcbc16..efc1de816ed 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence/sowt-with-video.mov.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence/sowt-with-video.mov.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch.dump index 433feb9b961..9f689da8039 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/sequence/sowt-with-video.mov_silence_halfPitch/sowt-with-video.mov_silence/sowt-with-video.mov_halfPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 44100 pcmEncoding = pcm-16 format video: diff --git a/libraries/test_data/src/test/assets/transformerdumps/ts/sample_h264.ts/transmuxed_with_inappmuxer.dump b/libraries/test_data/src/test/assets/transformerdumps/ts/sample_h264.ts/transmuxed_with_inappmuxer.dump index d15ef9275a2..ebb77098f79 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/ts/sample_h264.ts/transmuxed_with_inappmuxer.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/ts/sample_h264.ts/transmuxed_with_inappmuxer.dump @@ -45,7 +45,7 @@ track 0: data = length 6849, hash AD411A86 sample 3: time = 66666 - flags = 0 + flags = 67108864 data = length 794, hash 6289130F sample 4: time = 166666 @@ -53,7 +53,7 @@ track 0: data = length 1874, hash 8A60ABD5 sample 5: time = 133333 - flags = 0 + flags = 67108864 data = length 1209, hash 9947502F sample 6: time = 300000 @@ -65,11 +65,11 @@ track 0: data = length 325, hash E39A441C sample 8: time = 200000 - flags = 0 + flags = 67108864 data = length 89, hash 14CB8CBE sample 9: time = 266666 - flags = 0 + flags = 67108864 data = length 177, hash 2E796387 sample 10: time = 333333 @@ -85,11 +85,11 @@ track 0: data = length 302, hash 8F0A6A12 sample 13: time = 366666 - flags = 0 + flags = 67108864 data = length 96, hash 32B6396A sample 14: time = 433333 - flags = 0 + flags = 67108864 data = length 198, hash 9D64D0A9 sample 15: time = 600000 @@ -101,11 +101,11 @@ track 0: data = length 168, hash D1EBAB18 sample 17: time = 500000 - flags = 0 + flags = 67108864 data = length 98, hash 69F34C6E sample 18: time = 566666 - flags = 0 + flags = 67108864 data = length 108, hash ECDDA32A sample 19: time = 633333 @@ -125,11 +125,11 @@ track 0: data = length 336, hash F0603229 sample 23: time = 700000 - flags = 0 + flags = 67108864 data = length 105, hash EF20F2CC sample 24: time = 766666 - flags = 0 + flags = 67108864 data = length 106, hash C9702210 sample 25: time = 933333 @@ -141,11 +141,11 @@ track 0: data = length 109, hash 77764B9F sample 27: time = 833333 - flags = 0 + flags = 67108864 data = length 84, hash 74F4BEE sample 28: time = 900000 - flags = 0 + flags = 67108864 data = length 93, hash 9522F962 sample 29: time = 966666 diff --git a/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/48000hz.dump b/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/48000hz.dump index 5f43a048c80..6e13e5bc153 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/48000hz.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/48000hz.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 48000 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/doubleSpeed.dump b/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/doubleSpeed.dump index e2887ab7b25..d8cdc9ce820 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/doubleSpeed.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/doubleSpeed.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/highPitch_highPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/highPitch_highPitch.dump index eff8163a60b..7839ac518d1 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/highPitch_highPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/highPitch_highPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/highPitch_lowPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/highPitch_lowPitch.dump index 7cc8e49a4f9..1ab733853ea 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/highPitch_lowPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/highPitch_lowPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/mixed_sample.wav.dump b/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/mixed_sample.wav.dump index 404e86edc36..73a05e189de 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/mixed_sample.wav.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/mixed_sample.wav.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/mixed_sample.wav_48000hz.dump b/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/mixed_sample.wav_48000hz.dump index 09240029e09..16b5977fcb8 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/mixed_sample.wav_48000hz.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/mixed_sample.wav_48000hz.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 48000 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/original_original.dump b/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/original_original.dump index 94240266402..a9e24bc114b 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/original_original.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/original_original.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/silenceFromEffect.dump b/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/silenceFromEffect.dump index 8e60445bebb..0f7a916feaf 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/silenceFromEffect.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/wav/sample.wav/silenceFromEffect.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/wav/sample_ima_adpcm.wav/silenceFromEffect.dump b/libraries/test_data/src/test/assets/transformerdumps/wav/sample_ima_adpcm.wav/silenceFromEffect.dump index a9ce68e6712..701e1ef2c21 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/wav/sample_ima_adpcm.wav/silenceFromEffect.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/wav/sample_ima_adpcm.wav/silenceFromEffect.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 1 - channelMask = 0x00000004 sampleRate = 44100 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/highPitch_sample.wavHighPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/highPitch_sample.wavHighPitch.dump index bf697591eb8..e6464201ea9 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/highPitch_sample.wavHighPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/highPitch_sample.wavHighPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/highPitch_sample.wavLowPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/highPitch_sample.wavLowPitch.dump index d24e145d260..d34bbde601e 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/highPitch_sample.wavLowPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/highPitch_sample.wavLowPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/original_sample.wav.dump b/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/original_sample.wav.dump index 89da2dabfe4..4bd181e3c98 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/original_sample.wav.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/original_sample.wav.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/original_sample.wav_highPitch.dump b/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/original_sample.wav_highPitch.dump index 015072aa000..d7a80a403a8 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/original_sample.wav_highPitch.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/original_sample.wav_highPitch.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/silenceFromEffect.dump b/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/silenceFromEffect.dump index c42b6db44df..5920d802d51 100644 --- a/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/silenceFromEffect.dump +++ b/libraries/test_data/src/test/assets/transformerdumps/wav/sample_rf64.wav/silenceFromEffect.dump @@ -1,8 +1,6 @@ format audio: - averageBitrate = 131072 sampleMimeType = audio/raw channelCount = 2 - channelMask = 0x0000000C sampleRate = 48000 pcmEncoding = pcm-16 sample: diff --git a/libraries/test_exoplayer_playback/src/androidTest/java/androidx/media3/test/exoplayer/playback/gts/DashStreamingTest.java b/libraries/test_exoplayer_playback/src/androidTest/java/androidx/media3/test/exoplayer/playback/gts/DashStreamingTest.java index 99b24ef346a..f4728de6d36 100644 --- a/libraries/test_exoplayer_playback/src/androidTest/java/androidx/media3/test/exoplayer/playback/gts/DashStreamingTest.java +++ b/libraries/test_exoplayer_playback/src/androidTest/java/androidx/media3/test/exoplayer/playback/gts/DashStreamingTest.java @@ -30,7 +30,6 @@ import androidx.media3.test.utils.ActionSchedule; import androidx.media3.test.utils.HostActivity; import androidx.test.ext.junit.runners.AndroidJUnit4; -import androidx.test.filters.SdkSuppress; import androidx.test.rule.ActivityTestRule; import org.junit.After; import org.junit.Before; @@ -109,6 +108,18 @@ public final class DashStreamingTest { .clearVideoSurface() .delay(10000) .setVideoSurface() + // Wait 10 seconds, then detach and attach the surface 5 times in quick succession. + .delay(10000) + .clearVideoSurface() + .setVideoSurface() + .clearVideoSurface() + .setVideoSurface() + .clearVideoSurface() + .setVideoSurface() + .clearVideoSurface() + .setVideoSurface() + .clearVideoSurface() + .setVideoSurface() // Wait 10 seconds, then seek to near end. .delay(10000) .seek(120000) @@ -215,7 +226,7 @@ public void h265FixedV23() throws Exception { @Test public void h265AdaptiveV24() throws Exception { - if (SDK_INT < 24 || isPc()) { + if (isPc()) { // Pass. return; } @@ -231,7 +242,7 @@ public void h265AdaptiveV24() throws Exception { @Test public void h265AdaptiveWithSeekingV24() throws Exception { - if (SDK_INT < 24 || isPc()) { + if (isPc()) { // Pass. return; } @@ -248,7 +259,7 @@ public void h265AdaptiveWithSeekingV24() throws Exception { @Test public void h265AdaptiveWithRendererDisablingV24() throws Exception { - if (SDK_INT < 24 || isPc()) { + if (isPc()) { // Pass. return; } @@ -279,10 +290,6 @@ public void vp9Fixed360pV23() throws Exception { @Test public void vp9AdaptiveV24() throws Exception { - if (SDK_INT < 24) { - // Pass. - return; - } testRunner .setStreamName("test_vp9_adaptive") .setManifestUrl(DashTestData.VP9_MANIFEST) @@ -295,10 +302,6 @@ public void vp9AdaptiveV24() throws Exception { @Test public void vp9AdaptiveWithSeekingV24() throws Exception { - if (SDK_INT < 24) { - // Pass. - return; - } testRunner .setStreamName("test_vp9_adaptive_with_seeking") .setManifestUrl(DashTestData.VP9_MANIFEST) @@ -312,10 +315,6 @@ public void vp9AdaptiveWithSeekingV24() throws Exception { @Test public void vp9AdaptiveWithRendererDisablingV24() throws Exception { - if (SDK_INT < 24) { - // Pass. - return; - } testRunner .setStreamName("test_vp9_adaptive_with_renderer_disabling") .setManifestUrl(DashTestData.VP9_MANIFEST) @@ -461,7 +460,6 @@ public void widevineH265FixedV23() throws Exception { } @Test - @SdkSuppress(minSdkVersion = 24) public void widevineH265AdaptiveV24() throws Exception { assumeFalse(shouldSkipWidevineTest(testRule.getActivity())); assumeFalse(isPc()); @@ -479,7 +477,6 @@ public void widevineH265AdaptiveV24() throws Exception { } @Test - @SdkSuppress(minSdkVersion = 24) public void widevineH265AdaptiveWithSeekingV24() throws Exception { assumeFalse(shouldSkipWidevineTest(testRule.getActivity())); assumeFalse(isPc()); @@ -498,7 +495,6 @@ public void widevineH265AdaptiveWithSeekingV24() throws Exception { } @Test - @SdkSuppress(minSdkVersion = 24) public void widevineH265AdaptiveWithRendererDisablingV24() throws Exception { assumeFalse(shouldSkipWidevineTest(testRule.getActivity())); assumeFalse(isPc()); @@ -535,7 +531,6 @@ public void widevineVp9Fixed360pV23() throws Exception { } @Test - @SdkSuppress(minSdkVersion = 24) public void widevineVp9AdaptiveV24() throws Exception { assumeFalse(shouldSkipWidevineTest(testRule.getActivity())); @@ -552,7 +547,6 @@ public void widevineVp9AdaptiveV24() throws Exception { } @Test - @SdkSuppress(minSdkVersion = 24) public void widevineVp9AdaptiveWithSeekingV24() throws Exception { assumeFalse(shouldSkipWidevineTest(testRule.getActivity())); @@ -570,7 +564,6 @@ public void widevineVp9AdaptiveWithSeekingV24() throws Exception { } @Test - @SdkSuppress(minSdkVersion = 24) public void widevineVp9AdaptiveWithRendererDisablingV24() throws Exception { assumeFalse(shouldSkipWidevineTest(testRule.getActivity())); @@ -652,7 +645,6 @@ public void decoderInfoH264() throws Exception { } @Test - @SdkSuppress(minSdkVersion = 24) public void decoderInfoH265V24() throws Exception { assumeFalse(isPc()); @@ -664,7 +656,6 @@ public void decoderInfoH265V24() throws Exception { } @Test - @SdkSuppress(minSdkVersion = 24) public void decoderInfoVP9V24() throws Exception { assertThat( diff --git a/libraries/test_exoplayer_playback/src/androidTest/java/androidx/media3/test/exoplayer/playback/gts/DashTestData.java b/libraries/test_exoplayer_playback/src/androidTest/java/androidx/media3/test/exoplayer/playback/gts/DashTestData.java index 77ccb54f8ca..6ca08e965ed 100644 --- a/libraries/test_exoplayer_playback/src/androidTest/java/androidx/media3/test/exoplayer/playback/gts/DashTestData.java +++ b/libraries/test_exoplayer_playback/src/androidTest/java/androidx/media3/test/exoplayer/playback/gts/DashTestData.java @@ -15,8 +15,6 @@ */ package androidx.media3.test.exoplayer.playback.gts; -import static android.os.Build.VERSION.SDK_INT; - /** Test data for DASH tests. */ /* package */ final class DashTestData { @@ -54,20 +52,14 @@ public static final String H264_MAIN_480P_VIDEO_REPRESENTATION_ID = "avc-main-480"; // The highest quality H264 format mandated by the Android CDD. public static final String H264_CDD_FIXED = H264_MAIN_480P_VIDEO_REPRESENTATION_ID; - // Multiple H264 formats mandated by the Android CDD. Note: The CDD actually mandated main profile - // support from API level 23, but we opt to test only from 24 due to known issues on API level 23 - // when switching between baseline and main profiles on certain devices. + // Multiple H264 formats mandated by the Android CDD. public static final String[] H264_CDD_ADAPTIVE = - SDK_INT < 24 - ? new String[] { - H264_BASELINE_240P_VIDEO_REPRESENTATION_ID, H264_BASELINE_480P_VIDEO_REPRESENTATION_ID - } - : new String[] { - H264_BASELINE_240P_VIDEO_REPRESENTATION_ID, - H264_BASELINE_480P_VIDEO_REPRESENTATION_ID, - H264_MAIN_240P_VIDEO_REPRESENTATION_ID, - H264_MAIN_480P_VIDEO_REPRESENTATION_ID - }; + new String[] { + H264_BASELINE_240P_VIDEO_REPRESENTATION_ID, + H264_BASELINE_480P_VIDEO_REPRESENTATION_ID, + H264_MAIN_240P_VIDEO_REPRESENTATION_ID, + H264_MAIN_480P_VIDEO_REPRESENTATION_ID + }; public static final String H264_BASELINE_480P_23FPS_VIDEO_REPRESENTATION_ID = "avc-baseline-480-23"; @@ -104,21 +96,14 @@ // The highest quality H264 format mandated by the Android CDD. public static final String WIDEVINE_H264_CDD_FIXED = WIDEVINE_H264_MAIN_480P_VIDEO_REPRESENTATION_ID; - // Multiple H264 formats mandated by the Android CDD. Note: The CDD actually mandated main profile - // support from API level 23, but we opt to test only from 24 due to known issues on API level 23 - // when switching between baseline and main profiles on certain devices. + // Multiple H264 formats mandated by the Android CDD. public static final String[] WIDEVINE_H264_CDD_ADAPTIVE = - SDK_INT < 24 - ? new String[] { - WIDEVINE_H264_BASELINE_240P_VIDEO_REPRESENTATION_ID, - WIDEVINE_H264_BASELINE_480P_VIDEO_REPRESENTATION_ID - } - : new String[] { - WIDEVINE_H264_BASELINE_240P_VIDEO_REPRESENTATION_ID, - WIDEVINE_H264_BASELINE_480P_VIDEO_REPRESENTATION_ID, - WIDEVINE_H264_MAIN_240P_VIDEO_REPRESENTATION_ID, - WIDEVINE_H264_MAIN_480P_VIDEO_REPRESENTATION_ID - }; + new String[] { + WIDEVINE_H264_BASELINE_240P_VIDEO_REPRESENTATION_ID, + WIDEVINE_H264_BASELINE_480P_VIDEO_REPRESENTATION_ID, + WIDEVINE_H264_MAIN_240P_VIDEO_REPRESENTATION_ID, + WIDEVINE_H264_MAIN_480P_VIDEO_REPRESENTATION_ID + }; public static final String WIDEVINE_H264_BASELINE_480P_23FPS_VIDEO_REPRESENTATION_ID = "3"; public static final String WIDEVINE_H264_BASELINE_480P_24FPS_VIDEO_REPRESENTATION_ID = "3"; diff --git a/libraries/test_exoplayer_playback/src/androidTest/java/androidx/media3/test/exoplayer/playback/gts/DashTestRunner.java b/libraries/test_exoplayer_playback/src/androidTest/java/androidx/media3/test/exoplayer/playback/gts/DashTestRunner.java index 4b69012f773..aa809c40160 100644 --- a/libraries/test_exoplayer_playback/src/androidTest/java/androidx/media3/test/exoplayer/playback/gts/DashTestRunner.java +++ b/libraries/test_exoplayer_playback/src/androidTest/java/androidx/media3/test/exoplayer/playback/gts/DashTestRunner.java @@ -15,19 +15,24 @@ */ package androidx.media3.test.exoplayer.playback.gts; -import static android.os.Build.VERSION.SDK_INT; import static androidx.media3.common.C.WIDEVINE_UUID; import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkState; +import android.content.Context; import android.media.MediaDrm; import android.media.UnsupportedSchemeException; +import android.os.Build; +import android.view.Display; import android.view.Surface; +import android.view.WindowManager; import android.widget.FrameLayout; +import androidx.annotation.Nullable; import androidx.annotation.Size; import androidx.media3.common.C; import androidx.media3.common.Format; import androidx.media3.common.MediaItem; +import androidx.media3.common.MediaLibraryInfo; import androidx.media3.common.TrackGroup; import androidx.media3.common.util.Log; import androidx.media3.common.util.NullableType; @@ -35,8 +40,10 @@ import androidx.media3.datasource.DefaultDataSource; import androidx.media3.datasource.DefaultHttpDataSource; import androidx.media3.exoplayer.DecoderCounters; +import androidx.media3.exoplayer.DecoderReuseEvaluation; import androidx.media3.exoplayer.ExoPlayer; import androidx.media3.exoplayer.RendererCapabilities; +import androidx.media3.exoplayer.analytics.AnalyticsListener; import androidx.media3.exoplayer.dash.DashMediaSource; import androidx.media3.exoplayer.drm.DefaultDrmSessionManager; import androidx.media3.exoplayer.drm.DrmSessionManager; @@ -73,10 +80,6 @@ private static final long TEST_TIMEOUT_MS = 5 * 60 * 1000; - // Whether adaptive tests should enable video formats beyond those mandated by the Android CDD - // if the device advertises support for them. - private static final boolean ALLOW_ADDITIONAL_VIDEO_FORMATS = SDK_INT >= 24; - private static final String AUDIO_TAG_SUFFIX = ":Audio"; private static final String VIDEO_TAG_SUFFIX = ":Video"; @@ -146,8 +149,7 @@ public DashTestRunner setFullPlaybackNoSeeking(boolean fullPlaybackNoSeeking) { @CanIgnoreReturnValue public DashTestRunner setCanIncludeAdditionalVideoFormats( boolean canIncludeAdditionalVideoFormats) { - this.canIncludeAdditionalVideoFormats = - canIncludeAdditionalVideoFormats && ALLOW_ADDITIONAL_VIDEO_FORMATS; + this.canIncludeAdditionalVideoFormats = canIncludeAdditionalVideoFormats; return this; } @@ -225,6 +227,12 @@ private DashHostedTest createDashHostedTest( /** A {@link HostedTest} for DASH playback tests. */ private static final class DashHostedTest extends ExoHostedTest { + static { + // Disable all device and codec-specific workarounds, since we are interested in testing + // that the underlying platform is behaving correctly. + MediaLibraryInfo.setEnableWorkarounds(false); + } + private final String streamName; private final String manifestUrl; private final MetricsLogger metricsLogger; @@ -237,6 +245,8 @@ private static final class DashHostedTest extends ExoHostedTest { private final DataSource.Factory dataSourceFactory; private boolean needsCddLimitedRetry; + private float displayRefreshRate = Format.NO_VALUE; + private float contentFrameRate = Format.NO_VALUE; /** * @param tag A tag to use for logging. @@ -328,10 +338,23 @@ protected DrmSessionManager buildDrmSessionManager() { @Override protected ExoPlayer buildExoPlayer( HostActivity host, Surface surface, MappingTrackSelector trackSelector) { + displayRefreshRate = getMaxDisplayRefreshRate(host); ExoPlayer player = new ExoPlayer.Builder(host, new DebugRenderersFactory(host)) .setTrackSelector(trackSelector) .build(); + player.addAnalyticsListener( + new AnalyticsListener() { + @Override + public void onVideoInputFormatChanged( + EventTime eventTime, + Format format, + @Nullable DecoderReuseEvaluation decoderReuseEvaluation) { + if (format != null && format.frameRate != Format.NO_VALUE) { + contentFrameRate = format.frameRate; + } + } + }); player.setVideoSurface(surface); return player; } @@ -371,8 +394,18 @@ protected void assertPassed(DecoderCounters audioCounters, DecoderCounters video // We shouldn't have skipped any output buffers. DecoderCountersUtil.assertSkippedOutputBufferCount( tag + AUDIO_TAG_SUFFIX, audioCounters, 0); + int expectedSkips = 0; + int tolerance = 0; + if (displayRefreshRate > 0 + && contentFrameRate != Format.NO_VALUE + && contentFrameRate > displayRefreshRate) { + double expectedSkipFraction = 1.0 - (double) displayRefreshRate / contentFrameRate; + int totalBuffers = DecoderCountersUtil.getTotalBufferCount(videoCounters); + expectedSkips = (int) (totalBuffers * expectedSkipFraction); + tolerance = Math.max(5, (int) (totalBuffers * 0.1f)); + } DecoderCountersUtil.assertSkippedOutputBufferCount( - tag + VIDEO_TAG_SUFFIX, videoCounters, 0); + tag + VIDEO_TAG_SUFFIX, videoCounters, expectedSkips, tolerance); DecoderCountersUtil.assertTotalBufferCount(tag + AUDIO_TAG_SUFFIX, audioCounters); DecoderCountersUtil.assertTotalBufferCount(tag + VIDEO_TAG_SUFFIX, videoCounters); } @@ -401,6 +434,29 @@ protected void assertPassed(DecoderCounters audioCounters, DecoderCounters video } } } + + private static float getMaxDisplayRefreshRate(@Nullable Context context) { + if (context == null) { + return Format.NO_VALUE; + } + WindowManager windowManager = + (WindowManager) context.getSystemService(Context.WINDOW_SERVICE); + if (windowManager == null || windowManager.getDefaultDisplay() == null) { + return Format.NO_VALUE; + } + Display display = windowManager.getDefaultDisplay(); + float maxRefreshRate = display.getMode().getRefreshRate(); + if (Build.VERSION.SDK_INT >= 31) { + for (float alternativeRefreshRate : display.getMode().getAlternativeRefreshRates()) { + maxRefreshRate = Math.max(maxRefreshRate, alternativeRefreshRate); + } + } else { + for (Display.Mode mode : display.getSupportedModes()) { + maxRefreshRate = Math.max(maxRefreshRate, mode.getRefreshRate()); + } + } + return maxRefreshRate; + } } /** Provides a hook to skip dropped output buffer assertions in specific circumstances. */ diff --git a/libraries/test_exoplayer_playback/src/androidTest/java/androidx/media3/test/exoplayer/playback/gts/DebugRenderersFactory.java b/libraries/test_exoplayer_playback/src/androidTest/java/androidx/media3/test/exoplayer/playback/gts/DebugRenderersFactory.java index 94952f8afac..863b53b8083 100644 --- a/libraries/test_exoplayer_playback/src/androidTest/java/androidx/media3/test/exoplayer/playback/gts/DebugRenderersFactory.java +++ b/libraries/test_exoplayer_playback/src/androidTest/java/androidx/media3/test/exoplayer/playback/gts/DebugRenderersFactory.java @@ -161,13 +161,6 @@ protected void onProcessedOutputBuffer(long presentationTimeUs) { } } - // TODO: b/321230611 - Remove this override when 'late' buffers that result in identical release - // timestamps are reported as 'dropped' instead of 'skipped'. - @Override - protected boolean shouldSkipBuffersWithIdenticalReleaseTime() { - return false; - } - private void clearTimestamps() { startIndex = 0; queueSize = 0; diff --git a/libraries/test_proguard/build.gradle.kts b/libraries/test_proguard/build.gradle.kts index b4024ac4e7a..7b36aaa8e66 100644 --- a/libraries/test_proguard/build.gradle.kts +++ b/libraries/test_proguard/build.gradle.kts @@ -36,6 +36,7 @@ android { dependencies { coreLibraryDesugaring(libs.desugar.jdk.libs) + implementation(project(":lib-effect")) implementation(project(":lib-exoplayer")) implementation(project(":lib-exoplayer-dash")) implementation(project(":lib-exoplayer-hls")) @@ -43,6 +44,7 @@ dependencies { implementation(project(":lib-exoplayer-smoothstreaming")) implementation(project(":lib-transformer")) implementation(project(":lib-ui")) + implementation(project(":lib-ui-compose-material3")) implementation(project(":lib-datasource-rtmp")) implementation(project(":lib-decoder-vp9")) implementation(project(":lib-decoder-opus")) diff --git a/libraries/test_proguard/src/androidTest/java/androidx/media3/test/proguard/EffectModuleProguardTest.java b/libraries/test_proguard/src/androidTest/java/androidx/media3/test/proguard/EffectModuleProguardTest.java new file mode 100644 index 00000000000..37008c8a70e --- /dev/null +++ b/libraries/test_proguard/src/androidTest/java/androidx/media3/test/proguard/EffectModuleProguardTest.java @@ -0,0 +1,32 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.test.proguard; + +import androidx.test.ext.junit.runners.AndroidJUnit4; +import androidx.test.filters.SdkSuppress; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit test executing methods in {@link EffectModuleProguard}. */ +@SdkSuppress(minSdkVersion = 26) +@RunWith(AndroidJUnit4.class) +public final class EffectModuleProguardTest { + + @Test + public void bindEglImage_succeeds() { + EffectModuleProguard.bindEglImage(); + } +} diff --git a/libraries/test_proguard/src/androidTest/java/androidx/media3/test/proguard/UiComposeMaterial3ModuleProguardTest.kt b/libraries/test_proguard/src/androidTest/java/androidx/media3/test/proguard/UiComposeMaterial3ModuleProguardTest.kt new file mode 100644 index 00000000000..800707aa3b0 --- /dev/null +++ b/libraries/test_proguard/src/androidTest/java/androidx/media3/test/proguard/UiComposeMaterial3ModuleProguardTest.kt @@ -0,0 +1,32 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.test.proguard + +import androidx.test.core.app.ApplicationProvider.getApplicationContext +import androidx.test.ext.junit.runners.AndroidJUnit4 +import androidx.test.platform.app.InstrumentationRegistry.getInstrumentation +import org.junit.Test +import org.junit.runner.RunWith + +@RunWith(AndroidJUnit4::class) +class UiComposeMaterial3ModuleProguardTest { + + @Test + fun scrubbingModeUtil_succeeds() { + getInstrumentation().runOnMainSync { testScrubbingModeReflection(getApplicationContext()) } + } +} diff --git a/libraries/test_proguard/src/main/java/androidx/media3/test/proguard/Av1ModuleProguard.java b/libraries/test_proguard/src/main/java/androidx/media3/test/proguard/Av1ModuleProguard.java index 65622f56bd0..a65e09d41c9 100644 --- a/libraries/test_proguard/src/main/java/androidx/media3/test/proguard/Av1ModuleProguard.java +++ b/libraries/test_proguard/src/main/java/androidx/media3/test/proguard/Av1ModuleProguard.java @@ -18,6 +18,7 @@ import static androidx.annotation.VisibleForTesting.NONE; import androidx.annotation.VisibleForTesting; +import androidx.media3.common.C; import androidx.media3.decoder.av1.Dav1dDecoder; /** Class exercising native code in the AV1 module that relies on a correct proguard config. */ @@ -38,7 +39,8 @@ public static void createDav1dDecoder() throws Exception { /* initialInputBufferSize= */ 1024, /* threads= */ 1, /* maxFrameDelay= */ 2, - /* useCustomAllocator= */ false); + /* useCustomAllocator= */ false, + C.THREAD_PRIORITY_NO_OP); decoder.release(); } } diff --git a/libraries/test_proguard/src/main/java/androidx/media3/test/proguard/EffectModuleProguard.java b/libraries/test_proguard/src/main/java/androidx/media3/test/proguard/EffectModuleProguard.java new file mode 100644 index 00000000000..4766ca31a29 --- /dev/null +++ b/libraries/test_proguard/src/main/java/androidx/media3/test/proguard/EffectModuleProguard.java @@ -0,0 +1,36 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.test.proguard; + +import static androidx.annotation.VisibleForTesting.NONE; + +import androidx.annotation.RequiresApi; +import androidx.annotation.VisibleForTesting; +import androidx.media3.effect.HardwareBufferJni; + +/** Class exercising native code in the Effect module that relies on a correct proguard config. */ +@RequiresApi(26) +@VisibleForTesting(otherwise = NONE) +public final class EffectModuleProguard { + + private EffectModuleProguard() {} + + /** Exercises {@link HardwareBufferJni} that relies on unobfuscated native method names. */ + public static void bindEglImage() { + boolean unused = + HardwareBufferJni.INSTANCE.nativeBindEGLImage(/* target= */ 0, /* eglImageHandle= */ 0); + } +} diff --git a/libraries/test_proguard/src/main/java/androidx/media3/test/proguard/ExoPlayerModuleProguard.java b/libraries/test_proguard/src/main/java/androidx/media3/test/proguard/ExoPlayerModuleProguard.java index b6b8f1c042d..8fd328c4201 100644 --- a/libraries/test_proguard/src/main/java/androidx/media3/test/proguard/ExoPlayerModuleProguard.java +++ b/libraries/test_proguard/src/main/java/androidx/media3/test/proguard/ExoPlayerModuleProguard.java @@ -253,7 +253,11 @@ public boolean shouldIgnoreFrame( } }; VideoFrameReleaseControl videoFrameReleaseControl = - new VideoFrameReleaseControl(context, frameTimingEvaluator, /* allowedJoiningTimeMs= */ 0); + new VideoFrameReleaseControl( + context, + frameTimingEvaluator, + /* allowedJoiningTimeMs= */ 0, + /* skipBuffersWithIdenticalReleaseTime= */ true); PlaybackVideoGraphWrapper playbackVideoGraphWrapper = new PlaybackVideoGraphWrapper.Builder(context, videoFrameReleaseControl).build(); playbackVideoGraphWrapper.getSink(/* inputIndex= */ 0).initialize(new Format.Builder().build()); diff --git a/libraries/test_proguard/src/main/java/androidx/media3/test/proguard/UiComposeMaterial3ModuleProguard.kt b/libraries/test_proguard/src/main/java/androidx/media3/test/proguard/UiComposeMaterial3ModuleProguard.kt new file mode 100644 index 00000000000..1777b679ec6 --- /dev/null +++ b/libraries/test_proguard/src/main/java/androidx/media3/test/proguard/UiComposeMaterial3ModuleProguard.kt @@ -0,0 +1,34 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.test.proguard + +import android.content.Context +import androidx.media3.common.Player +import androidx.media3.exoplayer.ExoPlayer +import androidx.media3.transformer.CompositionPlayer +import androidx.media3.ui.compose.material3.util.isScrubbingModeEnabled +import androidx.media3.ui.compose.material3.util.setScrubbingModeEnabled + +internal fun testScrubbingModeReflection(context: Context) { + val exoPlayer: Player = ExoPlayer.Builder(context).build() + exoPlayer.setScrubbingModeEnabled(true) + check(exoPlayer.isScrubbingModeEnabled()) + + val compPlayer: Player = CompositionPlayer.Builder(context).build() + compPlayer.setScrubbingModeEnabled(true) + check(compPlayer.isScrubbingModeEnabled()) +} diff --git a/libraries/test_proguard_minimal/build.gradle.kts b/libraries/test_proguard_minimal/build.gradle.kts new file mode 100644 index 00000000000..0b56db809c5 --- /dev/null +++ b/libraries/test_proguard_minimal/build.gradle.kts @@ -0,0 +1,47 @@ +// Copyright 2026 The Android Open Source Project +// +// Licensed under the Apache License, Version 2.0 (the "License"); +// you may not use this file except in compliance with the License. +// You may obtain a copy of the License at +// +// http://www.apache.org/licenses/LICENSE-2.0 +// +// Unless required by applicable law or agreed to in writing, software +// distributed under the License is distributed on an "AS IS" BASIS, +// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +// See the License for the specific language governing permissions and +// limitations under the License. +plugins { id("media3.android-application") } + +android { + namespace = "androidx.media3.test.proguard.minimal" + + compileOptions.isCoreLibraryDesugaringEnabled = true + + buildTypes { + // Run R8 for all build types to discover potential proguard problems. + configureEach { + isDebuggable = false + isShrinkResources = true + isMinifyEnabled = true + proguardFiles( + "src/androidTest/proguard-rules.pgcfg", + "src/androidTest/proguard-checkdiscard-rules.pgcfg", + getDefaultProguardFile("proguard-android-optimize.txt"), + ) + testProguardFile("src/androidTest/proguard-test-rules.pgcfg") + } + } +} + +dependencies { + coreLibraryDesugaring(libs.desugar.jdk.libs) + compileOnly(libs.androidx.annotation) + implementation(project(":lib-exoplayer")) + // Explicit dependency on lib-effect so R8 verifies that unused effect classes + // are discarded at build time. + implementation(project(":lib-effect")) + androidTestImplementation(libs.androidx.test.core) + androidTestImplementation(libs.androidx.test.ext.junit) + androidTestImplementation(libs.androidx.test.runner) +} diff --git a/libraries/session/src/main/res/values/colors_material.xml b/libraries/test_proguard_minimal/src/androidTest/AndroidManifest.xml similarity index 66% rename from libraries/session/src/main/res/values/colors_material.xml rename to libraries/test_proguard_minimal/src/androidTest/AndroidManifest.xml index 13f01b63b2b..f335adf8ebb 100644 --- a/libraries/session/src/main/res/values/colors_material.xml +++ b/libraries/test_proguard_minimal/src/androidTest/AndroidManifest.xml @@ -1,5 +1,5 @@ - - - - - - #ffffffff - - #b3ffffff - + + + diff --git a/libraries/test_proguard_minimal/src/androidTest/java/androidx/media3/test/proguard/minimal/ExoPlayerMinimalModuleProguardTest.java b/libraries/test_proguard_minimal/src/androidTest/java/androidx/media3/test/proguard/minimal/ExoPlayerMinimalModuleProguardTest.java new file mode 100644 index 00000000000..5457363c147 --- /dev/null +++ b/libraries/test_proguard_minimal/src/androidTest/java/androidx/media3/test/proguard/minimal/ExoPlayerMinimalModuleProguardTest.java @@ -0,0 +1,104 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.test.proguard.minimal; + +import static androidx.test.core.app.ApplicationProvider.getApplicationContext; +import static androidx.test.platform.app.InstrumentationRegistry.getInstrumentation; +import static org.junit.Assert.assertThrows; +import static org.junit.Assert.assertTrue; + +import android.content.res.Resources; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.io.InputStream; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** + * Instrumentation test verifying that unused classes and resources are stripped in a minimal + * ExoPlayer build. + * + *

      + *
    • Build-time validation: R8 enforces the {@code -checkdiscard} rules in {@code + * proguard-checkdiscard-rules.pgcfg} (failing the build if {@code + * SingleInputVideoGraph$Factory} or {@code DefaultVideoFrameProcessor$Factory$Builder} is + * retained). + *
    • Runtime validation: Verifies basic player execution, unused class stripping, and resource + * stripping. + *
    + */ +@RunWith(AndroidJUnit4.class) +public final class ExoPlayerMinimalModuleProguardTest { + + @Test + public void createAndReleaseExoPlayer_succeeds() { + getInstrumentation() + .runOnMainSync( + () -> + ExoPlayerMinimalModuleProguard.createAndReleaseExoPlayer(getApplicationContext())); + } + + @Test + @SuppressWarnings({"UnnecessaryStringBuilder", "RedundantStringBuilderAppend"}) + public void effectClasses_areStripped() { + // LINT.IfChange + // b/463697143: Obfuscate class name to bypass R8/AppReduce static analysis. + assertThrows( + ClassNotFoundException.class, + () -> + Class.forName( + new StringBuilder("androidx.media3.effect.") + .append("SingleInputVideoGraph$Factory") + .toString())); + assertThrows( + ClassNotFoundException.class, + () -> + Class.forName( + new StringBuilder("androidx.media3.effect.") + .append("DefaultVideoFrameProcessor$Factory$Builder") + .toString())); + // LINT.ThenChange(../../../../../../../../../exoplayer/src/main/java/androidx/media3/exoplayer/video/PlaybackVideoGraphWrapper.java) + } + + @Test + public void unusedResource_isStrippedOrEmpty() throws Exception { + Resources resources = getApplicationContext().getResources(); + String packageName = getApplicationContext().getPackageName(); + + assertTrue(getRawResourceAvailableBytes(resources, "unused_raw_resource", packageName) <= 1); + } + + @Test + public void effectShaderResource_isStrippedOrEmpty() throws Exception { + Resources resources = getApplicationContext().getResources(); + String packageName = getApplicationContext().getPackageName(); + + assertTrue( + getRawResourceAvailableBytes(resources, "fragment_shader_copy_es2", packageName) <= 1); + } + + private static int getRawResourceAvailableBytes( + Resources resources, String resourceName, String packageName) throws Exception { + int resId = resources.getIdentifier(resourceName, "raw", packageName); + if (resId == 0) { + return 0; + } + try (InputStream inputStream = resources.openRawResource(resId)) { + // Shrunk resources are stubbed to empty/dummy bytes (0 or 1 byte) or removed by resource + // shrinker. + return inputStream.available(); + } + } +} diff --git a/libraries/test_proguard_minimal/src/androidTest/proguard-checkdiscard-rules.pgcfg b/libraries/test_proguard_minimal/src/androidTest/proguard-checkdiscard-rules.pgcfg new file mode 100644 index 00000000000..f22eb8e4ccf --- /dev/null +++ b/libraries/test_proguard_minimal/src/androidTest/proguard-checkdiscard-rules.pgcfg @@ -0,0 +1,5 @@ +# ProGuard / R8 rules asserting that optional classes are stripped when unused. + +# Assert that effect module classes are discarded when setVideoEffects is not called. +-checkdiscard class androidx.media3.effect.SingleInputVideoGraph$Factory +-checkdiscard class androidx.media3.effect.DefaultVideoFrameProcessor$Factory$Builder diff --git a/libraries/test_proguard_minimal/src/androidTest/proguard-rules.pgcfg b/libraries/test_proguard_minimal/src/androidTest/proguard-rules.pgcfg new file mode 100644 index 00000000000..07d05a7e0f1 --- /dev/null +++ b/libraries/test_proguard_minimal/src/androidTest/proguard-rules.pgcfg @@ -0,0 +1,3 @@ +# Keep all classes of the minimal proguard test APK except R classes so resource shrinking works. +-keep class !androidx.media3.test.proguard.minimal.R, !androidx.media3.test.proguard.minimal.R$*, androidx.media3.test.proguard.minimal.** {*;} +-keep class kotlin.** {*;} diff --git a/libraries/test_proguard_minimal/src/androidTest/proguard-test-rules.pgcfg b/libraries/test_proguard_minimal/src/androidTest/proguard-test-rules.pgcfg new file mode 100644 index 00000000000..575cafffd50 --- /dev/null +++ b/libraries/test_proguard_minimal/src/androidTest/proguard-test-rules.pgcfg @@ -0,0 +1,20 @@ +# Proguard rules for the androidTest target to ensure the test runners are kept. +-keep class androidx.test.** {*;} +-keep class com.vladium.** {*;} +-keep class org.jacoco.** {*;} +-keep class org.junit.** {*;} +-keepattributes *Annotation* +-dontwarn android.test.** +-dontwarn androidx.test.** +-dontwarn com.sun.jna.** +-dontwarn edu.umd.cs.findbugs.** +-dontwarn java.lang.instrument.** +-dontwarn java.lang.management.** +-dontwarn junit.framework.** +-dontwarn junit.runner.** +-dontwarn org.hamcrest.** +-dontwarn org.jacoco.** +-dontwarn org.junit.** +-dontwarn org.mockito.** +-dontwarn org.objectweb.asm.** +-dontwarn sun.misc.SignalHandler,sun.misc.Signal diff --git a/libraries/test_proguard_minimal/src/main/AndroidManifest.xml b/libraries/test_proguard_minimal/src/main/AndroidManifest.xml new file mode 100644 index 00000000000..f335adf8ebb --- /dev/null +++ b/libraries/test_proguard_minimal/src/main/AndroidManifest.xml @@ -0,0 +1,18 @@ + + + + + diff --git a/libraries/test_proguard_minimal/src/main/java/androidx/media3/test/proguard/minimal/ExoPlayerMinimalModuleProguard.java b/libraries/test_proguard_minimal/src/main/java/androidx/media3/test/proguard/minimal/ExoPlayerMinimalModuleProguard.java new file mode 100644 index 00000000000..3e83fe0ba8d --- /dev/null +++ b/libraries/test_proguard_minimal/src/main/java/androidx/media3/test/proguard/minimal/ExoPlayerMinimalModuleProguard.java @@ -0,0 +1,37 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.test.proguard.minimal; + +import android.content.Context; +import android.net.Uri; +import androidx.media3.common.MediaItem; +import androidx.media3.exoplayer.ExoPlayer; + +/** + * Class exercising only the core ExoPlayer functionality without optional features (e.g., without + * setVideoEffects or extension decoders) to ensure unused classes are stripped. + */ +public final class ExoPlayerMinimalModuleProguard { + + private ExoPlayerMinimalModuleProguard() {} + + public static void createAndReleaseExoPlayer(Context context) { + ExoPlayer player = new ExoPlayer.Builder(context).build(); + player.setMediaItem(MediaItem.fromUri(Uri.EMPTY)); + player.prepare(); + player.release(); + } +} diff --git a/libraries/test_proguard_minimal/src/main/java/androidx/media3/test/proguard/minimal/package-info.java b/libraries/test_proguard_minimal/src/main/java/androidx/media3/test/proguard/minimal/package-info.java new file mode 100644 index 00000000000..6e44ea853ad --- /dev/null +++ b/libraries/test_proguard_minimal/src/main/java/androidx/media3/test/proguard/minimal/package-info.java @@ -0,0 +1,19 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +@NonNullApi +package androidx.media3.test.proguard.minimal; + +import androidx.media3.common.util.NonNullApi; diff --git a/libraries/test_proguard_minimal/src/main/res/raw/unused_raw_resource b/libraries/test_proguard_minimal/src/main/res/raw/unused_raw_resource new file mode 100644 index 00000000000..f22bbdf4683 --- /dev/null +++ b/libraries/test_proguard_minimal/src/main/res/raw/unused_raw_resource @@ -0,0 +1 @@ +This resource is unused and should be shrunk or removed by resource shrinking. diff --git a/libraries/test_session_common/src/main/aidl/androidx/media3/test/session/common/IRemoteMediaSession.aidl b/libraries/test_session_common/src/main/aidl/androidx/media3/test/session/common/IRemoteMediaSession.aidl index 67a6a4be424..300e82b2140 100644 --- a/libraries/test_session_common/src/main/aidl/androidx/media3/test/session/common/IRemoteMediaSession.aidl +++ b/libraries/test_session_common/src/main/aidl/androidx/media3/test/session/common/IRemoteMediaSession.aidl @@ -86,6 +86,7 @@ interface IRemoteMediaSession { void setCurrentPeriodIndex(String sessionId, int index); void setTrackSelectionParameters(String sessionId, in Bundle parameters); void notifyTimelineChanged(String sessionId, int reason); + void notifyTimelineChangedWithMediaItemTransition(String sessionId); void notifyPlaylistMetadataChanged(String sessionId); void notifyShuffleModeEnabledChanged(String sessionId); void notifyRepeatModeChanged(String sessionId); diff --git a/libraries/test_session_common/src/main/java/androidx/media3/test/session/common/MediaSessionConstants.java b/libraries/test_session_common/src/main/java/androidx/media3/test/session/common/MediaSessionConstants.java index 6902907b53a..b86b1d9c2b0 100644 --- a/libraries/test_session_common/src/main/java/androidx/media3/test/session/common/MediaSessionConstants.java +++ b/libraries/test_session_common/src/main/java/androidx/media3/test/session/common/MediaSessionConstants.java @@ -58,6 +58,10 @@ public class MediaSessionConstants { public static final String CONNECTION_HINT_KEY_ASYNC_CONNECTION_DELAY_MS = "androidx.media3.session.async_connection_delay_ms"; + /** Connection hint key for requesting an async library root with a delay in milliseconds. */ + public static final String CONNECTION_HINT_KEY_ASYNC_LIBRARY_ROOT_DELAY_MS = + "androidx.media3.session.async_library_root_delay_ms"; + /** Connection hint key for requesting an async rejection with a delay in milliseconds. */ public static final String CONNECTION_HINT_KEY_ASYNC_CONNECTION_REJECT_DELAY_MS = "androidx.media3.session.async_connection_reject_delay_ms"; diff --git a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaBrowserCompatWithMediaLibraryServiceTest.java b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaBrowserCompatWithMediaLibraryServiceTest.java index 26f695a71a5..59571ff46de 100644 --- a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaBrowserCompatWithMediaLibraryServiceTest.java +++ b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaBrowserCompatWithMediaLibraryServiceTest.java @@ -66,6 +66,7 @@ import static androidx.media3.test.session.common.MediaBrowserConstants.SEARCH_RESULT_COUNT; import static androidx.media3.test.session.common.MediaBrowserConstants.SUBSCRIBE_PARENT_ID_2; import static androidx.media3.test.session.common.MediaSessionConstants.CONNECTION_HINT_KEY_ASYNC_CONNECTION_DELAY_MS; +import static androidx.media3.test.session.common.MediaSessionConstants.CONNECTION_HINT_KEY_ASYNC_LIBRARY_ROOT_DELAY_MS; import static androidx.media3.test.session.common.MediaSessionConstants.EXTRA_KEY_ASYNC_CONNECTION_CONFIRMATION; import static androidx.media3.test.session.common.MediaSessionConstants.KEY_CONTROLLER; import static androidx.media3.test.session.common.TestUtils.LONG_TIMEOUT_MS; @@ -218,6 +219,45 @@ public void getRoot_browserConnectionAsync() throws Exception { .isTrue(); } + @Test + public void getRoot_onGetLibraryRootAsync() throws Exception { + Bundle rootHints = new Bundle(); + rootHints.putLong(CONNECTION_HINT_KEY_ASYNC_LIBRARY_ROOT_DELAY_MS, 200L); + + connectAndWait(rootHints); + + assertThat(browserCompat.getRoot()).isEqualTo(ROOT_ID); + assertThat( + browserCompat + .getExtras() + .getInt(ROOT_EXTRAS_KEY, /* defaultValue= */ ROOT_EXTRAS_VALUE + 1)) + .isEqualTo(ROOT_EXTRAS_VALUE); + assertThat(TestUtils.contains(browserCompat.getExtras(), ROOT_EXTRAS)).isTrue(); + } + + @Test + public void getRoot_connectionAsyncAndOnGetLibraryRootAsync() throws Exception { + Bundle rootHints = new Bundle(); + rootHints.putLong(CONNECTION_HINT_KEY_ASYNC_CONNECTION_DELAY_MS, 100L); + rootHints.putLong(CONNECTION_HINT_KEY_ASYNC_LIBRARY_ROOT_DELAY_MS, 100L); + rootHints.putString(KEY_CONTROLLER, "getRoot_connectionAsyncAndOnGetLibraryRootAsync"); + + connectAndWait(rootHints); + + assertThat(browserCompat.getRoot()).isEqualTo(ROOT_ID); + assertThat( + browserCompat + .getExtras() + .getInt(ROOT_EXTRAS_KEY, /* defaultValue= */ ROOT_EXTRAS_VALUE + 1)) + .isEqualTo(ROOT_EXTRAS_VALUE); + assertThat(TestUtils.contains(browserCompat.getExtras(), ROOT_EXTRAS)).isTrue(); + assertThat( + browserCompat + .getExtras() + .getBoolean(EXTRA_KEY_ASYNC_CONNECTION_CONFIRMATION, /* defaultValue= */ false)) + .isTrue(); + } + @Test public void getItem_browsable() throws Exception { String mediaId = MEDIA_ID_GET_BROWSABLE_ITEM; diff --git a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaBrowserCompatWithMediaSessionServiceTest.java b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaBrowserCompatWithMediaSessionServiceTest.java index a0746a0807e..72b4b2df791 100644 --- a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaBrowserCompatWithMediaSessionServiceTest.java +++ b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaBrowserCompatWithMediaSessionServiceTest.java @@ -163,6 +163,24 @@ public void getRoot_connectionAsync() throws Exception { assertThat(connectionCallback.connectedLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); } + @Test + public void connect_asyncRejected() throws Exception { + Bundle rootHints = new Bundle(); + rootHints.putLong( + MediaSessionConstants.CONNECTION_HINT_KEY_ASYNC_CONNECTION_REJECT_DELAY_MS, 100L); + + handler.postAndSync( + () -> { + browserCompat = + new MediaBrowserCompat(context, getServiceComponent(), connectionCallback, rootHints); + }); + + browserCompat.connect(); + + assertThat(connectionCallback.failedLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(connectionCallback.connectedLatch.getCount()).isNotEqualTo(0); + } + @Ignore("Create a session service whose onConnect() returns null.") @Test public void connect_rejected() throws InterruptedException { diff --git a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaControllerCompatCallbackWithMediaSessionTest.java b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaControllerCompatCallbackWithMediaSessionTest.java index cbc36ede6b2..ccfa92dae3b 100644 --- a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaControllerCompatCallbackWithMediaSessionTest.java +++ b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaControllerCompatCallbackWithMediaSessionTest.java @@ -1804,6 +1804,103 @@ public void onPlaybackStateChanged(PlaybackStateCompat state) { assertThat(controllerCompat.getSessionActivity()).isNull(); } + @Test + public void + setSessionActivity_standaloneSessionWithoutNotificationController_changedWhenReceivedWithSetter() + throws Exception { + RemoteMediaSession standaloneSession = + new RemoteMediaSession("standaloneSession", context, Bundle.EMPTY); + MediaControllerCompat standaloneControllerCompat = + new MediaControllerCompat(context, standaloneSession.getCompatToken()); + waitUntilSessionReady(standaloneControllerCompat); + + Intent intent = new Intent(context, SurfaceActivity.class); + PendingIntent sessionActivity = + PendingIntent.getActivity( + context, 0, intent, PendingIntent.FLAG_IMMUTABLE | PendingIntent.FLAG_UPDATE_CURRENT); + CountDownLatch playingLatch = new CountDownLatch(1); + CountDownLatch bufferingLatch = new CountDownLatch(1); + MediaControllerCompat.Callback callback = + new MediaControllerCompat.Callback() { + @Override + public void onPlaybackStateChanged(PlaybackStateCompat state) { + if (state.getState() == PlaybackStateCompat.STATE_BUFFERING) { + if (standaloneControllerCompat.getSessionActivity() == null) { + bufferingLatch.countDown(); + } + } else if (state.getState() == PlaybackStateCompat.STATE_PLAYING) { + if (standaloneControllerCompat.getSessionActivity() != null) { + playingLatch.countDown(); + } + } + } + }; + standaloneControllerCompat.registerCallback(callback, handler); + assertThat(standaloneControllerCompat.getSessionActivity()).isNull(); + + standaloneSession.setSessionActivity(/* controllerKey= */ null, sessionActivity); + standaloneSession + .getMockPlayer() + .notifyPlayWhenReadyChanged( + true, + Player.PLAY_WHEN_READY_CHANGE_REASON_USER_REQUEST, + Player.PLAYBACK_SUPPRESSION_REASON_NONE); + standaloneSession.getMockPlayer().notifyPlaybackStateChanged(STATE_READY); + + assertThat(playingLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(standaloneControllerCompat.getSessionActivity()).isEqualTo(sessionActivity); + + standaloneSession.setSessionActivity(/* controllerKey= */ null, null); + standaloneSession.getMockPlayer().notifyPlaybackStateChanged(Player.STATE_BUFFERING); + + assertThat(bufferingLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(standaloneControllerCompat.getSessionActivity()).isNull(); + + standaloneSession.release(); + } + + @Test + public void + setPlaybackException_standaloneSessionWithoutNotificationController_updatesPlaybackStateToErrorAndRecovers() + throws Exception { + RemoteMediaSession standaloneSession = + new RemoteMediaSession("standaloneSessionForError", context, Bundle.EMPTY); + MediaControllerCompat standaloneControllerCompat = + new MediaControllerCompat(context, standaloneSession.getCompatToken()); + waitUntilSessionReady(standaloneControllerCompat); + + standaloneSession.getMockPlayer().notifyPlaybackStateChanged(STATE_READY); + List playbackStates = new ArrayList<>(); + CountDownLatch latch = new CountDownLatch(2); + MediaControllerCompat.Callback callback = + new MediaControllerCompat.Callback() { + @Override + public void onPlaybackStateChanged(PlaybackStateCompat state) { + if (playbackStates.isEmpty() && state.getState() != PlaybackStateCompat.STATE_ERROR) { + return; + } + playbackStates.add(state); + latch.countDown(); + } + }; + standaloneControllerCompat.registerCallback(callback, handler); + + PlaybackException testPlayerError = + new PlaybackException( + /* message= */ "standalone error", + /* cause= */ null, + PlaybackException.ERROR_CODE_REMOTE_ERROR); + standaloneSession.setPlaybackException(/* controllerKey= */ null, testPlayerError); + standaloneSession.setPlaybackException(/* controllerKey= */ null, /* playerError= */ null); + + assertThat(latch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(playbackStates.get(0).getState()).isEqualTo(PlaybackStateCompat.STATE_ERROR); + assertThat(playbackStates.get(0).getErrorMessage().toString()).isEqualTo("standalone error"); + assertThat(playbackStates.get(1).getState()).isNotEqualTo(PlaybackStateCompat.STATE_ERROR); + + standaloneSession.release(); + } + @SuppressWarnings("deprecation") // Testing access through deprecated androidx.media library @Test public void setSessionActivity_setToNotificationController_changedWhenReceivedWithSetter() diff --git a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaControllerCompatPlaybackStateCompatActionsWithMediaSessionTest.java b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaControllerCompatPlaybackStateCompatActionsWithMediaSessionTest.java index 26f00679c52..3fc7adcb7dd 100644 --- a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaControllerCompatPlaybackStateCompatActionsWithMediaSessionTest.java +++ b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaControllerCompatPlaybackStateCompatActionsWithMediaSessionTest.java @@ -25,6 +25,8 @@ import static java.util.concurrent.TimeUnit.MILLISECONDS; import static org.junit.Assert.assertThrows; +import android.app.PendingIntent; +import android.content.Intent; import android.net.Uri; import android.os.Bundle; import android.os.Handler; @@ -48,6 +50,7 @@ import androidx.media3.session.MediaSession.ConnectionResult; import androidx.media3.session.MediaSession.ConnectionResult.AcceptedResultBuilder; import androidx.media3.test.session.common.HandlerThreadTestRule; +import androidx.media3.test.session.common.SurfaceActivity; import androidx.media3.test.session.common.TestUtils; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; @@ -1535,6 +1538,70 @@ public ListenableFuture onConnectAsync( releasePlayer(player); } + @Test + public void + onConnect_mediaNotificationController_sessionExtrasAndActivityConfigured_setsExtrasAndActivityOnLegacySession() + throws Exception { + Player player = createDefaultPlayer(); + Bundle sessionExtras = new Bundle(); + sessionExtras.putString("test_key", "test_value"); + Intent intent = new Intent(ApplicationProvider.getApplicationContext(), SurfaceActivity.class); + PendingIntent sessionActivity = + PendingIntent.getActivity( + ApplicationProvider.getApplicationContext(), + 0, + intent, + PendingIntent.FLAG_IMMUTABLE | PendingIntent.FLAG_UPDATE_CURRENT); + MediaSession.Callback callback = + new MediaSession.Callback() { + @Override + public ListenableFuture onConnectAsync( + MediaSession session, MediaSession.ControllerInfo controller) { + return Futures.immediateFuture( + new AcceptedResultBuilder(session, controller) + .setSessionExtras(sessionExtras) + .setSessionActivity(sessionActivity) + .build()); + } + }; + MediaSession mediaSession = createMediaSession(player, callback); + connectMediaNotificationController(mediaSession); + MediaControllerCompat controllerCompat = createMediaControllerCompat(mediaSession); + + assertThat(controllerCompat.getExtras().getString("test_key")).isEqualTo("test_value"); + assertThat(controllerCompat.getSessionActivity()).isEqualTo(sessionActivity); + + mediaSession.release(); + releasePlayer(player); + } + + @Test + public void builder_withSessionExtrasAndActivity_setsExtrasAndActivityOnLegacySession() + throws Exception { + Player player = createDefaultPlayer(); + Bundle sessionExtras = new Bundle(); + sessionExtras.putString("test_key", "test_value"); + Intent intent = new Intent(ApplicationProvider.getApplicationContext(), SurfaceActivity.class); + PendingIntent sessionActivity = + PendingIntent.getActivity( + ApplicationProvider.getApplicationContext(), + 0, + intent, + PendingIntent.FLAG_IMMUTABLE | PendingIntent.FLAG_UPDATE_CURRENT); + MediaSession mediaSession = + new MediaSession.Builder(ApplicationProvider.getApplicationContext(), player) + .setSessionExtras(sessionExtras) + .setSessionActivity(sessionActivity) + .build(); + MediaControllerCompat controllerCompat = createMediaControllerCompat(mediaSession); + + assertThat(controllerCompat.getExtras().getString("test_key")).isEqualTo("test_value"); + assertThat(controllerCompat.getSessionActivity()).isEqualTo(sessionActivity); + + mediaSession.release(); + releasePlayer(player); + } + @Test public void playerWithCustomLayout_setCustomLayout_playbackStateChangedWithCustomActionsChanged() throws Exception { diff --git a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaControllerListenerTest.java b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaControllerListenerTest.java index 6a7f48d4e62..401306aed21 100644 --- a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaControllerListenerTest.java +++ b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaControllerListenerTest.java @@ -25,15 +25,20 @@ import static androidx.media3.common.Player.COMMAND_PREPARE; import static androidx.media3.common.Player.COMMAND_RELEASE; import static androidx.media3.common.Player.COMMAND_SEEK_BACK; +import static androidx.media3.common.Player.DISCONTINUITY_REASON_SEEK; import static androidx.media3.common.Player.EVENT_AVAILABLE_COMMANDS_CHANGED; import static androidx.media3.common.Player.EVENT_IS_PLAYING_CHANGED; +import static androidx.media3.common.Player.EVENT_MEDIA_ITEM_TRANSITION; import static androidx.media3.common.Player.EVENT_PLAYBACK_STATE_CHANGED; import static androidx.media3.common.Player.EVENT_PLAYER_ERROR; import static androidx.media3.common.Player.EVENT_PLAY_WHEN_READY_CHANGED; import static androidx.media3.common.Player.EVENT_RENDERED_FIRST_FRAME; +import static androidx.media3.common.Player.EVENT_TIMELINE_CHANGED; import static androidx.media3.common.Player.MEDIA_ITEM_TRANSITION_REASON_AUTO; +import static androidx.media3.common.Player.MEDIA_ITEM_TRANSITION_REASON_PLAYLIST_CHANGED; import static androidx.media3.common.Player.MEDIA_ITEM_TRANSITION_REASON_REPEAT; import static androidx.media3.common.Player.MEDIA_ITEM_TRANSITION_REASON_SEEK; +import static androidx.media3.common.Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED; import static androidx.media3.session.MediaTestUtils.createMediaItem; import static androidx.media3.session.MediaTestUtils.createTimeline; import static androidx.media3.session.MediaUtils.createPlayerCommandsWith; @@ -931,13 +936,13 @@ public void setPlayer_notifiesChangedValues() throws Exception { AtomicReference timelineRef = new AtomicReference<>(); AtomicReference playlistMetadataRef = new AtomicReference<>(); AtomicReference audioAttributesRef = new AtomicReference<>(); - AtomicBoolean isPlayingAdRef = new AtomicBoolean(); AtomicInteger currentAdGroupIndexRef = new AtomicInteger(); AtomicInteger currentAdIndexInAdGroupRef = new AtomicInteger(); AtomicBoolean shuffleModeEnabledRef = new AtomicBoolean(); AtomicInteger repeatModeRef = new AtomicInteger(); AtomicReference commandsRef = new AtomicReference<>(); - CountDownLatch latch = new CountDownLatch(8); + CountDownLatch latch = new CountDownLatch(7); + // Connect to session with empty player. MediaController controller = controllerTestRule.createController(remoteSession.getToken()); threadTestRule .getHandler() @@ -966,6 +971,8 @@ public void onPlaybackStateChanged(@Player.State int playbackState) { @Override public void onTimelineChanged( Timeline timeline, @Player.TimelineChangeReason int reason) { + currentAdGroupIndexRef.set(controller.getCurrentAdGroupIndex()); + currentAdIndexInAdGroupRef.set(controller.getCurrentAdIndexInAdGroup()); timelineRef.set(timeline); latch.countDown(); } @@ -976,17 +983,6 @@ public void onPlaylistMetadataChanged(MediaMetadata playlistMetadata) { latch.countDown(); } - @Override - public void onPositionDiscontinuity( - PositionInfo oldPosition, - PositionInfo newPosition, - @Player.DiscontinuityReason int reason) { - isPlayingAdRef.set(controller.isPlayingAd()); - currentAdGroupIndexRef.set(newPosition.adGroupIndex); - currentAdIndexInAdGroupRef.set(newPosition.adIndexInAdGroup); - latch.countDown(); - } - @Override public void onShuffleModeEnabledChanged(boolean shuffleModeEnabled) { shuffleModeEnabledRef.set(shuffleModeEnabled); @@ -1081,10 +1077,8 @@ public void setPlayer_updatesGetters() throws Exception { controller.addListener( new Player.Listener() { @Override - public void onPositionDiscontinuity( - PositionInfo oldPosition, - PositionInfo newPosition, - @Player.DiscontinuityReason int reason) { + public void onTimelineChanged( + Timeline timeline, @Player.TimelineChangeReason int reason) { currentPositionMsRef.set(controller.getCurrentPosition()); contentPositionMsRef.set(controller.getContentPosition()); durationMsRef.set(controller.getDuration()); @@ -1177,7 +1171,9 @@ public void onEvents(Player player, Player.Events events) { assertThat(onEventsCurrentMediaItemIndexRef.get()).isEqualTo(testMediaItemIndex); assertThat(currentPeriodIndexRef.get()).isEqualTo(testPeriodIndex); assertThat(onEventsCurrentPeriodIndexRef.get()).isEqualTo(testPeriodIndex); - assertThat(getEventsAsList(eventsRef.get())).contains(Player.EVENT_POSITION_DISCONTINUITY); + assertThat(getEventsAsList(eventsRef.get())) + .containsExactly( + EVENT_TIMELINE_CHANGED, EVENT_MEDIA_ITEM_TRANSITION, EVENT_PLAYBACK_STATE_CHANGED); } @Test @@ -1187,9 +1183,7 @@ public void onMediaItemTransition_mediaItemTransitionSeek_callbackCalledForReaso Timeline timeline = MediaTestUtils.createTimeline(/* windowCount= */ 5, /* buildWithUri= */ false); remoteSession.getMockPlayer().setTimeline(timeline); - remoteSession - .getMockPlayer() - .notifyTimelineChanged(Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); + remoteSession.getMockPlayer().notifyTimelineChanged(TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); remoteSession.getMockPlayer().setCurrentMediaItemIndex(currentIndex); remoteSession .getMockPlayer() @@ -1241,7 +1235,7 @@ public void onEvents(Player player, Player.Events events) { assertThat(mediaItemFromGetterRef.get()).isEqualTo(currentMediaItem); assertThat(mediaItemFromOnEventsRef.get()).isEqualTo(currentMediaItem); assertThat(reasonRef.get()).isEqualTo(testReason); - assertThat(getEventsAsList(eventsRef.get())).contains(Player.EVENT_MEDIA_ITEM_TRANSITION); + assertThat(getEventsAsList(eventsRef.get())).contains(EVENT_MEDIA_ITEM_TRANSITION); } @Test @@ -1283,7 +1277,7 @@ public void onMediaItemTransition( @Override public void onEvents(Player player, Player.Events events) { - if (events.contains(Player.EVENT_MEDIA_ITEM_TRANSITION)) { + if (events.contains(EVENT_MEDIA_ITEM_TRANSITION)) { eventsRef.set(events); mediaItemFromOnEventsRef.set(player.getCurrentMediaItem()); latch.countDown(); @@ -1293,9 +1287,7 @@ public void onEvents(Player player, Player.Events events) { remoteSession.getMockPlayer().setTimeline(timeline); remoteSession.getMockPlayer().setCurrentMediaItemIndex(/* index= */ 0); remoteSession.getMockPlayer().setCurrentPeriodIndex(/* index= */ 0); - remoteSession - .getMockPlayer() - .notifyTimelineChanged(Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); + remoteSession.getMockPlayer().notifyTimelineChanged(TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); assertThat(timelineLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); remoteSession.getMockPlayer().setCurrentMediaItemIndex(/* index= */ 1); remoteSession.getMockPlayer().setCurrentPeriodIndex(/* index= */ 1); @@ -1327,7 +1319,7 @@ public void onEvents(Player player, Player.Events events) { Player.DISCONTINUITY_REASON_AUTO_TRANSITION); assertThat(latch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); - assertThat(getEventsAsList(eventsRef.get())).contains(Player.EVENT_MEDIA_ITEM_TRANSITION); + assertThat(getEventsAsList(eventsRef.get())).contains(EVENT_MEDIA_ITEM_TRANSITION); assertThat(mediaItemFromParamRef.get()).isEqualTo(expectedMediaItemWithoutUri); assertThat(mediaItemFromGetterRef.get()).isEqualTo(expectedMediaItemWithoutUri); assertThat(mediaItemFromOnEventsRef.get()).isEqualTo(expectedMediaItemWithoutUri); @@ -1371,7 +1363,7 @@ public void onMediaItemTransition( @Override public void onEvents(Player player, Player.Events events) { - if (events.contains(Player.EVENT_MEDIA_ITEM_TRANSITION)) { + if (events.contains(EVENT_MEDIA_ITEM_TRANSITION)) { eventsRef.set(events); mediaItemFromOnEventsRef.set(player.getCurrentMediaItem()); latch.countDown(); @@ -1381,9 +1373,7 @@ public void onEvents(Player player, Player.Events events) { remoteSession.getMockPlayer().setTimeline(timeline); remoteSession.getMockPlayer().setCurrentMediaItemIndex(/* index= */ 0); remoteSession.getMockPlayer().setCurrentPeriodIndex(/* index= */ 0); - remoteSession - .getMockPlayer() - .notifyTimelineChanged(Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); + remoteSession.getMockPlayer().notifyTimelineChanged(TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); assertThat(timelineLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); remoteSession.getMockPlayer().setCurrentMediaItemIndex(/* index= */ 0); remoteSession.getMockPlayer().setCurrentPeriodIndex(/* index= */ 0); @@ -1421,7 +1411,7 @@ public void onEvents(Player player, Player.Events events) { assertThat(mediaItemFromGetterRef.get()).isEqualTo(expectedMediaItemWithoutUri); assertThat(mediaItemFromOnEventsRef.get()).isEqualTo(expectedMediaItemWithoutUri); assertThat(reasonRef.get()).isEqualTo(MEDIA_ITEM_TRANSITION_REASON_REPEAT); - assertThat(getEventsAsList(eventsRef.get())).contains(Player.EVENT_MEDIA_ITEM_TRANSITION); + assertThat(getEventsAsList(eventsRef.get())).contains(EVENT_MEDIA_ITEM_TRANSITION); } @Test @@ -1471,18 +1461,16 @@ public void onEvents(Player player, Player.Events events) { remoteSession.getMockPlayer().setTimeline(timeline); remoteSession.getMockPlayer().setCurrentMediaItemIndex(/* index= */ 0); remoteSession.getMockPlayer().setCurrentPeriodIndex(/* index= */ 0); - remoteSession - .getMockPlayer() - .notifyTimelineChanged(Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); + remoteSession.getMockPlayer().notifyTimelineChanged(TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); assertThat(timelineLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); - remoteSession.getMockPlayer().setCurrentMediaItemIndex(/* index= */ 0); - remoteSession.getMockPlayer().setCurrentPeriodIndex(/* index= */ 0); + remoteSession.getMockPlayer().setCurrentMediaItemIndex(/* index= */ 1); + remoteSession.getMockPlayer().setCurrentPeriodIndex(/* index= */ 1); MediaItem expectedMediaItemWithoutUri = createMediaItem("mediaId", /* buildWithUri= */ false); remoteSession .getMockPlayer() .notifyPositionDiscontinuity( - new PositionInfo( + /* oldPosition= */ new PositionInfo( /* windowUid= */ new Object(), /* mediaItemIndex= */ 0, mediaItem, @@ -1492,7 +1480,7 @@ public void onEvents(Player player, Player.Events events) { /* contentPositionMs= */ 10_000L, /* adGroupIndex= */ C.INDEX_UNSET, /* adIndexInAdGroup= */ C.INDEX_UNSET), - new PositionInfo( + /* newPosition= */ new PositionInfo( /* windowUid= */ new Object(), /* mediaItemIndex= */ 1, mediaItem, @@ -1509,7 +1497,125 @@ public void onEvents(Player player, Player.Events events) { assertThat(mediaItemFromGetterRef.get()).isEqualTo(expectedMediaItemWithoutUri); assertThat(mediaItemFromOnEventsRef.get()).isEqualTo(expectedMediaItemWithoutUri); assertThat(reasonRef.get()).isEqualTo(MEDIA_ITEM_TRANSITION_REASON_SEEK); - assertThat(getEventsAsList(eventsRef.get())).contains(Player.EVENT_MEDIA_ITEM_TRANSITION); + assertThat(getEventsAsList(eventsRef.get())).contains(EVENT_MEDIA_ITEM_TRANSITION); + } + + @Test + public void onMediaItemTransition_seekDiscontinuitySameItemWithEmptyTimeline_callbackNotCalled() + throws Exception { + MediaItem mediaItem2 = createMediaItem("mediaId2", /* buildWithUri= */ true); + Timeline timeline = + MediaTestUtils.createTimeline( + ImmutableList.of(createMediaItem("mediaId1", /* buildWithUri= */ true), mediaItem2)); + CountDownLatch timelineLatch = new CountDownLatch(1); + CountDownLatch expectedTransitionLatch = new CountDownLatch(1); + AtomicBoolean unexpectedTransitionDetected = new AtomicBoolean(false); + CountDownLatch discontinuityLatch = new CountDownLatch(2); + // Connect the controller before there are media items in the player. + MediaController controller = controllerTestRule.createController(remoteSession.getToken()); + threadTestRule + .getHandler() + .postAndSync( + () -> + controller.addListener( + new Player.Listener() { + @Override + public void onTimelineChanged( + Timeline timeline, @Player.TimelineChangeReason int reason) { + if (reason == TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED) { + timelineLatch.countDown(); + } + } + + @Override + public void onMediaItemTransition( + @Nullable MediaItem mediaItem, + @Player.MediaItemTransitionReason int reason) { + if (reason == MEDIA_ITEM_TRANSITION_REASON_PLAYLIST_CHANGED) { + expectedTransitionLatch.countDown(); + return; + } + unexpectedTransitionDetected.set(true); + } + + @Override + public void onPositionDiscontinuity( + PositionInfo oldPosition, + PositionInfo newPosition, + @Player.DiscontinuityReason int reason) { + if (reason == DISCONTINUITY_REASON_SEEK) { + discontinuityLatch.countDown(); + } + } + })); + // Move to index 1 (initial state). + remoteSession.getMockPlayer().setCurrentMediaItemIndex(/* index= */ 1); + remoteSession.getMockPlayer().setCurrentPeriodIndex(/* index= */ 1); + // Set the timeline to support a call to setMediaItems(items, 1) + remoteSession.getMockPlayer().setTimeline(timeline); + // Trigger timeline change and media item transitions in a single listener iteration. + remoteSession.getMockPlayer().notifyTimelineChangedWithMediaItemTransition(); + // Wait for the timeline update. That overrides the new position as well that was 0 in the ctrl. + assertThat(timelineLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(expectedTransitionLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + remoteSession.getMockPlayer().setCurrentPosition(20_000L); + + // Trigger seek discontinuities inside index 1. + remoteSession + .getMockPlayer() + .notifyPositionDiscontinuity( + /* oldPosition= */ new PositionInfo( + /* windowUid= */ new Object(), + /* mediaItemIndex= */ 1, + mediaItem2, + /* periodUid= */ new Object(), + /* periodIndex= */ 1, + /* positionMs= */ 10_000L, + /* contentPositionMs= */ 10_000L, + /* adGroupIndex= */ C.INDEX_UNSET, + /* adIndexInAdGroup= */ C.INDEX_UNSET), + /* newPosition= */ new PositionInfo( + /* windowUid= */ new Object(), + /* mediaItemIndex= */ 1, + mediaItem2, + /* periodUid= */ new Object(), + /* periodIndex= */ 1, + /* positionMs= */ 20_000L, + /* contentPositionMs= */ 20_000L, + /* adGroupIndex= */ C.INDEX_UNSET, + /* adIndexInAdGroup= */ C.INDEX_UNSET), + DISCONTINUITY_REASON_SEEK); + remoteSession.getMockPlayer().setCurrentPosition(30_000L); + // Trigger again to make sure we would see the media item transition in between. + remoteSession + .getMockPlayer() + .notifyPositionDiscontinuity( + /* oldPosition= */ new PositionInfo( + /* windowUid= */ new Object(), + /* mediaItemIndex= */ 1, + mediaItem2, + /* periodUid= */ new Object(), + /* periodIndex= */ 1, + /* positionMs= */ 20_000L, + /* contentPositionMs= */ 20_000L, + /* adGroupIndex= */ C.INDEX_UNSET, + /* adIndexInAdGroup= */ C.INDEX_UNSET), + /* newPosition= */ new PositionInfo( + /* windowUid= */ new Object(), + /* mediaItemIndex= */ 1, + mediaItem2, + /* periodUid= */ new Object(), + /* periodIndex= */ 1, + /* positionMs= */ 30_000L, + /* contentPositionMs= */ 30_000L, + /* adGroupIndex= */ C.INDEX_UNSET, + /* adIndexInAdGroup= */ C.INDEX_UNSET), + DISCONTINUITY_REASON_SEEK); + + // Wait for two discontinuities to be processed. + assertThat(discontinuityLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + // Verify that media item transition was NOT called for reasons other than playlist change. + assertThat(unexpectedTransitionDetected.get()).isFalse(); } @Test @@ -1551,9 +1657,7 @@ public void onEvents(Player player, Player.Events events) { })); remoteSession.getMockPlayer().setTimeline(Timeline.EMPTY); - remoteSession - .getMockPlayer() - .notifyTimelineChanged(Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); + remoteSession.getMockPlayer().notifyTimelineChanged(TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); remoteSession .getMockPlayer() .notifyMediaItemTransition( @@ -1563,7 +1667,7 @@ public void onEvents(Player player, Player.Events events) { assertThat(mediaItemFromParamRef.get()).isNull(); assertThat(mediaItemFromGetterRef.get()).isNull(); assertThat(mediaItemOnEventsRef.get()).isNull(); - assertThat(getEventsAsList(eventsRef.get())).contains(Player.EVENT_MEDIA_ITEM_TRANSITION); + assertThat(getEventsAsList(eventsRef.get())).contains(EVENT_MEDIA_ITEM_TRANSITION); } /** This also tests {@link MediaController#getPlaybackParameters()}. */ @@ -1743,7 +1847,7 @@ public void onEvents(Player player, Player.Events events) { MediaTestUtils.assertMediaIdEquals(timeline, timelineFromGetterRef.get()); MediaTestUtils.assertMediaIdEquals(timeline, timelineFromOnEventsRef.get()); assertThat(reasonRef.get()).isEqualTo(reason); - assertThat(getEventsAsList(eventsRef.get())).contains(Player.EVENT_TIMELINE_CHANGED); + assertThat(getEventsAsList(eventsRef.get())).contains(EVENT_TIMELINE_CHANGED); } @Test @@ -1790,7 +1894,7 @@ public void onEvents(Player player, Player.Events events) { assertThat(timelineFromOnEventsRef.get().getWindow(i, window).mediaItem.mediaId) .isEqualTo(TestUtils.getMediaIdInFakeTimeline(i)); } - assertThat(getEventsAsList(eventsRef.get())).contains(Player.EVENT_TIMELINE_CHANGED); + assertThat(getEventsAsList(eventsRef.get())).contains(EVENT_TIMELINE_CHANGED); } @Test @@ -1822,9 +1926,7 @@ public void onEvents(Player player, Player.Events events) { threadTestRule.getHandler().postAndSync(() -> controller.addListener(listener)); remoteSession.getMockPlayer().setTimeline(Timeline.EMPTY); - remoteSession - .getMockPlayer() - .notifyTimelineChanged(Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); + remoteSession.getMockPlayer().notifyTimelineChanged(TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); assertThat(latch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); assertThat(timelineFromParamRef.get().getWindowCount()).isEqualTo(0); @@ -1833,7 +1935,7 @@ public void onEvents(Player player, Player.Events events) { assertThat(timelineFromGetterRef.get().getPeriodCount()).isEqualTo(0); assertThat(timelineFromOnEventsRef.get().getWindowCount()).isEqualTo(0); assertThat(timelineFromOnEventsRef.get().getPeriodCount()).isEqualTo(0); - assertThat(getEventsAsList(eventsRef.get())).contains(Player.EVENT_TIMELINE_CHANGED); + assertThat(getEventsAsList(eventsRef.get())).contains(EVENT_TIMELINE_CHANGED); } /** This also tests {@link MediaController#getPlaylistMetadata()}. */ @@ -2090,7 +2192,7 @@ public void onShuffleModeEnabledChanged() throws Exception { MediaTestUtils.createMediaItems(/* size= */ 3, /* buildWithUri= */ true), /* shuffledIndices= */ new int[] {0, 2, 1}); player.setTimeline(timeline); - player.notifyTimelineChanged(Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); + player.notifyTimelineChanged(TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); player.setCurrentMediaItemIndex(2); player.setCurrentPeriodIndex(2); player.setShuffleModeEnabled(false); @@ -2149,7 +2251,7 @@ public void onRepeatModeChanged() throws Exception { Timeline timeline = MediaTestUtils.createTimeline(/* windowCount= */ 3, /* buildWithUri= */ true); player.setTimeline(timeline); - player.notifyTimelineChanged(Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); + player.notifyTimelineChanged(TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); player.setCurrentMediaItemIndex(2); player.setCurrentPeriodIndex(2); player.setRepeatMode(Player.REPEAT_MODE_OFF); @@ -2935,12 +3037,19 @@ public void onPositionDiscontinuity_isNotified() throws Exception { @Player.DiscontinuityReason int testReason = Player.DISCONTINUITY_REASON_INTERNAL; MediaController controller = controllerTestRule.createController(remoteSession.getToken()); CountDownLatch latch = new CountDownLatch(2); + CountDownLatch timelineLatch = new CountDownLatch(/* count= */ 1); AtomicReference oldPositionRef = new AtomicReference<>(); AtomicReference newPositionRef = new AtomicReference<>(); AtomicInteger positionDiscontinuityReasonRef = new AtomicInteger(); AtomicReference eventsRef = new AtomicReference<>(); Player.Listener listener = new Player.Listener() { + + @Override + public void onTimelineChanged(Timeline timeline, int reason) { + timelineLatch.countDown(); + } + @Override public void onPositionDiscontinuity( PositionInfo oldPosition, @@ -2954,11 +3063,23 @@ public void onPositionDiscontinuity( @Override public void onEvents(Player player, Player.Events events) { - eventsRef.set(events); - latch.countDown(); + if (events.contains(Player.EVENT_POSITION_DISCONTINUITY)) { + // Count down only after the discontinuity event. + eventsRef.set(events); + latch.countDown(); + } } }; threadTestRule.getHandler().postAndSync(() -> controller.addListener(listener)); + // Send down and await a timeline first. Discontinuity events are only emitted when the timeline + // is not empty. + remoteSession + .getMockPlayer() + .setTimeline(MediaTestUtils.createTimeline(/* windowCount= */ 4, /* buildWithUri= */ true)); + remoteSession.getMockPlayer().notifyTimelineChanged(TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); + assertThat(timelineLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + remoteSession.getMockPlayer().setCurrentMediaItemIndex(/* index= */ 3); + remoteSession.getMockPlayer().setCurrentPeriodIndex(/* index= */ 3); remoteSession .getMockPlayer() @@ -2969,7 +3090,8 @@ public void onEvents(Player player, Player.Events events) { assertThat(oldPositionRef.get()).isEqualTo(testOldPosition); assertThat(newPositionRef.get()).isEqualTo(testNewPosition); assertThat(getEventsAsList(eventsRef.get())) - .containsExactly(Player.EVENT_POSITION_DISCONTINUITY); + .containsExactly(Player.EVENT_MEDIA_ITEM_TRANSITION, Player.EVENT_POSITION_DISCONTINUITY) + .inOrder(); } @Test @@ -2997,8 +3119,8 @@ public void onPositionDiscontinuity_updatesGetters() throws Exception { testContentPositionMs, testCurrentAdGroupIndex, testCurrentAdIndexInAdGroup); - MediaController controller = controllerTestRule.createController(remoteSession.getToken()); + CountDownLatch timelineLatch = new CountDownLatch(1); CountDownLatch latch = new CountDownLatch(2); AtomicLong currentPositionMsRef = new AtomicLong(); AtomicLong contentPositionMsRef = new AtomicLong(); @@ -3015,6 +3137,12 @@ public void onPositionDiscontinuity_updatesGetters() throws Exception { AtomicReference eventsRef = new AtomicReference<>(); Player.Listener listener = new Player.Listener() { + + @Override + public void onTimelineChanged(Timeline timeline, int reason) { + timelineLatch.countDown(); + } + @Override public void onPositionDiscontinuity( PositionInfo oldPosition, @@ -3037,13 +3165,22 @@ public void onPositionDiscontinuity( @Override public void onEvents(Player player, Player.Events events) { - eventsRef.set(events); - latch.countDown(); + if (events.contains(Player.EVENT_POSITION_DISCONTINUITY)) { + // Count down only after the discontinuity event. + eventsRef.set(events); + latch.countDown(); + } } }; threadTestRule.getHandler().postAndSync(() -> controller.addListener(listener)); - RemoteMockPlayer remoteMockPlayer = remoteSession.getMockPlayer(); + // We need to set an initial non-empty timeline and await the timeline changed or the + // discontinuity event will not be emited. + remoteMockPlayer.setTimeline( + new PlaylistTimeline( + ImmutableList.of(new MediaItem.Builder().setUri("http://example.com").build()))); + remoteMockPlayer.notifyTimelineChanged(TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); + assertThat(timelineLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); remoteMockPlayer.setCurrentPosition(testCurrentPositionMs); remoteMockPlayer.setContentPosition(testContentPositionMs); remoteMockPlayer.setDuration(testDurationMs); @@ -3056,6 +3193,7 @@ public void onEvents(Player player, Player.Events events) { remoteMockPlayer.setIsPlayingAd(testIsPlayingAd); remoteMockPlayer.setCurrentAdGroupIndex(testCurrentAdGroupIndex); remoteMockPlayer.setCurrentAdIndexInAdGroup(testCurrentAdIndexInAdGroup); + remoteMockPlayer.notifyPositionDiscontinuity( /* oldPosition= */ SessionPositionInfo.DEFAULT_POSITION_INFO, newPositionInfo, @@ -3074,8 +3212,7 @@ public void onEvents(Player player, Player.Events events) { assertThat(isPlayingAdRef.get()).isEqualTo(testIsPlayingAd); assertThat(currentAdGroupIndexRef.get()).isEqualTo(testCurrentAdGroupIndex); assertThat(currentAdIndexInAdGroupRef.get()).isEqualTo(testCurrentAdIndexInAdGroup); - assertThat(getEventsAsList(eventsRef.get())) - .containsExactly(Player.EVENT_POSITION_DISCONTINUITY); + assertThat(getEventsAsList(eventsRef.get())).contains(Player.EVENT_POSITION_DISCONTINUITY); } /** This also tests {@link MediaController#getAvailableSessionCommands()}. */ @@ -3109,6 +3246,67 @@ public void onAvailableSessionCommandsChanged( assertThat(sessionCommandsFromGetterRef.get()).isEqualTo(commands); } + @Test + public void onPositionDiscontinuity_seekToWithEmptyTimeline_emitsPositionDiscontinuityEvent() + throws Exception { + PositionInfo testOldPosition = + new PositionInfo( + /* windowUid= */ null, + /* mediaItemIndex= */ 0, + new MediaItem.Builder().setMediaId("media-id").build(), + /* periodUid= */ null, + /* periodIndex= */ 0, + /* positionMs= */ 0L, + /* contentPositionMs= */ 0L, + /* adGroupIndex= */ C.INDEX_UNSET, + /* adIndexInAdGroup= */ C.INDEX_UNSET); + PositionInfo testNewPosition = + new PositionInfo( + /* windowUid= */ null, + /* mediaItemIndex= */ 0, + new MediaItem.Builder().setMediaId("media-id").build(), + /* periodUid= */ null, + /* periodIndex= */ 0, + /* positionMs= */ 10L, + /* contentPositionMs= */ 10L, + /* adGroupIndex= */ C.INDEX_UNSET, + /* adIndexInAdGroup= */ C.INDEX_UNSET); + MediaController controller = controllerTestRule.createController(remoteSession.getToken()); + CountDownLatch latch = new CountDownLatch(1); + AtomicReference oldPositionRef = new AtomicReference<>(); + AtomicReference newPositionRef = new AtomicReference<>(); + AtomicInteger positionDiscontinuityReasonRef = new AtomicInteger(); + threadTestRule + .getHandler() + .postAndSync( + () -> + controller.addListener( + new Player.Listener() { + + @Override + public void onPositionDiscontinuity( + PositionInfo oldPosition, + PositionInfo newPosition, + @Player.DiscontinuityReason int reason) { + oldPositionRef.set(oldPosition); + newPositionRef.set(newPosition); + positionDiscontinuityReasonRef.set(reason); + latch.countDown(); + } + })); + remoteSession.getMockPlayer().setCurrentMediaItemIndex(/* index= */ 3); + remoteSession.getMockPlayer().setCurrentPeriodIndex(/* index= */ 3); + + remoteSession + .getMockPlayer() + .notifyPositionDiscontinuity(testOldPosition, testNewPosition, DISCONTINUITY_REASON_SEEK); + + assertThat(latch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(positionDiscontinuityReasonRef.get()).isEqualTo(DISCONTINUITY_REASON_SEEK); + assertThat(oldPositionRef.get()).isEqualTo(testOldPosition); + assertThat(newPositionRef.get()).isEqualTo(testNewPosition); + } + /** This also tests {@link MediaController#getAvailableCommands()}. */ @Test public void onAvailableCommandsChanged_isCalledByPlayerChange() throws Exception { @@ -3196,7 +3394,7 @@ public void onEvents(Player player, Player.Events events) { assertThat(eventsList).hasSize(2); assertThat(getEventsAsList(eventsList.get(0))) .containsExactly(EVENT_AVAILABLE_COMMANDS_CHANGED); - assertThat(getEventsAsList(eventsList.get(1))).containsExactly(Player.EVENT_TIMELINE_CHANGED); + assertThat(getEventsAsList(eventsList.get(1))).containsExactly(EVENT_TIMELINE_CHANGED); } @Test @@ -3244,7 +3442,7 @@ public void onEvents(Player player, Player.Events events) { assertThat(eventsList).hasSize(2); assertThat(getEventsAsList(eventsList.get(0))) .containsExactly(EVENT_AVAILABLE_COMMANDS_CHANGED); - assertThat(getEventsAsList(eventsList.get(1))).containsExactly(Player.EVENT_TIMELINE_CHANGED); + assertThat(getEventsAsList(eventsList.get(1))).containsExactly(EVENT_TIMELINE_CHANGED); } /** This also tests {@link MediaController#getAvailableCommands()}. */ @@ -4739,7 +4937,7 @@ public void onTimelineChanged(Timeline timeline, int reason) { assertThat(discontinuityReceived).isTrue(); assertThat(receivedNewPosition.get().mediaItemIndex).isEqualTo(targetIndex); - remotePlayer.notifyTimelineChanged(Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); + remotePlayer.notifyTimelineChanged(TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); boolean timelineReceived = timelineLatch.await(TIMEOUT_MS, MILLISECONDS); assertThat(timelineReceived).isTrue(); assertThat(receivedTimeline.get().getWindowCount()).isEqualTo(newTimelineSize); diff --git a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaControllerWithMediaSessionCompatTest.java b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaControllerWithMediaSessionCompatTest.java index b79be12f305..d5bd599055e 100644 --- a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaControllerWithMediaSessionCompatTest.java +++ b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaControllerWithMediaSessionCompatTest.java @@ -2143,9 +2143,7 @@ public void prepare_withMetadata_callsPrepare() throws Exception { session.setPlaybackState( new PlaybackStateCompat.Builder() .setState(PlaybackStateCompat.STATE_NONE, /* position= */ 0, /* playbackSpeed= */ 0.0f) - .setActions( - PlaybackStateCompat.ACTION_PREPARE_FROM_MEDIA_ID - | PlaybackStateCompat.ACTION_PLAY_FROM_MEDIA_ID) + .setActions(PlaybackStateCompat.ACTION_PREPARE | PlaybackStateCompat.ACTION_PLAY) .build()); session.setMetadata( new MediaMetadataCompat.Builder() @@ -2188,9 +2186,7 @@ public void prepare_withMetadataAndActiveQueueItemId_callsPrepare() throws Excep new PlaybackStateCompat.Builder() .setActiveQueueItemId(4) .setState(PlaybackStateCompat.STATE_NONE, /* position= */ 0, /* playbackSpeed= */ 0.0f) - .setActions( - PlaybackStateCompat.ACTION_PREPARE_FROM_MEDIA_ID - | PlaybackStateCompat.ACTION_PLAY_FROM_MEDIA_ID) + .setActions(PlaybackStateCompat.ACTION_PREPARE | PlaybackStateCompat.ACTION_PLAY) .build()); session.setMetadata( new MediaMetadataCompat.Builder() @@ -2314,9 +2310,7 @@ public void prepare_withQueueAndMetadata_callsPrepare() throws Exception { session.setPlaybackState( new PlaybackStateCompat.Builder() .setState(PlaybackStateCompat.STATE_NONE, /* position= */ 0, /* playbackSpeed= */ 0.0f) - .setActions( - PlaybackStateCompat.ACTION_PREPARE_FROM_MEDIA_ID - | PlaybackStateCompat.ACTION_PLAY_FROM_MEDIA_ID) + .setActions(PlaybackStateCompat.ACTION_PREPARE | PlaybackStateCompat.ACTION_PLAY) .build()); session.setMetadata( new MediaMetadataCompat.Builder() diff --git a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaLibraryServiceTest.java b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaLibraryServiceTest.java index e7b7482b566..92a7ba9904a 100644 --- a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaLibraryServiceTest.java +++ b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaLibraryServiceTest.java @@ -18,6 +18,7 @@ import static android.os.Build.VERSION.SDK_INT; import static androidx.media3.test.session.common.TestUtils.TIMEOUT_MS; import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assume.assumeTrue; import android.app.UiAutomation; import android.content.ComponentName; @@ -25,6 +26,7 @@ import android.content.Intent; import android.content.pm.PackageManager; import android.content.pm.ResolveInfo; +import android.os.Build; import android.os.Bundle; import android.os.ParcelFileDescriptor; import androidx.media3.common.MediaItem; @@ -58,6 +60,10 @@ @MediumTest public class MediaLibraryServiceTest { + // Build.VERSION_CODES_FULL.CINNAMON_BUN_2 (SDK 37.2). Defined locally as a constant because + // the symbol is not present in all android-37 compile SDK stubs used by Gradle. + private static final int VERSION_CODES_FULL_CINNAMON_BUN_2 = 3700002; + @Rule public final RemoteControllerTestRule controllerTestRule = new RemoteControllerTestRule(); private Context context; @@ -132,9 +138,16 @@ public ListenableFuture> onGetItem( } @Ignore("Failing: b/517062718") - @SdkSuppress(minSdkVersion = 30) // Emulators up to API 29 don't have bluetooth service. + @SdkSuppress( + minSdkVersion = 30, + maxSdkVersion = 37) // Emulators up to API 29 don't have bluetooth service. @Test public void mediaLibraryService_isRecognizedAsAvrcpBrowsable() throws Exception { + // Bluetooth AVRCP lazy initialization (b/544875015) in Android 37.2+ trunk_staging defers + // MediaPlayerList initialization until an AVRCP device connects, so dumpsys + // bluetooth_manager no longer lists browsable packages without a connected device. + assumeTrue(getSdkIntFull() < VERSION_CODES_FULL_CINNAMON_BUN_2); + UiAutomation uiAutomation = InstrumentationRegistry.getInstrumentation().getUiAutomation(); String packageName = context.getPackageName(); @@ -216,4 +229,11 @@ private String executeShellCommand(UiAutomation uiAutomation, String command) th } return output.toString(); } + + private static int getSdkIntFull() { + if (SDK_INT < 36) { + return SDK_INT * 100_000; + } + return Build.VERSION.SDK_INT_FULL; + } } diff --git a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionCallbackWithMediaControllerCompatTest.java b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionCallbackWithMediaControllerCompatTest.java index bcaf81ee3db..e846acc0ae3 100644 --- a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionCallbackWithMediaControllerCompatTest.java +++ b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionCallbackWithMediaControllerCompatTest.java @@ -15,8 +15,6 @@ */ package androidx.media3.session; -import static android.os.Build.VERSION.SDK_INT; -import static androidx.media.MediaSessionManager.RemoteUserInfo.LEGACY_CONTROLLER; import static androidx.media3.common.Player.COMMAND_PLAY_PAUSE; import static androidx.media3.common.Player.COMMAND_PREPARE; import static androidx.media3.common.Player.COMMAND_SET_MEDIA_ITEM; @@ -100,8 +98,7 @@ public class MediaSessionCallbackWithMediaControllerCompatTest { private static final String TAG = "MSCallbackWithMCCTest"; private static final String TEST_URI = "http://test.test"; - private static final String EXPECTED_CONTROLLER_PACKAGE_NAME = - SDK_INT >= 24 ? SUPPORT_APP_PACKAGE_NAME : LEGACY_CONTROLLER; + private static final String EXPECTED_CONTROLLER_PACKAGE_NAME = SUPPORT_APP_PACKAGE_NAME; @ClassRule public static MainLooperTestRule mainLooperTestRule = new MainLooperTestRule(); diff --git a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionCompatCallbackWithMediaControllerTest.java b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionCompatCallbackWithMediaControllerTest.java index a7621ccd72a..ff4c0ab75c8 100644 --- a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionCompatCallbackWithMediaControllerTest.java +++ b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionCompatCallbackWithMediaControllerTest.java @@ -953,6 +953,32 @@ public void setMediaItems_thenPrepare_initializes() throws Exception { assertThat(sessionCallback.onPrepareFromMediaIdCalled).isTrue(); } + @Test + public void setMediaItems_thenPrepareWithOnlyPrepareFromUri_doesNotInitializeUntilPlay() + throws Exception { + MediaItem testItem = new MediaItem.Builder().setMediaId("id").build(); + session.setFlags(FLAG_HANDLES_QUEUE_COMMANDS); + setPlaybackStateAndActions( + PlaybackStateCompat.STATE_NONE, + PlaybackStateCompat.ACTION_PREPARE_FROM_URI + | PlaybackStateCompat.ACTION_PLAY_FROM_MEDIA_ID); + RemoteMediaController controller = createControllerAndWaitConnection(); + sessionCallback.reset(1); + + controller.setMediaItem(testItem); + controller.prepare(); + + assertThat(sessionCallback.await(NO_OP_TIMEOUT_MS)).isFalse(); + assertThat(sessionCallback.onPrepareFromMediaIdCalled).isFalse(); + assertThat(sessionCallback.onPlayFromMediaIdCalled).isFalse(); + + controller.play(); + + assertThat(sessionCallback.await(TIMEOUT_MS)).isTrue(); + assertThat(sessionCallback.onPlayFromMediaIdCalled).isTrue(); + assertThat(sessionCallback.mediaId).isEqualTo("id"); + } + @Test public void setMediaItems_withMediaUri_initializesWithPlayFromUri() throws Exception { Uri testUri = Uri.parse("http://test.com"); @@ -999,6 +1025,184 @@ public void setMediaItems_withSearchQuery_initializesWithPlayFromSearch() throws assertThat(sessionCallback.query).isEqualTo(testQuery); } + @Test + public void setMediaItems_pausedAndSupportsPrepareFromUri_initializesImmediately() + throws Exception { + Uri testUri = Uri.parse("https://example.com/media"); + MediaItem testItem = + new MediaItem.Builder() + .setMediaId("id") + .setRequestMetadata( + new MediaItem.RequestMetadata.Builder().setMediaUri(testUri).build()) + .build(); + session.setFlags(FLAG_HANDLES_QUEUE_COMMANDS); + setPlaybackStateAndActions( + PlaybackStateCompat.STATE_PAUSED, + PlaybackStateCompat.ACTION_PLAY + | PlaybackStateCompat.ACTION_PREPARE_FROM_URI + | PlaybackStateCompat.ACTION_PLAY_FROM_URI); + RemoteMediaController controller = createControllerAndWaitConnection(); + sessionCallback.reset(1); + + controller.setMediaItem(testItem); + + assertThat(sessionCallback.await(TIMEOUT_MS)).isTrue(); + assertThat(sessionCallback.onPrepareFromUriCalled).isTrue(); + assertThat(sessionCallback.uri).isEqualTo(testUri); + + sessionCallback.reset(1); + controller.play(); + + assertThat(sessionCallback.await(TIMEOUT_MS)).isTrue(); + assertThat(sessionCallback.onPlayCalledCount).isEqualTo(1); + assertThat(sessionCallback.onPlayFromUriCalled).isFalse(); + } + + @Test + public void setMediaItems_pausedAndSupportsPrepareFromSearch_initializesImmediately() + throws Exception { + String testQuery = "test query"; + MediaItem testItem = + new MediaItem.Builder() + .setMediaId("id") + .setRequestMetadata( + new MediaItem.RequestMetadata.Builder().setSearchQuery(testQuery).build()) + .build(); + session.setFlags(FLAG_HANDLES_QUEUE_COMMANDS); + setPlaybackStateAndActions( + PlaybackStateCompat.STATE_PAUSED, + PlaybackStateCompat.ACTION_PLAY + | PlaybackStateCompat.ACTION_PREPARE_FROM_SEARCH + | PlaybackStateCompat.ACTION_PLAY_FROM_SEARCH); + RemoteMediaController controller = createControllerAndWaitConnection(); + sessionCallback.reset(1); + + controller.setMediaItem(testItem); + + assertThat(sessionCallback.await(TIMEOUT_MS)).isTrue(); + assertThat(sessionCallback.onPrepareFromSearchCalled).isTrue(); + assertThat(sessionCallback.query).isEqualTo(testQuery); + + sessionCallback.reset(1); + controller.play(); + + assertThat(sessionCallback.await(TIMEOUT_MS)).isTrue(); + assertThat(sessionCallback.onPlayCalledCount).isEqualTo(1); + assertThat(sessionCallback.onPlayFromSearchCalled).isFalse(); + } + + @Test + public void setMediaItems_withMediaUriAndOnlyPrepareFromMediaId_doesNotInitializeUntilPlay() + throws Exception { + Uri testUri = Uri.parse("https://example.com/media"); + MediaItem testItem = + new MediaItem.Builder() + .setMediaId("id") + .setRequestMetadata( + new MediaItem.RequestMetadata.Builder().setMediaUri(testUri).build()) + .build(); + session.setFlags(FLAG_HANDLES_QUEUE_COMMANDS); + setPlaybackStateAndActions( + PlaybackStateCompat.STATE_PAUSED, + PlaybackStateCompat.ACTION_PREPARE_FROM_MEDIA_ID + | PlaybackStateCompat.ACTION_PLAY_FROM_URI); + RemoteMediaController controller = createControllerAndWaitConnection(); + sessionCallback.reset(1); + + controller.setMediaItem(testItem); + + assertThat(sessionCallback.await(NO_OP_TIMEOUT_MS)).isFalse(); + assertThat(sessionCallback.onPrepareFromUriCalled).isFalse(); + assertThat(sessionCallback.onPlayFromUriCalled).isFalse(); + + controller.play(); + + assertThat(sessionCallback.await(TIMEOUT_MS)).isTrue(); + assertThat(sessionCallback.onPlayFromUriCalled).isTrue(); + assertThat(sessionCallback.uri).isEqualTo(testUri); + } + + @Test + public void setMediaItems_withSearchQueryAndOnlyPrepareFromMediaId_doesNotInitializeUntilPlay() + throws Exception { + String testQuery = "test query"; + MediaItem testItem = + new MediaItem.Builder() + .setMediaId("id") + .setRequestMetadata( + new MediaItem.RequestMetadata.Builder().setSearchQuery(testQuery).build()) + .build(); + session.setFlags(FLAG_HANDLES_QUEUE_COMMANDS); + setPlaybackStateAndActions( + PlaybackStateCompat.STATE_PAUSED, + PlaybackStateCompat.ACTION_PREPARE_FROM_MEDIA_ID + | PlaybackStateCompat.ACTION_PLAY_FROM_SEARCH); + RemoteMediaController controller = createControllerAndWaitConnection(); + sessionCallback.reset(1); + + controller.setMediaItem(testItem); + + assertThat(sessionCallback.await(NO_OP_TIMEOUT_MS)).isFalse(); + assertThat(sessionCallback.onPrepareFromSearchCalled).isFalse(); + assertThat(sessionCallback.onPlayFromSearchCalled).isFalse(); + + controller.play(); + + assertThat(sessionCallback.await(TIMEOUT_MS)).isTrue(); + assertThat(sessionCallback.onPlayFromSearchCalled).isTrue(); + assertThat(sessionCallback.query).isEqualTo(testQuery); + } + + @Test + public void setMediaItems_withMediaIdAndOnlyPrepareFromUri_doesNotInitializeUntilPlay() + throws Exception { + MediaItem testItem = new MediaItem.Builder().setMediaId("id").build(); + session.setFlags(FLAG_HANDLES_QUEUE_COMMANDS); + setPlaybackStateAndActions( + PlaybackStateCompat.STATE_PAUSED, + PlaybackStateCompat.ACTION_PREPARE_FROM_URI + | PlaybackStateCompat.ACTION_PLAY_FROM_MEDIA_ID); + RemoteMediaController controller = createControllerAndWaitConnection(); + sessionCallback.reset(1); + + controller.setMediaItem(testItem); + + assertThat(sessionCallback.await(NO_OP_TIMEOUT_MS)).isFalse(); + assertThat(sessionCallback.onPrepareFromMediaIdCalled).isFalse(); + assertThat(sessionCallback.onPlayFromMediaIdCalled).isFalse(); + + controller.play(); + + assertThat(sessionCallback.await(TIMEOUT_MS)).isTrue(); + assertThat(sessionCallback.onPlayFromMediaIdCalled).isTrue(); + assertThat(sessionCallback.mediaId).isEqualTo("id"); + } + + @Test + public void setMediaItems_afterAlreadyInitializedWhilePaused_reinitializesOnPlay() + throws Exception { + List initialItems = + MediaTestUtils.createMediaItems(/* buildWithUri= */ false, "id1"); + MediaItem newItem = new MediaItem.Builder().setMediaId("id2").build(); + session.setQueue(MediaTestUtils.convertToQueueItemsWithoutBitmap(initialItems)); + session.setFlags(FLAG_HANDLES_QUEUE_COMMANDS); + setPlaybackStateAndActions( + PlaybackStateCompat.STATE_PAUSED, + PlaybackStateCompat.ACTION_PLAY + | PlaybackStateCompat.ACTION_PAUSE + | PlaybackStateCompat.ACTION_PLAY_FROM_MEDIA_ID); + RemoteMediaController controller = createControllerAndWaitConnection(); + sessionCallback.reset(1); + + controller.setMediaItem(newItem); + controller.play(); + + assertThat(sessionCallback.await(TIMEOUT_MS)).isTrue(); + assertThat(sessionCallback.onPlayFromMediaIdCalled).isTrue(); + assertThat(sessionCallback.onPlayCalledCount).isEqualTo(0); + assertThat(sessionCallback.mediaId).isEqualTo("id2"); + } + @Test public void setMediaItems_withStartPosition_seeksAfterPlayFromMediaId() throws Exception { long startPositionMs = 1000L; @@ -1187,6 +1391,8 @@ private static class MediaSessionCallback extends MediaSessionCompat.Callback { private boolean onCustomActionCalled; private boolean onPrepareCalled; private boolean onPrepareFromMediaIdCalled; + private boolean onPrepareFromSearchCalled; + private boolean onPrepareFromUriCalled; private boolean onSetRepeatModeCalled; private boolean onSetShuffleModeCalled; private int onAddQueueItemAtCalledCount; @@ -1226,6 +1432,8 @@ private void reset(int count) { onCustomActionCalled = false; onPrepareCalled = false; onPrepareFromMediaIdCalled = false; + onPrepareFromSearchCalled = false; + onPrepareFromUriCalled = false; onSetRepeatModeCalled = false; onSetShuffleModeCalled = false; onAddQueueItemAtCalledCount = 0; @@ -1358,6 +1566,7 @@ public void onPrepareFromMediaId(String mediaId, Bundle extras) { @Override public void onPrepareFromSearch(String query, Bundle extras) { + onPrepareFromSearchCalled = true; this.query = query; this.extras = extras; latch.countDown(); @@ -1365,6 +1574,7 @@ public void onPrepareFromSearch(String query, Bundle extras) { @Override public void onPrepareFromUri(Uri uri, Bundle extras) { + onPrepareFromUriCalled = true; this.uri = uri; this.extras = extras; latch.countDown(); diff --git a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionKeyEventTest.java b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionKeyEventTest.java index 0f902af472d..204fa2cb85c 100644 --- a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionKeyEventTest.java +++ b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionKeyEventTest.java @@ -16,7 +16,6 @@ package androidx.media3.session; import static android.os.Build.VERSION.SDK_INT; -import static androidx.media.MediaSessionManager.RemoteUserInfo.LEGACY_CONTROLLER; import static androidx.media3.common.Player.STATE_ENDED; import static androidx.media3.session.MediaSession.ControllerInfo.LEGACY_CONTROLLER_VERSION; import static androidx.media3.test.session.common.CommonConstants.SUPPORT_APP_PACKAGE_NAME; @@ -465,12 +464,9 @@ private static String getExpectedControllerPackageName(ControllerInfo controller if (SDK_INT >= 28) { // Above API 28: package of the app using AudioManager. return SUPPORT_APP_PACKAGE_NAME; - } else if (SDK_INT >= 24) { + } else { // API 24 - 27: KeyEvent from system service has the package name "android". return "android"; - } else { - // API 23: Fallback set by MediaSessionCompat#getCurrentControllerInfo - return LEGACY_CONTROLLER; } } diff --git a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionServiceTest.java b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionServiceTest.java index 32880fefe4c..6c8cd9c93f4 100644 --- a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionServiceTest.java +++ b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionServiceTest.java @@ -55,6 +55,7 @@ import androidx.media3.common.Player.Listener; import androidx.media3.common.Player.PositionInfo; import androidx.media3.common.util.ConditionVariable; +import androidx.media3.common.util.NullableType; import androidx.media3.common.util.Util; import androidx.media3.exoplayer.ExoPlayer; import androidx.media3.session.MediaLibraryService.MediaLibrarySession; @@ -70,6 +71,7 @@ import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; import androidx.test.filters.MediumTest; +import androidx.test.filters.SdkSuppress; import androidx.test.platform.app.InstrumentationRegistry; import com.google.common.collect.ImmutableList; import com.google.common.collect.Lists; @@ -221,6 +223,54 @@ public void onDisconnected( .isSameInstanceAs(connectedControllerManagerControllerInfos.get(0)); } + @Test + @SdkSuppress(minSdkVersion = 31, maxSdkVersion = 32) + public void play_serviceNotInForeground_getControllerForCurrentRequestReturnsCallingController() + throws Exception { + TestServiceRegistry testServiceRegistry = TestServiceRegistry.getInstance(); + AtomicReference onGetSessionControllerInfo = new AtomicReference<>(); + SettableFuture<@NullableType ControllerInfo> playCommandControllerInfo = + SettableFuture.create(); + AtomicReference session = new AtomicReference<>(); + testServiceRegistry.setOnGetSessionHandler( + controllerInfo -> { + if (session.get() != null) { + return session.get(); + } + onGetSessionControllerInfo.set(controllerInfo); + Player player = new ExoPlayer.Builder(context).build(); + player.setMediaItem(MediaItem.fromUri("asset:///media/mp4/sample.mp4")); + player.prepare(); + ForwardingPlayer forwardingPlayer = + new ForwardingPlayer(player) { + @Override + public void play() { + playCommandControllerInfo.set(session.get().getControllerForCurrentRequest()); + } + }; + session.set(new MediaSession.Builder(context, forwardingPlayer).build()); + return session.get(); + }); + RemoteMediaController controller = + controllerTestRule.createRemoteController( + token, /* waitForConnection= */ true, /* connectionHints= */ Bundle.EMPTY); + try { + controller.play(); + + assertThat(playCommandControllerInfo.get(TIMEOUT_MS, MILLISECONDS)) + .isEqualTo(onGetSessionControllerInfo.get()); + } finally { + controller.release(); + MainLooperTestRule.runOnMainSync( + () -> { + if (session.get() != null) { + session.get().getPlayer().release(); + session.get().release(); + } + }); + } + } + @Test public void controllerRelease_keepsControllerBoundUntilCommandsHandled() throws Exception { TestServiceRegistry testServiceRegistry = TestServiceRegistry.getInstance(); diff --git a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionTest.java b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionTest.java index 0964485c3af..b2fe2025901 100644 --- a/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionTest.java +++ b/libraries/test_session_current/src/androidTest/java/androidx/media3/session/MediaSessionTest.java @@ -15,7 +15,6 @@ */ package androidx.media3.session; -import static android.os.Build.VERSION.SDK_INT; import static android.view.KeyEvent.KEYCODE_HEADSETHOOK; import static android.view.KeyEvent.KEYCODE_MEDIA_FAST_FORWARD; import static android.view.KeyEvent.KEYCODE_MEDIA_NEXT; @@ -25,7 +24,10 @@ import static android.view.KeyEvent.KEYCODE_MEDIA_PREVIOUS; import static android.view.KeyEvent.KEYCODE_MEDIA_REWIND; import static android.view.KeyEvent.KEYCODE_MEDIA_STOP; +import static androidx.media3.common.Player.DISCONTINUITY_REASON_SEEK; +import static androidx.media3.common.Player.MEDIA_ITEM_TRANSITION_REASON_PLAYLIST_CHANGED; import static androidx.media3.common.Player.STATE_IDLE; +import static androidx.media3.common.Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED; import static androidx.media3.test.session.common.TestUtils.LONG_TIMEOUT_MS; import static androidx.media3.test.session.common.TestUtils.TIMEOUT_MS; import static androidx.test.core.app.ApplicationProvider.getApplicationContext; @@ -47,12 +49,18 @@ import android.os.SystemClock; import android.text.TextUtils; import android.view.KeyEvent; +import androidx.annotation.Nullable; import androidx.media.MediaSessionManager; +import androidx.media3.common.C; import androidx.media3.common.ForwardingPlayer; +import androidx.media3.common.MediaItem; import androidx.media3.common.MediaLibraryInfo; import androidx.media3.common.Player; +import androidx.media3.common.Player.PositionInfo; import androidx.media3.common.Timeline; import androidx.media3.common.util.Log; +import androidx.media3.exoplayer.ExoPlayer; +import androidx.media3.exoplayer.source.SilenceMediaSource; import androidx.media3.session.MediaSession.ControllerInfo; import androidx.media3.test.session.common.HandlerThreadTestRule; import androidx.media3.test.session.common.MainLooperTestRule; @@ -63,6 +71,7 @@ import androidx.test.filters.LargeTest; import androidx.test.filters.SdkSuppress; import androidx.test.platform.app.InstrumentationRegistry; +import com.google.common.collect.ImmutableList; import com.google.common.util.concurrent.FutureCallback; import com.google.common.util.concurrent.Futures; import com.google.common.util.concurrent.ListenableFuture; @@ -539,8 +548,7 @@ public void getPlatformToken_returnsCompatibleWithPlatformMediaController() thro public ListenableFuture onConnectAsync( MediaSession session, ControllerInfo controller) { if (TextUtils.equals( - getControllerCallerPackageName(controller), - controller.getPackageName())) { + context.getPackageName(), controller.getPackageName())) { return MediaSession.Callback.super.onConnectAsync(session, controller); } return immediateFuture(MediaSession.ConnectionResult.reject()); @@ -790,8 +798,7 @@ controllerInfo, getMediaButtonIntent(KEYCODE_HEADSETHOOK))) public ListenableFuture onConnectAsync( MediaSession session, ControllerInfo controller) { if (TextUtils.equals( - getControllerCallerPackageName(controller), - controller.getPackageName())) { + context.getPackageName(), controller.getPackageName())) { return MediaSession.Callback.super.onConnectAsync(session, controller); } return immediateFuture(MediaSession.ConnectionResult.reject()); @@ -860,7 +867,7 @@ controllerInfo, getMediaButtonIntent(KEYCODE_MEDIA_STOP))) for (ControllerInfo info : callerCollectorPlayer.callingControllers) { assertThat(session.get().isMediaNotificationController(info)).isFalse(); assertThat(info.getControllerVersion()).isEqualTo(ControllerInfo.LEGACY_CONTROLLER_VERSION); - assertThat(info.getPackageName()).isEqualTo(getControllerCallerPackageName(info)); + assertThat(info.getPackageName()).isEqualTo(context.getPackageName()); } } @@ -880,8 +887,7 @@ controllerInfo, getMediaButtonIntent(KEYCODE_MEDIA_STOP))) public ListenableFuture onConnectAsync( MediaSession session, ControllerInfo controller) { if (TextUtils.equals( - getControllerCallerPackageName(controller), - controller.getPackageName())) { + context.getPackageName(), controller.getPackageName())) { return MediaSession.Callback.super.onConnectAsync(session, controller); } return immediateFuture(MediaSession.ConnectionResult.reject()); @@ -1297,25 +1303,116 @@ public void testGetBitmapDimensionLimit() { assertThat(limit).isAtMost(4096); } - /** - * Returns the expected {@link MediaSessionManager.RemoteUserInfo#getPackageName()} of a - * controller hosted in the test companion app. - * - *

    After API 23 the package name is {@link Context#getPackageName()} of the {@link - * ApplicationProvider#getApplicationContext() application under test}. - * - *

    On API 23, the platform MediaSession doesn't report the caller package name. Instead the - * package of the RemoteUserInfo is set for all external controllers to the same {@code - * MediaSessionManager.RemoteUserInfo.LEGACY_CONTROLLER} (see - * MediaSessionCompat.MediaSessionCallback.setCurrentControllerInfo()). - * - *

    Calling this method should only be required to test legacy behaviour. - */ - private static String getControllerCallerPackageName(ControllerInfo controllerInfo) { - return (SDK_INT > 23 - || controllerInfo.getControllerVersion() != ControllerInfo.LEGACY_CONTROLLER_VERSION) - ? ApplicationProvider.getApplicationContext().getPackageName() - : MediaSessionManager.RemoteUserInfo.LEGACY_CONTROLLER; + @Test + public void setMediaItems_withMediaItemIndexNotZero_silentPositionUpdateOnTimelineChange() + throws Exception { + AtomicReference player = new AtomicReference<>(); + try { + player.set( + new ExoPlayer.Builder(ApplicationProvider.getApplicationContext()) + .setLooper(handler.getLooper()) + .build()); + MediaSession testSession = + sessionTestRule.ensureReleaseAfterTest( + new MediaSession.Builder(context, player.get()) + .setId("setMediaItems_withMediaItemIndexNotZero_") + .build()); + CountDownLatch disconnectedLatch = new CountDownLatch(1); + MediaController testController = + new MediaController.Builder(context, testSession.getToken()) + .setListener( + new MediaController.Listener() { + @Override + public void onDisconnected(MediaController controller) { + disconnectedLatch.countDown(); + } + }) + .setApplicationLooper(handler.getLooper()) + .buildAsync() + .get(TIMEOUT_MS, MILLISECONDS); + assertThat(testController.isConnected()).isTrue(); + CountDownLatch timelineLatch = new CountDownLatch(/* count= */ 1); + CountDownLatch discontinuityLatch = new CountDownLatch(/* count= */ 2); + CountDownLatch expectedMediaItemTransitionLatch = new CountDownLatch(/* count= */ 1); + AtomicBoolean unexpectedMediaItemTransitionCall = new AtomicBoolean(false); + List discontinuityMediaItemIndices = new ArrayList<>(); + List discontinuityReasons = new ArrayList<>(); + List transitionReasons = new ArrayList<>(); + handler.postAndSync( + () -> { + testController.addListener( + new Player.Listener() { + @Override + public void onTimelineChanged( + Timeline timeline, @Player.TimelineChangeReason int reason) { + if (reason == TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED) { + timelineLatch.countDown(); + } + } + + @Override + public void onMediaItemTransition( + @Nullable MediaItem mediaItem, @Player.MediaItemTransitionReason int reason) { + transitionReasons.add(reason); + if (reason == MEDIA_ITEM_TRANSITION_REASON_PLAYLIST_CHANGED) { + expectedMediaItemTransitionLatch.countDown(); + return; + } + unexpectedMediaItemTransitionCall.set(true); + } + + @Override + public void onPositionDiscontinuity( + PositionInfo oldPosition, + PositionInfo newPosition, + @Player.DiscontinuityReason int reason) { + discontinuityReasons.add(reason); + discontinuityMediaItemIndices.add(oldPosition.mediaItemIndex); + discontinuityMediaItemIndices.add(newPosition.mediaItemIndex); + if (reason == DISCONTINUITY_REASON_SEEK) { + discontinuityLatch.countDown(); + } + } + }); + }); + // Set the media source with start position that triggers the initial events. + handler.postAndSync( + () -> { + SilenceMediaSource silenceMediaSource1 = + new SilenceMediaSource(/* durationUs= */ 10_000_000L); + SilenceMediaSource silenceMediaSource2 = + new SilenceMediaSource(/* durationUs= */ 20_000_000L); + player + .get() + .setMediaSources( + ImmutableList.of(silenceMediaSource1, silenceMediaSource2), + /* startMediaItemIndex= */ 1, + /* startPositionMs= */ C.TIME_UNSET); + }); + assertThat(timelineLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(expectedMediaItemTransitionLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(discontinuityMediaItemIndices).isEmpty(); + assertThat(unexpectedMediaItemTransitionCall.get()).isFalse(); + + // No media item transition because the media item index was silently updated. + handler.postAndSync( + () -> { + player.get().seekTo(/* mediaItemIndex= */ 1, /* positionMs= */ 2_000L); + }); + handler.postAndSync( + () -> { + player.get().seekTo(/* mediaItemIndex= */ 1, /* positionMs= */ 10_000_000L); + }); + + assertThat(discontinuityLatch.await(TIMEOUT_MS, MILLISECONDS)).isTrue(); + assertThat(unexpectedMediaItemTransitionCall.get()).isFalse(); + assertThat(transitionReasons).containsExactly(MEDIA_ITEM_TRANSITION_REASON_PLAYLIST_CHANGED); + assertThat(discontinuityMediaItemIndices).containsExactly(1, 1, 1, 1); + assertThat(discontinuityReasons) + .containsExactly(DISCONTINUITY_REASON_SEEK, DISCONTINUITY_REASON_SEEK); + } finally { + threadTestRule.getHandler().postAndSync(() -> player.get().release()); + } } private static ControllerInfo createMediaButtonCaller() { diff --git a/libraries/test_session_current/src/main/java/androidx/media3/session/MediaSessionProviderService.java b/libraries/test_session_current/src/main/java/androidx/media3/session/MediaSessionProviderService.java index 55be276beac..11a957dc6dc 100644 --- a/libraries/test_session_current/src/main/java/androidx/media3/session/MediaSessionProviderService.java +++ b/libraries/test_session_current/src/main/java/androidx/media3/session/MediaSessionProviderService.java @@ -108,6 +108,7 @@ import androidx.media3.common.Player.DiscontinuityReason; import androidx.media3.common.Player.PositionInfo; import androidx.media3.common.Timeline; +import androidx.media3.common.Timeline.Window; import androidx.media3.common.TrackGroup; import androidx.media3.common.TrackSelectionParameters; import androidx.media3.common.Tracks; @@ -1460,6 +1461,24 @@ public void notifyTimelineChanged(String sessionId, @Player.TimelineChangeReason }); } + @Override + public void notifyTimelineChangedWithMediaItemTransition(String sessionId) + throws RemoteException { + runOnHandler( + () -> { + MediaSession session = sessionMap.get(sessionId); + MockPlayer player = (MockPlayer) session.getPlayer(); + player.notifyTimelineChanged(Player.TIMELINE_CHANGE_REASON_PLAYLIST_CHANGED); + MediaItem mediaItem = null; + if (player.currentMediaItemIndex < player.timeline.getWindowCount()) { + mediaItem = + player.timeline.getWindow(player.currentMediaItemIndex, new Window()).mediaItem; + } + player.notifyMediaItemTransition( + mediaItem, Player.MEDIA_ITEM_TRANSITION_REASON_PLAYLIST_CHANGED); + }); + } + @Override public void notifyPlaylistMetadataChanged(String sessionId) throws RemoteException { runOnHandler( diff --git a/libraries/test_session_current/src/main/java/androidx/media3/session/MockMediaLibraryService.java b/libraries/test_session_current/src/main/java/androidx/media3/session/MockMediaLibraryService.java index e5023ed4000..20d40bd38b2 100644 --- a/libraries/test_session_current/src/main/java/androidx/media3/session/MockMediaLibraryService.java +++ b/libraries/test_session_current/src/main/java/androidx/media3/session/MockMediaLibraryService.java @@ -66,6 +66,7 @@ import static androidx.media3.test.session.common.MediaBrowserConstants.SUBSCRIBE_PARENT_ID_2; import static androidx.media3.test.session.common.MediaSessionConstants.CONNECTION_HINT_KEY_ASYNC_CONNECTION_DELAY_MS; import static androidx.media3.test.session.common.MediaSessionConstants.CONNECTION_HINT_KEY_ASYNC_CONNECTION_REJECT_DELAY_MS; +import static androidx.media3.test.session.common.MediaSessionConstants.CONNECTION_HINT_KEY_ASYNC_LIBRARY_ROOT_DELAY_MS; import static androidx.media3.test.session.common.MediaSessionConstants.EXTRA_KEY_ASYNC_CONNECTION_CONFIRMATION; import static androidx.media3.test.session.common.MediaSessionConstants.KEY_CONTROLLER; import static com.google.common.util.concurrent.Futures.immediateFuture; @@ -400,6 +401,16 @@ public ListenableFuture> onGetLibraryRoot( rootExtras.putBoolean(EXTRA_KEY_ASYNC_CONNECTION_CONFIRMATION, true); rootParams = new LibraryParams.Builder().setExtras(rootExtras).build(); } + if (params != null + && params.extras.containsKey(CONNECTION_HINT_KEY_ASYNC_LIBRARY_ROOT_DELAY_MS)) { + long delayMs = params.extras.getLong(CONNECTION_HINT_KEY_ASYNC_LIBRARY_ROOT_DELAY_MS); + SettableFuture> future = SettableFuture.create(); + MediaItem finalRootItem = rootItem; + LibraryParams finalRootParams = rootParams; + handler.postDelayed( + () -> future.set(LibraryResult.ofItem(finalRootItem, finalRootParams)), delayMs); + return future; + } return immediateFuture(LibraryResult.ofItem(rootItem, rootParams)); } diff --git a/libraries/test_session_current/src/main/java/androidx/media3/session/RemoteMediaSession.java b/libraries/test_session_current/src/main/java/androidx/media3/session/RemoteMediaSession.java index 5c48e247c50..bb69768108b 100644 --- a/libraries/test_session_current/src/main/java/androidx/media3/session/RemoteMediaSession.java +++ b/libraries/test_session_current/src/main/java/androidx/media3/session/RemoteMediaSession.java @@ -447,6 +447,10 @@ public void notifyTimelineChanged(@Player.TimelineChangeReason int reason) binder.notifyTimelineChanged(sessionId, reason); } + public void notifyTimelineChangedWithMediaItemTransition() throws RemoteException { + binder.notifyTimelineChangedWithMediaItemTransition(sessionId); + } + public void notifyPlaylistMetadataChanged() throws RemoteException { binder.notifyPlaylistMetadataChanged(sessionId); } diff --git a/libraries/test_utils/src/androidTest/java/androidx/media3/test/utils/CountDownFutureTest.java b/libraries/test_utils/src/androidTest/java/androidx/media3/test/utils/CountDownFutureTest.java new file mode 100644 index 00000000000..53318efe020 --- /dev/null +++ b/libraries/test_utils/src/androidTest/java/androidx/media3/test/utils/CountDownFutureTest.java @@ -0,0 +1,85 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.test.utils; + +import static com.google.common.truth.Truth.assertThat; +import static java.util.concurrent.TimeUnit.SECONDS; +import static org.junit.Assert.assertThrows; + +import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link CountDownFuture}. */ +@RunWith(AndroidJUnit4.class) +public final class CountDownFutureTest { + + @Test + public void countDown_targetCountReached_completes() throws Exception { + CountDownFuture future = new CountDownFuture(3); + assertThat(future.isDone()).isFalse(); + + future.countDown(); + assertThat(future.isDone()).isFalse(); + + future.countDown(); + assertThat(future.isDone()).isFalse(); + + future.countDown(); + assertThat(future.isDone()).isTrue(); + assertThat(future.get()).isNull(); + } + + @Test + public void zeroCount_completesImmediately() throws Exception { + CountDownFuture zeroFuture = new CountDownFuture(0); + assertThat(zeroFuture.isDone()).isTrue(); + assertThat(zeroFuture.get()).isNull(); + } + + @Test + public void negativeCount_throwsIllegalArgumentException() { + assertThrows(IllegalArgumentException.class, () -> new CountDownFuture(-5)); + } + + @Test + public void countDown_excessCalls_isHarmlessNoOp() throws Exception { + CountDownFuture future = new CountDownFuture(1); + future.countDown(); + assertThat(future.isDone()).isTrue(); + + future.countDown(); + future.countDown(); + assertThat(future.isDone()).isTrue(); + } + + @Test + public void countDown_concurrentCalls_completesSafely() throws Exception { + int limit = 50; + CountDownFuture future = new CountDownFuture(limit); + ExecutorService executor = Executors.newFixedThreadPool(10); + try { + for (int i = 0; i < limit; i++) { + executor.execute(future::countDown); + } + future.get(5, SECONDS); + } finally { + executor.shutdownNow(); + } + } +} diff --git a/libraries/test_utils/src/androidTest/java/androidx/media3/test/utils/PlayerFenceJavaTest.java b/libraries/test_utils/src/androidTest/java/androidx/media3/test/utils/PlayerFenceJavaTest.java index 41e034e4b74..66852aaf3b1 100644 --- a/libraries/test_utils/src/androidTest/java/androidx/media3/test/utils/PlayerFenceJavaTest.java +++ b/libraries/test_utils/src/androidTest/java/androidx/media3/test/utils/PlayerFenceJavaTest.java @@ -109,7 +109,7 @@ public void entersPlaybackState_timeout_cancelsFuture() throws Exception { Stopwatch stopwatch = Stopwatch.createStarted(new ElapsedRealtimeTicker()); assertThrows(CancellationException.class, playerReadyFuture::get); - assertThat(stopwatch.elapsed(MILLISECONDS)).isLessThan(200); + assertThat(stopwatch.elapsed(MILLISECONDS)).isLessThan(1000); } @Test @@ -199,6 +199,7 @@ public void isPlaying() throws Exception { isPlayingFuture.setFuture(futureWhen(player).isPlaying(true)); player.setMediaItem(SHORT_MP3_ITEM); + player.setRepeatMode(Player.REPEAT_MODE_ALL); player.prepare(); player.play(); }); @@ -267,4 +268,31 @@ public void passesContentPosition() throws Exception { player.get().release(); }); } + + @Test + public void withSupervision_customFuture_completesWhenResolved() throws Exception { + SettableFuture trigger = SettableFuture.create(); + SettableFuture playerReadyFuture = SettableFuture.create(); + SettableFuture supervisedFuture = SettableFuture.create(); + getInstrumentation() + .runOnMainSync( + () -> { + player = + new ExoPlayer.Builder(getInstrumentation().getContext().getApplicationContext()) + .build(); + playerReadyFuture.setFuture( + futureWhen(player).entersPlaybackState(Player.STATE_READY)); + supervisedFuture.setFuture(futureWhen(player).withSupervision(trigger)); + + player.setMediaItem(SHORT_MP3_ITEM); + player.prepare(); + player.play(); + }); + + playerReadyFuture.get(); + assertThat(supervisedFuture.isDone()).isFalse(); + + trigger.set(null); + supervisedFuture.get(); + } } diff --git a/libraries/test_utils/src/androidTest/java/androidx/media3/test/utils/PlayerFenceTest.kt b/libraries/test_utils/src/androidTest/java/androidx/media3/test/utils/PlayerFenceTest.kt index 76422442f15..e0773864db8 100644 --- a/libraries/test_utils/src/androidTest/java/androidx/media3/test/utils/PlayerFenceTest.kt +++ b/libraries/test_utils/src/androidTest/java/androidx/media3/test/utils/PlayerFenceTest.kt @@ -34,6 +34,7 @@ import androidx.media3.exoplayer.ExoPlayer import androidx.media3.exoplayer.FormatHolder import androidx.media3.exoplayer.Renderer import androidx.media3.exoplayer.RenderersFactory +import androidx.media3.exoplayer.analytics.AnalyticsListener import androidx.media3.exoplayer.audio.AudioRendererEventListener import androidx.media3.exoplayer.audio.AudioSink import androidx.media3.exoplayer.audio.MediaCodecAudioRenderer @@ -46,8 +47,10 @@ import com.google.common.truth.Truth.assertThat import com.google.common.truth.Truth.assertWithMessage import java.util.concurrent.TimeUnit import java.util.concurrent.atomic.AtomicBoolean +import kotlin.coroutines.resume import kotlin.test.assertFailsWith import kotlin.time.Duration.Companion.milliseconds +import kotlin.time.Duration.Companion.seconds import kotlinx.coroutines.CompletableDeferred import kotlinx.coroutines.CoroutineScope import kotlinx.coroutines.CoroutineStart @@ -56,9 +59,16 @@ import kotlinx.coroutines.TimeoutCancellationException import kotlinx.coroutines.android.HandlerDispatcher import kotlinx.coroutines.android.asCoroutineDispatcher import kotlinx.coroutines.async +import kotlinx.coroutines.channels.awaitClose import kotlinx.coroutines.delay +import kotlinx.coroutines.flow.callbackFlow +import kotlinx.coroutines.flow.filter +import kotlinx.coroutines.flow.take +import kotlinx.coroutines.flow.toList import kotlinx.coroutines.launch import kotlinx.coroutines.runBlocking +import kotlinx.coroutines.suspendCancellableCoroutine +import kotlinx.coroutines.withTimeout import kotlinx.coroutines.yield import org.junit.After import org.junit.Test @@ -187,7 +197,7 @@ class PlayerFenceTest { assertFailsWith { player.awaitPlaybackState(Player.STATE_READY, timeout = 100.milliseconds) } - assertThat(stopwatch.elapsed(TimeUnit.MILLISECONDS)).isLessThan(200) + assertThat(stopwatch.elapsed(TimeUnit.MILLISECONDS)).isLessThan(1000) player.release() } @@ -256,7 +266,6 @@ class PlayerFenceTest { val player = ExoPlayer.Builder(getInstrumentation().context.applicationContext).build() player.setMediaItem(SHORT_MP3_ITEM) player.prepare() - player.play() try { player.awaitPlaybackState(Player.STATE_READY) @@ -273,6 +282,7 @@ class PlayerFenceTest { runBlocking(Dispatchers.Main) { player = ExoPlayer.Builder(getInstrumentation().context.applicationContext).build() player.setMediaItem(SHORT_MP3_ITEM) + player.repeatMode = Player.REPEAT_MODE_ALL player.prepare() val isPlaying = async { player.awaitIsPlaying(true) } @@ -313,6 +323,7 @@ class PlayerFenceTest { runBlocking(Dispatchers.Main) { player = ExoPlayer.Builder(getInstrumentation().context.applicationContext).build() player.setMediaItem(SHORT_MP3_ITEM) + player.repeatMode = Player.REPEAT_MODE_ALL player.prepare() player.play() val playerPlaying = CompletableDeferred() @@ -356,6 +367,7 @@ class PlayerFenceTest { .setRenderersFactory(FailingAudioRenderer.Factory()) .build() player.setMediaItem(SHORT_MP3_ITEM) + player.repeatMode = Player.REPEAT_MODE_ALL player.prepare() player.play() @@ -425,8 +437,7 @@ class PlayerFenceTest { player.prepare() player.play() - val exception = - assertFailsWith(IllegalStateException::class) { player.awaitFirstFrameRendered() } + val exception = assertFailsWith { player.awaitFirstFrameRendered() } assertThat(exception).hasMessageThat().contains("FailingAudioRenderer") player.clearVideoSurface() @@ -549,7 +560,7 @@ class PlayerFenceTest { player.play() val exception = - assertFailsWith(IllegalStateException::class) { player.awaitContentPositionAtLeast(5_000) } + assertFailsWith { player.awaitContentPositionAtLeast(5_000) } assertThat(exception).hasMessageThat().contains("Playback ended at position") assertThat(exception).hasMessageThat().contains("before target of 5000ms") } @@ -568,7 +579,7 @@ class PlayerFenceTest { player.play() val exception = - assertFailsWith(IllegalStateException::class) { player.awaitContentPositionAtLeast(500) } + assertFailsWith { player.awaitContentPositionAtLeast(500) } assertThat(exception) .hasMessageThat() .contains("Playback left item 0 before reaching position 500m") @@ -586,7 +597,7 @@ class PlayerFenceTest { player.play() val throwable = - assertFailsWith(IllegalStateException::class) { + assertFailsWith { player.awaitContentPositionAtLeast(targetPositionMs = 500) } assertThat(throwable).hasMessageThat().contains("FailingAudioRenderer") @@ -608,6 +619,134 @@ class PlayerFenceTest { assertThat(player.contentPosition).isAtLeast(500) } + @Test + fun awaitSupervised_withOneShotDeferred() = + runBlocking(Dispatchers.Main) { + player = ExoPlayer.Builder(getInstrumentation().context.applicationContext).build() + val audioPlayoutStartSystemTimeMs = CompletableDeferred() + val listener = + object : AnalyticsListener { + override fun onAudioPositionAdvancing( + eventTime: AnalyticsListener.EventTime, + playoutStartSystemTimeMs: Long, + ) { + audioPlayoutStartSystemTimeMs.complete(playoutStartSystemTimeMs) + } + } + player.addAnalyticsListener(listener) + + player.setMediaItem(SHORT_MP3_ITEM) + player.repeatMode = Player.REPEAT_MODE_ALL + player.prepare() + player.play() + + val advancingTime = player.awaitSupervised { audioPlayoutStartSystemTimeMs.await() } + assertThat(advancingTime).isGreaterThan(0) + // For completeness, we could remove the listener in a try-finally block, but the player + // will get released at the end of the test anyway, so no need. The other example tests below + // with suspendCancellableCoroutine and callbackFlow enforce/facilitate this clean-up more + // explicitly so we do it there as a demonstration. + } + + /** + * This test shows how [Player.awaitSupervised] can be used with [suspendCancellableCoroutine]. + */ + @Test + fun awaitSupervised_withSuspendCancellableCoroutine_resolvesValue() = + runBlocking(Dispatchers.Main) { + val exoPlayer = ExoPlayer.Builder(getInstrumentation().context.applicationContext).build() + player = exoPlayer + exoPlayer.setMediaItem(SHORT_MP3_ITEM) + exoPlayer.repeatMode = Player.REPEAT_MODE_ALL + exoPlayer.prepare() + exoPlayer.play() + + val audioPlayoutStartTimeSystemTimeMs = player.awaitSupervised { + suspendCancellableCoroutine { continuation -> + val listener = + object : AnalyticsListener { + override fun onAudioPositionAdvancing( + eventTime: AnalyticsListener.EventTime, + playoutStartSystemTimeMs: Long, + ) { + if (continuation.isActive) { + continuation.resume(playoutStartSystemTimeMs) + } + } + } + exoPlayer.addAnalyticsListener(listener) + continuation.invokeOnCancellation { exoPlayer.removeAnalyticsListener(listener) } + } + } + + assertThat(audioPlayoutStartTimeSystemTimeMs).isGreaterThan(0) + } + + /** + * This test shows how to use a [callbackFlow] with [Player.awaitSupervised], which works well for + * use-cases that need to count multiple events or collect multiple values. + */ + @Test + fun awaitSupervised_withCallbackFlow() = + runBlocking(Dispatchers.Main) { + player = ExoPlayer.Builder(getInstrumentation().context.applicationContext).build() + val flow = callbackFlow { + val listener = + object : Player.Listener { + override fun onPlaybackStateChanged(playbackState: Int) { + trySend(playbackState).getOrThrow() + } + } + player.addListener(listener) + awaitClose { player.removeListener(listener) } + } + + player.setMediaItem(SHORT_MP3_ITEM) + player.prepare() + player.play() + + val transitions = player.awaitSupervised { + flow.filter { it == Player.STATE_READY || it == Player.STATE_ENDED }.take(2).toList() + } + + assertThat(transitions).containsExactly(Player.STATE_READY, Player.STATE_ENDED).inOrder() + } + + @Test + fun awaitSupervised_innerTimeout_timesOutAndCancelsFast() = + runBlocking(Dispatchers.Main) { + player = ExoPlayer.Builder(getInstrumentation().context.applicationContext).build() + player.setMediaItem(SHORT_MP3_ITEM) + player.prepare() + player.play() + + val stopwatch = Stopwatch.createStarted(ElapsedRealtimeTicker()) + assertFailsWith { + player.awaitSupervised(timeout = 10.seconds) { + withTimeout(100.milliseconds) { + CompletableDeferred().await() // Never completes + } + } + } + // Should fail fast (~100ms), far before the outer 10s timeout + assertThat(stopwatch.elapsed(TimeUnit.MILLISECONDS)).isLessThan(1000) + } + + @Test + fun awaitSupervised_playbackExceptionWhileSuspended_throwsPlaybackException(): Unit = + runBlocking(Dispatchers.Main) { + player = ExoPlayer.Builder(getInstrumentation().context.applicationContext).build() + player.setMediaItem(MediaItem.fromUri("file:///not/a/real/file")) + + assertFailsWith { + player.awaitSupervised(failOnNonFatalErrors = false) { + player.prepare() + // Block forever to give a chance for the playback error to propagate. + CompletableDeferred().await() + } + } + } + private fun CoroutineScope.assertDoesntSuspend(block: suspend () -> Unit) { var didSuspend = true // Use UNDISPATCHED so that control-flow returns to the outer scope immediately if `block()` diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/AssetInfo.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/AssetInfo.java index dd30152ccab..448e5128ed5 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/AssetInfo.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/AssetInfo.java @@ -717,6 +717,25 @@ public AssetInfo build() { .setVideoDurationUs(300_000) .build(); + public static final AssetInfo MP4_ASSET_COLOR_TEST_720P_STRIP_HLG = + new AssetInfo.Builder("asset:///media/mp4/hlg_1280x720_primary_gradient.mp4") + .setVideoFormat( + new Format.Builder() + .setSampleMimeType(VIDEO_H265) + .setWidth(1280) + .setHeight(720) + .setFrameRate(30.000f) + .setColorInfo( + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorRange(C.COLOR_RANGE_LIMITED) + .setColorTransfer(C.COLOR_TRANSFER_HLG) + .build()) + .setCodecs("hvc1.2.4.L93.90") + .build()) + .setVideoDurationUs(1_000_000) + .build(); + public static final AssetInfo MP4_ASSET_720P_4_SECOND_HDR10 = new AssetInfo.Builder("asset:///media/mp4/hdr10-720p.mp4") .setVideoFormat( diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/BindCanaryMode.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/BindCanaryMode.java new file mode 100644 index 00000000000..5d5a6ad152d --- /dev/null +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/BindCanaryMode.java @@ -0,0 +1,38 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.test.utils; + +import static java.lang.annotation.ElementType.FIELD; +import static java.lang.annotation.RetentionPolicy.RUNTIME; + +import androidx.media3.common.util.UnstableApi; +import com.google.errorprone.annotations.Keep; +import java.lang.annotation.Documented; +import java.lang.annotation.Retention; +import java.lang.annotation.Target; + +/** + * Binds a boolean test parameter field to Media3 canary mode. + * + *

    When used in conjunction with {@link Media3FlagsRule}, the rule will automatically enable or + * disable canary mode before each test based on the field's value. + */ +@Documented +@Retention(RUNTIME) +@Target(FIELD) +@Keep +@UnstableApi +public @interface BindCanaryMode {} diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/BindFlag.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/BindFlag.java new file mode 100644 index 00000000000..12f1d890d1c --- /dev/null +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/BindFlag.java @@ -0,0 +1,44 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.test.utils; + +import static java.lang.annotation.ElementType.FIELD; +import static java.lang.annotation.RetentionPolicy.RUNTIME; + +import androidx.media3.common.Flags; +import androidx.media3.common.util.UnstableApi; +import com.google.errorprone.annotations.Keep; +import java.lang.annotation.Documented; +import java.lang.annotation.Retention; +import java.lang.annotation.Target; + +/** + * Binds a boolean test parameter field to a Media3 feature flag. + * + *

    When used in conjunction with {@link Media3FlagsRule}, the rule will automatically enable or + * disable the specified flag before each test based on the field's value. + */ +@Documented +@Retention(RUNTIME) +@Target(FIELD) +@Keep +@UnstableApi +public @interface BindFlag { + + /** The {@link Flags.Flag} identifier. */ + @Flags.Flag + int value(); +} diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/BinderStressCreator.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/BinderStressCreator.java index edcec07578e..444ce38749e 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/BinderStressCreator.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/BinderStressCreator.java @@ -24,6 +24,7 @@ import android.content.Context; import android.media.AudioManager; import android.net.ConnectivityManager; +import android.net.Network; import androidx.media3.common.C; import androidx.media3.common.util.Consumer; import androidx.media3.common.util.UnstableApi; @@ -118,8 +119,11 @@ public BinderStressCreator(Context context) throws InterruptedException { // process and are unlikely to work without binder calls even in the future. ConnectivityManager connectivityManager = (ConnectivityManager) context.getSystemService(Context.CONNECTIVITY_SERVICE); - checkNotNull(connectivityManager) - .requestBandwidthUpdate(checkNotNull(connectivityManager.getActiveNetwork())); + checkNotNull(connectivityManager); + Network activeNetwork = connectivityManager.getActiveNetwork(); + if (activeNetwork != null) { + connectivityManager.requestBandwidthUpdate(activeNetwork); + } AudioManager audioManager = (AudioManager) context.getSystemService(Context.AUDIO_SERVICE); checkNotNull(audioManager).setStreamVolume(C.STREAM_TYPE_DEFAULT, 1, 0); diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/BitmapPixelTestUtil.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/BitmapPixelTestUtil.java index 92fc6169bb8..e59208a499a 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/BitmapPixelTestUtil.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/BitmapPixelTestUtil.java @@ -23,6 +23,7 @@ import static com.google.common.truth.Truth.assertThat; import static java.lang.Math.abs; import static java.lang.Math.max; +import static java.lang.Math.min; import android.content.Context; import android.graphics.Bitmap; @@ -35,8 +36,10 @@ import android.opengl.GLES20; import android.opengl.GLES30; import android.os.Build; +import android.util.Half; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; +import androidx.media3.common.GlTextureInfo; import androidx.media3.common.util.GlUtil; import androidx.media3.common.util.Log; import androidx.media3.common.util.UnstableApi; @@ -47,6 +50,9 @@ import java.io.OutputStream; import java.lang.reflect.Method; import java.nio.ByteBuffer; +import java.nio.ByteOrder; +import java.nio.FloatBuffer; +import java.nio.ShortBuffer; import java.util.Arrays; /** Utilities for pixel tests. */ @@ -284,6 +290,41 @@ public static Bitmap createArgb8888BitmapWithSolidColor(int width, int height, i return Bitmap.createBitmap(colors, width, height, Bitmap.Config.ARGB_8888); } + /** + * Returns a solid {@link Bitmap.Config#RGBA_F16} {@link Bitmap} with every pixel having the same + * color and an alpha value of {@code 1.0f}. + * + * @param width The width of image to create, in pixels. + * @param height The height of image to create, in pixels. + * @param r The red component. + * @param g The green component. + * @param b The blue component. + */ + @RequiresApi(26) + @SuppressWarnings("HalfFloat") + public static Bitmap createFp16BitmapWithSolidColor( + int width, int height, float r, float g, float b) { + long rHalf = (long) Half.toHalf(r) & 0xFFFFL; + long gHalf = (long) Half.toHalf(g) & 0xFFFFL; + long bHalf = (long) Half.toHalf(b) & 0xFFFFL; + long aHalf = (long) Half.toHalf(1.0f) & 0xFFFFL; + long pixelColor; + if (ByteOrder.nativeOrder() == ByteOrder.LITTLE_ENDIAN) { + pixelColor = rHalf | (gHalf << 16) | (bHalf << 32) | (aHalf << 48); + } else { + pixelColor = (rHalf << 48) | (gHalf << 32) | (bHalf << 16) | aHalf; + } + int numPixels = width * height; + long[] pixels = new long[numPixels]; + Arrays.fill(pixels, pixelColor); + ByteBuffer buffer = ByteBuffer.allocateDirect(numPixels * 8).order(ByteOrder.nativeOrder()); + buffer.asLongBuffer().put(pixels); + buffer.rewind(); + Bitmap bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.RGBA_F16); + bitmap.copyPixelsFromBuffer(buffer); + return bitmap; + } + /** * Returns the average difference between the expected and actual bitmaps. * @@ -526,8 +567,82 @@ public static Bitmap createUnpremultipliedArgb8888BitmapFromFocusedGlFramebuffer @RequiresApi(26) // Bitmap.Config.RGBA_F16 public static Bitmap createFp16BitmapFromFocusedGlFramebuffer(int width, int height) throws GlUtil.GlException { + int channelCount = 4; + int bytesPerFp16 = 2; + int pixelSize = channelCount * bytesPerFp16; + int[] readType = new int[1]; + GLES20.glGetIntegerv(GLES20.GL_IMPLEMENTATION_COLOR_READ_TYPE, readType, /* offset= */ 0); + GlUtil.checkGlError(); + if (readType[0] == GLES20.GL_FLOAT) { + int floatsPerRow = width * channelCount; + int bytesPerRow = floatsPerRow * Float.BYTES; + // Cap the direct float buffer size to avoid OutOfMemoryError. + int maxBufferBytes = 16 * 1024 * 1024; + int rowsPerRead = max(1, min(height, maxBufferBytes / bytesPerRow)); + FloatBuffer floatBuffer = + ByteBuffer.allocateDirect(rowsPerRead * bytesPerRow) + .order(ByteOrder.nativeOrder()) + .asFloatBuffer(); + ByteBuffer halfBuffer = + ByteBuffer.allocateDirect(width * height * pixelSize).order(ByteOrder.nativeOrder()); + ShortBuffer shortBuffer = halfBuffer.asShortBuffer(); + for (int y = 0; y < height; y += rowsPerRead) { + int rowsToRead = min(rowsPerRead, height - y); + floatBuffer.rewind(); + GLES20.glReadPixels( + /* x= */ 0, y, width, rowsToRead, GLES20.GL_RGBA, GLES20.GL_FLOAT, floatBuffer); + GlUtil.checkGlError(); + floatBuffer.rewind(); + for (int r = 0; r < rowsToRead; r++) { + // Flip vertically: OpenGL row (y + r) from the bottom maps to Bitmap row + // (height - 1 - (y + r)) from the top. + shortBuffer.position((height - 1 - (y + r)) * floatsPerRow); + copyFloatsToHalfBuffer(floatBuffer, shortBuffer, floatsPerRow); + } + } + halfBuffer.rewind(); + Bitmap bitmap = Bitmap.createBitmap(width, height, Bitmap.Config.RGBA_F16); + bitmap.copyPixelsFromBuffer(halfBuffer); + return bitmap; + } return createBitmapFromFocusedGlFrameBuffer( - width, height, /* pixelSize= */ 8, GLES30.GL_HALF_FLOAT, Bitmap.Config.RGBA_F16); + width, height, pixelSize, GLES30.GL_HALF_FLOAT, Bitmap.Config.RGBA_F16); + } + + /** + * Creates a {@link Bitmap.Config#ARGB_8888} bitmap with the values of the specified {@link + * GlTextureInfo}. + * + *

    Must be called with an active OpenGL context on the current thread. + */ + public static Bitmap createArgb8888BitmapFromGlTexture(GlTextureInfo glTextureInfo) + throws GlUtil.GlException { + int fboId = GlUtil.createFboForTexture(glTextureInfo.texId); + try { + GlUtil.focusFramebufferUsingCurrentContext(fboId, glTextureInfo.width, glTextureInfo.height); + return createArgb8888BitmapFromFocusedGlFramebuffer( + glTextureInfo.width, glTextureInfo.height); + } finally { + GlUtil.deleteFbo(fboId); + } + } + + /** + * Creates a {@link Bitmap.Config#RGBA_F16} bitmap with the values of the specified {@link + * GlTextureInfo}. + * + *

    Must be called with an active OpenGL context on the current thread. + */ + @RequiresApi(26) + public static Bitmap createFp16BitmapFromGlTexture(GlTextureInfo glTextureInfo) + throws GlUtil.GlException { + int fboId = GlUtil.createFboForTexture(glTextureInfo.texId); + try { + GlUtil.focusFramebufferUsingCurrentContext(fboId, glTextureInfo.width, glTextureInfo.height); + return createFp16BitmapFromFocusedGlFramebuffer(glTextureInfo.width, glTextureInfo.height); + } finally { + GlUtil.deleteFbo(fboId); + } } private static Bitmap createBitmapFromFocusedGlFrameBuffer( @@ -546,6 +661,15 @@ private static Bitmap createBitmapFromFocusedGlFrameBuffer( return flipBitmapVertically(bitmap); } + @RequiresApi(26) // Half.toHalf + @SuppressWarnings("HalfFloat") // Writes 16-bit half-precision floats into the buffer. + private static void copyFloatsToHalfBuffer( + FloatBuffer source, ShortBuffer destination, int count) { + for (int i = 0; i < count; i++) { + destination.put(Half.toHalf(source.get())); + } + } + /** * Creates a {@link GLES20#GL_TEXTURE_2D 2-dimensional OpenGL texture} with the bitmap's contents. * diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/CapturingAudioSink.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/CapturingAudioSink.java index 9555bc2d05b..96d8e40bed5 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/CapturingAudioSink.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/CapturingAudioSink.java @@ -54,6 +54,9 @@ public class CapturingAudioSink extends ForwardingAudioSink implements Dumper.Du private long lastPresentationTimeUs; @Nullable private ByteBuffer currentBuffer; private @MonotonicNonNull Format format; + private boolean isPlaying; + private boolean audioTrackFull; + private long firstPendingPresentationTimeUs; /** Creates the capturing audio sink. */ public static CapturingAudioSink create() { @@ -89,6 +92,7 @@ protected CapturingAudioSink(AudioSink sink, boolean shouldCaptureIndividualBuff audioSink = sink; interceptedData = new ArrayList<>(); this.shouldCaptureIndividualBuffers = shouldCaptureIndividualBuffers; + firstPendingPresentationTimeUs = C.TIME_UNSET; } /** Returns the wrapped {@link AudioSink}. */ @@ -115,6 +119,23 @@ public void handleDiscontinuity() { public boolean handleBuffer( ByteBuffer buffer, long presentationTimeUs, int encodedAccessUnitCount) throws InitializationException, WriteException { + // Simulate a finite AudioTrack buffer: while not playing, only buffer up to one AudioTrack + // buffer worth of audio ahead of the first pending sample before reporting the sink as full. + // A real AudioTrack applies this backpressure once its buffer fills while paused; the fake + // AudioTrack used in tests otherwise accepts everything, letting the codec decode the whole + // stream while paused (see b/174737370). + if (!isPlaying) { + long audioTrackBufferSizeUs = getAudioTrackBufferSizeUs(); + if (firstPendingPresentationTimeUs == C.TIME_UNSET) { + firstPendingPresentationTimeUs = presentationTimeUs; + } else if (audioTrackBufferSizeUs != C.TIME_UNSET + && audioTrackBufferSizeUs > 0 + && presentationTimeUs - firstPendingPresentationTimeUs >= audioTrackBufferSizeUs) { + audioTrackFull = true; + return false; + } + } + audioTrackFull = false; lastPresentationTimeUs = presentationTimeUs; // The handleBuffer is called repeatedly with the same buffer until it's been fully consumed by // the sink. We only want to dump each buffer once. @@ -132,6 +153,40 @@ public boolean handleBuffer( return fullyBuffered; } + @Override + public boolean hasPendingData() { + return audioTrackFull || super.hasPendingData(); + } + + @Override + public void play() { + isPlaying = true; + audioTrackFull = false; + firstPendingPresentationTimeUs = C.TIME_UNSET; + super.play(); + } + + @Override + public void pause() { + isPlaying = false; + super.pause(); + } + + @Override + public void flush() { + audioTrackFull = false; + firstPendingPresentationTimeUs = C.TIME_UNSET; + super.flush(); + } + + @Override + public void reset() { + isPlaying = false; + audioTrackFull = false; + firstPendingPresentationTimeUs = C.TIME_UNSET; + super.reset(); + } + @Override public void dump(Dumper dumper) { if (interceptedData.isEmpty()) { diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/CapturingFrameProcessor.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/CapturingFrameProcessor.java new file mode 100644 index 00000000000..68564b99305 --- /dev/null +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/CapturingFrameProcessor.java @@ -0,0 +1,224 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.test.utils; + +import static com.google.common.base.Preconditions.checkNotNull; +import static com.google.common.base.Preconditions.checkState; +import static com.google.common.collect.ImmutableList.toImmutableList; + +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import androidx.media3.common.C; +import androidx.media3.common.util.Consumer; +import androidx.media3.common.util.ExperimentalApi; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.FrameProcessor; +import androidx.media3.common.video.FrameWriter; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.Iterables; +import java.util.List; +import java.util.Objects; +import java.util.concurrent.CopyOnWriteArrayList; +import java.util.concurrent.Executor; + +/** + * A {@link FrameProcessor} that delegates to an underlying {@link FrameProcessor} and records + * queued frames and EOS signals for testing. + */ +@RequiresApi(26) +@ExperimentalApi // TODO: b/498176910 Remove once FrameProcessor is production ready. +public class CapturingFrameProcessor implements FrameProcessor { + + /** A factory for {@link CapturingFrameProcessor} instances. */ + public static class Factory implements FrameProcessor.Factory { + + private final FrameProcessor.Factory underlyingFactory; + @Nullable private final Consumer> onQueueListener; + @Nullable private volatile CapturingFrameProcessor createdProcessor; + + /** + * Creates a new instance. + * + * @param underlyingFactory The factory that creates the underlying {@link FrameProcessor}. + */ + public Factory(FrameProcessor.Factory underlyingFactory) { + this(underlyingFactory, /* onQueueListener= */ null); + } + + /** + * Creates a new instance. + * + * @param underlyingFactory The factory that creates the underlying {@link FrameProcessor}. + * @param onQueueListener A listener that is invoked after each successful {@link #queue(List)} + * call. + */ + public Factory( + FrameProcessor.Factory underlyingFactory, + @Nullable Consumer> onQueueListener) { + this.underlyingFactory = checkNotNull(underlyingFactory); + this.onQueueListener = onQueueListener; + } + + @Override + public CapturingFrameProcessor create( + FrameWriter output, Executor listenerExecutor, Listener listener) { + checkState(createdProcessor == null, "Factory can only create one processor"); + FrameProcessor underlyingProcessor = + underlyingFactory.create(output, listenerExecutor, listener); + createdProcessor = new CapturingFrameProcessor(underlyingProcessor, onQueueListener); + return checkNotNull(createdProcessor); + } + + /** Returns the created {@link CapturingFrameProcessor} instance. */ + @Nullable + public CapturingFrameProcessor getCreatedProcessor() { + return createdProcessor; + } + } + + /** Marker interface for ordering frames and EOS signals. */ + public interface Event {} + + /** Holds the frames queued in a {@link #queue} call. */ + public static final class FramesEvent implements Event { + public final ImmutableList frames; + + /** + * Creates a new instance. + * + * @param frames The frames to record. Will be defensively copied. + */ + public FramesEvent(List frames) { + this.frames = ImmutableList.copyOf(frames); + } + + /** + * {@inheritDoc} + * + *

    Performs an identity-based comparison of the frames in the list, as {@link AsyncFrame} + * does not implement {@code equals()}. + */ + @Override + public boolean equals(@Nullable Object o) { + if (this == o) { + return true; + } + if (o == null || getClass() != o.getClass()) { + return false; + } + FramesEvent that = (FramesEvent) o; + return frames.equals(that.frames); + } + + @Override + public int hashCode() { + return Objects.hashCode(frames); + } + + @Override + public String toString() { + ImmutableList contentTimesUs = + frames.stream() + .map(asyncFrame -> asyncFrame.frame.getContentTimeUs()) + .collect(toImmutableList()); + return "FramesEvent(" + contentTimesUs + ")"; + } + } + + /** Marker class representing a {@link #signalEndOfStream()} call. */ + public static final class EosEvent implements Event { + @Override + public boolean equals(@Nullable Object o) { + return o instanceof EosEvent; + } + + @Override + public int hashCode() { + return EosEvent.class.hashCode(); + } + + @Override + public String toString() { + return "EOS"; + } + } + + private final FrameProcessor underlyingProcessor; + @Nullable private final Consumer> onQueueListener; + + /** All the frames and EOS events queued to this {@link FrameProcessor}. */ + private final CopyOnWriteArrayList queuedEvents; + + private CapturingFrameProcessor( + FrameProcessor underlyingProcessor, @Nullable Consumer> onQueueListener) { + this.underlyingProcessor = underlyingProcessor; + this.onQueueListener = onQueueListener; + this.queuedEvents = new CopyOnWriteArrayList<>(); + } + + @Override + public boolean queue(List frames) { + FramesEvent event = new FramesEvent(frames); + boolean queued = underlyingProcessor.queue(frames); + if (queued) { + queuedEvents.add(event); + if (onQueueListener != null) { + onQueueListener.accept(frames); + } + } + return queued; + } + + @Override + public void signalEndOfStream() { + queuedEvents.add(new EosEvent()); + underlyingProcessor.signalEndOfStream(); + } + + @Override + public void close() { + underlyingProcessor.close(); + } + + /** Returns true if the last {@link Event} queued was an {@link EosEvent}. */ + public boolean isEnded() { + return !queuedEvents.isEmpty() && Iterables.getLast(queuedEvents) instanceof EosEvent; + } + + /** Returns an {@link ImmutableList} of all queued events. */ + public ImmutableList getQueuedEvents() { + return ImmutableList.copyOf(queuedEvents); + } + + /** + * Returns a list of timestamp lists for all queued events. The list for an EOS event contains a + * single {@link C#TIME_UNSET} value. + */ + public ImmutableList> getQueuedContentTimesUs() { + return queuedEvents.stream() + .map( + event -> { + if (event instanceof EosEvent) { + return ImmutableList.of(C.TIME_UNSET); + } + FramesEvent framesEvent = (FramesEvent) event; + return framesEvent.frames.stream() + .map(asyncFrame -> asyncFrame.frame.getContentTimeUs()) + .collect(toImmutableList()); + }) + .collect(toImmutableList()); + } +} diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/CompositionAssetInfo.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/CompositionAssetInfo.java index 7b414f3a37a..22ffaea4aa7 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/CompositionAssetInfo.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/CompositionAssetInfo.java @@ -197,6 +197,15 @@ public Composition getComposition() { .build(); } + public boolean hasVideo() { + for (SequenceAssetInfo sequence : sequences) { + if (sequence.trackTypes.contains(C.TRACK_TYPE_VIDEO)) { + return true; + } + } + return false; + } + public ImmutableList getExpectedVideoTimestampsUs() { // When there are multiple sequences, output timestamps should match those of the primary // sequence. @@ -226,7 +235,7 @@ public Format getVideoEncoderInputFormat() { return null; } // If the first asset is a gap, the encoder will be configured with [16, 16], HEVC. - if (firstAsset.editedMediaItem != null && firstAsset.editedMediaItem.removeVideo) { + if (firstAsset.getEditedMediaItem().removeVideo) { format = format .buildUpon() diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/CountDownFuture.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/CountDownFuture.java new file mode 100644 index 00000000000..134bd7a140d --- /dev/null +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/CountDownFuture.java @@ -0,0 +1,55 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.test.utils; + +import static com.google.common.base.Preconditions.checkArgument; +import static java.lang.Math.max; + +import androidx.media3.common.util.UnstableApi; +import com.google.common.util.concurrent.AbstractFuture; +import com.google.common.util.concurrent.ListenableFuture; +import java.util.concurrent.atomic.AtomicInteger; + +/** + * A {@link ListenableFuture} that completes when {@link #countDown()} has been called a specified + * number of times. + */ +@UnstableApi +public final class CountDownFuture extends AbstractFuture { + private final AtomicInteger remaining; + + /** + * Creates an instance initialized with the specified count. + * + * @param count The number of times {@link #countDown()} must be invoked before completing. If 0, + * the future completes immediately. + * @throws IllegalArgumentException if count is negative. + */ + public CountDownFuture(int count) { + checkArgument(count >= 0, "count < 0"); + this.remaining = new AtomicInteger(count); + if (count == 0) { + set(null); + } + } + + /** Decrements the remaining count, completing the future when it reaches 0. */ + public void countDown() { + if (remaining.getAndUpdate(v -> max(0, v - 1)) == 1) { + set(null); + } + } +} diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/DecoderCountersUtil.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/DecoderCountersUtil.java index a7bd04fda33..64db64c3bd7 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/DecoderCountersUtil.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/DecoderCountersUtil.java @@ -40,14 +40,38 @@ public static int getTotalBufferCount(DecoderCounters counters) { + counters.renderedOutputBufferCount; } + /** + * Asserts that the skipped output buffer count is {@code expected}. + * + * @param name The name of the decoder. + * @param counters The decoder counters. + * @param expected The expected number of skipped output buffers. + */ public static void assertSkippedOutputBufferCount( String name, DecoderCounters counters, int expected) { + assertSkippedOutputBufferCount(name, counters, expected, /* tolerance= */ 0); + } + + /** + * Asserts that the skipped output buffer count is within {@code tolerance} of {@code expected}. + * + * @param name The name of the decoder. + * @param counters The decoder counters. + * @param expected The expected number of skipped output buffers. + * @param tolerance The allowed tolerance around the expected count. + */ + public static void assertSkippedOutputBufferCount( + String name, DecoderCounters counters, int expected, int tolerance) { counters.ensureUpdated(); int actual = counters.skippedOutputBufferCount; assertWithMessage( - "Codec(%s) skipped an unexpected number of buffers. Counters:\n%s", name, counters) + "Codec(%s) skipped unexpected number of buffers (expected=%s, tolerance=%s)." + + " Counters:\n" + + "%s", + name, expected, tolerance, counters) .that(actual) - .isEqualTo(expected); + .isWithin(tolerance) + .of(expected); } /** Asserts that the input and output values in {@code counters} are self-consistent. */ diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/EditedMediaItemAssetInfo.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/EditedMediaItemAssetInfo.java index d9854506644..ac91454dcaa 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/EditedMediaItemAssetInfo.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/EditedMediaItemAssetInfo.java @@ -18,6 +18,7 @@ import static androidx.media3.common.util.Util.usToMs; import static androidx.media3.test.utils.AssetInfo.MP4_ADVANCED_ASSET; import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_SRGB; +import static androidx.media3.test.utils.AssetInfo.MP4_SIMPLE_ASSET; import static androidx.media3.test.utils.AssetInfo.MP4_VIDEO_ONLY_ASSET; import static androidx.media3.test.utils.AssetInfo.PNG_ASSET; import static androidx.media3.test.utils.AssetInfo.WAV_ASSET; @@ -34,6 +35,7 @@ import androidx.media3.common.util.UnstableApi; import androidx.media3.transformer.EditedMediaItem; import androidx.media3.transformer.Effects; +import com.google.common.base.Supplier; import com.google.common.collect.ImmutableList; import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.util.Collection; @@ -46,7 +48,7 @@ public final class EditedMediaItemAssetInfo { /** A builder for {@link EditedMediaItemAssetInfo} instances. */ private static final class Builder { private @MonotonicNonNull AssetInfo originalAssetInfo; - private @MonotonicNonNull EditedMediaItem editedMediaItem; + private @MonotonicNonNull Supplier editedMediaItemSupplier; private @MonotonicNonNull String name; @Nullable private ImmutableList videoTimestampsUs; @@ -59,10 +61,12 @@ public Builder setOriginalAssetInfo(AssetInfo originalAssetInfo) { return this; } - /** Sets the {@link EditedMediaItem} containing the transformations to apply. */ + /** + * Sets the {@link Supplier} of {@link EditedMediaItem} containing the transformations to apply. + */ @CanIgnoreReturnValue - public Builder setEditedMediaItem(EditedMediaItem editedMediaItem) { - this.editedMediaItem = editedMediaItem; + public Builder setEditedMediaItemSupplier(Supplier editedMediaItemSupplier) { + this.editedMediaItemSupplier = editedMediaItemSupplier; return this; } @@ -83,7 +87,7 @@ public Builder setName(String name) { /** Creates an {@link EditedMediaItemAssetInfo} instance. */ public EditedMediaItemAssetInfo build() { checkNotNull(originalAssetInfo); - checkNotNull(editedMediaItem); + checkNotNull(editedMediaItemSupplier); checkNotNull(name); return new EditedMediaItemAssetInfo(this); } @@ -123,15 +127,16 @@ public long getNextSpeedChangeTimeUs(long timeUs) { public static final EditedMediaItemAssetInfo IMAGE = new EditedMediaItemAssetInfo.Builder() .setOriginalAssetInfo(PNG_ASSET) - .setEditedMediaItem( - new EditedMediaItem.Builder( - new MediaItem.Builder() - .setUri(PNG_ASSET.uri) - .setImageDurationMs(usToMs(/* timeUs= */ 500_000)) - .build()) - .setDurationUs(500_000) - .setFrameRate(30) - .build()) + .setEditedMediaItemSupplier( + () -> + new EditedMediaItem.Builder( + new MediaItem.Builder() + .setUri(PNG_ASSET.uri) + .setImageDurationMs(usToMs(/* timeUs= */ 500_000)) + .build()) + .setDurationUs(500_000) + .setFrameRate(30) + .build()) .setVideoTimestampsUs( ImmutableList.of( 0L, 33_333L, 66_667L, 100_000L, 133_333L, 166_667L, 200_000L, 233_333L, 266_667L, @@ -139,15 +144,16 @@ public long getNextSpeedChangeTimeUs(long timeUs) { .setName("Image") .build(); - /** An {@link EditedMediaItemAssetInfo} wrapper around {@link AssetInfo#MP4_ADVANCED_ASSET}. */ + /** An {@link EditedMediaItemAssetInfo} wrapper around {@link AssetInfo#MP4_SIMPLE_ASSET}. */ public static final EditedMediaItemAssetInfo VIDEO = new EditedMediaItemAssetInfo.Builder() - .setOriginalAssetInfo(MP4_ADVANCED_ASSET) - .setEditedMediaItem( - new EditedMediaItem.Builder( - new MediaItem.Builder().setUri(MP4_ADVANCED_ASSET.uri).build()) - .setDurationUs(MP4_ADVANCED_ASSET.videoDurationUs) - .build()) + .setOriginalAssetInfo(MP4_SIMPLE_ASSET) + .setEditedMediaItemSupplier( + () -> + new EditedMediaItem.Builder( + new MediaItem.Builder().setUri(MP4_SIMPLE_ASSET.uri).build()) + .setDurationUs(MP4_SIMPLE_ASSET.videoDurationUs) + .build()) .setName("Video") .build(); @@ -155,11 +161,12 @@ public long getNextSpeedChangeTimeUs(long timeUs) { public static final EditedMediaItemAssetInfo VIDEO_SRGB = new EditedMediaItemAssetInfo.Builder() .setOriginalAssetInfo(MP4_ASSET_SRGB) - .setEditedMediaItem( - new EditedMediaItem.Builder( - new MediaItem.Builder().setUri(MP4_ASSET_SRGB.uri).build()) - .setDurationUs(MP4_ASSET_SRGB.videoDurationUs) - .build()) + .setEditedMediaItemSupplier( + () -> + new EditedMediaItem.Builder( + new MediaItem.Builder().setUri(MP4_ASSET_SRGB.uri).build()) + .setDurationUs(MP4_ASSET_SRGB.videoDurationUs) + .build()) .setName("Video_srgb") .build(); @@ -170,11 +177,12 @@ public long getNextSpeedChangeTimeUs(long timeUs) { public static final EditedMediaItemAssetInfo VIDEO_WITHOUT_AUDIO = new EditedMediaItemAssetInfo.Builder() .setOriginalAssetInfo(MP4_ADVANCED_ASSET) - .setEditedMediaItem( - new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)) - .setDurationUs(MP4_ADVANCED_ASSET.videoDurationUs) - .setRemoveAudio(true) - .build()) + .setEditedMediaItemSupplier( + () -> + new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)) + .setDurationUs(MP4_ADVANCED_ASSET.videoDurationUs) + .setRemoveAudio(true) + .build()) .setName("Video_no_audio") .build(); @@ -196,15 +204,17 @@ public long getNextSpeedChangeTimeUs(long timeUs) { public static final EditedMediaItemAssetInfo VIDEO_ONLY_CLIPPED_TWICE_SPEED = new EditedMediaItemAssetInfo.Builder() .setOriginalAssetInfo(MP4_VIDEO_ONLY_ASSET) - .setEditedMediaItem( - new EditedMediaItem.Builder(VIDEO_ONLY_CLIPPED_MEDIA_ITEM) - .setDurationUs(MP4_VIDEO_ONLY_ASSET.videoDurationUs) - .setRemoveAudio(true) - .setEffects( - new Effects( - /* audioProcessors= */ ImmutableList.of(), - /* videoEffects= */ ImmutableList.of(TWICE_SPEED_CHANGE_EFFECTS.second))) - .build()) + .setEditedMediaItemSupplier( + () -> + new EditedMediaItem.Builder(VIDEO_ONLY_CLIPPED_MEDIA_ITEM) + .setDurationUs(MP4_VIDEO_ONLY_ASSET.videoDurationUs) + .setRemoveAudio(true) + .setEffects( + new Effects( + /* audioProcessors= */ ImmutableList.of(), + /* videoEffects= */ ImmutableList.of( + TWICE_SPEED_CHANGE_EFFECTS.second))) + .build()) .setVideoTimestampsUs( ImmutableList.of( 250L, 16933L, 33617L, 50300L, 66983L, 83667L, 100350L, 117033L, 133717L, 150400L, @@ -219,15 +229,17 @@ public long getNextSpeedChangeTimeUs(long timeUs) { public static final EditedMediaItemAssetInfo VIDEO_ONLY_CLIPPED_HALF_SPEED = new EditedMediaItemAssetInfo.Builder() .setOriginalAssetInfo(MP4_VIDEO_ONLY_ASSET) - .setEditedMediaItem( - new EditedMediaItem.Builder(VIDEO_ONLY_CLIPPED_MEDIA_ITEM) - .setDurationUs(MP4_VIDEO_ONLY_ASSET.videoDurationUs) - .setRemoveAudio(true) - .setEffects( - new Effects( - /* audioProcessors= */ ImmutableList.of(), - /* videoEffects= */ ImmutableList.of(HALF_SPEED_CHANGE_EFFECTS.second))) - .build()) + .setEditedMediaItemSupplier( + () -> + new EditedMediaItem.Builder(VIDEO_ONLY_CLIPPED_MEDIA_ITEM) + .setDurationUs(MP4_VIDEO_ONLY_ASSET.videoDurationUs) + .setRemoveAudio(true) + .setEffects( + new Effects( + /* audioProcessors= */ ImmutableList.of(), + /* videoEffects= */ ImmutableList.of( + HALF_SPEED_CHANGE_EFFECTS.second))) + .build()) .setVideoTimestampsUs( ImmutableList.of( 1000L, 67732L, 134466L, 201200L, 267932L, 334666L, 401400L, 468132L, 534866L, @@ -239,10 +251,11 @@ public long getNextSpeedChangeTimeUs(long timeUs) { public static final EditedMediaItemAssetInfo AUDIO = new EditedMediaItemAssetInfo.Builder() .setOriginalAssetInfo(WAV_ASSET) - .setEditedMediaItem( - new EditedMediaItem.Builder(MediaItem.fromUri(WAV_ASSET.uri)) - .setDurationUs(1_000_000) - .build()) + .setEditedMediaItemSupplier( + () -> + new EditedMediaItem.Builder(MediaItem.fromUri(WAV_ASSET.uri)) + .setDurationUs(1_000_000) + .build()) .setVideoTimestampsUs(ImmutableList.of()) .setName("Audio") .build(); @@ -254,10 +267,11 @@ public long getNextSpeedChangeTimeUs(long timeUs) { public static final EditedMediaItemAssetInfo AUDIO_WITH_VIDEO_TIMESTAMPS = new EditedMediaItemAssetInfo.Builder() .setOriginalAssetInfo(WAV_ASSET) - .setEditedMediaItem( - new EditedMediaItem.Builder(MediaItem.fromUri(WAV_ASSET.uri)) - .setDurationUs(1_000_000) - .build()) + .setEditedMediaItemSupplier( + () -> + new EditedMediaItem.Builder(MediaItem.fromUri(WAV_ASSET.uri)) + .setDurationUs(1_000_000) + .build()) .setVideoTimestampsUs( ImmutableList.of( 0L, 33_333L, 66_667L, 100_000L, 133_333L, 166_667L, 200_000L, 233_333L, 266_667L, @@ -274,11 +288,12 @@ public long getNextSpeedChangeTimeUs(long timeUs) { public static final EditedMediaItemAssetInfo VIDEO_WITH_REMOVE_VIDEO = new EditedMediaItemAssetInfo.Builder() .setOriginalAssetInfo(MP4_ADVANCED_ASSET) - .setEditedMediaItem( - new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)) - .setDurationUs(MP4_ADVANCED_ASSET.videoDurationUs) - .setRemoveVideo(true) - .build()) + .setEditedMediaItemSupplier( + () -> + new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)) + .setDurationUs(MP4_ADVANCED_ASSET.videoDurationUs) + .setRemoveVideo(true) + .build()) .setVideoTimestampsUs( ImmutableList.of( 0L, @@ -315,8 +330,7 @@ public long getNextSpeedChangeTimeUs(long timeUs) { .setName("Video_with_remove_video_set") .build(); - /** The {@link EditedMediaItem} to be processed. */ - public final EditedMediaItem editedMediaItem; + private final Supplier editedMediaItemSupplier; /** Expected output video timestamps in microseconds. */ public final ImmutableList videoTimestampsUs; @@ -328,7 +342,7 @@ public long getNextSpeedChangeTimeUs(long timeUs) { public final String name; private EditedMediaItemAssetInfo(Builder builder) { - this.editedMediaItem = checkNotNull(builder.editedMediaItem); + this.editedMediaItemSupplier = checkNotNull(builder.editedMediaItemSupplier); this.videoTimestampsUs = builder.videoTimestampsUs != null ? builder.videoTimestampsUs @@ -337,6 +351,11 @@ private EditedMediaItemAssetInfo(Builder builder) { this.videoFormat = checkNotNull(builder.originalAssetInfo).videoFormat; } + /** Returns the {@link EditedMediaItem} to be processed. */ + public EditedMediaItem getEditedMediaItem() { + return editedMediaItemSupplier.get(); + } + @Override public String toString() { return name; diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/ExoHostedTest.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/ExoHostedTest.java index 8e2823a1e37..07891897a4a 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/ExoHostedTest.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/ExoHostedTest.java @@ -23,7 +23,6 @@ import android.widget.FrameLayout; import androidx.annotation.Nullable; import androidx.annotation.Size; -import androidx.media3.common.MediaLibraryInfo; import androidx.media3.common.PlaybackException; import androidx.media3.common.Player; import androidx.media3.common.util.Clock; @@ -48,12 +47,6 @@ @UnstableApi public abstract class ExoHostedTest implements HostedTest { - static { - // Disable all device and codec-specific workarounds, since we're interested in testing that the - // underlying platform is behaving correctly. - MediaLibraryInfo.setEnableWorkarounds(false); - } - public static final long MAX_PLAYING_TIME_DISCREPANCY_MS = 5000; public static final long EXPECTED_PLAYING_TIME_MEDIA_DURATION_MS = -2; public static final long EXPECTED_PLAYING_TIME_UNSET = -1; diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/ExoPlayerTestRunner.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/ExoPlayerTestRunner.java index 1288473bc1b..070c8b2ba68 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/ExoPlayerTestRunner.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/ExoPlayerTestRunner.java @@ -345,19 +345,6 @@ public Builder setAnalyticsListener(AnalyticsListener analyticsListener) { return this; } - /** - * Sets whether {@link ExoPlayer} will enable per-stream media period progression. - * - * @param perStreamMediaProgressionEnabled Whether {@link ExoPlayer} will enable per-stream - * media period progression. - * @return This builder. - */ - @CanIgnoreReturnValue - public Builder setPerStreamMediaProgressionEnabled(boolean perStreamMediaProgressionEnabled) { - testPlayerBuilder.setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled); - return this; - } - /** * Sets the number of times the test runner is expected to reach the {@link Player#STATE_ENDED} * or {@link Player#STATE_IDLE}. The default is 1. This affects how long {@link diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeAdaptiveMediaPeriod.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeAdaptiveMediaPeriod.java index ba1d80f5c75..313363722e3 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeAdaptiveMediaPeriod.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeAdaptiveMediaPeriod.java @@ -45,7 +45,6 @@ import androidx.media3.exoplayer.upstream.Allocator; import androidx.media3.exoplayer.upstream.DefaultLoadErrorHandlingPolicy; import com.google.common.collect.ImmutableList; -import com.google.common.collect.ImmutableMap; import com.google.common.collect.Lists; import java.io.IOException; import java.util.ArrayList; @@ -109,7 +108,9 @@ public void release() { @Override public void prepare(Callback callback, long positionUs) { mediaSourceEventDispatcher.loadStarted( - new LoadEventInfo(fakePreparationLoadTaskId, FAKE_DATA_SPEC, SystemClock.elapsedRealtime()), + new LoadEventInfo.Builder( + fakePreparationLoadTaskId, FAKE_DATA_SPEC, SystemClock.elapsedRealtime()) + .build(), C.DATA_TYPE_MEDIA, C.TRACK_TYPE_UNKNOWN, /* trackFormat= */ null, @@ -122,14 +123,10 @@ public void prepare(Callback callback, long positionUs) { prepared = true; Util.castNonNull(this.callback).onPrepared(this); mediaSourceEventDispatcher.loadCompleted( - new LoadEventInfo( - fakePreparationLoadTaskId, - FAKE_DATA_SPEC, - FAKE_DATA_SPEC.uri, - /* responseHeaders= */ ImmutableMap.of(), - SystemClock.elapsedRealtime(), - /* loadDurationMs= */ 0, - /* bytesLoaded= */ 100), + new LoadEventInfo.Builder( + fakePreparationLoadTaskId, FAKE_DATA_SPEC, SystemClock.elapsedRealtime()) + .setBytesLoaded(100) + .build(), C.DATA_TYPE_MEDIA, C.TRACK_TYPE_UNKNOWN, /* trackFormat= */ null, diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeChunkSource.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeChunkSource.java index 079c18be413..61e7a563daa 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeChunkSource.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeChunkSource.java @@ -159,6 +159,7 @@ public void getNextChunk( * *

    Subclasses can override this method to return custom {@link MediaChunk} implementations. */ + @SuppressWarnings("deprecation") // Uses deprecated SingleSampleMediaChunk. protected MediaChunk createMediaChunk( DataSource dataSource, DataSpec dataSpec, diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeClock.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeClock.java index cef6d862e3d..bd7b711867c 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeClock.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeClock.java @@ -324,9 +324,8 @@ protected synchronized void addPendingHandlerMessage(HandlerMessage message) { } else { // Make sure the current looper message is finished before handling the new message. activeMessageLooper = currentLooper; - try { - new Handler(currentLooper).post(() -> onMessageHandled(currentLooper)); - } catch (IllegalStateException e) { + // Prevent deadlocks when the current looper rejects the message without throwing. + if (!postOnMessageHandled(currentLooper, new Handler(currentLooper))) { onMessageHandled(currentLooper); } } @@ -427,7 +426,9 @@ private synchronized void maybeTriggerMessage() { realHandler.sendMessage( realHandler.obtainMessage(message.what, message.arg1, message.arg2, message.obj)); } - messageSent &= message.handler.internalHandler.post(() -> onMessageHandled(targetLooper)); + if (messageSent) { + messageSent = postOnMessageHandled(targetLooper, message.handler.internalHandler); + } } catch (IllegalStateException e) { // In Robolectric PAUSED looper mode, sending a message to a Handler on a dead thread throws // IllegalStateException instead of returning false. @@ -452,6 +453,14 @@ private synchronized void advanceTimeInternal(long timeDiffMs) { } } + private boolean postOnMessageHandled(Looper looper, Handler handler) { + try { + return handler.post(() -> onMessageHandled(looper)); + } catch (IllegalStateException e) { + return false; + } + } + private static synchronized long getNextMessageId() { return messageIdProvider++; } diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeExoMediaDrm.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeExoMediaDrm.java index ac62383abfc..d7cdc34b5f8 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeExoMediaDrm.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeExoMediaDrm.java @@ -602,14 +602,10 @@ public Response executeProvisionRequest(UUID uuid, ProvisionRequest request) if (Arrays.equals(request.getData(), FAKE_PROVISION_REQUEST.getData())) { return new Response.Builder(Bytes.toArray(VALID_PROVISION_RESPONSE)) .setLoadEventInfo( - new LoadEventInfo( - /* loadTaskId= */ -1, - new DataSpec(uri), - uri, - /* responseHeaders= */ ImmutableMap.of(), - SystemClock.elapsedRealtime(), - /* loadDurationMs= */ 0, - /* bytesLoaded= */ VALID_PROVISION_RESPONSE.size())) + new LoadEventInfo.Builder( + /* loadTaskId= */ -1, new DataSpec(uri), SystemClock.elapsedRealtime()) + .setBytesLoaded(VALID_PROVISION_RESPONSE.size()) + .build()) .build(); } else { return new Response(Util.EMPTY_BYTE_ARRAY); @@ -636,14 +632,10 @@ public Response executeKeyRequest(UUID uuid, KeyRequest request) Uri uri = Uri.parse(request.getLicenseServerUrl()); return new Response.Builder(Bytes.toArray(response)) .setLoadEventInfo( - new LoadEventInfo( - /* loadTaskId= */ -1, - new DataSpec(uri), - uri, - /* responseHeaders= */ ImmutableMap.of(), - SystemClock.elapsedRealtime(), - /* loadDurationMs= */ 0, - /* bytesLoaded= */ response.size())) + new LoadEventInfo.Builder( + /* loadTaskId= */ -1, new DataSpec(uri), SystemClock.elapsedRealtime()) + .setBytesLoaded(response.size()) + .build()) .build(); } diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeFrameProcessor.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeFrameProcessor.java index bb60cf82966..ca8a8d112c4 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeFrameProcessor.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeFrameProcessor.java @@ -15,12 +15,9 @@ */ package androidx.media3.test.utils; -import static com.google.common.collect.ImmutableList.toImmutableList; import static com.google.common.util.concurrent.MoreExecutors.directExecutor; import androidx.annotation.Nullable; -import androidx.annotation.RequiresApi; -import androidx.media3.common.C; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.util.ExperimentalApi; import androidx.media3.common.video.AsyncFrame; @@ -28,21 +25,50 @@ import androidx.media3.common.video.FrameProcessor; import androidx.media3.common.video.FrameWriter; import androidx.media3.common.video.SyncFenceWrapper; -import com.google.common.collect.ImmutableList; -import com.google.common.collect.Iterables; -import java.util.ArrayList; import java.util.List; +import java.util.concurrent.CopyOnWriteArrayList; import java.util.concurrent.Executor; +import java.util.concurrent.atomic.AtomicBoolean; +import java.util.concurrent.atomic.AtomicReference; -/** A no-op {@link FrameProcessor} that holds a reference to all queued frames for testing. */ -@RequiresApi(26) +/** A fake {@link FrameProcessor} implementation. */ @ExperimentalApi // TODO: b/498176910 Remove once FrameProcessor is production ready. public class FakeFrameProcessor implements FrameProcessor { + /** A fake {@link FrameProcessor.Listener} implementation for tests. */ + public static class Listener implements FrameProcessor.Listener { + + public final List processedFrames; + public final AtomicReference error; + + /** Creates a new instance. */ + public Listener() { + processedFrames = new CopyOnWriteArrayList<>(); + error = new AtomicReference<>(); + } + + @Override + public void onWakeup() {} + + @Override + public void onError(VideoFrameProcessingException exception) { + error.set(exception); + } + + @Override + public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper onCompleteFence) { + if (onCompleteFence != null) { + onCompleteFence.close(); + } + processedFrames.add(frame); + } + } + /** A factory for {@link FakeFrameProcessor} implementations. */ public static class Factory implements FrameProcessor.Factory { @Nullable public FakeFrameProcessor createdProcessor; + private final boolean shouldCompleteIncomingFrames; /** @@ -57,7 +83,7 @@ public Factory(boolean shouldCompleteIncomingFrames) { @Override public FakeFrameProcessor create( - FrameWriter output, Executor listenerExecutor, Listener listener) { + FrameWriter output, Executor listenerExecutor, FrameProcessor.Listener listener) { createdProcessor = new FakeFrameProcessor(output, listenerExecutor, listener, shouldCompleteIncomingFrames); return createdProcessor; @@ -65,60 +91,30 @@ public FakeFrameProcessor create( /** Helper create method for tests that don't need to specify listener. */ public FakeFrameProcessor create(FrameWriter output) { - createdProcessor = - new FakeFrameProcessor( - output, - /* listenerExecutor= */ Runnable::run, - new Listener() { - @Override - public void onWakeup() {} - - @Override - public void onError(VideoFrameProcessingException exception) {} - - @Override - public void onFrameProcessed( - Frame frame, @Nullable SyncFenceWrapper onCompleteFence) {} - }, - shouldCompleteIncomingFrames); - return createdProcessor; + return create(output, directExecutor(), new Listener()); } } - /** Marker interface for ordering frames and EOS signals. */ - public interface Event {} - - /** Holds the frames queued in a {@link #queue} call. */ - public static final class FramesEvent implements Event { - public final List frames; - - public FramesEvent(List frames) { - this.frames = frames; - } - } + public final FrameWriter output; + public final AtomicBoolean eosSignaled; + public final AtomicBoolean closed; - /** Marker class representing a {@link #signalEndOfStream()} call. */ - public static final class EosEvent implements Event {} + public int queuedFramesCount; - private final FrameWriter output; private final Executor listenerExecutor; - private final Listener listener; - - /** All the frames and EOS events queued to this {@link FrameProcessor}. */ - private final List queuedEvents; - + private final FrameProcessor.Listener listener; private final boolean shouldCompleteIncomingFrames; - @Nullable public List lastFrames; private boolean configured; private FakeFrameProcessor( FrameWriter output, Executor listenerExecutor, - Listener listener, + FrameProcessor.Listener listener, boolean shouldCompleteIncomingFrames) { - this.queuedEvents = new ArrayList<>(); this.output = output; + this.eosSignaled = new AtomicBoolean(); + this.closed = new AtomicBoolean(); this.listenerExecutor = listenerExecutor; this.listener = listener; this.shouldCompleteIncomingFrames = shouldCompleteIncomingFrames; @@ -126,6 +122,7 @@ private FakeFrameProcessor( @Override public boolean queue(List frames) { + queuedFramesCount++; if (!configured && !frames.isEmpty()) { output.configure(frames.get(0).frame.getFormat(), /* usage= */ 0); configured = true; @@ -133,12 +130,13 @@ public boolean queue(List frames) { AsyncFrame placeholderFrame = output.dequeueInputFrame(directExecutor(), /* wakeupListener= */ () -> {}); if (placeholderFrame != null) { + if (placeholderFrame.acquireFence != null) { + placeholderFrame.acquireFence.close(); + } // This forces lazy configuration on some FrameWriter implementations. output.queueInputFrame(placeholderFrame.frame, /* writeCompleteFence= */ null); } } - queuedEvents.add(new FramesEvent(frames)); - this.lastFrames = frames; if (shouldCompleteIncomingFrames) { for (AsyncFrame asyncFrame : frames) { listenerExecutor.execute( @@ -150,41 +148,13 @@ public boolean queue(List frames) { @Override public void signalEndOfStream() { - queuedEvents.add(new EosEvent()); + eosSignaled.set(true); output.signalEndOfStream(); } @Override - public void close() {} - - /** Returns true if the last {@link Event} queued was an {@link EosEvent}. */ - public boolean isEnded() { - return !queuedEvents.isEmpty() && Iterables.getLast(queuedEvents) instanceof EosEvent; - } - - /** Returns an {@link ImmutableList} of all queued events. */ - public ImmutableList getQueuedEvents() { - return ImmutableList.copyOf(queuedEvents); - } - - /** - * Returns an {@link ImmutableList} of content timestamps for all queued frames. EOS frames return - * {@link C#TIME_UNSET}. - */ - public ImmutableList> getQueuedContentTimesUs() { - ImmutableList currentEvents = getQueuedEvents(); - return currentEvents.stream() - .map( - event -> { - if (event instanceof EosEvent) { - return ImmutableList.of(C.TIME_UNSET); - } - FramesEvent framesEvent = (FakeFrameProcessor.FramesEvent) event; - return framesEvent.frames.stream() - .map(asyncFrame -> asyncFrame.frame.getContentTimeUs()) - .collect(toImmutableList()); - }) - .collect(toImmutableList()); + public void close() { + closed.set(true); } /** Simulates a video frame processing error by notifying the listener. */ diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeFrameWriter.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeFrameWriter.java new file mode 100644 index 00000000000..a73d0f9ebeb --- /dev/null +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeFrameWriter.java @@ -0,0 +1,111 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.test.utils; + +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import androidx.media3.common.Format; +import androidx.media3.common.util.ExperimentalApi; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.Frame; +import androidx.media3.common.video.FrameWriter; +import androidx.media3.common.video.SyncFenceWrapper; +import java.util.List; +import java.util.Queue; +import java.util.concurrent.ConcurrentLinkedQueue; +import java.util.concurrent.CopyOnWriteArrayList; +import java.util.concurrent.CountDownLatch; +import java.util.concurrent.Executor; +import java.util.concurrent.atomic.AtomicBoolean; + +/** A fake {@link FrameWriter} implementation for testing. */ +@RequiresApi(26) +@ExperimentalApi // TODO: b/498176910 - Remove once FrameWriter is production ready. +public class FakeFrameWriter implements FrameWriter { + + public final List queuedFrames; + public final AtomicBoolean eosSignaled; + public final CountDownLatch configuredLatch; + public final CountDownLatch eosLatch; + public final CountDownLatch dequeueAttemptedLatch; + + @Nullable public volatile Format configuredFormat; + public volatile CountDownLatch frameLatch; + + private final Queue availableFrames; + + @Nullable private volatile Executor wakeupExecutor; + @Nullable private volatile Runnable wakeupListener; + + /** Creates a new instance. */ + public FakeFrameWriter() { + queuedFrames = new CopyOnWriteArrayList<>(); + eosSignaled = new AtomicBoolean(false); + configuredLatch = new CountDownLatch(1); + eosLatch = new CountDownLatch(1); + dequeueAttemptedLatch = new CountDownLatch(1); + frameLatch = new CountDownLatch(1); + availableFrames = new ConcurrentLinkedQueue<>(); + } + + @Override + public void configure(Format format, @Frame.Usage long usage) { + this.configuredFormat = format; + configuredLatch.countDown(); + } + + @Override + @Nullable + public AsyncFrame dequeueInputFrame(Executor wakeupExecutor, Runnable wakeupListener) { + this.wakeupExecutor = wakeupExecutor; + this.wakeupListener = wakeupListener; + dequeueAttemptedLatch.countDown(); + return availableFrames.poll(); + } + + @Override + public void queueInputFrame(Frame frame, @Nullable SyncFenceWrapper writeCompleteFence) { + if (writeCompleteFence != null) { + writeCompleteFence.close(); + } + queuedFrames.add(frame); + frameLatch.countDown(); + } + + @Override + public void signalEndOfStream() { + eosSignaled.set(true); + eosLatch.countDown(); + } + + @Override + public Info getInfo() { + return (format, usage) -> true; + } + + @Override + public void close() {} + + /** Makes a frame available to be returned by the next {@link #dequeueInputFrame} call. */ + public void prepareInputFrame(Frame frame) { + availableFrames.add(new AsyncFrame(frame, /* acquireFence= */ null)); + @Nullable Executor executor = this.wakeupExecutor; + @Nullable Runnable listener = this.wakeupListener; + if (executor != null && listener != null) { + executor.execute(listener); + } + } +} diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeHardwareBufferJniWrapper.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeHardwareBufferJniWrapper.java new file mode 100644 index 00000000000..079b514eac5 --- /dev/null +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeHardwareBufferJniWrapper.java @@ -0,0 +1,55 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.test.utils; + +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; + +import android.hardware.HardwareBuffer; +import androidx.annotation.RequiresApi; +import androidx.annotation.RestrictTo; +import androidx.media3.effect.HardwareBufferJniWrapper; + +/** + * Fake {@link HardwareBufferJniWrapper} that doesn't call into native code. + * + *

    The bitmap and buffer copy methods are inherited from {@link FakeHardwareBufferNativeHelpers}; + * this class only adds fakes for the EGLImage methods. All operations return a successful result if + * {@link #shouldSucceed} is {@code true} (the default), or a failure result otherwise. + */ +@RequiresApi(26) +@RestrictTo(LIBRARY_GROUP) +public final class FakeHardwareBufferJniWrapper extends FakeHardwareBufferNativeHelpers + implements HardwareBufferJniWrapper { + + /** The fake EGLImage handle returned on success. */ + public static final long FAKE_EGL_IMAGE_HANDLE = 1L; + + @Override + public long nativeCreateEglImageFromHardwareBuffer( + long displayHandle, HardwareBuffer hardwareBuffer) { + return shouldSucceed ? FAKE_EGL_IMAGE_HANDLE : 0L; + } + + @Override + public boolean nativeBindEGLImage(int target, long eglImageHandle) { + return shouldSucceed; + } + + @Override + public boolean nativeDestroyEGLImage(long displayHandle, long imageHandle) { + return shouldSucceed; + } +} diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeHardwareBufferNativeHelpers.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeHardwareBufferNativeHelpers.java index 70b8c1c9f85..6b3638023dc 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeHardwareBufferNativeHelpers.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeHardwareBufferNativeHelpers.java @@ -22,7 +22,6 @@ import androidx.media3.common.video.HardwareBufferNativeHelpers; /** Fake class to simulate {@link HardwareBufferNativeHelpers}. */ -@RequiresApi(26) @ExperimentalApi // TODO: b/498176910 - Remove once Frame is production ready. public class FakeHardwareBufferNativeHelpers implements HardwareBufferNativeHelpers { @@ -33,11 +32,13 @@ public FakeHardwareBufferNativeHelpers() { shouldSucceed = true; } + @RequiresApi(26) @Override public boolean nativeCopyBitmapToHardwareBuffer(Bitmap bitmap, HardwareBuffer hb) { return shouldSucceed; } + @RequiresApi(26) @Override public boolean nativeCopyHardwareBufferToHardwareBuffer( HardwareBuffer srcHb, HardwareBuffer dstHb) { diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeMediaChunk.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeMediaChunk.java index 6a1bb6be839..adbede9c65a 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeMediaChunk.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeMediaChunk.java @@ -63,7 +63,8 @@ public FakeMediaChunk( /* trackSelectionData= */ null, startTimeUs, endTimeUs, - /* chunkIndex= */ 0); + /* chunkIndex= */ 0, + /* steeredPathwayId= */ null); } @Override diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeMediaPeriod.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeMediaPeriod.java index 2aa61f39dbb..348df5b75c4 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeMediaPeriod.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeMediaPeriod.java @@ -52,7 +52,6 @@ import com.google.common.math.DoubleMath; import java.io.IOException; import java.math.RoundingMode; -import java.util.Collections; import java.util.List; import java.util.Set; @@ -304,7 +303,9 @@ public void release() { @Override public synchronized void prepare(Callback callback, long positionUs) { mediaSourceEventDispatcher.loadStarted( - new LoadEventInfo(fakePreparationLoadTaskId, FAKE_DATA_SPEC, SystemClock.elapsedRealtime()), + new LoadEventInfo.Builder( + fakePreparationLoadTaskId, FAKE_DATA_SPEC, SystemClock.elapsedRealtime()) + .build(), C.DATA_TYPE_MEDIA, C.TRACK_TYPE_UNKNOWN, /* trackFormat= */ null, @@ -518,14 +519,10 @@ private void finishPreparation() { prepared = true; Util.castNonNull(prepareCallback).onPrepared(this); mediaSourceEventDispatcher.loadCompleted( - new LoadEventInfo( - fakePreparationLoadTaskId, - FAKE_DATA_SPEC, - FAKE_DATA_SPEC.uri, - /* responseHeaders= */ Collections.emptyMap(), - SystemClock.elapsedRealtime(), - /* loadDurationMs= */ 0, - /* bytesLoaded= */ 100), + new LoadEventInfo.Builder( + fakePreparationLoadTaskId, FAKE_DATA_SPEC, SystemClock.elapsedRealtime()) + .setBytesLoaded(100) + .build(), C.DATA_TYPE_MEDIA, C.TRACK_TYPE_UNKNOWN, /* trackFormat= */ null, diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeMediaSource.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeMediaSource.java index 46ccca97add..bb84bc682ed 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeMediaSource.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakeMediaSource.java @@ -49,7 +49,6 @@ import androidx.media3.exoplayer.source.TrackGroupArray; import androidx.media3.exoplayer.upstream.Allocator; import androidx.media3.test.utils.FakeMediaPeriod.TrackDataFactory; -import com.google.common.collect.ImmutableMap; import com.google.common.collect.Iterables; import com.google.common.collect.ObjectArrays; import com.google.errorprone.annotations.CanIgnoreReturnValue; @@ -577,25 +576,13 @@ private void finishSourcePreparation(boolean sendManifestLoadEvents) { createEventDispatcher(/* mediaPeriodId= */ null); long loadTaskId = LoadEventInfo.getNewId(); eventDispatcher.loadStarted( - new LoadEventInfo( - loadTaskId, - FAKE_DATA_SPEC, - FAKE_DATA_SPEC.uri, - /* responseHeaders= */ ImmutableMap.of(), - elapsedRealTimeMs, - /* loadDurationMs= */ 0, - /* bytesLoaded= */ 0), + new LoadEventInfo.Builder(loadTaskId, FAKE_DATA_SPEC, elapsedRealTimeMs).build(), mediaLoadData, /* retryCount= */ 0); eventDispatcher.loadCompleted( - new LoadEventInfo( - loadTaskId, - FAKE_DATA_SPEC, - FAKE_DATA_SPEC.uri, - /* responseHeaders= */ ImmutableMap.of(), - elapsedRealTimeMs, - /* loadDurationMs= */ 0, - /* bytesLoaded= */ MANIFEST_LOAD_BYTES), + new LoadEventInfo.Builder(loadTaskId, FAKE_DATA_SPEC, elapsedRealTimeMs) + .setBytesLoaded(MANIFEST_LOAD_BYTES) + .build(), mediaLoadData); } } diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakePlayer.kt b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakePlayer.kt index 5fef1652300..e145d32f0b0 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakePlayer.kt +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FakePlayer.kt @@ -26,12 +26,13 @@ import androidx.media3.common.PlaybackParameters import androidx.media3.common.Player import androidx.media3.common.SimpleBasePlayer import androidx.media3.common.TrackSelectionParameters +import androidx.media3.common.Tracks import androidx.media3.common.VideoSize +import androidx.media3.common.text.CueGroup import androidx.media3.common.util.UnstableApi import androidx.media3.common.util.Util import androidx.media3.common.util.Util.getAvailableCommands import androidx.media3.common.util.Util.msToUs -import com.google.common.base.Preconditions.checkState import com.google.common.collect.ImmutableList import com.google.common.util.concurrent.Futures import com.google.common.util.concurrent.ListenableFuture @@ -45,6 +46,7 @@ import kotlinx.coroutines.test.TestCoroutineScheduler * @param playWhenReady Whether playback should start automatically when ready. * @param playlist The initial list of media items to play. * @param playbackSpeed The initial playback speed. + * @param trackSelectionParameters The initial track selection parameters. * @param bufferingDelayMs The time spent in [Player#STATE_BUFFERING] when preparing or seeking, in * milliseconds. A value of zero will result in the player transitioning directly to * [Player#STATE_READY] without entering [Player#STATE_BUFFERING]. Must not be negative. @@ -55,6 +57,7 @@ class FakePlayer( playWhenReady: Boolean = false, playlist: List = ImmutableList.of(), playbackSpeed: Float = 1f, + trackSelectionParameters: TrackSelectionParameters = TrackSelectionParameters.DEFAULT, private val bufferingDelayMs: Long = 100, ) : SimpleBasePlayer(Looper.myLooper()!!) { private val handler = Util.createHandlerForCurrentLooper() @@ -66,6 +69,7 @@ class FakePlayer( .setPlayWhenReady(playWhenReady, PLAY_WHEN_READY_CHANGE_REASON_USER_REQUEST) .setPlaylist(playlist) .setPlaybackParameters(PlaybackParameters(playbackSpeed)) + .setTrackSelectionParameters(trackSelectionParameters) .setDeviceInfo(DeviceInfo.Builder(DeviceInfo.PLAYBACK_TYPE_LOCAL).build()) .build() @@ -137,16 +141,19 @@ class FakePlayer( override fun handleSeek(mediaItemIndex: Int, positionMs: Long, seekCommand: @Player.Command Int) = handleStateUpdate { - val resultingState: @Player.State Int - if (bufferingDelayMs > 0) { - resultingState = STATE_BUFFERING - handler.postDelayed({ updateState { setPlaybackState(STATE_READY) } }, bufferingDelayMs) - } else { - resultingState = STATE_READY + if (playbackState != STATE_IDLE) { + val resultingState: @Player.State Int + if (state.timeline.isEmpty) { + resultingState = STATE_ENDED + } else if (bufferingDelayMs > 0) { + resultingState = STATE_BUFFERING + handler.postDelayed({ updateState { setPlaybackState(STATE_READY) } }, bufferingDelayMs) + } else { + resultingState = STATE_READY + } + setPlaybackState(resultingState) } - setPlaybackState(resultingState) - .setCurrentMediaItemIndex(mediaItemIndex) - .setContentPositionMs(positionMs) + setCurrentMediaItemIndex(mediaItemIndex).setContentPositionMs(positionMs) } override fun handleSetShuffleModeEnabled(shuffleModeEnabled: Boolean) = handleStateUpdate { @@ -221,8 +228,8 @@ class FakePlayer( * * assuming the playback speed does not change after the supplier is set */ fun setPositionSupplierDrivenBy(testScheduler: TestCoroutineScheduler) { - checkState(state.playlist.size == 1, "Playlist must contain exactly one item") - checkState(isPlaying, "Player must be playing") + check(state.playlist.size == 1) { "Playlist must contain exactly one item" } + check(isPlaying) { "Player must be playing" } val startTime = testScheduler.timeSource.markNow() val currentPositionMs = currentPosition val positionSupplier = { @@ -243,6 +250,16 @@ class FakePlayer( invalidateState() } + /** + * Sets the current cues. + * + * @param cues The current cues. + */ + fun setCurrentCues(cues: CueGroup) { + state = state.buildUpon().setCurrentCues(cues).build() + invalidateState() + } + /** * Sets the duration of the media item in the playlist, in milliseconds. * @@ -311,6 +328,18 @@ class FakePlayer( invalidateState() } + /** + * Sets the playback suppression reason. + * + * @param playbackSuppressionReason The playback suppression reason. + */ + fun setPlaybackSuppressionReason( + playbackSuppressionReason: @Player.PlaybackSuppressionReason Int + ) { + state = state.buildUpon().setPlaybackSuppressionReason(playbackSuppressionReason).build() + invalidateState() + } + /** * Sets whether a frame has been rendered for the first time since setting the surface, a * rendering reset, or since the stream being rendered was changed. @@ -347,6 +376,39 @@ class FakePlayer( invalidateState() } + /** + * Sets the [Tracks] of the current media item. + * + * @param tracks The [Tracks] to set. + */ + fun setTracks(tracks: Tracks) { + val index = currentMediaItemIndex + if (index < 0 || index >= state.playlist.size) { + return + } + val modifiedPlaylist = state.playlist.toMutableList() + modifiedPlaylist[index] = modifiedPlaylist[index].buildUpon().setTracks(tracks).build() + state = state.buildUpon().setPlaylist(modifiedPlaylist).build() + invalidateState() + } + + /** Sets the [DeviceInfo] of the player. */ + fun setDeviceInfo(deviceInfo: DeviceInfo) { + state = state.buildUpon().setDeviceInfo(deviceInfo).build() + invalidateState() + } + + /** + * Sets the current ad being played. + * + * @param adGroupIndex The index of the ad group being played. + * @param adIndexInAdGroup The index of the ad within the ad group being played. + */ + fun setCurrentAd(adGroupIndex: Int, adIndexInAdGroup: Int) { + state = state.buildUpon().setCurrentAd(adGroupIndex, adIndexInAdGroup).build() + invalidateState() + } + private fun handleStateUpdate(stateUpdate: State.Builder.() -> Unit): ListenableFuture<*> { updateState(stateUpdate) return Futures.immediateVoidFuture() diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/FrameProcessorTestUtil.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FrameProcessorTestUtil.java new file mode 100644 index 00000000000..430c26e1f79 --- /dev/null +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/FrameProcessorTestUtil.java @@ -0,0 +1,333 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.test.utils; + +import static android.os.Build.VERSION.SDK_INT; +import static com.google.common.base.Preconditions.checkNotNull; +import static com.google.common.truth.Truth.assertThat; +import static java.lang.Math.max; +import static java.util.Objects.hash; + +import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; +import androidx.media3.common.C; +import androidx.media3.common.MediaItem; +import androidx.media3.common.util.UnstableApi; +import androidx.media3.common.util.Util; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.Frame; +import androidx.media3.common.video.FrameProcessor; +import androidx.media3.transformer.Composition; +import androidx.media3.transformer.CompositionFrameMetadata; +import androidx.media3.transformer.EditedMediaItemSequence; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.Iterables; +import java.util.ArrayDeque; +import java.util.ArrayList; +import java.util.List; +import java.util.Queue; + +/** Test utilities for asserting {@link FrameProcessor} inputs. */ +@RequiresApi(26) +@UnstableApi +public final class FrameProcessorTestUtil { + + /** Represents a frame that is expected to be presented. */ + public static class ExpectedFrame { + public final long contentTimeUs; + public final int sequenceIndex; + + /** + * Creates an instance. + * + * @param contentTimeUs The expected presentation timestamp, in microseconds. + * @param sequenceIndex The index of the sequence this frame belongs to. + */ + public ExpectedFrame(long contentTimeUs, int sequenceIndex) { + this.contentTimeUs = contentTimeUs; + this.sequenceIndex = sequenceIndex; + } + + @Override + public boolean equals(@Nullable Object o) { + if (this == o) { + return true; + } + if (!(o instanceof ExpectedFrame)) { + return false; + } + ExpectedFrame that = (ExpectedFrame) o; + return contentTimeUs == that.contentTimeUs && sequenceIndex == that.sequenceIndex; + } + + @Override + public int hashCode() { + return hash(contentTimeUs, sequenceIndex); + } + + @Override + public String toString() { + return Util.formatInvariant("(t=%d, seq=%d)", contentTimeUs, sequenceIndex); + } + } + + /** + * Asserts the {@link CapturingFrameProcessor} consumes frames matching the {@link + * CompositionAssetInfo}, if it was played from start to finish. + * + *

    This method verifies both the timing/structure of the playback (timestamps, sequence + * indices, EOS) and that the correct {@link MediaItem} metadata is associated with each frame. + */ + public static void assertPlaybackOutput( + CapturingFrameProcessor frameProcessor, CompositionAssetInfo compositionAssetInfo) { + ImmutableList actualEvents = frameProcessor.getQueuedEvents(); + assertThat(Iterables.getLast(actualEvents)) + .isInstanceOf(CapturingFrameProcessor.EosEvent.class); + + ImmutableList actualEventsWithoutEos = + actualEvents.subList(0, actualEvents.size() - 1); + ImmutableList> actual = toExpectedEvents(actualEventsWithoutEos); + ImmutableList> expected = getExpectedEvents(compositionAssetInfo); + assertThat(actual).isEqualTo(expected); + + for (int i = 0; i < actualEventsWithoutEos.size(); i++) { + List expectedFrames = expected.get(i); + CapturingFrameProcessor.Event actualEvent = actualEvents.get(i); + + assertThat(actualEvent).isInstanceOf(CapturingFrameProcessor.FramesEvent.class); + CapturingFrameProcessor.FramesEvent actualFrames = + (CapturingFrameProcessor.FramesEvent) actualEvent; + + for (int j = 0; j < actualFrames.frames.size(); j++) { + AsyncFrame asyncFrame = actualFrames.frames.get(j); + Frame frame = asyncFrame.frame; + ExpectedFrame expectedFrame = expectedFrames.get(j); + + CompositionFrameMetadata compositionFrameMetadata = + checkNotNull( + (CompositionFrameMetadata) + frame + .getMetadata() + .get(CompositionFrameMetadata.KEY_COMPOSITION_FRAME_METADATA)); + assertThat(compositionFrameMetadata).isNotNull(); + + MediaItem itemFromMetadata = itemFromMetadata(compositionFrameMetadata); + MediaItem expectedItemAtTime = + expectedItemAtTime( + compositionAssetInfo, expectedFrame.sequenceIndex, expectedFrame.contentTimeUs); + assertThat(itemFromMetadata).isEqualTo(expectedItemAtTime); + } + } + } + + /** + * Returns the events expected to be consumed by the {@link + * androidx.media3.common.video.FrameProcessor} if the {@linkplain + * androidx.media3.test.utils.CompositionAssetInfo#getComposition() Composition} was played from + * start to finish. + */ + public static ImmutableList> getExpectedEvents( + CompositionAssetInfo compositionAssetInfo) { + List videoSequences = new ArrayList<>(); + List videoSequenceIndices = new ArrayList<>(); + for (int i = 0; i < compositionAssetInfo.sequences.size(); i++) { + SequenceAssetInfo seq = compositionAssetInfo.sequences.get(i); + if (seq.trackTypes.contains(C.TRACK_TYPE_VIDEO)) { + videoSequences.add(seq); + videoSequenceIndices.add(i); + } + } + if (videoSequences.isEmpty()) { + return ImmutableList.of(); + } + + return getExpectedEvents( + videoSequences.get(0).getExpectedVideoTimestampsUs(), + videoSequenceIndices.get(0), + extractContentTimesUs(videoSequences.subList(/* fromIndex= */ 1, videoSequences.size())), + videoSequenceIndices.subList(/* fromIndex= */ 1, videoSequenceIndices.size())); + } + + /** + * Calculates the expected events from the primary and secondary sequence timestamps. + * + *

    For each primary timestamp, selects the closest secondary timestamp between the preceding + * and following frames on API 29+, or the smallest secondary timestamp greater than or equal to + * the primary timestamp below API 29. If there is no matching secondary timestamp, that sequence + * is omitted. + * + * @param primaryContentTimesUs Timestamps of the primary sequence. + * @param primarySeqIndex Index of the primary sequence. + * @param secondarySequencesContentTimesUs Timestamps of the secondary sequences. + * @param secondarySeqIndices Indices of the secondary sequences. + * @return A list of expected events. + */ + private static ImmutableList> getExpectedEvents( + List primaryContentTimesUs, + int primarySeqIndex, + List> secondarySequencesContentTimesUs, + List secondarySeqIndices) { + + ImmutableList.Builder> expectedEvents = ImmutableList.builder(); + + // Always add one primary timestamp per packet. + for (long primaryTime : primaryContentTimesUs) { + List packet = new ArrayList<>(); + packet.add(new ExpectedFrame(primaryTime, primarySeqIndex)); + + for (int i = 0; i < secondarySequencesContentTimesUs.size(); i++) { + Queue queue = secondarySequencesContentTimesUs.get(i); + int seqIndex = secondarySeqIndices.get(i); + if (SDK_INT >= 29) { + advanceToClosestSecondaryTimeUs(queue, primaryTime); + } else { + // Drain the secondary sequence while it's timestamp is less than the primary timestamp. + while (!queue.isEmpty()) { + long peekedTime = checkNotNull(queue.peek()); + if (peekedTime < primaryTime) { + queue.poll(); + } else { + break; + } + } + } + + if (!queue.isEmpty()) { + // Guaranteed to be the smallest timestamp >= to the primary time on API < 29, or the + // closest timestamp on API 29+. + packet.add(new ExpectedFrame(checkNotNull(queue.peek()), seqIndex)); + } + } + expectedEvents.add(packet); + } + return expectedEvents.build(); + } + + /** + * Advances {@code queue} to the frame closest to {@code targetTimeUs}. + * + *

    Retains at most one frame at or before {@code targetTimeUs}, or empties {@code queue} if the + * secondary sequence has ended before {@code targetTimeUs}. + */ + private static void advanceToClosestSecondaryTimeUs(Queue queue, long targetTimeUs) { + if (queue.isEmpty()) { + return; + } + int precedingFrameCount = 0; + for (long timeUs : queue) { + if (timeUs <= targetTimeUs) { + precedingFrameCount++; + } else { + break; + } + } + int framesToDiscard = max(0, precedingFrameCount - 1); + for (int i = 0; i < framesToDiscard; i++) { + queue.poll(); + } + + long firstTimeUs = checkNotNull(queue.peek()); + // No preceding frame exists, or it matches targetTimeUs. + if (firstTimeUs >= targetTimeUs) { + return; + } + + if (queue.size() >= 2) { + long previousTimeUs = firstTimeUs; + long nextTimeUs = Iterables.get(queue, 1); + long previousDiffUs = targetTimeUs - previousTimeUs; + long nextDiffUs = nextTimeUs - targetTimeUs; + if (nextDiffUs < previousDiffUs) { + queue.poll(); + } + return; + } + + queue.poll(); + } + + /** + * Converts the recorded {@link CapturingFrameProcessor.Event}s to expected events format. + * + * @param actualEvents The events recorded by the capturing processor. + * @return The mapped expected events. + */ + private static ImmutableList> toExpectedEvents( + List actualEvents) { + List> result = new ArrayList<>(); + for (CapturingFrameProcessor.Event event : actualEvents) { + assertThat(event).isInstanceOf(CapturingFrameProcessor.FramesEvent.class); + CapturingFrameProcessor.FramesEvent framesEvent = (CapturingFrameProcessor.FramesEvent) event; + List packet = new ArrayList<>(); + for (AsyncFrame asyncFrame : framesEvent.frames) { + Frame frame = asyncFrame.frame; + long contentTimeUs = frame.getContentTimeUs(); + CompositionFrameMetadata metadata = + (CompositionFrameMetadata) + frame.getMetadata().get(CompositionFrameMetadata.KEY_COMPOSITION_FRAME_METADATA); + int sequenceIndex = metadata != null ? metadata.sequenceIndex : C.INDEX_UNSET; + packet.add(new ExpectedFrame(contentTimeUs, sequenceIndex)); + } + result.add(packet); + } + return ImmutableList.copyOf(result); + } + + /** Extracts the expected video timestamps from a list of sequences. */ + private static ImmutableList> extractContentTimesUs( + List sequences) { + ImmutableList.Builder> list = new ImmutableList.Builder<>(); + for (SequenceAssetInfo seq : sequences) { + list.add(new ArrayDeque<>(seq.getExpectedVideoTimestampsUs())); + } + return list.build(); + } + + /** Extracts the {@link MediaItem} that produced the frame from its metadata. */ + private static MediaItem itemFromMetadata(CompositionFrameMetadata metadata) { + return metadata + .composition + .sequences + .get(metadata.sequenceIndex) + .editedMediaItems + .get(metadata.itemIndex) + .mediaItem; + } + + /** + * Resolves the expected {@link MediaItem} that should be active in a sequence at a given + * presentation time. + */ + private static MediaItem expectedItemAtTime( + CompositionAssetInfo compositionAssetInfo, int sequenceIndex, long presentationTimeUs) { + Composition composition = compositionAssetInfo.getComposition(); + EditedMediaItemSequence sequence = composition.sequences.get(sequenceIndex); + int itemIndex = 0; + while (itemIndex < sequence.editedMediaItems.size()) { + long itemDurationUs = sequence.editedMediaItems.get(itemIndex).getPresentationDurationUs(); + if (presentationTimeUs < itemDurationUs) { + break; + } + presentationTimeUs -= itemDurationUs; + itemIndex++; + } + assertThat(itemIndex).isLessThan(sequence.editedMediaItems.size()); + return sequence.editedMediaItems.get(itemIndex).mediaItem; + } + + /** Prevents instantiation. */ + private FrameProcessorTestUtil() {} +} diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/HostActivity.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/HostActivity.java index 40fc14dd7e1..ed633bc96bb 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/HostActivity.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/HostActivity.java @@ -15,7 +15,6 @@ */ package androidx.media3.test.utils; -import static android.os.Build.VERSION.SDK_INT; import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.base.Preconditions.checkState; @@ -194,20 +193,10 @@ public void onStart() { super.onStart(); } - @Override - public void onPause() { - super.onPause(); - if (SDK_INT == 23) { - maybeStopHostedTest(); - } - } - @Override public void onStop() { super.onStop(); - if (SDK_INT > 23) { - maybeStopHostedTest(); - } + maybeStopHostedTest(); if (wakeLock != null) { wakeLock.release(); wakeLock = null; diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/HttpDataSourceTestEnv.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/HttpDataSourceTestEnv.java index 9cc0d7a33e9..c48d7ab8f89 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/HttpDataSourceTestEnv.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/HttpDataSourceTestEnv.java @@ -91,6 +91,15 @@ public class HttpDataSourceTestEnv extends ExternalResource { .setExtraResponseHeaders(EXTRA_HEADERS) .build(); + public static final WebServerDispatcher.Resource RANGE_NOT_SUPPORTED_NON_AUTHORITATIVE = + new WebServerDispatcher.Resource.Builder() + .setPath("/doesnt/support/range-requests-non-authoritative") + .setData(TestUtil.buildTestData(/* length= */ 20, seed++)) + .supportsRangeRequests(false) + .setNonRangeResponseCode(203) + .setExtraResponseHeaders(EXTRA_HEADERS) + .build(); + public static final WebServerDispatcher.Resource GZIP_ENABLED = new WebServerDispatcher.Resource.Builder() .setPath("/gzip/enabled") @@ -139,6 +148,7 @@ public ImmutableList getServedResources() { createTestResource("range not supported", RANGE_NOT_SUPPORTED), createTestResource( "range not supported, length unknown", RANGE_NOT_SUPPORTED_LENGTH_UNKNOWN), + createTestResource("range not supported (203)", RANGE_NOT_SUPPORTED_NON_AUTHORITATIVE), // TODO: crbug.com/41313195 - Set these back to false when CronetDataSource and // HttpEngineDataSource are able to set Accept-Encoding: identity. createTestResource("gzip enabled", GZIP_ENABLED, /* mayResolveToUnknownLength= */ true), @@ -196,6 +206,7 @@ protected void before() throws Throwable { RANGE_SUPPORTED_LENGTH_UNKNOWN, RANGE_NOT_SUPPORTED, RANGE_NOT_SUPPORTED_LENGTH_UNKNOWN, + RANGE_NOT_SUPPORTED_NON_AUTHORITATIVE, GZIP_ENABLED, GZIP_FORCED, POST_EMPTY_REQUEST_BODY, diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/ImageReaderSurfaceHolder.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/ImageReaderSurfaceHolder.java index 52d4d43ff18..82cff612811 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/ImageReaderSurfaceHolder.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/ImageReaderSurfaceHolder.java @@ -145,7 +145,10 @@ public void drainSurface() { private void triggerCallbacks() { ImageReader reader = imageReader; - if (reader != null && (reader.getWidth() != width || reader.getHeight() != height)) { + if (reader != null + && (reader.getWidth() != width + || reader.getHeight() != height + || reader.getImageFormat() != format)) { reader.getSurface().release(); reader.close(); imageReader = null; diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/InMemoryDatabaseRule.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/InMemoryDatabaseRule.java index 8f38801e02e..1739cb53ebe 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/InMemoryDatabaseRule.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/InMemoryDatabaseRule.java @@ -15,6 +15,8 @@ */ package androidx.media3.test.utils; +import static com.google.common.base.StandardSystemProperty.JAVA_IO_TMPDIR; + import android.database.sqlite.SQLiteDatabase; import android.database.sqlite.SQLiteOpenHelper; import androidx.media3.common.util.UnstableApi; @@ -22,6 +24,7 @@ import androidx.media3.database.DefaultDatabaseProvider; import androidx.media3.datasource.cache.NoOpCacheEvictor; import androidx.media3.datasource.cache.SimpleCache; +import java.io.File; import java.io.IOException; import java.util.ArrayList; import java.util.List; @@ -51,8 +54,27 @@ private InMemoryDatabaseRule(RuleImpl ruleImpl, RuleChain ruleChain) { public static InMemoryDatabaseRule create() { TemporaryFolder temporaryFolder = new TemporaryFolder(); RuleImpl ruleImpl = new RuleImpl(temporaryFolder); + ExternalResource tmpDirSetupResource = + new ExternalResource() { + @Override + protected void before() throws Throwable { + // On some Android environments, particularly physical devices on clean runs, the + // directory returned by JAVA_IO_TMPDIR may not exist, causing issues with + // TemporaryFolder. + String tmpDirProperty = JAVA_IO_TMPDIR.value(); + if (tmpDirProperty != null) { + File tmpDir = new File(tmpDirProperty); + if (!tmpDir.exists()) { + if (!tmpDir.mkdirs()) { + throw new IOException("Failed to create java.io.tmpdir: " + tmpDir); + } + } + } + } + }; return new InMemoryDatabaseRule( - ruleImpl, RuleChain.outerRule(temporaryFolder).around(ruleImpl)); + ruleImpl, + RuleChain.outerRule(tmpDirSetupResource).around(temporaryFolder).around(ruleImpl)); } @Override diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/Media3FlagsRule.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/Media3FlagsRule.java new file mode 100644 index 00000000000..a0216b85697 --- /dev/null +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/Media3FlagsRule.java @@ -0,0 +1,199 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.test.utils; + +import static com.google.common.base.Preconditions.checkArgument; +import static com.google.common.base.Preconditions.checkNotNull; + +import androidx.media3.common.Flags; +import androidx.media3.common.util.UnstableApi; +import java.lang.reflect.Field; +import org.junit.rules.TestRule; +import org.junit.runner.Description; +import org.junit.runners.model.Statement; + +/** + * A JUnit {@link TestRule} for managing {@link Flags} states in tests. + * + *

    This rule ensures that all flag overrides and canary mode configurations are reset after each + * test to prevent test pollution. + * + *

    It automatically inspects the target object (usually {@code this}) and its superclasses for + * fields annotated with {@link BindFlag} and {@link BindCanaryMode}, and configures {@link Flags} + * before each test runs. + * + *

    Usage Examples: + * + *

    1. Automatic flag reset for manual flag overrides + * + *

    Applying the rule ensures that any flag or canary mode modifications made during test + * execution (e.g. via {@link Flags#enableFlag(int)} or {@link Flags#setCanaryModeEnabled(boolean)}) + * are automatically reset after each test finishes: + * + *

    + * public class MyPlaybackTest {
    + *
    + *   @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this);
    + *
    + *   @Test
    + *   public void playback_withFeatureFlagEnabled() {
    + *     Flags.enableFlag(Flags.FLAG_SOME_FEATURE);
    + *     // Test behavior with the flag enabled...
    + *   }
    + * }
    + * 
    + * + *

    2. Parameterized test on a feature flag with {@link BindFlag} + * + *

    The rule automatically enables or disables the flag before each test iteration according to + * the parameter field's value: + * + *

    + * @RunWith(ParameterizedRobolectricTestRunner.class)
    + * public class MyPlaybackTest {
    + *
    + *   @Parameter(0)
    + *   @BindFlag(Flags.FLAG_SOME_FEATURE)
    + *   public boolean someFeatureEnabled;
    + *
    + *   @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this);
    + *
    + *   @Test
    + *   public void playback() {
    + *     // Runs once with the flag disabled, and once with the flag enabled.
    + *   }
    + * }
    + * 
    + * + *

    3. Parameterized test on canary mode with {@link BindCanaryMode} + * + *

    The rule enables or disables canary mode before each test iteration: + * + *

    + * @RunWith(ParameterizedRobolectricTestRunner.class)
    + * public class MyPlaybackTest {
    + *
    + *   @Parameter(0)
    + *   @BindCanaryMode
    + *   public boolean canaryMode;
    + *
    + *   @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this);
    + *
    + *   @Test
    + *   public void playback() {
    + *     // Runs once with canary mode disabled, and once with canary mode enabled.
    + *   }
    + * }
    + * 
    + * + *

    4. Multiple flags or combined flag and canary mode bindings + * + *

    Multiple annotated fields can be declared to test combinations of flags: + * + *

    + * @RunWith(ParameterizedRobolectricTestRunner.class)
    + * public class MyPlaybackTest {
    + *
    + *   @Parameter(0)
    + *   @BindFlag(Flags.FLAG_FEATURE_A)
    + *   public boolean featureA;
    + *
    + *   @Parameter(1)
    + *   @BindFlag(Flags.FLAG_FEATURE_B)
    + *   public boolean featureB;
    + *
    + *   @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this);
    + *
    + *   @Test
    + *   public void playback() {
    + *     // Runs for each combination of flags.
    + *   }
    + * }
    + * 
    + */ +@UnstableApi +public final class Media3FlagsRule implements TestRule { + + private final Object target; + + /** + * Creates a rule that binds annotated fields on the {@code target} before each test and resets + * flags after each test. + * + * @param target The test instance (usually {@code this}) containing fields annotated with {@link + * BindFlag} or {@link BindCanaryMode}, or simply the test instance to ensure flags are reset. + */ + public Media3FlagsRule(Object target) { + this.target = checkNotNull(target); + } + + @Override + public Statement apply(Statement base, Description description) { + return new Statement() { + @Override + public void evaluate() throws Throwable { + Flags.resetForTesting(); + try { + bindFlags(target); + base.evaluate(); + } finally { + Flags.resetForTesting(); + } + } + }; + } + + private static void bindFlags(Object target) { + for (Class clazz = target.getClass(); clazz != null; clazz = clazz.getSuperclass()) { + for (Field field : clazz.getDeclaredFields()) { + if (field.isAnnotationPresent(BindCanaryMode.class)) { + field.setAccessible(true); + Object value; + try { + value = field.get(target); + } catch (IllegalAccessException e) { + throw new IllegalStateException("Failed to access field: " + field.getName(), e); + } + checkArgument( + value instanceof Boolean, + "Field %s annotated with @BindCanaryMode must be a boolean", + field.getName()); + Flags.setCanaryModeEnabled((Boolean) value); + } + if (field.isAnnotationPresent(BindFlag.class)) { + BindFlag bindFlag = checkNotNull(field.getAnnotation(BindFlag.class)); + field.setAccessible(true); + Object value; + try { + value = field.get(target); + } catch (IllegalAccessException e) { + throw new IllegalStateException("Failed to access field: " + field.getName(), e); + } + checkArgument( + value instanceof Boolean, + "Field %s annotated with @BindFlag must be a boolean", + field.getName()); + boolean enabled = (Boolean) value; + if (enabled) { + Flags.enableFlag(bindFlag.value()); + } else { + Flags.disableFlag(bindFlag.value()); + } + } + } + } + } +} diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/PlayerFence.kt b/libraries/test_utils/src/main/java/androidx/media3/test/utils/PlayerFence.kt index 3aaeb40ac92..f1be777b272 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/PlayerFence.kt +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/PlayerFence.kt @@ -38,14 +38,17 @@ import kotlin.time.Duration.Companion.milliseconds import kotlin.time.Duration.Companion.seconds import kotlinx.coroutines.CompletableDeferred import kotlinx.coroutines.CoroutineScope +import kotlinx.coroutines.CoroutineStart import kotlinx.coroutines.Job import kotlinx.coroutines.android.asCoroutineDispatcher import kotlinx.coroutines.channels.awaitClose +import kotlinx.coroutines.coroutineScope import kotlinx.coroutines.delay import kotlinx.coroutines.flow.Flow import kotlinx.coroutines.flow.callbackFlow import kotlinx.coroutines.flow.channelFlow import kotlinx.coroutines.flow.first +import kotlinx.coroutines.guava.await import kotlinx.coroutines.guava.future import kotlinx.coroutines.launch import kotlinx.coroutines.withTimeout @@ -71,32 +74,25 @@ suspend fun Player.awaitPlaybackState( failOnNonFatalErrors: Boolean = true, timeout: Duration? = null, ) { - playerError?.let { throw it } - - if (playbackState == targetState) { - return - } - val stateSeen = CompletableDeferred() val playerListener = - object : ErrorFailingPlayerListener(stateSeen::completeExceptionally) { + object : Player.Listener { override fun onPlaybackStateChanged(playbackState: @Player.State Int) { if (targetState == playbackState) { stateSeen.complete(Unit) } } } - addListener(playerListener) - val analyticsListener = - maybeAddAnalyticsListener(failOnNonFatalErrors) { - NonFatalFailingAnalyticsListener(stateSeen::completeExceptionally) - } try { - withTimeout(timeout) { stateSeen.await() } + awaitSupervised(failOnNonFatalErrors, timeout) { + if (playbackState == targetState) { + return@awaitSupervised + } + stateSeen.await() + } } finally { removeListener(playerListener) - maybeRemoveAnalyticsListener(analyticsListener) } } @@ -119,13 +115,9 @@ suspend fun Player.awaitIsPlaying( failOnNonFatalErrors: Boolean = true, timeout: Duration? = null, ) { - playerError?.let { throw it } - - if (isPlaying == targetIsPlaying) return - val conditionMet = CompletableDeferred() val playerListener = - object : ErrorFailingPlayerListener(conditionMet::completeExceptionally) { + object : Player.Listener { override fun onIsPlayingChanged(isPlaying: Boolean) { if (targetIsPlaying == isPlaying) { conditionMet.complete(Unit) @@ -134,15 +126,13 @@ suspend fun Player.awaitIsPlaying( } addListener(playerListener) - val analyticsListener = - maybeAddAnalyticsListener(failOnNonFatalErrors) { - NonFatalFailingAnalyticsListener(conditionMet::completeExceptionally) - } try { - withTimeout(timeout) { conditionMet.await() } + awaitSupervised(failOnNonFatalErrors, timeout) { + if (isPlaying == targetIsPlaying) return@awaitSupervised + conditionMet.await() + } } finally { removeListener(playerListener) - maybeRemoveAnalyticsListener(analyticsListener) } } @@ -164,25 +154,18 @@ suspend fun Player.awaitFirstFrameRendered( failOnNonFatalErrors: Boolean = true, timeout: Duration? = null, ) { - playerError?.let { throw it } - val renderedFirstFrame = CompletableDeferred() val playerListener = - object : ErrorFailingPlayerListener(renderedFirstFrame::completeExceptionally) { + object : Player.Listener { override fun onRenderedFirstFrame() { renderedFirstFrame.complete(Unit) } } addListener(playerListener) - val analyticsListener = - maybeAddAnalyticsListener(failOnNonFatalErrors) { - NonFatalFailingAnalyticsListener(renderedFirstFrame::completeExceptionally) - } try { - withTimeout(timeout) { renderedFirstFrame.await() } + awaitSupervised(failOnNonFatalErrors, timeout) { renderedFirstFrame.await() } } finally { removeListener(playerListener) - maybeRemoveAnalyticsListener(analyticsListener) } } @@ -212,21 +195,11 @@ suspend fun Player.awaitContentPositionAtLeast( timeout: Duration? = null, ) { require(targetPositionMs != C.TIME_UNSET) - playerError?.let { throw it } - - if (contentPosition >= targetPositionMs) { - return - } - if (playbackState == Player.STATE_ENDED) { - throw IllegalStateException( - "Playback already ended at position ${contentPosition}ms, before target of ${targetPositionMs}ms" - ) - } val currentMediaItemIndex = currentMediaItemIndex val speedChanges: Flow = callbackFlow { val listener = - object : ErrorFailingPlayerListener(::close) { + object : Player.Listener { override fun onPlaybackStateChanged(playbackState: @Player.State Int) { if (playbackState == Player.STATE_ENDED) { val contentPositionMs = contentPosition @@ -271,15 +244,9 @@ suspend fun Player.awaitContentPositionAtLeast( } addListener(listener) - val analyticsListener = - maybeAddAnalyticsListener(failOnNonFatalErrors) { NonFatalFailingAnalyticsListener(::close) } - send(playbackParameters.speed) - awaitClose { - removeListener(listener) - maybeRemoveAnalyticsListener(analyticsListener) - } + awaitClose { removeListener(listener) } } val targetReachedEvents: Flow = channelFlow { @@ -301,7 +268,62 @@ suspend fun Player.awaitContentPositionAtLeast( } } - withTimeout(timeout) { targetReachedEvents.first() } + awaitSupervised(failOnNonFatalErrors, timeout) { + if (contentPosition >= targetPositionMs) { + return@awaitSupervised + } + if (playbackState == Player.STATE_ENDED) { + throw IllegalStateException( + "Playback already ended at position ${contentPosition}ms, before target of ${targetPositionMs}ms" + ) + } + targetReachedEvents.first() + } +} + +/** + * Suspends until [block] completes, or a playback error occurs, or [timeout] expires. + * + * If a fatal [Player.playerError] or non-fatal [AnalyticsListener] error occurs, or if [timeout] + * expires, the [block] coroutine is cancelled immediately and the exception is thrown. + * + * Must be called on the player's application looper thread. + * + * @param failOnNonFatalErrors Whether non-fatal errors (such as those from + * [AnalyticsListener.onAudioCodecError]) cause an exception to be thrown immediately. + * @param timeout The max time to wait for, or `null` to use a default timeout of 10 seconds. + * @param block The suspending computation to execute within the supervision scope. + */ +@UnstableApi +suspend fun Player.awaitSupervised( + failOnNonFatalErrors: Boolean = true, + timeout: Duration? = null, + block: suspend CoroutineScope.() -> T, +): T { + playerError?.let { throw it } + + return coroutineScope { + val result = CompletableDeferred() + val errorListener = ErrorFailingPlayerListener(result::completeExceptionally) + addListener(errorListener) + val analyticsListener = + maybeAddAnalyticsListener(failOnNonFatalErrors) { + NonFatalFailingAnalyticsListener(result::completeExceptionally) + } + launch(start = CoroutineStart.UNDISPATCHED) { + try { + result.complete(block()) + } catch (t: Throwable) { + result.completeExceptionally(t) + } + } + try { + withTimeout(timeout ?: DEFAULT_TIMEOUT) { result.await() } + } finally { + removeListener(errorListener) + maybeRemoveAnalyticsListener(analyticsListener) + } + } } /** @@ -379,6 +401,21 @@ private constructor( player.awaitContentPositionAtLeast(targetPositionMs, failOnNonFatalErrors, timeout) } + /** + * Returns a future that completes when the provided [future] completes, supervising it against + * [Player] playback errors and timeouts. + * + * Must be called on the player's application looper thread. + */ + fun withSupervision(future: ListenableFuture): ListenableFuture { + val looper = Looper.myLooper() ?: error("Must be called on a Looper thread") + val dispatcher = Handler(looper).asCoroutineDispatcher("JavaFenceDispatcher") + + return CoroutineScope(dispatcher).future { + player.awaitSupervised(failOnNonFatalErrors, timeout) { future.await() } + } + } + private fun createFuture(block: suspend () -> Unit): ListenableFuture { val looper = Looper.myLooper() ?: error("Must be called on a Looper thread") val dispatcher = Handler(looper).asCoroutineDispatcher("JavaFenceDispatcher") @@ -427,9 +464,6 @@ private fun Player.maybeRemoveAnalyticsListener(analyticsListener: AnalyticsList } } -private suspend fun withTimeout(duration: Duration?, block: suspend CoroutineScope.() -> T): T = - withTimeout(duration ?: DEFAULT_TIMEOUT, block) - private open class ErrorFailingPlayerListener(private val exceptionConsumer: (Exception) -> Unit) : Player.Listener { override fun onPlayerError(error: PlaybackException) { diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/RecordingHardwareBufferEffectsPipeline.kt b/libraries/test_utils/src/main/java/androidx/media3/test/utils/RecordingHardwareBufferEffectsPipeline.kt deleted file mode 100644 index 39d71357032..00000000000 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/RecordingHardwareBufferEffectsPipeline.kt +++ /dev/null @@ -1,84 +0,0 @@ -/* - * Copyright 2026 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package androidx.media3.test.utils - -import android.content.Context -import androidx.annotation.RequiresApi -import androidx.media3.common.util.Consumer -import androidx.media3.common.util.ExperimentalApi -import androidx.media3.effect.DefaultHardwareBufferEffectsPipeline -import androidx.media3.effect.HardwareBufferFrame -import androidx.media3.effect.HardwareBufferFrameQueue -import androidx.media3.effect.HardwareBufferJniWrapper -import androidx.media3.effect.PacketConsumer.Packet -import androidx.media3.effect.RenderingPacketConsumer -import com.google.common.collect.ImmutableList - -/** - * Wraps a [DefaultHardwareBufferEffectsPipeline] and triggers a callback on every [queuePacket] - * call. - */ -@ExperimentalApi // TODO: b/449956776 - Remove once FrameConsumer API is finalized. -class RecordingHardwareBufferEffectsPipeline -private constructor( - private val internalPipeline: - RenderingPacketConsumer, HardwareBufferFrameQueue>, - private val onQueue: (ImmutableList) -> ImmutableList?, -) : RenderingPacketConsumer, HardwareBufferFrameQueue> { - - override fun setRenderOutput(output: HardwareBufferFrameQueue?) { - internalPipeline.setRenderOutput(output) - } - - override fun setErrorConsumer(errorConsumer: Consumer) { - internalPipeline.setErrorConsumer(errorConsumer) - } - - override suspend fun queuePacket(packet: Packet>) { - when (packet) { - is Packet.EndOfStream -> internalPipeline.queuePacket(Packet.EndOfStream) - is Packet.Payload -> { - onQueue(packet.payload)?.let { processedPayload -> - internalPipeline.queuePacket(Packet.of(processedPayload)) - } - } - } - } - - override suspend fun release() { - internalPipeline.release() - } - - companion object { - - /** - * Create a [RecordingHardwareBufferEffectsPipeline] with a - * [DefaultHardwareBufferEffectsPipeline]. - */ - @JvmStatic - @RequiresApi(26) - fun create( - context: Context, - hardwareBufferJniWrapper: HardwareBufferJniWrapper, - onQueue: (ImmutableList) -> ImmutableList?, - ): RecordingHardwareBufferEffectsPipeline { - return RecordingHardwareBufferEffectsPipeline( - DefaultHardwareBufferEffectsPipeline.create(context, hardwareBufferJniWrapper), - onQueue, - ) - } - } -} diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/SequenceAssetInfo.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/SequenceAssetInfo.java index 1681dd202c0..69743726741 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/SequenceAssetInfo.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/SequenceAssetInfo.java @@ -23,13 +23,14 @@ import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableSet; import com.google.common.collect.Iterables; +import java.util.List; import java.util.Set; /** Test assets that describe various {@link EditedMediaItemSequence} configurations. */ @UnstableApi public final class SequenceAssetInfo { public final ImmutableList assets; - private final ImmutableSet<@TrackType Integer> trackTypes; + public final ImmutableSet<@TrackType Integer> trackTypes; private final boolean isLooping; public SequenceAssetInfo( @@ -54,15 +55,20 @@ public EditedMediaItemSequence getEditedMediaItemSequence() { EditedMediaItemSequence.Builder sequenceBuilder = new EditedMediaItemSequence.Builder(trackTypes); for (EditedMediaItemAssetInfo asset : assets) { - sequenceBuilder.addItem(checkNotNull(asset.editedMediaItem)); + sequenceBuilder.addItem(checkNotNull(asset.getEditedMediaItem())); } sequenceBuilder.setIsLooping(isLooping); return sequenceBuilder.build(); } public ImmutableList getExpectedVideoTimestampsUs() { + return getExpectedVideoTimestampsUs(assets, /* offsetUs= */ 0); + } + + public static ImmutableList getExpectedVideoTimestampsUs( + List assets, long offsetUs) { ImmutableList.Builder expectedVideoTimestampsUs = new ImmutableList.Builder<>(); - long previousDuration = 0; + long previousDuration = offsetUs; for (EditedMediaItemAssetInfo asset : assets) { long finalPreviousDuration = previousDuration; ImmutableList videoTimestampsUs = @@ -70,7 +76,7 @@ public ImmutableList getExpectedVideoTimestampsUs() { expectedVideoTimestampsUs.addAll( Iterables.transform( videoTimestampsUs, timestampUs -> finalPreviousDuration + timestampUs)); - previousDuration += checkNotNull(asset.editedMediaItem).getPresentationDurationUs(); + previousDuration += checkNotNull(asset.getEditedMediaItem()).getPresentationDurationUs(); } return expectedVideoTimestampsUs.build(); } diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/StubExoPlayer.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/StubExoPlayer.java index 0d0eedc6dfe..148b85616d6 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/StubExoPlayer.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/StubExoPlayer.java @@ -274,11 +274,23 @@ public int getVideoChangeFrameRateStrategy() { } @Override + public void addVideoFrameMetadataListener(VideoFrameMetadataListener listener) { + throw new UnsupportedOperationException(); + } + + @Override + public void removeVideoFrameMetadataListener(VideoFrameMetadataListener listener) { + throw new UnsupportedOperationException(); + } + + @Override + @Deprecated public void setVideoFrameMetadataListener(VideoFrameMetadataListener listener) { throw new UnsupportedOperationException(); } @Override + @Deprecated public void clearVideoFrameMetadataListener(VideoFrameMetadataListener listener) { throw new UnsupportedOperationException(); } diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/TestExoPlayerBuilder.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/TestExoPlayerBuilder.java index 726f0a14a97..11ed154a959 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/TestExoPlayerBuilder.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/TestExoPlayerBuilder.java @@ -15,6 +15,7 @@ */ package androidx.media3.test.utils; +import static androidx.annotation.RestrictTo.Scope.LIBRARY_GROUP; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.truth.Truth.assertThat; @@ -22,7 +23,9 @@ import android.os.Looper; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; +import androidx.annotation.RestrictTo; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.util.Clock; import androidx.media3.common.util.HandlerWrapper; import androidx.media3.common.util.UnstableApi; @@ -62,8 +65,6 @@ public class TestExoPlayerBuilder { private boolean deviceVolumeControlEnabled; private boolean suppressPlaybackWhenUnsuitableOutput; @Nullable private ExoPlayer.PreloadConfiguration preloadConfiguration; - private boolean dynamicSchedulingEnabled; - private boolean perStreamMediaProgressionEnabled; private int stuckPlayingDetectionTimeoutMs; private int stuckSuppressedDetectionTimeoutMs; private boolean enforceAdPlaybackOnTimelineRefresh; @@ -278,6 +279,7 @@ public TestExoPlayerBuilder setPlaybackLooper(Looper playbackLooper) { */ @CanIgnoreReturnValue @RequiresApi(35) + @RestrictTo(LIBRARY_GROUP) public TestExoPlayerBuilder setSuitableOutputChecker( SuitableOutputChecker suitableOutputChecker) { this.suitableOutputChecker = suitableOutputChecker; @@ -400,19 +402,6 @@ public TestExoPlayerBuilder setSuppressPlaybackOnUnsuitableOutput( return this; } - /** - * See {@link ExoPlayer.Builder#experimentalSetDynamicSchedulingEnabled(boolean)} for details. - * - * @param dynamicSchedulingEnabled Whether the player should enable dynamically schedule its - * playback loop for when {@link Renderer} progress can be made. - * @return This builder. - */ - @CanIgnoreReturnValue - public TestExoPlayerBuilder setDynamicSchedulingEnabled(boolean dynamicSchedulingEnabled) { - this.dynamicSchedulingEnabled = dynamicSchedulingEnabled; - return this; - } - /** * See {@link ExoPlayer.Builder#setStuckPlayingDetectionTimeoutMs} for details. * @@ -454,19 +443,6 @@ public TestExoPlayerBuilder setEnforceAdPlaybackOnTimelineRefresh( return this; } - /** - * See {@link ExoPlayer.Builder#enablePerStreamMediaProgression} for details. - * - * @param perStreamMediaProgressionEnabled Whether to enable per stream media period progression. - * @return This builder. - */ - @CanIgnoreReturnValue // TODO: b/510217604 - Remove this method. - public TestExoPlayerBuilder setPerStreamMediaProgressionEnabled( - boolean perStreamMediaProgressionEnabled) { - this.perStreamMediaProgressionEnabled = perStreamMediaProgressionEnabled; - return this; - } - /** Builds an {@link ExoPlayer} using the provided values or their defaults. */ public ExoPlayer build() { checkNotNull( @@ -491,7 +467,7 @@ public ExoPlayer build() { }; }; } - ExoPlayer.Builder.experimentalEnableStuckPlayingDetection = true; + Flags.enableFlag(Flags.FLAG_ENABLE_STUCK_PLAYING_DETECTION); ExoPlayer.Builder builder = new ExoPlayer.Builder(context, playerRenderersFactory) .setTrackSelector(trackSelector) @@ -506,11 +482,9 @@ public ExoPlayer build() { .setMaxSeekToPreviousPositionMs(maxSeekToPreviousPositionMs) .setDeviceVolumeControlEnabled(deviceVolumeControlEnabled) .setSuppressPlaybackOnUnsuitableOutput(suppressPlaybackWhenUnsuitableOutput) - .experimentalSetDynamicSchedulingEnabled(dynamicSchedulingEnabled) .setStuckPlayingDetectionTimeoutMs(stuckPlayingDetectionTimeoutMs) .setStuckSuppressedDetectionTimeoutMs(stuckSuppressedDetectionTimeoutMs) - .setEnforceAdPlaybackOnTimelineRefresh(enforceAdPlaybackOnTimelineRefresh) - .enablePerStreamMediaProgression(perStreamMediaProgressionEnabled); + .setEnforceAdPlaybackOnTimelineRefresh(enforceAdPlaybackOnTimelineRefresh); if (suitableOutputChecker != null) { builder.setSuitableOutputChecker(suitableOutputChecker); } diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/TestTransformerBuilder.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/TestTransformerBuilder.java index d0ceced2c9e..2a4be03195f 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/TestTransformerBuilder.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/TestTransformerBuilder.java @@ -34,6 +34,7 @@ import androidx.media3.transformer.Composition; import androidx.media3.transformer.DefaultEncoderFactory; import androidx.media3.transformer.DefaultMuxer; +import androidx.media3.transformer.ExportException; import androidx.media3.transformer.ExportResult; import androidx.media3.transformer.Transformer; import com.google.common.collect.ImmutableList; diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/TestUtil.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/TestUtil.java index 8610ce7ac68..868df25379f 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/TestUtil.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/TestUtil.java @@ -27,6 +27,7 @@ import android.database.sqlite.SQLiteDatabase; import android.database.sqlite.SQLiteOpenHelper; import android.graphics.Bitmap; +import android.graphics.Bitmap.Config; import android.graphics.Color; import android.media.MediaCodec; import android.net.Uri; @@ -35,6 +36,7 @@ import android.os.Parcel; import android.view.SurfaceView; import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; import androidx.media3.common.C; import androidx.media3.common.Format; import androidx.media3.common.MediaItem; @@ -160,6 +162,15 @@ public static float[] buildFloatTestSamples(int length, Random random) { return source; } + /** Returns an array of random doubles between {@code [-1; 1]} with the specified length. */ + public static double[] buildDoubleTestSamples(int length, Random random) { + double[] source = new double[length]; + for (int i = 0; i < length; i++) { + source[i] = (random.nextDouble() * 2.0) - 1.0; + } + return source; + } + /** * Generates a random string with the specified length. * @@ -439,6 +450,51 @@ public static void assertBitmapsAreSimilar( assertThat(getPsnr(expectedBitmap, actualBitmap)).isAtLeast(psnrThresholdDb); } + /** + * Asserts whether actual FP16 bitmap is very similar to the expected FP16 bitmap at some quality + * level. + * + *

    This is defined as their PSNR value is greater than or equal to the threshold. The higher + * the threshold, the more similar they are. + * + * @param expectedBitmap The expected FP16 bitmap. + * @param actualBitmap The actual FP16 bitmap. + * @param psnrThresholdDb The PSNR threshold (in dB), at or above which bitmaps are considered + * very similar. + */ + @RequiresApi(29) + public static void assertFp16BitmapsAreSimilar( + Bitmap expectedBitmap, Bitmap actualBitmap, double psnrThresholdDb) { + assertThat(getPsnrFloat(expectedBitmap, actualBitmap)).isAtLeast(psnrThresholdDb); + } + + @RequiresApi(29) + private static double getPsnrFloat(Bitmap firstBitmap, Bitmap secondBitmap) { + assertThat(firstBitmap.getWidth()).isEqualTo(secondBitmap.getWidth()); + assertThat(firstBitmap.getHeight()).isEqualTo(secondBitmap.getHeight()); + assertThat(firstBitmap.getConfig()).isEqualTo(Config.RGBA_F16); + assertThat(secondBitmap.getConfig()).isEqualTo(Config.RGBA_F16); + int width = firstBitmap.getWidth(); + int height = firstBitmap.getHeight(); + double mse = 0.0; + + for (int y = 0; y < height; y++) { + for (int x = 0; x < width; x++) { + Color firstColor = firstBitmap.getColor(x, y); + Color secondColor = secondBitmap.getColor(x, y); + + float dr = firstColor.red() - secondColor.red(); + float dg = firstColor.green() - secondColor.green(); + float db = firstColor.blue() - secondColor.blue(); + + mse += (dr * dr) + (dg * dg) + (db * db); + } + } + + double normalizedMse = mse / (width * height * 3); + return 10.0 * Math.log10(1.0 / normalizedMse); + } + /** * Calculates the Peak-Signal-to-Noise-Ratio value for 2 bitmaps. * diff --git a/libraries/test_utils/src/main/java/androidx/media3/test/utils/WebServerDispatcher.java b/libraries/test_utils/src/main/java/androidx/media3/test/utils/WebServerDispatcher.java index 18900e1a43c..da762046bf8 100644 --- a/libraries/test_utils/src/main/java/androidx/media3/test/utils/WebServerDispatcher.java +++ b/libraries/test_utils/src/main/java/androidx/media3/test/utils/WebServerDispatcher.java @@ -121,6 +121,7 @@ public static final class Builder { private byte[] requestBody; private byte @MonotonicNonNull [] data; private boolean supportsRangeRequests; + private int nonRangeResponseCode; private boolean includesContentLengthInRangeResponses; private boolean resolvesToUnknownLength; private @GzipSupport int gzipSupport; @@ -131,6 +132,7 @@ public Builder() { this.httpMethod = HTTP_METHOD_GET; this.requestHeaders = ImmutableListMultimap.of(); this.requestBody = EMPTY_BYTE_ARRAY; + this.nonRangeResponseCode = 200; this.includesContentLengthInRangeResponses = true; this.gzipSupport = GZIP_SUPPORT_DISABLED; this.extraResponseHeaders = ImmutableListMultimap.of(); @@ -143,6 +145,7 @@ private Builder(Resource resource) { this.requestBody = resource.getRequestBody(); this.data = resource.getData(); this.supportsRangeRequests = resource.supportsRangeRequests(); + this.nonRangeResponseCode = resource.getNonRangeResponseCode(); this.includesContentLengthInRangeResponses = resource.includesContentLengthInRangeResponses(); this.resolvesToUnknownLength = resource.resolvesToUnknownLength(); @@ -229,6 +232,19 @@ public Builder supportsRangeRequests(boolean supportsRangeRequests) { return this; } + /** + * Sets the HTTP response code to use when a range is not supported or not requested. + * + *

    Defaults to 200. + * + * @return this builder, for convenience. + */ + @CanIgnoreReturnValue + public Builder setNonRangeResponseCode(int nonRangeResponseCode) { + this.nonRangeResponseCode = nonRangeResponseCode; + return this; + } + /** * Sets if RFC 7233 HTTP 206 range responses should include a {@code Content-Length} header. * @@ -304,6 +320,7 @@ public Resource build() { requestBody, checkNotNull(data), supportsRangeRequests, + nonRangeResponseCode, includesContentLengthInRangeResponses, resolvesToUnknownLength, gzipSupport, @@ -317,6 +334,7 @@ public Resource build() { private final byte[] requestBody; private final byte[] data; private final boolean supportsRangeRequests; + private final int nonRangeResponseCode; private final boolean includesContentLengthInRangeResponses; private final boolean resolvesToUnknownLength; private final @GzipSupport int gzipSupport; @@ -329,6 +347,7 @@ private Resource( byte[] requestBody, byte[] data, boolean supportsRangeRequests, + int nonRangeResponseCode, boolean includesContentLengthInRangeResponses, boolean resolvesToUnknownLength, @GzipSupport int gzipSupport, @@ -339,12 +358,18 @@ private Resource( this.requestBody = requestBody; this.data = data; this.supportsRangeRequests = supportsRangeRequests; + this.nonRangeResponseCode = nonRangeResponseCode; this.includesContentLengthInRangeResponses = includesContentLengthInRangeResponses; this.resolvesToUnknownLength = resolvesToUnknownLength; this.gzipSupport = gzipSupport; this.extraResponseHeaders = extraResponseHeaders; } + /** The HTTP response code when a range is not supported or not requested. */ + public int getNonRangeResponseCode() { + return nonRangeResponseCode; + } + /** Returns the path this resource is available at. */ public String getPath() { return path; @@ -497,6 +522,7 @@ public MockResponse dispatch(RecordedRequest request) { @Nullable String rangeHeader = request.getHeader(HttpHeaders.RANGE); if (!resource.supportsRangeRequests() || rangeHeader == null) { + response.setResponseCode(resource.getNonRangeResponseCode()); setResponseBody( response, preferredContentCoding, diff --git a/libraries/test_utils/src/test/java/androidx/media3/test/utils/CapturingFrameProcessorTest.java b/libraries/test_utils/src/test/java/androidx/media3/test/utils/CapturingFrameProcessorTest.java new file mode 100644 index 00000000000..1400c17630d --- /dev/null +++ b/libraries/test_utils/src/test/java/androidx/media3/test/utils/CapturingFrameProcessorTest.java @@ -0,0 +1,144 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.test.utils; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.media3.common.Format; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.Frame; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import java.util.List; +import java.util.concurrent.atomic.AtomicReference; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Tests for {@link CapturingFrameProcessor}. */ +@RunWith(AndroidJUnit4.class) +public final class CapturingFrameProcessorTest { + + @Test + public void signalEndOfStream_addsEosEventAndSignalsOutput() { + FakeFrameWriter frameWriter = new FakeFrameWriter(); + FakeFrameProcessor.Factory fakeFactory = + new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ false); + CapturingFrameProcessor.Factory capturingFactory = + new CapturingFrameProcessor.Factory(fakeFactory); + CapturingFrameProcessor processor = + capturingFactory.create(frameWriter, Runnable::run, new FakeFrameProcessor.Listener()); + + processor.signalEndOfStream(); + + ImmutableList events = processor.getQueuedEvents(); + assertThat(events).hasSize(1); + assertThat(events.get(0)).isInstanceOf(CapturingFrameProcessor.EosEvent.class); + assertThat(frameWriter.eosSignaled.get()).isTrue(); + } + + @Test + public void queueAndSignalEndOfStream_maintainsOrder() { + FakeFrameWriter frameWriter = new FakeFrameWriter(); + FakeFrameProcessor.Factory fakeFactory = + new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ false); + CapturingFrameProcessor.Factory capturingFactory = + new CapturingFrameProcessor.Factory(fakeFactory); + CapturingFrameProcessor processor = + capturingFactory.create(frameWriter, Runnable::run, new FakeFrameProcessor.Listener()); + ImmutableList frames = ImmutableList.of(); + + boolean queued = processor.queue(frames); + processor.signalEndOfStream(); + + assertThat(queued).isTrue(); + ImmutableList events = processor.getQueuedEvents(); + assertThat(events).hasSize(2); + assertThat(events.get(0)).isInstanceOf(CapturingFrameProcessor.FramesEvent.class); + assertThat(events.get(1)).isInstanceOf(CapturingFrameProcessor.EosEvent.class); + } + + @Test + public void queue_multipleTimes_addsMultipleEvents() throws Exception { + FakeFrameWriter frameWriter = new FakeFrameWriter(); + FakeFrameProcessor.Factory fakeFactory = + new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ false); + CapturingFrameProcessor.Factory capturingFactory = + new CapturingFrameProcessor.Factory(fakeFactory); + CapturingFrameProcessor processor = + capturingFactory.create(frameWriter, Runnable::run, new FakeFrameProcessor.Listener()); + + // First queue call + AsyncFrame frame1 = new AsyncFrame(createPlaceholderFrame(), /* acquireFence= */ null); + ImmutableList frames1 = ImmutableList.of(frame1); + + boolean queued1 = processor.queue(frames1); + + // Second queue call + AsyncFrame frame2 = new AsyncFrame(createPlaceholderFrame(), /* acquireFence= */ null); + ImmutableList frames2 = ImmutableList.of(frame2); + + boolean queued2 = processor.queue(frames2); + + assertThat(queued1).isTrue(); + assertThat(queued2).isTrue(); + + ImmutableList events = processor.getQueuedEvents(); + assertThat(events).hasSize(2); + assertThat(((CapturingFrameProcessor.FramesEvent) events.get(0)).frames) + .isSameInstanceAs(frames1); + assertThat(((CapturingFrameProcessor.FramesEvent) events.get(1)).frames) + .isSameInstanceAs(frames2); + } + + @Test + public void queue_withOnQueueListener_invokesListenerWhenQueued() { + FakeFrameWriter frameWriter = new FakeFrameWriter(); + FakeFrameProcessor.Factory fakeFactory = + new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ false); + AtomicReference> receivedFrames = new AtomicReference<>(); + CapturingFrameProcessor.Factory capturingFactory = + new CapturingFrameProcessor.Factory(fakeFactory, receivedFrames::set); + CapturingFrameProcessor processor = + capturingFactory.create(frameWriter, Runnable::run, new FakeFrameProcessor.Listener()); + AsyncFrame frame = new AsyncFrame(createPlaceholderFrame(), /* acquireFence= */ null); + ImmutableList frames = ImmutableList.of(frame); + + boolean queued = processor.queue(frames); + + assertThat(queued).isTrue(); + assertThat(receivedFrames.get()).isSameInstanceAs(frames); + } + + private static Frame createPlaceholderFrame() { + return new Frame() { + @Override + public Format getFormat() { + return new Format.Builder().build(); + } + + @Override + public ImmutableMap getMetadata() { + return ImmutableMap.of(); + } + + @Override + public long getContentTimeUs() { + return 0; + } + }; + } +} diff --git a/libraries/test_utils/src/test/java/androidx/media3/test/utils/FakeClockTest.java b/libraries/test_utils/src/test/java/androidx/media3/test/utils/FakeClockTest.java index aa6a1652fa4..e16da13f20f 100644 --- a/libraries/test_utils/src/test/java/androidx/media3/test/utils/FakeClockTest.java +++ b/libraries/test_utils/src/test/java/androidx/media3/test/utils/FakeClockTest.java @@ -713,6 +713,34 @@ public void backgroundLooperQuit_doesNotStallMainLooper() throws Exception { assertThat(mainMessageRun.get()).isTrue(); } + @Test(timeout = 1000) + public void postFromQuittingExternalLooper_doesNotStallMainLooper() throws Exception { + HandlerThread handlerThread = new HandlerThread("ExternalBackgroundThread"); + handlerThread.start(); + FakeClock fakeClock = new FakeClock(/* initialTimeMs= */ 0); + HandlerWrapper mainHandler = + fakeClock.createHandler(Looper.getMainLooper(), /* callback= */ null); + Handler rawBackgroundHandler = new Handler(handlerThread.getLooper()); + AtomicBoolean mainMessageRun = new AtomicBoolean(); + ConditionVariable backgroundTaskFinished = new ConditionVariable(); + + rawBackgroundHandler.post( + () -> { + handlerThread.quit(); + mainHandler.post(() -> mainMessageRun.set(true)); + backgroundTaskFinished.open(); + }); + + backgroundTaskFinished.block(); + quitAndJoin(handlerThread); + + // Trigger the message on the main thread. If the clock is locked up, this will timeout or do + // nothing. + ShadowLooper.idleMainLooper(); + + assertThat(mainMessageRun.get()).isTrue(); + } + private static class TestActivity extends Activity { public Button button; diff --git a/libraries/test_utils/src/test/java/androidx/media3/test/utils/FakeFrameProcessorTest.java b/libraries/test_utils/src/test/java/androidx/media3/test/utils/FakeFrameProcessorTest.java index 53dd3d922c8..ddc79518db6 100644 --- a/libraries/test_utils/src/test/java/androidx/media3/test/utils/FakeFrameProcessorTest.java +++ b/libraries/test_utils/src/test/java/androidx/media3/test/utils/FakeFrameProcessorTest.java @@ -17,18 +17,13 @@ import static com.google.common.truth.Truth.assertThat; -import androidx.annotation.Nullable; import androidx.media3.common.Format; import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.video.AsyncFrame; import androidx.media3.common.video.Frame; -import androidx.media3.common.video.FrameProcessor; -import androidx.media3.common.video.FrameWriter; -import androidx.media3.common.video.SyncFenceWrapper; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableMap; -import java.util.concurrent.Executor; import org.junit.Test; import org.junit.runner.RunWith; @@ -40,152 +35,49 @@ public final class FakeFrameProcessorTest { public void factoryCreate_returnsProcessorAndMaintainsReference() { FakeFrameProcessor.Factory factory = new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ false); - TestFrameWriter frameWriter = new TestFrameWriter(); - FakeFrameProcessor processor = factory.create(frameWriter); + FakeFrameProcessor processor = factory.create(new FakeFrameWriter()); assertThat(processor).isNotNull(); assertThat(factory.createdProcessor).isSameInstanceAs(processor); } - @Test - public void signalEndOfStream_addsEosEventAndSignalsOutput() { - TestFrameWriter frameWriter = new TestFrameWriter(); - FakeFrameProcessor processor = - new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ false) - .create(frameWriter); - - processor.signalEndOfStream(); - - ImmutableList events = processor.getQueuedEvents(); - assertThat(events).hasSize(1); - assertThat(events.get(0)).isInstanceOf(FakeFrameProcessor.EosEvent.class); - assertThat(frameWriter.signalEndOfStreamCalled).isTrue(); - } - - @Test - public void queueAndSignalEndOfStream_maintainsOrder() { - TestFrameWriter frameWriter = new TestFrameWriter(); - FakeFrameProcessor processor = - new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ false) - .create(frameWriter); - ImmutableList frames = ImmutableList.of(); - - boolean queued = processor.queue(frames); - processor.signalEndOfStream(); - - assertThat(queued).isTrue(); - ImmutableList events = processor.getQueuedEvents(); - assertThat(events).hasSize(2); - assertThat(events.get(0)).isInstanceOf(FakeFrameProcessor.FramesEvent.class); - assertThat(events.get(1)).isInstanceOf(FakeFrameProcessor.EosEvent.class); - } - @Test public void close_doesNotThrow() { - TestFrameWriter frameWriter = new TestFrameWriter(); FakeFrameProcessor processor = new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ false) - .create(frameWriter); + .create(new FakeFrameWriter()); // Should complete without exceptions. processor.close(); } - @Test - public void queue_multipleTimes_updatesLastFieldsAndAddsMultipleEvents() throws Exception { - TestFrameWriter frameWriter = new TestFrameWriter(); - FrameProcessor.Listener listener = - new FrameProcessor.Listener() { - @Override - public void onWakeup() {} - - @Override - public void onError(VideoFrameProcessingException exception) {} - - @Override - public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper onCompleteFence) {} - }; - FakeFrameProcessor processor = - new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ false) - .create(frameWriter, Runnable::run, listener); - - // First queue call - AsyncFrame frame1 = new AsyncFrame(createPlaceholderFrame(), /* acquireFence= */ null); - ImmutableList frames1 = ImmutableList.of(frame1); - - boolean queued1 = processor.queue(frames1); - - // Second queue call - AsyncFrame frame2 = new AsyncFrame(createPlaceholderFrame(), /* acquireFence= */ null); - ImmutableList frames2 = ImmutableList.of(frame2); - - boolean queued2 = processor.queue(frames2); - - assertThat(queued1).isTrue(); - assertThat(queued2).isTrue(); - assertThat(processor.lastFrames).isSameInstanceAs(frames2); - - ImmutableList events = processor.getQueuedEvents(); - assertThat(events).hasSize(2); - assertThat(((FakeFrameProcessor.FramesEvent) events.get(0)).frames).isSameInstanceAs(frames1); - assertThat(((FakeFrameProcessor.FramesEvent) events.get(1)).frames).isSameInstanceAs(frames2); - } - @Test public void queue_withCompleteIncomingFramesTrue_invokesCompletionListener() throws Exception { - TestFrameWriter frameWriter = new TestFrameWriter(); - boolean[] listenerInvoked = new boolean[1]; - FrameProcessor.Listener listener = - new FrameProcessor.Listener() { - @Override - public void onWakeup() {} - - @Override - public void onError(VideoFrameProcessingException exception) {} - - @Override - public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper onCompleteFence) { - listenerInvoked[0] = true; - } - }; + FakeFrameProcessor.Listener listener = new FakeFrameProcessor.Listener(); FakeFrameProcessor processor = new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ true) - .create(frameWriter, Runnable::run, listener); + .create(new FakeFrameWriter(), Runnable::run, listener); AsyncFrame frame = new AsyncFrame(createPlaceholderFrame(), /* acquireFence= */ null); ImmutableList frames = ImmutableList.of(frame); boolean queued = processor.queue(frames); assertThat(queued).isTrue(); - assertThat(listenerInvoked[0]).isTrue(); + assertThat(listener.processedFrames).containsExactly(frame.frame); } @Test public void triggerError_invokesOnErrorListener() throws Exception { - TestFrameWriter frameWriter = new TestFrameWriter(); - VideoFrameProcessingException[] exceptionHolder = new VideoFrameProcessingException[1]; - FrameProcessor.Listener listener = - new FrameProcessor.Listener() { - @Override - public void onWakeup() {} - - @Override - public void onError(VideoFrameProcessingException exception) { - exceptionHolder[0] = exception; - } - - @Override - public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper onCompleteFence) {} - }; + FakeFrameProcessor.Listener listener = new FakeFrameProcessor.Listener(); FakeFrameProcessor processor = new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ false) - .create(frameWriter, Runnable::run, listener); + .create(new FakeFrameWriter(), Runnable::run, listener); VideoFrameProcessingException exception = new VideoFrameProcessingException("Test error"); processor.triggerError(exception); - assertThat(exceptionHolder[0]).isSameInstanceAs(exception); + assertThat(listener.error.get()).isSameInstanceAs(exception); } private static Frame createPlaceholderFrame() { @@ -206,35 +98,4 @@ public long getContentTimeUs() { } }; } - - /** A simple Fake {@link FrameWriter} for testing. */ - private static final class TestFrameWriter implements FrameWriter { - - private boolean signalEndOfStreamCalled; - - @Override - public Info getInfo() { - throw new UnsupportedOperationException(); - } - - @Override - public void configure(Format format, @Frame.Usage long usage) {} - - @Nullable - @Override - public AsyncFrame dequeueInputFrame(Executor wakeupExecutor, Runnable wakeupListener) { - return null; - } - - @Override - public void queueInputFrame(Frame frame, @Nullable SyncFenceWrapper writeCompleteFence) {} - - @Override - public void signalEndOfStream() { - signalEndOfStreamCalled = true; - } - - @Override - public void close() {} - } } diff --git a/libraries/test_utils/src/test/java/androidx/media3/test/utils/FakePlayerTest.kt b/libraries/test_utils/src/test/java/androidx/media3/test/utils/FakePlayerTest.kt new file mode 100644 index 00000000000..8d29fd9a318 --- /dev/null +++ b/libraries/test_utils/src/test/java/androidx/media3/test/utils/FakePlayerTest.kt @@ -0,0 +1,83 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.test.utils + +import androidx.media3.common.Player +import androidx.media3.common.SimpleBasePlayer.MediaItemData +import androidx.test.ext.junit.runners.AndroidJUnit4 +import com.google.common.truth.Truth.assertThat +import java.util.concurrent.TimeUnit +import org.junit.Test +import org.junit.runner.RunWith +import org.robolectric.shadows.ShadowLooper + +/** Unit tests for [FakePlayer]. */ +@RunWith(AndroidJUnit4::class) +class FakePlayerTest { + + private val seekablePlaylist = listOf(MediaItemData.Builder("item").setIsSeekable(true).build()) + + @Test + fun seekTo_whenIdle_keepsIdleStateAndUpdatesPosition() { + val player = FakePlayer(playlist = seekablePlaylist, bufferingDelayMs = 100) + + player.seekTo(/* positionMs= */ 500L) + ShadowLooper.idleMainLooper(100, TimeUnit.MILLISECONDS) + + assertThat(player.playbackState).isEqualTo(Player.STATE_IDLE) + assertThat(player.currentPosition).isEqualTo(500L) + } + + @Test + fun seekTo_whenReadyWithZeroBufferingDelay_staysReadyAndUpdatesPosition() { + val player = FakePlayer(playlist = seekablePlaylist, bufferingDelayMs = 0) + player.prepare() + + player.seekTo(/* positionMs= */ 500L) + + assertThat(player.playbackState).isEqualTo(Player.STATE_READY) + assertThat(player.currentPosition).isEqualTo(500L) + } + + @Test + fun seekTo_whenReadyWithBufferingDelay_buffersThenTransitionsToReady() { + val player = + FakePlayer( + playbackState = Player.STATE_READY, + playlist = seekablePlaylist, + bufferingDelayMs = 100, + ) + + player.seekTo(/* positionMs= */ 500L) + + assertThat(player.playbackState).isEqualTo(Player.STATE_BUFFERING) + assertThat(player.currentPosition).isEqualTo(500L) + + ShadowLooper.idleMainLooper(100, TimeUnit.MILLISECONDS) + + assertThat(player.playbackState).isEqualTo(Player.STATE_READY) + } + + @Test + fun seekTo_whenEndedWithEmptyTimeline_staysEnded() { + val player = FakePlayer(playbackState = Player.STATE_ENDED, bufferingDelayMs = 0) + + player.seekTo(/* mediaItemIndex= */ 0, /* positionMs= */ 500L) + + assertThat(player.playbackState).isEqualTo(Player.STATE_ENDED) + } +} diff --git a/libraries/test_utils/src/test/java/androidx/media3/test/utils/Media3FlagsRuleTest.java b/libraries/test_utils/src/test/java/androidx/media3/test/utils/Media3FlagsRuleTest.java new file mode 100644 index 00000000000..7acbeed73c4 --- /dev/null +++ b/libraries/test_utils/src/test/java/androidx/media3/test/utils/Media3FlagsRuleTest.java @@ -0,0 +1,252 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.test.utils; + +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; + +import androidx.media3.common.Flags; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import org.junit.Test; +import org.junit.runner.Description; +import org.junit.runner.RunWith; +import org.junit.runners.model.Statement; + +/** Unit tests for {@link Media3FlagsRule}. */ +@RunWith(AndroidJUnit4.class) +public class Media3FlagsRuleTest { + + private static final @Flags.Flag int TEST_FLAG_1 = 1_000_000; + private static final @Flags.Flag int TEST_FLAG_2 = 1_000_001; + + private static final Statement EMPTY_STATEMENT = + new Statement() { + @Override + public void evaluate() {} + }; + + @Test + public void constructor_withNullTarget_throwsNullPointerException() { + assertThrows(NullPointerException.class, () -> new Media3FlagsRule(null)); + } + + @Test + public void apply_withoutAnnotatedFields_resetsFlagsAfterTest() throws Throwable { + Media3FlagsRule rule = new Media3FlagsRule(new Object()); + Statement testStatement = + new Statement() { + @Override + public void evaluate() { + Flags.enableFlag(TEST_FLAG_1); + assertThat(Flags.isEnabled(TEST_FLAG_1)).isTrue(); + } + }; + + rule.apply(testStatement, Description.EMPTY).evaluate(); + + assertThat(Flags.getEnabledOverrides()).asList().doesNotContain(TEST_FLAG_1); + } + + @Test + public void apply_withoutAnnotatedFields_resetsFlagsEvenIfTestThrows() { + Media3FlagsRule rule = new Media3FlagsRule(new Object()); + Statement testStatement = + new Statement() { + @Override + public void evaluate() { + Flags.enableFlag(TEST_FLAG_1); + throw new RuntimeException("Test failure"); + } + }; + Statement appliedStatement = rule.apply(testStatement, Description.EMPTY); + + assertThrows(RuntimeException.class, appliedStatement::evaluate); + + assertThat(Flags.getEnabledOverrides()).asList().doesNotContain(TEST_FLAG_1); + } + + @Test + public void apply_withBindFlagEnabled_enablesFlagDuringTestAndResetsAfter() throws Throwable { + Media3FlagsRule rule = new Media3FlagsRule(new TargetWithBindFlagTrue()); + Statement testStatement = + new Statement() { + @Override + public void evaluate() { + assertThat(Flags.isEnabled(TEST_FLAG_1)).isTrue(); + assertThat(Flags.getEnabledOverrides()).asList().contains(TEST_FLAG_1); + } + }; + + rule.apply(testStatement, Description.EMPTY).evaluate(); + + assertThat(Flags.getEnabledOverrides()).asList().doesNotContain(TEST_FLAG_1); + } + + @Test + public void apply_withBindFlagDisabled_disablesFlagDuringTestAndResetsAfter() throws Throwable { + Media3FlagsRule rule = new Media3FlagsRule(new TargetWithBindFlagFalse()); + Statement testStatement = + new Statement() { + @Override + public void evaluate() { + assertThat(Flags.isEnabled(TEST_FLAG_1)).isFalse(); + assertThat(Flags.getDisabledOverrides()).asList().contains(TEST_FLAG_1); + } + }; + + rule.apply(testStatement, Description.EMPTY).evaluate(); + + assertThat(Flags.getDisabledOverrides()).asList().doesNotContain(TEST_FLAG_1); + } + + @Test + public void apply_withBoxedBooleanBindFlag_bindsCorrectly() throws Throwable { + Media3FlagsRule rule = new Media3FlagsRule(new TargetWithBoxedBooleanBindFlag()); + Statement testStatement = + new Statement() { + @Override + public void evaluate() { + assertThat(Flags.isEnabled(TEST_FLAG_1)).isTrue(); + } + }; + + rule.apply(testStatement, Description.EMPTY).evaluate(); + } + + @Test + public void apply_withBindCanaryMode_bindsCanaryModeStateAndResetsAfter() throws Throwable { + Media3FlagsRule rule = new Media3FlagsRule(new TargetWithBindCanaryModeDisabled()); + Statement testStatement = + new Statement() { + @Override + public void evaluate() { + assertThat(Flags.isCanaryModeEnabled()).isFalse(); + } + }; + + rule.apply(testStatement, Description.EMPTY).evaluate(); + + assertThat(Flags.isCanaryModeEnabled()).isTrue(); + } + + @Test + public void apply_withMultipleFlagsAndCanaryMode_bindsAll() throws Throwable { + Media3FlagsRule rule = new Media3FlagsRule(new TargetWithMultipleFlagsAndCanaryMode()); + Statement testStatement = + new Statement() { + @Override + public void evaluate() { + assertThat(Flags.isCanaryModeEnabled()).isFalse(); + assertThat(Flags.isEnabled(TEST_FLAG_1)).isTrue(); + assertThat(Flags.isEnabled(TEST_FLAG_2)).isFalse(); + } + }; + + rule.apply(testStatement, Description.EMPTY).evaluate(); + + assertThat(Flags.isCanaryModeEnabled()).isTrue(); + assertThat(Flags.getEnabledOverrides()).asList().doesNotContain(TEST_FLAG_1); + assertThat(Flags.getDisabledOverrides()).asList().doesNotContain(TEST_FLAG_2); + } + + @Test + public void apply_withInheritedAnnotatedFields_bindsSuperclassFields() throws Throwable { + Media3FlagsRule rule = new Media3FlagsRule(new SubTestClass()); + Statement testStatement = + new Statement() { + @Override + public void evaluate() { + assertThat(Flags.isEnabled(TEST_FLAG_1)).isTrue(); + assertThat(Flags.isEnabled(TEST_FLAG_2)).isFalse(); + } + }; + + rule.apply(testStatement, Description.EMPTY).evaluate(); + } + + @Test + public void apply_withNonBooleanFieldForBindFlag_throwsIllegalArgumentException() { + Media3FlagsRule rule = new Media3FlagsRule(new TargetWithNonBooleanFieldForBindFlag()); + Statement appliedStatement = rule.apply(EMPTY_STATEMENT, Description.EMPTY); + + assertThrows(IllegalArgumentException.class, appliedStatement::evaluate); + } + + @Test + public void apply_withNonBooleanFieldForBindCanaryMode_throwsIllegalArgumentException() { + Media3FlagsRule rule = new Media3FlagsRule(new TargetWithNonBooleanFieldForBindCanaryMode()); + Statement appliedStatement = rule.apply(EMPTY_STATEMENT, Description.EMPTY); + + assertThrows(IllegalArgumentException.class, appliedStatement::evaluate); + } + + @SuppressWarnings("FieldCanBeStatic") + private static class TargetWithBindFlagTrue { + @BindFlag(TEST_FLAG_1) + private final boolean flagEnabled = true; + } + + @SuppressWarnings("FieldCanBeStatic") + private static class TargetWithBindFlagFalse { + @BindFlag(TEST_FLAG_1) + private final boolean flagEnabled = false; + } + + @SuppressWarnings("FieldCanBeStatic") + private static class TargetWithBoxedBooleanBindFlag { + @BindFlag(TEST_FLAG_1) + private final Boolean flagEnabled = Boolean.TRUE; + } + + @SuppressWarnings("FieldCanBeStatic") + private static class TargetWithBindCanaryModeDisabled { + @BindCanaryMode private final boolean canaryMode = false; + } + + @SuppressWarnings("FieldCanBeStatic") + private static class TargetWithMultipleFlagsAndCanaryMode { + @BindCanaryMode private final boolean canaryMode = false; + + @BindFlag(TEST_FLAG_1) + private final boolean flag1 = true; + + @BindFlag(TEST_FLAG_2) + private final boolean flag2 = false; + } + + @SuppressWarnings("FieldCanBeStatic") + private static class BaseTestClass { + @BindFlag(TEST_FLAG_1) + private final boolean baseFlag = true; + } + + @SuppressWarnings("FieldCanBeStatic") + private static class SubTestClass extends BaseTestClass { + @BindFlag(TEST_FLAG_2) + private final boolean subFlag = false; + } + + @SuppressWarnings("FieldCanBeStatic") + private static class TargetWithNonBooleanFieldForBindFlag { + @BindFlag(TEST_FLAG_1) + private final int invalidType = 123; + } + + @SuppressWarnings("FieldCanBeStatic") + private static class TargetWithNonBooleanFieldForBindCanaryMode { + @BindCanaryMode private final String invalidType = "true"; + } +} diff --git a/libraries/test_utils/src/test/java/androidx/media3/test/utils/WebServerDispatcherTest.java b/libraries/test_utils/src/test/java/androidx/media3/test/utils/WebServerDispatcherTest.java index 5ccd1fa7fba..5af0653a139 100644 --- a/libraries/test_utils/src/test/java/androidx/media3/test/utils/WebServerDispatcherTest.java +++ b/libraries/test_utils/src/test/java/androidx/media3/test/utils/WebServerDispatcherTest.java @@ -26,6 +26,7 @@ import okhttp3.Request; import okhttp3.Response; import okhttp3.mockwebserver.MockWebServer; +import org.junit.After; import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; @@ -118,6 +119,11 @@ public void setupServer() { .build()))); } + @After + public void teardownServer() throws Exception { + mockWebServer.shutdown(); + } + @Test public void rangeRequestsSupported_handlesConventionalRequest() throws Exception { OkHttpClient client = new OkHttpClient(); diff --git a/libraries/test_utils_robolectric/src/main/java/androidx/media3/test/utils/robolectric/CapturingRenderersFactory.java b/libraries/test_utils_robolectric/src/main/java/androidx/media3/test/utils/robolectric/CapturingRenderersFactory.java index cfec77944b0..d872e4405b4 100644 --- a/libraries/test_utils_robolectric/src/main/java/androidx/media3/test/utils/robolectric/CapturingRenderersFactory.java +++ b/libraries/test_utils_robolectric/src/main/java/androidx/media3/test/utils/robolectric/CapturingRenderersFactory.java @@ -26,12 +26,10 @@ import android.util.SparseArray; import androidx.annotation.Nullable; import androidx.media3.common.C; -import androidx.media3.common.MimeTypes; +import androidx.media3.common.Flags; import androidx.media3.common.util.Clock; -import androidx.media3.common.util.ExperimentalApi; import androidx.media3.common.util.UnstableApi; import androidx.media3.exoplayer.DefaultRenderersFactory; -import androidx.media3.exoplayer.ExoPlayer; import androidx.media3.exoplayer.Renderer; import androidx.media3.exoplayer.RenderersFactory; import androidx.media3.exoplayer.audio.AudioRendererEventListener; @@ -80,7 +78,6 @@ public class CapturingRenderersFactory implements RenderersFactory, Dumper.Dumpa private ImageDecoder.Factory imageDecoderFactory; private TextRendererFactory textRendererFactory; - private boolean parseAv1SampleDependencies; /** * Creates an instance. @@ -107,7 +104,7 @@ public CapturingRenderersFactory( this.audioSink = capturingAudioSink; this.imageDecoderFactory = new BitmapFactoryImageDecoder.Factory(context); this.textRendererFactory = TextRenderer::new; - ExoPlayer.Builder.experimentalEnableStuckPlayingDetection = true; + Flags.enableFlag(Flags.FLAG_ENABLE_STUCK_PLAYING_DETECTION); } /** @@ -138,25 +135,6 @@ public CapturingRenderersFactory setTextRendererFactory(TextRendererFactory text return this; } - /** - * Sets whether {@link MimeTypes#VIDEO_AV1} bitstream parsing for sample dependency information is - * enabled. Knowing which input frames are not depended on can speed up seeking and reduce dropped - * frames. - * - *

    Defaults to {@code false}. - * - *

    This method is experimental and will be renamed or removed in a future release. - * - * @param parseAv1SampleDependencies Whether bitstream parsing is enabled. - */ - @CanIgnoreReturnValue - @ExperimentalApi // TODO: b/470365670 - Remove method once config is enabled by default. - public final CapturingRenderersFactory experimentalSetParseAv1SampleDependencies( - boolean parseAv1SampleDependencies) { - this.parseAv1SampleDependencies = parseAv1SampleDependencies; - return this; - } - @Override public Renderer[] createRenderers( Handler eventHandler, @@ -218,8 +196,7 @@ protected MediaCodecVideoRenderer createMediaCodecVideoRenderer( /* enableDecoderFallback= */ false, eventHandler, videoRendererEventListener, - DefaultRenderersFactory.MAX_DROPPED_VIDEO_FRAME_COUNT_TO_NOTIFY, - parseAv1SampleDependencies); + DefaultRenderersFactory.MAX_DROPPED_VIDEO_FRAME_COUNT_TO_NOTIFY); } /** @@ -250,8 +227,7 @@ protected CapturingMediaCodecVideoRenderer( boolean enableDecoderFallback, @Nullable Handler eventHandler, @Nullable VideoRendererEventListener eventListener, - int maxDroppedFramesToNotify, - boolean parseAv1SampleDependencies) { + int maxDroppedFramesToNotify) { super( new Builder(context) .setCodecAdapterFactory(codecAdapterFactory) @@ -261,9 +237,10 @@ protected CapturingMediaCodecVideoRenderer( .setEventHandler(eventHandler) .setEventListener(eventListener) .setMaxDroppedFramesToNotify(maxDroppedFramesToNotify) - .experimentalSetParseAv1SampleDependencies(parseAv1SampleDependencies) // Do not drop input buffers due to slow processing. - .experimentalSetLateThresholdToDropDecoderInputUs(C.TIME_UNSET)); + .experimentalSetLateThresholdToDropDecoderInputUs(C.TIME_UNSET) + // Do not skip buffers with identical vsync times as we can't control this from tests. + .setSkipBuffersWithIdenticalReleaseTime(false)); } @Override @@ -279,12 +256,6 @@ protected boolean shouldDropBuffersToKeyframe( // Do not drop output buffers due to slow processing. return false; } - - @Override - protected boolean shouldSkipBuffersWithIdenticalReleaseTime() { - // Do not skip buffers with identical vsync times as we can't control this from tests. - return false; - } } /** diff --git a/libraries/test_utils_robolectric/src/main/java/androidx/media3/test/utils/robolectric/FakeDownloader.java b/libraries/test_utils_robolectric/src/main/java/androidx/media3/test/utils/robolectric/FakeDownloader.java index f82bc8e92cd..87c4ba457d2 100644 --- a/libraries/test_utils_robolectric/src/main/java/androidx/media3/test/utils/robolectric/FakeDownloader.java +++ b/libraries/test_utils_robolectric/src/main/java/androidx/media3/test/utils/robolectric/FakeDownloader.java @@ -35,11 +35,14 @@ public class FakeDownloader implements Downloader { /** Timeout to use when blocking on conditions that we expect to become unblocked. */ private static final int TIMEOUT_MS = 10_000; + private static final int NEGATIVE_TIMEOUT_MS = 500; + private final DownloadRequest request; private final ConditionVariable downloadStarted; private final ConditionVariable removeStarted; private final ConditionVariable cancelStarted; - private final ConditionVariable finished; + private final ConditionVariable downloadFinished; + private final ConditionVariable removeFinished; private final ConditionVariable blocker; private final AtomicInteger downloadStartCount; private final AtomicInteger removeStartCount; @@ -48,6 +51,13 @@ public class FakeDownloader implements Downloader { private volatile boolean canceled; private volatile boolean enableDownloadIoException; + /** + * Tracks whether the most recent operation on this downloader is {@link #remove()}. Used by + * {@link #finish()} and {@link #fail()} to determine which condition variable ({@link + * #downloadFinished} vs {@link #removeFinished}) to await. + */ + private volatile boolean isRemoving; + /** * Creates a {@link FakeDownloader}. * @@ -58,7 +68,8 @@ public FakeDownloader(DownloadRequest request) { downloadStarted = createRobolectricConditionVariable(); removeStarted = createRobolectricConditionVariable(); cancelStarted = createRobolectricConditionVariable(); - finished = createRobolectricConditionVariable(); + downloadFinished = createRobolectricConditionVariable(); + removeFinished = createRobolectricConditionVariable(); blocker = createRobolectricConditionVariable(); downloadStartCount = new AtomicInteger(); removeStartCount = new AtomicInteger(); @@ -74,6 +85,7 @@ public void cancel() { @Override public void download(@Nullable ProgressListener progressListener) throws IOException { + isRemoving = false; downloadStartCount.incrementAndGet(); downloadStarted.open(); try { @@ -90,32 +102,41 @@ public void download(@Nullable ProgressListener progressListener) throws IOExcep throw new IOException(); } } finally { - finished.open(); + downloadFinished.open(); } } @Override public void remove() { + isRemoving = true; removeStartCount.incrementAndGet(); removeStarted.open(); try { block(); } finally { - finished.open(); + removeFinished.open(); } } /** Finishes the {@link #download} or {@link #remove} without an error. */ public void finish() throws InterruptedException { blocker.open(); - blockUntilFinished(); + if (isRemoving) { + blockUntilRemoveFinished(); + } else { + blockUntilDownloadFinished(); + } } /** Fails {@link #download} or {@link #remove} with an error. */ public void fail() throws InterruptedException { enableDownloadIoException = true; blocker.open(); - blockUntilFinished(); + if (isRemoving) { + blockUntilRemoveFinished(); + } else { + blockUntilDownloadFinished(); + } } /** Increments the number of bytes that the fake downloader has downloaded. */ @@ -147,7 +168,7 @@ public void assertRemoveStarted(boolean started) throws InterruptedException { assertThat(removeStarted.block(TIMEOUT_MS)).isTrue(); removeStarted.close(); } else { - assertThat(removeStarted.block(TIMEOUT_MS)).isFalse(); + assertThat(removeStarted.block(NEGATIVE_TIMEOUT_MS)).isFalse(); } } @@ -155,10 +176,10 @@ public void assertRemoveStarted(boolean started) throws InterruptedException { public void assertCanceled(boolean canceled) throws InterruptedException { if (canceled) { assertThat(cancelStarted.block(TIMEOUT_MS)).isTrue(); - blockUntilFinished(); + blockUntilDownloadFinished(); cancelStarted.close(); } else { - assertThat(cancelStarted.block(TIMEOUT_MS)).isFalse(); + assertThat(cancelStarted.block(NEGATIVE_TIMEOUT_MS)).isFalse(); } } @@ -174,8 +195,13 @@ private void block() { } } - private void blockUntilFinished() throws InterruptedException { - assertThat(finished.block(TIMEOUT_MS)).isTrue(); - finished.close(); + private void blockUntilDownloadFinished() throws InterruptedException { + assertThat(downloadFinished.block(TIMEOUT_MS)).isTrue(); + downloadFinished.close(); + } + + private void blockUntilRemoveFinished() throws InterruptedException { + assertThat(removeFinished.block(TIMEOUT_MS)).isTrue(); + removeFinished.close(); } } diff --git a/libraries/test_utils_robolectric/src/main/java/androidx/media3/test/utils/robolectric/TestPlayerRunHelper.java b/libraries/test_utils_robolectric/src/main/java/androidx/media3/test/utils/robolectric/TestPlayerRunHelper.java index 5a7ff8401aa..79ba367a0ba 100644 --- a/libraries/test_utils_robolectric/src/main/java/androidx/media3/test/utils/robolectric/TestPlayerRunHelper.java +++ b/libraries/test_utils_robolectric/src/main/java/androidx/media3/test/utils/robolectric/TestPlayerRunHelper.java @@ -373,6 +373,54 @@ public PlayerRunResult ignoringNonFatalErrors() { player, playBeforeWaiting, /* throwNonFatalErrors= */ false, timeoutMs); } + /** + * Runs tasks of the main {@link Looper} until the specified condition becomes true independent + * of a message on the main {@link Looper}. + * + *

    This method is useful for cases where the condition may change outside of a main {@link + * Looper} message, for example because it's checking a volatile variable or shared synchronized + * state that is updated on a background thread, or because checking the condition itself may + * cause it to become true. + * + *

    This method ensures the condition is checked within artificially created main {@link + * Looper} messages. When using a {@link androidx.media3.test.utils.FakeClock}, this guarantees + * the remainder of the test method is not executed in parallel with other background thread + * messages. + * + * @param clock The {@link Clock} to create a {@link HandlerWrapper} from. + * @param backgroundThreadCondition The condition to wait for. + * @throws PlaybackException If a playback error occurs. + * @throws IllegalStateException If non-fatal playback errors occur, and aren't {@linkplain + * #ignoringNonFatalErrors() ignored} (the non-fatal exceptions will be attached with {@link + * Throwable#addSuppressed(Throwable)}). + * @throws TimeoutException If the timeout is exceeded. The timeout is {@link + * RobolectricUtil#DEFAULT_TIMEOUT_MS} by default, or a specific value if this instance is + * created via {@link #withTimeoutMs(long)}. + */ + public void untilBackgroundThreadCondition( + Clock clock, Supplier backgroundThreadCondition) + throws PlaybackException, TimeoutException { + if (backgroundThreadCondition.get()) { + return; + } + AtomicBoolean conditionTrue = new AtomicBoolean(); + HandlerWrapper handler = + clock.createHandler(Util.getCurrentOrMainLooper(), /* callback= */ null); + Runnable checkCondition = + new Runnable() { + @Override + public void run() { + if (backgroundThreadCondition.get()) { + conditionTrue.set(true); + } else { + handler.postDelayed(this, /* delayMs= */ 1); + } + } + }; + handler.post(checkCondition); + runUntil(conditionTrue::get); + } + /** Runs the main {@link Looper} until {@code predicate} returns true or an error occurs. */ protected final void runUntil(Supplier predicate) throws PlaybackException, TimeoutException { @@ -639,25 +687,7 @@ public void untilPendingCommandsAreFullyHandled() throws TimeoutException { */ public void untilBackgroundThreadCondition(Supplier backgroundThreadCondition) throws PlaybackException, TimeoutException { - if (backgroundThreadCondition.get()) { - return; - } - AtomicBoolean conditionTrue = new AtomicBoolean(); - HandlerWrapper handler = - player.getClock().createHandler(Util.getCurrentOrMainLooper(), /* callback= */ null); - Runnable checkCondition = - new Runnable() { - @Override - public void run() { - if (backgroundThreadCondition.get()) { - conditionTrue.set(true); - } else { - handler.postDelayed(this, /* delayMs= */ 1); - } - } - }; - handler.post(checkCondition); - runUntil(conditionTrue::get); + this.untilBackgroundThreadCondition(player.getClock(), backgroundThreadCondition); } /** @@ -734,6 +764,34 @@ public void untilPendingCommandsAreFullyHandled() throws TimeoutException { player.getClock(), checkNotNull(player.getPlaybackLooper())); } + /** + * Runs tasks of the main {@link Looper} until the specified condition becomes true independent + * of a message on the main {@link Looper}. + * + *

    This method is useful for cases where the condition may change outside of a main {@link + * Looper} message, for example because it's checking a volatile variable or shared synchronized + * state that is updated on a background thread, or because checking the condition itself may + * cause it to become true. + * + *

    This method ensures the condition is checked within artificially created main {@link + * Looper} messages. When using a {@link androidx.media3.test.utils.FakeClock}, this guarantees + * the remainder of the test method is not executed in parallel with other background thread + * messages. + * + * @param backgroundThreadCondition The condition to wait for. + * @throws PlaybackException If a playback error occurs. + * @throws IllegalStateException If non-fatal playback errors occur, and aren't {@linkplain + * #ignoringNonFatalErrors() ignored} (the non-fatal exceptions will be attached with {@link + * Throwable#addSuppressed(Throwable)}). + * @throws TimeoutException If the timeout is exceeded. The timeout is {@link + * RobolectricUtil#DEFAULT_TIMEOUT_MS} by default, or a specific value if this instance is + * created via {@link #withTimeoutMs(long)}. + */ + public void untilBackgroundThreadCondition(Supplier backgroundThreadCondition) + throws PlaybackException, TimeoutException { + this.untilBackgroundThreadCondition(player.getClock(), backgroundThreadCondition); + } + @Override public CompositionPlayerRunResult withTimeoutMs(long timeoutMs) { checkState(!hasBeenUsed); diff --git a/libraries/transformer/build.gradle.kts b/libraries/transformer/build.gradle.kts index 7870a29f48c..d0e82829581 100644 --- a/libraries/transformer/build.gradle.kts +++ b/libraries/transformer/build.gradle.kts @@ -39,8 +39,10 @@ dependencies { testImplementation(project(":test-data")) testImplementation(libs.androidx.runner) testImplementation(libs.robolectric) + testImplementation(project(":lib-inspector")) testImplementation(libs.test.parameter.injector) testImplementation(libs.truth) + testImplementation(project(":lib-inspector")) androidTestRuntimeOnly(libs.dexmaker.mockito) androidTestImplementation(libs.junit) androidTestImplementation(libs.androidx.test.runner) @@ -50,6 +52,5 @@ dependencies { androidTestImplementation(libs.test.parameter.injector) androidTestImplementation(project(":lib-inspector")) androidTestImplementation(project(":lib-inspector-frame")) - androidTestImplementation(project(":lib-effect-ndk")) androidTestImplementation(project(":test-utils")) } diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/AndroidTestUtil.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/AndroidTestUtil.java index bd2bd6cbfe4..51745042938 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/AndroidTestUtil.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/AndroidTestUtil.java @@ -19,6 +19,7 @@ import static androidx.media3.test.utils.TestUtil.retrieveTrackFormat; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.truth.Truth.assertThat; +import static com.google.common.truth.Truth.assertWithMessage; import static com.google.common.util.concurrent.Futures.immediateFuture; import static java.util.concurrent.TimeUnit.SECONDS; @@ -31,6 +32,7 @@ import android.opengl.EGLContext; import android.opengl.EGLDisplay; import android.os.Handler; +import android.util.Rational; import androidx.annotation.Nullable; import androidx.media3.common.C; import androidx.media3.common.ColorInfo; @@ -46,6 +48,8 @@ import androidx.media3.common.util.GlUtil; import androidx.media3.common.util.NullableType; import androidx.media3.common.util.Size; +import androidx.media3.common.util.ThrowingRunnable; +import androidx.media3.common.util.Util; import androidx.media3.effect.ByteBufferGlEffect; import androidx.media3.effect.DefaultGlObjectsProvider; import androidx.media3.effect.GlEffect; @@ -65,6 +69,7 @@ import androidx.media3.muxer.BufferInfo; import androidx.media3.muxer.Muxer; import androidx.media3.muxer.MuxerException; +import androidx.media3.test.utils.AssetInfo; import androidx.media3.test.utils.BitmapPixelTestUtil; import androidx.media3.test.utils.FakeExtractorOutput; import androidx.media3.test.utils.FakeTrackOutput; @@ -73,7 +78,9 @@ import com.google.common.collect.ImmutableList; import com.google.common.collect.Iterables; import com.google.common.util.concurrent.ListenableFuture; +import java.io.File; import java.io.IOException; +import java.math.RoundingMode; import java.nio.ByteBuffer; import java.util.ArrayList; import java.util.List; @@ -108,6 +115,14 @@ protected PlaybackVideoGraphWrapper createPlaybackVideoGraphWrapper( private static final String TAG = "AndroidTestUtil"; + /** + * The minimum SDK version required for the hardware buffer frame processor pipeline. + * + *

    API 28 is required for Java {@link android.hardware.HardwareBuffer} operations such as + * {@link Image#getHardwareBuffer()} and {@link Bitmap#getHardwareBuffer()}. + */ + public static final int HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK = 28; + /** An {@link Effects} instance that forces video transcoding. */ public static final Effects FORCE_TRANSCODE_VIDEO_EFFECTS = new Effects( @@ -176,7 +191,7 @@ public VideoGraph create( } /** Runs the given task and blocks until it completes, or timeoutSeconds has elapsed. */ - public static void runAsyncTaskAndWait(ThrowingRunnable task, int timeoutSeconds) + public static void runAsyncTaskAndWait(ThrowingRunnable task, int timeoutSeconds) throws TimeoutException, InterruptedException { CountDownLatch countDownLatch = new CountDownLatch(1); AtomicReference<@NullableType Exception> unexpectedExceptionReference = @@ -260,11 +275,6 @@ protected boolean shouldDropBuffersToKeyframe( } } - /** A type that can be used to succinctly wrap throwing {@link Runnable} objects. */ - public interface ThrowingRunnable { - void run() throws Exception; - } - /** * Creates the GL objects needed to set up a GL environment including an {@link EGLDisplay} and an * {@link EGLContext}. @@ -377,6 +387,11 @@ public Codec createForVideoEncoding(Format format, @Nullable LogSessionId logSes return encoderFactory.createForVideoEncoding(format, logSessionId); } + @Override + public boolean isVideoFormatSupported(Format format) { + return encoderFactory.isVideoFormatSupported(format); + } + @Override public boolean audioNeedsEncoding() { return true; @@ -615,5 +630,47 @@ public static ImmutableList getVideoSampleTimesUs(String filePath) throws .getSampleTimesUs(); } + /** + * Asserts that the exported video frames are paced at a constant frame rate. + * + * @param filePath The {@link String filepath} of the exported video. + * @param expectedFps The expected {@link Rational frame rate}. + */ + public static void assertExportedVideoFrameRateIsConstant(String filePath, Rational expectedFps) + throws IOException { + ImmutableList sampleTimesUs = getVideoSampleTimesUs(filePath); + long expectedDeltaUs = + Util.scaleLargeValue( + 1_000_000L, + expectedFps.getDenominator(), + expectedFps.getNumerator(), + RoundingMode.HALF_UP); + for (int i = 1; i < sampleTimesUs.size(); i++) { + long deltaUs = sampleTimesUs.get(i) - sampleTimesUs.get(i - 1); + // Absolute timestamps rounded to the nearest microsecond can cause + // deltas to fluctuate by 1us from the exact mathematical duration. + assertWithMessage( + "Time between frames %s (%s) and %s (%s) does not match expected %s fps", + i - 1, sampleTimesUs.get(i - 1), i, sampleTimesUs.get(i), expectedFps) + .that(deltaUs) + .isWithin(1L) + .of(expectedDeltaUs); + } + } + + /** + * Asserts that the exported video matches the original video's frame count and timestamps. + * + * @param result The {@link ExportTestResult} from the transformer. + * @param assetInfo The original {@link AssetInfo} to compare against. + */ + public static void assertExportResultHasOriginalVideoTimestamps( + ExportTestResult result, AssetInfo assetInfo) throws IOException { + assertThat(new File(checkNotNull(result.filePath)).length()).isGreaterThan(0); + assertThat(getVideoSampleTimesUs(result.filePath)) + .containsExactlyElementsIn(assetInfo.videoTimestampsUs) + .inOrder(); + } + private AndroidTestUtil() {} } diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/BitmapToHardwareBufferConverterTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/BitmapToHardwareBufferConverterTest.java new file mode 100644 index 00000000000..26689b36687 --- /dev/null +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/BitmapToHardwareBufferConverterTest.java @@ -0,0 +1,418 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static androidx.media3.transformer.TransformerUtil.releaseIfNeeded; +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.util.concurrent.MoreExecutors.directExecutor; +import static java.util.concurrent.TimeUnit.MILLISECONDS; +import static org.junit.Assert.assertThrows; + +import android.graphics.Bitmap; +import android.graphics.Bitmap.Config; +import android.graphics.Color; +import android.graphics.ColorSpace; +import android.graphics.Matrix; +import android.hardware.HardwareBuffer; +import android.opengl.EGLContext; +import android.opengl.EGLDisplay; +import android.opengl.EGLSurface; +import android.opengl.GLES20; +import android.os.Build; +import androidx.media3.common.util.GlUtil; +import androidx.media3.common.video.HardwareBufferFrame; +import androidx.media3.effect.HardwareBufferJni; +import androidx.media3.test.utils.BitmapPixelTestUtil; +import androidx.test.filters.SdkSuppress; +import com.google.testing.junit.testparameterinjector.TestParameter; +import com.google.testing.junit.testparameterinjector.TestParameterInjector; +import java.io.IOException; +import java.util.ArrayList; +import java.util.List; +import java.util.concurrent.ExecutionException; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import java.util.concurrent.TimeoutException; +import org.checkerframework.checker.nullness.qual.MonotonicNonNull; +import org.junit.After; +import org.junit.AssumptionViolatedException; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TestName; +import org.junit.runner.RunWith; + +/** Instrumentation tests for {@link BitmapToHardwareBufferConverter}. */ +@RunWith(TestParameterInjector.class) +@SdkSuppress(minSdkVersion = 26) +public final class BitmapToHardwareBufferConverterTest { + + private enum BitmapType { + HARDWARE, + ARGB_8888, + RGBA_1010102, + RGBA_F16, + } + + private static final float MAX_AVG_PIXEL_DIFFERENCE = 1.0f; + private static final String INPUT_PATH = "media/png/first_frame_1920x1080.png"; + private static final long TEST_TIMEOUT_MS = 1000L; + + private @MonotonicNonNull ExecutorService executorService; + private @MonotonicNonNull BitmapToHardwareBufferConverter converter; + private @MonotonicNonNull List resourcesToClose; + + @Rule public final TestName testName = new TestName(); + + @Before + public void setUp() { + executorService = Executors.newSingleThreadExecutor(); + converter = + new BitmapToHardwareBufferConverter( + HardwareBufferJni.INSTANCE, + /* internalExecutor= */ executorService, + /* errorExecutor= */ directExecutor(), + /* errorCallback= */ (e) -> { + throw new AssertionError(e); + }); + resourcesToClose = new ArrayList<>(); + } + + @After + public void tearDown() throws Exception { + if (converter != null) { + converter.close(); + } + if (executorService != null) { + executorService.shutdown(); + } + for (AutoCloseable resource : resourcesToClose) { + resource.close(); + } + } + + @Test + public void getOrCreateRetainedFrame_validBitmap_copiesPixelsCorrectly( + @TestParameter BitmapType bitmapType) throws Exception { + checkBitmapTypeSupported(bitmapType); + Bitmap inputBitmap = readBitmap(bitmapType); + // The pixel comparison can only run on ARGB_8888 bitmaps, if the input is HARDWARE, copy it to + // a ARGB_8888 for the assertion. + Bitmap expectedBitmap = + bitmapType == BitmapType.ARGB_8888 + ? inputBitmap + : inputBitmap.copy(Config.ARGB_8888, /* isMutable= */ false); + + HardwareBufferFrame outputFrame = converter.getOrCreateRetainedFrame(inputBitmap); + + assertThat(outputFrame.getHardwareBuffer()).isNotNull(); + HardwareBuffer hardwareBuffer = outputFrame.getHardwareBuffer(); + assertThat(hardwareBuffer.getWidth()).isEqualTo(inputBitmap.getWidth()); + assertThat(hardwareBuffer.getHeight()).isEqualTo(inputBitmap.getHeight()); + Bitmap outputBitmap = readBitmapFromHardwareBuffer(hardwareBuffer); + assertThat( + BitmapPixelTestUtil.getBitmapAveragePixelAbsoluteDifferenceArgb8888( + expectedBitmap, outputBitmap, testName.getMethodName())) + .isLessThan(MAX_AVG_PIXEL_DIFFERENCE); + releaseIfNeeded(outputFrame, /* releaseFence= */ null); + } + + @Test + public void getOrCreateRetainedFrame_recycledBitmap_throwsIllegalStateException() + throws Exception { + Bitmap inputBitmap = BitmapPixelTestUtil.readBitmap(INPUT_PATH); + inputBitmap.recycle(); + + assertThrows( + IllegalStateException.class, () -> converter.getOrCreateRetainedFrame(inputBitmap)); + } + + @Test + public void getOrCreateRetainedFrame_repeatedBitmap_reusesSameBuffer( + @TestParameter BitmapType bitmapType) throws IOException { + checkBitmapTypeSupported(bitmapType); + + Bitmap inputBitmap = readBitmap(bitmapType); + + HardwareBufferFrame outputFrame1 = converter.getOrCreateRetainedFrame(inputBitmap); + HardwareBufferFrame outputFrame2 = converter.getOrCreateRetainedFrame(inputBitmap); + + assertThat(outputFrame1.getHardwareBuffer()).isSameInstanceAs(outputFrame2.getHardwareBuffer()); + assertThat(outputFrame1.getHardwareBuffer().isClosed()).isFalse(); + + releaseIfNeeded(outputFrame1, /* releaseFence= */ null); + releaseIfNeeded(outputFrame2, /* releaseFence= */ null); + } + + @Test + public void getOrCreateRetainedFrame_repeatedBitmapAfterRelease_reusesSameBuffer( + @TestParameter BitmapType bitmapType) throws IOException { + checkBitmapTypeSupported(bitmapType); + + Bitmap inputBitmap = readBitmap(bitmapType); + + HardwareBufferFrame outputFrame1 = converter.getOrCreateRetainedFrame(inputBitmap); + HardwareBuffer buffer1 = outputFrame1.getHardwareBuffer(); + releaseIfNeeded(outputFrame1, /* releaseFence= */ null); + HardwareBufferFrame outputFrame2 = converter.getOrCreateRetainedFrame(inputBitmap); + HardwareBuffer buffer2 = outputFrame2.getHardwareBuffer(); + + assertThat(buffer1).isSameInstanceAs(buffer2); + assertThat(buffer1.isClosed()).isFalse(); + + releaseIfNeeded(outputFrame2, /* releaseFence= */ null); + } + + @Test + public void getOrCreateRetainedFrame_differentSoftwareBitmap_createsNewBufferAndClosesOldBuffer( + @TestParameter({"ARGB_8888", "RGBA_1010102", "RGBA_F16"}) BitmapType bitmapType) + throws IOException, ExecutionException, InterruptedException, TimeoutException { + checkBitmapTypeSupported(bitmapType); + + Bitmap bitmap1 = readBitmap(bitmapType); + Bitmap bitmap2 = readBitmap(bitmapType); + + HardwareBufferFrame outputFrame1 = converter.getOrCreateRetainedFrame(bitmap1); + HardwareBufferFrame outputFrame2 = converter.getOrCreateRetainedFrame(bitmap2); + + HardwareBuffer buffer1 = outputFrame1.getHardwareBuffer(); + HardwareBuffer buffer2 = outputFrame2.getHardwareBuffer(); + + assertThat(buffer1).isNotSameInstanceAs(buffer2); + + // Converter released its hold on buffer1 when bitmap2 was converted. + // buffer1 is still held by outputFrame1. + assertThat(buffer1.isClosed()).isFalse(); + + releaseIfNeeded(outputFrame1, /* releaseFence= */ null); + // Ensure there are no pending tasks left on the executor. + executorService.submit(() -> {}).get(TEST_TIMEOUT_MS, MILLISECONDS); + + assertThat(buffer1.isClosed()).isTrue(); + + releaseIfNeeded(outputFrame2, /* releaseFence= */ null); + } + + @Test + @SdkSuppress(minSdkVersion = 26, maxSdkVersion = 30) + public void + getOrCreateRetainedFrame_differentHardwareBitmap_belowApi31_createsNewBufferAndClosesOldBuffer() + throws IOException, ExecutionException, InterruptedException, TimeoutException { + Bitmap bitmap1 = readBitmap(BitmapType.HARDWARE); + Bitmap bitmap2 = readBitmap(BitmapType.HARDWARE); + + HardwareBufferFrame outputFrame1 = converter.getOrCreateRetainedFrame(bitmap1); + HardwareBufferFrame outputFrame2 = converter.getOrCreateRetainedFrame(bitmap2); + + HardwareBuffer buffer1 = outputFrame1.getHardwareBuffer(); + HardwareBuffer buffer2 = outputFrame2.getHardwareBuffer(); + + assertThat(buffer1).isNotSameInstanceAs(buffer2); + + // Converter released its hold on buffer1 when bitmap2 was converted. + // buffer1 is still held by outputFrame1. + assertThat(buffer1.isClosed()).isFalse(); + + releaseIfNeeded(outputFrame1, /* releaseFence= */ null); + // Ensure there are no pending tasks left on the executor. + executorService.submit(() -> {}).get(TEST_TIMEOUT_MS, MILLISECONDS); + + assertThat(buffer1.isClosed()).isTrue(); + + releaseIfNeeded(outputFrame2, /* releaseFence= */ null); + } + + @Test + @SdkSuppress(minSdkVersion = 31) + public void getOrCreateRetainedFrame_differentHardwareBitmap_api31OrAbove_doesNotCloseOldBuffer() + throws IOException, ExecutionException, InterruptedException, TimeoutException { + Bitmap bitmap1 = readBitmap(BitmapType.HARDWARE); + Bitmap bitmap2 = readBitmap(BitmapType.HARDWARE); + + HardwareBufferFrame outputFrame1 = converter.getOrCreateRetainedFrame(bitmap1); + HardwareBufferFrame outputFrame2 = converter.getOrCreateRetainedFrame(bitmap2); + + HardwareBuffer buffer1 = outputFrame1.getHardwareBuffer(); + HardwareBuffer buffer2 = outputFrame2.getHardwareBuffer(); + + assertThat(buffer1).isNotSameInstanceAs(buffer2); + + // Converter released its hold on buffer1 when bitmap2 was converted. + // buffer1 is still held by outputFrame1. + assertThat(buffer1.isClosed()).isFalse(); + + releaseIfNeeded(outputFrame1, /* releaseFence= */ null); + // Ensure there are no pending tasks left on the executor. + executorService.submit(() -> {}).get(TEST_TIMEOUT_MS, MILLISECONDS); + + // The HardwareBuffer is owned by bitmap1, so releasing outputFrame1 does not close it. + assertThat(buffer1.isClosed()).isFalse(); + + releaseIfNeeded(outputFrame2, /* releaseFence= */ null); + } + + @Test + public void getOrCreateRetainedFrame_differentGenerationId_createsNewBuffer() { + // Cannot be tested with Config.Hardware because hardware backed bitmaps are immutable. + Bitmap bitmap = Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888); + + HardwareBufferFrame outputFrame1 = converter.getOrCreateRetainedFrame(bitmap); + bitmap.eraseColor(Color.RED); + HardwareBufferFrame outputFrame2 = converter.getOrCreateRetainedFrame(bitmap); + + HardwareBuffer buffer1 = outputFrame1.getHardwareBuffer(); + HardwareBuffer buffer2 = outputFrame2.getHardwareBuffer(); + + assertThat(buffer1).isNotSameInstanceAs(buffer2); + + releaseIfNeeded(outputFrame1, /* releaseFence= */ null); + releaseIfNeeded(outputFrame2, /* releaseFence= */ null); + } + + @Test + public void releaseOutputFrame_sharedBuffer_doesNotCloseSharedBuffer( + @TestParameter BitmapType bitmapType) throws Exception { + checkBitmapTypeSupported(bitmapType); + + Bitmap inputBitmap = readBitmap(bitmapType); + + HardwareBufferFrame outputFrame1 = converter.getOrCreateRetainedFrame(inputBitmap); + HardwareBufferFrame outputFrame2 = converter.getOrCreateRetainedFrame(inputBitmap); + + releaseIfNeeded(outputFrame1, /* releaseFence= */ null); + // Ensure there are no pending tasks left on the executor. + executorService.submit(() -> {}).get(TEST_TIMEOUT_MS, MILLISECONDS); + + assertThat(outputFrame1.getHardwareBuffer().isClosed()).isFalse(); + assertThat(outputFrame2.getHardwareBuffer().isClosed()).isFalse(); + + releaseIfNeeded(outputFrame2, /* releaseFence= */ null); + // Ensure there are no pending tasks left on the executor. + executorService.submit(() -> {}).get(TEST_TIMEOUT_MS, MILLISECONDS); + + assertThat(outputFrame1.getHardwareBuffer().isClosed()).isFalse(); + assertThat(outputFrame2.getHardwareBuffer().isClosed()).isFalse(); + } + + @Test + public void close_withoutErrors_shutsDownInternalExecutor() { + Bitmap bitmap = Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888); + HardwareBufferFrame outputFrame = converter.getOrCreateRetainedFrame(bitmap); + releaseIfNeeded(outputFrame, /* releaseFence= */ null); + + converter.close(); + + assertThat(executorService.isShutdown()).isTrue(); + } + + @Test + public void releaseOutputFrame_afterClose_closesBuffer() throws IOException { + Bitmap inputBitmap = BitmapPixelTestUtil.readBitmap(INPUT_PATH); + HardwareBufferFrame outputFrame = converter.getOrCreateRetainedFrame(inputBitmap); + + converter.close(); + + // Converter released its reference, but the frame still holds one. + assertThat(outputFrame.getHardwareBuffer().isClosed()).isFalse(); + + releaseIfNeeded(outputFrame, /* releaseFence= */ null); + + // This will run synchronously because the internal executor is shutdown. + assertThat(outputFrame.getHardwareBuffer().isClosed()).isTrue(); + } + + @Test + public void getOrCreateRetainedFrame_afterClose_throwsIllegalStateException() { + Bitmap bitmap = Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888); + + converter.close(); + + assertThrows(IllegalStateException.class, () -> converter.getOrCreateRetainedFrame(bitmap)); + } + + private Bitmap readBitmapFromHardwareBuffer(HardwareBuffer hardwareBuffer) throws Exception { + if (Build.VERSION.SDK_INT >= 29) { + Bitmap hardwareBitmap = + Bitmap.wrapHardwareBuffer(hardwareBuffer, ColorSpace.get(ColorSpace.Named.SRGB)); + assertThat(hardwareBitmap).isNotNull(); + return hardwareBitmap.copy(Bitmap.Config.ARGB_8888, false); + } + + int width = hardwareBuffer.getWidth(); + int height = hardwareBuffer.getHeight(); + EGLDisplay eglDisplay = GlUtil.getDefaultEglDisplay(); + EGLContext eglContext = GlUtil.createEglContext(eglDisplay); + EGLSurface eglSurface = GlUtil.createFocusedPlaceholderEglSurface(eglContext, eglDisplay); + + int texId = GlUtil.createTexture(width, height, false); + long eglImage = + HardwareBufferJni.INSTANCE.nativeCreateEglImageFromHardwareBuffer( + eglDisplay.getNativeHandle(), hardwareBuffer); + assertThat(eglImage).isNotEqualTo(0L); + + GLES20.glBindTexture(GLES20.GL_TEXTURE_2D, texId); + assertThat(HardwareBufferJni.INSTANCE.nativeBindEGLImage(GLES20.GL_TEXTURE_2D, eglImage)) + .isTrue(); + + int fboId = GlUtil.createFboForTexture(texId); + GlUtil.focusFramebufferUsingCurrentContext(fboId, width, height); + Bitmap bitmap = BitmapPixelTestUtil.createArgb8888BitmapFromFocusedGlFramebuffer(width, height); + + GLES20.glDeleteTextures(1, new int[] {texId}, 0); + assertThat( + HardwareBufferJni.INSTANCE.nativeDestroyEGLImage( + eglDisplay.getNativeHandle(), eglImage)) + .isTrue(); + GlUtil.destroyEglSurface(eglDisplay, eglSurface); + GlUtil.destroyEglContext(eglDisplay, eglContext); + + // OpenGL returns an upside down bitmap due to coordinate system differences, flip it before + // returning. + Matrix matrix = new Matrix(); + matrix.postScale(1, -1, bitmap.getWidth() / 2f, bitmap.getHeight() / 2f); + return Bitmap.createBitmap(bitmap, 0, 0, bitmap.getWidth(), bitmap.getHeight(), matrix, true); + } + + private static Bitmap readBitmap(BitmapType bitmapType) throws IOException { + switch (bitmapType) { + case ARGB_8888: + return BitmapPixelTestUtil.readBitmap(INPUT_PATH); + case HARDWARE: + return BitmapPixelTestUtil.readBitmap(INPUT_PATH) + .copy(Config.HARDWARE, /* isMutable= */ false); + case RGBA_1010102: + if (Build.VERSION.SDK_INT >= 33) { + return BitmapPixelTestUtil.readBitmap(INPUT_PATH) + .copy(Config.RGBA_1010102, /* isMutable= */ false); + } + return BitmapPixelTestUtil.readBitmap(INPUT_PATH); + case RGBA_F16: + if (Build.VERSION.SDK_INT >= 33) { + return BitmapPixelTestUtil.readBitmap(INPUT_PATH) + .copy(Config.RGBA_F16, /* isMutable= */ false); + } + return BitmapPixelTestUtil.readBitmap(INPUT_PATH); + } + throw new IllegalArgumentException(); + } + + private static void checkBitmapTypeSupported(BitmapType bitmapType) { + if ((bitmapType == BitmapType.RGBA_1010102 || bitmapType == BitmapType.RGBA_F16) + && Build.VERSION.SDK_INT < 33) { + throw new AssumptionViolatedException("HDR bitmap formats require API level 33+"); + } + } +} diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlaybackTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlaybackTest.java index a010aeae8c6..09bcdc4d6d9 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlaybackTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlaybackTest.java @@ -16,12 +16,11 @@ package androidx.media3.transformer; -import static android.os.Build.VERSION.SDK_INT; import static androidx.media3.common.Player.DISCONTINUITY_REASON_AUTO_TRANSITION; import static androidx.media3.common.Player.REPEAT_MODE_ALL; import static androidx.media3.common.Player.REPEAT_MODE_OFF; -import static androidx.media3.common.util.Util.isRunningOnEmulator; import static androidx.media3.test.utils.AssetInfo.MP4_ADVANCED_ASSET; +import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_WITH_INCREASING_TIMESTAMPS; import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_GAMMA22_1S; import static androidx.media3.test.utils.AssetInfo.MP4_SIMPLE_ASSET; import static androidx.media3.test.utils.AssetInfo.WAV_80KHZ_MONO_20_REPEATING_1_SAMPLES_ASSET; @@ -29,41 +28,42 @@ import static androidx.media3.test.utils.PlayerFence.futureWhen; import static androidx.test.platform.app.InstrumentationRegistry.getInstrumentation; import static com.google.common.truth.Truth.assertThat; -import static java.util.concurrent.TimeUnit.MILLISECONDS; import android.content.Context; import androidx.annotation.Nullable; import androidx.media3.common.C; import androidx.media3.common.Effect; +import androidx.media3.common.Flags; import androidx.media3.common.MediaItem; import androidx.media3.common.MediaItem.ClippingConfiguration; -import androidx.media3.common.PlaybackException; import androidx.media3.common.Player; import androidx.media3.common.audio.SpeedProvider; -import androidx.media3.common.util.NullableType; import androidx.media3.effect.GlEffect; import androidx.media3.exoplayer.DecoderCounters; import androidx.media3.exoplayer.audio.AudioSink; import androidx.media3.exoplayer.audio.DefaultAudioSink; import androidx.media3.exoplayer.audio.TeeAudioProcessor; +import androidx.media3.test.utils.CountDownFuture; +import androidx.media3.test.utils.Media3FlagsRule; import androidx.media3.test.utils.PassthroughAudioProcessor; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableSet; import com.google.common.collect.Iterables; +import com.google.common.util.concurrent.ListenableFuture; import com.google.common.util.concurrent.SettableFuture; import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.nio.ByteBuffer; import java.nio.ShortBuffer; import java.util.List; import java.util.concurrent.CopyOnWriteArrayList; -import java.util.concurrent.CountDownLatch; import java.util.concurrent.ExecutionException; import java.util.concurrent.atomic.AtomicInteger; import java.util.concurrent.atomic.AtomicReference; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; import org.junit.After; import org.junit.Ignore; +import org.junit.Rule; import org.junit.Test; import org.junit.runner.RunWith; @@ -71,7 +71,8 @@ @RunWith(AndroidJUnit4.class) public class CompositionPlaybackTest { - private static final long TEST_TIMEOUT_MS = isRunningOnEmulator() ? 20_000 : 10_000; + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); + private static final MediaItem VIDEO_MEDIA_ITEM = MediaItem.fromUri(MP4_ADVANCED_ASSET.uri); private static final long VIDEO_DURATION_US = MP4_ADVANCED_ASSET.videoDurationUs; private static final ImmutableList VIDEO_TIMESTAMPS_US = @@ -405,9 +406,9 @@ public void playback_withRepeatModeSet_succeeds() throws Exception { new Composition.Builder( EditedMediaItemSequence.withAudioAndVideoFrom(ImmutableList.of(editedMediaItem))) .build(); - CountDownLatch repetitionEndedLatch = new CountDownLatch(2); - AtomicReference<@NullableType PlaybackException> playbackException = new AtomicReference<>(); + CountDownFuture twoRepetitions = new CountDownFuture(2); + AtomicReference> supervisedFuture = new AtomicReference<>(); SettableFuture endedFuture = SettableFuture.create(); getInstrumentation() .runOnMainSync( @@ -422,27 +423,18 @@ public void onPositionDiscontinuity( Player.PositionInfo newPosition, int reason) { if (reason == DISCONTINUITY_REASON_AUTO_TRANSITION) { - repetitionEndedLatch.countDown(); - } - } - - @Override - public void onPlayerError(PlaybackException error) { - playbackException.set(error); - while (repetitionEndedLatch.getCount() > 0) { - repetitionEndedLatch.countDown(); + twoRepetitions.countDown(); } } }); + supervisedFuture.set(futureWhen(player).withSupervision(twoRepetitions)); player.setComposition(composition); player.setRepeatMode(REPEAT_MODE_ALL); player.prepare(); player.play(); }); - boolean latchTimedOut = !repetitionEndedLatch.await(TEST_TIMEOUT_MS, MILLISECONDS); + supervisedFuture.get().get(); - assertThat(playbackException.get()).isNull(); - assertThat(latchTimedOut).isFalse(); getInstrumentation().runOnMainSync(() -> player.setRepeatMode(REPEAT_MODE_OFF)); endedFuture.get(); } @@ -656,13 +648,8 @@ public void playback_withEditedMediaItemFrameRateSet_correctlyDropFrames() throw 566_666L, 600_000L, 633_333L, 666_666L, 700_000L, 733_333L, 766_666L, 800_000L, 833_333L, 866_666L, 900_000L, 933_333L, 966_666L, 983_333L) .inOrder(); - if (SDK_INT >= 34) { - // Frames are dropped by MediaCodec internally. - assertThat(videoDecoderCounters.skippedOutputBufferCount).isEqualTo(0); - } else { - // Frames are dropped after MediaCodec output. - assertThat(videoDecoderCounters.skippedOutputBufferCount).isEqualTo(29); - } + // Frames are dropped after MediaCodec output. + assertThat(videoDecoderCounters.skippedOutputBufferCount).isEqualTo(29); } @Test @@ -731,21 +718,20 @@ public void playback_clippedMediaWithOutOfOrderFrames_effectsReceiveCorrectTimes @Ignore("TODO: b/521754606 - Fix flakiness and re-enable") @Test - public void playback_withVeryShortItems_succeeds() throws Exception { + public void playback_withVeryShortClippedMediaItems_succeeds() throws Exception { // Use items with duration of 33ms to showcase the stuck playback issue (b/475201870). - String uri = "asset:///media/mp4/h264_4k_30fps_10sec.mp4"; ImmutableList.Builder editedMediaItems = new ImmutableList.Builder<>(); EditedMediaItem editedMediaItem = new EditedMediaItem.Builder( new MediaItem.Builder() - .setUri(uri) + .setUri(MP4_ASSET_WITH_INCREASING_TIMESTAMPS.uri) .setClippingConfiguration( new MediaItem.ClippingConfiguration.Builder() .setStartPositionMs(0) .setEndPositionMs(33L) .build()) .build()) - .setDurationUs(10_000_000) + .setDurationUs(MP4_ASSET_WITH_INCREASING_TIMESTAMPS.videoDurationUs) .build(); for (int i = 0; i < 3; i++) { editedMediaItems.add(editedMediaItem); @@ -784,6 +770,11 @@ private DecoderCounters runCompositionPlayer( boolean videoPrewarmingEnabled, boolean perStreamMediaProgressionEnabled) throws InterruptedException, ExecutionException { + if (perStreamMediaProgressionEnabled) { + Flags.enableFlag(Flags.FLAG_PER_STREAM_MEDIA_PROGRESSION); + } else { + Flags.disableFlag(Flags.FLAG_PER_STREAM_MEDIA_PROGRESSION); + } SettableFuture endedFuture = SettableFuture.create(); DecoderCountersListener decoderCountersListener = new DecoderCountersListener(); getInstrumentation() @@ -793,7 +784,6 @@ private DecoderCounters runCompositionPlayer( new CompositionPlayer.Builder(context) .setVideoPrewarmingEnabled(videoPrewarmingEnabled) .experimentalSetLateThresholdToDropInputUs(C.TIME_UNSET) - .setPerStreamMediaProgressionEnabled(perStreamMediaProgressionEnabled) .build(); endedFuture.setFuture(futureWhen(player).entersPlaybackState(Player.STATE_ENDED)); player.addAnalyticsListener(decoderCountersListener); diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerDefaultGlFrameProcessorTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerDefaultGlFrameProcessorTest.java index 452b549c57f..efa701088d4 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerDefaultGlFrameProcessorTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerDefaultGlFrameProcessorTest.java @@ -17,37 +17,34 @@ import static androidx.media3.common.Player.STATE_ENDED; import static androidx.media3.common.util.Util.isRunningOnEmulator; +import static androidx.media3.test.utils.AssetInfo.JPG_ULTRA_HDR_ASSET; import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_5S; import static androidx.media3.test.utils.PlayerFence.futureWhen; -import static androidx.media3.transformer.GlFrameProcessorTestUtil.closeTestingGlResources; import static com.google.common.truth.Truth.assertThat; import static java.util.concurrent.TimeUnit.MILLISECONDS; +import static org.junit.Assume.assumeTrue; import android.app.Instrumentation; import android.content.Context; import android.view.SurfaceView; -import androidx.annotation.Nullable; import androidx.media3.common.C; import androidx.media3.common.GlObjectsProvider; import androidx.media3.common.MediaItem; import androidx.media3.common.Player; -import androidx.media3.common.util.GlUtil; import androidx.media3.common.util.Util; import androidx.media3.effect.DefaultGlFrameProcessor; import androidx.media3.effect.DefaultGlObjectsProvider; import androidx.media3.effect.FrameProcessorUtils; -import androidx.media3.effect.ndk.HardwareBufferJni; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.rules.ActivityScenarioRule; import androidx.test.ext.junit.runners.AndroidJUnit4; import androidx.test.filters.SdkSuppress; import androidx.test.platform.app.InstrumentationRegistry; import com.google.common.collect.ImmutableList; -import com.google.common.util.concurrent.ListeningExecutorService; -import com.google.common.util.concurrent.MoreExecutors; import com.google.common.util.concurrent.SettableFuture; import java.util.Queue; import java.util.concurrent.ConcurrentLinkedQueue; +import java.util.concurrent.ExecutorService; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; import org.junit.After; import org.junit.Before; @@ -58,7 +55,7 @@ /** Instrumentation tests for {@link CompositionPlayer} using {@link DefaultGlFrameProcessor}. */ @RunWith(AndroidJUnit4.class) -@SdkSuppress(minSdkVersion = 28) +@SdkSuppress(minSdkVersion = AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) public final class CompositionPlayerDefaultGlFrameProcessorTest { private static final long TEST_TIMEOUT_MS = isRunningOnEmulator() ? 20_000 : 10_000; @@ -86,10 +83,11 @@ public final class CompositionPlayerDefaultGlFrameProcessorTest { private final Instrumentation instrumentation; private final Context context; + @Rule + public final GlFrameProcessorTestRule glFrameProcessorTestRule = new GlFrameProcessorTestRule(); + private @MonotonicNonNull CompositionPlayer compositionPlayer; private @MonotonicNonNull SurfaceView surfaceView; - private @MonotonicNonNull ListeningExecutorService glExecutorService; - private @MonotonicNonNull GlObjectsProvider glObjectsProvider; public CompositionPlayerDefaultGlFrameProcessorTest() { instrumentation = InstrumentationRegistry.getInstrumentation(); @@ -97,25 +95,12 @@ public CompositionPlayerDefaultGlFrameProcessorTest() { } @Before - public void setupSurfacesAndExecutor() throws Exception { + public void setupSurfaces() { rule.getScenario() .onActivity( activity -> { surfaceView = activity.getSurfaceView(); }); - glExecutorService = - MoreExecutors.listeningDecorator(Util.newSingleThreadExecutor("CompositionTest:GL")); - glObjectsProvider = new DefaultGlObjectsProvider(); - glExecutorService - .submit( - () -> { - try { - FrameProcessorUtils.setupOpenGl(glObjectsProvider); - } catch (GlUtil.GlException e) { - throw new AssertionError(e); - } - }) - .get(TEST_TIMEOUT_MS, MILLISECONDS); } @After @@ -127,15 +112,6 @@ public void releaseResources() { } }); rule.getScenario().close(); - @Nullable Exception releasingException = null; - if (glExecutorService != null) { - releasingException = - closeTestingGlResources(glExecutorService, glObjectsProvider, TEST_TIMEOUT_MS); - glExecutorService.shutdown(); - } - if (releasingException != null) { - throw new AssertionError(releasingException); - } } @Test @@ -198,15 +174,80 @@ public void compositionPlayer_playAfterEnded_doesNotTimeout() throws Exception { endedAfterPlayFuture.get(TEST_TIMEOUT_MS, MILLISECONDS); } + @SdkSuppress(minSdkVersion = 34) + @Test + public void + compositionPlayer_withUltraHdrImageAndDefaultGlFrameProcessor_outputsFrameBeforeEnding() + throws Exception { + assumeTrue(isOpenGlVersion3Supported()); + + SettableFuture endedFuture = SettableFuture.create(); + Queue videoTimestamps = new ConcurrentLinkedQueue<>(); + // Set a duration shorter than 1 frame period (1/30s ~ 33ms) so that exactly one frame at + // presentation timestamp 0 is requested before ending playback. + EditedMediaItem ultraHdrItem = + new EditedMediaItem.Builder(MediaItem.fromUri(JPG_ULTRA_HDR_ASSET.uri)) + .setDurationUs(10_000) + .setFrameRate(30) + .build(); + Composition composition = + new Composition.Builder( + EditedMediaItemSequence.withVideoFrom(ImmutableList.of(ultraHdrItem))) + .build(); + + instrumentation.runOnMainSync( + () -> { + compositionPlayer = createCompositionPlayer(); + endedFuture.setFuture( + futureWhen(compositionPlayer).entersPlaybackState(Player.STATE_ENDED)); + compositionPlayer.setVideoSurfaceView(surfaceView); + compositionPlayer.setVideoFrameMetadataListener( + (presentationTimeUs, releaseTimeNs, format, mediaFormat) -> { + videoTimestamps.add(presentationTimeUs); + }); + compositionPlayer.addListener( + new Player.Listener() { + @Override + public void onPlaybackStateChanged(int playbackState) { + if (playbackState == STATE_ENDED) { + videoTimestamps.add(-1L); + } + } + }); + compositionPlayer.setComposition(composition); + compositionPlayer.prepare(); + compositionPlayer.play(); + }); + endedFuture.get(TEST_TIMEOUT_MS, MILLISECONDS); + + assertThat(videoTimestamps).containsExactly(0L, -1L).inOrder(); + } + private CompositionPlayer createCompositionPlayer() { - DefaultGlFrameProcessor.Factory frameProcessorFactory = - new DefaultGlFrameProcessor.Factory( - context, glObjectsProvider, HardwareBufferJni.INSTANCE, glExecutorService); - return new CompositionPlayer.Builder(context) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setFrameProcessorFactory(frameProcessorFactory) + return glFrameProcessorTestRule + .createCompositionPlayerBuilder(context) // Disable frame dropping. .experimentalSetLateThresholdToDropInputUs(C.TIME_UNSET) .build(); } + + private static boolean isOpenGlVersion3Supported() throws Exception { + ExecutorService glExecutorService = Util.newSingleThreadExecutor("Test:GlThread"); + GlObjectsProvider glObjectsProvider = new DefaultGlObjectsProvider(); + try { + return glExecutorService + .submit( + () -> { + try { + return FrameProcessorUtils.setupOpenGl(glObjectsProvider) + == FrameProcessorUtils.OPEN_GL_VERSION_3; + } finally { + FrameProcessorUtils.releaseOpenGl(glObjectsProvider); + } + }) + .get(); + } finally { + FrameProcessorUtils.shutdownGlExecutorService(glExecutorService); + } + } } diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerParameterizedPlaybackTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerParameterizedPlaybackTest.java index 9b33e42c5b6..f5d2b62a217 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerParameterizedPlaybackTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerParameterizedPlaybackTest.java @@ -15,11 +15,12 @@ */ package androidx.media3.transformer; -import static androidx.media3.common.C.TRACK_TYPE_VIDEO; import static androidx.media3.common.util.Util.isRunningOnEmulator; import static androidx.media3.test.utils.CompositionAssetInfo.MULTI_SEQUENCE_CONFIGS; import static androidx.media3.test.utils.CompositionAssetInfo.MULTI_SEQUENCE_VIDEO_CONFIGS; import static androidx.media3.test.utils.CompositionAssetInfo.SINGLE_SEQUENCE_CONFIGS; +import static androidx.media3.test.utils.EditedMediaItemAssetInfo.VIDEO_ONLY_CLIPPED_HALF_SPEED; +import static androidx.media3.test.utils.EditedMediaItemAssetInfo.VIDEO_ONLY_CLIPPED_TWICE_SPEED; import static androidx.test.platform.app.InstrumentationRegistry.getInstrumentation; import static com.google.common.truth.Truth.assertThat; import static com.google.common.truth.TruthJUnit.assume; @@ -28,19 +29,16 @@ import android.view.SurfaceView; import androidx.annotation.RequiresApi; import androidx.media3.common.C; -import androidx.media3.common.MediaItem; import androidx.media3.common.Player; import androidx.media3.common.VideoGraph; +import androidx.media3.common.video.FrameProcessor; import androidx.media3.effect.GlEffect; -import androidx.media3.effect.HardwareBufferFrame; -import androidx.media3.effect.HardwareBufferFrameQueue; import androidx.media3.effect.MultipleInputVideoGraph; -import androidx.media3.effect.RenderingPacketConsumer; import androidx.media3.effect.SingleInputVideoGraph; -import androidx.media3.effect.ndk.HardwareBufferJni; +import androidx.media3.test.utils.CapturingFrameProcessor; import androidx.media3.test.utils.CompositionAssetInfo; +import androidx.media3.test.utils.FrameProcessorTestUtil; import androidx.media3.test.utils.PlayerFence; -import androidx.media3.test.utils.RecordingHardwareBufferEffectsPipeline; import androidx.test.ext.junit.rules.ActivityScenarioRule; import androidx.test.filters.SdkSuppress; import com.google.common.collect.ImmutableList; @@ -49,7 +47,6 @@ import com.google.testing.junit.testparameterinjector.TestParameterInjector; import com.google.testing.junit.testparameterinjector.TestParameterValuesProvider; import java.util.List; -import java.util.concurrent.CopyOnWriteArrayList; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; import org.junit.After; import org.junit.Before; @@ -72,6 +69,9 @@ public class CompositionPlayerParameterizedPlaybackTest { private @MonotonicNonNull CompositionPlayer player; private @MonotonicNonNull SurfaceView surfaceView; + @Rule + public final GlFrameProcessorTestRule glFrameProcessorTestRule = new GlFrameProcessorTestRule(); + private static class SingleInputVideoGraphConfigsProvider extends TestParameterValuesProvider { @Override protected List provideValues( @@ -95,11 +95,24 @@ private static class FrameConsumerConfigsProvider extends TestParameterValuesPro @Override protected List provideValues( TestParameterValuesProvider.Context context) { - return new ImmutableList.Builder() - .addAll(SINGLE_SEQUENCE_CONFIGS) - .addAll(MULTI_SEQUENCE_VIDEO_CONFIGS) - .addAll(MULTI_SEQUENCE_CONFIGS) - .build(); + ImmutableList allConfigs = + new ImmutableList.Builder() + .addAll(SINGLE_SEQUENCE_CONFIGS) + .addAll(MULTI_SEQUENCE_VIDEO_CONFIGS) + .addAll(MULTI_SEQUENCE_CONFIGS) + .build(); + ImmutableList.Builder filteredConfigs = new ImmutableList.Builder<>(); + // DefaultGlFrameProcessor doesn't support speed changing video effects. + // TODO: b/530108514 - Move tests to use EditedMediaItem.setSpeed(). + for (CompositionAssetInfo config : allConfigs) { + String configString = config.toString(); + if (configString.contains(VIDEO_ONLY_CLIPPED_HALF_SPEED.name) + || configString.contains(VIDEO_ONLY_CLIPPED_TWICE_SPEED.name)) { + continue; + } + filteredConfigs.add(config); + } + return filteredConfigs.build(); } } @@ -181,56 +194,31 @@ public void playback_multipleInputVideoGraph( } @Test - @SdkSuppress(minSdkVersion = 28) - public void playback_packetConsumer( + @SdkSuppress(minSdkVersion = AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) + public void playback_frameProcessor( @TestParameter(valuesProvider = FrameConsumerConfigsProvider.class) CompositionAssetInfo compositionAssetInfo) throws Exception { - // The MediaCodec decoder's output surface is sometimes dropping frames on emulator despite - // using MediaFormat.KEY_ALLOW_FRAME_DROP. assume() .withMessage("Skipped on emulator due to surface dropping frames") .that(isRunningOnEmulator()) .isFalse(); - List> queuedPackets = new CopyOnWriteArrayList<>(); - RecordingHardwareBufferEffectsPipeline pipeline = - RecordingHardwareBufferEffectsPipeline.create( - context, - HardwareBufferJni.INSTANCE, - (frames) -> { - queuedPackets.add(frames); - return frames; - }); - ImmutableList expectedVideoTimestampsUs = - compositionAssetInfo.getExpectedVideoTimestampsUs(); + CapturingFrameProcessor.Factory recordingFrameProcessorFactory = + new CapturingFrameProcessor.Factory( + glFrameProcessorTestRule.createDefaultGlFrameProcessorFactory(context)); Composition composition = compositionAssetInfo.getComposition(); - runCompositionPlayer(composition, pipeline); - - // TODO: b/449956936 - add EOS to CompositionPlayer packet consumer and wait until its received. + runCompositionPlayer(composition, recordingFrameProcessorFactory); - // Some decoders output an extra trailing decoded frame. Relax the check to verify that we - // received at least the expected N frames, at most N + 1 frames, and match the expected N - // timestamps. - int expectedFrameCount = expectedVideoTimestampsUs.size(); - assertThat(queuedPackets.size()).isAnyOf(expectedFrameCount, expectedFrameCount + 1); - for (int packetIndex = 0; packetIndex < expectedFrameCount; packetIndex++) { - long sequencePresentationTimeUs = - queuedPackets.get(packetIndex).get(0).sequencePresentationTimeUs; - assertThat(sequencePresentationTimeUs).isEqualTo(expectedVideoTimestampsUs.get(packetIndex)); - assertThat(queuedPackets.get(0)).hasSize(getNumVideoSequences(composition)); - for (int i = 0; i < queuedPackets.get(packetIndex).size(); ++i) { - HardwareBufferFrame.Metadata metadata = queuedPackets.get(packetIndex).get(i).getMetadata(); - assertThat(metadata).isInstanceOf(CompositionFrameMetadata.class); - CompositionFrameMetadata compositionFrameMetadata = (CompositionFrameMetadata) metadata; - int sequenceIndex = compositionFrameMetadata.sequenceIndex; - // CompositionPlayer replaces TimestampAdjustment effects with InactiveTimestampAdjustment. - // Assert on the non-edited MediaItem. - MediaItem itemFromMetadata = itemFromMetadata(compositionFrameMetadata); - MediaItem expectedItemAtTime = - expectedItemAtTime(composition, sequenceIndex, sequencePresentationTimeUs); - assertThat(itemFromMetadata).isEqualTo(expectedItemAtTime); - } + CapturingFrameProcessor frameProcessor = recordingFrameProcessorFactory.getCreatedProcessor(); + assertThat(frameProcessor).isNotNull(); + if (compositionAssetInfo.hasVideo()) { + assertThat(frameProcessor.isEnded()).isTrue(); + FrameProcessorTestUtil.assertPlaybackOutput(frameProcessor, compositionAssetInfo); + } else { + // TODO: b/534326875 - Remove this once audio only Compositions are handled. + assertThat(frameProcessor.isEnded()).isFalse(); + assertThat(frameProcessor.getQueuedEvents()).isEmpty(); } } @@ -256,20 +244,16 @@ private void runCompositionPlayer(Composition composition, VideoGraph.Factory vi endedFuture.get(); } - @RequiresApi(28) + @RequiresApi(AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) private void runCompositionPlayer( - Composition composition, - RenderingPacketConsumer, HardwareBufferFrameQueue> - hardwareBufferEffectsPipeline) - throws Exception { + Composition composition, FrameProcessor.Factory frameProcessorFactory) throws Exception { SettableFuture endedFuture = SettableFuture.create(); getInstrumentation() .runOnMainSync( () -> { player = new CompositionPlayer.Builder(context) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setHardwareBufferEffectsPipeline(hardwareBufferEffectsPipeline) + .setFrameProcessorFactory(frameProcessorFactory) .experimentalSetLateThresholdToDropInputUs(C.TIME_UNSET) .build(); // Set a surface on the player even though there is no UI on this test. We need a @@ -283,41 +267,6 @@ private void runCompositionPlayer( endedFuture.get(); } - private static MediaItem itemFromMetadata(CompositionFrameMetadata metadata) { - return metadata - .composition - .sequences - .get(metadata.sequenceIndex) - .editedMediaItems - .get(metadata.itemIndex) - .mediaItem; - } - - private static MediaItem expectedItemAtTime( - Composition composition, int sequenceIndex, long presentationTimeUs) { - EditedMediaItemSequence sequence = composition.sequences.get(sequenceIndex); - int itemIndex = 0; - while (itemIndex < sequence.editedMediaItems.size()) { - long itemDurationUs = sequence.editedMediaItems.get(itemIndex).getPresentationDurationUs(); - if (presentationTimeUs < itemDurationUs) { - break; - } - presentationTimeUs -= itemDurationUs; - itemIndex++; - } - return sequence.editedMediaItems.get(itemIndex).mediaItem; - } - - private static int getNumVideoSequences(Composition composition) { - int numVideoSequences = 0; - for (int i = 0; i < composition.sequences.size(); i++) { - if (composition.sequences.get(i).trackTypes.contains(TRACK_TYPE_VIDEO)) { - numVideoSequences++; - } - } - return numVideoSequences; - } - private static PlayerFence futureWhen(Player player) { return PlayerFence.futureWhen(player).withTimeoutMs(TEST_TIMEOUT_MS); } diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerSeekTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerSeekTest.java index 70a8d066189..105ee03260e 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerSeekTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerSeekTest.java @@ -23,6 +23,7 @@ import static androidx.media3.test.utils.AssetInfo.MP4_SIMPLE_ASSET; import static androidx.media3.test.utils.AssetInfo.PNG_ASSET; import static androidx.media3.test.utils.AssetInfo.RAW_AAC_ASSET; +import static androidx.media3.transformer.AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK; import static androidx.test.platform.app.InstrumentationRegistry.getInstrumentation; import static com.google.common.collect.Iterables.getLast; import static com.google.common.collect.Iterables.skip; @@ -47,20 +48,18 @@ import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.VideoGraph; import androidx.media3.common.util.ConditionVariable; -import androidx.media3.common.util.HandlerWrapper; import androidx.media3.common.util.NullableType; import androidx.media3.common.util.Util; +import androidx.media3.common.video.Frame; +import androidx.media3.common.video.FrameProcessor; import androidx.media3.effect.GlEffect; -import androidx.media3.effect.HardwareBufferFrame; import androidx.media3.effect.SingleInputVideoGraph; -import androidx.media3.effect.ndk.HardwareBufferJni; +import androidx.media3.test.utils.CapturingFrameProcessor; import androidx.media3.test.utils.PassthroughAudioProcessor; import androidx.media3.test.utils.PlayerFence; -import androidx.media3.test.utils.RecordingHardwareBufferEffectsPipeline; import androidx.test.ext.junit.rules.ActivityScenarioRule; import androidx.test.filters.SdkSuppress; import com.google.common.base.Ascii; -import com.google.common.base.Supplier; import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableSet; import com.google.common.collect.Iterables; @@ -110,6 +109,9 @@ public class CompositionPlayerSeekTest { public ActivityScenarioRule rule = new ActivityScenarioRule<>(SurfaceTestActivity.class); + @Rule + public final GlFrameProcessorTestRule glFrameProcessorTestRule = new GlFrameProcessorTestRule(); + @TestParameter boolean isScrubbingModeEnabled; private final Context applicationContext = @@ -1038,30 +1040,19 @@ protected void onFlush(StreamMetadata streamMetadata) { new Composition.Builder(EditedMediaItemSequence.withAudioFrom(ImmutableList.of(item))) .build(); + // Advance the player first to seek backwards. + SettableFuture passed500MsFuture = SettableFuture.create(); getInstrumentation() .runOnMainSync( () -> { player.set(new CompositionPlayer.Builder(applicationContext).build()); player.get().setComposition(composition); + passed500MsFuture.setFuture( + futureWhen(player.get()).passesContentPosition(/* targetPositionMs= */ 500)); player.get().prepare(); player.get().play(); }); - - HandlerWrapper handler = - player - .get() - .getClock() - .createHandler(player.get().getApplicationLooper(), /* callback= */ null); - - // Advance the player first to seek backwards. - assertWithMessage("Player position did not advance to 500ms.") - .that( - pollingWaitUntilCondition( - /* timeoutMs= */ 2_000, - /* pollIntervalMs= */ 100, - handler, - () -> player.get().getCurrentPosition() >= 500)) - .isTrue(); + passed500MsFuture.get(); SettableFuture readyFuture = SettableFuture.create(); // Seek backwards to avoid any seeking optimization (e.g. decode forward). @@ -1103,30 +1094,19 @@ protected void onFlush(StreamMetadata streamMetadata) { new Composition.Builder(EditedMediaItemSequence.withAudioFrom(ImmutableList.of(item))) .build(); + // Advance the player first to seek backwards. + SettableFuture passed500MsFuture = SettableFuture.create(); getInstrumentation() .runOnMainSync( () -> { player.set(new CompositionPlayer.Builder(applicationContext).build()); player.get().setComposition(composition); + passed500MsFuture.setFuture( + futureWhen(player.get()).passesContentPosition(/* targetPositionMs= */ 500)); player.get().prepare(); player.get().play(); }); - - HandlerWrapper handler = - player - .get() - .getClock() - .createHandler(player.get().getApplicationLooper(), /* callback= */ null); - - // Advance the player first to seek backwards. - assertWithMessage("Player position did not advance to 500ms.") - .that( - pollingWaitUntilCondition( - /* timeoutMs= */ 2_000, - /* pollIntervalMs= */ 100, - handler, - () -> player.get().getCurrentPosition() >= 500)) - .isTrue(); + passed500MsFuture.get(); SettableFuture readyFuture = SettableFuture.create(); // Seek backwards to avoid any seeking optimization (e.g. decode forward). @@ -1150,23 +1130,21 @@ protected void onFlush(StreamMetadata streamMetadata) { @Ignore("b/506959477 - Fix flakiness and re-enable") @Test - @SdkSuppress(minSdkVersion = 28) - public void packetConsumer_oneVideoSequence_seekForwardsAndBackwards_outputsCorrectFrames() + @SdkSuppress(minSdkVersion = HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) + public void frameProcessor_oneVideoSequence_seekForwardsAndBackwards_outputsCorrectFrames() throws Exception { SettableFuture firstFrameRenderedFuture = SettableFuture.create(); AtomicBoolean isPlaying = new AtomicBoolean(); - AtomicReference lastQueuedFrame = new AtomicReference<>(); - ConditionVariable packetQueued = new ConditionVariable(); - AtomicInteger queuedPackets = new AtomicInteger(); - RecordingHardwareBufferEffectsPipeline pipeline = - RecordingHardwareBufferEffectsPipeline.create( - applicationContext, - HardwareBufferJni.INSTANCE, - /* onQueue= */ frames -> { - lastQueuedFrame.set(frames.get(0)); - queuedPackets.incrementAndGet(); - packetQueued.open(); - return frames; + AtomicReference lastQueuedFrame = new AtomicReference<>(); + ConditionVariable frameQueued = new ConditionVariable(); + AtomicInteger queuedFramesCount = new AtomicInteger(); + FrameProcessor.Factory frameProcessorFactory = + new CapturingFrameProcessor.Factory( + glFrameProcessorTestRule.createDefaultGlFrameProcessorFactory(applicationContext), + /* onQueueListener= */ frames -> { + lastQueuedFrame.set(frames.get(0).frame); + queuedFramesCount.incrementAndGet(); + frameQueued.open(); }); Composition composition = @@ -1182,9 +1160,9 @@ public void packetConsumer_oneVideoSequence_seekForwardsAndBackwards_outputsCorr .runOnMainSync( () -> { player.set( - new CompositionPlayer.Builder(applicationContext) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setHardwareBufferEffectsPipeline(pipeline) + glFrameProcessorTestRule + .createCompositionPlayerBuilder(applicationContext) + .setFrameProcessorFactory(frameProcessorFactory) .build()); player.get().setVideoSurfaceView(surfaceView); firstFrameRenderedFuture.setFuture(futureWhen(player.get()).rendersFirstFrame()); @@ -1193,13 +1171,13 @@ public void packetConsumer_oneVideoSequence_seekForwardsAndBackwards_outputsCorr }); firstFrameRenderedFuture.get(); - assertThat(packetQueued.isOpen()).isTrue(); - assertThat(queuedPackets.get()).isEqualTo(1); - assertThat(lastQueuedFrame.get().presentationTimeUs).isEqualTo(0); + assertThat(frameQueued.isOpen()).isTrue(); + assertThat(queuedFramesCount.get()).isEqualTo(1); + assertThat(lastQueuedFrame.get().getContentTimeUs()).isEqualTo(0); getInstrumentation().runOnMainSync(() -> assertThat(player.get().getPlayWhenReady()).isFalse()); // Seek forwards - packetQueued.close(); + frameQueued.close(); getInstrumentation() .runOnMainSync( () -> { @@ -1208,14 +1186,14 @@ public void packetConsumer_oneVideoSequence_seekForwardsAndBackwards_outputsCorr player.get().setScrubbingModeEnabled(false); }); - assertThat(packetQueued.block(TEST_TIMEOUT_MS)).isTrue(); - assertThat(queuedPackets.get()).isEqualTo(2); - assertThat(lastQueuedFrame.get().presentationTimeUs).isEqualTo(500_500L); + assertThat(frameQueued.block(TEST_TIMEOUT_MS)).isTrue(); + assertThat(queuedFramesCount.get()).isEqualTo(2); + assertThat(lastQueuedFrame.get().getContentTimeUs()).isEqualTo(500_500L); getInstrumentation().runOnMainSync(() -> isPlaying.set(player.get().getPlayWhenReady())); assertThat(isPlaying.get()).isFalse(); // Seek backwards - packetQueued.close(); + frameQueued.close(); getInstrumentation() .runOnMainSync( () -> { @@ -1224,14 +1202,14 @@ public void packetConsumer_oneVideoSequence_seekForwardsAndBackwards_outputsCorr player.get().setScrubbingModeEnabled(false); }); - assertThat(packetQueued.block(TEST_TIMEOUT_MS)).isTrue(); - assertThat(queuedPackets.get()).isEqualTo(3); - assertThat(lastQueuedFrame.get().presentationTimeUs).isEqualTo(200_200L); + assertThat(frameQueued.block(TEST_TIMEOUT_MS)).isTrue(); + assertThat(queuedFramesCount.get()).isEqualTo(3); + assertThat(lastQueuedFrame.get().getContentTimeUs()).isEqualTo(200_200L); getInstrumentation().runOnMainSync(() -> isPlaying.set(player.get().getPlayWhenReady())); assertThat(isPlaying.get()).isFalse(); // Seek forwards - packetQueued.close(); + frameQueued.close(); getInstrumentation() .runOnMainSync( () -> { @@ -1240,32 +1218,30 @@ public void packetConsumer_oneVideoSequence_seekForwardsAndBackwards_outputsCorr player.get().setScrubbingModeEnabled(false); }); - assertThat(packetQueued.block(TEST_TIMEOUT_MS)).isTrue(); - assertThat(queuedPackets.get()).isEqualTo(4); - assertThat(lastQueuedFrame.get().presentationTimeUs).isEqualTo(767_433L); + assertThat(frameQueued.block(TEST_TIMEOUT_MS)).isTrue(); + assertThat(queuedFramesCount.get()).isEqualTo(4); + assertThat(lastQueuedFrame.get().getContentTimeUs()).isEqualTo(767_433L); getInstrumentation().runOnMainSync(() -> isPlaying.set(player.get().getPlayWhenReady())); assertThat(isPlaying.get()).isFalse(); } @Ignore("b/506959477 - Fix flakiness and re-enable") @Test - @SdkSuppress(minSdkVersion = 28) - public void packetConsumer_twoVideoSequences_seekForwardsAndBackwards_outputsCorrectFrames() + @SdkSuppress(minSdkVersion = HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) + public void frameProcessor_twoVideoSequences_seekForwardsAndBackwards_outputsCorrectFrames() throws Exception { SettableFuture firstFrameRenderedFuture = SettableFuture.create(); AtomicBoolean isPlaying = new AtomicBoolean(); - AtomicReference lastQueuedFrame = new AtomicReference<>(); - ConditionVariable packetQueued = new ConditionVariable(); - AtomicInteger queuedPackets = new AtomicInteger(); - RecordingHardwareBufferEffectsPipeline pipeline = - RecordingHardwareBufferEffectsPipeline.create( - applicationContext, - HardwareBufferJni.INSTANCE, - /* onQueue= */ frames -> { - lastQueuedFrame.set(frames.get(0)); - queuedPackets.incrementAndGet(); - packetQueued.open(); - return frames; + AtomicReference lastQueuedFrame = new AtomicReference<>(); + ConditionVariable frameQueued = new ConditionVariable(); + AtomicInteger queuedFramesCount = new AtomicInteger(); + FrameProcessor.Factory frameProcessorFactory = + new CapturingFrameProcessor.Factory( + glFrameProcessorTestRule.createDefaultGlFrameProcessorFactory(applicationContext), + /* onQueueListener= */ frames -> { + lastQueuedFrame.set(frames.get(0).frame); + queuedFramesCount.incrementAndGet(); + frameQueued.open(); }); Composition composition = @@ -1286,9 +1262,9 @@ public void packetConsumer_twoVideoSequences_seekForwardsAndBackwards_outputsCor .runOnMainSync( () -> { player.set( - new CompositionPlayer.Builder(applicationContext) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setHardwareBufferEffectsPipeline(pipeline) + glFrameProcessorTestRule + .createCompositionPlayerBuilder(applicationContext) + .setFrameProcessorFactory(frameProcessorFactory) .build()); player.get().setVideoSurfaceView(surfaceView); firstFrameRenderedFuture.setFuture(futureWhen(player.get()).rendersFirstFrame()); @@ -1297,13 +1273,13 @@ public void packetConsumer_twoVideoSequences_seekForwardsAndBackwards_outputsCor }); firstFrameRenderedFuture.get(); - assertThat(packetQueued.isOpen()).isTrue(); - assertThat(queuedPackets.get()).isEqualTo(1); - assertThat(lastQueuedFrame.get().presentationTimeUs).isEqualTo(0); + assertThat(frameQueued.isOpen()).isTrue(); + assertThat(queuedFramesCount.get()).isEqualTo(1); + assertThat(lastQueuedFrame.get().getContentTimeUs()).isEqualTo(0); getInstrumentation().runOnMainSync(() -> assertThat(player.get().getPlayWhenReady()).isFalse()); // Seek forwards - packetQueued.close(); + frameQueued.close(); getInstrumentation() .runOnMainSync( () -> { @@ -1312,14 +1288,14 @@ public void packetConsumer_twoVideoSequences_seekForwardsAndBackwards_outputsCor player.get().setScrubbingModeEnabled(false); }); - assertThat(packetQueued.block(TEST_TIMEOUT_MS)).isTrue(); - assertThat(queuedPackets.get()).isEqualTo(2); - assertThat(lastQueuedFrame.get().presentationTimeUs).isEqualTo(500_500L); + assertThat(frameQueued.block(TEST_TIMEOUT_MS)).isTrue(); + assertThat(queuedFramesCount.get()).isEqualTo(2); + assertThat(lastQueuedFrame.get().getContentTimeUs()).isEqualTo(500_500L); getInstrumentation().runOnMainSync(() -> isPlaying.set(player.get().getPlayWhenReady())); assertThat(isPlaying.get()).isFalse(); // Seek backwards - packetQueued.close(); + frameQueued.close(); getInstrumentation() .runOnMainSync( () -> { @@ -1328,14 +1304,14 @@ public void packetConsumer_twoVideoSequences_seekForwardsAndBackwards_outputsCor player.get().setScrubbingModeEnabled(false); }); - assertThat(packetQueued.block(TEST_TIMEOUT_MS)).isTrue(); - assertThat(queuedPackets.get()).isEqualTo(3); - assertThat(lastQueuedFrame.get().presentationTimeUs).isEqualTo(200_200L); + assertThat(frameQueued.block(TEST_TIMEOUT_MS)).isTrue(); + assertThat(queuedFramesCount.get()).isEqualTo(3); + assertThat(lastQueuedFrame.get().getContentTimeUs()).isEqualTo(200_200L); getInstrumentation().runOnMainSync(() -> isPlaying.set(player.get().getPlayWhenReady())); assertThat(isPlaying.get()).isFalse(); // Seek forwards - packetQueued.close(); + frameQueued.close(); getInstrumentation() .runOnMainSync( () -> { @@ -1344,31 +1320,29 @@ public void packetConsumer_twoVideoSequences_seekForwardsAndBackwards_outputsCor player.get().setScrubbingModeEnabled(false); }); - assertThat(packetQueued.block(TEST_TIMEOUT_MS)).isTrue(); - assertThat(queuedPackets.get()).isEqualTo(4); - assertThat(lastQueuedFrame.get().presentationTimeUs).isEqualTo(767_433L); + assertThat(frameQueued.block(TEST_TIMEOUT_MS)).isTrue(); + assertThat(queuedFramesCount.get()).isEqualTo(4); + assertThat(lastQueuedFrame.get().getContentTimeUs()).isEqualTo(767_433L); getInstrumentation().runOnMainSync(() -> isPlaying.set(player.get().getPlayWhenReady())); assertThat(isPlaying.get()).isFalse(); } @Ignore("b/506959477 - Fix flakiness and re-enable") @Test - @SdkSuppress(minSdkVersion = 28) - public void packetConsumer_oneVideoSequence_seekThenPlay_outputsPacketAndEnds() throws Exception { + @SdkSuppress(minSdkVersion = HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) + public void frameProcessor_oneVideoSequence_seekThenPlay_outputsPacketAndEnds() throws Exception { SettableFuture firstFrameRenderedFuture = SettableFuture.create(); AtomicBoolean isPlaying = new AtomicBoolean(); - AtomicReference lastQueuedFrame = new AtomicReference<>(); - ConditionVariable packetQueued = new ConditionVariable(); - AtomicInteger queuedPackets = new AtomicInteger(); - RecordingHardwareBufferEffectsPipeline pipeline = - RecordingHardwareBufferEffectsPipeline.create( - applicationContext, - HardwareBufferJni.INSTANCE, - /* onQueue= */ frames -> { - lastQueuedFrame.set(frames.get(0)); - queuedPackets.incrementAndGet(); - packetQueued.open(); - return frames; + AtomicReference lastQueuedFrame = new AtomicReference<>(); + ConditionVariable frameQueued = new ConditionVariable(); + AtomicInteger queuedFramesCount = new AtomicInteger(); + FrameProcessor.Factory frameProcessorFactory = + new CapturingFrameProcessor.Factory( + glFrameProcessorTestRule.createDefaultGlFrameProcessorFactory(applicationContext), + /* onQueueListener= */ frames -> { + lastQueuedFrame.set(frames.get(0).frame); + queuedFramesCount.incrementAndGet(); + frameQueued.open(); }); Composition composition = @@ -1384,9 +1358,9 @@ public void packetConsumer_oneVideoSequence_seekThenPlay_outputsPacketAndEnds() .runOnMainSync( () -> { player.set( - new CompositionPlayer.Builder(applicationContext) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setHardwareBufferEffectsPipeline(pipeline) + glFrameProcessorTestRule + .createCompositionPlayerBuilder(applicationContext) + .setFrameProcessorFactory(frameProcessorFactory) .build()); player.get().setVideoSurfaceView(surfaceView); firstFrameRenderedFuture.setFuture(futureWhen(player.get()).rendersFirstFrame()); @@ -1395,12 +1369,12 @@ public void packetConsumer_oneVideoSequence_seekThenPlay_outputsPacketAndEnds() }); firstFrameRenderedFuture.get(); - assertThat(packetQueued.isOpen()).isTrue(); - assertThat(queuedPackets.get()).isEqualTo(1); - assertThat(lastQueuedFrame.get().presentationTimeUs).isEqualTo(0); + assertThat(frameQueued.isOpen()).isTrue(); + assertThat(queuedFramesCount.get()).isEqualTo(1); + assertThat(lastQueuedFrame.get().getContentTimeUs()).isEqualTo(0); getInstrumentation().runOnMainSync(() -> assertThat(player.get().getPlayWhenReady()).isFalse()); - packetQueued.close(); + frameQueued.close(); getInstrumentation() .runOnMainSync( () -> { @@ -1409,9 +1383,9 @@ public void packetConsumer_oneVideoSequence_seekThenPlay_outputsPacketAndEnds() player.get().setScrubbingModeEnabled(false); }); - assertThat(packetQueued.block(TEST_TIMEOUT_MS)).isTrue(); - assertThat(queuedPackets.get()).isEqualTo(2); - assertThat(lastQueuedFrame.get().presentationTimeUs).isEqualTo(767_433L); + assertThat(frameQueued.block(TEST_TIMEOUT_MS)).isTrue(); + assertThat(queuedFramesCount.get()).isEqualTo(2); + assertThat(lastQueuedFrame.get().getContentTimeUs()).isEqualTo(767_433L); getInstrumentation().runOnMainSync(() -> isPlaying.set(player.get().getPlayWhenReady())); assertThat(isPlaying.get()).isFalse(); @@ -1429,23 +1403,21 @@ public void packetConsumer_oneVideoSequence_seekThenPlay_outputsPacketAndEnds() @Ignore("b/506959477 - Fix flakiness and re-enable") @Test - @SdkSuppress(minSdkVersion = 28) - public void packetConsumer_twoVideoSequences_seekThenPlay_outputsPacketAndEnds() + @SdkSuppress(minSdkVersion = HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) + public void frameProcessor_twoVideoSequences_seekThenPlay_outputsPacketAndEnds() throws Exception { SettableFuture firstFrameRenderedFuture = SettableFuture.create(); AtomicBoolean isPlaying = new AtomicBoolean(); - AtomicReference lastQueuedFrame = new AtomicReference<>(); - ConditionVariable packetQueued = new ConditionVariable(); - AtomicInteger queuedPackets = new AtomicInteger(); - RecordingHardwareBufferEffectsPipeline pipeline = - RecordingHardwareBufferEffectsPipeline.create( - applicationContext, - HardwareBufferJni.INSTANCE, - /* onQueue= */ frames -> { - lastQueuedFrame.set(frames.get(0)); - queuedPackets.incrementAndGet(); - packetQueued.open(); - return frames; + AtomicReference lastQueuedFrame = new AtomicReference<>(); + ConditionVariable frameQueued = new ConditionVariable(); + AtomicInteger queuedFramesCount = new AtomicInteger(); + FrameProcessor.Factory frameProcessorFactory = + new CapturingFrameProcessor.Factory( + glFrameProcessorTestRule.createDefaultGlFrameProcessorFactory(applicationContext), + /* onQueueListener= */ frames -> { + lastQueuedFrame.set(frames.get(0).frame); + queuedFramesCount.incrementAndGet(); + frameQueued.open(); }); Composition composition = @@ -1466,9 +1438,9 @@ public void packetConsumer_twoVideoSequences_seekThenPlay_outputsPacketAndEnds() .runOnMainSync( () -> { player.set( - new CompositionPlayer.Builder(applicationContext) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setHardwareBufferEffectsPipeline(pipeline) + glFrameProcessorTestRule + .createCompositionPlayerBuilder(applicationContext) + .setFrameProcessorFactory(frameProcessorFactory) .build()); player.get().setVideoSurfaceView(surfaceView); firstFrameRenderedFuture.setFuture(futureWhen(player.get()).rendersFirstFrame()); @@ -1477,12 +1449,12 @@ public void packetConsumer_twoVideoSequences_seekThenPlay_outputsPacketAndEnds() }); firstFrameRenderedFuture.get(); - assertThat(packetQueued.isOpen()).isTrue(); - assertThat(queuedPackets.get()).isEqualTo(1); - assertThat(lastQueuedFrame.get().presentationTimeUs).isEqualTo(0); + assertThat(frameQueued.isOpen()).isTrue(); + assertThat(queuedFramesCount.get()).isEqualTo(1); + assertThat(lastQueuedFrame.get().getContentTimeUs()).isEqualTo(0); getInstrumentation().runOnMainSync(() -> assertThat(player.get().getPlayWhenReady()).isFalse()); - packetQueued.close(); + frameQueued.close(); getInstrumentation() .runOnMainSync( () -> { @@ -1491,9 +1463,9 @@ public void packetConsumer_twoVideoSequences_seekThenPlay_outputsPacketAndEnds() player.get().setScrubbingModeEnabled(false); }); - assertThat(packetQueued.block(TEST_TIMEOUT_MS)).isTrue(); - assertThat(queuedPackets.get()).isEqualTo(2); - assertThat(lastQueuedFrame.get().presentationTimeUs).isEqualTo(767_433L); + assertThat(frameQueued.block(TEST_TIMEOUT_MS)).isTrue(); + assertThat(queuedFramesCount.get()).isEqualTo(2); + assertThat(lastQueuedFrame.get().getContentTimeUs()).isEqualTo(767_433L); getInstrumentation().runOnMainSync(() -> isPlaying.set(player.get().getPlayWhenReady())); assertThat(isPlaying.get()).isFalse(); @@ -1509,27 +1481,6 @@ public void packetConsumer_twoVideoSequences_seekThenPlay_outputsPacketAndEnds() endedFuture.get(); } - private static boolean pollingWaitUntilCondition( - long timeoutMs, long pollIntervalMs, HandlerWrapper handler, Supplier predicate) - throws InterruptedException { - ConditionVariable isDone = new ConditionVariable(); - handler.postDelayed(() -> evaluate(isDone, pollIntervalMs, handler, predicate), pollIntervalMs); - return isDone.block(timeoutMs); - } - - private static void evaluate( - ConditionVariable isDone, - long pollIntervalMs, - HandlerWrapper handler, - Supplier predicate) { - if (predicate.get()) { - isDone.open(); - } else { - handler.postDelayed( - () -> evaluate(isDone, pollIntervalMs, handler, predicate), pollIntervalMs); - } - } - /** * Plays the first {@code numberOfFramesBeforeSeeking} frames of the provided sequence, seeks to * {@code seekTimeMs}, resumes playback until it ends, and returns the timestamps of the processed diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerSurfaceViewDisplayTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerSurfaceViewDisplayTest.java new file mode 100644 index 00000000000..54367f51181 --- /dev/null +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerSurfaceViewDisplayTest.java @@ -0,0 +1,185 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static androidx.media3.common.util.Util.isRunningOnEmulator; +import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_5S; +import static androidx.media3.test.utils.BitmapPixelTestUtil.maybeSaveTestBitmap; +import static androidx.media3.test.utils.PlayerFence.futureWhen; +import static java.util.concurrent.TimeUnit.MILLISECONDS; +import static org.junit.Assume.assumeTrue; + +import android.app.Instrumentation; +import android.content.Context; +import android.graphics.Bitmap; +import android.graphics.Rect; +import android.view.SurfaceView; +import androidx.media3.common.MediaItem; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.rules.ActivityScenarioRule; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import androidx.test.filters.SdkSuppress; +import androidx.test.platform.app.InstrumentationRegistry; +import com.google.common.collect.ImmutableList; +import com.google.common.util.concurrent.SettableFuture; +import java.util.concurrent.TimeoutException; +import java.util.concurrent.atomic.AtomicBoolean; +import org.checkerframework.checker.nullness.qual.MonotonicNonNull; +import org.junit.After; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TestName; +import org.junit.runner.RunWith; + +/** + * Instrumentation tests verifying that {@link CompositionPlayer} frames actually reach the display + * when output to a {@link SurfaceView}. + * + *

    These tests deliberately avoid {@link android.view.PixelCopy}. {@code PixelCopy} on a {@link + * SurfaceView} reads the last buffer queued by the producer, so it still succeeds when + * {@code BLASTBufferQueue} rejects every buffer (for example because the {@link + * android.view.SurfaceHolder} was never resized to the video size) and the user only sees a black + * screen. Screen content is instead captured through {@link + * android.app.UiAutomation#takeScreenshot()}, which goes through real SurfaceFlinger composition. + */ +@RunWith(AndroidJUnit4.class) +@SdkSuppress(minSdkVersion = AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) +public final class CompositionPlayerSurfaceViewDisplayTest { + + private static final long TEST_TIMEOUT_MS = isRunningOnEmulator() ? 30_000 : 10_000; + private static final long DISPLAY_TIMEOUT_MS = isRunningOnEmulator() ? 20_000 : 10_000; + private static final long DISPLAY_POLL_INTERVAL_MS = 100; + + @Rule + public final ActivityScenarioRule activityRule = + new ActivityScenarioRule<>(SurfaceTestActivity.class); + + @Rule public final TestName testName = new TestName(); + + @Rule + public final GlFrameProcessorTestRule glFrameProcessorTestRule = new GlFrameProcessorTestRule(); + + private final Instrumentation instrumentation = InstrumentationRegistry.getInstrumentation(); + private final Context context = ApplicationProvider.getApplicationContext(); + + private @MonotonicNonNull CompositionPlayer compositionPlayer; + private @MonotonicNonNull SurfaceView surfaceView; + private String testId; + + @Before + public void setUp() { + testId = testName.getMethodName(); + activityRule.getScenario().onActivity(activity -> surfaceView = activity.getSurfaceView()); + } + + @After + public void tearDown() { + instrumentation.runOnMainSync( + () -> { + if (compositionPlayer != null) { + compositionPlayer.release(); + } + }); + activityRule.getScenario().close(); + } + + @Test + public void prepare_withFrameProcessorAndSurfaceView_displaysFrameOnScreen() throws Exception { + SettableFuture firstFrameRenderedFuture = SettableFuture.create(); + instrumentation.runOnMainSync( + () -> { + compositionPlayer = + glFrameProcessorTestRule.createCompositionPlayerBuilder(context).build(); + compositionPlayer.setVideoSurfaceView(surfaceView); + firstFrameRenderedFuture.setFuture(futureWhen(compositionPlayer).rendersFirstFrame()); + compositionPlayer.setComposition(createSingleItemComposition()); + compositionPlayer.prepare(); + }); + try { + firstFrameRenderedFuture.get(TEST_TIMEOUT_MS, MILLISECONDS); + } catch (TimeoutException e) { + assumeSurfaceViewHasWindowFocus(); + throw e; + } + + assertSurfaceViewDisplaysContent(); + } + + private static Composition createSingleItemComposition() { + return new Composition.Builder( + EditedMediaItemSequence.withVideoFrom( + ImmutableList.of( + new EditedMediaItem.Builder( + MediaItem.fromUri( + MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_5S.uri)) + .setDurationUs( + MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_5S.videoDurationUs) + .build()))) + .build(); + } + + private void assertSurfaceViewDisplaysContent() throws InterruptedException { + long timeoutMs = DISPLAY_TIMEOUT_MS; + Bitmap lastCapture = null; + while (timeoutMs > 0) { + lastCapture = captureSurfaceViewFromScreen(); + int bottomY = lastCapture.getHeight() - 1; + // Check that the SurfaceView contents changed from a uniform background. + if (lastCapture.getPixel(0, bottomY) + != lastCapture.getPixel(lastCapture.getWidth() / 2, bottomY)) { + return; + } + Thread.sleep(DISPLAY_POLL_INTERVAL_MS); + timeoutMs -= DISPLAY_POLL_INTERVAL_MS; + } + assumeSurfaceViewHasWindowFocus(); + maybeSaveTestBitmap(testId, "failure_lastCapture", lastCapture, /* path= */ null); + throw new AssertionError( + "Timed out after " + DISPLAY_TIMEOUT_MS + " ms waiting for SurfaceView content."); + } + + private void assumeSurfaceViewHasWindowFocus() { + AtomicBoolean hasWindowFocus = new AtomicBoolean(); + instrumentation.runOnMainSync(() -> hasWindowFocus.set(surfaceView.hasWindowFocus())); + assumeTrue("SurfaceView lost window focus", hasWindowFocus.get()); + } + + private Bitmap captureSurfaceViewFromScreen() { + Bitmap screenshot = instrumentation.getUiAutomation().takeScreenshot(); + Rect bounds = getSurfaceViewScreenBounds(screenshot.getWidth(), screenshot.getHeight()); + return Bitmap.createBitmap( + screenshot, bounds.left, bounds.top, bounds.width(), bounds.height()); + } + + private Rect getSurfaceViewScreenBounds(int screenshotWidth, int screenshotHeight) { + Rect bounds = new Rect(); + instrumentation.runOnMainSync( + () -> { + int[] locationOnScreen = new int[2]; + surfaceView.getLocationOnScreen(locationOnScreen); + bounds.set( + locationOnScreen[0], + locationOnScreen[1], + locationOnScreen[0] + surfaceView.getWidth(), + locationOnScreen[1] + surfaceView.getHeight()); + }); + if (!bounds.intersect(0, 0, screenshotWidth, screenshotHeight)) { + bounds.setEmpty(); + } + return bounds; + } +} diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerTest.java index fc353a92b31..be78b6cdfd4 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/CompositionPlayerTest.java @@ -21,6 +21,7 @@ import static androidx.media3.test.utils.AssetInfo.MP4_ADVANCED_ASSET; import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_5S; import static androidx.media3.test.utils.PlayerFence.futureWhen; +import static androidx.media3.transformer.AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK; import static com.google.common.base.Preconditions.checkState; import static com.google.common.truth.Truth.assertThat; import static com.google.common.util.concurrent.Futures.immediateFailedFuture; @@ -66,13 +67,11 @@ import androidx.media3.datasource.AssetDataSource; import androidx.media3.datasource.DataSourceUtil; import androidx.media3.datasource.DataSpec; -import androidx.media3.effect.DefaultHardwareBufferEffectsPipeline; import androidx.media3.effect.DefaultVideoFrameProcessor; import androidx.media3.effect.GlEffect; import androidx.media3.effect.GlShaderProgram; import androidx.media3.effect.PassthroughShaderProgram; import androidx.media3.effect.SingleInputVideoGraph; -import androidx.media3.effect.ndk.HardwareBufferJni; import androidx.media3.exoplayer.analytics.AnalyticsListener; import androidx.media3.exoplayer.audio.DefaultAudioSink; import androidx.media3.exoplayer.audio.ForwardingAudioSink; @@ -118,6 +117,9 @@ public class CompositionPlayerTest { public ActivityScenarioRule rule = new ActivityScenarioRule<>(SurfaceTestActivity.class); + @Rule + public final GlFrameProcessorTestRule glFrameProcessorTestRule = new GlFrameProcessorTestRule(); + private final Instrumentation instrumentation = InstrumentationRegistry.getInstrumentation(); private final Context applicationContext = instrumentation.getContext().getApplicationContext(); @@ -862,20 +864,16 @@ public void onPlayerReleased(EventTime eventTime) { } @Test - @SdkSuppress(minSdkVersion = 28) - public void compositionPlayer_withPacketConsumer_outputsFrameBeforeEnding() throws Exception { + @SdkSuppress(minSdkVersion = HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) + public void compositionPlayer_withFrameProcessor_outputsFrameBeforeEnding() throws Exception { SettableFuture endedFuture = SettableFuture.create(); Queue videoTimestamps = new ConcurrentLinkedQueue<>(); instrumentation.runOnMainSync( () -> { - DefaultHardwareBufferEffectsPipeline packetProcessor = - DefaultHardwareBufferEffectsPipeline.create( - applicationContext, HardwareBufferJni.INSTANCE); compositionPlayer = - new CompositionPlayer.Builder(applicationContext) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setHardwareBufferEffectsPipeline(packetProcessor) + glFrameProcessorTestRule + .createCompositionPlayerBuilder(applicationContext) .experimentalSetLateThresholdToDropInputUs(C.TIME_UNSET) .build(); compositionPlayer.setVideoSurfaceView(surfaceView); @@ -921,19 +919,16 @@ public void onPlaybackStateChanged(int playbackState) { } @Test - @SdkSuppress(minSdkVersion = 28) - public void compositionPlayer_playAfterEnded_doesNotTimeout() throws Exception { + @SdkSuppress(minSdkVersion = HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) + public void compositionPlayer_withFrameProcessor_playAfterEnded_doesNotTimeout() + throws Exception { SettableFuture endedFuture = SettableFuture.create(); instrumentation.runOnMainSync( () -> { - DefaultHardwareBufferEffectsPipeline packetProcessor = - DefaultHardwareBufferEffectsPipeline.create( - applicationContext, HardwareBufferJni.INSTANCE); compositionPlayer = - new CompositionPlayer.Builder(applicationContext) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setHardwareBufferEffectsPipeline(packetProcessor) + glFrameProcessorTestRule + .createCompositionPlayerBuilder(applicationContext) .experimentalSetLateThresholdToDropInputUs(C.TIME_UNSET) .build(); compositionPlayer.setVideoSurfaceView(surfaceView); diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/EncoderFrameWriterAndroidTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/EncoderFrameWriterAndroidTest.java index 36bb9d7f5c2..dbcd017306f 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/EncoderFrameWriterAndroidTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/EncoderFrameWriterAndroidTest.java @@ -127,21 +127,6 @@ public void tearDown() { } } - @Test - public void configure_withImpossibleFormat_invokesErrorListener() { - Format impossibleFormat = - new Format.Builder() - .setWidth(1) - .setHeight(1) - .setSampleMimeType(MimeTypes.VIDEO_H264) - .build(); - - encoderFrameWriter.configure(impossibleFormat, Frame.USAGE_VIDEO_ENCODE); - - assertThat(createdEncoder.get()).isNull(); - assertThat(errorException.get()).isNotNull(); - } - @Test public void dequeueInputFrame_beforeConfigure_throwIllegalStateException() { Executor wakeupExecutor = directExecutor(); diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/ExportTestResult.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/ExportTestResult.java index c83a65a610e..ca261b7de51 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/ExportTestResult.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/ExportTestResult.java @@ -24,12 +24,12 @@ import org.json.JSONObject; /** A test only class for holding the details of a test export. */ -public class ExportTestResult { +public final class ExportTestResult { /** Represents an unset or unknown SSIM score. */ public static final double SSIM_UNSET = -1.0d; /** A builder for {@link ExportTestResult}. */ - public static class Builder { + public static final class Builder { private final ExportResult exportResult; @Nullable private String filePath; diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/GlEncoderFrameWriterAndroidTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/GlEncoderFrameWriterAndroidTest.java index db77e390312..f9c8310d322 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/GlEncoderFrameWriterAndroidTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/GlEncoderFrameWriterAndroidTest.java @@ -29,7 +29,7 @@ import androidx.media3.common.video.Frame; import androidx.media3.common.video.HardwareBufferFrame; import androidx.media3.effect.DefaultGlObjectsProvider; -import androidx.media3.effect.ndk.HardwareBufferJni; +import androidx.media3.effect.HardwareBufferJni; import androidx.media3.transformer.AndroidTestUtil.ForceEncodeEncoderFactory; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/GlFrameProcessorTestRule.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/GlFrameProcessorTestRule.java new file mode 100644 index 00000000000..08a309104a1 --- /dev/null +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/GlFrameProcessorTestRule.java @@ -0,0 +1,58 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import android.content.Context; +import androidx.annotation.RequiresApi; +import androidx.media3.effect.DefaultGlFrameProcessor; +import androidx.media3.effect.HardwareBufferJni; +import org.junit.rules.ExternalResource; + +/** + * A JUnit rule that creates components configured with {@link DefaultGlFrameProcessor.Factory} for + * testing. + * + *

    Each created {@link DefaultGlFrameProcessor} creates and releases its own GL resources. + */ +public final class GlFrameProcessorTestRule extends ExternalResource { + + /** Returns a {@link DefaultGlFrameProcessor.Factory}. */ + @RequiresApi(26) + public DefaultGlFrameProcessor.Factory createDefaultGlFrameProcessorFactory(Context context) { + return new DefaultGlFrameProcessor.Factory.Builder(context, HardwareBufferJni.INSTANCE).build(); + } + + /** + * Returns a {@link Transformer.Builder} configured with {@link DefaultGlFrameProcessor.Factory}. + */ + @RequiresApi(AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) + public Transformer.Builder createTransformerBuilder(Context context) { + return new Transformer.Builder(context) + .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) + .setFrameProcessorFactory(createDefaultGlFrameProcessorFactory(context)); + } + + /** + * Returns a {@link CompositionPlayer.Builder} configured with {@link + * DefaultGlFrameProcessor.Factory}. + */ + @RequiresApi(AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) + public CompositionPlayer.Builder createCompositionPlayerBuilder(Context context) { + return new CompositionPlayer.Builder(context) + .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) + .setFrameProcessorFactory(createDefaultGlFrameProcessorFactory(context)); + } +} diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/HardwareBufferFrameReaderAndroidTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/HardwareBufferFrameReaderAndroidTest.java index e0b85047718..c22bbacca90 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/HardwareBufferFrameReaderAndroidTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/HardwareBufferFrameReaderAndroidTest.java @@ -17,6 +17,8 @@ import static androidx.media3.test.utils.AssetInfo.MP4_ADVANCED_ASSET; import static androidx.media3.transformer.EditedMediaItemSequence.withAudioFrom; +import static androidx.media3.transformer.TransformerUtil.END_OF_STREAM_ASYNC_FRAME; +import static androidx.media3.transformer.TransformerUtil.releaseIfNeeded; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.base.Preconditions.checkState; import static com.google.common.truth.Truth.assertThat; @@ -28,6 +30,7 @@ import android.hardware.HardwareBuffer; import android.media.Image; import android.media.ImageWriter; +import android.os.Handler; import android.os.HandlerThread; import androidx.media3.common.C; import androidx.media3.common.ColorInfo; @@ -36,12 +39,18 @@ import androidx.media3.common.MimeTypes; import androidx.media3.common.util.ConstantRateTimestampIterator; import androidx.media3.common.util.SystemClock; +import androidx.media3.common.util.TimestampIterator; import androidx.media3.common.util.Util; -import androidx.media3.effect.HardwareBufferFrame; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.DefaultHardwareBufferFrame; +import androidx.media3.common.video.Frame; +import androidx.media3.common.video.HardwareBufferFrame; +import androidx.media3.effect.HardwareBufferJni; import androidx.test.ext.junit.runners.AndroidJUnit4; import androidx.test.filters.SdkSuppress; import com.google.common.collect.ImmutableList; import java.util.concurrent.BlockingQueue; +import java.util.concurrent.CountDownLatch; import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.atomic.AtomicBoolean; import java.util.concurrent.atomic.AtomicReference; @@ -55,9 +64,10 @@ * frames into a Surface seems unsupported on robolectric. */ @RunWith(AndroidJUnit4.class) +@SdkSuppress(minSdkVersion = 28) public class HardwareBufferFrameReaderAndroidTest { - private static final long TEST_TIMEOUT_MS = 100; + private static final long TEST_TIMEOUT_MS = 10_000; private static final Format TEST_FORMAT = new Format.Builder() .setWidth(10) @@ -66,16 +76,19 @@ public class HardwareBufferFrameReaderAndroidTest { .build(); private Composition composition; - private BlockingQueue receivedFrames; + private BlockingQueue receivedFrames; private HardwareBufferFrameReader hardwareBufferFrameReader; private HandlerThread handlerThread; private AtomicReference hardwareBufferFrameReaderException; @Before public void setUp() { - EditedMediaItem editedMediaItem = + EditedMediaItem editedMediaItem1 = new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)).build(); - EditedMediaItemSequence sequence = withAudioFrom(ImmutableList.of(editedMediaItem)); + EditedMediaItem editedMediaItem2 = + new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)).build(); + EditedMediaItemSequence sequence = + withAudioFrom(ImmutableList.of(editedMediaItem1, editedMediaItem2)); composition = new Composition.Builder(sequence).build(); handlerThread = new HandlerThread("HardwareBufferFrameReaderTest"); handlerThread.start(); @@ -91,7 +104,8 @@ public void setUp() { /* defaultSurfacePixelFormat= */ ImageFormat.YUV_420_888, new DefaultImageReaderAdapter.Factory(), e -> hardwareBufferFrameReaderException.set(e), - SystemClock.DEFAULT.createHandler(Util.getCurrentOrMainLooper(), null)); + SystemClock.DEFAULT.createHandler(Util.getCurrentOrMainLooper(), null), + /* hardwareBufferJniWrapper= */ HardwareBufferJni.INSTANCE); } @After @@ -101,28 +115,27 @@ public void tearDown() { } @Test + @SuppressWarnings("deprecation") // Uses deprecated CompositionFrameMetadata. public void frameReader_queueFrameViaSurface_receivesFrame() throws Exception { hardwareBufferFrameReader.queueFrameViaSurface( /* presentationTimeUs= */ 0, /* sequenceOffsetUs= */ 0, /* indexOfItem= */ 0, TEST_FORMAT); produceFrameToFrameReaderSurface(/* presentationTimeUs= */ 0); - HardwareBufferFrame receivedFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + AsyncFrame receivedFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); assertThat(hardwareBufferFrameReaderException.get()).isNull(); - assertThat(receivedFrame.presentationTimeUs).isEqualTo(0); - assertThat(receivedFrame.sequencePresentationTimeUs).isEqualTo(0); - assertThat(receivedFrame.format).isEqualTo(TEST_FORMAT); - assertThat(receivedFrame.internalFrame).isNotNull(); - assertThat(receivedFrame.getMetadata()).isInstanceOf(CompositionFrameMetadata.class); - CompositionFrameMetadata compositionFrameMetadata = - (CompositionFrameMetadata) receivedFrame.getMetadata(); + assertThat(receivedFrame).isNotNull(); + assertThat(getPresentationTimeUs(receivedFrame)).isEqualTo(0); + assertThat(receivedFrame.frame.getContentTimeUs()).isEqualTo(0); + assertThat(receivedFrame.frame.getFormat()).isEqualTo(TEST_FORMAT); + assertThat(getInternalImage(receivedFrame)).isNotNull(); + CompositionFrameMetadata compositionFrameMetadata = getCompositionFrameMetadata(receivedFrame); assertThat(compositionFrameMetadata.composition).isEqualTo(composition); assertThat(compositionFrameMetadata.sequenceIndex).isEqualTo(0); assertThat(compositionFrameMetadata.itemIndex).isEqualTo(0); } @Test - @SdkSuppress(minSdkVersion = 28) public void frameReader_releaseSurfaceFrame_closesTheHardwareBuffer() throws Exception { hardwareBufferFrameReader.queueFrameViaSurface( /* presentationTimeUs= */ 1234, @@ -130,11 +143,13 @@ public void frameReader_releaseSurfaceFrame_closesTheHardwareBuffer() throws Exc /* indexOfItem= */ 0, TEST_FORMAT); produceFrameToFrameReaderSurface(/* presentationTimeUs= */ 1234); - HardwareBufferFrame receivedFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); - HardwareBuffer hardwareBuffer = checkNotNull(receivedFrame.hardwareBuffer); - receivedFrame.release(/* releaseFence= */ null); - handlerThread.join(TEST_TIMEOUT_MS); + AsyncFrame receivedFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + assertThat(receivedFrame).isNotNull(); + HardwareBuffer hardwareBuffer = checkNotNull(getHardwareBuffer(receivedFrame)); + + releaseIfNeeded(receivedFrame.frame, /* releaseFence= */ null); + flushHandlerThread(); assertThat(hardwareBuffer.isClosed()).isTrue(); assertThat(hardwareBufferFrameReaderException.get()).isNull(); @@ -149,11 +164,13 @@ public void frameReader_releaseBitmapFrame_doesNotCloseTheHardwareBuffer() throw new ConstantRateTimestampIterator(/* durationUs= */ 1_000_000, /* frameRate= */ 30f), /* sequenceOffsetUs= */ 0, /* indexOfItem= */ 1); - HardwareBufferFrame receivedFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); - HardwareBuffer hardwareBuffer = checkNotNull(receivedFrame.hardwareBuffer); - receivedFrame.release(/* releaseFence= */ null); - handlerThread.join(TEST_TIMEOUT_MS); + AsyncFrame receivedFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + assertThat(receivedFrame).isNotNull(); + HardwareBuffer hardwareBuffer = checkNotNull(getHardwareBuffer(receivedFrame)); + + releaseIfNeeded(receivedFrame.frame, /* releaseFence= */ null); + flushHandlerThread(); // Closing the HardwareBuffer is handled by garbage collection. assertThat(hardwareBuffer.isClosed()).isFalse(); @@ -169,10 +186,12 @@ public void frameReader_releaseOutputFrame_freesUpCapacity() throws Exception { TEST_FORMAT); checkState(!hardwareBufferFrameReader.canAcceptFrameViaSurface()); produceFrameToFrameReaderSurface(/* presentationTimeUs= */ 1234); - HardwareBufferFrame receivedFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); - receivedFrame.release(/* releaseFence= */ null); - handlerThread.join(TEST_TIMEOUT_MS); + AsyncFrame receivedFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + assertThat(receivedFrame).isNotNull(); + + releaseIfNeeded(receivedFrame.frame, /* releaseFence= */ null); + flushHandlerThread(); assertThat(hardwareBufferFrameReader.canAcceptFrameViaSurface()).isTrue(); assertThat(hardwareBufferFrameReaderException.get()).isNull(); @@ -189,15 +208,18 @@ public void frameReader_releaseOutputFrame_callsWakeupListener() throws Exceptio hardwareBufferFrameReader.addRendererWakeupListener(() -> onWakeupCalled.set(true)); assertThat(hardwareBufferFrameReader.canAcceptFrameViaSurface()).isFalse(); produceFrameToFrameReaderSurface(/* presentationTimeUs= */ 1234); - HardwareBufferFrame receivedFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); - receivedFrame.release(/* releaseFence= */ null); - handlerThread.join(TEST_TIMEOUT_MS); + AsyncFrame receivedFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + assertThat(receivedFrame).isNotNull(); + + releaseIfNeeded(receivedFrame.frame, /* releaseFence= */ null); + flushHandlerThread(); assertThat(onWakeupCalled.get()).isTrue(); } @Test + @SuppressWarnings("deprecation") // Uses deprecated CompositionFrameMetadata. public void produceSurfaceFrame_withPendingBitmap_outputsBitmap() throws Exception { // SRGB ColorTransfer is replaced with SDR. ColorInfo expectedColorInfo = @@ -226,15 +248,14 @@ public void produceSurfaceFrame_withPendingBitmap_outputsBitmap() throws Excepti /* indexOfItem= */ 1); produceFrameToFrameReaderSurface(/* presentationTimeUs= */ 1234); - HardwareBufferFrame firstFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); - HardwareBufferFrame secondFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + AsyncFrame firstFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + AsyncFrame secondFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); assertThat(firstFrame).isNotNull(); - assertThat(secondFrame.internalFrame).isInstanceOf(Bitmap.class); - assertThat(secondFrame.getMetadata()).isInstanceOf(CompositionFrameMetadata.class); - assertThat(secondFrame.format).isEqualTo(expectedBitmapFormat); - CompositionFrameMetadata bitmapFrameMetadata = - (CompositionFrameMetadata) secondFrame.getMetadata(); + assertThat(secondFrame).isNotNull(); + assertThat(getInternalImage(secondFrame)).isInstanceOf(Bitmap.class); + assertThat(secondFrame.frame.getFormat()).isEqualTo(expectedBitmapFormat); + CompositionFrameMetadata bitmapFrameMetadata = getCompositionFrameMetadata(secondFrame); assertThat(bitmapFrameMetadata.composition).isEqualTo(composition); assertThat(bitmapFrameMetadata.sequenceIndex).isEqualTo(0); assertThat(bitmapFrameMetadata.itemIndex).isEqualTo(1); @@ -254,16 +275,17 @@ public void frameReader_queueFrameViaSurfaceThenQueueEndOfStream_receivesFrameTh produceFrameToFrameReaderSurface(frameTimeUs); - HardwareBufferFrame firstFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); - HardwareBufferFrame secondFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); - HardwareBufferFrame shouldBeNull = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + AsyncFrame firstFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + AsyncFrame secondFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + AsyncFrame shouldBeNull = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + flushHandlerThread(); assertThat(hardwareBufferFrameReaderException.get()).isNull(); assertThat(firstFrame).isNotNull(); - assertThat(firstFrame.presentationTimeUs).isEqualTo(frameTimeUs); - assertThat(firstFrame.sequencePresentationTimeUs).isEqualTo(frameTimeUs); - assertThat(firstFrame.format).isEqualTo(TEST_FORMAT); - assertThat(secondFrame).isEqualTo(HardwareBufferFrame.END_OF_STREAM_FRAME); + assertThat(getPresentationTimeUs(firstFrame)).isEqualTo(frameTimeUs); + assertThat(firstFrame.frame.getContentTimeUs()).isEqualTo(frameTimeUs); + assertThat(firstFrame.frame.getFormat()).isEqualTo(TEST_FORMAT); + assertThat(secondFrame).isEqualTo(END_OF_STREAM_ASYNC_FRAME); assertThat(shouldBeNull).isNull(); } @@ -278,21 +300,23 @@ public void frameReader_queueEndOfStreamThenQueueFrameViaSurface_receivesEosThen /* indexOfItem= */ 0, TEST_FORMAT); - HardwareBufferFrame receivedEos = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + AsyncFrame receivedEos = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); assertThat(hardwareBufferFrameReaderException.get()).isNull(); - assertThat(receivedEos).isEqualTo(HardwareBufferFrame.END_OF_STREAM_FRAME); + assertThat(receivedEos).isEqualTo(END_OF_STREAM_ASYNC_FRAME); produceFrameToFrameReaderSurface(frameTimeUs); - HardwareBufferFrame firstFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); - HardwareBufferFrame shouldBeNull = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + AsyncFrame firstFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + AsyncFrame shouldBeNull = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + flushHandlerThread(); assertThat(hardwareBufferFrameReaderException.get()).isNull(); assertThat(firstFrame).isNotNull(); - assertThat(firstFrame.presentationTimeUs).isEqualTo(frameTimeUs); - assertThat(firstFrame.sequencePresentationTimeUs).isEqualTo(frameTimeUs); - assertThat(firstFrame.format).isEqualTo(TEST_FORMAT); + + assertThat(getPresentationTimeUs(firstFrame)).isEqualTo(frameTimeUs); + assertThat(firstFrame.frame.getContentTimeUs()).isEqualTo(frameTimeUs); + assertThat(firstFrame.frame.getFormat()).isEqualTo(TEST_FORMAT); assertThat(shouldBeNull).isNull(); } @@ -326,32 +350,33 @@ public void frameReader_queueMultipleSurfaceFramesAndEos_receivesInOrder() throw format3); produceFrameToFrameReaderSurface(frameTimeUs1); - HardwareBufferFrame recFrame1 = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + AsyncFrame recFrame1 = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); assertThat(recFrame1).isNotNull(); - assertThat(recFrame1.presentationTimeUs).isEqualTo(frameTimeUs1); - assertThat(recFrame1.sequencePresentationTimeUs).isEqualTo(frameTimeUs1); - assertThat(recFrame1.format).isEqualTo(format1); - recFrame1.release(/* releaseFence= */ null); + assertThat(getPresentationTimeUs(recFrame1)).isEqualTo(frameTimeUs1); + assertThat(recFrame1.frame.getContentTimeUs()).isEqualTo(frameTimeUs1); + assertThat(recFrame1.frame.getFormat()).isEqualTo(format1); + releaseIfNeeded(recFrame1.frame, /* releaseFence= */ null); produceFrameToFrameReaderSurface(frameTimeUs2); - HardwareBufferFrame recFrame2 = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + AsyncFrame recFrame2 = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); assertThat(recFrame2).isNotNull(); - assertThat(recFrame2.presentationTimeUs).isEqualTo(frameTimeUs2); - assertThat(recFrame2.sequencePresentationTimeUs).isEqualTo(frameTimeUs2); - assertThat(recFrame2.format).isEqualTo(format2); - recFrame2.release(/* releaseFence= */ null); + assertThat(getPresentationTimeUs(recFrame2)).isEqualTo(frameTimeUs2); + assertThat(recFrame2.frame.getContentTimeUs()).isEqualTo(frameTimeUs2); + assertThat(recFrame2.frame.getFormat()).isEqualTo(format2); + releaseIfNeeded(recFrame2.frame, /* releaseFence= */ null); - HardwareBufferFrame recEos = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); - assertThat(recEos).isEqualTo(HardwareBufferFrame.END_OF_STREAM_FRAME); + AsyncFrame recEos = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + assertThat(recEos).isEqualTo(END_OF_STREAM_ASYNC_FRAME); produceFrameToFrameReaderSurface(frameTimeUs3); - HardwareBufferFrame recFrame3 = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + AsyncFrame recFrame3 = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); assertThat(recFrame3).isNotNull(); - assertThat(recFrame3.presentationTimeUs).isEqualTo(frameTimeUs3); - assertThat(recFrame3.sequencePresentationTimeUs).isEqualTo(frameTimeUs3); - assertThat(recFrame3.format).isEqualTo(format3); - recFrame3.release(/* releaseFence= */ null); + assertThat(getPresentationTimeUs(recFrame3)).isEqualTo(frameTimeUs3); + assertThat(recFrame3.frame.getContentTimeUs()).isEqualTo(frameTimeUs3); + assertThat(recFrame3.frame.getFormat()).isEqualTo(format3); + releaseIfNeeded(recFrame3.frame, /* releaseFence= */ null); + flushHandlerThread(); assertThat(receivedFrames).isEmpty(); assertThat(hardwareBufferFrameReaderException.get()).isNull(); } @@ -370,8 +395,9 @@ public void queueFrameViaSurface_nullColorInfo_setsDefaultColorInfo() throws Exc nullColorInfoFormat); produceFrameToFrameReaderSurface(frameTimeUs1); - HardwareBufferFrame recFrame1 = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); - assertThat(recFrame1.format).isEqualTo(expectedFormat); + AsyncFrame recFrame1 = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + assertThat(recFrame1).isNotNull(); + assertThat(recFrame1.frame.getFormat()).isEqualTo(expectedFormat); } @Test @@ -397,8 +423,153 @@ public void queueFrameViaSurface_unsetColorInfo_setsDefaultColorInfo() throws Ex unsetColorSpaceFormat); produceFrameToFrameReaderSurface(frameTimeUs1); - HardwareBufferFrame recFrame1 = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); - assertThat(recFrame1.format).isEqualTo(expectedFormat); + AsyncFrame recFrame1 = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + assertThat(recFrame1).isNotNull(); + assertThat(recFrame1.frame.getFormat()).isEqualTo(expectedFormat); + } + + @Test + public void + outputBitmap_withBitmapToHardwareBufferConverter_reusesHardwareBufferForRepeatedBitmap() + throws Exception { + HardwareBufferFrameReader frameReader = + new HardwareBufferFrameReader( + composition, + /* sequenceIndex= */ 0, + hardwareBufferFrame -> receivedFrames.add(hardwareBufferFrame), + handlerThread.getLooper(), + /* defaultSurfacePixelFormat= */ ImageFormat.YUV_420_888, + new DefaultImageReaderAdapter.Factory(), + e -> hardwareBufferFrameReaderException.set(e), + SystemClock.DEFAULT.createHandler(Util.getCurrentOrMainLooper(), null), + HardwareBufferJni.INSTANCE); + + try { + Bitmap bitmap = Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888); + frameReader.outputBitmap( + bitmap, + new ConstantRateTimestampIterator(/* durationUs= */ 1_000_000, /* frameRate= */ 30f), + /* sequenceOffsetUs= */ 0, + /* indexOfItem= */ 0); + + AsyncFrame frame1 = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + AsyncFrame frame2 = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + + assertThat(hardwareBufferFrameReaderException.get()).isNull(); + assertThat(frame1).isNotNull(); + assertThat(frame2).isNotNull(); + HardwareBuffer hardwareBuffer1 = getHardwareBuffer(frame1); + HardwareBuffer hardwareBuffer2 = getHardwareBuffer(frame2); + assertThat(hardwareBuffer1).isNotNull(); + assertThat(hardwareBuffer2).isNotNull(); + assertThat(hardwareBuffer1).isSameInstanceAs(hardwareBuffer2); + assertThat(hardwareBuffer1.isClosed()).isFalse(); + assertThat(getInternalImage(frame1)).isSameInstanceAs(bitmap); + assertThat(getInternalImage(frame2)).isSameInstanceAs(bitmap); + assertThat(getPresentationTimeUs(frame1)).isEqualTo(0); + assertThat(frame1.frame.getContentTimeUs()).isEqualTo(0); + assertThat(getPresentationTimeUs(frame2)).isEqualTo(33_333); + assertThat(frame2.frame.getContentTimeUs()).isEqualTo(33_333); + + releaseIfNeeded(frame1.frame, /* releaseFence= */ null); + releaseIfNeeded(frame2.frame, /* releaseFence= */ null); + } finally { + frameReader.release(); + } + } + + @Test + public void + outputBitmap_withBitmapToHardwareBufferConverter_differentBitmap_createsNewHardwareBuffer() + throws Exception { + HardwareBufferFrameReader frameReader = + new HardwareBufferFrameReader( + composition, + /* sequenceIndex= */ 0, + hardwareBufferFrame -> receivedFrames.add(hardwareBufferFrame), + handlerThread.getLooper(), + /* defaultSurfacePixelFormat= */ ImageFormat.YUV_420_888, + new DefaultImageReaderAdapter.Factory(), + e -> hardwareBufferFrameReaderException.set(e), + SystemClock.DEFAULT.createHandler(Util.getCurrentOrMainLooper(), null), + HardwareBufferJni.INSTANCE); + + try { + TimestampIterator singleFrame1 = + new ConstantRateTimestampIterator(/* durationUs= */ 1_000, /* frameRate= */ 1f); + TimestampIterator singleFrame2 = + new ConstantRateTimestampIterator(/* durationUs= */ 1_000, /* frameRate= */ 1f); + Bitmap bitmap1 = Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888); + Bitmap bitmap2 = Bitmap.createBitmap(20, 20, Bitmap.Config.ARGB_8888); + + frameReader.outputBitmap( + bitmap1, singleFrame1, /* sequenceOffsetUs= */ 0, /* indexOfItem= */ 0); + frameReader.outputBitmap( + bitmap2, singleFrame2, /* sequenceOffsetUs= */ 1_000, /* indexOfItem= */ 1); + + AsyncFrame frame1 = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + AsyncFrame frame2 = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + + assertThat(hardwareBufferFrameReaderException.get()).isNull(); + assertThat(frame1).isNotNull(); + assertThat(frame2).isNotNull(); + HardwareBuffer hardwareBuffer1 = getHardwareBuffer(frame1); + HardwareBuffer hardwareBuffer2 = getHardwareBuffer(frame2); + assertThat(hardwareBuffer1).isNotNull(); + assertThat(hardwareBuffer2).isNotNull(); + assertThat(hardwareBuffer1).isNotSameInstanceAs(hardwareBuffer2); + + releaseIfNeeded(frame1.frame, /* releaseFence= */ null); + releaseIfNeeded(frame2.frame, /* releaseFence= */ null); + } finally { + frameReader.release(); + } + } + + @Test + public void + outputBitmap_withBitmapToHardwareBufferConverter_releaseOutputFrameAndReader_closesHardwareBuffer() + throws Exception { + HardwareBufferFrameReader frameReader = + new HardwareBufferFrameReader( + composition, + /* sequenceIndex= */ 0, + hardwareBufferFrame -> receivedFrames.add(hardwareBufferFrame), + handlerThread.getLooper(), + /* defaultSurfacePixelFormat= */ ImageFormat.YUV_420_888, + new DefaultImageReaderAdapter.Factory(), + e -> hardwareBufferFrameReaderException.set(e), + SystemClock.DEFAULT.createHandler(Util.getCurrentOrMainLooper(), null), + HardwareBufferJni.INSTANCE); + + try { + Bitmap bitmap = Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888); + frameReader.outputBitmap( + bitmap, + new ConstantRateTimestampIterator(/* durationUs= */ 1_000, /* frameRate= */ 1f), + /* sequenceOffsetUs= */ 0, + /* indexOfItem= */ 0); + + AsyncFrame receivedFrame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + assertThat(receivedFrame).isNotNull(); + HardwareBuffer hardwareBuffer = checkNotNull(getHardwareBuffer(receivedFrame)); + + releaseIfNeeded(receivedFrame.frame, /* releaseFence= */ null); + frameReader.release(); + + flushHandlerThread(); + + assertThat(hardwareBuffer.isClosed()).isTrue(); + } finally { + frameReader.release(); + } + } + + private void flushHandlerThread() throws Exception { + Handler handler = new Handler(handlerThread.getLooper()); + CountDownLatch latch = new CountDownLatch(1); + handler.post(latch::countDown); + assertThat(latch.await(TEST_TIMEOUT_MS, MILLISECONDS)).isTrue(); } private void produceFrameToFrameReaderSurface(long presentationTimeUs) { @@ -409,4 +580,26 @@ private void produceFrameToFrameReaderSurface(long presentationTimeUs) { imageWriter.queueInputImage(image); } } + + private static long getPresentationTimeUs(AsyncFrame asyncFrame) { + return checkNotNull((Long) asyncFrame.frame.getMetadata().get(Frame.KEY_PRESENTATION_TIME_US)); + } + + private static Object getInternalImage(AsyncFrame asyncFrame) { + return ((DefaultHardwareBufferFrame) asyncFrame.frame).getInternalImage(); + } + + private static HardwareBuffer getHardwareBuffer(AsyncFrame asyncFrame) { + return ((HardwareBufferFrame) asyncFrame.frame).getHardwareBuffer(); + } + + @SuppressWarnings("deprecation") // Uses deprecated CompositionFrameMetadata. + private static CompositionFrameMetadata getCompositionFrameMetadata(AsyncFrame asyncFrame) { + return (CompositionFrameMetadata) + checkNotNull( + asyncFrame + .frame + .getMetadata() + .get(CompositionFrameMetadata.KEY_COMPOSITION_FRAME_METADATA)); + } } diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/HardwareBufferSampleConsumerAndroidTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/HardwareBufferSampleConsumerAndroidTest.java index e9530d3ffda..6a2bd4a07f6 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/HardwareBufferSampleConsumerAndroidTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/HardwareBufferSampleConsumerAndroidTest.java @@ -17,10 +17,13 @@ import static androidx.media3.test.utils.AssetInfo.MP4_ADVANCED_ASSET; import static androidx.media3.transformer.EditedMediaItemSequence.withAudioFrom; +import static androidx.media3.transformer.HardwareBufferFrameReader.CAPACITY; +import static androidx.media3.transformer.TransformerUtil.releaseIfNeeded; import static com.google.common.truth.Truth.assertThat; import static java.util.concurrent.TimeUnit.MILLISECONDS; import android.graphics.Bitmap; +import android.hardware.HardwareBuffer; import android.os.HandlerThread; import android.os.Looper; import androidx.media3.common.Format; @@ -30,12 +33,16 @@ import androidx.media3.common.util.Consumer; import androidx.media3.common.util.HandlerWrapper; import androidx.media3.common.util.SystemClock; -import androidx.media3.effect.HardwareBufferFrame; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.DefaultHardwareBufferFrame; +import androidx.media3.common.video.HardwareBufferFrame; +import androidx.media3.effect.HardwareBufferJni; import androidx.test.ext.junit.runners.AndroidJUnit4; +import androidx.test.filters.SdkSuppress; import com.google.common.collect.ImmutableList; -import java.util.ArrayList; -import java.util.List; +import java.util.concurrent.BlockingQueue; import java.util.concurrent.CountDownLatch; +import java.util.concurrent.LinkedBlockingQueue; import java.util.concurrent.atomic.AtomicReference; import org.junit.After; import org.junit.Before; @@ -47,11 +54,12 @@ * hardware buffers seems unsupported on robolectric. */ @RunWith(AndroidJUnit4.class) +@SdkSuppress(minSdkVersion = 26) public class HardwareBufferSampleConsumerAndroidTest { private static final long TEST_TIMEOUT_MS = 500L; private HardwareBufferSampleConsumer sampleConsumer; - private List receivedFrames; + private BlockingQueue receivedFrames; private HandlerThread handlerThread; private AtomicReference errorRef; @@ -60,7 +68,7 @@ public void setUp() { handlerThread = new HandlerThread("HardwareBufferSampleConsumerTest"); handlerThread.start(); errorRef = new AtomicReference<>(); - receivedFrames = new ArrayList<>(); + receivedFrames = new LinkedBlockingQueue<>(); } @After @@ -72,7 +80,7 @@ public void tearDown() { @Test public void queueInputARGB888Bitmap_outputsHardwareBufferFrameWithBitmap() throws InterruptedException { - CountDownLatch framesReceivedLatch = new CountDownLatch(2); + CountDownLatch framesReceivedLatch = new CountDownLatch(CAPACITY); sampleConsumer = createSampleConsumer( /* onFrame= */ (frame) -> { @@ -96,14 +104,18 @@ public void queueInputARGB888Bitmap_outputsHardwareBufferFrameWithBitmap() .isEqualTo(GraphInput.INPUT_RESULT_SUCCESS); assertThat(framesReceivedLatch.await(TEST_TIMEOUT_MS, MILLISECONDS)).isTrue(); - assertThat(receivedFrames).hasSize(2); - assertThat(receivedFrames.get(0).hardwareBuffer).isNull(); - assertThat(receivedFrames.get(0).internalFrame).isSameInstanceAs(bitmap); - assertThat(receivedFrames.get(1).hardwareBuffer).isNull(); - assertThat(receivedFrames.get(1).internalFrame).isSameInstanceAs(bitmap); + assertThat(receivedFrames).hasSize(CAPACITY); + for (int i = 0; i < 3; i++) { + AsyncFrame frame = receivedFrames.poll(TEST_TIMEOUT_MS, MILLISECONDS); + assertThat(frame).isNotNull(); + assertThat(getHardwareBuffer(frame)).isNotNull(); + assertThat(getInternalImage(frame)).isSameInstanceAs(bitmap); + releaseIfNeeded(frame.frame, /* releaseFence= */ null); + } + assertThat(receivedFrames).isEmpty(); } - private HardwareBufferSampleConsumer createSampleConsumer(Consumer onFrame) { + private HardwareBufferSampleConsumer createSampleConsumer(Consumer onFrame) { EditedMediaItem editedMediaItem = new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)).build(); EditedMediaItemSequence sequence = withAudioFrom(ImmutableList.of(editedMediaItem)); @@ -118,6 +130,15 @@ private HardwareBufferSampleConsumer createSampleConsumer(Consumer errorRef.set(error)); + error -> errorRef.set(error), + HardwareBufferJni.INSTANCE); + } + + private static HardwareBuffer getHardwareBuffer(AsyncFrame asyncFrame) { + return ((HardwareBufferFrame) asyncFrame.frame).getHardwareBuffer(); + } + + private static Object getInternalImage(AsyncFrame asyncFrame) { + return ((DefaultHardwareBufferFrame) asyncFrame.frame).getInternalImage(); } } diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/ParameterizedInputSequenceExportTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/ParameterizedInputSequenceExportTest.java index 551858659fc..ec80833527e 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/ParameterizedInputSequenceExportTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/ParameterizedInputSequenceExportTest.java @@ -157,7 +157,12 @@ public void export_completesWithCorrectFrameCount() throws Exception { .build() .run(testId, sequence.buildComposition(Effects.EMPTY)); - assertThat(result.exportResult.videoFrameCount).isEqualTo(sequence.totalExpectedFrameCount); + // Rarely, MediaCodec decoders output frames in the wrong order. + // When the MediaCodec encoder sees frames in the wrong order, fewer output frames are produced. + // Use a tolerance when comparing frame counts. See b/343476417#comment5. + assertThat(result.exportResult.videoFrameCount) + .isWithin(2) + .of(sequence.totalExpectedFrameCount); assertThat(new File(result.filePath).length()).isGreaterThan(0); } @@ -189,7 +194,12 @@ public void export_withCompositionEffect_completesWithCorrectFrameCount() throws /* height= */ 360, Presentation.LAYOUT_SCALE_TO_FIT))))); - assertThat(result.exportResult.videoFrameCount).isEqualTo(sequence.totalExpectedFrameCount); + // Rarely, MediaCodec decoders output frames in the wrong order. + // When the MediaCodec encoder sees frames in the wrong order, fewer output frames are produced. + // Use a tolerance when comparing frame counts. See b/343476417#comment5. + assertThat(result.exportResult.videoFrameCount) + .isWithin(2) + .of(sequence.totalExpectedFrameCount); assertThat(new File(result.filePath).length()).isGreaterThan(0); } } diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/SequenceEffectTestUtil.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/SequenceEffectTestUtil.java index 6a138007b78..03f6d725155 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/SequenceEffectTestUtil.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/SequenceEffectTestUtil.java @@ -35,6 +35,7 @@ import androidx.media3.common.util.Util; import androidx.media3.effect.Presentation; import androidx.media3.exoplayer.mediacodec.MediaCodecInfo; +import androidx.media3.test.utils.BitmapPixelTestUtil; import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableSet; import java.io.IOException; @@ -116,14 +117,29 @@ public static EditedMediaItem oneFrameFromImage(String uri, List effects } /** - * Assert that the bitmaps output in {@link #PNG_ASSET_BASE_PATH} match those written in {code - * actualBitmaps}. + * Asserts that the bitmaps output in {@link #PNG_ASSET_BASE_PATH} match those written in {@code + * actualBitmaps} with luma threshold {@link + * BitmapPixelTestUtil#MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE_LUMA}. * *

    Also saves {@code actualBitmaps} bitmaps, in case they differ from expected bitmaps, stored * at {@link #PNG_ASSET_BASE_PATH}/{@code testId}_id.png. */ public static void assertBitmapsMatchExpectedAndSave(List actualBitmaps, String testId) throws IOException { + assertBitmapsMatchExpectedAndSave( + actualBitmaps, testId, MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE_LUMA); + } + + /** + * Asserts that the bitmaps output in {@link #PNG_ASSET_BASE_PATH} match those written in {@code + * actualBitmaps} with {@code maxAveragePixelAbsoluteDifference}. + * + *

    Also saves {@code actualBitmaps} bitmaps, in case they differ from expected bitmaps, stored + * at {@link #PNG_ASSET_BASE_PATH}/{@code testId}_id.png. + */ + public static void assertBitmapsMatchExpectedAndSave( + List actualBitmaps, String testId, float maxAveragePixelAbsoluteDifference) + throws IOException { for (int i = 0; i < actualBitmaps.size(); i++) { maybeSaveTestBitmap( testId, /* bitmapLabel= */ String.valueOf(i), actualBitmaps.get(i), /* path= */ null); @@ -139,7 +155,7 @@ public static void assertBitmapsMatchExpectedAndSave(List actualBitmaps, expectedBitmap, actualBitmaps.get(i), subTestId); assertWithMessage("For expected bitmap " + expectedPath) .that(averagePixelAbsoluteDifference) - .isAtMost(MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE_LUMA); + .isAtMost(maxAveragePixelAbsoluteDifference); } } diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/SingleFrameCapturingEncoder.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/SingleFrameCapturingEncoder.java new file mode 100644 index 00000000000..2328874b3d5 --- /dev/null +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/SingleFrameCapturingEncoder.java @@ -0,0 +1,220 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static android.os.Build.VERSION.SDK_INT; +import static androidx.media3.test.utils.BitmapPixelTestUtil.createArgb8888BitmapFromRgba8888Image; +import static com.google.common.base.Preconditions.checkArgument; + +import android.graphics.Bitmap; +import android.graphics.PixelFormat; +import android.hardware.HardwareBuffer; +import android.media.Image; +import android.media.ImageReader; +import android.media.MediaCodec.BufferInfo; +import android.media.metrics.LogSessionId; +import android.view.Surface; +import androidx.annotation.Nullable; +import androidx.media3.common.Format; +import androidx.media3.common.MimeTypes; +import androidx.media3.decoder.DecoderInputBuffer; +import java.nio.ByteBuffer; +import java.util.concurrent.atomic.AtomicReference; + +/** + * A {@link Codec} that captures a single output video frame as a {@linkplain Bitmap}. + * + *

    This encoder is intended for testing single-frame exports (such as static image inputs + * configured with a single-frame duration). It does not drain frames continuously; providing an + * input with multiple frames may cause the GL pipeline to block waiting for free buffers. + */ +/* package */ final class SingleFrameCapturingEncoder implements Codec { + + /** A {@link Codec.EncoderFactory} that creates {@link SingleFrameCapturingEncoder} instances. */ + /* package */ static final class Factory implements Codec.EncoderFactory { + + private final AtomicReference lastVideoEncoder; + + /* package */ Factory() { + lastVideoEncoder = new AtomicReference<>(); + } + + @Override + public Codec createForAudioEncoding(Format format, @Nullable LogSessionId logSessionId) { + throw new UnsupportedOperationException(); + } + + @Override + public Codec createForVideoEncoding(Format format, @Nullable LogSessionId logSessionId) { + SingleFrameCapturingEncoder encoder = new SingleFrameCapturingEncoder(format); + lastVideoEncoder.set(encoder); + return encoder; + } + + @Override + public boolean isVideoFormatSupported(Format format) { + return SingleFrameCapturingEncoder.isVideoFormatSupported(format); + } + + @Override + public boolean audioNeedsEncoding() { + return false; + } + + @Override + public boolean videoNeedsEncoding() { + return true; + } + + /** + * Returns the last captured {@linkplain Bitmap bitmap}, or {@code null} if no frame was + * captured. + */ + @Nullable + /* package */ Bitmap getLastBitmap() { + @Nullable SingleFrameCapturingEncoder encoder = lastVideoEncoder.get(); + return encoder != null ? encoder.getLastBitmap() : null; + } + } + + private static final int MAX_PENDING_FRAME_COUNT = 1; + + private final Format configurationFormat; + private final ImageReader imageReader; + + @Nullable private volatile Bitmap lastBitmap; + private volatile boolean inputStreamEnded; + + /* package */ SingleFrameCapturingEncoder(Format configurationFormat) { + checkArgument( + isVideoFormatSupported(configurationFormat), + "Unsupported configuration format: %s", + configurationFormat); + this.configurationFormat = configurationFormat; + this.imageReader = + SDK_INT >= 29 + ? ImageReader.newInstance( + configurationFormat.width, + configurationFormat.height, + PixelFormat.RGBA_8888, + /* maxImages= */ MAX_PENDING_FRAME_COUNT, + HardwareBuffer.USAGE_GPU_COLOR_OUTPUT | HardwareBuffer.USAGE_CPU_READ_OFTEN) + : ImageReader.newInstance( + configurationFormat.width, + configurationFormat.height, + PixelFormat.RGBA_8888, + /* maxImages= */ MAX_PENDING_FRAME_COUNT); + } + + @Override + public String getName() { + return "SingleFrameCapturingEncoder"; + } + + @Override + public Format getConfigurationFormat() { + return configurationFormat; + } + + @Override + public Format getInputFormat() { + return configurationFormat; + } + + @Override + public Surface getInputSurface() { + return imageReader.getSurface(); + } + + @Override + public int getMaxPendingFrameCount() { + return MAX_PENDING_FRAME_COUNT; + } + + @Override + public boolean maybeDequeueInputBuffer(DecoderInputBuffer inputBuffer) { + throw new UnsupportedOperationException(); + } + + @Override + public void queueInputBuffer(DecoderInputBuffer inputBuffer) { + throw new UnsupportedOperationException(); + } + + @Override + public void signalEndOfInputStream() { + try (@Nullable Image image = imageReader.acquireLatestImage()) { + if (image != null) { + lastBitmap = createArgb8888BitmapFromRgba8888Image(image); + } + } finally { + try { + imageReader.close(); + } finally { + inputStreamEnded = true; + } + } + } + + @Override + @Nullable + public Format getOutputFormat() { + return configurationFormat; + } + + @Override + @Nullable + public ByteBuffer getOutputBuffer() { + return null; + } + + @Override + @Nullable + public BufferInfo getOutputBufferInfo() { + return null; + } + + @Override + public void releaseOutputBuffer(boolean render) {} + + @Override + public void releaseOutputBuffer(long renderPresentationTimeUs) {} + + @Override + public boolean isEnded() { + return inputStreamEnded; + } + + @Override + public void release() { + imageReader.close(); + } + + /** + * Returns the last captured {@linkplain Bitmap bitmap}, or {@code null} if no frame was captured. + */ + @Nullable + /* package */ Bitmap getLastBitmap() { + return lastBitmap; + } + + private static boolean isVideoFormatSupported(Format format) { + return format.sampleMimeType != null + && MimeTypes.isVideo(format.sampleMimeType) + && format.width > 0 + && format.height > 0; + } +} diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/SurfaceTestActivity.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/SurfaceTestActivity.java index c5c28aa13ee..06451a8faf8 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/SurfaceTestActivity.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/SurfaceTestActivity.java @@ -22,6 +22,7 @@ import android.os.Bundle; import android.view.SurfaceView; import android.view.TextureView; +import android.view.Window; import android.view.WindowManager; import androidx.media3.transformer.test.R; @@ -34,6 +35,7 @@ public final class SurfaceTestActivity extends Activity { @Override protected void onCreate(Bundle savedInstanceState) { super.onCreate(savedInstanceState); + requestWindowFeature(Window.FEATURE_NO_TITLE); setKeepScreenOn(this); setContentView(R.layout.surface_test_activity); surfaceView = findViewById(R.id.surface_view); diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerAndroidTestRunner.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerAndroidTestRunner.java index 305a4e9c703..4b33451722a 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerAndroidTestRunner.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerAndroidTestRunner.java @@ -55,19 +55,18 @@ import org.json.JSONObject; /** An android instrumentation test runner for {@link Transformer}. */ -public class TransformerAndroidTestRunner { +public final class TransformerAndroidTestRunner { private static final String TAG = "TransformerAndroidTest"; /** The default export timeout value. */ public static final int DEFAULT_TIMEOUT_SECONDS = 120; /** A {@link Builder} for {@link TransformerAndroidTestRunner} instances. */ - public static class Builder { + public static final class Builder { private final Context context; private final Transformer transformer; private boolean requestCalculateSsim; private int timeoutSeconds; - private boolean suppressAnalysisExceptions; @Nullable private Map inputValues; /** @@ -116,26 +115,6 @@ public Builder setRequestCalculateSsim(boolean requestCalculateSsim) { return this; } - /** - * Sets whether to suppress failures that occurs as a result of post-export analysis, such as - * SSIM calculation. - * - *

    Regardless of this value, analysis exceptions are attached to the analysis file. - * - *

    It's recommended to add a comment explaining why this suppression is needed, ideally with - * a bug number. - * - *

    The default value is {@code false}. - * - * @param suppressAnalysisExceptions Whether to suppress analysis exceptions. - * @return This {@link Builder}. - */ - @CanIgnoreReturnValue - public Builder setSuppressAnalysisExceptions(boolean suppressAnalysisExceptions) { - this.suppressAnalysisExceptions = suppressAnalysisExceptions; - return this; - } - /** * Sets a {@link Map} of transformer input values, which are propagated to the export summary * JSON file. @@ -155,12 +134,7 @@ public Builder setInputValues(@Nullable Map inputValues) { /** Builds the {@link TransformerAndroidTestRunner}. */ public TransformerAndroidTestRunner build() { return new TransformerAndroidTestRunner( - context, - transformer, - timeoutSeconds, - requestCalculateSsim, - suppressAnalysisExceptions, - inputValues); + context, transformer, timeoutSeconds, requestCalculateSsim, inputValues); } } @@ -168,7 +142,6 @@ public TransformerAndroidTestRunner build() { private final Transformer transformer; private final int timeoutSeconds; private final boolean requestCalculateSsim; - private final boolean suppressAnalysisExceptions; @Nullable private final Map inputValues; private TransformerAndroidTestRunner( @@ -176,13 +149,11 @@ private TransformerAndroidTestRunner( Transformer transformer, int timeoutSeconds, boolean requestCalculateSsim, - boolean suppressAnalysisExceptions, @Nullable Map inputValues) { this.context = context; this.transformer = transformer; this.timeoutSeconds = timeoutSeconds; this.requestCalculateSsim = requestCalculateSsim; - this.suppressAnalysisExceptions = suppressAnalysisExceptions; this.inputValues = inputValues; } @@ -279,8 +250,7 @@ public ExportTestResult run(String testId, MediaItem mediaItem) throws Exception * @param oldFilePath The old output file path to resume the export from. Passing {@code null} * will restart the export from the beginning. * @return The {@link ExportTestResult}. - * @throws Exception The cause of the export not completing, or an analysis exception if {@code - * suppressAnalysisExceptions} is {@code false}. + * @throws Exception The cause of the export not completing. */ private ExportTestResult runAndLog(String testId, Object mediaInput, @Nullable String oldFilePath) throws Exception { @@ -297,7 +267,7 @@ private ExportTestResult runAndLog(String testId, Object mediaInput, @Nullable S if (exportTestResult.exportResult.exportException != null) { throw exportTestResult.exportResult.exportException; } - if (!suppressAnalysisExceptions && exportTestResult.analysisException != null) { + if (exportTestResult.analysisException != null) { throw exportTestResult.analysisException; } return exportTestResult; diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerEndToEndTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerEndToEndTest.java index 76925ccf943..ae92e5e14c0 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerEndToEndTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerEndToEndTest.java @@ -27,6 +27,7 @@ import static androidx.media3.test.utils.AssetInfo.MP3_ASSET; import static androidx.media3.test.utils.AssetInfo.MP3_ASSET_CBR_TRAILING_GARBAGE; import static androidx.media3.test.utils.AssetInfo.MP3_ASSET_CBR_TRAILING_ID3V1; +import static androidx.media3.test.utils.AssetInfo.MP4_15FPS; import static androidx.media3.test.utils.AssetInfo.MP4_ADVANCED_ASSET; import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_DOLBY_VISION_HDR; import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_PHOTOS_TRIM_OPTIMIZATION_VIDEO; @@ -37,6 +38,7 @@ import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_WITH_SHORTER_AUDIO; import static androidx.media3.test.utils.AssetInfo.MP4_PORTRAIT_ASSET; import static androidx.media3.test.utils.AssetInfo.MP4_POSITIVE_SHIFT_EDIT_LIST; +import static androidx.media3.test.utils.AssetInfo.MP4_SIMPLE_ASSET; import static androidx.media3.test.utils.AssetInfo.MP4_TRIM_OPTIMIZATION; import static androidx.media3.test.utils.AssetInfo.MP4_TRIM_OPTIMIZATION_180; import static androidx.media3.test.utils.AssetInfo.MP4_TRIM_OPTIMIZATION_270; @@ -48,11 +50,14 @@ import static androidx.media3.test.utils.AssetInfo.WAV_96KHZ_ASSET; import static androidx.media3.test.utils.AssetInfo.WAV_ASSET; import static androidx.media3.test.utils.AssetInfo.WEBP_LARGE; +import static androidx.media3.test.utils.FormatSupportAssumptions.assumeAllFormatsSupported; import static androidx.media3.test.utils.FormatSupportAssumptions.assumeFormatsSupported; import static androidx.media3.test.utils.TestSummaryLogger.recordTestSkipped; import static androidx.media3.test.utils.TestUtil.createByteCountingAudioProcessor; import static androidx.media3.test.utils.TestUtil.extractAllSamplesFromFilePath; import static androidx.media3.test.utils.TestUtil.retrieveTrackFormat; +import static androidx.media3.transformer.AndroidTestUtil.assertExportResultHasOriginalVideoTimestamps; +import static androidx.media3.transformer.AndroidTestUtil.assertExportedVideoFrameRateIsConstant; import static androidx.media3.transformer.AndroidTestUtil.assumeCanEncodeWithProfile; import static androidx.media3.transformer.AndroidTestUtil.createFrameCountingEffect; import static androidx.media3.transformer.AndroidTestUtil.createOpenGlObjects; @@ -87,10 +92,14 @@ import android.os.Looper; import android.os.Message; import android.util.Pair; +import android.util.Rational; import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; import androidx.media3.common.C; import androidx.media3.common.Effect; import androidx.media3.common.Format; +import androidx.media3.common.GlObjectsProvider; +import androidx.media3.common.GlTextureInfo; import androidx.media3.common.MediaItem; import androidx.media3.common.MimeTypes; import androidx.media3.common.OnInputFrameProcessedListener; @@ -109,25 +118,28 @@ import androidx.media3.datasource.DataSourceBitmapLoader; import androidx.media3.effect.Contrast; import androidx.media3.effect.DefaultGlObjectsProvider; -import androidx.media3.effect.DefaultHardwareBufferEffectsPipeline; import androidx.media3.effect.DefaultVideoFrameProcessor; import androidx.media3.effect.FrameCache; import androidx.media3.effect.FrameDropEffect; import androidx.media3.effect.GlEffect; +import androidx.media3.effect.GlShaderProgram; +import androidx.media3.effect.PassthroughShaderProgram; import androidx.media3.effect.Presentation; import androidx.media3.effect.RgbFilter; import androidx.media3.effect.ScaleAndRotateTransformation; import androidx.media3.effect.SpeedChangeEffect; import androidx.media3.effect.TimestampWrapper; -import androidx.media3.effect.ndk.HardwareBufferJni; import androidx.media3.exoplayer.audio.TeeAudioProcessor; +import androidx.media3.exoplayer.mediacodec.MediaCodecUtil; import androidx.media3.extractor.mp4.Mp4Extractor; import androidx.media3.extractor.text.DefaultSubtitleParserFactory; import androidx.media3.inspector.MediaExtractorCompat; import androidx.media3.inspector.MetadataRetriever; +import androidx.media3.muxer.Muxer; +import androidx.media3.muxer.MuxerException; +import androidx.media3.test.utils.AssetInfo; import androidx.media3.test.utils.FakeExtractorOutput; import androidx.media3.test.utils.FakeTrackOutput; -import androidx.media3.test.utils.RecordingHardwareBufferEffectsPipeline; import androidx.media3.test.utils.TestSpeedProvider; import androidx.media3.transformer.AssetLoader.CompositionSettings; import androidx.test.core.app.ApplicationProvider; @@ -137,6 +149,7 @@ import com.google.common.collect.ImmutableSet; import com.google.common.util.concurrent.ListenableFuture; import java.io.File; +import java.io.IOException; import java.nio.ByteBuffer; import java.nio.ShortBuffer; import java.util.ArrayList; @@ -145,6 +158,7 @@ import java.util.concurrent.atomic.AtomicInteger; import java.util.concurrent.atomic.AtomicReference; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; +import org.json.JSONException; import org.junit.Before; import org.junit.Rule; import org.junit.Test; @@ -159,6 +173,10 @@ @RunWith(AndroidJUnit4.class) public class TransformerEndToEndTest { + private static final Rational FPS_60 = new Rational(60, 1); + private static final Rational FPS_15 = new Rational(15, 1); + private static final Rational FPS_HALF = new Rational(1, 2); + private static final GlEffect NO_OP_EFFECT = new Contrast(0f); private static final int MIN_BLANK_FRAME_OUTPUT_WIDTH = 320; private static final int MIN_BLANK_FRAME_OUTPUT_HEIGHT = 240; @@ -168,6 +186,9 @@ public class TransformerEndToEndTest { private String testId; + @Rule + public final GlFrameProcessorTestRule glFrameProcessorTestRule = new GlFrameProcessorTestRule(); + private volatile @MonotonicNonNull TextureAssetLoader textureAssetLoader; @Before @@ -827,6 +848,164 @@ public void videoEditing_withMultiSequenceAndCompositionEffect_appliesEffect() t assertThat(timestampRecordingShaderProgram.getInputTimestampsUs()).isNotEmpty(); } + @Test + public void videoEditing_withAggregationFps60AndNoFrameProcessor_ignoresAggregationParameters() + throws Exception { + Composition composition = + assumeFormatsAndSetupComposition(context, testId, FPS_60, MP4_SIMPLE_ASSET); + + // The Transformer has no frame processor, so the aggregator is not active. + // The frame aggregation parameters should be ignored. + Transformer transformer = + new Transformer.Builder(context) + .setEncoderFactory( + new DefaultEncoderFactory.Builder(context).setEnableFallback(false).build()) + .build(); + + ExportTestResult result = + new TransformerAndroidTestRunner.Builder(context, transformer) + .build() + .run(testId, composition); + + // The output should have the original frame count of 30, not upsampled to 60. + assertExportResultHasOriginalVideoTimestamps(result, MP4_SIMPLE_ASSET); + } + + @Test + @SdkSuppress(minSdkVersion = AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) + public void videoEditing_withAggregationFps60AndFrameProcessor_upsamplesTo60Fps() + throws Exception { + Composition composition = + assumeFormatsAndSetupComposition(context, testId, FPS_60, MP4_ADVANCED_ASSET); + + Transformer transformer = buildTransformerWithFrameProcessorFactory(context); + + ExportTestResult result = + new TransformerAndroidTestRunner.Builder(context, transformer) + .build() + .run(testId, composition); + + // As defined in AssetInfo, the last frame timestamp of the input is 967,633us. + // At 60 fps, the 59th frame is targeted at 966,666us. The 60th frame (targeted at + // 983,333us) exceeds the last available timestamp and is dropped, yielding exactly 59 frames. + assertThat(result.exportResult.videoFrameCount).isEqualTo(59); + assertThat(new File(result.filePath).length()).isGreaterThan(0); + assertExportedVideoFrameRateIsConstant(result.filePath, FPS_60); + } + + @Test + @SdkSuppress(minSdkVersion = AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) + public void videoEditing_multiSequenceWithAggregationFps60AndFrameProcessor_upsamplesTo60Fps() + throws Exception { + Composition composition = + assumeFormatsAndSetupComposition( + context, testId, FPS_60, MP4_ADVANCED_ASSET, MP4_PORTRAIT_ASSET); + + Transformer transformer = buildTransformerWithFrameProcessorFactory(context); + + ExportTestResult result = + new TransformerAndroidTestRunner.Builder(context, transformer) + .build() + .run(testId, composition); + + // As defined in AssetInfo, the last frame timestamp of the input is 967,633us. + // At 60 fps, the 59th frame is targeted at 966,666us. The 60th frame (targeted at + // 983,333us) exceeds the last available timestamp and is dropped, yielding exactly 59 frames. + assertThat(result.exportResult.videoFrameCount).isEqualTo(59); + assertThat(new File(result.filePath).length()).isGreaterThan(0); + assertExportedVideoFrameRateIsConstant(result.filePath, FPS_60); + } + + @Test + @SdkSuppress(minSdkVersion = AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) + public void videoEditing_multiSequenceWithNoAggregationFpsAndFrameProcessor_keepsIntrinsicFps() + throws Exception { + Composition composition = + assumeFormatsAndSetupComposition( + context, testId, /* targetVideoFrameRate= */ null, MP4_15FPS, MP4_PORTRAIT_ASSET); + + Transformer transformer = buildTransformerWithFrameProcessorFactory(context); + + ExportTestResult result = + new TransformerAndroidTestRunner.Builder(context, transformer) + .build() + .run(testId, composition); + + // The Composition has no target frame rate, so the Transformer passes the + // frames through with their exact original timestamps, yielding exactly 15 frames. + assertExportResultHasOriginalVideoTimestamps(result, MP4_15FPS); + } + + @Test + @SdkSuppress(minSdkVersion = AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) + public void videoEditing_singleSequenceWithNoAggregationFpsAndFrameProcessor_keepsIntrinsicFps() + throws Exception { + Composition composition = + assumeFormatsAndSetupComposition( + context, testId, /* targetVideoFrameRate= */ null, MP4_15FPS); + + Transformer transformer = buildTransformerWithFrameProcessorFactory(context); + + ExportTestResult result = + new TransformerAndroidTestRunner.Builder(context, transformer) + .build() + .run(testId, composition); + + // The Composition has no target frame rate, so the Transformer passes the + // frames through with their exact original timestamps, yielding exactly 15 frames. + assertExportResultHasOriginalVideoTimestamps(result, MP4_15FPS); + } + + @Test + @SdkSuppress(minSdkVersion = AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) + public void videoEditing_withAggregationFps15AndFrameProcessor_downsamplesTo15Fps() + throws Exception { + // As defined in AssetInfo, MP4_ADVANCED_ASSET has a duration of 1024ms and + // contains 30 frames (~29.3 fps). The Composition downsamples it to exactly 15 fps. + Composition composition = + assumeFormatsAndSetupComposition(context, testId, FPS_15, MP4_ADVANCED_ASSET); + + Transformer transformer = buildTransformerWithFrameProcessorFactory(context); + + ExportTestResult result = + new TransformerAndroidTestRunner.Builder(context, transformer) + .build() + .run(testId, composition); + + // As defined in AssetInfo, the last frame timestamp of the input is 967,633us. + // At 15 fps, the 15th frame is targeted at 933,333us. The 16th frame (targeted at + // 1,000,000us) exceeds the last available timestamp and is dropped, yielding exactly 15 frames. + assertThat(result.exportResult.videoFrameCount).isEqualTo(15); + assertThat(new File(result.filePath).length()).isGreaterThan(0); + assertExportedVideoFrameRateIsConstant(result.filePath, FPS_15); + } + + @Test + @SdkSuppress(minSdkVersion = AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) + public void videoEditing_withAggregationFpsHalfAndFrameProcessor_downsamplesToFpsHalf() + throws Exception { + // As defined in AssetInfo, MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_5S has a duration + // of 5019ms and contains 300 frames (~59.8 fps). The Composition downsamples it to exactly 0.5 + // fps. + Composition composition = + assumeFormatsAndSetupComposition( + context, testId, FPS_HALF, MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_5S); + + Transformer transformer = buildTransformerWithFrameProcessorFactory(context); + + ExportTestResult result = + new TransformerAndroidTestRunner.Builder(context, transformer) + .build() + .run(testId, composition); + + // As defined in AssetInfo, the last frame timestamp of the input is around 4,983,333us. + // At 0.5 fps, the 3rd frame is targeted at 4,000,000us. The 4th frame (targeted at + // 6,000,000us) exceeds the last available timestamp and is dropped, yielding exactly 3 frames. + assertThat(result.exportResult.videoFrameCount).isEqualTo(3); + assertThat(new File(result.filePath).length()).isGreaterThan(0); + assertExportedVideoFrameRateIsConstant(result.filePath, FPS_HALF); + } + @Test public void videoOnly_completesWithConsistentDuration() throws Exception { assumeFormatsSupported( @@ -2354,7 +2533,7 @@ public void transcode_withOutputAudioMimeTypeOpus_completesSuccessfully() throws } @Test - @SdkSuppress(minSdkVersion = 28) + @SdkSuppress(minSdkVersion = AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) public void cancelExport_withPacketConsumerAndImageComposition_doesNotCrash() throws Exception { assumeFormatsSupported( context, @@ -2364,22 +2543,31 @@ public void cancelExport_withPacketConsumerAndImageComposition_doesNotCrash() th CountDownLatch firstFrameLatch = new CountDownLatch(1); AtomicReference<@NullableType ExportException> exceptionReference = new AtomicReference<>(); AtomicBoolean completedReference = new AtomicBoolean(false); + GlEffect countingEffect = + new GlEffect() { + @Override + public GlShaderProgram toGlShaderProgram(Context context, boolean useHdr) { + return new PassthroughShaderProgram() { + @Override + public void queueInputFrame( + GlObjectsProvider glObjectsProvider, + GlTextureInfo inputTexture, + long presentationTimeUs) { + super.queueInputFrame(glObjectsProvider, inputTexture, presentationTimeUs); + firstFrameLatch.countDown(); + } + }; + } + }; EditedMediaItem imageItem = new EditedMediaItem.Builder( new MediaItem.Builder().setUri(JPG_ASSET.uri).setImageDurationMs(50000).build()) .setFrameRate(30) + .setEffects(new Effects(ImmutableList.of(), ImmutableList.of(countingEffect))) .build(); Transformer transformer = - new Transformer.Builder(context) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setHardwareBufferEffectsPipeline( - RecordingHardwareBufferEffectsPipeline.create( - context, - HardwareBufferJni.INSTANCE, - (frames) -> { - firstFrameLatch.countDown(); - return frames; - })) + glFrameProcessorTestRule + .createTransformerBuilder(context) .addListener( new Transformer.Listener() { @Override @@ -2416,6 +2604,76 @@ public void onError( assertThat(completedReference.get()).isFalse(); } + @Test + public void cancelExport_withFrameworkMuxer_doesNotThrow() throws Exception { + assumeFormatsSupported( + context, + testId, + /* inputFormat= */ MP4_ASSET_WITH_INCREASING_TIMESTAMPS.videoFormat, + /* outputFormat= */ MP4_ASSET_WITH_INCREASING_TIMESTAMPS.videoFormat); + AtomicReference<@NullableType ExportException> exceptionReference = new AtomicReference<>(); + AtomicBoolean completedReference = new AtomicBoolean(false); + EditedMediaItem item = + new EditedMediaItem.Builder( + MediaItem.fromUri(Uri.parse(MP4_ASSET_WITH_INCREASING_TIMESTAMPS.uri))) + .build(); + + CountDownLatch muxerCreatedLatch = new CountDownLatch(1); + FrameworkMuxer.Factory frameworkMuxerFactory = new FrameworkMuxer.Factory(); + Muxer.Factory latchMuxerFactory = + new Muxer.Factory() { + @Override + public Muxer create(String path) throws MuxerException { + Muxer muxer = frameworkMuxerFactory.create(path); + muxerCreatedLatch.countDown(); + return muxer; + } + + @Override + public ImmutableList getSupportedSampleMimeTypes(@C.TrackType int trackType) { + return frameworkMuxerFactory.getSupportedSampleMimeTypes(trackType); + } + }; + + Transformer transformer = + new Transformer.Builder(context) + .setMuxerFactory(latchMuxerFactory) + .addListener( + new Transformer.Listener() { + @Override + public void onCompleted(Composition composition, ExportResult exportResult) { + completedReference.set(true); + } + + @Override + public void onError( + Composition composition, + ExportResult exportResult, + ExportException exportException) { + exceptionReference.set(exportException); + } + }) + .build(); + + TransformerAndroidTestRunner runner = + new TransformerAndroidTestRunner.Builder(context, transformer).build(); + ListenableFuture unused = runner.runAsync(testId, item); + assertThat(muxerCreatedLatch.await(TEST_TIMEOUT_MS, MILLISECONDS)).isTrue(); + + getInstrumentation() + .runOnMainSync( + () -> { + try { + transformer.cancel(); + } catch (RuntimeException e) { + exceptionReference.set(ExportException.createForUnexpected(e)); + } + }); + + assertThat(exceptionReference.get()).isNull(); + assertThat(completedReference.get()).isFalse(); + } + @Test @SdkSuppress(minSdkVersion = 33) public void transcode_withOutputVideoMimeTypeAv1_andPacketConsumer_completesSuccessfully() @@ -2433,10 +2691,8 @@ public void transcode_withOutputVideoMimeTypeAv1_andPacketConsumer_completesSucc MediaItem mediaItem = MediaItem.fromUri(Uri.parse(MP4_ADVANCED_ASSET.uri)); EditedMediaItem editedMediaItem = new EditedMediaItem.Builder(mediaItem).build(); Transformer transformer = - new Transformer.Builder(context) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setHardwareBufferEffectsPipeline( - DefaultHardwareBufferEffectsPipeline.create(context, HardwareBufferJni.INSTANCE)) + glFrameProcessorTestRule + .createTransformerBuilder(context) .setVideoMimeType(MimeTypes.VIDEO_AV1) .build(); @@ -2456,13 +2712,8 @@ public void transcode_withOutputVideoMimeTypeAv1_andPacketConsumer_completesSucc @Test @SdkSuppress(minSdkVersion = 33) - public void export_withHardwareBufferEffectsPipeline_forcesVideoTranscoding() throws Exception { - Transformer transformer = - new Transformer.Builder(context) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setHardwareBufferEffectsPipeline( - DefaultHardwareBufferEffectsPipeline.create(context, HardwareBufferJni.INSTANCE)) - .build(); + public void export_withDefaultGlFrameProcessor_forcesVideoTranscoding() throws Exception { + Transformer transformer = buildTransformerWithFrameProcessorFactory(context); MediaItem mediaItem = MediaItem.fromUri(MP4_ADVANCED_ASSET.uri); ExportTestResult result = @@ -2755,44 +3006,6 @@ public void setSpeed_withAudioOnly_outputsExpectedNumberOfSamples() throws Excep assertThat(readBytes.get() / 2).isWithin(3).of(38588); } - @Test - public void setSpeed_withNoOtherEffects_transcodesAndAppliesEffect() throws Exception { - Transformer transformer = new Transformer.Builder(context).build(); - SpeedProvider speedProvider = - TestSpeedProvider.createWithStartTimes( - new long[] {0L, 300_000, 600_000}, new float[] {4f, 0.5f, 2f}); - // Use a WAV file without any effects so that the EditedMediaItem is a candidate for - // transmuxing. - EditedMediaItem input = - new EditedMediaItem.Builder(MediaItem.fromUri(WAV_ASSET.uri)) - .setSpeed(speedProvider) - .build(); - - TransformerAndroidTestRunner testRunner = - new TransformerAndroidTestRunner.Builder(context, transformer).build(); - ExportTestResult result = testRunner.run(testId, input); - - // TODO: b/479474095 - Enable this assertion once ExportResult reports the right operation for - // single asset exports that bypass the decoder. - // assertThat(result.exportResult.audioConversionProcess).isEqualTo(CONVERSION_PROCESS_TRANSCODED); - // Assert that decoder is bypassed for WAV file. - assertThat(result.exportResult.processedInputs.get(0).audioDecoderName).isNull(); - - // When Transformer transmuxes a file, it skips the entire effects pipeline. We should make sure - // that the file is processed and the speed adjustment is applied. - AtomicInteger bytesRead = new AtomicInteger(); - EditedMediaItem speedAdjustedOutput = - new EditedMediaItem.Builder(MediaItem.fromUri(result.filePath)) - .setEffects(fromProcessors(createByteCountingAudioProcessor(bytesRead))) - .build(); - - testRunner.run(testId, speedAdjustedOutput); - - // Allow 0.05ms of tolerance because of encoder dropping frames (b/475182836). - // 300ms / 4 + 300ms / 0.5 + 400ms / 2 = 875ms -> 38588 frames. - assertThat(bytesRead.get() / 2).isWithin(2300).of(38588); - } - // HE-AAC profile support is available from API 29. @SdkSuppress(minSdkVersion = 29) @Test @@ -2825,12 +3038,39 @@ public void export_setAudioEncodingProfile_changesProfile() throws Exception { assertThat(profileAndLevel.first).isEqualTo(AACObjectHE); } - // On API 23, the encoder output format does not seem to contain bitrate, hence the test fails. - @SdkSuppress(minSdkVersion = 24) + @Test + public void export_withRawAudioAndSetBitrate_encodes() throws Exception { + Context context = ApplicationProvider.getApplicationContext(); + Transformer transformer = + new Transformer.Builder(context) + .setEncoderFactory( + new DefaultEncoderFactory.Builder(context) + .setRequestedAudioEncoderSettings( + new AudioEncoderSettings.Builder().setBitrate(1024).build()) + .build()) + .build(); + MediaItem mediaItem = MediaItem.fromUri(WAV_ASSET.uri); + + ExportTestResult result = + new TransformerAndroidTestRunner.Builder(context, transformer) + .build() + .run(testId, mediaItem); + + assertThat(result.exportResult.audioMimeType).isEqualTo(MimeTypes.AUDIO_AAC); + + try (MetadataRetriever r = + new MetadataRetriever.Builder(context, MediaItem.fromUri(result.filePath)).build()) { + assertThat(r.retrieveTrackGroups().get().get(0).getFormat(0).sampleMimeType) + .isEqualTo(MimeTypes.AUDIO_AAC); + long duration = r.retrieveDurationUs().get(); + // Allow at most 2048 samples of divergence (2048@44.1KHz -> 46_440us). + assertThat(duration).isWithin(46_440).of(1_000_000); + } + } + @Test public void export_setAudioEncodingBitrate_configuresEncoderWithRequestedBitrate() throws Exception { - // On API 23, the encoder output format does not seem to contain bitrate, hence the test fails. Context context = ApplicationProvider.getApplicationContext(); int requestedBitrate = 60_000; // The MediaMuxer is not writing the bitrate hence use the InAppMuxer. @@ -3437,6 +3677,51 @@ public void export_withBFramesInputAndEditedMediaItemFrameRateSetForceTranscode_ .inOrder(); } + private static Composition assumeFormatsAndSetupComposition( + Context context, + String testId, + @Nullable Rational targetVideoFrameRate, + AssetInfo... assetInfos) + throws IOException, JSONException, MediaCodecUtil.DecoderQueryException { + ImmutableList.Builder inputFormats = new ImmutableList.Builder<>(); + for (AssetInfo assetInfo : assetInfos) { + inputFormats.add(assetInfo.videoFormat); + } + assumeAllFormatsSupported( + context, testId, inputFormats.build(), /* outputFormat= */ assetInfos[0].videoFormat); + + ImmutableList.Builder sequences = new ImmutableList.Builder<>(); + for (int i = 0; i < assetInfos.length; i++) { + AssetInfo assetInfo = assetInfos[i]; + MediaItem mediaItem = MediaItem.fromUri(Uri.parse(assetInfo.uri)); + EditedMediaItem editedMediaItem = new EditedMediaItem.Builder(mediaItem).build(); + if (i == 0) { + sequences.add( + EditedMediaItemSequence.withAudioAndVideoFrom(ImmutableList.of(editedMediaItem))); + } else { + sequences.add( + new EditedMediaItemSequence.Builder(ImmutableSet.of(C.TRACK_TYPE_VIDEO)) + .addItem(editedMediaItem) + .build()); + } + } + Composition.Builder builder = new Composition.Builder(sequences.build()); + if (targetVideoFrameRate != null) { + builder.setVideoFrameAggregationParameters( + new VideoFrameAggregationParameters.Builder().setFrameRate(targetVideoFrameRate).build()); + } + return builder.build(); + } + + @RequiresApi(AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) + private Transformer buildTransformerWithFrameProcessorFactory(Context context) { + return glFrameProcessorTestRule + .createTransformerBuilder(context) + .setEncoderFactory( + new DefaultEncoderFactory.Builder(context).setEnableFallback(false).build()) + .build(); + } + private static AudioProcessor createSonic(float pitch) { SonicAudioProcessor sonic = new SonicAudioProcessor(); sonic.setPitch(pitch); diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerFrameProcessorTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerFrameProcessorTest.java index d668127fafc..e92621f1909 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerFrameProcessorTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerFrameProcessorTest.java @@ -16,27 +16,39 @@ */ package androidx.media3.transformer; +import static android.graphics.Bitmap.Config.RGBA_1010102; +import static androidx.media3.common.C.MICROS_PER_SECOND; import static androidx.media3.test.utils.AssetInfo.MP4_SIMPLE_ASSET; +import static androidx.media3.test.utils.AssetInfo.PNG_ASSET; import static androidx.media3.test.utils.FormatSupportAssumptions.assumeFormatsSupported; +import static androidx.media3.transformer.ExperimentalAnalyzerModeFactory.buildAnalyzer; import static com.google.common.truth.Truth.assertThat; +import static com.google.common.util.concurrent.Futures.immediateFuture; import android.content.Context; import android.graphics.Bitmap; import android.hardware.HardwareBuffer; -import androidx.annotation.RequiresApi; +import android.net.Uri; import androidx.media3.common.MediaItem; import androidx.media3.common.VideoCompositorSettings; +import androidx.media3.common.util.BitmapLoader; +import androidx.media3.common.video.AsyncFrame; import androidx.media3.common.video.Frame; import androidx.media3.common.video.FrameProcessor; +import androidx.media3.common.video.HardwareBufferFrame; import androidx.media3.effect.AlphaScale; import androidx.media3.effect.DefaultGlFrameProcessor; -import androidx.media3.effect.HardwareBufferJniWrapper; +import androidx.media3.test.utils.CapturingFrameProcessor; import androidx.media3.test.utils.FakeFrameProcessor; +import androidx.media3.test.utils.FakeHardwareBufferJniWrapper; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; import androidx.test.filters.SdkSuppress; import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableMap; +import com.google.common.util.concurrent.ListenableFuture; +import java.util.List; +import java.util.concurrent.atomic.AtomicInteger; import org.junit.Before; import org.junit.Rule; import org.junit.Test; @@ -68,8 +80,10 @@ public void export_compositionWithFrameProcessor_populatesRequiredMetadataFields /* inputFormat= */ MP4_SIMPLE_ASSET.videoFormat, /* outputFormat= */ MP4_SIMPLE_ASSET.videoFormat); - FakeFrameProcessor.Factory frameProcessorFactory = + FakeFrameProcessor.Factory fakeFrameProcessorFactory = new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ true); + CapturingFrameProcessor.Factory frameProcessorFactory = + new CapturingFrameProcessor.Factory(fakeFrameProcessorFactory); Transformer transformer = new Transformer.Builder(context) .setFrameProcessorFactory(frameProcessorFactory) @@ -95,9 +109,9 @@ public void export_compositionWithFrameProcessor_populatesRequiredMetadataFields new TransformerAndroidTestRunner.Builder(context, transformer).build().run(testId, composition); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; - FakeFrameProcessor.FramesEvent framesEvent = - (FakeFrameProcessor.FramesEvent) frameProcessor.getQueuedEvents().get(0); + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); + CapturingFrameProcessor.FramesEvent framesEvent = + (CapturingFrameProcessor.FramesEvent) frameProcessor.getQueuedEvents().get(0); Frame frame = framesEvent.frames.get(0).frame; ImmutableMap metadata = frame.getMetadata(); @@ -113,34 +127,78 @@ public void export_compositionWithFrameProcessor_populatesRequiredMetadataFields .isEqualTo(composition.effects.videoEffects); } - /** A no-op {@link HardwareBufferJniWrapper} that always succeeds. */ - @RequiresApi(26) - private static final class FakeHardwareBufferJniWrapper implements HardwareBufferJniWrapper { - @Override - public long nativeCreateEglImageFromHardwareBuffer( - long displayHandle, HardwareBuffer hardwareBuffer) { - return 1L; - } - - @Override - public boolean nativeBindEGLImage(int target, long eglImageHandle) { - return true; - } - - @Override - public boolean nativeDestroyEGLImage(long displayHandle, long imageHandle) { - return true; - } - - @Override - public boolean nativeCopyBitmapToHardwareBuffer(Bitmap bitmap, HardwareBuffer hb) { - return true; - } - - @Override - public boolean nativeCopyHardwareBufferToHardwareBuffer( - HardwareBuffer srcHb, HardwareBuffer dstHb) { - return true; - } + @Test + @SdkSuppress(minSdkVersion = 34) + public void export_withProgrammaticHdrImage_queuesHardwareBufferWithCorrectFormat() + throws Exception { + AtomicInteger receivedFormat = new AtomicInteger(); + // Create a fake BitmapLoader that strictly outputs a true 10-bit SDR/HDR bitmap. + BitmapLoader fakeBitmapLoader = + new BitmapLoader() { + @Override + public boolean supportsMimeType(String mimeType) { + return true; + } + + @Override + public ListenableFuture decodeBitmap(byte[] data) { + return immediateFuture(Bitmap.createBitmap(1, 1, RGBA_1010102)); + } + + @Override + public ListenableFuture loadBitmap(Uri uri) { + return immediateFuture(Bitmap.createBitmap(1, 1, RGBA_1010102)); + } + }; + FakeFrameProcessor.Factory fakeFrameProcessorFactory = + new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ true); + FrameProcessor.Factory interceptingFactory = + (output, listenerExecutor, listener) -> { + FrameProcessor realProcessor = + fakeFrameProcessorFactory.create(output, listenerExecutor, listener); + + return new FrameProcessor() { + @Override + public boolean queue(List frames) { + for (AsyncFrame asyncFrame : frames) { + if (asyncFrame.frame instanceof HardwareBufferFrame) { + HardwareBufferFrame hardwareBufferFrame = (HardwareBufferFrame) asyncFrame.frame; + // Safely capture the format synchronously before the buffer is closed later + receivedFormat.set(hardwareBufferFrame.getHardwareBuffer().getFormat()); + } + } + return realProcessor.queue(frames); + } + + @Override + public void signalEndOfStream() { + realProcessor.signalEndOfStream(); + } + + @Override + public void close() { + realProcessor.close(); + } + }; + }; + Transformer transformer = + buildAnalyzer( + context, + new Transformer.Builder(context) + .setAssetLoaderFactory(new ImageAssetLoader.Factory(context, fakeBitmapLoader)) + .setFrameProcessorFactory(interceptingFactory) + .setNativeHardwareBufferHelpers(new FakeHardwareBufferJniWrapper()) + .build()); + EditedMediaItem editedMediaItem = + new EditedMediaItem.Builder(MediaItem.fromUri(PNG_ASSET.uri)) + .setDurationUs(MICROS_PER_SECOND / 2) + .setFrameRate(30) + .build(); + + new TransformerAndroidTestRunner.Builder(context, transformer) + .build() + .run(testId, editedMediaItem); + + assertThat(receivedFormat.get()).isEqualTo(HardwareBuffer.RGBA_1010102); } } diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerParameterizedPacketConsumerAndroidTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerParameterizedFrameProcessorAndroidTest.java similarity index 60% rename from libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerParameterizedPacketConsumerAndroidTest.java rename to libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerParameterizedFrameProcessorAndroidTest.java index b88eafc8af9..694b216c4d9 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerParameterizedPacketConsumerAndroidTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerParameterizedFrameProcessorAndroidTest.java @@ -15,7 +15,6 @@ */ package androidx.media3.transformer; -import static androidx.media3.common.util.Util.isRunningOnEmulator; import static androidx.media3.test.utils.AssetInfo.MP4_ADVANCED_ASSET; import static androidx.media3.test.utils.CompositionAssetInfo.MULTI_SEQUENCE_CONFIGS; import static androidx.media3.test.utils.CompositionAssetInfo.MULTI_SEQUENCE_VIDEO_CONFIGS; @@ -24,35 +23,20 @@ import static androidx.media3.test.utils.EditedMediaItemAssetInfo.VIDEO_ONLY_CLIPPED_TWICE_SPEED; import static androidx.media3.test.utils.FormatSupportAssumptions.assumeAllFormatsSupported; import static androidx.media3.transformer.ExportResult.CONVERSION_PROCESS_TRANSCODED; -import static androidx.media3.transformer.GlFrameProcessorTestUtil.closeTestingGlResources; import static com.google.common.truth.Truth.assertThat; -import static java.util.concurrent.TimeUnit.MILLISECONDS; import android.content.Context; -import androidx.annotation.Nullable; import androidx.media3.common.C; -import androidx.media3.common.GlObjectsProvider; import androidx.media3.common.MediaItem; -import androidx.media3.common.util.GlUtil; -import androidx.media3.common.util.Util; -import androidx.media3.effect.DefaultGlFrameProcessor; -import androidx.media3.effect.DefaultGlObjectsProvider; -import androidx.media3.effect.DefaultHardwareBufferEffectsPipeline; -import androidx.media3.effect.FrameProcessorUtils; -import androidx.media3.effect.ndk.HardwareBufferJni; import androidx.media3.test.utils.CompositionAssetInfo; import androidx.test.core.app.ApplicationProvider; import androidx.test.filters.SdkSuppress; import com.google.common.collect.ImmutableList; -import com.google.common.util.concurrent.ListeningExecutorService; -import com.google.common.util.concurrent.MoreExecutors; import com.google.testing.junit.testparameterinjector.TestParameter; import com.google.testing.junit.testparameterinjector.TestParameterInjector; import com.google.testing.junit.testparameterinjector.TestParameterValuesProvider; import java.io.File; import java.util.List; -import org.checkerframework.checker.nullness.qual.MonotonicNonNull; -import org.junit.After; import org.junit.Before; import org.junit.Rule; import org.junit.Test; @@ -62,22 +46,17 @@ /** Parameterized end-to-end export tests for {@link Transformer}. */ @RunWith(TestParameterInjector.class) // TODO: b/479415308 - Expand API versions below 28 once supported. -@SdkSuppress(minSdkVersion = 28) -public final class TransformerParameterizedPacketConsumerAndroidTest { - - private static final String PACKET_PROCESSOR = "packet_processor"; - private static final String DEFAULT_GL_FRAME_PROCESSOR = "default_gl_frame_processor"; - private static final long TEST_TIMEOUT_MS = isRunningOnEmulator() ? 20_000 : 10_000; +@SdkSuppress(minSdkVersion = AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) +public final class TransformerParameterizedFrameProcessorAndroidTest { @Rule public final TestName testName; private final Context context; - private @MonotonicNonNull ListeningExecutorService glExecutorService; - private @MonotonicNonNull GlObjectsProvider glObjectsProvider; - private String testId; - @TestParameter({PACKET_PROCESSOR, DEFAULT_GL_FRAME_PROCESSOR}) - public String pipelineMode; + @Rule + public final GlFrameProcessorTestRule glFrameProcessorTestRule = new GlFrameProcessorTestRule(); + + private String testId; private static class TestConfigProvider extends TestParameterValuesProvider { @Override @@ -104,55 +83,24 @@ protected List provideValues( } } - public TransformerParameterizedPacketConsumerAndroidTest() { + public TransformerParameterizedFrameProcessorAndroidTest() { context = ApplicationProvider.getApplicationContext(); testName = new TestName(); } @Before - public void setUp() throws Exception { + public void setUp() { testId = testName.getMethodName(); - glExecutorService = - MoreExecutors.listeningDecorator(Util.newSingleThreadExecutor("PacketProcessor:Effect")); - if (pipelineMode.equals(DEFAULT_GL_FRAME_PROCESSOR)) { - glObjectsProvider = new DefaultGlObjectsProvider(); - glExecutorService - .submit( - () -> { - try { - FrameProcessorUtils.setupOpenGl(glObjectsProvider); - } catch (GlUtil.GlException e) { - throw new AssertionError(e); - } - }) - .get(TEST_TIMEOUT_MS, MILLISECONDS); - } - } - - @After - public void tearDown() { - @Nullable Exception releasingException = null; - if (pipelineMode.equals(DEFAULT_GL_FRAME_PROCESSOR)) { - releasingException = - closeTestingGlResources(glExecutorService, glObjectsProvider, TEST_TIMEOUT_MS); - } - if (glExecutorService != null) { - glExecutorService.shutdown(); - } - if (releasingException != null) { - throw new AssertionError(releasingException); - } } @Test - public void export_completesSuccessfully( + public void export_withDefaultGlFrameProcessor_completesSuccessfully( @TestParameter(valuesProvider = TestConfigProvider.class) CompositionAssetInfo testConfig) throws Exception { - String testId = "export_" + pipelineMode + "_" + testConfig; Composition composition = testConfig.getComposition(); assumeAllFormatsSupported( context, - testId, + testId + "_" + testConfig, /* inputFormats= */ testConfig.getAllVideoFormats(), /* outputFormat= */ testConfig.getVideoEncoderInputFormat()); Transformer transformer = buildTransformer(); @@ -185,18 +133,6 @@ public void export_forcesVideoTranscoding() throws Exception { } private Transformer buildTransformer() { - Transformer.Builder transformerBuilder = - new Transformer.Builder(context).setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE); - if (pipelineMode.equals(PACKET_PROCESSOR)) { - transformerBuilder.setHardwareBufferEffectsPipeline( - DefaultHardwareBufferEffectsPipeline.create(context, HardwareBufferJni.INSTANCE)); - } else if (pipelineMode.equals(DEFAULT_GL_FRAME_PROCESSOR)) { - transformerBuilder.setFrameProcessorFactory( - new DefaultGlFrameProcessor.Factory( - context, glObjectsProvider, HardwareBufferJni.INSTANCE, glExecutorService)); - } else { - throw new UnsupportedOperationException(pipelineMode); - } - return transformerBuilder.build(); + return glFrameProcessorTestRule.createTransformerBuilder(context).build(); } } diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerSequenceEffectTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerSequenceEffectTest.java index e39386e7b8c..e9fd22e2a7b 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerSequenceEffectTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/TransformerSequenceEffectTest.java @@ -17,6 +17,7 @@ package androidx.media3.transformer; +import static android.os.Build.VERSION.SDK_INT; import static androidx.media3.test.utils.AssetInfo.BT601_MOV_ASSET; import static androidx.media3.test.utils.AssetInfo.JPG_ASSET; import static androidx.media3.test.utils.AssetInfo.JPG_PORTRAIT_ASSET; @@ -27,14 +28,16 @@ import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_15S; import static androidx.media3.test.utils.AssetInfo.MP4_PORTRAIT_ASSET; import static androidx.media3.test.utils.AssetInfo.PNG_ASSET_LINES_1080P; +import static androidx.media3.test.utils.BitmapPixelTestUtil.MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE_LUMA; +import static androidx.media3.test.utils.BitmapPixelTestUtil.maybeSaveTestBitmap; import static androidx.media3.test.utils.BitmapPixelTestUtil.readBitmap; import static androidx.media3.test.utils.FormatSupportAssumptions.assumeFormatsSupported; +import static androidx.media3.test.utils.TestUtil.assertBitmapsAreSimilar; import static androidx.media3.transformer.AndroidTestUtil.extractBitmapsFromVideo; import static androidx.media3.transformer.SequenceEffectTestUtil.NO_EFFECT; import static androidx.media3.transformer.SequenceEffectTestUtil.PSNR_THRESHOLD; import static androidx.media3.transformer.SequenceEffectTestUtil.PSNR_THRESHOLD_HD; import static androidx.media3.transformer.SequenceEffectTestUtil.SINGLE_30_FPS_VIDEO_FRAME_THRESHOLD_MS; -import static androidx.media3.transformer.SequenceEffectTestUtil.assertBitmapsMatchExpectedAndSave; import static androidx.media3.transformer.SequenceEffectTestUtil.assertFramesMatchExpectedPsnrAndSave; import static androidx.media3.transformer.SequenceEffectTestUtil.clippedVideo; import static androidx.media3.transformer.SequenceEffectTestUtil.createVideoOnlyComposition; @@ -43,16 +46,22 @@ import static androidx.media3.transformer.SequenceEffectTestUtil.tryToExportCompositionWithDecoder; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assume.assumeFalse; +import static org.junit.Assume.assumeTrue; import android.content.Context; -import android.os.Build; +import android.graphics.Bitmap; +import android.net.Uri; import androidx.annotation.Nullable; import androidx.media3.common.C; import androidx.media3.common.Effect; import androidx.media3.common.Format; import androidx.media3.common.MediaItem; +import androidx.media3.common.MimeTypes; import androidx.media3.effect.BitmapOverlay; +import androidx.media3.effect.DefaultGlFrameProcessor; import androidx.media3.effect.DefaultVideoFrameProcessor; +import androidx.media3.effect.HardwareBufferJni; import androidx.media3.effect.LanczosResample; import androidx.media3.effect.OverlayEffect; import androidx.media3.effect.Presentation; @@ -61,8 +70,6 @@ import androidx.media3.exoplayer.mediacodec.MediaCodecInfo; import androidx.media3.exoplayer.mediacodec.MediaCodecSelector; import androidx.test.core.app.ApplicationProvider; -import androidx.test.ext.junit.runners.AndroidJUnit4; -import com.google.common.base.Ascii; import com.google.common.collect.ImmutableList; import java.io.File; import java.io.IOException; @@ -72,12 +79,15 @@ import org.junit.Test; import org.junit.rules.TestName; import org.junit.runner.RunWith; +import org.junit.runners.Parameterized; +import org.junit.runners.Parameterized.Parameter; +import org.junit.runners.Parameterized.Parameters; /** * Tests for using different {@linkplain Effect effects} for {@link MediaItem MediaItems} in one * {@link EditedMediaItemSequence}. */ -@RunWith(AndroidJUnit4.class) +@RunWith(Parameterized.class) public final class TransformerSequenceEffectTest { private static final String OVERLAY_PNG_ASSET_PATH = "media/png/media3test.png"; @@ -88,11 +98,37 @@ public final class TransformerSequenceEffectTest { private final Context context = ApplicationProvider.getApplicationContext(); @Rule public final TestName testName = new TestName(); + @Parameters(name = "useDefaultGlFrameProcessor={0}") + public static ImmutableList parameters() { + // When false, DefaultVideoFrameProcessor is used. + return ImmutableList.of(true, false); + } + + @Parameter public boolean useDefaultGlFrameProcessor; + private String testId; @Before - public void setUpTestId() { + public void setUp() throws Exception { testId = testName.getMethodName(); + // Remove the parameter part from the test ID to locate the correct test golden files. + int bracketIndex = testId.indexOf('['); + if (bracketIndex != -1) { + testId = testId.substring(0, bracketIndex); + } + + if (useDefaultGlFrameProcessor) { + assumeTrue(SDK_INT >= 28); + } + } + + private Transformer.Builder createTransformerBuilder() { + Transformer.Builder builder = new Transformer.Builder(context); + if (shouldUseDefaultGlFrameProcessor()) { + builder.setFrameProcessorFactory( + new DefaultGlFrameProcessor.Factory.Builder(context, HardwareBufferJni.INSTANCE).build()); + } + return builder; } @Test @@ -128,7 +164,7 @@ public void export_withNoCompositionPresentationAndWithPerMediaItemEffects() thr overlayEffect))); ExportTestResult result = - new TransformerAndroidTestRunner.Builder(context, new Transformer.Builder(context).build()) + new TransformerAndroidTestRunner.Builder(context, createTransformerBuilder().build()) .build() .run(testId, composition); @@ -333,69 +369,47 @@ public void export854x356_withAllAvailableDecoders_doesNotStretchOutputOnAny() t @Test public void export_image_samplesFromTextureCorrectly() throws Exception { - assumeFormatsSupported( - context, - testId, - /* inputFormat= */ MP4_ASSET_WITH_INCREASING_TIMESTAMPS.videoFormat, - /* outputFormat= */ MP4_ASSET_WITH_INCREASING_TIMESTAMPS.videoFormat); Composition composition = createVideoOnlyComposition( /* presentation= */ null, new EditedMediaItem.Builder( new MediaItem.Builder() .setUri(PNG_ASSET_LINES_1080P.uri) - .setImageDurationMs(C.MILLIS_PER_SECOND / 4) + // A duration of one frame at 30 fps produces exactly 1 frame. + .setImageDurationMs(C.MILLIS_PER_SECOND / 30) .build()) .setFrameRate(30) .build()); - // Some devices need a very high bitrate to avoid encoding artifacts. - int bitrate = 30_000_000; - if (Ascii.equalsIgnoreCase(Build.MODEL, "mi a2 lite") - || Ascii.equalsIgnoreCase(Build.MODEL, "redmi 8") - || Ascii.equalsIgnoreCase(Build.MODEL, "sm-f711u1") - || Ascii.equalsIgnoreCase(Build.MODEL, "sm-t870") - || Ascii.equalsIgnoreCase(Build.MODEL, "sm-f916u1") - || Ascii.equalsIgnoreCase(Build.MODEL, "sm-f926u1") - || Ascii.equalsIgnoreCase(Build.MODEL, "sm-g781n") - || Ascii.equalsIgnoreCase(Build.MODEL, "sm-g781v") - || Ascii.equalsIgnoreCase(Build.MODEL, "sm-g981u1") - || Ascii.equalsIgnoreCase(Build.MODEL, "sm-g986u1") - || Ascii.equalsIgnoreCase(Build.MODEL, "sm-n981u") - || Ascii.equalsIgnoreCase(Build.MODEL, "tb-q706") - || Ascii.equalsIgnoreCase(Build.MODEL, "moto g04") - || Ascii.equalsIgnoreCase(Build.MODEL, "moto e13") - || Ascii.equalsIgnoreCase(Build.MODEL, "rmx3760")) { - // And some devices need a lower bitrate because VideoDecodingWrapper fails to decode high - // bitrate output, or FrameworkMuxer fails to mux. - bitrate = 10_000_000; - } - Codec.EncoderFactory encoderFactory = - new DefaultEncoderFactory.Builder(context) - .setRequestedVideoEncoderSettings( - new VideoEncoderSettings.Builder().setBitrate(bitrate).build()) - .build(); + SingleFrameCapturingEncoder.Factory capturingEncoderFactory = + new SingleFrameCapturingEncoder.Factory(); Transformer transformer = - new Transformer.Builder(context) - .setEncoderFactory(new AndroidTestUtil.ForceEncodeEncoderFactory(encoderFactory)) - .setVideoMimeType("video/avc") + createTransformerBuilder() + .setEncoderFactory(capturingEncoderFactory) + .setMaxDelayBetweenMuxerSamplesMs(C.TIME_UNSET) + .setMuxerFactory( + new NoWriteMuxer.Factory( + /* audioMimeTypes= */ ImmutableList.of(MimeTypes.AUDIO_AAC), + /* videoMimeTypes= */ ImmutableList.of(MimeTypes.VIDEO_H264))) .build(); - ExportTestResult result = - new TransformerAndroidTestRunner.Builder(context, transformer) - .build() - .run(testId, composition); + new TransformerAndroidTestRunner.Builder(context, transformer).build().run(testId, composition); - assertThat(new File(result.filePath).length()).isGreaterThan(0); + Bitmap actualBitmap = capturingEncoderFactory.getLastBitmap(); + assertThat(actualBitmap).isNotNull(); + Bitmap expectedBitmap = + readBitmap( + checkNotNull(Uri.parse(PNG_ASSET_LINES_1080P.uri).getPath()).replaceFirst("^/", "")); + maybeSaveTestBitmap(testId, /* bitmapLabel= */ "0", actualBitmap, /* path= */ null); // The PSNR threshold was chosen based on: // Pixel 8 with coordinate rounding error during texture sampling, gets PSNR 23.4. // After fix -> 29.5 // rmx3563 with bug fix achieves PSNR 28.8 - assertFramesMatchExpectedPsnrAndSave( - context, - testId, - checkNotNull(result.filePath), - /* psnrThreshold= */ 28.5f, - /* frameCount= */ 2); + assertBitmapsAreSimilar( + expectedBitmap, + actualBitmap, + // TODO: b/530130453 - Lowering PSNR because DefaultGlFrameProcessor doesn't yet process + // frames in linear colors. + useDefaultGlFrameProcessor ? 24 : 28.5f); } @Test @@ -436,7 +450,7 @@ public void export_imageWithLanczosResample_completesWithHighPsnr() throws Excep new VideoEncoderSettings.Builder().setBitrate(bitrate).build()) .build(); Transformer transformer = - new Transformer.Builder(context) + createTransformerBuilder() .setEncoderFactory(new AndroidTestUtil.ForceEncodeEncoderFactory(encoderFactory)) .setVideoMimeType("video/avc") .build(); @@ -460,6 +474,7 @@ public void export_imageWithLanczosResample_completesWithHighPsnr() throws Excep @Test public void export_withCompositionPresentationAndWithPerMediaItemEffects() throws Exception { + assumeFalse("OpenGL pipeline doesn't convert color yet.", useDefaultGlFrameProcessor); assumeFormatsSupported( context, testId, @@ -500,6 +515,45 @@ public void export_withCompositionPresentationAndWithPerMediaItemEffects() throw extractBitmapsFromVideo(context, checkNotNull(result.filePath)), testId); } + @Test + public void export_withCompositionPresentationAndWithPerMediaItemEffectsLessVideo() + throws Exception { + assumeFormatsSupported( + context, + testId, + /* inputFormat= */ MP4_ADVANCED_ASSET.videoFormat, + /* outputFormat= */ MP4_ADVANCED_ASSET.videoFormat); + Composition composition = + createVideoOnlyComposition( + Presentation.createForWidthAndHeight( + /* width= */ SQUARE_SIZE, + /* height= */ SQUARE_SIZE, + Presentation.LAYOUT_SCALE_TO_FIT), + oneFrameFromImage( + JPG_ASSET.uri, + ImmutableList.of( + new ScaleAndRotateTransformation.Builder().setRotationDegrees(90).build(), + Presentation.createForWidthAndHeight( + EXPORT_WIDTH, EXPORT_HEIGHT, Presentation.LAYOUT_SCALE_TO_FIT))), + oneFrameFromImage(JPG_ASSET.uri, NO_EFFECT), + clippedVideo( + MP4_ADVANCED_ASSET.uri, + ImmutableList.of( + Presentation.createForWidthAndHeight( + EXPORT_WIDTH / 2, EXPORT_HEIGHT, Presentation.LAYOUT_SCALE_TO_FIT), + createOverlayEffect()), + SINGLE_30_FPS_VIDEO_FRAME_THRESHOLD_MS)); + + ExportTestResult result = + new TransformerAndroidTestRunner.Builder(context, getLinearColorSpaceTransformer()) + .build() + .run(testId, composition); + + assertThat(new File(result.filePath).length()).isGreaterThan(0); + assertBitmapsMatchExpectedAndSave( + extractBitmapsFromVideo(context, checkNotNull(result.filePath)), testId); + } + @Test public void export_withCompositionPresentationAndNoVideoEffects() throws Exception { assumeFormatsSupported( @@ -528,6 +582,7 @@ public void export_withCompositionPresentationAndNoVideoEffects() throws Excepti @Test public void export_withCompositionPresentationAndNoVideoEffectsForFirstMediaItem() throws Exception { + assumeFalse("OpenGL pipeline doesn't convert color yet.", useDefaultGlFrameProcessor); assumeFormatsSupported( context, testId, @@ -554,6 +609,7 @@ public void export_withCompositionPresentationAndNoVideoEffectsForFirstMediaItem @Test public void export_withBt601AndBt709MediaItems() throws Exception { + assumeFalse("OpenGL pipeline doesn't convert color yet.", useDefaultGlFrameProcessor); assumeFormatsSupported( context, testId, @@ -583,6 +639,7 @@ public void export_withBt601AndBt709MediaItems() throws Exception { @Test public void export_withBt601VideoAndBt709ImageMediaItems() throws Exception { + assumeFalse("OpenGL pipeline doesn't convert color yet.", useDefaultGlFrameProcessor); assumeFormatsSupported( context, testId, @@ -611,12 +668,14 @@ public void export_withBt601VideoAndBt709ImageMediaItems() throws Exception { private Transformer getLinearColorSpaceTransformer() { // Use linear color space for grayscale effects. - return new Transformer.Builder(context) - .setVideoFrameProcessorFactory( - new DefaultVideoFrameProcessor.Factory.Builder() - .setSdrWorkingColorSpace(DefaultVideoFrameProcessor.WORKING_COLOR_SPACE_LINEAR) - .build()) - .build(); + Transformer.Builder builder = createTransformerBuilder(); + if (!useDefaultGlFrameProcessor) { + builder.setVideoFrameProcessorFactory( + new DefaultVideoFrameProcessor.Factory.Builder() + .setSdrWorkingColorSpace(DefaultVideoFrameProcessor.WORKING_COLOR_SPACE_LINEAR) + .build()); + } + return builder.build(); } private static OverlayEffect createOverlayEffect() throws IOException { @@ -624,4 +683,18 @@ private static OverlayEffect createOverlayEffect() throws IOException { ImmutableList.of( BitmapOverlay.createStaticBitmapOverlay(readBitmap(OVERLAY_PNG_ASSET_PATH)))); } + + private void assertBitmapsMatchExpectedAndSave(List actualBitmaps, String testId) + throws IOException { + // TODO: b/530130453 - Using a higher pixel difference because DefaultGlFrameProcessor doesn't + // yet process frames in linear colors. + float maxPixelDifference = + useDefaultGlFrameProcessor ? 20.0f : MAXIMUM_AVERAGE_PIXEL_ABSOLUTE_DIFFERENCE_LUMA; + SequenceEffectTestUtil.assertBitmapsMatchExpectedAndSave( + actualBitmaps, testId, maxPixelDifference); + } + + private boolean shouldUseDefaultGlFrameProcessor() { + return useDefaultGlFrameProcessor && SDK_INT >= 28; + } } diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/CompositionPlayerPacketConsumerSurfaceViewPixelTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/CompositionPlayerFrameProcessorSurfaceViewPixelTest.java similarity index 78% rename from libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/CompositionPlayerPacketConsumerSurfaceViewPixelTest.java rename to libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/CompositionPlayerFrameProcessorSurfaceViewPixelTest.java index cf84ea20ab1..35b396c2d60 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/CompositionPlayerPacketConsumerSurfaceViewPixelTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/CompositionPlayerFrameProcessorSurfaceViewPixelTest.java @@ -26,9 +26,7 @@ import static androidx.media3.test.utils.FormatSupportAssumptions.assumeFormatsSupported; import static androidx.media3.test.utils.PlayerFence.futureWhen; import static androidx.media3.test.utils.TestUtil.assertBitmapsAreSimilar; -import static androidx.media3.transformer.GlFrameProcessorTestUtil.closeTestingGlResources; import static com.google.common.truth.Truth.assertThat; -import static java.util.concurrent.TimeUnit.MILLISECONDS; import static org.junit.Assume.assumeTrue; import android.app.Instrumentation; @@ -53,33 +51,23 @@ import androidx.media3.common.C; import androidx.media3.common.ColorInfo; import androidx.media3.common.Format; -import androidx.media3.common.GlObjectsProvider; import androidx.media3.common.MediaItem; import androidx.media3.common.Player; import androidx.media3.common.VideoSize; import androidx.media3.common.util.ConditionVariable; -import androidx.media3.common.util.GlUtil.GlException; -import androidx.media3.common.util.Util; -import androidx.media3.effect.DefaultGlFrameProcessor; -import androidx.media3.effect.DefaultGlObjectsProvider; -import androidx.media3.effect.DefaultHardwareBufferEffectsPipeline; -import androidx.media3.effect.FrameProcessorUtils; -import androidx.media3.effect.SimpleGlFrameProcessor; -import androidx.media3.effect.ndk.HardwareBufferJni; +import androidx.media3.transformer.AndroidTestUtil; import androidx.media3.transformer.Composition; import androidx.media3.transformer.CompositionPlayer; import androidx.media3.transformer.EditedMediaItem; import androidx.media3.transformer.EditedMediaItemSequence; -import androidx.media3.transformer.FrameWriterToHardwareBufferFrameQueueAdapter; -import androidx.media3.transformer.PacketConsumerToFrameProcessorAdapter; +import androidx.media3.transformer.GlFrameProcessorTestRule; import androidx.media3.transformer.SurfaceTestActivity; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.rules.ActivityScenarioRule; +import androidx.test.ext.junit.runners.AndroidJUnit4; import androidx.test.filters.SdkSuppress; import androidx.test.platform.app.InstrumentationRegistry; import com.google.common.collect.ImmutableList; -import com.google.common.util.concurrent.ListeningExecutorService; -import com.google.common.util.concurrent.MoreExecutors; import com.google.common.util.concurrent.SettableFuture; import java.util.List; import java.util.Queue; @@ -94,35 +82,19 @@ import org.junit.Test; import org.junit.rules.TestName; import org.junit.runner.RunWith; -import org.junit.runners.Parameterized; -import org.junit.runners.Parameterized.Parameter; -import org.junit.runners.Parameterized.Parameters; /** - * Pixel tests for {@link CompositionPlayer} when outputting to a {@link android.view.SurfaceView}, - * parameterized to test different effects pipeline paths. + * Pixel tests for {@link CompositionPlayer} when outputting to a {@link android.view.SurfaceView}. */ @Ignore("Only intended to run on internal infra: b/396671260") -@RunWith(Parameterized.class) -@SdkSuppress(minSdkVersion = 28) -public class CompositionPlayerPacketConsumerSurfaceViewPixelTest { - - private static final String PACKET_CONSUMER = "packet_consumer"; - private static final String FRAME_PROCESSOR_ADAPTER = "frame_processor_adapter"; - private static final String FRAME_PROCESSOR = "frame_processor"; - private static final String DEFAULT_GL_FRAME_PROCESSOR = "default_gl_frame_processor"; +@RunWith(AndroidJUnit4.class) +@SdkSuppress(minSdkVersion = AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) +public class CompositionPlayerFrameProcessorSurfaceViewPixelTest { + private static final long TEST_TIMEOUT_MS = isRunningOnEmulator() ? 20_000 : 10_000; private static final float PSNR_THRESHOLD = 23f; private static final String GOLDEN_ASSET_FOLDER_PATH = - "test-generated-goldens/CompositionPlayerPacketConsumerSurfaceViewPixelTest/"; - - @Parameters(name = "{0}") - public static ImmutableList params() { - return ImmutableList.of( - PACKET_CONSUMER, FRAME_PROCESSOR_ADAPTER, FRAME_PROCESSOR, DEFAULT_GL_FRAME_PROCESSOR); - } - - @Parameter public String mode; + "test-generated-goldens/CompositionPlayerFrameProcessorSurfaceViewPixelTest/"; @Rule public ActivityScenarioRule rule = @@ -136,9 +108,9 @@ public static ImmutableList params() { private @MonotonicNonNull CompositionPlayer compositionPlayer; private @MonotonicNonNull SurfaceView surfaceView; private @MonotonicNonNull ImageReaderSurfaceHolder surfaceHolder; - private @MonotonicNonNull ListeningExecutorService glExecutorService; - // Only used for DefaultGlFrameProcessor - private @MonotonicNonNull GlObjectsProvider glObjectsProvider; + + @Rule + public final GlFrameProcessorTestRule glFrameProcessorTestRule = new GlFrameProcessorTestRule(); private String testId; @@ -146,21 +118,6 @@ public static ImmutableList params() { public void setUp() throws Exception { testId = testName.getMethodName(); rule.getScenario().onActivity(activity -> surfaceView = activity.getSurfaceView()); - glExecutorService = - MoreExecutors.listeningDecorator(Util.newSingleThreadExecutor("PacketProcessor:Effect")); - glObjectsProvider = new DefaultGlObjectsProvider(); - if (mode.equals(DEFAULT_GL_FRAME_PROCESSOR)) { - glExecutorService - .submit( - () -> { - try { - FrameProcessorUtils.setupOpenGl(glObjectsProvider); - } catch (GlException e) { - throw new AssertionError(e); - } - }) - .get(TEST_TIMEOUT_MS, MILLISECONDS); - } } @After @@ -175,17 +132,6 @@ public void tearDown() { if (surfaceHolder != null) { surfaceHolder.release(); } - @Nullable Exception releasingException = null; - if (mode.equals(DEFAULT_GL_FRAME_PROCESSOR)) { - releasingException = - closeTestingGlResources(glExecutorService, glObjectsProvider, TEST_TIMEOUT_MS); - } - if (glExecutorService != null) { - glExecutorService.shutdown(); - } - if (releasingException != null) { - throw new AssertionError(releasingException); - } } @Test @@ -195,7 +141,7 @@ public void compositionPlayer_reportsVideoSizeChanged() throws InterruptedExcept instrumentation.runOnMainSync( () -> { - compositionPlayer = createCompositionPlayerBuilder(context, mode).build(); + compositionPlayer = createCompositionPlayerBuilder(context).build(); compositionPlayer.setVideoSurfaceView(surfaceView); compositionPlayer.addListener( new Player.Listener() { @@ -230,13 +176,13 @@ public void onVideoSizeChanged(VideoSize videoSize) { } @Test - public void compositionPlayer_withPacketConsumer_rendersFirstFrameAndReturnsBitmap() + public void compositionPlayer_withFrameProcessor_rendersFirstFrameAndReturnsBitmap() throws Exception { SettableFuture firstFrameRenderedFuture = SettableFuture.create(); instrumentation.runOnMainSync( () -> { - compositionPlayer = createCompositionPlayerBuilder(context, mode).build(); + compositionPlayer = createCompositionPlayerBuilder(context).build(); compositionPlayer.setVideoSurfaceView(surfaceView); firstFrameRenderedFuture.setFuture(futureWhen(compositionPlayer).rendersFirstFrame()); compositionPlayer.setComposition( @@ -275,14 +221,14 @@ public void compositionPlayer_withPacketConsumer_rendersFirstFrameAndReturnsBitm Bitmap expectedBitmap = readBitmap( /* assetString= */ GOLDEN_ASSET_FOLDER_PATH - + "compositionPlayer_withPacketConsumer_rendersFirstFrameAndReturnsBitmap.png"); + + "compositionPlayer_withFrameProcessor_rendersFirstFrameAndReturnsBitmap.png"); maybeSaveTestBitmap(testId, "firstFrame", bitmap, /* path= */ null); assertBitmapsAreSimilar(expectedBitmap, bitmap, PSNR_THRESHOLD); } @Test - public void compositionPlayer_withPacketConsumer_backsUpAndRestoresFrameOnLifecycleChange() + public void compositionPlayer_withFrameProcessor_backsUpAndRestoresFrameOnLifecycleChange() throws Exception { SettableFuture firstFrameRenderedFuture = SettableFuture.create(); // rule.getScenario().moveToState(RESUMED); hangs if the device setup is not completed. @@ -309,13 +255,7 @@ public void surfaceDestroyed(SurfaceHolder holder) { instrumentation.runOnMainSync( () -> { surfaceView.getHolder().addCallback(callback); - DefaultHardwareBufferEffectsPipeline packetProcessor = - DefaultHardwareBufferEffectsPipeline.create(context, HardwareBufferJni.INSTANCE); - compositionPlayer = - new CompositionPlayer.Builder(context) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setHardwareBufferEffectsPipeline(packetProcessor) - .build(); + compositionPlayer = createCompositionPlayerBuilder(context).build(); firstFrameRenderedFuture.setFuture(futureWhen(compositionPlayer).rendersFirstFrame()); compositionPlayer.setVideoSurfaceView(surfaceView); compositionPlayer.setComposition( @@ -370,26 +310,108 @@ public void surfaceDestroyed(SurfaceHolder holder) { Bitmap expectedBitmap = readBitmap( /* assetString= */ GOLDEN_ASSET_FOLDER_PATH - + "compositionPlayer_withPacketConsumer_rendersFirstFrameAndReturnsBitmap.png"); + + "compositionPlayer_withFrameProcessor_rendersFirstFrameAndReturnsBitmap.png"); maybeSaveTestBitmap(testId, "restoredFrame", bitmap, /* path= */ null); assertBitmapsAreSimilar(expectedBitmap, bitmap, PSNR_THRESHOLD); } + @Ignore("b/517525358 - HDR10 input not supported yet") @Test - public void compositionPlayer_withPacketConsumer_usesMetadataListener() throws Exception { + @SdkSuppress( + minSdkVersion = 34) // RGBA_1010102 only supported in ImageReader/SurfaceView from API 34. + public void compositionPlayer_withFrameProcessor_hdr_backsUpAndRestoresFrameOnLifecycleChange() + throws Exception { + assumeTrue(isDeviceReady()); + assumeFormatsSupported( + context, + testId, + /* inputFormat= */ MP4_ASSET_COLOR_TEST_1080P_HLG10.videoFormat, + /* outputFormat= */ null); + SettableFuture firstFrameRenderedFuture = SettableFuture.create(); + ConditionVariable surfaceDestroyed = new ConditionVariable(); + ConditionVariable surfaceChanged = new ConditionVariable(); + + SurfaceHolder.Callback callback = + new SurfaceHolder.Callback() { + @Override + public void surfaceCreated(SurfaceHolder holder) {} + + @Override + public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) { + surfaceChanged.open(); + } + + @Override + public void surfaceDestroyed(SurfaceHolder holder) { + surfaceDestroyed.open(); + } + }; + + instrumentation.runOnMainSync( + () -> { + surfaceView.getHolder().addCallback(callback); + compositionPlayer = createCompositionPlayerBuilder(context).build(); + firstFrameRenderedFuture.setFuture(futureWhen(compositionPlayer).rendersFirstFrame()); + compositionPlayer.setVideoSurfaceView(surfaceView); + compositionPlayer.setComposition( + new Composition.Builder( + EditedMediaItemSequence.withVideoFrom( + ImmutableList.of( + new EditedMediaItem.Builder( + MediaItem.fromUri(MP4_ASSET_COLOR_TEST_1080P_HLG10.uri)) + .setDurationUs(MP4_ASSET_COLOR_TEST_1080P_HLG10.videoDurationUs) + .build()))) + .build()); + compositionPlayer.prepare(); + compositionPlayer.setPlayWhenReady(false); + }); + + firstFrameRenderedFuture.get(); + + // Move activity to stopped state (destroys surface, triggers backup). + rule.getScenario().moveToState(CREATED); + + // Wait for surface destruction to complete. + assertThat(surfaceDestroyed.block(TEST_TIMEOUT_MS)).isTrue(); + + // Reset the condition variable to wait for the next recreation. + surfaceChanged.close(); + + // Move activity back to resumed state (re-creates surface, triggers restore). + rule.getScenario().moveToState(RESUMED); + + // Wait for surface re-creation. + assertThat(surfaceChanged.block(TEST_TIMEOUT_MS)).isTrue(); + + Bitmap bitmap = Bitmap.createBitmap(/* width= */ 1920, /* height= */ 1080, Config.RGBA_1010102); + ConditionVariable pixelCopyFinished = new ConditionVariable(); + + instrumentation.runOnMainSync( + () -> + PixelCopy.request( + surfaceView, + bitmap, + result -> { + if (result == PixelCopy.SUCCESS) { + pixelCopyFinished.open(); + } + }, + surfaceView.getHandler())); + assertThat(pixelCopyFinished.block(TEST_TIMEOUT_MS)).isTrue(); + assertThat(isBitmapBlackOrEmpty(bitmap)).isFalse(); + } + + @Test + public void compositionPlayer_withFrameProcessor_usesMetadataListener() throws Exception { SettableFuture endedFuture = SettableFuture.create(); Queue videoTimestamps = new ConcurrentLinkedQueue<>(); AtomicReference formatAtomicReference = new AtomicReference<>(); instrumentation.runOnMainSync( () -> { - DefaultHardwareBufferEffectsPipeline packetProcessor = - DefaultHardwareBufferEffectsPipeline.create(context, HardwareBufferJni.INSTANCE); compositionPlayer = - new CompositionPlayer.Builder(context) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setHardwareBufferEffectsPipeline(packetProcessor) + createCompositionPlayerBuilder(context) .experimentalSetLateThresholdToDropInputUs(C.TIME_UNSET) .build(); compositionPlayer.setVideoSurfaceView(surfaceView); @@ -436,27 +458,20 @@ public void compositionPlayer_withPacketConsumer_usesMetadataListener() throws E .inOrder(); assertThat(formatAtomicReference.get().width).isEqualTo(320); assertThat(formatAtomicReference.get().height).isEqualTo(240); - assertThat(formatAtomicReference.get().colorInfo) - .isEqualTo( - new ColorInfo.Builder() - .setColorRange(C.COLOR_RANGE_FULL) - .setColorSpace(C.COLOR_SPACE_BT601) - .setColorTransfer(C.COLOR_TRANSFER_SDR) - .setChromaBitdepth(8) - .setLumaBitdepth(8) - .build()); + ColorInfo expectedColorInfo = ColorInfo.SDR_BT709_LIMITED; + assertThat(formatAtomicReference.get().colorInfo).isEqualTo(expectedColorInfo); } @Test @SdkSuppress(minSdkVersion = 33) - public void compositionPlayer_withPacketConsumer_andSdrVideo_outputsCorrectDataSpace() + public void compositionPlayer_withFrameProcessor_andSdrVideo_outputsCorrectDataSpace() throws Exception { SettableFuture firstFrameRenderedFuture = SettableFuture.create(); surfaceHolder = new ImageReaderSurfaceHolder(); instrumentation.runOnMainSync( () -> { - compositionPlayer = createCompositionPlayerBuilder(context, mode).build(); + compositionPlayer = createCompositionPlayerBuilder(context).build(); compositionPlayer.setVideoSurfaceHolder(surfaceHolder); firstFrameRenderedFuture.setFuture(futureWhen(compositionPlayer).rendersFirstFrame()); compositionPlayer.setComposition( @@ -477,11 +492,12 @@ public void compositionPlayer_withPacketConsumer_andSdrVideo_outputsCorrectDataS firstFrameRenderedFuture.get(); int actualDataSpace = surfaceHolder.getLatestDataSpace(); - assertThat(DataSpace.getStandard(actualDataSpace)).isEqualTo(DataSpace.STANDARD_BT601_625); + assertThat(DataSpace.getStandard(actualDataSpace)).isEqualTo(DataSpace.STANDARD_BT709); assertThat(DataSpace.getTransfer(actualDataSpace)).isEqualTo(DataSpace.TRANSFER_SMPTE_170M); - assertThat(DataSpace.getRange(actualDataSpace)).isEqualTo(DataSpace.RANGE_FULL); + assertThat(DataSpace.getRange(actualDataSpace)).isEqualTo(DataSpace.RANGE_LIMITED); } + @Ignore("b/517525358 - HDR10 input not supported yet") @Test @SdkSuppress(minSdkVersion = 34) // RGBA_1010102 only supported in ImageReader from API 34. public void compositionPlayer_andHdrVideo_outputsCorrectDataSpace() throws Exception { @@ -495,7 +511,7 @@ public void compositionPlayer_andHdrVideo_outputsCorrectDataSpace() throws Excep instrumentation.runOnMainSync( () -> { - compositionPlayer = createCompositionPlayerBuilder(context, mode).build(); + compositionPlayer = createCompositionPlayerBuilder(context).build(); compositionPlayer.setVideoSurfaceHolder(surfaceHolder); firstFrameRenderedFuture.setFuture(futureWhen(compositionPlayer).rendersFirstFrame()); compositionPlayer.setComposition( @@ -518,30 +534,10 @@ public void compositionPlayer_andHdrVideo_outputsCorrectDataSpace() throws Excep assertThat(DataSpace.getRange(actualDataSpace)).isEqualTo(DataSpace.RANGE_LIMITED); } - private CompositionPlayer.Builder createCompositionPlayerBuilder(Context context, String mode) { - CompositionPlayer.Builder builder = - new CompositionPlayer.Builder(context) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE); - if (mode.equals(PACKET_CONSUMER)) { - return builder.setHardwareBufferEffectsPipeline( - DefaultHardwareBufferEffectsPipeline.create(context, HardwareBufferJni.INSTANCE)); - } else if (mode.equals(FRAME_PROCESSOR_ADAPTER)) { - DefaultHardwareBufferEffectsPipeline pipeline = - DefaultHardwareBufferEffectsPipeline.create(context, HardwareBufferJni.INSTANCE); - return builder.setFrameProcessorFactory( - (output, listenerExecutor, listener) -> { - pipeline.setRenderOutput(new FrameWriterToHardwareBufferFrameQueueAdapter(output)); - return new PacketConsumerToFrameProcessorAdapter(pipeline, listenerExecutor, listener); - }); - } else if (mode.equals(FRAME_PROCESSOR)) { - return builder.setFrameProcessorFactory( - new SimpleGlFrameProcessor.Factory(context, HardwareBufferJni.INSTANCE)); - } else if (mode.equals(DEFAULT_GL_FRAME_PROCESSOR)) { - return builder.setFrameProcessorFactory( - new DefaultGlFrameProcessor.Factory( - context, glObjectsProvider, HardwareBufferJni.INSTANCE, glExecutorService)); - } - throw new IllegalArgumentException("Unknown mode: " + mode); + private CompositionPlayer.Builder createCompositionPlayerBuilder(Context context) { + return new CompositionPlayer.Builder(context) + .setFrameProcessorFactory( + glFrameProcessorTestRule.createDefaultGlFrameProcessorFactory(context)); } private static boolean isDeviceReady() { @@ -554,6 +550,19 @@ private static boolean isDeviceReady() { return deviceProvisioned && userSetupComplete; } + private static boolean isBitmapBlackOrEmpty(Bitmap bitmap) { + int width = bitmap.getWidth(); + int height = bitmap.getHeight(); + for (int x = 0; x < width; x += 10) { + for (int y = 0; y < height; y += 10) { + if ((bitmap.getPixel(x, y) & 0xFFFFFF) != 0) { + return false; + } + } + } + return true; + } + /** An implementation of {@link SurfaceHolder} which is backed by an {@link ImageReader}. */ private static final class ImageReaderSurfaceHolder implements SurfaceHolder { private final List callbacks = new CopyOnWriteArrayList<>(); diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/ExportTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/ExportTest.java index 19a0f49920d..e2335449f2f 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/ExportTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/ExportTest.java @@ -59,9 +59,12 @@ import androidx.media3.transformer.VideoEncoderSettings; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; +import androidx.test.platform.app.InstrumentationRegistry; +import androidx.test.uiautomator.UiDevice; import com.google.common.base.Ascii; import com.google.common.collect.ImmutableList; import java.io.File; +import java.io.IOException; import org.junit.AssumptionViolatedException; import org.junit.Before; import org.junit.Ignore; @@ -189,17 +192,22 @@ public void export4K60() throws Exception { public void export8K24() throws Exception { // Reference: b/244711282#comment5 assumeFalse( - "Some devices are capable of instantiating only either one 8K decoder or one 8K encoder", - Ascii.equalsIgnoreCase(Build.MODEL, "tb-q706") - || Ascii.equalsIgnoreCase(Build.MODEL, "sm-f916u1") - || Ascii.equalsIgnoreCase(Build.MODEL, "sm-g981u1") - || Ascii.equalsIgnoreCase(Build.MODEL, "sm-g781n") - || Ascii.equalsIgnoreCase(Build.MODEL, "sm-g781v") - || Ascii.equalsIgnoreCase(Build.MODEL, "sm-g986u1") - || Ascii.equalsIgnoreCase(Build.MODEL, "sm-n981u") - || Ascii.equalsIgnoreCase(Build.MODEL, "le2121") - || Ascii.equalsIgnoreCase(Build.MODEL, "seahawk") - || Ascii.equalsIgnoreCase(Build.MODEL, "sm-t870")); + "Some SoCs are unable to create two concurrent 8k codecs.", + SDK_INT >= 31 + && Ascii.equalsIgnoreCase(Build.SOC_MANUFACTURER, "qti") + && Ascii.toLowerCase(Build.SOC_MODEL).contains("sm8250")); + + // On API 31- there is no standardized way to access SoC information, so we need to rely on + // OEM-specific workarounds. In this case, ro.hardware.chipname is specific to Samsung devices. + String socManufacturer = getSysProp("ro.hardware"); + String socModel = getSysProp("ro.hardware.chipname"); + + assumeFalse( + "Some SoCs are unable to create two concurrent 8k codecs.", + SDK_INT < 31 + && Ascii.equalsIgnoreCase(socManufacturer, "qcom") + && Ascii.equalsIgnoreCase(socModel, "sm8250")); + assumeFormatsSupported( context, testId, /* inputFormat= */ MP4_ASSET_8K24.videoFormat, /* outputFormat= */ null); Transformer transformer = @@ -423,4 +431,9 @@ public void export_setEncodingProfileLevel_changesProfileAndLevel() throws Excep assertThat(profileAndLevel.first).isAtMost(AVCProfileHigh); assertThat(profileAndLevel.second).isAtMost(AVCLevel41); } + + private static String getSysProp(String name) throws IOException { + UiDevice uiDevice = UiDevice.getInstance(InstrumentationRegistry.getInstrumentation()); + return uiDevice.executeShellCommand("getprop " + name).trim(); + } } diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/PlaybackVideoGraphWrapperTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/PlaybackVideoGraphWrapperTest.java index 4785b7bc006..34365981314 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/PlaybackVideoGraphWrapperTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/PlaybackVideoGraphWrapperTest.java @@ -172,6 +172,9 @@ public boolean shouldIgnoreFrame( } }; return new VideoFrameReleaseControl( - context, frameTimingEvaluator, /* allowedJoiningTimeMs= */ 0); + context, + frameTimingEvaluator, + /* allowedJoiningTimeMs= */ 0, + /* skipBuffersWithIdenticalReleaseTime= */ true); } } diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/TranscodeForegroundSpeedTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/TranscodeForegroundSpeedTest.java index da727839ed1..764d50fb394 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/TranscodeForegroundSpeedTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/TranscodeForegroundSpeedTest.java @@ -148,10 +148,7 @@ private static ExportTestResult exportVideoAndAudioTo720pWithDynamicScheduling( String testId, Uri mediaUri, long durationMs) throws Exception { Context context = ApplicationProvider.getApplicationContext(); Codec.DecoderFactory decoderFactory = - new DefaultDecoderFactory.Builder(context) - .experimentalSetDynamicSchedulingEnabled(true) - .setShouldConfigureOperatingRate(true) - .build(); + new DefaultDecoderFactory.Builder(context).setShouldConfigureOperatingRate(true).build(); AssetLoader.Factory assetLoaderFactory = new DefaultAssetLoaderFactory( context, decoderFactory, Clock.DEFAULT, /* logSessionId= */ null); diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/TranscodeQualityTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/TranscodeQualityTest.java index ecd93d1804f..cf51646d728 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/TranscodeQualityTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/TranscodeQualityTest.java @@ -17,7 +17,6 @@ package androidx.media3.transformer.mh; import static android.os.Build.VERSION.SDK_INT; -import static androidx.media3.common.util.Util.isRunningOnEmulator; import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_COLOR_TEST_1080P_HLG10; import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_WITH_INCREASING_TIMESTAMPS; import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_15S; @@ -28,10 +27,8 @@ import static androidx.media3.test.utils.HdrCapabilitiesUtil.assumeDeviceSupportsHdrEditing; import static androidx.media3.test.utils.TestUtil.assertBitmapsAreSimilar; import static androidx.media3.test.utils.TestUtil.retrieveTrackFormat; -import static androidx.media3.transformer.GlFrameProcessorTestUtil.closeTestingGlResources; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.truth.Truth.assertThat; -import static java.util.concurrent.TimeUnit.MILLISECONDS; import static java.util.concurrent.TimeUnit.SECONDS; import static org.junit.Assume.assumeFalse; @@ -39,39 +36,26 @@ import android.graphics.Bitmap; import android.net.Uri; import android.os.Build; -import androidx.annotation.Nullable; import androidx.media3.common.C; -import androidx.media3.common.GlObjectsProvider; import androidx.media3.common.MediaItem; import androidx.media3.common.MimeTypes; -import androidx.media3.common.util.GlUtil.GlException; import androidx.media3.common.util.Log; import androidx.media3.common.util.Util; -import androidx.media3.effect.DefaultGlFrameProcessor; -import androidx.media3.effect.DefaultGlObjectsProvider; -import androidx.media3.effect.DefaultHardwareBufferEffectsPipeline; -import androidx.media3.effect.FrameProcessorUtils; -import androidx.media3.effect.SimpleGlFrameProcessor; -import androidx.media3.effect.ndk.HardwareBufferJni; import androidx.media3.inspector.frame.FrameExtractor; +import androidx.media3.transformer.AndroidTestUtil; import androidx.media3.transformer.AndroidTestUtil.ForceEncodeEncoderFactory; import androidx.media3.transformer.EditedMediaItem; import androidx.media3.transformer.ExportTestResult; -import androidx.media3.transformer.FrameWriterToHardwareBufferFrameQueueAdapter; -import androidx.media3.transformer.PacketConsumerToFrameProcessorAdapter; +import androidx.media3.transformer.GlFrameProcessorTestRule; import androidx.media3.transformer.Transformer; import androidx.media3.transformer.TransformerAndroidTestRunner; import androidx.test.core.app.ApplicationProvider; import androidx.test.filters.SdkSuppress; import com.google.common.collect.ImmutableList; -import com.google.common.util.concurrent.ListeningExecutorService; -import com.google.common.util.concurrent.MoreExecutors; import java.io.OutputStream; import java.lang.reflect.Method; import java.nio.file.Files; import java.nio.file.Paths; -import org.checkerframework.checker.nullness.qual.MonotonicNonNull; -import org.junit.After; import org.junit.Before; import org.junit.Ignore; import org.junit.Rule; @@ -93,69 +77,29 @@ public final class TranscodeQualityTest { "test-generated-goldens/FrameExtractorTest/hlg10-color-test_0.000.png"; private static final String EXPECTED_ROTATED_VIDEO_EXTRACTED_PNG_PATH = "test-generated-goldens/FrameExtractorTest/internal_emulator_transformer_output_270_rotated_0.000.png"; - private static final long TEST_TIMEOUT_MS = isRunningOnEmulator() ? 20_000 : 10_000; - private static final String LEGACY = "legacy"; - private static final String PACKET_CONSUMER_NDK = "packet_consumer_ndk"; - private static final String FRAME_PROCESSOR_ADAPTER_NDK = "frame_processor_adapter_ndk"; - private static final String FRAME_PROCESSOR_NDK = "frame_processor_ndk"; private static final String DEFAULT_GL_FRAME_PROCESSOR_NDK = "default_gl_frame_processor_ndk"; - private static final int FRAME_PROCESSOR_MIN_SDK = 28; @Rule public final TestName testName = new TestName(); @Parameters(name = "{0}") public static ImmutableList params() { - if (SDK_INT >= FRAME_PROCESSOR_MIN_SDK) { - return ImmutableList.of( - LEGACY, - PACKET_CONSUMER_NDK, - FRAME_PROCESSOR_ADAPTER_NDK, - FRAME_PROCESSOR_NDK, - DEFAULT_GL_FRAME_PROCESSOR_NDK); + if (SDK_INT >= AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) { + return ImmutableList.of(LEGACY, DEFAULT_GL_FRAME_PROCESSOR_NDK); } return ImmutableList.of(LEGACY); } @Parameter public String mode; - private @MonotonicNonNull ListeningExecutorService glExecutorService; - private @MonotonicNonNull GlObjectsProvider glObjectsProvider; + @Rule + public final GlFrameProcessorTestRule glFrameProcessorTestRule = new GlFrameProcessorTestRule(); + private String testId; @Before public void setUp() throws Exception { testId = testName.getMethodName(); - glExecutorService = - MoreExecutors.listeningDecorator(Util.newSingleThreadExecutor("PacketProcessor:Effect")); - if (mode.equals(DEFAULT_GL_FRAME_PROCESSOR_NDK)) { - glObjectsProvider = new DefaultGlObjectsProvider(); - glExecutorService - .submit( - () -> { - try { - FrameProcessorUtils.setupOpenGl(checkNotNull(glObjectsProvider)); - } catch (GlException e) { - throw new AssertionError(e); - } - }) - .get(TEST_TIMEOUT_MS, MILLISECONDS); - } - } - - @After - public void tearDown() { - @Nullable Exception releasingException = null; - if (mode.equals(DEFAULT_GL_FRAME_PROCESSOR_NDK)) { - releasingException = - closeTestingGlResources(glExecutorService, glObjectsProvider, TEST_TIMEOUT_MS); - } - if (glExecutorService != null) { - glExecutorService.shutdown(); - } - if (releasingException != null) { - throw new AssertionError(releasingException); - } } @Test @@ -222,12 +166,8 @@ public void transcodeAvcToAvc320x240_ssimIsGreaterThan90Percent() throws Excepti @SdkSuppress(minSdkVersion = 34) // HDR Bitmap extraction requires API 34+. public void transcode_hlg10_outputsHlg() throws Exception { Context context = ApplicationProvider.getApplicationContext(); - // TODO: b/286211012 - Enable once DefaultHardwareBufferEffectsPipeline supports HDR. - assumeFalse( - mode.equals(PACKET_CONSUMER_NDK) - || mode.equals(FRAME_PROCESSOR_ADAPTER_NDK) - || mode.equals(FRAME_PROCESSOR_NDK) - || mode.equals(DEFAULT_GL_FRAME_PROCESSOR_NDK)); + // TODO: b/286211012 - Enable once DefaultGlFrameProcessor supports HDR. + assumeFalse(mode.equals(DEFAULT_GL_FRAME_PROCESSOR_NDK)); assumeDeviceSupportsHdrEditing(testId, MP4_ASSET_COLOR_TEST_1080P_HLG10.videoFormat); assumeFormatsSupported( context, @@ -296,39 +236,11 @@ public void transcode_rotated270_outputsRotated270() throws Exception { } private Transformer.Builder createBuilder(Context context, String mode) { - if (SDK_INT < FRAME_PROCESSOR_MIN_SDK) { + if (SDK_INT < AndroidTestUtil.HARDWARE_BUFFER_FRAME_PROCESSOR_MIN_SDK) { return new Transformer.Builder(context); } - if (mode.equals(PACKET_CONSUMER_NDK)) { - return new Transformer.Builder(context) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setHardwareBufferEffectsPipeline( - DefaultHardwareBufferEffectsPipeline.create(context, HardwareBufferJni.INSTANCE)); - } else if (mode.equals(FRAME_PROCESSOR_ADAPTER_NDK)) { - DefaultHardwareBufferEffectsPipeline pipeline = - DefaultHardwareBufferEffectsPipeline.create(context, HardwareBufferJni.INSTANCE); - return new Transformer.Builder(context) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setFrameProcessorFactory( - (output, listenerExecutor, listener) -> { - pipeline.setRenderOutput(new FrameWriterToHardwareBufferFrameQueueAdapter(output)); - return new PacketConsumerToFrameProcessorAdapter( - pipeline, listenerExecutor, listener); - }); - } else if (mode.equals(FRAME_PROCESSOR_NDK)) { - return new Transformer.Builder(context) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setFrameProcessorFactory( - new SimpleGlFrameProcessor.Factory(context, HardwareBufferJni.INSTANCE)); - } else if (mode.equals(DEFAULT_GL_FRAME_PROCESSOR_NDK)) { - return new Transformer.Builder(context) - .setNativeHardwareBufferHelpers(HardwareBufferJni.INSTANCE) - .setFrameProcessorFactory( - new DefaultGlFrameProcessor.Factory( - context, - checkNotNull(glObjectsProvider), - HardwareBufferJni.INSTANCE, - checkNotNull(glExecutorService))); + if (mode.equals(DEFAULT_GL_FRAME_PROCESSOR_NDK)) { + return glFrameProcessorTestRule.createTransformerBuilder(context); } return new Transformer.Builder(context); } diff --git a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/analysis/ExoPlayerFrameDropAnalysisTest.java b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/analysis/ExoPlayerFrameDropAnalysisTest.java index 286c397b9dc..4e6829bdc52 100644 --- a/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/analysis/ExoPlayerFrameDropAnalysisTest.java +++ b/libraries/transformer/src/androidTest/java/androidx/media3/transformer/mh/analysis/ExoPlayerFrameDropAnalysisTest.java @@ -23,7 +23,6 @@ import android.content.Context; import android.net.Uri; import android.view.SurfaceView; -import androidx.annotation.RequiresApi; import androidx.media3.common.C; import androidx.media3.common.MediaItem; import androidx.media3.common.Player; @@ -84,7 +83,6 @@ public class ExoPlayerFrameDropAnalysisTest { public TestConfig testConfig; @Parameters(name = "{0}") - @RequiresApi(24) public static List parameters() { return Sets.cartesianProduct( INPUT_ASSETS, @@ -151,8 +149,6 @@ public void onVideoDecoderInitialized( new ExoPlayer.Builder( context, new DefaultRenderersFactory(context) - .experimentalSetParseAv1SampleDependencies( - /* parseAv1SampleDependencies= */ testConfig.lateThresholdUs != 0) .experimentalSetLateThresholdToDropDecoderInputUs( testConfig.lateThresholdUs)) .setClock(new JumpingClock()) diff --git a/libraries/transformer/src/androidTest/res/layout/surface_test_activity.xml b/libraries/transformer/src/androidTest/res/layout/surface_test_activity.xml index d93d55364ff..951e404b4ba 100644 --- a/libraries/transformer/src/androidTest/res/layout/surface_test_activity.xml +++ b/libraries/transformer/src/androidTest/res/layout/surface_test_activity.xml @@ -18,7 +18,8 @@ xmlns:android="http://schemas.android.com/apk/res/android" android:id="@+id/root_view_group" android:layout_width="match_parent" - android:layout_height="match_parent"> + android:layout_height="match_parent" + android:fitsSystemWindows="true"> This processor caches the underlying {@link android.hardware.HardwareBuffer} as long as the - * input {@link Bitmap} remains the same (verified via {@link Bitmap#getGenerationId()}). It uses - * JNI to copy pixels from the bitmap to the hardware buffer. + *

    This converter caches the underlying {@link HardwareBuffer} as long as the input {@link + * Bitmap} remains the same (verified via {@link Bitmap#getGenerationId()}). It uses JNI to copy + * pixels from the bitmap to the hardware buffer. * - *

    The processor manages the lifecycle of the hardware buffer using reference counting, ensuring - * it is only closed once all consumer frames and the processor itself have released their + *

    The converter manages the lifecycle of the hardware buffer using reference counting, ensuring + * it is only closed once all consumer frames and the converter itself have released their * references. */ @RequiresApi(26) -@ExperimentalApi // TODO: b/479415385 - remove when packet consumer is production-ready. -public class BitmapToHardwareBufferProcessor implements HardwareBufferFrameProcessor { +/* package */ final class BitmapToHardwareBufferConverter implements AutoCloseable { private static final Duration RELEASE_TIMEOUT = Duration.ofMillis(500); @@ -61,7 +62,7 @@ public class BitmapToHardwareBufferProcessor implements HardwareBufferFrameProce @GuardedBy("this") @Nullable - private HardwareBufferFrame currentFrame; + private DefaultHardwareBufferFrame currentFrame; @GuardedBy("this") @Nullable @@ -82,7 +83,7 @@ public class BitmapToHardwareBufferProcessor implements HardwareBufferFrameProce * @param errorCallback The {@link Consumer} called when waiting or * closing the created {@link HardwareBuffer}s fails. */ - public BitmapToHardwareBufferProcessor( + /* package */ BitmapToHardwareBufferConverter( HardwareBufferJniWrapper hardwareBufferJniWrapper, ExecutorService internalExecutor, Executor errorExecutor, @@ -93,16 +94,14 @@ public BitmapToHardwareBufferProcessor( this.errorCallback = errorCallback; } - @Override - public HardwareBufferFrame process(HardwareBufferFrame inputFrame) { - if (inputFrame.hardwareBuffer != null || !(inputFrame.internalFrame instanceof Bitmap)) { - return inputFrame; - } - Bitmap nextBitmap = checkNotNull((Bitmap) inputFrame.internalFrame); - HardwareBufferFrame outputFrame; - + /** + * Returns a retained {@link DefaultHardwareBufferFrame} containing the {@link HardwareBuffer} for + * the given {@link Bitmap}. + * + *

    The caller must release the returned frame when finished with it. + */ + /* package */ DefaultHardwareBufferFrame getOrCreateRetainedFrame(Bitmap nextBitmap) { synchronized (this) { - // process should not be called after close. checkState(!internalExecutor.isShutdown()); // Check whether the current bitmap should be updated. if (currentFrame != null @@ -111,20 +110,18 @@ public HardwareBufferFrame process(HardwareBufferFrame inputFrame) { // Release this reference, the buffer will be cleaned up once other references are released. checkNotNull(currentFrame).release(/* releaseFence= */ null); currentFrame = null; + currentBitmap = null; + currentBitmapGenerationId = C.INDEX_UNSET; } if (currentFrame == null) { HardwareBuffer buffer = null; + boolean bitmapOwnsBuffer = false; try { - if (SDK_INT >= 31) { - if (nextBitmap.getConfig() == Config.HARDWARE) { - // Input is HARDWARE and API >= 31: Direct access to HardwareBuffer is possible. - buffer = nextBitmap.getHardwareBuffer(); - } else { - // Input is not HARDWARE and API >= 31: We can create a HARDWARE Bitmap copy - // and get its HardwareBuffer. - buffer = nextBitmap.copy(Config.HARDWARE, /* isMutable= */ false).getHardwareBuffer(); - } + if (SDK_INT >= 31 && nextBitmap.getConfig() == Config.HARDWARE) { + // Input is HARDWARE and API >= 31: Direct access to HardwareBuffer is possible. + buffer = nextBitmap.getHardwareBuffer(); + bitmapOwnsBuffer = buffer != null; } // Fallback to the native helper when the HardwareBuffer retrieved from the Bitmap was // null, or SDK_INT < 31. @@ -134,7 +131,11 @@ public HardwareBufferFrame process(HardwareBufferFrame inputFrame) { // We must first copy to a software Bitmap (e.g. ARGB_8888) // and then copy that to a new HardwareBuffer via JNI. Bitmap softwareBitmap = nextBitmap.copy(Config.ARGB_8888, /* isMutable= */ false); - buffer = copyCpuBitmapToHardwareBuffer(softwareBitmap, hardwareBufferJniWrapper); + try { + buffer = copyCpuBitmapToHardwareBuffer(softwareBitmap, hardwareBufferJniWrapper); + } finally { + softwareBitmap.recycle(); + } } else { // Input is not HARDWARE: Copy the software Bitmap to a new HardwareBuffer via JNI. buffer = copyCpuBitmapToHardwareBuffer(nextBitmap, hardwareBufferJniWrapper); @@ -143,49 +144,50 @@ public HardwareBufferFrame process(HardwareBufferFrame inputFrame) { currentBitmap = nextBitmap; HardwareBuffer currentBuffer = buffer; + boolean shouldCloseBuffer = !bitmapOwnsBuffer; currentFrame = - new HardwareBufferFrame.Builder( + new DefaultHardwareBufferFrame.Builder( buffer, /* releaseExecutor= */ directExecutor(), - /* releaseCallback= */ (fence) -> releaseBuffer(currentBuffer, fence)) + /* releaseCallback= */ (fence) -> { + if (shouldCloseBuffer) { + releaseBuffer(currentBuffer, fence); + } + }) .build(); // Save the generationId from after the native copy, as AndroidBitmap_unlockPixels can // cause the generationId to increment. currentBitmapGenerationId = nextBitmap.getGenerationId(); } catch (IllegalStateException e) { - // If the native copy failed, the buffer is not wrapped in a HardwareBufferFrame, so - // it needs to be closed here to avoid leaking. - if (buffer != null) { + // If the native copy failed, the buffer is not wrapped in a DefaultHardwareBufferFrame, + // so it needs to be closed here to avoid leaking. + if (buffer != null && !bitmapOwnsBuffer) { buffer.close(); } throw e; } } - outputFrame = checkNotNull(currentFrame).retain(); + return checkNotNull(currentFrame).buildUpon().shouldIncrementReferenceCount().build(); } - - inputFrame.release(/* releaseFence= */ null); - return outputFrame - .buildUpon() - .setFormat(inputFrame.format) - .setInternalFrame(nextBitmap) - .setReleaseTimeNs(inputFrame.releaseTimeNs) - .setPresentationTimeUs(inputFrame.presentationTimeUs) - .setSequencePresentationTimeUs(inputFrame.sequencePresentationTimeUs) - .setMetadata(inputFrame.getMetadata()) - .build(); } - @Override - public void close() { + /** Releases the cached frame and resets state. */ + /* package */ void flush() { synchronized (this) { if (currentFrame != null) { currentFrame.release(/* releaseFence= */ null); currentFrame = null; currentBitmap = null; - currentBitmapGenerationId = 0; + currentBitmapGenerationId = C.INDEX_UNSET; } + } + } + + @Override + public void close() { + synchronized (this) { + flush(); if (!internalExecutor.isShutdown()) { internalExecutor.shutdown(); } @@ -193,14 +195,14 @@ public void close() { } /** - * Copies a {@link Bitmap.Config#ARGB_8888}, {@link Bitmap.Config#RGBA_F16} or {@link - * Bitmap.Config#RGBA_1010102} {@link Bitmap} to a {@link HardwareBuffer} using JNI. + * Copies a software {@link Bitmap} to a {@link HardwareBuffer} using JNI. The pixel format of the + * created buffer is derived from the source {@link Bitmap.Config} via {@link + * #getHardwareBufferPixelFormat} so that the source bit depth is preserved. * *

    The created buffer will have {@linkplain HardwareBuffer#USAGE_GPU_SAMPLED_IMAGE GPU read}, * {@linkplain HardwareBuffer#USAGE_GPU_COLOR_OUTPUT GPU write}, {@linkplain * HardwareBuffer#USAGE_CPU_READ_OFTEN CPU read} and {@linkplain - * HardwareBuffer#USAGE_CPU_WRITE_OFTEN CPU write} usage flags set, and pixelFormat of {@link - * HardwareBuffer#RGBA_8888}. + * HardwareBuffer#USAGE_CPU_WRITE_OFTEN CPU write} usage flags set. */ private static HardwareBuffer copyCpuBitmapToHardwareBuffer( Bitmap bitmap, HardwareBufferJniWrapper hardwareBufferJniWrapper) { @@ -208,17 +210,38 @@ private static HardwareBuffer copyCpuBitmapToHardwareBuffer( HardwareBuffer.create( bitmap.getWidth(), bitmap.getHeight(), - /* pixelFormat= */ HardwareBuffer.RGBA_8888, + getHardwareBufferPixelFormat(bitmap.getConfig()), /* layers= */ 1, /* usageFlags= */ HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE | HardwareBuffer.USAGE_GPU_COLOR_OUTPUT | HardwareBuffer.USAGE_CPU_READ_OFTEN | HardwareBuffer.USAGE_CPU_WRITE_OFTEN); - checkState(hardwareBufferJniWrapper.nativeCopyBitmapToHardwareBuffer(bitmap, buffer)); + if (!hardwareBufferJniWrapper.nativeCopyBitmapToHardwareBuffer(bitmap, buffer)) { + buffer.close(); + throw new IllegalStateException("Failed to natively copy bitmap to hardware buffer."); + } return buffer; } + /** + * Returns the {@link HardwareBuffer} pixel format that matches the source bit depth of the given + * {@link Bitmap.Config} on API 33+. Falls back to {@link HardwareBuffer#RGBA_8888} for configs + * whose bit depth is 8 bpc or unknown, or on API levels below 33. + */ + // TODO: b/545085046 - Investigate supporting RGBA_FP16 on API 30-32. + private static int getHardwareBufferPixelFormat(@Nullable Config config) { + if (SDK_INT >= 33) { + if (config == Config.RGBA_1010102) { + return HardwareBuffer.RGBA_1010102; + } + if (config == Config.RGBA_F16) { + return HardwareBuffer.RGBA_FP16; + } + } + return HardwareBuffer.RGBA_8888; + } + private void releaseBuffer(HardwareBuffer buffer, @Nullable SyncFenceWrapper releaseFence) { if (releaseFence == null) { buffer.close(); diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/CapturingEncoderFactory.java b/libraries/transformer/src/main/java/androidx/media3/transformer/CapturingEncoderFactory.java index 03a3a04f976..6660bb4eedf 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/CapturingEncoderFactory.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/CapturingEncoderFactory.java @@ -46,6 +46,11 @@ public Codec createForVideoEncoding(Format format, @Nullable LogSessionId logSes return videoEncoder; } + @Override + public boolean isVideoFormatSupported(Format format) { + return encoderFactory.isVideoFormatSupported(format); + } + @Override public boolean audioNeedsEncoding() { return encoderFactory.audioNeedsEncoding(); diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/CodecDbLite.java b/libraries/transformer/src/main/java/androidx/media3/transformer/CodecDbLite.java index 22a311d8981..50b1b97924b 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/CodecDbLite.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/CodecDbLite.java @@ -17,6 +17,8 @@ import static android.os.Build.VERSION.SDK_INT; import static com.google.common.base.Preconditions.checkArgument; +import static java.lang.Math.max; +import static java.lang.Math.min; import static java.lang.Math.round; import android.os.Build; @@ -59,49 +61,78 @@ public final class CodecDbLite { MimeTypes.VIDEO_H265, /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 37538929, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("Google", "Tensor G2"), new VideoEncoderEntry( MimeTypes.VIDEO_H264, /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 32739600, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("Google", "Tensor G3"), new VideoEncoderEntry( MimeTypes.VIDEO_H265, /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 37538350, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("Google", "Tensor G3"), new VideoEncoderEntry( MimeTypes.VIDEO_H264, /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 32750593, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("Google", "Tensor G4"), new VideoEncoderEntry( MimeTypes.VIDEO_AV1, /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 32844500, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("Google", "Tensor G4"), new VideoEncoderEntry( MimeTypes.VIDEO_H265, /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 51851802, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("Google", "Tensor G4"), new VideoEncoderEntry( MimeTypes.VIDEO_H264, /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 44206216, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("Mediatek", "MT6761"), new VideoEncoderEntry( @@ -157,14 +188,16 @@ public final class CodecDbLite { MimeTypes.VIDEO_H265, /* maxBFrames= */ 0, /* bFrameResolutionCutoff= */ 0, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("Mediatek", "MT6785"), new VideoEncoderEntry( MimeTypes.VIDEO_H264, /* maxBFrames= */ 0, /* bFrameResolutionCutoff= */ 0, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("Mediatek", "MT6789V/CD"), new VideoEncoderEntry( @@ -199,28 +232,36 @@ public final class CodecDbLite { MimeTypes.VIDEO_H265, /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 34028841, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("Mediatek", "MT6893"), new VideoEncoderEntry( MimeTypes.VIDEO_H264, /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 457499715, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) - .put( - new Chipset("Mediatek", "MT6983"), - new VideoEncoderEntry( - MimeTypes.VIDEO_H265, - /* maxBFrames= */ 1, - /* bFrameResolutionCutoff= */ 36134374, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("Mediatek", "MT6983"), new VideoEncoderEntry( MimeTypes.VIDEO_H264, /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 189533581, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("QTI", "SDM450"), new VideoEncoderEntry( @@ -228,13 +269,6 @@ public final class CodecDbLite { /* maxBFrames= */ 0, /* bFrameResolutionCutoff= */ 0, VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) - .put( - new Chipset("QTI", "SM4350"), - new VideoEncoderEntry( - MimeTypes.VIDEO_H265, - /* maxBFrames= */ 0, - /* bFrameResolutionCutoff= */ 0, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) .put( new Chipset("QTI", "SM4350"), new VideoEncoderEntry( @@ -242,13 +276,6 @@ public final class CodecDbLite { /* maxBFrames= */ 0, /* bFrameResolutionCutoff= */ 0, VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) - .put( - new Chipset("QTI", "SM6125"), - new VideoEncoderEntry( - MimeTypes.VIDEO_H265, - /* maxBFrames= */ 0, - /* bFrameResolutionCutoff= */ 0, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) .put( new Chipset("QTI", "SM6125"), new VideoEncoderEntry( @@ -256,13 +283,6 @@ public final class CodecDbLite { /* maxBFrames= */ 0, /* bFrameResolutionCutoff= */ 0, VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) - .put( - new Chipset("QTI", "SM6225"), - new VideoEncoderEntry( - MimeTypes.VIDEO_H265, - /* maxBFrames= */ 0, - /* bFrameResolutionCutoff= */ 0, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) .put( new Chipset("QTI", "SM6225"), new VideoEncoderEntry( @@ -270,13 +290,6 @@ public final class CodecDbLite { /* maxBFrames= */ 0, /* bFrameResolutionCutoff= */ 0, VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) - .put( - new Chipset("QTI", "SM6375"), - new VideoEncoderEntry( - MimeTypes.VIDEO_H265, - /* maxBFrames= */ 0, - /* bFrameResolutionCutoff= */ 0, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) .put( new Chipset("QTI", "SM6375"), new VideoEncoderEntry( @@ -290,28 +303,44 @@ public final class CodecDbLite { MimeTypes.VIDEO_H265, /* maxBFrames= */ 0, /* bFrameResolutionCutoff= */ 0, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("QTI", "SM8250"), new VideoEncoderEntry( MimeTypes.VIDEO_H264, /* maxBFrames= */ 0, /* bFrameResolutionCutoff= */ 0, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("QTI", "SM8350"), new VideoEncoderEntry( MimeTypes.VIDEO_H265, /* maxBFrames= */ 0, /* bFrameResolutionCutoff= */ 0, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("QTI", "SM8350"), new VideoEncoderEntry( MimeTypes.VIDEO_H264, /* maxBFrames= */ 0, /* bFrameResolutionCutoff= */ 0, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("QTI", "SM8450"), new VideoEncoderEntry( @@ -319,7 +348,11 @@ public final class CodecDbLite { /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 497664000, VideoEncoderEntry.FormatOptimization - .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES)) + .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("QTI", "SM8450"), new VideoEncoderEntry( @@ -327,7 +360,11 @@ public final class CodecDbLite { /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 497664000, VideoEncoderEntry.FormatOptimization - .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES)) + .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("QTI", "SM8475"), new VideoEncoderEntry( @@ -335,7 +372,9 @@ public final class CodecDbLite { /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 497664000, VideoEncoderEntry.FormatOptimization - .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES)) + .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("QTI", "SM8475"), new VideoEncoderEntry( @@ -343,7 +382,9 @@ public final class CodecDbLite { /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 497664000, VideoEncoderEntry.FormatOptimization - .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES)) + .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("QTI", "SM8550"), new VideoEncoderEntry( @@ -351,7 +392,13 @@ public final class CodecDbLite { /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 497664000, VideoEncoderEntry.FormatOptimization - .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES)) + .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("QTI", "SM8550"), new VideoEncoderEntry( @@ -359,7 +406,13 @@ public final class CodecDbLite { /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 110196681, VideoEncoderEntry.FormatOptimization - .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES)) + .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("QTI", "SM8650"), new VideoEncoderEntry( @@ -367,7 +420,13 @@ public final class CodecDbLite { /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 34344411, VideoEncoderEntry.FormatOptimization - .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES)) + .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("QTI", "SM8650"), new VideoEncoderEntry( @@ -375,7 +434,13 @@ public final class CodecDbLite { /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 132451733, VideoEncoderEntry.FormatOptimization - .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES)) + .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("QTI", "SM8750"), new VideoEncoderEntry( @@ -383,7 +448,13 @@ public final class CodecDbLite { /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 52435727, VideoEncoderEntry.FormatOptimization - .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES)) + .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("QTI", "SM8750"), new VideoEncoderEntry( @@ -391,49 +462,69 @@ public final class CodecDbLite { /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 159007069, VideoEncoderEntry.FormatOptimization - .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES)) - .put( - new Chipset("Samsung", "Exynos 850"), - new VideoEncoderEntry( - MimeTypes.VIDEO_H265, - /* maxBFrames= */ 0, - /* bFrameResolutionCutoff= */ 0, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("Samsung", "Exynos 850"), new VideoEncoderEntry( MimeTypes.VIDEO_H264, /* maxBFrames= */ 0, /* bFrameResolutionCutoff= */ 0, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("Samsung", "s5e8825"), new VideoEncoderEntry( MimeTypes.VIDEO_H265, /* maxBFrames= */ 0, /* bFrameResolutionCutoff= */ 0, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("Samsung", "s5e8825"), new VideoEncoderEntry( MimeTypes.VIDEO_H264, /* maxBFrames= */ 0, /* bFrameResolutionCutoff= */ 0, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("Samsung", "s5e9925"), new VideoEncoderEntry( MimeTypes.VIDEO_H265, /* maxBFrames= */ 1, /* bFrameResolutionCutoff= */ 51506898, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("Samsung", "s5e9925"), new VideoEncoderEntry( MimeTypes.VIDEO_H264, /* maxBFrames= */ 2, /* bFrameResolutionCutoff= */ 40856748, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) + VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P + | VideoEncoderEntry.FormatOptimization + .FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K)) .put( new Chipset("Spreadtrum", "SC9863A"), new VideoEncoderEntry( @@ -441,13 +532,6 @@ public final class CodecDbLite { /* maxBFrames= */ 0, /* bFrameResolutionCutoff= */ 0, VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) - .put( - new Chipset("Spreadtrum", "SC9863A"), - new VideoEncoderEntry( - MimeTypes.VIDEO_H265, - /* maxBFrames= */ 0, - /* bFrameResolutionCutoff= */ 0, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) .put( new Chipset("Spreadtrum", "T606"), new VideoEncoderEntry( @@ -455,13 +539,6 @@ public final class CodecDbLite { /* maxBFrames= */ 0, /* bFrameResolutionCutoff= */ 0, VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) - .put( - new Chipset("Spreadtrum", "T606"), - new VideoEncoderEntry( - MimeTypes.VIDEO_H265, - /* maxBFrames= */ 0, - /* bFrameResolutionCutoff= */ 0, - VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE)) .build(); private static final VideoEncoderEntry ENCODER_DEFAULT = @@ -489,6 +566,25 @@ public final class CodecDbLite { /* x0Sigmoid= */ 1.956e+07, /* offsetSigmoid= */ 8760); + /** + * Frame dimension thresholds delimiting the resolution scopes used by the {@linkplain + * VideoEncoderEntry.FormatOptimization non-linear rate model flags}. + * + *

    A format belongs to a scope if either its long edge reaches the scope's long-edge + * threshold or its short edge reaches the scope's short-edge threshold. + */ + private static final int SCOPE_4K_LONG_EDGE = 3840; + + private static final int SCOPE_4K_SHORT_EDGE = 2160; + + private static final int SCOPE_1080P_LONG_EDGE = 1920; + + private static final int SCOPE_1080P_SHORT_EDGE = 1080; + + private static final int SCOPE_720P_LONG_EDGE = 1280; + + private static final int SCOPE_720P_SHORT_EDGE = 720; + private CodecDbLite() {} /** Returns the MIME type recommended for video encoding on the runtime device. */ @@ -508,6 +604,10 @@ public static String getRecommendedVideoMimeType() { /** * Returns the recommended {@link VideoEncoderSettings} for the provided format. * + *

    The returned {@linkplain VideoEncoderSettings#bitrate bitrate} is {@link + * VideoEncoderSettings#NO_VALUE} if no recommendation is available for the current device and the + * provided format. + * * @param format the video format to recommend settings for which must include {@link * Format#sampleMimeType} to identify the codec. For best results, {@link Format#width}, * {@link Format#height}, {@link Format#frameRate} should be set since some optimizations are @@ -550,16 +650,17 @@ public static VideoEncoderSettings getRecommendedVideoEncoderSettings(Format for } } - if (format.width != Format.NO_VALUE + if (entryForCodec != null + && format.width != Format.NO_VALUE && format.height != Format.NO_VALUE && format.frameRate != Format.NO_VALUE) { - - // Use H.264 MIME type if not provided, so that we always recommend a valid video bitrate. - String mimeForBitrate = - format.sampleMimeType == null ? MimeTypes.VIDEO_H264 : format.sampleMimeType; - settingsBuilder.setBitrate( - getRecommendedBitrate(mimeForBitrate, format.width, format.height, format.frameRate)); + getRecommendedBitrate( + entryForCodec.mimeType, + format.width, + format.height, + format.frameRate, + entryForCodec.formatOptimizations)); } return settingsBuilder.build(); @@ -575,11 +676,21 @@ public static VideoEncoderSettings getRecommendedVideoEncoderSettings(Format for * @param width The video width. * @param height The video height. * @param frameRate The video frame rate. + * @param formatOptimizations The {@linkplain VideoEncoderEntry.FormatOptimization format + * optimizations} recommended for the encoder, which determine the resolution scopes the + * non-linear rate model has been validated for. * @return The recommended bitrate in bits per second, or {@link VideoEncoderSettings#NO_VALUE} if - * no recommendation is available for the given MIME type. + * no recommendation is available for the given MIME type, or if {@code formatOptimizations} + * does not opt in to the non-linear rate model for the resolution scope that the frame + * dimensions fall into. Callers are expected to fall back to their default bitrate + * calculation when {@link VideoEncoderSettings#NO_VALUE} is returned. */ private static int getRecommendedBitrate( - String mimeType, int width, int height, float frameRate) { + String mimeType, + int width, + int height, + float frameRate, + @VideoEncoderEntry.FormatOptimization int formatOptimizations) { BitrateParameters params; if (mimeType.equals(MimeTypes.VIDEO_H264)) { @@ -590,6 +701,30 @@ private static int getRecommendedBitrate( return VideoEncoderSettings.NO_VALUE; } + // The rate model is a positive opt-in per resolution scope, so resolve the scope a format + // falls into and require the matching flag. 1440p (2560x1440) resolves to the 1080p scope + // because its long edge reaches 1920 but not 3840. + int longEdge = max(width, height); + int shortEdge = min(width, height); + @VideoEncoderEntry.FormatOptimization int requiredRateModelFlag; + if (longEdge >= SCOPE_4K_LONG_EDGE || shortEdge >= SCOPE_4K_SHORT_EDGE) { + requiredRateModelFlag = + VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K; + } else if (longEdge >= SCOPE_1080P_LONG_EDGE || shortEdge >= SCOPE_1080P_SHORT_EDGE) { + requiredRateModelFlag = + VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P; + } else if (longEdge >= SCOPE_720P_LONG_EDGE || shortEdge >= SCOPE_720P_SHORT_EDGE) { + requiredRateModelFlag = + VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P; + } else { + // The rate model is not validated below the 720p scope. + return VideoEncoderSettings.NO_VALUE; + } + + if ((formatOptimizations & requiredRateModelFlag) == 0) { + return VideoEncoderSettings.NO_VALUE; + } + double x = (double) width * height * frameRate; double a = params.aPower * Math.pow(x, params.bPower) @@ -747,7 +882,10 @@ private VideoEncoderEntry( value = { VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NONE, VideoEncoderEntry.FormatOptimization - .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES + .FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES, + VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P, + VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P, + VideoEncoderEntry.FormatOptimization.FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K }) private @interface FormatOptimization { /** Flag indicating that no additional optimizations need to be made to the encoder. */ @@ -758,6 +896,24 @@ private VideoEncoderEntry( * android.media.MediaFormat#KEY_TEMPORAL_LAYERING} must also be set. */ int FORMAT_OPTIMIZATION_SET_TEMPORAL_LAYERING_FOR_B_FRAMES = 1; + + /** + * Flag indicating that the non-linear rate model has been validated for this encoder in the + * '720p' resolution scope. + */ + int FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_720P = 1 << 1; + + /** + * Flag indicating that the non-linear rate model has been validated for this encoder in the + * '1080p' resolution scope. + */ + int FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_1080P = 1 << 2; + + /** + * Flag indicating that the non-linear rate model has been validated for this encoder in the + * '4k' resolution scope. + */ + int FORMAT_OPTIMIZATION_NON_LINEAR_RATE_MODEL_4K = 1 << 3; } } } diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/Composition.java b/libraries/transformer/src/main/java/androidx/media3/transformer/Composition.java index 43cd16e5d6c..ff9b8e8f03e 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/Composition.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/Composition.java @@ -28,6 +28,7 @@ import androidx.media3.common.util.Log; import androidx.media3.common.util.UnstableApi; import androidx.media3.common.video.Frame; +import androidx.media3.common.video.FrameProcessor; import androidx.media3.effect.DefaultGlFrameProcessor; import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableSet; @@ -61,6 +62,7 @@ public static final class Builder { private boolean transmuxVideo; private @HdrMode int hdrMode; private boolean retainHdrFromUltraHdrImage; + private VideoFrameAggregationParameters videoFrameAggregationParameters; /** * Creates an instance. @@ -88,6 +90,7 @@ public Builder(List sequences) { this.sequences = ImmutableList.copyOf(sequences); videoCompositorSettings = VideoCompositorSettings.DEFAULT; effects = Effects.EMPTY; + videoFrameAggregationParameters = VideoFrameAggregationParameters.DEFAULT; } /** Creates a new instance to build upon the provided {@link Composition}. */ @@ -100,6 +103,7 @@ private Builder(Composition composition) { transmuxVideo = composition.transmuxVideo; hdrMode = composition.hdrMode; retainHdrFromUltraHdrImage = composition.retainHdrFromUltraHdrImage; + videoFrameAggregationParameters = composition.videoFrameAggregationParameters; } /** @@ -130,6 +134,27 @@ public Builder setEffects(Effects effects) { return this; } + /** + * Sets the {@link VideoFrameAggregationParameters} to apply to the {@link Composition}. + * + *

    The default value is {@link VideoFrameAggregationParameters#DEFAULT}. + * + *

    Note: This configuration is only applied when a {@link FrameProcessor} is enabled (e.g., + * by setting a {@link FrameProcessor.Factory} via {@link + * Transformer.Builder#setFrameProcessorFactory(FrameProcessor.Factory)} or {@link + * CompositionPlayer.Builder#setFrameProcessorFactory(FrameProcessor.Factory)}). + * + * @param videoFrameAggregationParameters The {@link VideoFrameAggregationParameters} to apply. + * @return This builder. + */ + @CanIgnoreReturnValue + @ExperimentalApi // TODO: b/526781983 - Remove @ExperimentalApi. + public Builder setVideoFrameAggregationParameters( + VideoFrameAggregationParameters videoFrameAggregationParameters) { + this.videoFrameAggregationParameters = videoFrameAggregationParameters; + return this; + } + /** * @deprecated Use {@link EditedMediaItemSequence.Builder#Builder(Set)} to set sequence track * types instead. @@ -272,7 +297,8 @@ public Composition build() { transmuxAudio, transmuxVideo, hdrMode, - retainHdrFromUltraHdrImage && hdrMode == HDR_MODE_KEEP_HDR); + retainHdrFromUltraHdrImage && hdrMode == HDR_MODE_KEEP_HDR, + videoFrameAggregationParameters); } /** @@ -431,6 +457,14 @@ public Composition build() { */ public final boolean retainHdrFromUltraHdrImage; + /** + * The {@link VideoFrameAggregationParameters} to apply to the composition. + * + *

    The default value is {@link VideoFrameAggregationParameters#DEFAULT}. + */ + @ExperimentalApi // TODO: b/526781983 - Remove @ExperimentalApi. + public final VideoFrameAggregationParameters videoFrameAggregationParameters; + /** Returns a {@link Composition.Builder} initialized with the values of this instance. */ /* package */ Builder buildUpon() { return new Builder(this); @@ -444,7 +478,8 @@ private Composition( boolean transmuxAudio, boolean transmuxVideo, @HdrMode int hdrMode, - boolean retainHdrFromUltraHdrImage) { + boolean retainHdrFromUltraHdrImage, + VideoFrameAggregationParameters videoFrameAggregationParameters) { checkArgument( !transmuxAudio || !forceAudioTrack, "Audio transmuxing and audio track forcing are not allowed together."); @@ -459,6 +494,7 @@ private Composition( this.forceAudioTrack = forceAudioTrack; this.hdrMode = hdrMode; this.retainHdrFromUltraHdrImage = retainHdrFromUltraHdrImage; + this.videoFrameAggregationParameters = videoFrameAggregationParameters; } @Override @@ -493,6 +529,8 @@ public String toString() { jsonObject.put("transmuxVideo", transmuxVideo); jsonObject.put("hdrMode", hdrMode); jsonObject.put("retainHdrFromUltraHdrImage", retainHdrFromUltraHdrImage); + jsonObject.put( + "videoFrameAggregationParameters", videoFrameAggregationParameters.toJsonObject()); return jsonObject; } catch (JSONException e) { Log.w(/* tag= */ "Composition", "JSON conversion failed.", e); diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/CompositionFrameMetadata.java b/libraries/transformer/src/main/java/androidx/media3/transformer/CompositionFrameMetadata.java index e665f0a22d9..c5f7e5e2836 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/CompositionFrameMetadata.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/CompositionFrameMetadata.java @@ -26,19 +26,18 @@ import androidx.media3.common.util.ExperimentalApi; import androidx.media3.common.video.Frame; import androidx.media3.effect.DefaultGlFrameProcessor; -import androidx.media3.effect.HardwareBufferFrame; import com.google.common.collect.ImmutableMap; /** - * Metadata included on {@link HardwareBufferFrame}s in {@link CompositionPlayer}. + * Metadata included on frames in {@link CompositionPlayer}. * *

    Applications can extend this class to add custom metadata. * * @deprecated Use similar keys in {@link DefaultGlFrameProcessor} instead. */ @ExperimentalApi // TODO: b/470355043 - Publish CompositionPlayer. -@Deprecated // TODO: b/498547782 - Remove with effect.HardwareBufferFrame. -public class CompositionFrameMetadata implements HardwareBufferFrame.Metadata { +@Deprecated // TODO: b/498547782 - Remove once all callers migrate to Frame metadata keys. +public class CompositionFrameMetadata { /** Converts the {@link CompositionFrameMetadata} to a map for {@link Frame#getMetadata()}. */ @RequiresApi(26) diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/CompositionPlayer.java b/libraries/transformer/src/main/java/androidx/media3/transformer/CompositionPlayer.java index 589476c272c..9ab4883caaa 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/CompositionPlayer.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/CompositionPlayer.java @@ -24,6 +24,10 @@ import static androidx.media3.effect.DebugTraceUtil.EVENT_SET_COMPOSITION; import static androidx.media3.effect.DebugTraceUtil.EVENT_SET_VIDEO_OUTPUT; import static androidx.media3.exoplayer.video.PlaybackVideoGraphWrapper.LATE_US_TO_DROP_INPUT_FRAME; +import static androidx.media3.transformer.FrameAggregator.STRATEGY_EXPECT_NO_FRAMES; +import static androidx.media3.transformer.FrameAggregator.STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET; +import static androidx.media3.transformer.FrameAggregator.STRATEGY_MATCH_FRAME_CLOSEST_TO_TARGET; +import static androidx.media3.transformer.TransformerUtil.END_OF_STREAM_ASYNC_FRAME; import static androidx.media3.transformer.TransformerUtil.containsSpeedChangingEffects; import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkNotNull; @@ -48,8 +52,6 @@ import androidx.annotation.IntRange; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; -import androidx.annotation.RestrictTo; -import androidx.annotation.RestrictTo.Scope; import androidx.annotation.VisibleForTesting; import androidx.media3.common.AudioAttributes; import androidx.media3.common.C; @@ -75,6 +77,7 @@ import androidx.media3.common.util.ConditionVariable; import androidx.media3.common.util.ExperimentalApi; import androidx.media3.common.util.GlUtil; +import androidx.media3.common.util.HandlerExecutor; import androidx.media3.common.util.HandlerWrapper; import androidx.media3.common.util.Log; import androidx.media3.common.util.NullableType; @@ -84,15 +87,11 @@ import androidx.media3.common.video.FrameProcessor; import androidx.media3.common.video.SurfaceHolderFrameWriter; import androidx.media3.common.video.SyncFenceWrapper; -import androidx.media3.effect.BitmapToHardwareBufferProcessor; import androidx.media3.effect.DebugTraceUtil; import androidx.media3.effect.DefaultGlObjectsProvider; import androidx.media3.effect.DefaultVideoFrameProcessor; -import androidx.media3.effect.HardwareBufferFrame; -import androidx.media3.effect.HardwareBufferFrameProcessor; -import androidx.media3.effect.HardwareBufferFrameQueue; +import androidx.media3.effect.HardwareBufferJni; import androidx.media3.effect.HardwareBufferJniWrapper; -import androidx.media3.effect.RenderingPacketConsumer; import androidx.media3.effect.SingleInputVideoGraph; import androidx.media3.effect.TimestampAdjustment; import androidx.media3.exoplayer.DecoderCounters; @@ -127,6 +126,7 @@ import androidx.media3.extractor.DefaultExtractorsFactory; import androidx.media3.extractor.amr.AmrExtractor; import androidx.media3.extractor.ts.AdtsExtractor; +import androidx.media3.transformer.FrameAggregator.AggregationStrategy; import com.google.common.base.Supplier; import com.google.common.base.Suppliers; import com.google.common.collect.ImmutableList; @@ -181,18 +181,15 @@ public static final class Builder { private boolean handleAudioFocus; private VideoGraph.@MonotonicNonNull Factory videoGraphFactory; - @Nullable - private RenderingPacketConsumer, HardwareBufferFrameQueue> - packetProcessor; - private FrameProcessor.@MonotonicNonNull Factory frameProcessorFactory; @Nullable private HardwareBufferJniWrapper hardwareBufferJniWrapper; private Supplier imageReaderAdapterFactorySupplier; + @Nullable private AnalyticsListener sequencePlayerAnalyticsListener; + @Nullable private VideoFrameReleaseControl.FrameTimingEvaluator frameTimingEvaluator; private boolean videoPrewarmingEnabled; - private boolean perStreamMediaProgressionEnabled; private boolean enableReplayableCache; private long lateThresholdToDropInputUs; private boolean built; @@ -327,22 +324,6 @@ public Builder setImageDecoderFactory(ImageDecoder.Factory imageDecoderFactory) return this; } - /** - * Sets whether to enable per-stream media progression in the player. - * - *

    The default value is {@code false}. - * - * @param perStreamMediaProgressionEnabled Whether to enable per-stream media progression in the - * player. - * @return This builder, for convenience. - */ - @CanIgnoreReturnValue - @ExperimentalApi // TODO: b/528260159 - Enable this by default. - public Builder setPerStreamMediaProgressionEnabled(boolean perStreamMediaProgressionEnabled) { - this.perStreamMediaProgressionEnabled = perStreamMediaProgressionEnabled; - return this; - } - /** * Sets the {@link Clock} that will be used by the player. * @@ -486,22 +467,6 @@ public Builder experimentalSetLateThresholdToDropInputUs(long lateThresholdToDro return this; } - /** - * @deprecated Use {@link #setFrameProcessorFactory} instead. - */ - @Deprecated - @RequiresApi(28) - @CanIgnoreReturnValue - @RestrictTo(Scope.LIBRARY_GROUP) // TODO: b/498547782 - Remove once usages have been migrated. - public Builder setHardwareBufferEffectsPipeline( - RenderingPacketConsumer, HardwareBufferFrameQueue> - packetProcessor) { - checkState(videoGraphFactory == null); - checkState(frameProcessorFactory == null); - this.packetProcessor = packetProcessor; - return this; - } - /** * Sets the {@link FrameProcessor.Factory} to be used to create {@link FrameProcessor} instances * that are used to process {@link androidx.media3.common.video.HardwareBufferFrame}s. @@ -510,13 +475,10 @@ public Builder setHardwareBufferEffectsPipeline( * used and {@link CompositionPlayer} will not process {@link * androidx.media3.common.video.HardwareBufferFrame}s. * - *

    If used on API 32 and below, the {@linkplain #setNativeHardwareBufferHelpers native - * helpers} must be set. - * * @param frameProcessorFactory The {@link FrameProcessor.Factory}. * @return This builder. - * @throws IllegalStateException if a {@linkplain #setVideoGraphFactory videoGraphFactory} or - * {@linkplain #setHardwareBufferEffectsPipeline HardwareBufferEffectsPipeline} is set. + * @throws IllegalStateException if a {@linkplain #setVideoGraphFactory videoGraphFactory} is + * set. */ @RequiresApi(28) @CanIgnoreReturnValue @@ -524,7 +486,6 @@ public Builder setHardwareBufferEffectsPipeline( public Builder setFrameProcessorFactory(FrameProcessor.Factory frameProcessorFactory) { checkNotNull(frameProcessorFactory); checkState(videoGraphFactory == null); - checkState(packetProcessor == null); this.frameProcessorFactory = frameProcessorFactory; return this; } @@ -532,9 +493,11 @@ public Builder setFrameProcessorFactory(FrameProcessor.Factory frameProcessorFac /** * Sets the {@link HardwareBufferJniWrapper} used to provide native helpers. * + *

    The default value is {@link HardwareBufferJni#INSTANCE}. + * *

    This method is experimental and will be renamed or removed in a future release. * - *

    This will only be used if {@link #setHardwareBufferEffectsPipeline} is set. + *

    This will only be used if {@link #setFrameProcessorFactory} is set. * * @param hardwareBufferJniWrapper The {@link HardwareBufferJniWrapper} to provide native * helpers. @@ -567,6 +530,32 @@ public Builder setNativeHardwareBufferHelpers( return this; } + /** + * Sets a {@link AnalyticsListener} to receive events from internal players for testing + * purposes. + * + * @return This builder. + */ + @CanIgnoreReturnValue + @VisibleForTesting + /* package */ Builder setSequencePlayerAnalyticsListener(AnalyticsListener listener) { + sequencePlayerAnalyticsListener = listener; + return this; + } + + /** + * Sets a {@link VideoFrameReleaseControl.FrameTimingEvaluator} for testing purposes. + * + * @return This builder. + */ + @CanIgnoreReturnValue + @VisibleForTesting + /* package */ Builder setFrameTimingEvaluator( + VideoFrameReleaseControl.FrameTimingEvaluator frameTimingEvaluator) { + this.frameTimingEvaluator = frameTimingEvaluator; + return this; + } + /** * Builds the {@link CompositionPlayer} instance. Must be called at most once. * @@ -585,6 +574,9 @@ public CompositionPlayer build() { .build(); videoGraphFactory = new SingleInputVideoGraph.Factory(videoFrameProcessorFactory); } + if (SDK_INT >= 28 && frameProcessorFactory != null && hardwareBufferJniWrapper == null) { + hardwareBufferJniWrapper = HardwareBufferJni.INSTANCE; + } CompositionPlayer compositionPlayer = new CompositionPlayer(this); built = true; return compositionPlayer; @@ -635,7 +627,6 @@ public CompositionPlayer build() { private final ImageDecoder.Factory imageDecoderFactory; private final VideoGraph.Factory videoGraphFactory; private final boolean videoPrewarmingEnabled; - private final boolean perStreamMediaProgressionEnabled; private final HandlerWrapper compositionInternalListenerHandler; private final LoadControl loadControl; private final boolean enableReplayableCache; @@ -655,8 +646,7 @@ public CompositionPlayer build() { private final ImageReaderAdapter.Factory imageReaderAdapterFactory; @Nullable private final SurfaceHolderFrameWriter surfaceHolderFrameWriter; - - @Nullable private final HardwareBufferFrameProcessor hardwareBufferPostProcessor; + @Nullable private final HardwareBufferJniWrapper hardwareBufferJniWrapper; private final HandlerThread playbackThread; private final HandlerWrapper playbackThreadHandler; @@ -665,6 +655,8 @@ public CompositionPlayer build() { private final SparseBooleanArray videoTracksSelected; private final AnalyticsCollector analyticsCollector; + @Nullable private final AnalyticsListener sequencePlayerAnalyticsListener; + @Nullable private final VideoFrameReleaseControl.FrameTimingEvaluator frameTimingEvaluator; private @MonotonicNonNull CompositionPlayerInternal compositionPlayerInternal; private @MonotonicNonNull ImmutableList playlist; @@ -716,12 +708,13 @@ private CompositionPlayer(Builder builder) { imageDecoderFactory = new GapHandlingDecoderFactory(builder.imageDecoderFactorySupplier.get()); videoGraphFactory = checkNotNull(builder.videoGraphFactory); videoPrewarmingEnabled = builder.videoPrewarmingEnabled; - perStreamMediaProgressionEnabled = builder.perStreamMediaProgressionEnabled; compositionInternalListenerHandler = clock.createHandler(builder.looper, /* callback= */ null); loadControl = builder.loadControlSupplier.get(); this.enableReplayableCache = builder.enableReplayableCache; lateThresholdToDropInputUs = builder.lateThresholdToDropInputUs; imageReaderAdapterFactory = builder.imageReaderAdapterFactorySupplier.get(); + sequencePlayerAnalyticsListener = builder.sequencePlayerAnalyticsListener; + frameTimingEvaluator = builder.frameTimingEvaluator; videoTracksSelected = new SparseBooleanArray(); playerHolders = new ArrayList<>(); compositionDurationUs = C.TIME_UNSET; @@ -739,12 +732,8 @@ private CompositionPlayer(Builder builder) { new AudioFocusManager( context, applicationHandler.getLooper(), /* playerControl= */ internalListener); playbackAudioGraphWrapper = new PlaybackAudioGraphWrapper(audioMixerFactory, finalAudioSink); - HardwareBufferJniWrapper hardwareBufferJniWrapper = builder.hardwareBufferJniWrapper; - @Nullable - FrameProcessor.Factory frameProcessorFactory = - builder.frameProcessorFactory != null - ? builder.frameProcessorFactory - : getRenderingPacketConsumerFactory(builder.packetProcessor); + hardwareBufferJniWrapper = builder.hardwareBufferJniWrapper; + @Nullable FrameProcessor.Factory frameProcessorFactory = builder.frameProcessorFactory; playbackThread = new HandlerThread(/* name= */ "CompositionPlaybackThread", Process.THREAD_PRIORITY_AUDIO); try { @@ -755,31 +744,13 @@ private CompositionPlayer(Builder builder) { new HandlerExecutor(playbackThreadHandler, internalListener); Executor applicationThreadExecutor = new HandlerExecutor(applicationHandler, internalListener); - // Convert CPU Bitmaps to HardwareBuffers when the native helpers are available. - if (hardwareBufferJniWrapper != null) { - hardwareBufferPostProcessor = - new BitmapToHardwareBufferProcessor( - hardwareBufferJniWrapper, - /* internalExecutor= */ Util.newSingleThreadExecutor( - "BitmapToHardwareBufferProcessor::Thread"), - /* errorExecutor= */ playbackThreadExecutor, - /* errorCallback= */ internalListener::onError); - } else { - hardwareBufferPostProcessor = null; - } surfaceHolderFrameWriter = - SDK_INT >= 33 - ? SurfaceHolderFrameWriter.create( - /* surfaceHolder= */ null, - /* surfaceHolderExecutor= */ applicationThreadExecutor, - internalListener, - applicationThreadExecutor) - : SurfaceHolderFrameWriter.create( - /* surfaceHolder= */ null, - /* surfaceHolderExecutor= */ applicationThreadExecutor, - internalListener, - applicationThreadExecutor, - checkNotNull(hardwareBufferJniWrapper)); + SurfaceHolderFrameWriter.create( + /* surfaceHolder= */ null, + /* surfaceHolderExecutor= */ applicationThreadExecutor, + internalListener, + applicationThreadExecutor, + checkNotNull(hardwareBufferJniWrapper)); frameProcessor = frameProcessorFactory.create( surfaceHolderFrameWriter, @@ -789,18 +760,20 @@ private CompositionPlayer(Builder builder) { VideoFrameReleaseControl videoFrameReleaseControl = new VideoFrameReleaseControl( this.context, - /* frameTimingEvaluator= */ new CompositionFrameTimingEvaluator( - // Convert lateThresholdToDropInputUs to early time. - lateThresholdToDropInputUs != C.TIME_UNSET - ? -lateThresholdToDropInputUs - : C.TIME_UNSET), - /* allowedJoiningTimeMs= */ 0); + /* frameTimingEvaluator= */ frameTimingEvaluator != null + ? frameTimingEvaluator + : new CompositionFrameTimingEvaluator( + // Convert lateThresholdToDropInputUs to early time. + lateThresholdToDropInputUs != C.TIME_UNSET + ? -lateThresholdToDropInputUs + : C.TIME_UNSET), + /* allowedJoiningTimeMs= */ 0, + /* skipBuffersWithIdenticalReleaseTime= */ false); videoFrameReleaseControl.setClock(clock); videoPacketReleaseControl = new CompositionVideoPacketReleaseControl( videoFrameReleaseControl, frameProcessor, internalListener); } else { - hardwareBufferPostProcessor = null; frameProcessor = null; surfaceHolderFrameWriter = null; frameAggregator = null; @@ -855,7 +828,9 @@ public void setComposition(Composition composition, @IntRange(from = 0) long sta composition = applyPreProcessingEffects(composition); - if (composition.sequences.size() > 1 && !videoGraphFactory.supportsMultipleInputs()) { + if (composition.sequences.size() > 1 + && frameProcessor == null + && !videoGraphFactory.supportsMultipleInputs()) { Log.w(TAG, "Setting multi-sequence Composition with single input video graph."); } @@ -1095,14 +1070,6 @@ protected ListenableFuture handleRelease() { waitingForFrameAfterSeek = false; pendingSeekPositionMs = C.TIME_UNSET; removeSurfaceCallbacks(); - if (hardwareBufferPostProcessor != null) { - try { - hardwareBufferPostProcessor.close(); - } catch (Exception e) { - // Ignore exceptions during release. - Log.e(TAG, "Failed to release hardwareBufferPostProcessor.", e); - } - } // TODO: b/518679527 - Move close calls of frameAggregator and frameProcessor to the // playback thread. if (frameAggregator != null) { @@ -1333,6 +1300,7 @@ private static EditedMediaItem transformSpeedChangingEffects(EditedMediaItem ite */ // TODO: b/449937111 - Delete this once Effects#createExperimentalSpeedAdjustingEffect() is // removed. + @SuppressWarnings("deprecation") // Uses deprecated TimestampAdjustment. private static EditedMediaItem deactivateSpeedAdjustingVideoEffects(EditedMediaItem item) { ImmutableList videoEffects = item.effects.videoEffects; List newVideoEffects = new ArrayList<>(); @@ -1533,9 +1501,12 @@ private void prepareCompositionPlayerInternal() { VideoFrameReleaseControl videoFrameReleaseControl = new VideoFrameReleaseControl( context, - new CompositionFrameTimingEvaluator( - CompositionFrameTimingEvaluator.DEFAULT_FRAME_LATE_THRESHOLD_US), - /* allowedJoiningTimeMs= */ 0); + /* frameTimingEvaluator= */ frameTimingEvaluator != null + ? frameTimingEvaluator + : new CompositionFrameTimingEvaluator( + CompositionFrameTimingEvaluator.DEFAULT_FRAME_LATE_THRESHOLD_US), + /* allowedJoiningTimeMs= */ 0, + /* skipBuffersWithIdenticalReleaseTime= */ false); playbackVideoGraphWrapper = new PlaybackVideoGraphWrapper.Builder(context, videoFrameReleaseControl) .setVideoGraphFactory(checkNotNull(videoGraphFactory)) @@ -1579,6 +1550,7 @@ private void setCompositionInternal(Composition composition, long startPositionM frameAggregator = new FrameAggregator( composition.sequences.size(), + composition.videoFrameAggregationParameters.frameRate, videoPacketReleaseControl::queue, videoPacketReleaseControl::flush); } @@ -1670,17 +1642,12 @@ private SequencePlayerHolder createSequencePlayer(Composition composition, int s new HardwareBufferFrameReader( composition, sequenceIndex, - /* frameConsumer= */ hardwareBufferFrame -> { - if (hardwareBufferFrame == HardwareBufferFrame.END_OF_STREAM_FRAME) { + /* frameConsumer= */ asyncFrame -> { + if (asyncFrame == END_OF_STREAM_ASYNC_FRAME) { checkNotNull(currentFrameAggregator).queueEndOfStream(sequenceIndex); - } else if (hardwareBufferPostProcessor != null) { - HardwareBufferFrame processedFrame = - hardwareBufferPostProcessor.process(hardwareBufferFrame); - checkNotNull(currentFrameAggregator) - .queueFrame(processedFrame, sequenceIndex); } else { checkNotNull(currentFrameAggregator) - .queueFrame(hardwareBufferFrame, sequenceIndex); + .queueFrame(asyncFrame, sequenceIndex); } }, checkNotNull(playbackThread).getLooper(), @@ -1691,7 +1658,8 @@ private SequencePlayerHolder createSequencePlayer(Composition composition, int s "HardwareBufferFrameReader error", e, PlaybackException.ERROR_CODE_UNSPECIFIED), - compositionInternalListenerHandler)); + compositionInternalListenerHandler, + checkNotNull(hardwareBufferJniWrapper))); renderersFactory = SequenceRenderersFactory.createForHardwareBuffer( context, @@ -1702,8 +1670,18 @@ private SequencePlayerHolder createSequencePlayer(Composition composition, int s checkNotNull(videoPacketReleaseControl), hardwareBufferFrameReaderSupplier, lateThresholdToDropInputUs); - // Ensure the FrameAggregator ignores audio only sequences. - checkNotNull(currentFrameAggregator).registerSequence(sequenceIndex, sequenceContainsVideo); + @AggregationStrategy int aggregationStrategy; + if (!sequenceContainsVideo) { + // Audio only sequences never produce video frames. + aggregationStrategy = STRATEGY_EXPECT_NO_FRAMES; + } else if (HardwareBufferFrameReader.CAPACITY >= 3) { + // Retaining the previous frame in FrameAggregator requires reader capacity >= 3 since + // CompositionVideoPacketReleaseControl also retains a frame downstream. + aggregationStrategy = STRATEGY_MATCH_FRAME_CLOSEST_TO_TARGET; + } else { + aggregationStrategy = STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET; + } + checkNotNull(currentFrameAggregator).registerSequence(sequenceIndex, aggregationStrategy); } else { VideoSink inputSink = checkNotNull(playbackVideoGraphWrapper).getSink(sequenceIndex); renderersFactory = @@ -1729,6 +1707,9 @@ private SequencePlayerHolder createSequencePlayer(Composition composition, int s playerHolder.player.addListener(new PlayerListener(sequenceIndex)); playerHolder.player.addAnalyticsListener(new PlayerAnalyticsListener()); playerHolder.player.addAnalyticsListener(new EventLogger(TAG + "-" + sequenceIndex)); + if (sequencePlayerAnalyticsListener != null) { + playerHolder.player.addAnalyticsListener(sequencePlayerAnalyticsListener); + } // Audio focus is handled directly by CompositionPlayer, not by sequence players. playerHolder.player.setAudioAttributes(audioAttributes, /* handleAudioFocus= */ false); playerHolder.player.setPauseAtEndOfMediaItems(true); @@ -2380,16 +2361,12 @@ private SequencePlayerHolder( .setHandleAudioBecomingNoisy(true) .setLoadControl(loadControl) .setClock(clock) - // Use dynamic scheduling to show the first video/image frame more promptly when the - // player is paused (which is common in editing applications). - .experimentalSetDynamicSchedulingEnabled(true) .setTrackSelector(trackSelector) // TODO: b/489733731 - Reenable stuck player detection. .setStuckBufferingDetectionTimeoutMs(Integer.MAX_VALUE) .setStuckPlayingDetectionTimeoutMs(Integer.MAX_VALUE) .setStuckPlayingNotEndingTimeoutMs(Integer.MAX_VALUE) - .setStuckSuppressedDetectionTimeoutMs(Integer.MAX_VALUE) - .enablePerStreamMediaProgression(perStreamMediaProgressionEnabled); + .setStuckSuppressedDetectionTimeoutMs(Integer.MAX_VALUE); player = playerBuilder.build(); this.renderersFactory = renderersFactory; this.hardwareBufferFrameReaderSupplier = hardwareBufferFrameReaderSupplier; @@ -2460,7 +2437,8 @@ private final class InternalListener PlaybackVideoGraphWrapper.Listener, CompositionVideoPacketReleaseControl.Listener, SurfaceHolderFrameWriter.Listener, - FrameProcessor.Listener { + FrameProcessor.Listener, + HandlerExecutor.Listener { // AudioFocusManager.PlayerControl methods. Called on the application thread. @@ -2586,18 +2564,6 @@ public void onEnded() { } } - @Override - public void onError(VideoFrameProcessingException videoFrameProcessingException) { - // The error will also be surfaced from the underlying ExoPlayer instance via - // PlayerListener.onPlayerError, and it will arrive to the composition player twice. - applicationHandler.post( - () -> - maybeUpdatePlaybackError( - "Error processing video frames", - videoFrameProcessingException, - PlaybackException.ERROR_CODE_VIDEO_FRAME_PROCESSING_FAILED)); - } - // FrameProcessor.Listener methods @Override @@ -2611,46 +2577,24 @@ public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper onCompleteF videoPacketReleaseControl.onFrameProcessed(frame, onCompleteFence); } } - } - // TODO: b/510766403 - Remove once PacketConsumer entrypoint is removed. - /** Converts a {@link RenderingPacketConsumer} to a {@link FrameProcessor}. */ - @Nullable - private static FrameProcessor.Factory getRenderingPacketConsumerFactory( - @Nullable - RenderingPacketConsumer, HardwareBufferFrameQueue> - packetProcessor) { - if (SDK_INT >= 26 && packetProcessor != null) { - return (output, listenerExecutor, listener) -> { - HardwareBufferFrameQueue adaptedQueue = - new FrameWriterToHardwareBufferFrameQueueAdapter(output); - packetProcessor.setRenderOutput(adaptedQueue); - return new PacketConsumerToFrameProcessorAdapter( - packetProcessor, listenerExecutor, listener); - }; + @Override + public void onError(VideoFrameProcessingException videoFrameProcessingException) { + // The error will also be surfaced from the underlying ExoPlayer instance via + // PlayerListener.onPlayerError, and it will arrive to the composition player twice. + applicationHandler.post( + () -> + maybeUpdatePlaybackError( + "Error processing video frames", + videoFrameProcessingException, + PlaybackException.ERROR_CODE_VIDEO_FRAME_PROCESSING_FAILED)); } - return null; - } - private static final class HandlerExecutor implements Executor { - private final HandlerWrapper handler; - private final InternalListener internalListener; - - private HandlerExecutor(HandlerWrapper handler, InternalListener internalListener) { - this.handler = handler; - this.internalListener = internalListener; - } + // HandlerExecutor.Listener methods @Override - public void execute(Runnable command) { - handler.post( - () -> { - try { - command.run(); - } catch (RuntimeException e) { - internalListener.onError(e); - } - }); + public void onError(RuntimeException e) { + onError(VideoFrameProcessingException.from(e)); } } } diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/CompositionTrackSelector.java b/libraries/transformer/src/main/java/androidx/media3/transformer/CompositionTrackSelector.java index 027d5590cad..87a6862be57 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/CompositionTrackSelector.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/CompositionTrackSelector.java @@ -37,6 +37,7 @@ import org.checkerframework.checker.nullness.qual.MonotonicNonNull; /** A {@link TrackSelector} implementation for {@link CompositionPlayer}. */ +// TODO(b/540734725): Refactor and add tests for CompositionTrackSelector. /* package */ final class CompositionTrackSelector extends TrackSelector { public interface Listener { @@ -57,6 +58,11 @@ public void init(InvalidationListener listener, BandwidthMeter bandwidthMeter) { trackSelectorInternal.init(listener, bandwidthMeter); } + @Override + public void onParametersActivated(@Nullable TrackSelectionParameters parameters) { + trackSelectorInternal.onParametersActivated(parameters); + } + @Override public TrackSelectorResult selectTracks( RendererCapabilities[] rendererCapabilities, diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/CompositionVideoPacketReleaseControl.java b/libraries/transformer/src/main/java/androidx/media3/transformer/CompositionVideoPacketReleaseControl.java index df6448585ff..e45710f7500 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/CompositionVideoPacketReleaseControl.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/CompositionVideoPacketReleaseControl.java @@ -17,24 +17,20 @@ import static androidx.media3.effect.DefaultGlFrameProcessor.KEY_FRAME_DISCONTINUITY_NUMBER; import static androidx.media3.exoplayer.video.VideoSink.RELEASE_FIRST_FRAME_IMMEDIATELY; -import static androidx.media3.transformer.CompositionFrameMetadata.asFrameMetadata; +import static androidx.media3.transformer.TransformerUtil.END_OF_STREAM_ASYNC_FRAME; import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.base.Preconditions.checkState; -import androidx.annotation.GuardedBy; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; import androidx.media3.common.util.ExperimentalApi; -import androidx.media3.common.util.Log; import androidx.media3.common.util.SystemClock; import androidx.media3.common.video.AsyncFrame; import androidx.media3.common.video.DefaultHardwareBufferFrame; import androidx.media3.common.video.Frame; import androidx.media3.common.video.FrameProcessor; import androidx.media3.common.video.SyncFenceWrapper; -import androidx.media3.effect.GlTextureFrame; -import androidx.media3.effect.HardwareBufferFrame; import androidx.media3.exoplayer.ExoPlaybackException; import androidx.media3.exoplayer.video.FixedFrameRateEstimator; import androidx.media3.exoplayer.video.VideoFrameReleaseControl; @@ -42,36 +38,29 @@ import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableMap; import java.util.ArrayDeque; -import java.util.HashMap; import java.util.List; -import java.util.Map; // TODO: b/449956936 - This is a placeholder implementation, revisit the threading logic to make it // more robust. -/** Computes the release time for each {@linkplain List packet}. */ +/** Computes the release time for each {@linkplain List packet}. */ @RequiresApi(26) @ExperimentalApi // TODO: b/449956776 - Remove once FrameConsumer API is finalized. /* package */ class CompositionVideoPacketReleaseControl implements CompositionRendererListener, AutoCloseable { - private static final String TAG = "CompositionReleaseCtrl"; - private final VideoFrameReleaseControl videoFrameReleaseControl; private final FrameProcessor downstreamFrameProcessor; + private final InFlightFrameManager inFlightFrameManager; // Accessed on the playback thread only. - private final ArrayDeque> packetQueue; + private final ArrayDeque> packetQueue; private final VideoFrameReleaseControl.FrameReleaseInfo videoFrameReleaseInfo; private final FixedFrameRateEstimator frameRateEstimator; private volatile boolean isEnded; private final Listener listener; // Accessed on the playback thread only. private int currentStreamDiscontinuityNumber; - private final Object lock; - - @GuardedBy("lock") - private final Map inFlightFrames; - @Nullable private ImmutableList lastQueuedPacket; + @Nullable private ImmutableList lastQueuedPacket; /** Listener for {@link CompositionVideoPacketReleaseControl} events. */ public interface Listener { @@ -88,9 +77,8 @@ public interface Listener { * Creates a new {@link CompositionVideoPacketReleaseControl}. * * @param videoFrameReleaseControl Controls when frames are released. - * @param downstreamFrameProcessor Receives the {@linkplain List packet}, - * with each {@link HardwareBufferFrame} having the same {@linkplain - * HardwareBufferFrame#releaseTimeNs} release time}. + * @param downstreamFrameProcessor Receives the {@linkplain List packet}, with each + * {@link Frame} having the same release time. * @param listener The listener for {@link CompositionVideoPacketReleaseControl} events. */ public CompositionVideoPacketReleaseControl( @@ -104,24 +92,24 @@ public CompositionVideoPacketReleaseControl( frameRate -> videoFrameReleaseControl.setSurfaceMediaFrameRate(frameRate)); this.listener = listener; this.downstreamFrameProcessor = downstreamFrameProcessor; + inFlightFrameManager = new InFlightFrameManager(); packetQueue = new ArrayDeque<>(); videoFrameReleaseInfo = new VideoFrameReleaseControl.FrameReleaseInfo(); - lock = new Object(); - inFlightFrames = new HashMap<>(); // Allow the first frame to be rendered before playback starts. videoFrameReleaseControl.onStreamChanged(RELEASE_FIRST_FRAME_IMMEDIATELY); } /** - * Queues a {@linkplain List packet}. + * Queues a {@linkplain List packet}. * - *

    Once called, the caller must not modify the {@link HardwareBufferFrame}s in the packet. + *

    Once called, the caller must not modify the {@linkplain AsyncFrame async frames} in the + * packet. * *

    Called on the playback thread. * - * @param packet The {@link List} to queue. + * @param packet The {@link List} to queue. */ - public void queue(List packet) { + public void queue(List packet) { checkArgument(!packet.isEmpty()); packetQueue.add(ImmutableList.copyOf(packet)); } @@ -129,7 +117,7 @@ public void queue(List packet) { /** * {@inheritDoc} * - *

    Computes the release action and release time of queued {@linkplain List + *

    Computes the release action and release time of queued {@linkplain List * packets}, forwards them {@linkplain #downstreamFrameProcessor downstream} if applicable or * drops them. Continues until a packet should be held until a later {@code positionUs}. * @@ -143,10 +131,10 @@ public void onRender( throws ExoPlaybackException { // Remove packet from the packet queue to ensure frames are not simultaneously released by // queueFrame and forwarded downstream. - @Nullable ImmutableList packet; + @Nullable ImmutableList packet; while ((packet = packetQueue.poll()) != null) { checkState(!packet.isEmpty()); - if (packet.get(0).equals(HardwareBufferFrame.END_OF_STREAM_FRAME)) { + if (packet.get(0) == END_OF_STREAM_ASYNC_FRAME) { if (packetQueue.peek() == null) { isEnded = true; downstreamFrameProcessor.signalEndOfStream(); @@ -156,7 +144,7 @@ public void onRender( // Ignore EOS frames if there are more frames to be rendered. continue; } - long presentationTimeUs = checkNotNull(packet).get(0).sequencePresentationTimeUs; + long presentationTimeUs = checkNotNull(packet).get(0).frame.getContentTimeUs(); frameRateEstimator.onNextFrame(presentationTimeUs * 1000); @VideoFrameReleaseControl.FrameReleaseAction int frameReleaseAction = @@ -179,9 +167,9 @@ public void onRender( } /** - * {@linkplain HardwareBufferFrame#release Releases} all frames that have not been sent - * downstream, and {@link VideoFrameReleaseControl#reset() resets} the release control, when the - * primary sequence is flushed. + * Releases all frames that have not been sent downstream, and {@link + * VideoFrameReleaseControl#reset() resets} the release control, when the primary sequence is + * flushed. * *

    Called on the playback thread. */ @@ -218,30 +206,18 @@ public void onStopped() { /** Called when a frame has been fully processed by the downstream {@link FrameProcessor}. */ public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper releaseFence) { - @Nullable HardwareBufferFrame hardwareBufferFrame; - synchronized (lock) { - hardwareBufferFrame = inFlightFrames.remove(frame); - } - if (hardwareBufferFrame != null) { - hardwareBufferFrame.release(releaseFence); - } else { - if (releaseFence != null) { - releaseFence.close(); - } - Log.d(TAG, "onFrameProcessed: Frame not found: " + frame); - } + inFlightFrameManager.onFrameProcessed(frame, releaseFence); } /** - * {@linkplain HardwareBufferFrame#release Releases} all frames that have not been sent - * downstream, and {@link VideoFrameReleaseControl#reset() resets} the release control. + * Releases all frames that have not been sent downstream, and {@link + * VideoFrameReleaseControl#reset() resets} the release control. * - *

    This method does not release the frames in {@link #inFlightFrames} which have been sent - * downstream. The {@link #downstreamFrameProcessor} is responsible for releasing in flight - * frames. + *

    This method does not release the in-flight frames which have been sent downstream. The + * {@link #downstreamFrameProcessor} is responsible for releasing in flight frames. */ private void reset() { - @Nullable ImmutableList packet; + @Nullable ImmutableList packet; while ((packet = packetQueue.poll()) != null) { releasePacket(packet); } @@ -274,19 +250,13 @@ public void close() { /** Releases any resources held by this release control. */ private void releaseRetainedFrames() { - ImmutableList framesToRelease; - synchronized (lock) { - // Copy frames to release them without holding the lock. - framesToRelease = ImmutableList.copyOf(inFlightFrames.values()); - inFlightFrames.clear(); - } - releasePacket(framesToRelease); + inFlightFrameManager.releaseAll(); releasePacket(lastQueuedPacket); lastQueuedPacket = null; } - private void updateLastQueuedPacket(ImmutableList newlyQueuedPacket) { - ImmutableList lastQueuedPacket = this.lastQueuedPacket; + private void updateLastQueuedPacket(ImmutableList newlyQueuedPacket) { + ImmutableList lastQueuedPacket = this.lastQueuedPacket; // The newlyQueuedPacket is retained so that it's kept alive for replays. this.lastQueuedPacket = retainFrames(newlyQueuedPacket); // When replaying, the newly queued packet is the same as the last queued packet. When queueing @@ -298,27 +268,32 @@ private void updateLastQueuedPacket(ImmutableList newlyQueu } } - private ImmutableList retainFrames( - ImmutableList newlyQueuedPacket) { - ImmutableList.Builder retainedFrames = new ImmutableList.Builder<>(); + private ImmutableList retainFrames(ImmutableList newlyQueuedPacket) { + ImmutableList.Builder retainedFrames = new ImmutableList.Builder<>(); for (int i = 0; i < newlyQueuedPacket.size(); i++) { - HardwareBufferFrame hardwareBufferFrame = newlyQueuedPacket.get(i); - retainedFrames.add(hardwareBufferFrame.retain()); + AsyncFrame asyncFrame = newlyQueuedPacket.get(i); + checkState(asyncFrame.frame instanceof DefaultHardwareBufferFrame); + DefaultHardwareBufferFrame retainedFrame = + ((DefaultHardwareBufferFrame) asyncFrame.frame) + .buildUpon() + .shouldIncrementReferenceCount() + .build(); + retainedFrames.add(new AsyncFrame(retainedFrame, asyncFrame.acquireFence)); } return retainedFrames.build(); } /** - * Determines how the {@link HardwareBufferFrame} should be handled given the release action. + * Determines how the {@link AsyncFrame} should be handled given the release action. * * @param frameReleaseAction The release action for this frame. - * @param packet The {@link ImmutableList} to send downstream. - * @return {@code true} if the {@link HardwareBufferFrame} should be removed from the internal - * {@link #packetQueue}. + * @param packet The {@link ImmutableList} to send downstream. + * @return {@code true} if the {@link AsyncFrame} should be removed from the internal {@link + * #packetQueue}. */ private boolean maybeQueuePacketDownstream( @VideoFrameReleaseControl.FrameReleaseAction int frameReleaseAction, - ImmutableList packet) { + ImmutableList packet) { switch (frameReleaseAction) { case VideoFrameReleaseControl.FRAME_RELEASE_TRY_AGAIN_LATER: case VideoFrameReleaseControl.FRAME_RELEASE_IGNORE: @@ -349,62 +324,62 @@ private boolean maybeQueuePacketDownstream( } /** - * Updates the release time of all {@link HardwareBufferFrame}s and forwards them to a downstream - * consumer of {@link GlTextureFrame}. + * Updates the release time of all {@link AsyncFrame} instances and forwards them to a downstream + * consumer. * - *

    The downstream consumer is responsible for releasing the {@link GlTextureFrame} packet. + *

    The downstream consumer is responsible for releasing the packet. * - * @param packet The list of {@link HardwareBufferFrame} to send downstream. + * @param packet The list of {@link AsyncFrame} to send downstream. * @param releaseTimeNs The time the packet should be rendered on screen. * @return Whether the frame was queued downstream. */ - @SuppressWarnings("deprecation") private boolean setReleaseTimeAndQueueDownstream( - ImmutableList packet, long releaseTimeNs) { - ImmutableList.Builder asyncFrameListBuilder = ImmutableList.builder(); - Map pendingInFlightFrames = new HashMap<>(); + ImmutableList packet, long releaseTimeNs) { + ImmutableList.Builder downstreamAsyncFrames = ImmutableList.builder(); + for (int i = 0; i < packet.size(); i++) { - HardwareBufferFrame effectFrame = packet.get(i); + AsyncFrame asyncFrame = packet.get(i); + Frame inputFrame = asyncFrame.frame; + ImmutableMap.Builder metadataBuilder = ImmutableMap.builder() - .put(Frame.KEY_PRESENTATION_TIME_US, effectFrame.presentationTimeUs) - .put(Frame.KEY_DISPLAY_TIME_NS, releaseTimeNs); - if (effectFrame.getMetadata() instanceof CompositionFrameMetadata) { - CompositionFrameMetadata compositionFrameMetadata = - (CompositionFrameMetadata) effectFrame.getMetadata(); - metadataBuilder - .put(CompositionFrameMetadata.KEY_COMPOSITION_FRAME_METADATA, compositionFrameMetadata) - .putAll(asFrameMetadata(compositionFrameMetadata)); - } - metadataBuilder.put(KEY_FRAME_DISCONTINUITY_NUMBER, currentStreamDiscontinuityNumber); - DefaultHardwareBufferFrame commonFrame = - new DefaultHardwareBufferFrame.Builder(checkNotNull(effectFrame.hardwareBuffer)) - .setFormat(effectFrame.format) - .setContentTimeUs(effectFrame.sequencePresentationTimeUs) - .setMetadata(metadataBuilder.buildOrThrow()) - .setInternalImage(effectFrame.internalFrame) + .putAll(inputFrame.getMetadata()) + .put(Frame.KEY_DISPLAY_TIME_NS, releaseTimeNs) + .put(KEY_FRAME_DISCONTINUITY_NUMBER, currentStreamDiscontinuityNumber); + + checkState(inputFrame instanceof DefaultHardwareBufferFrame); + Frame downstreamFrame = + ((DefaultHardwareBufferFrame) inputFrame) + .buildUpon() + .shouldIncrementReferenceCount() + .setMetadata(metadataBuilder.buildKeepingLast()) .build(); - asyncFrameListBuilder.add(new AsyncFrame(commonFrame, effectFrame.acquireFence)); - pendingInFlightFrames.put(commonFrame, effectFrame); + downstreamAsyncFrames.add(new AsyncFrame(downstreamFrame, asyncFrame.acquireFence)); } - boolean queued = downstreamFrameProcessor.queue(asyncFrameListBuilder.build()); - if (queued) { - synchronized (lock) { - inFlightFrames.putAll(pendingInFlightFrames); + ImmutableList asyncFrameList = downstreamAsyncFrames.build(); + boolean queued = false; + try { + queued = + inFlightFrameManager.trackIfSuccessful(downstreamFrameProcessor::queue, asyncFrameList); + return queued; + } finally { + if (queued) { + updateLastQueuedPacket(packet); + releasePacket(packet); + } else { + releasePacket(asyncFrameList); } - updateLastQueuedPacket(packet); } - return queued; } - private static void releasePacket(@Nullable ImmutableList packet) { + private static void releasePacket(@Nullable List packet) { if (packet == null) { return; } for (int i = 0; i < packet.size(); i++) { - packet.get(i).release(/* releaseFence= */ null); + TransformerUtil.releaseIfNeeded(packet.get(i).frame, /* releaseFence= */ null); } } } diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/DefaultDecoderFactory.java b/libraries/transformer/src/main/java/androidx/media3/transformer/DefaultDecoderFactory.java index 78f4cd6ff1c..4c6c074b707 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/DefaultDecoderFactory.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/DefaultDecoderFactory.java @@ -36,7 +36,6 @@ import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; import androidx.media3.common.util.CodecSpecificDataUtil; -import androidx.media3.common.util.ExperimentalApi; import androidx.media3.common.util.Log; import androidx.media3.common.util.MediaFormatUtil; import androidx.media3.common.util.UnstableApi; @@ -48,6 +47,7 @@ import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.util.ArrayList; import java.util.List; +import java.util.Objects; /** * Default implementation of {@link Codec.DecoderFactory} that uses {@link MediaCodec} for decoding. @@ -80,7 +80,6 @@ public static final class Builder { private @C.Priority int codecPriority; private boolean shouldConfigureOperatingRate; private MediaCodecSelector mediaCodecSelector; - private boolean dynamicSchedulingEnabled; /** Creates a new {@link Builder}. */ public Builder(Context context) { @@ -168,29 +167,6 @@ public Builder setMediaCodecSelector(MediaCodecSelector mediaCodecSelector) { return this; } - /** - * Sets whether decoder dynamic scheduling is enabled. - * - *

    If enabled, the {@link ExoPlayerAssetLoader} can change how often the rendering loop for - * {@linkplain DefaultCodec decoders} created by this factory is run. - * - *

    On some devices, setting this to {@code true} will {@linkplain - * DefaultCodec#queueInputBuffer feed} and {@linkplain DefaultCodec#releaseOutputBuffer drain} - * decoders more frequently, and will lead to improved performance. - * - *

    The default value is {@code false}. - * - *

    This method is experimental, and will be renamed or removed in a future release. - * - * @param dynamicSchedulingEnabled Whether to enable dynamic scheduling. - */ - @CanIgnoreReturnValue - @ExperimentalApi // TODO: b/369523131 - Remove once feature is enabled by default. - public Builder experimentalSetDynamicSchedulingEnabled(boolean dynamicSchedulingEnabled) { - this.dynamicSchedulingEnabled = dynamicSchedulingEnabled; - return this; - } - /** Creates an instance of {@link DefaultDecoderFactory}, using defaults if values are unset. */ public DefaultDecoderFactory build() { return new DefaultDecoderFactory(this); @@ -203,7 +179,6 @@ public DefaultDecoderFactory build() { private final @C.Priority int codecPriority; private final boolean shouldConfigureOperatingRate; private final MediaCodecSelector mediaCodecSelector; - private final boolean dynamicSchedulingEnabled; /** * @deprecated Use {@link Builder} instead. @@ -237,7 +212,6 @@ private DefaultDecoderFactory(Builder builder) { this.codecPriority = builder.codecPriority; this.shouldConfigureOperatingRate = builder.shouldConfigureOperatingRate; this.mediaCodecSelector = builder.mediaCodecSelector; - this.dynamicSchedulingEnabled = builder.dynamicSchedulingEnabled; } @Override @@ -278,6 +252,14 @@ public DefaultCodec createForVideoDecoding( throw createExportException( format, /* reason= */ "Decoding 8k is not supported on this device."); } + @Nullable + Pair codecProfileAndLevel = + CodecSpecificDataUtil.getCodecProfileAndLevel(format); + if (codecProfileAndLevel == null + && Objects.equals(format.sampleMimeType, MimeTypes.VIDEO_DOLBY_VISION)) { + throw createExportException( + format, /* reason= */ "Unsupported Dolby Vision profile or level"); + } if (deviceNeedsNoFrameRateWorkaround()) { format = format.buildUpon().setFrameRate(Format.NO_VALUE).build(); } @@ -293,9 +275,6 @@ public DefaultCodec createForVideoDecoding( MediaFormat.KEY_COLOR_TRANSFER_REQUEST, MediaFormat.COLOR_TRANSFER_SDR_VIDEO); } - @Nullable - Pair codecProfileAndLevel = - CodecSpecificDataUtil.getCodecProfileAndLevel(format); if (codecProfileAndLevel != null) { MediaFormatUtil.maybeSetInteger( mediaFormat, MediaFormat.KEY_PROFILE, codecProfileAndLevel.first); @@ -372,15 +351,6 @@ private DefaultCodec createCodecForMediaFormat( return codec; } - /** - * Returns whether decoder dynamic scheduling is enabled. - * - *

    See {@link Builder#experimentalSetDynamicSchedulingEnabled}. - */ - public boolean isDynamicSchedulingEnabled() { - return dynamicSchedulingEnabled; - } - private DefaultCodec createCodecFromDecoderInfos( Context context, List decoderInfos, @@ -418,8 +388,8 @@ private static void configureOperatingRate(MediaFormat mediaFormat, float maxSup } if (deviceNeedsPriorityWorkaround()) { - // Setting KEY_PRIORITY to 1 leads to worse performance on many devices. - mediaFormat.setInteger(MediaFormat.KEY_PRIORITY, 1); + // Setting KEY_PRIORITY to non-realtime leads to worse performance on many devices. + mediaFormat.setInteger(MediaFormat.KEY_PRIORITY, C.MEDIA_CODEC_PRIORITY_NON_REALTIME); } mediaFormat.setFloat(MediaFormat.KEY_OPERATING_RATE, maxSupportedFrameRate); @@ -465,6 +435,11 @@ private static boolean deviceNeedsDisableToneMappingWorkaround( // using it (see b/290725189). return true; } + if (SDK_INT == 34 && Ascii.equalsIgnoreCase(Build.MODEL, "xq-dq72")) { + // The Sony Xperia 1 V drops empty EOS output buffer when tone-mapping HDR to SDR (see + // b/558024432). + return true; + } return false; } diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/DefaultEncoderFactory.java b/libraries/transformer/src/main/java/androidx/media3/transformer/DefaultEncoderFactory.java index 6d52bc8cfd9..0daa1a16161 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/DefaultEncoderFactory.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/DefaultEncoderFactory.java @@ -20,6 +20,7 @@ import static androidx.media3.common.ColorInfo.isTransferHdr; import static androidx.media3.common.util.MediaFormatUtil.createMediaFormatFromFormat; import static androidx.media3.transformer.EncoderUtil.getCodecProfilesForHdrFormat; +import static androidx.media3.transformer.EncoderUtil.toTemporalLayeringSchema; import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.base.Preconditions.checkState; @@ -47,7 +48,6 @@ import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.util.ArrayList; import java.util.List; -import java.util.Locale; import org.checkerframework.checker.nullness.qual.RequiresNonNull; /** A default implementation of {@link Codec.EncoderFactory}. */ @@ -58,9 +58,6 @@ public final class DefaultEncoderFactory implements Codec.EncoderFactory { private static final int DEFAULT_AUDIO_BITRATE = 128 * 1024; private static final int DEFAULT_FRAME_RATE = 30; - /** Best effort, or as-fast-as-possible priority setting for {@link MediaFormat#KEY_PRIORITY}. */ - private static final int PRIORITY_BEST_EFFORT = 1; - /** A builder for {@link DefaultEncoderFactory} instances. */ public static final class Builder { private final Context context; @@ -366,15 +363,12 @@ public DefaultCodec createForVideoEncoding(Format format, @Nullable LogSessionId mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, round(encoderSupportedFormat.frameRate)); if (supportedVideoEncoderSettings.profile != VideoEncoderSettings.NO_VALUE - && supportedVideoEncoderSettings.level != VideoEncoderSettings.NO_VALUE - && SDK_INT >= 24) { - // For API levels below 24, setting profile and level can lead to failures in MediaCodec - // configuration. The encoder selects the profile/level when we don't set them. + && supportedVideoEncoderSettings.level != VideoEncoderSettings.NO_VALUE) { // Set profile and level at the same time to maximize compatibility, or the encoder will pick // the values. mediaFormat.setInteger(MediaFormat.KEY_PROFILE, supportedVideoEncoderSettings.profile); mediaFormat.setInteger(MediaFormat.KEY_LEVEL, supportedVideoEncoderSettings.level); - } else if (SDK_INT >= 24 && ColorInfo.isTransferHdr(format.colorInfo)) { + } else if (ColorInfo.isTransferHdr(format.colorInfo)) { ImmutableList codecProfilesForHdrFormat = getCodecProfilesForHdrFormat(mimeType, checkNotNull(format.colorInfo).colorTransfer); mediaFormat.setInteger(MediaFormat.KEY_PROFILE, codecProfilesForHdrFormat.get(0)); @@ -453,21 +447,14 @@ public DefaultCodec createForVideoEncoding(Format format, @Nullable LogSessionId int numBidirectionalTemporalLayers = supportedVideoEncoderSettings.numBidirectionalTemporalLayers; if (SDK_INT >= 29 && numNonBidirectionalTemporalLayers >= 0) { - String temporalSchema; - if (numNonBidirectionalTemporalLayers == 0) { - temporalSchema = "none"; - } else if (numBidirectionalTemporalLayers > 0) { - temporalSchema = - String.format( - Locale.ROOT, - "android.generic.%d+%d", - numNonBidirectionalTemporalLayers, - numBidirectionalTemporalLayers); - } else { - temporalSchema = - String.format(Locale.ROOT, "android.generic.%d", numNonBidirectionalTemporalLayers); - } - mediaFormat.setString(MediaFormat.KEY_TEMPORAL_LAYERING, temporalSchema); + mediaFormat.setString( + MediaFormat.KEY_TEMPORAL_LAYERING, + toTemporalLayeringSchema( + numNonBidirectionalTemporalLayers, numBidirectionalTemporalLayers)); + } + + if (supportedVideoEncoderSettings.complexity != VideoEncoderSettings.NO_VALUE) { + mediaFormat.setInteger(MediaFormat.KEY_COMPLEXITY, supportedVideoEncoderSettings.complexity); } return new DefaultCodec( @@ -607,7 +594,9 @@ private static VideoEncoderConfiguration resolveVideoEncoderConfiguration( } else if (requestedFormat.averageBitrate != Format.NO_VALUE) { Log.d(TAG, "Using bitrate from requested Format"); requestedBitrate = requestedFormat.averageBitrate; - } else if (enableCodecDbLiteBitrate && recommendedVideoEncoderSettings != null) { + } else if (enableCodecDbLiteBitrate + && recommendedVideoEncoderSettings != null + && recommendedVideoEncoderSettings.bitrate != VideoEncoderSettings.NO_VALUE) { Log.d(TAG, "Using bitrate from CodecDB Lite"); requestedBitrate = recommendedVideoEncoderSettings.bitrate; } else { @@ -805,7 +794,7 @@ private static void adjustMediaFormatForEncoderPerformanceSettings(MediaFormat m return; } - mediaFormat.setInteger(MediaFormat.KEY_PRIORITY, PRIORITY_BEST_EFFORT); + mediaFormat.setInteger(MediaFormat.KEY_PRIORITY, C.MEDIA_CODEC_PRIORITY_NON_REALTIME); if (SDK_INT == 26) { mediaFormat.setInteger(MediaFormat.KEY_OPERATING_RATE, DEFAULT_FRAME_RATE); @@ -844,7 +833,7 @@ private static void adjustMediaFormatForH264EncoderSettings( @Nullable ColorInfo colorInfo, MediaCodecInfo encoderInfo, MediaFormat mediaFormat) { // TODO: b/445454172 - Remove overriding profile/level (before API 29). String mimeType = MimeTypes.VIDEO_H264; - if (SDK_INT >= 29) { + if (SDK_INT >= 29 && !deviceNeedsNoH264HighProfileWorkaround()) { int expectedEncodingProfile = MediaCodecInfo.CodecProfileLevel.AVCProfileHigh; if (colorInfo != null) { int colorTransfer = colorInfo.colorTransfer; @@ -882,7 +871,7 @@ private static void adjustMediaFormatForH264EncoderSettings( // TODO: b/445616792 - Set KEY_LATENCY to 2 to enable B-frame production. mediaFormat.setInteger(MediaFormat.KEY_LATENCY, 1); } - } else if (SDK_INT >= 24) { + } else { int expectedEncodingProfile = MediaCodecInfo.CodecProfileLevel.AVCProfileBaseline; int supportedLevel = EncoderUtil.findHighestSupportedEncodingLevel( @@ -895,8 +884,6 @@ private static void adjustMediaFormatForH264EncoderSettings( mediaFormat.setInteger(MediaFormat.KEY_LEVEL, supportedLevel); } } - // For API levels below 24, setting profile and level can lead to failures in MediaCodec - // configuration. The encoder selects the profile/level when we don't set them. } private interface EncoderFallbackCost { @@ -998,8 +985,12 @@ private static boolean deviceNeedsDefaultFrameRateWorkaround() { private static boolean deviceNeedsNoH264HighProfileWorkaround() { // The H.264/AVC encoder produces B-frames when high profile is chosen despite configuration to - // turn them off, so force not using high profile on these devices (see b/306617392). + // turn them off (see b/306617392), or fails during configure (see b/558023801), so force not + // using high profile on these devices. // TODO: b/229420356 - Remove once the in-app muxer is the default and B-frames are supported. - return SDK_INT == 27 && (Build.DEVICE.equals("ASUS_X00T_3") || Build.DEVICE.equals("TC77")); + return (SDK_INT == 27 && (Build.DEVICE.equals("ASUS_X00T_3") || Build.DEVICE.equals("TC77"))) + + // Redmi 7a + || Build.DEVICE.equals("pine"); } } diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/DefaultExportOperation.java b/libraries/transformer/src/main/java/androidx/media3/transformer/DefaultExportOperation.java index 758002c0b2d..6204de78880 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/DefaultExportOperation.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/DefaultExportOperation.java @@ -19,6 +19,8 @@ import android.content.Context; import android.media.metrics.LogSessionId; +import android.net.Uri; +import android.os.ParcelFileDescriptor; import androidx.annotation.Nullable; import androidx.media3.common.C.TrackType; import androidx.media3.common.DebugViewProvider; @@ -32,6 +34,7 @@ import androidx.media3.transformer.ExportResult.ProcessedInput; import androidx.media3.transformer.Transformer.ProgressState; import com.google.common.collect.ImmutableList; +import java.io.IOException; /** An {@link ExportOperation} implementation for single export operations. */ /* package */ final class DefaultExportOperation implements ExportOperation { @@ -53,15 +56,16 @@ @Nullable private final FrameProcessor.Factory frameProcessorFactory; - @Nullable HardwareBufferJniWrapper hardwareBufferJniWrapper; + @Nullable private final HardwareBufferJniWrapper hardwareBufferJniWrapper; @Nullable private final LogSessionId logSessionId; private final boolean applyMp4EditListTrim; private final Muxer.Factory muxerFactory; - private final String outputFilePath; + private final TransformerOutput transformerOutput; private final boolean fileStartsOnVideoFrameEnabled; private final ExportResult.Builder exportResultBuilder; private final ComponentListener componentListener; + @Nullable private Uri mediaStoreUri; @Nullable private TransformerInternal transformerInternal; @@ -85,7 +89,7 @@ public DefaultExportOperation( @Nullable LogSessionId logSessionId, boolean applyMp4EditListTrim, Muxer.Factory muxerFactory, - String outputFilePath, + TransformerOutput transformerOutput, boolean fileStartsOnVideoFrameEnabled) { this.context = context; this.composition = composition; @@ -105,7 +109,7 @@ public DefaultExportOperation( this.hardwareBufferJniWrapper = hardwareBufferJniWrapper; this.logSessionId = logSessionId; this.muxerFactory = muxerFactory; - this.outputFilePath = outputFilePath; + this.transformerOutput = transformerOutput; this.fileStartsOnVideoFrameEnabled = fileStartsOnVideoFrameEnabled; this.applyMp4EditListTrim = applyMp4EditListTrim; exportResultBuilder = new ExportResult.Builder(); @@ -114,14 +118,28 @@ public DefaultExportOperation( @Override public void start() { - MuxerWrapper muxerWrapper = - new MuxerWrapper( - outputFilePath, - muxerFactory, - componentListener, - MuxerWrapper.MUXER_MODE_DEFAULT, - /* dropSamplesBeforeFirstVideoSample= */ fileStartsOnVideoFrameEnabled, - /* appendVideoFormat= */ null); + MuxerWrapper muxerWrapper; + if (transformerOutput.outputType == TransformerOutput.TYPE_MEDIA_STORE) { + try { + muxerWrapper = createMuxerWrapperForMediaStore(); + } catch (IOException | RuntimeException e) { + // The entry may have been inserted before opening its file descriptor failed. + deleteMediaStoreEntryIfPresent(); + listener.onError( + exportResultBuilder.build(), + ExportException.createForMuxer(e, ExportException.ERROR_CODE_MUXING_FAILED)); + return; + } + } else { + muxerWrapper = + new MuxerWrapper( + checkNotNull(transformerOutput.path), + muxerFactory, + componentListener, + MuxerWrapper.MUXER_MODE_DEFAULT, + /* dropSamplesBeforeFirstVideoSample= */ fileStartsOnVideoFrameEnabled, + /* appendVideoFormat= */ null); + } transformerInternal = new TransformerInternal( context, @@ -158,8 +176,11 @@ public void start() { @Override public void cancel() { if (transformerInternal != null) { + // Blocks until the export pipeline has ended, releasing the muxer and closing the file + // descriptor, so that the MediaStore entry can be safely deleted below. transformerInternal.cancel(); } + deleteMediaStoreEntryIfPresent(); } @Override @@ -167,6 +188,34 @@ public void endWithException(ExportException exportException) { checkNotNull(transformerInternal).endWithException(exportException); } + private MuxerWrapper createMuxerWrapperForMediaStore() throws IOException { + Uri itemUri = + TransformerOutputResolver.insertMediaStoreEntry( + context, + checkNotNull(transformerOutput.collectionUri), + checkNotNull(transformerOutput.contentValues)); + this.mediaStoreUri = itemUri; + ParcelFileDescriptor pfd = TransformerOutputResolver.openFileDescriptor(context, itemUri); + return new MuxerWrapper( + pfd, + muxerFactory, + componentListener, + MuxerWrapper.MUXER_MODE_DEFAULT, + /* dropSamplesBeforeFirstVideoSample= */ fileStartsOnVideoFrameEnabled, + /* appendVideoFormat= */ null); + } + + /** + * Deletes the MediaStore entry created for this export, if any, so that an export that did not + * complete successfully does not leave an orphaned entry behind. + */ + private void deleteMediaStoreEntryIfPresent() { + if (mediaStoreUri != null) { + TransformerOutputResolver.deleteMediaStoreEntry(context, mediaStoreUri); + mediaStoreUri = null; + } + } + private final class ComponentListener implements TransformerInternal.Listener, MuxerWrapper.Listener { @@ -177,6 +226,11 @@ public void onCompleted( ImmutableList processedInputs, @Nullable String audioEncoderName, @Nullable String videoEncoderName) { + if (mediaStoreUri != null) { + TransformerOutputResolver.clearMediaStorePendingFlag(context, mediaStoreUri); + // The entry is published, so it must not be deleted by a subsequent cancel(). + mediaStoreUri = null; + } ExportResultUtil.updateProcessingDetails( exportResultBuilder, processedInputs, audioEncoderName, videoEncoderName); listener.onCompleted(exportResultBuilder.build()); @@ -189,6 +243,7 @@ public void onError( @Nullable String audioEncoderName, @Nullable String videoEncoderName, ExportException exportException) { + deleteMediaStoreEntryIfPresent(); ExportResultUtil.updateProcessingDetails( exportResultBuilder, processedInputs, audioEncoderName, videoEncoderName); listener.onError( diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/DefaultMuxer.java b/libraries/transformer/src/main/java/androidx/media3/transformer/DefaultMuxer.java index 7be75cba65c..831adc6f5a6 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/DefaultMuxer.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/DefaultMuxer.java @@ -15,6 +15,7 @@ */ package androidx.media3.transformer; +import android.os.ParcelFileDescriptor; import androidx.media3.common.C; import androidx.media3.common.Format; import androidx.media3.common.Metadata; @@ -71,6 +72,11 @@ public Muxer create(String path) throws MuxerException { return new DefaultMuxer(muxerFactory.create(path)); } + @Override + public Muxer create(ParcelFileDescriptor pfd) throws MuxerException { + return new DefaultMuxer(muxerFactory.create(pfd)); + } + @Override public ImmutableList getSupportedSampleMimeTypes(@C.TrackType int trackType) { return muxerFactory.getSupportedSampleMimeTypes(trackType); diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/EditedMediaItem.java b/libraries/transformer/src/main/java/androidx/media3/transformer/EditedMediaItem.java index fc97a10a324..1d7f3bb4fb2 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/EditedMediaItem.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/EditedMediaItem.java @@ -412,7 +412,10 @@ public Builder buildUpon() { */ @RestrictTo(Scope.LIBRARY_GROUP) public long getPresentationDurationUs() { - checkState(durationUs != C.TIME_UNSET); + checkState( + durationUs != C.TIME_UNSET, + "durationUs must be set via EditedMediaItem.Builder.setDurationUs() for" + + " CompositionPlayer."); if (presentationDurationUs == C.TIME_UNSET) { presentationDurationUs = getClippedDuration(mediaItem.clippingConfiguration, durationUs); @@ -437,7 +440,16 @@ public long getPresentationDurationUs() { jsonObject.put("effects", effects.toJsonObject()); jsonObject.put("removeAudio", removeAudio); jsonObject.put("removeVideo", removeVideo); + jsonObject.put("flattenForSlowMotion", flattenForSlowMotion); jsonObject.put("durationUs", durationUs); + jsonObject.put("frameRate", frameRate != C.RATE_UNSET_INT ? frameRate : "UNSET"); + if (!Objects.equals(speedParameters.speedProvider, SpeedProvider.DEFAULT)) { + JSONObject speedParametersJsonObject = new JSONObject(); + speedParametersJsonObject.put( + "speedProvider", speedParameters.speedProvider.getClass().getSimpleName()); + speedParametersJsonObject.put("shouldMaintainPitch", speedParameters.shouldMaintainPitch); + jsonObject.put("speedParameters", speedParametersJsonObject); + } // Calling getPresentationDurationUs() without a set duration will crash. jsonObject.put( "presentationDuration", durationUs != C.TIME_UNSET ? getPresentationDurationUs() : "N/A"); diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/EditingMetricsCollector.java b/libraries/transformer/src/main/java/androidx/media3/transformer/EditingMetricsCollector.java index 413aabad914..b3e66094240 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/EditingMetricsCollector.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/EditingMetricsCollector.java @@ -20,25 +20,34 @@ import android.content.Context; import android.hardware.DataSpace; +import android.media.metrics.BundleSession; import android.media.metrics.EditingEndedEvent; import android.media.metrics.EditingSession; import android.media.metrics.LogSessionId; import android.media.metrics.MediaItemInfo; import android.media.metrics.MediaMetricsManager; +import android.os.PersistableBundle; import android.util.Size; import android.util.SparseIntArray; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; +import androidx.annotation.VisibleForTesting; import androidx.media3.common.C; import androidx.media3.common.ColorInfo; +import androidx.media3.common.Effect; import androidx.media3.common.Format; +import androidx.media3.common.MetricsProvider; import androidx.media3.common.MimeTypes; +import androidx.media3.common.audio.AudioProcessor; import androidx.media3.common.util.Log; import androidx.media3.common.util.SystemClock; +import androidx.media3.effect.TimestampWrapper; import com.google.common.collect.ImmutableList; import com.google.common.collect.Iterables; import java.util.ArrayList; +import java.util.HashSet; import java.util.List; +import java.util.Set; /** * A metrics collector that collects editing events and forwards them to {@link MetricsReporter}. @@ -60,6 +69,9 @@ interface Factory { *

    The method should be called at most once. */ void reportMetrics(EditingEndedEvent editingEndedEvent); + + /** Reports effect metrics using a {@link PersistableBundle}. */ + default void reportEffectMetrics(PersistableBundle bundle) {} } /** @@ -89,11 +101,13 @@ public MetricsReporter create() { /** The {@link EditingSession} to report collected metrics to. */ @Nullable private EditingSession editingSession; + @Nullable private final MediaMetricsManager mediaMetricsManager; + private boolean metricsReported; + private boolean effectMetricsReported; private DefaultMetricsReporter(Context context) { - @Nullable - MediaMetricsManager mediaMetricsManager = + mediaMetricsManager = (MediaMetricsManager) context.getSystemService(Context.MEDIA_METRICS_SERVICE); if (mediaMetricsManager != null) { editingSession = mediaMetricsManager.createEditingSession(); @@ -108,6 +122,23 @@ public void reportMetrics(EditingEndedEvent editingEndedEvent) { } } + @Override + public void reportEffectMetrics(PersistableBundle bundle) { + if (!effectMetricsReported && mediaMetricsManager != null) { + if (editingSession != null) { + LogSessionId logSessionId = editingSession.getSessionId(); + if (logSessionId != null) { + bundle.putString(KEY_PRIMARY_LOG_SESSION_ID, logSessionId.getStringId()); + } + } + bundle.putInt(BundleSession.KEY_STATSD_ATOM, STATSD_ATOM_MEDIA_PROCESSING_EVENT_REPORTED); + try (BundleSession bundleSession = mediaMetricsManager.createBundleSession()) { + bundleSession.reportBundleMetrics(bundle); + effectMetricsReported = true; + } + } + } + @Override public void close() { if (editingSession != null) { @@ -212,6 +243,36 @@ public LogSessionId getLogSessionId() { } private static final int SUCCESS_PROGRESS_PERCENTAGE = 100; + + // Constants for Atom 1279 (MediaProcessingEventReported) and BundleSession keys. + @VisibleForTesting static final String KEY_PRIMARY_LOG_SESSION_ID = "primary_log_session_id"; + @VisibleForTesting static final String KEY_PROCESSOR_NAME = "processor_name"; + @VisibleForTesting static final String KEY_COMPONENTS = "components"; + @VisibleForTesting static final String KEY_COMPONENT_SCOPES = "component_scopes"; + @VisibleForTesting static final String KEY_METRICS = "metrics"; + @VisibleForTesting static final String KEY_METRIC_VALUES = "metric_values"; + private static final int STATSD_ATOM_MEDIA_PROCESSING_EVENT_REPORTED = 1279; + + // Media3 Component category enums for Atom 1279. + @VisibleForTesting static final int COMPONENT_MEDIA3_EFFECT_COLOR = 100; + @VisibleForTesting static final int COMPONENT_MEDIA3_EFFECT_SPATIAL = 101; + @VisibleForTesting static final int COMPONENT_MEDIA3_EFFECT_CONVOLUTION = 102; + @VisibleForTesting static final int COMPONENT_MEDIA3_EFFECT_OVERLAY = 103; + @VisibleForTesting static final int COMPONENT_MEDIA3_EFFECT_TEMPORAL = 104; + @VisibleForTesting static final int COMPONENT_MEDIA3_EFFECT_BUFFER = 105; + @VisibleForTesting static final int COMPONENT_MEDIA3_EFFECT_CUSTOM = 199; + + @VisibleForTesting static final int COMPONENT_MEDIA3_AUDIO_SPEED_AND_PITCH = 200; + @VisibleForTesting static final int COMPONENT_MEDIA3_AUDIO_CHANNEL_MANIPULATION = 201; + @VisibleForTesting static final int COMPONENT_MEDIA3_AUDIO_TEMPORAL_TRIM = 202; + @VisibleForTesting static final int COMPONENT_MEDIA3_AUDIO_PROCESSOR_CUSTOM = 299; + + // Quantitative Metric keys for Atom 1279. + @VisibleForTesting static final int METRIC_SPEED_MULTIPLIER = 300; + @VisibleForTesting static final int METRIC_ROTATION_DEGREES = 301; + @VisibleForTesting static final int METRIC_FILTER_SAMPLE_COUNT = 302; + @VisibleForTesting static final int METRIC_OVERLAY_COUNT = 303; + private final long startTimeMs; private final boolean compositionHasAudioProcessors; private final boolean compositionHasVideoEffects; @@ -250,9 +311,13 @@ public EditingMetricsCollector( /** * Called when export completes with success. * + * @param composition The exported {@link Composition}. * @param exportResult The {@link ExportResult} of the export. + * @param isExportResumed Whether the export was resumed. */ - public void onExportSuccess(ExportResult exportResult, boolean isExportResumed) { + public void onExportSuccess( + Composition composition, ExportResult exportResult, boolean isExportResumed) { + reportEffectMetrics(composition); EditingEndedEvent.Builder editingEndedEventBuilder = createEditingEndedEventBuilder(EditingEndedEvent.FINAL_STATE_SUCCEEDED) .setFinalProgressPercent(SUCCESS_PROGRESS_PERCENTAGE); @@ -277,16 +342,20 @@ public void onExportSuccess(ExportResult exportResult, boolean isExportResumed) /** * Called when export completes with an error. * + * @param composition The exported {@link Composition}. * @param progressPercentage The progress of the export operation in percent. Value is {@link * C#PERCENTAGE_UNSET} if unknown or between 0 and 100 inclusive. * @param exportException The {@link ExportException} describing the exception. * @param exportResult The {@link ExportResult} of the export. + * @param isExportResumed Whether the export was resumed. */ public void onExportError( + Composition composition, int progressPercentage, ExportException exportException, ExportResult exportResult, boolean isExportResumed) { + reportEffectMetrics(composition); EditingEndedEvent.Builder editingEndedEventBuilder = createEditingEndedEventBuilder(EditingEndedEvent.FINAL_STATE_ERROR) .setErrorCode(getEditingEndedEventErrorCode(exportException.errorCode)); @@ -314,10 +383,12 @@ public void onExportError( /** * Called when export is cancelled. * + * @param composition The exported {@link Composition}. * @param progressPercentage The progress of the export operation in percent. Value is {@link * C#PERCENTAGE_UNSET} if unknown or between 0 and 100 inclusive. */ - public void onExportCancelled(int progressPercentage) { + public void onExportCancelled(Composition composition, int progressPercentage) { + reportEffectMetrics(composition); EditingEndedEvent.Builder editingEndedEventBuilder = createEditingEndedEventBuilder(EditingEndedEvent.FINAL_STATE_CANCELED); if (progressPercentage != C.PERCENTAGE_UNSET) { @@ -516,4 +587,219 @@ private void populateOperationTypes( editingEndedEventBuilder.addOperationType(EditingEndedEvent.OPERATION_TYPE_VIDEO_EDIT); } } + + /** + * Reports video effects and audio processors diagnostic metrics for the given {@link + * Composition}. + */ + private void reportEffectMetrics(Composition composition) { + ExtractedEffectMetrics metrics = extractEffectMetrics(composition); + if (metrics.components.isEmpty()) { + return; + } + + PersistableBundle bundle = new PersistableBundle(); + bundle.putString(KEY_PROCESSOR_NAME, exporterName); + + int[] componentsArray = new int[metrics.components.size()]; + int[] scopesArray = new int[metrics.scopes.size()]; + for (int i = 0; i < metrics.components.size(); i++) { + componentsArray[i] = metrics.components.get(i); + scopesArray[i] = metrics.scopes.get(i); + } + bundle.putIntArray(KEY_COMPONENTS, componentsArray); + bundle.putIntArray(KEY_COMPONENT_SCOPES, scopesArray); + + if (!metrics.metricKeys.isEmpty()) { + int[] metricsArray = new int[metrics.metricKeys.size()]; + long[] metricValuesArray = new long[metrics.metricValues.size()]; + for (int i = 0; i < metrics.metricKeys.size(); i++) { + metricsArray[i] = metrics.metricKeys.get(i); + metricValuesArray[i] = metrics.metricValues.get(i); + } + bundle.putIntArray(KEY_METRICS, metricsArray); + bundle.putLongArray(KEY_METRIC_VALUES, metricValuesArray); + } + + metricsReporter.reportEffectMetrics(bundle); + } + + private static ExtractedEffectMetrics extractEffectMetrics(Composition composition) { + ImmutableList.Builder componentListBuilder = ImmutableList.builder(); + ImmutableList.Builder scopeListBuilder = ImmutableList.builder(); + ImmutableList.Builder metricKeyListBuilder = ImmutableList.builder(); + ImmutableList.Builder metricValueListBuilder = ImmutableList.builder(); + + Set processedTuples = new HashSet<>(); + + // Scope 0: Composition level + extractFromEffects( + composition.effects.videoEffects, + composition.effects.audioProcessors, + /* scope= */ 0, + componentListBuilder, + scopeListBuilder, + metricKeyListBuilder, + metricValueListBuilder, + processedTuples); + + // Scope 1..N: MediaItem level + int itemScope = 1; + for (EditedMediaItemSequence sequence : composition.sequences) { + for (EditedMediaItem item : sequence.editedMediaItems) { + extractFromEffects( + item.effects.videoEffects, + item.effects.audioProcessors, + itemScope++, + componentListBuilder, + scopeListBuilder, + metricKeyListBuilder, + metricValueListBuilder, + processedTuples); + } + } + + return new ExtractedEffectMetrics( + componentListBuilder.build(), + scopeListBuilder.build(), + metricKeyListBuilder.build(), + metricValueListBuilder.build()); + } + + private static void extractFromEffects( + List videoEffects, + List audioProcessors, + int scope, + ImmutableList.Builder componentListBuilder, + ImmutableList.Builder scopeListBuilder, + ImmutableList.Builder metricKeyListBuilder, + ImmutableList.Builder metricValueListBuilder, + Set processedTuples) { + for (Effect effect : videoEffects) { + if (effect instanceof TimestampWrapper) { + effect = ((TimestampWrapper) effect).glEffect; + } + ComponentMetricConsumer consumer = + new ComponentMetricConsumer( + COMPONENT_MEDIA3_EFFECT_CUSTOM, metricKeyListBuilder, metricValueListBuilder); + if (effect instanceof MetricsProvider) { + ((MetricsProvider) effect).populateMetrics(consumer); + } + int componentEnum = consumer.getComponentCategory(); + String tupleKey = componentEnum + ":" + scope; + if (processedTuples.add(tupleKey)) { + componentListBuilder.add(componentEnum); + scopeListBuilder.add(scope); + } + } + for (AudioProcessor audioProcessor : audioProcessors) { + ComponentMetricConsumer consumer = + new ComponentMetricConsumer( + COMPONENT_MEDIA3_AUDIO_PROCESSOR_CUSTOM, + metricKeyListBuilder, + metricValueListBuilder); + if (audioProcessor instanceof MetricsProvider) { + ((MetricsProvider) audioProcessor).populateMetrics(consumer); + } + int componentEnum = consumer.getComponentCategory(); + String tupleKey = componentEnum + ":" + scope; + if (processedTuples.add(tupleKey)) { + componentListBuilder.add(componentEnum); + scopeListBuilder.add(scope); + } + } + } + + private static int mapCategoryToComponentEnum( + @MetricsProvider.MetricConsumer.Category int category) { + switch (category) { + case MetricsProvider.MetricConsumer.CATEGORY_EFFECT_COLOR: + return COMPONENT_MEDIA3_EFFECT_COLOR; + case MetricsProvider.MetricConsumer.CATEGORY_EFFECT_SPATIAL: + return COMPONENT_MEDIA3_EFFECT_SPATIAL; + case MetricsProvider.MetricConsumer.CATEGORY_EFFECT_CONVOLUTION: + return COMPONENT_MEDIA3_EFFECT_CONVOLUTION; + case MetricsProvider.MetricConsumer.CATEGORY_EFFECT_OVERLAY: + return COMPONENT_MEDIA3_EFFECT_OVERLAY; + case MetricsProvider.MetricConsumer.CATEGORY_EFFECT_TEMPORAL: + return COMPONENT_MEDIA3_EFFECT_TEMPORAL; + case MetricsProvider.MetricConsumer.CATEGORY_EFFECT_BUFFER: + return COMPONENT_MEDIA3_EFFECT_BUFFER; + case MetricsProvider.MetricConsumer.CATEGORY_EFFECT_CUSTOM: + return COMPONENT_MEDIA3_EFFECT_CUSTOM; + case MetricsProvider.MetricConsumer.CATEGORY_AUDIO_SPEED_AND_PITCH: + return COMPONENT_MEDIA3_AUDIO_SPEED_AND_PITCH; + case MetricsProvider.MetricConsumer.CATEGORY_AUDIO_CHANNEL_MANIPULATION: + return COMPONENT_MEDIA3_AUDIO_CHANNEL_MANIPULATION; + case MetricsProvider.MetricConsumer.CATEGORY_AUDIO_TEMPORAL_TRIM: + return COMPONENT_MEDIA3_AUDIO_TEMPORAL_TRIM; + case MetricsProvider.MetricConsumer.CATEGORY_AUDIO_PROCESSOR_CUSTOM: + return COMPONENT_MEDIA3_AUDIO_PROCESSOR_CUSTOM; + default: + throw new IllegalArgumentException("Unknown category: " + category); + } + } + + private static int mapMetricKeyToId(@MetricsProvider.MetricConsumer.MetricKey int metricKey) { + switch (metricKey) { + case MetricsProvider.MetricConsumer.METRIC_SPEED_MULTIPLIER: + return METRIC_SPEED_MULTIPLIER; + case MetricsProvider.MetricConsumer.METRIC_ROTATION_DEGREES: + return METRIC_ROTATION_DEGREES; + case MetricsProvider.MetricConsumer.METRIC_FILTER_SAMPLE_COUNT: + return METRIC_FILTER_SAMPLE_COUNT; + case MetricsProvider.MetricConsumer.METRIC_OVERLAY_COUNT: + return METRIC_OVERLAY_COUNT; + default: + throw new IllegalArgumentException("Unknown metric key: " + metricKey); + } + } + + private static final class ComponentMetricConsumer implements MetricsProvider.MetricConsumer { + private final ImmutableList.Builder metricKeyListBuilder; + private final ImmutableList.Builder metricValueListBuilder; + private int componentCategory; + + /* package */ ComponentMetricConsumer( + int defaultCategory, + ImmutableList.Builder metricKeyListBuilder, + ImmutableList.Builder metricValueListBuilder) { + this.componentCategory = defaultCategory; + this.metricKeyListBuilder = metricKeyListBuilder; + this.metricValueListBuilder = metricValueListBuilder; + } + + @Override + public void setCategory(@Category int category) { + this.componentCategory = mapCategoryToComponentEnum(category); + } + + @Override + public void addMetric(@MetricKey int metricKey, long metricValue) { + metricKeyListBuilder.add(mapMetricKeyToId(metricKey)); + metricValueListBuilder.add(metricValue); + } + + /* package */ int getComponentCategory() { + return componentCategory; + } + } + + private static final class ExtractedEffectMetrics { + /* package */ final ImmutableList components; + /* package */ final ImmutableList scopes; + /* package */ final ImmutableList metricKeys; + /* package */ final ImmutableList metricValues; + + /* package */ ExtractedEffectMetrics( + ImmutableList components, + ImmutableList scopes, + ImmutableList metricKeys, + ImmutableList metricValues) { + this.components = components; + this.scopes = scopes; + this.metricKeys = metricKeys; + this.metricValues = metricValues; + } + } } diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/Effects.java b/libraries/transformer/src/main/java/androidx/media3/transformer/Effects.java index e2f1572c5f1..ae88204df11 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/Effects.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/Effects.java @@ -81,6 +81,7 @@ public Effects(List audioProcessors, List videoEffects) * timestamps. * @deprecated Use {@link EditedMediaItem.Builder#setSpeed(SpeedProvider)} instead. */ + @SuppressWarnings("deprecation") // Uses deprecated TimestampAdjustment. @Deprecated public static Pair createExperimentalSpeedChangingEffect( SpeedProvider speedProvider) { diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/EncoderFrameWriter.java b/libraries/transformer/src/main/java/androidx/media3/transformer/EncoderFrameWriter.java index d08841d20f9..0c07d3b72fa 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/EncoderFrameWriter.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/EncoderFrameWriter.java @@ -173,6 +173,7 @@ public void queueInputFrame(Frame frame, @Nullable SyncFenceWrapper writeComplet checkState(imageWriter != null); checkArgument(frame instanceof DefaultHardwareBufferFrame); DefaultHardwareBufferFrame hardwareBufferFrame = (DefaultHardwareBufferFrame) frame; + hardwareBufferFrame.getHardwareBuffer().close(); @Nullable Image image = (Image) hardwareBufferFrame.getInternalImage(); checkArgument(image != null); // The downstream encoder uses the image timestamp as the presentation timestamp. @@ -182,6 +183,8 @@ public void queueInputFrame(Frame frame, @Nullable SyncFenceWrapper writeComplet try { image.setFence(writeCompleteFence.asSyncFence()); } catch (IOException e) { + image.close(); + writeCompleteFence.close(); listenerExecutor.execute(() -> listener.onError(VideoFrameProcessingException.from(e))); return; } diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/EncoderUtil.java b/libraries/transformer/src/main/java/androidx/media3/transformer/EncoderUtil.java index fede8318d40..180243b1be2 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/EncoderUtil.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/EncoderUtil.java @@ -17,6 +17,7 @@ package androidx.media3.transformer; import static android.os.Build.VERSION.SDK_INT; +import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkNotNull; import static java.lang.Math.floor; import static java.lang.Math.max; @@ -44,6 +45,9 @@ import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableSet; import com.google.common.primitives.Ints; +import java.util.Locale; +import java.util.regex.Matcher; +import java.util.regex.Pattern; /** Utility methods for {@link MediaCodec} encoders. */ @UnstableApi @@ -52,6 +56,11 @@ public final class EncoderUtil { /** A value to indicate the encoding level is not set. */ public static final int LEVEL_UNSET = Format.NO_VALUE; + private static final String TEMPORAL_LAYERING_SCHEMA_NONE = "none"; + private static final String TEMPORAL_LAYERING_SCHEMA_WEBRTC_PREFIX = "webrtc."; + private static final Pattern TEMPORAL_LAYERING_SCHEMA_PATTERN = + Pattern.compile("^android\\.generic\\.([1-9]\\d*)(?:\\+(\\d+))?$"); + @GuardedBy("EncoderUtil.class") private static final ArrayListMultimap mimeTypeToEncoders = ArrayListMultimap.create(); @@ -362,6 +371,74 @@ public static boolean isBitrateModeSupported( return encoderCapabilities.isBitrateModeSupported(bitrateMode); } + /** + * Returns the {@link android.media.MediaFormat#KEY_TEMPORAL_LAYERING} schema string for the given + * temporal layer counts. + * + *

    Returns {@code "none"} if {@code numNonBidirectionalLayers} is zero, {@code + * "android.generic.N"} if {@code numBidirectionalLayers} is zero, and {@code + * "android.generic.N+M"} otherwise. + * + * @param numNonBidirectionalLayers The number of non-bidirectional temporal layers. Must be + * non-negative. + * @param numBidirectionalLayers The number of bidirectional temporal layers. Must be non-negative + * if {@code numNonBidirectionalLayers} is positive, and zero or {@link Format#NO_VALUE} if it + * is zero. + * @throws IllegalArgumentException If {@code numNonBidirectionalLayers} is negative, if it is + * positive and {@code numBidirectionalLayers} is negative, or if it is zero and {@code + * numBidirectionalLayers} is neither zero nor {@link Format#NO_VALUE}. + */ + public static String toTemporalLayeringSchema( + int numNonBidirectionalLayers, int numBidirectionalLayers) { + checkArgument(numNonBidirectionalLayers >= 0); + if (numNonBidirectionalLayers == 0) { + checkArgument(numBidirectionalLayers == 0 || numBidirectionalLayers == Format.NO_VALUE); + return TEMPORAL_LAYERING_SCHEMA_NONE; + } + checkArgument(numBidirectionalLayers >= 0); + if (numBidirectionalLayers > 0) { + return String.format( + Locale.ROOT, "android.generic.%d+%d", numNonBidirectionalLayers, numBidirectionalLayers); + } + return String.format(Locale.ROOT, "android.generic.%d", numNonBidirectionalLayers); + } + + /** + * Parses a {@link android.media.MediaFormat#KEY_TEMPORAL_LAYERING} schema string into a {@link + * Pair} of the {@linkplain VideoEncoderSettings#numNonBidirectionalTemporalLayers number of + * non-bidirectional temporal layers} and the {@linkplain + * VideoEncoderSettings#numBidirectionalTemporalLayers number of bidirectional temporal layers}. + * + *

    This method only supports {@code "none"}, which is parsed as zero layers, and the {@code + * "android.generic.N[+M]"} schemas defined by {@link + * android.media.MediaFormat#KEY_TEMPORAL_LAYERING}. + * + * @param schema The temporal layering schema string. + * @throws UnsupportedOperationException If the schema is a {@code "webrtc.*"} schema. + * @throws IllegalArgumentException If the schema is malformed or a layer count does not fit in an + * int. + */ + public static Pair parseTemporalLayeringSchema(String schema) { + if (schema.equals(TEMPORAL_LAYERING_SCHEMA_NONE)) { + return Pair.create(0, 0); + } + if (schema.startsWith(TEMPORAL_LAYERING_SCHEMA_WEBRTC_PREFIX)) { + throw new UnsupportedOperationException("Unsupported temporal layering schema: " + schema); + } + Matcher matcher = TEMPORAL_LAYERING_SCHEMA_PATTERN.matcher(schema); + checkArgument(matcher.matches(), "Malformed temporal layering schema: %s", schema); + try { + int numNonBidirectionalLayers = Integer.parseInt(checkNotNull(matcher.group(1))); + @Nullable String bidirectionalLayers = matcher.group(2); + int numBidirectionalLayers = + bidirectionalLayers == null ? 0 : Integer.parseInt(bidirectionalLayers); + return Pair.create(numNonBidirectionalLayers, numBidirectionalLayers); + } catch (NumberFormatException e) { + throw new IllegalArgumentException( + "Temporal layer count does not fit in an int: " + schema, e); + } + } + /** * Returns a {@link ImmutableList list} of supported {@linkplain * MediaCodecInfo.CodecCapabilities#colorFormats color formats} for the given {@linkplain diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/ExoPlayerAssetLoader.java b/libraries/transformer/src/main/java/androidx/media3/transformer/ExoPlayerAssetLoader.java index d60ef273dd8..f81ba0fde25 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/ExoPlayerAssetLoader.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/ExoPlayerAssetLoader.java @@ -297,10 +297,6 @@ private ExoPlayerAssetLoader( .setStuckPlayingDetectionTimeoutMs(Integer.MAX_VALUE) .setStuckPlayingNotEndingTimeoutMs(Integer.MAX_VALUE) .setUsePlatformDiagnostics(false); - if (decoderFactory instanceof DefaultDecoderFactory) { - playerBuilder.experimentalSetDynamicSchedulingEnabled( - ((DefaultDecoderFactory) decoderFactory).isDynamicSchedulingEnabled()); - } if (clock != Clock.DEFAULT) { // Transformer.Builder#setClock is also @VisibleForTesting, so if we're using a non-default // clock we must be in a test context. diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/ForwardingEncoderFactory.java b/libraries/transformer/src/main/java/androidx/media3/transformer/ForwardingEncoderFactory.java new file mode 100644 index 00000000000..192c16e9208 --- /dev/null +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/ForwardingEncoderFactory.java @@ -0,0 +1,61 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import android.media.metrics.LogSessionId; +import androidx.annotation.Nullable; +import androidx.media3.common.Format; +import androidx.media3.transformer.Codec.EncoderFactory; + +/** + * An {@link EncoderFactory} implementation that forwards all calls to another {@link + * EncoderFactory} instance. + */ +/* package */ class ForwardingEncoderFactory implements EncoderFactory { + + private final EncoderFactory factory; + + public ForwardingEncoderFactory(EncoderFactory factory) { + this.factory = factory; + } + + @Override + public Codec createForAudioEncoding(Format format, @Nullable LogSessionId logSessionId) + throws ExportException { + return factory.createForAudioEncoding(format, logSessionId); + } + + @Override + public Codec createForVideoEncoding(Format format, @Nullable LogSessionId logSessionId) + throws ExportException { + return factory.createForVideoEncoding(format, logSessionId); + } + + @Override + public boolean isVideoFormatSupported(Format format) { + return factory.isVideoFormatSupported(format); + } + + @Override + public boolean audioNeedsEncoding() { + return factory.audioNeedsEncoding(); + } + + @Override + public boolean videoNeedsEncoding() { + return factory.videoNeedsEncoding(); + } +} diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/FrameAggregator.java b/libraries/transformer/src/main/java/androidx/media3/transformer/FrameAggregator.java index aaa53db3e26..1817960ba4d 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/FrameAggregator.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/FrameAggregator.java @@ -16,90 +16,161 @@ package androidx.media3.transformer; +import static androidx.media3.transformer.TransformerUtil.END_OF_STREAM_ASYNC_FRAME; import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.base.Preconditions.checkState; +import static java.lang.Math.max; +import static java.lang.annotation.ElementType.TYPE_USE; +import static java.lang.annotation.RetentionPolicy.SOURCE; +import android.util.Rational; +import androidx.annotation.IntDef; import androidx.annotation.Nullable; +import androidx.media3.common.C; +import androidx.media3.common.Format; import androidx.media3.common.util.Consumer; -import androidx.media3.effect.HardwareBufferFrame; +import androidx.media3.common.util.Util; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.DefaultHardwareBufferFrame; +import androidx.media3.common.video.Frame; import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import com.google.common.collect.Iterables; +import java.lang.annotation.Documented; +import java.lang.annotation.Retention; +import java.lang.annotation.Target; +import java.math.RoundingMode; import java.util.ArrayDeque; import java.util.ArrayList; import java.util.List; import java.util.Queue; /** - * Combines multiple sequences of {@link HardwareBufferFrame}s into one sequence of {@link - * ImmutableList}. + * Combines multiple sequences of {@link AsyncFrame}s into one sequence of {@link + * ImmutableList}. */ /* package */ class FrameAggregator implements AutoCloseable { - private final Consumer> downstreamConsumer; + + /** Defines how frames from a registered sequence are used during aggregation. */ + @Documented + @Retention(SOURCE) + @Target(TYPE_USE) + @IntDef({ + STRATEGY_EXPECT_NO_FRAMES, + STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET, + STRATEGY_MATCH_FRAME_CLOSEST_TO_TARGET + }) + /* package */ @interface AggregationStrategy {} + + /** + * Expects no frames from the sequence, so the aggregator never waits for it. + * + *

    Used for sequences with no video track. + */ + /* package */ static final int STRATEGY_EXPECT_NO_FRAMES = 1; + + /** Matches the first frame with a timestamp at or after the target timestamp. */ + /* package */ static final int STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET = 2; + + /** + * Matches whichever of the preceding and following frames is closest to the target timestamp, + * breaking ties in favor of the preceding frame. + */ + /* package */ static final int STRATEGY_MATCH_FRAME_CLOSEST_TO_TARGET = 3; + + private final Consumer> downstreamConsumer; private final Consumer onFlush; private final List inputFrameQueues; private final int numSequences; + + @Nullable private final Rational frameRate; + private volatile boolean isEnded; private volatile boolean isClosed; + // Index of the next virtual reference tick, or C.INDEX_UNSET if awaiting initial frame. + private volatile long nextVirtualFrameIndex; + @Nullable private volatile AsyncFrame cachedVirtualFrame; + private volatile long cachedVirtualFrameIndex; /** * Creates a new {@link FrameAggregator}. * - * @param numSequences The number of sequences. - * @param downstreamConsumer Receives the aggregated {@linkplain - * ImmutableList frames}. + *

    When {@code frameRate} is {@code null}, secondary sequences are aligned to the presentation + * timestamps of frames queued to the primary sequence (at index 0), and aggregation ends when the + * primary sequence ends. + * + *

    When {@code frameRate} is set, an internal virtual clock generates reference ticks at the + * requested rate, and all sequences are retimed and aligned to these reference ticks. In this + * case, aggregation ends when all active sequences have completed and drained. + * + * @param numSequences The number of sequences to expect frames from. + * @param frameRate The target frame rate in frames per second, or {@code null} to use the primary + * sequence as the reference timeline. + * @param downstreamConsumer Receives the aggregated {@linkplain ImmutableList + * frames}. * @param onFlush Callback triggered when {@link #flush(int)} is called. - * @throws IllegalArgumentException If {@code numSequences} is less than 1. + * @throws IllegalArgumentException If {@code numSequences} is less than 1, or if {@code + * frameRate} has a zero or negative numerator or denominator. */ - public FrameAggregator( + /* package */ FrameAggregator( int numSequences, - Consumer> downstreamConsumer, + @Nullable Rational frameRate, + Consumer> downstreamConsumer, Consumer onFlush) { checkArgument(numSequences > 0, "numSequences must be at least 1."); + checkArgument( + frameRate == null || (frameRate.getNumerator() > 0 && frameRate.getDenominator() > 0)); this.numSequences = numSequences; + this.frameRate = frameRate; this.downstreamConsumer = downstreamConsumer; this.onFlush = onFlush; inputFrameQueues = new ArrayList<>(); for (int i = 0; i < numSequences; i++) { inputFrameQueues.add(new FrameQueue()); } + nextVirtualFrameIndex = C.INDEX_UNSET; + cachedVirtualFrame = null; + cachedVirtualFrameIndex = C.INDEX_UNSET; } /** - * Registers the given {@code sequenceIndex} with the {@link FrameAggregator}, and indicates - * whether it should be considered when aggregating frames. + * Registers the given {@code sequenceIndex} with the {@link FrameAggregator}. * *

    All sequences must be registered before frames are queued. * + * @param sequenceIndex The index of the sequence to register. + * @param aggregationStrategy The {@link AggregationStrategy} for this sequence. * @throws IllegalArgumentException If {@code sequenceIndex} is negative or greater than or equal * to the number of sequences. */ - public void registerSequence(int sequenceIndex, boolean shouldAggregate) { + public void registerSequence(int sequenceIndex, @AggregationStrategy int aggregationStrategy) { checkArgument(sequenceIndex >= 0); checkArgument(sequenceIndex < numSequences); checkState(!isClosed); - inputFrameQueues.get(sequenceIndex).initialize(shouldAggregate); + inputFrameQueues.get(sequenceIndex).initialize(aggregationStrategy); } /** - * Queues a {@link HardwareBufferFrame} at the given sequence. + * Queues an {@link AsyncFrame} at the given sequence. * - *

    Once called, the caller must not modify the {@link HardwareBufferFrame}. + *

    Once called, the caller must not modify the {@link AsyncFrame}. * *

    If the aggregator {@link #isEnded}, the frame will be released immediately. * - * @param frame The {@link HardwareBufferFrame} to queue. - * @param sequenceIndex The index of the sequence the queued {@link HardwareBufferFrame} is from. + * @param frame The {@link AsyncFrame} to queue. + * @param sequenceIndex The index of the sequence the queued {@link AsyncFrame} is from. * @throws IllegalArgumentException If {@code sequenceIndex} is negative or greater than or equal * to the number of sequences. */ - public void queueFrame(HardwareBufferFrame frame, int sequenceIndex) { + public void queueFrame(AsyncFrame frame, int sequenceIndex) { checkArgument(sequenceIndex >= 0); checkArgument(sequenceIndex < numSequences); checkState(inputFrameQueues.get(sequenceIndex).isRegistered); // Release frames immediately if the primary sequence has already ended, or the aggregator is // closed. if (isEnded || isClosed) { - frame.release(/* releaseFence= */ null); + TransformerUtil.releaseIfNeeded(frame.frame, /* releaseFence= */ null); return; } inputFrameQueues.get(sequenceIndex).frames.add(frame); @@ -140,86 +211,235 @@ public void flush(int sequenceIndex) { if (isClosed) { return; } - @Nullable HardwareBufferFrame nextFrame; + @Nullable AsyncFrame nextFrame; while ((nextFrame = inputFrameQueues.get(sequenceIndex).frames.poll()) != null) { - nextFrame.release(/* releaseFence= */ null); + TransformerUtil.releaseIfNeeded(nextFrame.frame, /* releaseFence= */ null); } + // CompositionPlayer does not support independent sequence flushing and always flushes all + // sequences iteratively. We can just reset the global isEnded state when sequence 0 is flushed. if (sequenceIndex == 0) { isEnded = false; } inputFrameQueues.get(sequenceIndex).setIsEnded(/* isEnded= */ false); + // The virtual clock index is currently cleared whenever any sequence is flushed. This is + // safe because CompositionPlayer does not support independent sequence flushing and always + // flushes all sequences iteratively. If independent flushing is supported in the future, + // this behavior may need to be re-evaluated based on the new requirements. + if (frameRate != null) { + nextVirtualFrameIndex = C.INDEX_UNSET; + clearCachedVirtualFrame(); + } onFlush.accept(sequenceIndex); } /** - * {@linkplain HardwareBufferFrame#release Releases } all frames that have not been sent - * downstream, and forces this instance to immediately release any newly queued frames. + * Releases all frames that have not been sent downstream, and forces this instance to immediately + * release any newly queued frames. */ @Override public void close() { isClosed = true; for (int i = 0; i < inputFrameQueues.size(); i++) { - @Nullable HardwareBufferFrame nextFrame; + @Nullable AsyncFrame nextFrame; while ((nextFrame = inputFrameQueues.get(i).frames.poll()) != null) { - nextFrame.release(/* releaseFence= */ null); + TransformerUtil.releaseIfNeeded(nextFrame.frame, /* releaseFence= */ null); } } + clearCachedVirtualFrame(); } /** * Selects the next frame greater than or equal to the current frame from the primary input stream * from each secondary stream. */ + @SuppressWarnings("NonAtomicVolatileUpdate") private void maybeAggregate() { while (!isEnded) { - // TODO: b/496904840 - This assumes the primary sequence can be aggregated against. Update it - // to when this is not the case. - @Nullable - HardwareBufferFrame nextPrimaryFrame = checkNotNull(inputFrameQueues.get(0).frames).peek(); - if (nextPrimaryFrame == null) { + @Nullable AsyncFrame referenceFrame = getNextReferenceFrame(); + if (referenceFrame == null) { handlePrimaryEndOfStream(); return; } + if (referenceFrame == END_OF_STREAM_ASYNC_FRAME) { + downstreamConsumer.accept(ImmutableList.of(END_OF_STREAM_ASYNC_FRAME)); + isEnded = true; + return; + } - ImmutableList matches = findMatchingSecondaryFrames(nextPrimaryFrame); + ImmutableList matches = findMatchingFrames(referenceFrame); if (matches == null) { return; } - ImmutableList.Builder outputFramesBuilder = - new ImmutableList.Builder<>(); - outputFramesBuilder.add(nextPrimaryFrame); + ImmutableList.Builder outputFramesBuilder = new ImmutableList.Builder<>(); + if (frameRate == null) { + outputFramesBuilder.add(referenceFrame); + } // Retain the matched secondary frames. We need to reference count them because // the same secondary frame might be reused for multiple future primary frames. - for (HardwareBufferFrame match : matches) { - outputFramesBuilder.add(match.retain()); + for (int i = 0; i < matches.size(); i++) { + checkState(matches.get(i).frame instanceof DefaultHardwareBufferFrame); + DefaultHardwareBufferFrame matchedFrame = (DefaultHardwareBufferFrame) matches.get(i).frame; + DefaultHardwareBufferFrame.Builder frameBuilder = + matchedFrame.buildUpon().shouldIncrementReferenceCount(); + // In the frameRate-unset case, physical Sequence 0 defines the primary output presentation + // timestamp. Under a virtual clock, primary track frames may be duplicated and those new + // frames must be retimed. We retime all the frames here for the sake of consistency and + // simplicity. Note that the maximum timestamp shift is bounded by the inter-frame + // duration of the track. + if (frameRate != null) { + long referenceContentTimeUs = referenceFrame.frame.getContentTimeUs(); + long shiftDeltaUs = referenceContentTimeUs - matchedFrame.getContentTimeUs(); + long matchPresentationTimeUs = + (Long) checkNotNull(matchedFrame.getMetadata().get(Frame.KEY_PRESENTATION_TIME_US)); + long newPresentationTimeUs = matchPresentationTimeUs + shiftDeltaUs; + ImmutableMap newMetadata = + ImmutableMap.builder() + .putAll(matchedFrame.getMetadata()) + // TODO(b/553446699): Verify that KEY_PRESENTATION_TIME_US is used correctly. + .put(Frame.KEY_PRESENTATION_TIME_US, newPresentationTimeUs) + .put(Frame.KEY_DISPLAY_TIME_NS, referenceContentTimeUs * 1_000L) + .buildKeepingLast(); + frameBuilder.setContentTimeUs(referenceContentTimeUs).setMetadata(newMetadata); + } + outputFramesBuilder.add(new AsyncFrame(frameBuilder.build(), matches.get(i).acquireFence)); + } + if (frameRate == null) { + // Remove the primary frame because each one is only matched once. + checkNotNull(inputFrameQueues.get(0).frames).poll(); + } else { + // Only advance the virtual clock tick once all input queues are ready (matches != null) + // and an output packet is successfully assembled. nextVirtualFrameIndex is guaranteed + // to be initialized by getNextReferenceFrame(). + checkState(nextVirtualFrameIndex != C.INDEX_UNSET); + nextVirtualFrameIndex++; + } + ImmutableList outputFrames = outputFramesBuilder.build(); + if (outputFrames.isEmpty()) { + downstreamConsumer.accept(ImmutableList.of(END_OF_STREAM_ASYNC_FRAME)); + isEnded = true; + return; } - // Remove the primary frame because each one is only matched once. - checkNotNull(inputFrameQueues.get(0).frames).poll(); - downstreamConsumer.accept(outputFramesBuilder.build()); + downstreamConsumer.accept(outputFrames); } } + /** + * Returns the next reference frame. + * + *

    When a target frame rate is set, this synthesizes a virtual reference frame ticking at the + * requested rate. Otherwise, this returns the physical frame sitting at the head of Sequence 0. + * + * @return The next reference frame, or {@code null} if awaiting input data, or {@link + * TransformerUtil#END_OF_STREAM_ASYNC_FRAME} if all active sequences have fully completed. + */ + @Nullable + private AsyncFrame getNextReferenceFrame() { + if (frameRate != null) { + if (isAllSequencesEndedAndEmpty()) { + return END_OF_STREAM_ASYNC_FRAME; + } + if (nextVirtualFrameIndex == C.INDEX_UNSET) { + long firstAvailableTimeUs = getFirstAvailableFrameTimeUs(); + if (firstAvailableTimeUs == C.TIME_UNSET) { + return null; // Await initial frame. + } + // By ceiling the index, we guarantee that the corresponding virtual tick's timestamp is + // always >= firstAvailableTimeUs. This ensures that the virtual clock never produces frames + // with timestamps earlier than the current available frames (e.g. before the seek + // position). + nextVirtualFrameIndex = getVirtualFrameIndexCeil(firstAvailableTimeUs, frameRate); + } + return getOrGenerateVirtualFrame(); + } else { + // TODO: b/496904840 - This assumes the primary sequence can be aggregated against. Update it + // to when this is not the case. + return checkNotNull(inputFrameQueues.get(0).frames).peek(); + } + } + + @Nullable + private AsyncFrame getOrGenerateVirtualFrame() { + if (isClosed || frameRate == null) { + return null; + } + if (cachedVirtualFrame != null && cachedVirtualFrameIndex == nextVirtualFrameIndex) { + return cachedVirtualFrame; + } + long frameDurationUsNumerator = 1_000_000L * frameRate.getDenominator(); + long frameDurationUsDenominator = frameRate.getNumerator(); + long targetTimeUs = + Util.scaleLargeValue( + /* value= */ nextVirtualFrameIndex, + /* multiplier= */ frameDurationUsNumerator, + /* divisor= */ frameDurationUsDenominator, + RoundingMode.HALF_UP); + AsyncFrame newFrame = new AsyncFrame(new VirtualFrame(targetTimeUs), /* acquireFence= */ null); + cachedVirtualFrame = newFrame; + cachedVirtualFrameIndex = nextVirtualFrameIndex; + return newFrame; + } + + private void clearCachedVirtualFrame() { + if (cachedVirtualFrame != null) { + cachedVirtualFrame = null; + cachedVirtualFrameIndex = C.INDEX_UNSET; + } + } + + /** Returns whether all active input queues are ended and empty. */ + private boolean isAllSequencesEndedAndEmpty() { + for (int i = 0; i < numSequences; i++) { + FrameQueue queue = inputFrameQueues.get(i); + if (!queue.getIsEnded() || !queue.frames.isEmpty()) { + return false; + } + } + return true; + } + + /** + * Returns the earliest presentation time across all active input queues, or {@link C#TIME_UNSET} + * if no frames are currently available. + */ + private long getFirstAvailableFrameTimeUs() { + long firstAvailableTimeUs = Long.MAX_VALUE; + for (int i = 0; i < numSequences; i++) { + FrameQueue queue = inputFrameQueues.get(i); + if (!queue.getIsEnded() && queue.frames.isEmpty()) { + // Must wait for all active sequences to provide their first frame. + return C.TIME_UNSET; + } + if (!queue.frames.isEmpty()) { + firstAvailableTimeUs = + Math.min( + firstAvailableTimeUs, checkNotNull(queue.frames.peek()).frame.getContentTimeUs()); + } + } + return firstAvailableTimeUs == Long.MAX_VALUE ? C.TIME_UNSET : firstAvailableTimeUs; + } + private void handlePrimaryEndOfStream() { - // When the primary sequence ends, send an EOS frame downstream. - if (inputFrameQueues.get(0).isEnded) { - downstreamConsumer.accept(ImmutableList.of(HardwareBufferFrame.END_OF_STREAM_FRAME)); + // Without a target frame rate, ending Sequence 0 ends the composition. + if (frameRate == null && inputFrameQueues.get(0).isEnded) { + downstreamConsumer.accept(ImmutableList.of(END_OF_STREAM_ASYNC_FRAME)); isEnded = true; } } /** - * Finds matching frames from secondary queues for the given primary frame. Returns null if any - * secondary queue is not yet at or past the primary frame's timestamp and is not yet ended. + * Finds matching frames from the input queues for the given reference frame. Returns null if any + * active queue is not yet at or past the reference frame's timestamp and is not yet ended. */ @Nullable - private ImmutableList findMatchingSecondaryFrames( - HardwareBufferFrame nextPrimaryFrame) { - ImmutableList.Builder matchesBuilder = ImmutableList.builder(); - for (int i = 1; i < numSequences; i++) { + private ImmutableList findMatchingFrames(AsyncFrame referenceFrame) { + ImmutableList.Builder matchesBuilder = ImmutableList.builder(); + // In virtual-reference mode, align all physical sequences (index 0+). + // In physical-reference mode (Sequence 0), only match secondary sequences (index 1+). + int firstSequenceToMatch = frameRate != null ? 0 : 1; + for (int i = firstSequenceToMatch; i < numSequences; i++) { FrameQueue secondaryQueue = inputFrameQueues.get(i); - @Nullable - HardwareBufferFrame matchingFrame = secondaryQueue.getMatchingFrame(nextPrimaryFrame); + @Nullable AsyncFrame matchingFrame = secondaryQueue.getMatchingFrame(referenceFrame); if (matchingFrame == null) { if (!secondaryQueue.getIsEnded()) { // Need to wait for more frames in this secondary queue. @@ -234,23 +454,44 @@ private ImmutableList findMatchingSecondaryFrames( return matchesBuilder.build(); } - /** A helper class representing a {@link Queue} that can end. */ + /** + * Returns the index of the virtual tick that occurs at or immediately following the given {@code + * timeUs}, or {@link C#INDEX_UNSET} if {@code frameRate} is {@code null} or {@code timeUs} is + * {@link C#TIME_UNSET}. + */ + private static long getVirtualFrameIndexCeil(long timeUs, @Nullable Rational frameRate) { + if (frameRate == null || timeUs == C.TIME_UNSET) { + return C.INDEX_UNSET; + } + // Virtual clock ticks represent fractional values (e.g., 66_666.666... us at 30 fps), but + // upstream timestamps are integer-quantized and may be rounded up to the next microsecond + // (e.g., 66_667 us). Strict ceiling division on such rounded-up timestamps would overshoot to + // the next virtual tick (e.g., 100_000 us). Subtract 1 us to absorb this quantization error. + long adjustedTimeUs = Math.max(0, timeUs - 1); + return Util.scaleLargeValue( + /* value= */ adjustedTimeUs, + /* multiplier= */ frameRate.getNumerator(), + /* divisor= */ 1_000_000L * frameRate.getDenominator(), + RoundingMode.CEILING); + } + + /** A helper class representing a {@link Queue} that can end. */ private static class FrameQueue { - final Queue frames; + final Queue frames; private boolean isRegistered; - private boolean shouldAggregate; + private @AggregationStrategy int aggregationStrategy; private boolean isEnded; FrameQueue() { frames = new ArrayDeque<>(); // Force FrameAggregator to wait for frames for this sequence, until initialize is called. - shouldAggregate = true; + aggregationStrategy = STRATEGY_MATCH_FRAME_CLOSEST_TO_TARGET; } - void initialize(boolean shouldAggregate) { + void initialize(@AggregationStrategy int aggregationStrategy) { checkState(!isRegistered); this.isRegistered = true; - this.shouldAggregate = shouldAggregate; + this.aggregationStrategy = aggregationStrategy; } void setIsEnded(boolean isEnded) { @@ -259,33 +500,45 @@ void setIsEnded(boolean isEnded) { boolean getIsEnded() { checkState(isRegistered); - return isEnded || !shouldAggregate; + return isEnded || aggregationStrategy == STRATEGY_EXPECT_NO_FRAMES; } /** * Finds the best matching frame for the given target frame's sequence presentation timestamp. * - *

    This method iterates through the frame queue, releasing any frames with presentation - * timestamps strictly less than the {@code targetFrame}'s sequencePresentationTimeUs. This - * ensures that we don't hold onto frames that can no longer be used for matching. + *

    When {@code aggregationStrategy} is {@link #STRATEGY_MATCH_FRAME_CLOSEST_TO_TARGET}, + * retains the previous frame relative to {@code targetFrame} and selects the closer frame + * between the previous and following frames, breaking ties in favor of the previous frame. * - *

    The first frame remaining in the queue, which will have a presentation timestamp greater - * than or equal to the {@code targetFrame}'s sequencePresentationTimeUs, is considered the - * match. This frame is not removed from the queue by this method, as it might be needed to + *

    When {@code aggregationStrategy} is {@link #STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET}, this + * method iterates through the frame queue, releasing any frames with presentation timestamps + * strictly less than {@code targetFrame}'s sequence presentation timestamp. The first remaining + * frame (with timestamp greater than or equal to {@code targetFrame}) is considered the match. + * + *

    Matched frames are not removed from the queue by this method, as they might be needed to * match subsequent primary frames (i.e., upsampling the secondary stream). * *

    This matching strategy is deterministic: for a given state of the queue and a given {@code - * targetFrame}, the result of this method will always be the same. It's designed to be - * efficient by only retaining the necessary frames in the secondary queue – effectively keeping - * only the secondary frames that are current or in the future relative to the last processed - * primary frame timestamp. + * targetFrame}, the result of this method will always be the same. It is designed to be + * efficient by keeping only secondary frames with timestamps greater than or equal to {@code + * targetFrame}, plus at most one preceding frame when {@code aggregationStrategy} is {@link + * #STRATEGY_MATCH_FRAME_CLOSEST_TO_TARGET}. * *

    Corner Cases: * *

      - *
    • If the queue is empty, returns null. - *
    • If all frames in the queue are older than {@code targetFrame}, all frames are released, - * and it returns null. + *
    • If the queue is empty, returns {@code null}. + *
    • If all frames in the queue are older than {@code targetFrame}, returns {@code null}: + *
        + *
      • When {@code aggregationStrategy} is {@link + * #STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET}, all queued frames are released + * immediately. + *
      • When {@code aggregationStrategy} is {@link + * #STRATEGY_MATCH_FRAME_CLOSEST_TO_TARGET}, all queued frames except the most + * recent one are released; the most recent frame remains queued to be compared + * against a future frame (or until released by {@link #flush(int)} or {@link + * #close()}). + *
      *
    * * @param targetFrame The primary frame for which to find a matching secondary frame. @@ -293,24 +546,105 @@ boolean getIsEnded() { * available. */ @Nullable - private HardwareBufferFrame getMatchingFrame(HardwareBufferFrame targetFrame) { - long targetTime = targetFrame.sequencePresentationTimeUs; + private AsyncFrame getMatchingFrame(AsyncFrame targetFrame) { + long targetTimeUs = targetFrame.frame.getContentTimeUs(); if (frames.isEmpty()) { return null; } + if (aggregationStrategy == STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET) { + discardAllFramesBefore(targetTimeUs); + return frames.peek(); + } + + discardPrecedingFramesExceptMostRecent(targetTimeUs); + + AsyncFrame firstFrame = checkNotNull(frames.peek()); + if (firstFrame.frame.getContentTimeUs() >= targetTimeUs) { + return firstFrame; + } + + if (frames.size() >= 2) { + AsyncFrame previousFrame = firstFrame; + AsyncFrame nextFrame = Iterables.get(frames, 1); + long previousDiffUs = targetTimeUs - previousFrame.frame.getContentTimeUs(); + long nextDiffUs = nextFrame.frame.getContentTimeUs() - targetTimeUs; + if (nextDiffUs < previousDiffUs) { + frames.poll(); + TransformerUtil.releaseIfNeeded(previousFrame.frame, /* releaseFence= */ null); + return nextFrame; + } + return previousFrame; + } + + return null; + } + + /** + * Discards all queued frames with presentation timestamps strictly less than {@code + * targetTimeUs}. + */ + private void discardAllFramesBefore(long targetTimeUs) { while (!frames.isEmpty()) { - HardwareBufferFrame nextFrame = checkNotNull(frames.peek()); - if (nextFrame.sequencePresentationTimeUs < targetTime) { + AsyncFrame asyncFrame = checkNotNull(frames.peek()); + if (asyncFrame.frame.getContentTimeUs() < targetTimeUs) { frames.poll(); - nextFrame.release(/* releaseFence= */ null); + TransformerUtil.releaseIfNeeded(asyncFrame.frame, /* releaseFence= */ null); + } else { + break; + } + } + } + + /** + * Discards all frames with presentation timestamp <= {@code targetTimeUs} except the most + * recent one. + * + *

    Any older frames are discarded because they are strictly farther from {@code targetTimeUs} + * than the retained frame. + */ + private void discardPrecedingFramesExceptMostRecent(long targetTimeUs) { + int precedingFrameCount = 0; + for (AsyncFrame asyncFrame : frames) { + if (asyncFrame.frame.getContentTimeUs() <= targetTimeUs) { + precedingFrameCount++; } else { - // Found the first frame >= targetTime break; } } + int framesToDiscard = max(0, precedingFrameCount - 1); + for (int i = 0; i < framesToDiscard; i++) { + AsyncFrame asyncFrame = checkNotNull(frames.poll()); + TransformerUtil.releaseIfNeeded(asyncFrame.frame, /* releaseFence= */ null); + } + } + } + + private static final class VirtualFrame implements Frame { + private final Format format = new Format.Builder().build(); + private final ImmutableMap metadata; + private final long contentTimeUs; + + VirtualFrame(long targetTimeUs) { + this.contentTimeUs = targetTimeUs; + this.metadata = + ImmutableMap.of( + KEY_PRESENTATION_TIME_US, targetTimeUs, KEY_DISPLAY_TIME_NS, targetTimeUs * 1_000L); + } + + @Override + public Format getFormat() { + return format; + } + + @Override + public ImmutableMap getMetadata() { + return metadata; + } - return frames.peek(); + @Override + public long getContentTimeUs() { + return contentTimeUs; } } } diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/FrameWriterToHardwareBufferFrameQueueAdapter.java b/libraries/transformer/src/main/java/androidx/media3/transformer/FrameWriterToHardwareBufferFrameQueueAdapter.java deleted file mode 100644 index 72afe5ccb4a..00000000000 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/FrameWriterToHardwareBufferFrameQueueAdapter.java +++ /dev/null @@ -1,137 +0,0 @@ -/* - * Copyright 2026 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package androidx.media3.transformer; - -import static androidx.media3.common.video.Frame.KEY_PRESENTATION_TIME_US; -import static com.google.common.base.Preconditions.checkNotNull; -import static com.google.common.base.Preconditions.checkState; -import static com.google.common.util.concurrent.MoreExecutors.directExecutor; - -import android.hardware.HardwareBuffer; -import androidx.annotation.Nullable; -import androidx.annotation.RequiresApi; -import androidx.media3.common.C; -import androidx.media3.common.Format; -import androidx.media3.common.util.ExperimentalApi; -import androidx.media3.common.video.AsyncFrame; -import androidx.media3.common.video.DefaultHardwareBufferFrame; -import androidx.media3.common.video.FrameWriter; -import androidx.media3.effect.HardwareBufferFrame; -import androidx.media3.effect.HardwareBufferFrameQueue; -import com.google.common.collect.ImmutableMap; -import java.util.Map; -import java.util.concurrent.ConcurrentHashMap; - -/** Adapts a {@link FrameWriter} to the {@link HardwareBufferFrameQueue} interface. */ -@RequiresApi(26) -@ExperimentalApi // TODO: b/449956776 - Remove once FrameConsumer API is finalized. -public final class FrameWriterToHardwareBufferFrameQueueAdapter - implements HardwareBufferFrameQueue { - - private final FrameWriter writer; - private final Map frameMap; - @Nullable private Format configuredFormat; - - public FrameWriterToHardwareBufferFrameQueueAdapter(FrameWriter writer) { - this.writer = writer; - this.frameMap = new ConcurrentHashMap<>(); - } - - @Nullable - @Override - public HardwareBufferFrame dequeue(FrameFormat format, Runnable wakeupListener) { - Format commonFormat = - new Format.Builder() - .setWidth(format.width) - .setHeight(format.height) - .setRotationDegrees(format.rotationDegrees) - .setColorInfo(format.colorInfo) - .setPixelFormat(format.pixelFormat) - .build(); - - if (!commonFormat.equals(configuredFormat)) { - writer.configure(commonFormat, format.usageFlags); - configuredFormat = commonFormat; - } - - AsyncFrame asyncFrame = writer.dequeueInputFrame(directExecutor(), wakeupListener); - - if (asyncFrame == null) { - return null; - } - - checkState( - asyncFrame.frame instanceof DefaultHardwareBufferFrame, - "Expected DefaultHardwareBufferFrame"); - DefaultHardwareBufferFrame hardwareBufferFrame = (DefaultHardwareBufferFrame) asyncFrame.frame; - HardwareBuffer hardwareBuffer = hardwareBufferFrame.getHardwareBuffer(); - - frameMap.put(hardwareBuffer, asyncFrame); - - long presentationTimeUs = - (long) - checkNotNull( - hardwareBufferFrame - .getMetadata() - .getOrDefault(KEY_PRESENTATION_TIME_US, C.TIME_UNSET)); - - return new HardwareBufferFrame.Builder( - hardwareBuffer, - directExecutor(), - releaseFence -> { - // Frames returned by dequeue should be queued back. - }) - .setPresentationTimeUs(presentationTimeUs) - .setAcquireFence(asyncFrame.acquireFence) - .setFormat(hardwareBufferFrame.getFormat()) - .setInternalFrame(hardwareBufferFrame.getInternalImage()) - .build(); - } - - @Override - public void queue(HardwareBufferFrame frame) { - HardwareBuffer hardwareBuffer = frame.hardwareBuffer; - AsyncFrame asyncFrame = - checkNotNull(frameMap.remove(hardwareBuffer), "Matching AsyncFrame not found"); - - ImmutableMap.Builder metadataBuilder = ImmutableMap.builder(); - metadataBuilder.putAll(asyncFrame.frame.getMetadata()); - if (frame.getMetadata() instanceof CompositionFrameMetadata) { - metadataBuilder.put( - CompositionFrameMetadata.KEY_COMPOSITION_FRAME_METADATA, frame.getMetadata()); - } - DefaultHardwareBufferFrame newFrame = - ((DefaultHardwareBufferFrame.Builder) - ((DefaultHardwareBufferFrame) asyncFrame.frame).buildUpon()) - .setFormat(frame.format) - .setContentTimeUs(frame.sequencePresentationTimeUs) - .setMetadata(metadataBuilder.buildOrThrow()) - .build(); - - writer.queueInputFrame(newFrame, frame.acquireFence); - } - - @Override - public void signalEndOfStream() { - writer.signalEndOfStream(); - } - - @Override - public void release() { - writer.close(); - frameMap.clear(); - } -} diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/FrameworkMuxer.java b/libraries/transformer/src/main/java/androidx/media3/transformer/FrameworkMuxer.java index dbda0015fa9..9588fb93925 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/FrameworkMuxer.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/FrameworkMuxer.java @@ -30,6 +30,7 @@ import android.media.MediaMuxer; import android.util.Pair; import android.util.SparseArray; +import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; import androidx.media3.common.C; import androidx.media3.common.Format; @@ -124,6 +125,7 @@ public ImmutableList getSupportedSampleMimeTypes(@C.TrackType int trackT private boolean isStarted; private boolean isReleased; + private boolean suppressReleaseException; private FrameworkMuxer(MediaMuxer mediaMuxer, long videoDurationUs) { this.mediaMuxer = mediaMuxer; @@ -275,6 +277,12 @@ public void close() throws MuxerException { try { stopMuxer(mediaMuxer); } catch (RuntimeException e) { + if (suppressReleaseException) { + // Suppress OS MediaMuxer.stop() errors on cancel. + // Platform muxer implementations throw if stopped without samples (b/488422335, + // b/243008015). + return; + } throw new MuxerException(MUXER_STOPPING_FAILED_ERROR_MESSAGE, e); } finally { mediaMuxer.release(); @@ -282,6 +290,14 @@ public void close() throws MuxerException { } } + /** + * Configures the muxer to suppress stopping failures from the underlying {@link MediaMuxer} in + * {@link #close()}. + */ + /* package */ void setSuppressReleaseException() { + this.suppressReleaseException = true; + } + private void startMuxer() throws MuxerException { try { mediaMuxer.start(); @@ -322,10 +338,11 @@ private static void stopMuxer(MediaMuxer mediaMuxer) { private static ImmutableList getSupportedVideoSampleMimeTypes() { ImmutableList.Builder supportedMimeTypes = new ImmutableList.Builder() - .add(MimeTypes.VIDEO_H264, MimeTypes.VIDEO_H263, MimeTypes.VIDEO_MP4V); - if (SDK_INT >= 24) { - supportedMimeTypes.add(MimeTypes.VIDEO_H265); - } + .add( + MimeTypes.VIDEO_H264, + MimeTypes.VIDEO_H263, + MimeTypes.VIDEO_MP4V, + MimeTypes.VIDEO_H265); if (SDK_INT >= 33) { supportedMimeTypes.add(MimeTypes.VIDEO_DOLBY_VISION); } @@ -353,10 +370,15 @@ private static int getDvProfile() { /** Get Dolby Vision level. */ @RequiresApi(33) - private static int getDvLevel(Format format) { + private static int getDvLevel(Format format) throws MuxerException { if (format.codecs != null) { - Pair profileAndLevel = getCodecProfileAndLevel(format); - return checkNotNull(profileAndLevel).second; + @Nullable Pair profileAndLevel = getCodecProfileAndLevel(format); + if (profileAndLevel == null) { + throw new MuxerException( + "Unsupported Dolby Vision profile and level for format=" + format, + new IllegalArgumentException("Unsupported Dolby Vision codecs: " + format.codecs)); + } + return profileAndLevel.second; } return calculateMediaCodecDolbyVisionLevel(format.width, format.height, format.frameRate); } diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/HardwareBufferFrameQueueToFrameWriterAdapter.java b/libraries/transformer/src/main/java/androidx/media3/transformer/HardwareBufferFrameQueueToFrameWriterAdapter.java deleted file mode 100644 index cff20f00faa..00000000000 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/HardwareBufferFrameQueueToFrameWriterAdapter.java +++ /dev/null @@ -1,153 +0,0 @@ -/* - * Copyright 2026 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package androidx.media3.transformer; - -import static com.google.common.base.Preconditions.checkNotNull; -import static com.google.common.base.Preconditions.checkState; - -import android.hardware.HardwareBuffer; -import androidx.annotation.Nullable; -import androidx.annotation.RequiresApi; -import androidx.media3.common.ColorInfo; -import androidx.media3.common.Format; -import androidx.media3.common.util.ExperimentalApi; -import androidx.media3.common.video.AsyncFrame; -import androidx.media3.common.video.DefaultHardwareBufferFrame; -import androidx.media3.common.video.Frame; -import androidx.media3.common.video.FrameWriter; -import androidx.media3.common.video.SyncFenceWrapper; -import androidx.media3.effect.HardwareBufferFrame; -import androidx.media3.effect.HardwareBufferFrameQueue; -import java.util.Map; -import java.util.concurrent.ConcurrentHashMap; -import java.util.concurrent.Executor; - -/** Adapts a {@link HardwareBufferFrameQueue} to the {@link FrameWriter} interface. */ -@RequiresApi(26) -@ExperimentalApi // TODO: b/449956776 - Remove once FrameConsumer API is finalized. -/* package */ final class HardwareBufferFrameQueueToFrameWriterAdapter implements FrameWriter { - - private final HardwareBufferFrameQueue queue; - private final Map frameMap; - @Nullable private Format format; - @Frame.Usage long usage; - - public HardwareBufferFrameQueueToFrameWriterAdapter(HardwareBufferFrameQueue queue) { - this.queue = queue; - this.frameMap = new ConcurrentHashMap<>(); - } - - @Override - public Info getInfo() { - return (format, usage) -> true; // Assume supported for simplicity in adapter - } - - @Override - public void configure(Format format, @Frame.Usage long usage) { - this.format = format; - this.usage = usage; - } - - @Nullable - @Override - public AsyncFrame dequeueInputFrame(Executor wakeupExecutor, Runnable wakeupListener) { - Format localFormat = checkNotNull(format, "configure() must be called first"); - int pixelFormat = localFormat.pixelFormat; - if (pixelFormat == Format.NO_VALUE) { - pixelFormat = - ColorInfo.isTransferHdr(localFormat.colorInfo) - ? HardwareBuffer.RGBA_1010102 - : HardwareBuffer.RGBA_8888; - } - HardwareBufferFrameQueue.FrameFormat.Builder frameFormatBuilder = - new HardwareBufferFrameQueue.FrameFormat.Builder() - .setWidth(localFormat.width) - .setHeight(localFormat.height) - .setUsageFlags(usage) - .setRotationDegrees(localFormat.rotationDegrees) - .setPixelFormat(pixelFormat); - if (localFormat.colorInfo != null) { - frameFormatBuilder.setColorInfo(localFormat.colorInfo); - } - HardwareBufferFrameQueue.FrameFormat frameFormat = frameFormatBuilder.build(); - - Runnable adaptedWakeupListener = () -> wakeupExecutor.execute(wakeupListener); - - HardwareBufferFrame effectFrame = queue.dequeue(frameFormat, adaptedWakeupListener); - - if (effectFrame == null) { - return null; - } - - if (effectFrame.hardwareBuffer != null) { - frameMap.put(effectFrame.hardwareBuffer, effectFrame); - } - - HardwareBuffer hardwareBuffer = checkNotNull(effectFrame.hardwareBuffer); - DefaultHardwareBufferFrame commonFrame = - new DefaultHardwareBufferFrame.Builder(hardwareBuffer) - .setFormat(localFormat) - .setContentTimeUs(effectFrame.presentationTimeUs) - .setInternalImage(effectFrame.internalFrame) - .build(); - - return new AsyncFrame(commonFrame, effectFrame.acquireFence); - } - - @Override - public void queueInputFrame(Frame frame, @Nullable SyncFenceWrapper writeCompleteFence) { - checkState(frame instanceof DefaultHardwareBufferFrame, "Expected DefaultHardwareBufferFrame"); - DefaultHardwareBufferFrame hardwareBufferFrame = (DefaultHardwareBufferFrame) frame; - HardwareBuffer hardwareBuffer = hardwareBufferFrame.getHardwareBuffer(); - - HardwareBufferFrame effectFrame = - checkNotNull(frameMap.remove(hardwareBuffer), "Matching effect Frame not found"); - - HardwareBufferFrame.Builder frameBuilder = - effectFrame.buildUpon().setAcquireFence(writeCompleteFence).setFormat(frame.getFormat()); - - Object metadata = - frame.getMetadata().get(CompositionFrameMetadata.KEY_COMPOSITION_FRAME_METADATA); - if (metadata instanceof CompositionFrameMetadata) { - frameBuilder.setMetadata((CompositionFrameMetadata) metadata); - } - - Long presentationTimeUs = - (Long) hardwareBufferFrame.getMetadata().get(Frame.KEY_PRESENTATION_TIME_US); - if (presentationTimeUs != null) { - frameBuilder.setPresentationTimeUs(presentationTimeUs); - } else { - frameBuilder.setPresentationTimeUs(hardwareBufferFrame.getContentTimeUs()); - } - - frameBuilder.setSequencePresentationTimeUs(hardwareBufferFrame.getContentTimeUs()); - // When encoding, releaseTime and contentTime are the same. - frameBuilder.setReleaseTimeNs(hardwareBufferFrame.getContentTimeUs() * 1000); - - queue.queue(frameBuilder.build()); - } - - @Override - public void signalEndOfStream() { - queue.signalEndOfStream(); - } - - @Override - public void close() { - queue.release(); - frameMap.clear(); - } -} diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/HardwareBufferFrameReader.java b/libraries/transformer/src/main/java/androidx/media3/transformer/HardwareBufferFrameReader.java index 76fab66ea92..71b2623f1f6 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/HardwareBufferFrameReader.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/HardwareBufferFrameReader.java @@ -16,37 +16,48 @@ package androidx.media3.transformer; import static android.os.Build.VERSION.SDK_INT; +import static androidx.media3.transformer.CompositionFrameMetadata.asFrameMetadata; +import static androidx.media3.transformer.TransformerUtil.END_OF_STREAM_ASYNC_FRAME; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.base.Preconditions.checkState; import android.graphics.Bitmap; +import android.graphics.ColorSpace; +import android.hardware.DataSpace; import android.hardware.HardwareBuffer; import android.os.Handler; import android.os.Looper; import android.view.Surface; import androidx.annotation.GuardedBy; import androidx.annotation.Nullable; +import androidx.annotation.RequiresApi; import androidx.media3.common.C; import androidx.media3.common.ColorInfo; import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; import androidx.media3.common.util.Consumer; +import androidx.media3.common.util.HandlerExecutor; import androidx.media3.common.util.HandlerWrapper; +import androidx.media3.common.util.Log; import androidx.media3.common.util.TimestampIterator; +import androidx.media3.common.util.Util; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.DefaultHardwareBufferFrame; +import androidx.media3.common.video.Frame; +import androidx.media3.common.video.ReleaseCallback; import androidx.media3.common.video.SyncFenceWrapper; -import androidx.media3.effect.HardwareBufferFrame; +import androidx.media3.effect.HardwareBufferJniWrapper; import androidx.media3.exoplayer.Renderer; +import com.google.common.collect.ImmutableMap; import java.util.ArrayDeque; import java.util.ArrayList; import java.util.List; +import java.util.Objects; import java.util.Queue; import java.util.concurrent.Executor; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; -/** - * An adaptor between a sequence player and a {@link Consumer} of {@link HardwareBufferFrame} - * instances. - */ +/** An adaptor between a sequence player and a {@link Consumer} of {@link AsyncFrame} instances. */ /* package */ final class HardwareBufferFrameReader { /** A listener for events. */ @@ -77,18 +88,18 @@ public interface RendererWakeupListener { * *

    Using 2 images seems to work on older API levels in our tests, and allows double buffering. */ - private static final int CAPACITY = 2; + /* package */ static final int CAPACITY = SDK_INT >= 29 ? 3 : 2; private static final int DEFAULT_FRAME_RATE = 30; private static final String TAG = "HBFrameReader"; + private static final long RELEASE_TIMEOUT_MS = 2000; private final Composition composition; private final int sequenceIndex; - private final Consumer frameConsumer; + private final Consumer frameConsumer; private final ImageReaderAdapter imageReader; private final Listener listener; - private final HandlerWrapper listenerHandler; - private final PlaybackExecutor playbackExecutor; + private final Executor playbackExecutor; // TODO: b/478781219 - Ensure this class is only accessed from a single thread. /** @@ -108,6 +119,9 @@ public interface RendererWakeupListener { @Nullable private Format lastFormat; private @MonotonicNonNull Format lastAdjustedFormat; + private final HardwareBufferJniWrapper hardwareBufferJniWrapper; + @Nullable private BitmapToHardwareBufferConverter bitmapToHardwareBufferConverter; + // TODO: b/478781219 - Ensure this class is only accessed from a single thread. /** The number of frames that are currently in use by the downstream consumer. */ @GuardedBy("this") @@ -126,21 +140,24 @@ public interface RendererWakeupListener { * internal {@link ImageReaderAdapter}. * @param listener The listener. * @param listenerHandler A {@link HandlerWrapper} to dispatch {@link Listener} callbacks. + * @param hardwareBufferJniWrapper A {@link HardwareBufferJniWrapper} used to convert software + * bitmaps to hardware buffers. */ /* package */ HardwareBufferFrameReader( Composition composition, int sequenceIndex, - Consumer frameConsumer, + Consumer frameConsumer, Looper playbackLooper, int defaultSurfacePixelFormat, ImageReaderAdapter.Factory imageReaderAdapterFactory, Listener listener, - HandlerWrapper listenerHandler) { + HandlerWrapper listenerHandler, + HardwareBufferJniWrapper hardwareBufferJniWrapper) { this.composition = composition; this.sequenceIndex = sequenceIndex; this.frameConsumer = frameConsumer; this.listener = listener; - this.listenerHandler = listenerHandler; + this.hardwareBufferJniWrapper = hardwareBufferJniWrapper; // The width and height are sensible defaults for tests and are typically ignored when writing // from MediaCodec. this.imageReader = @@ -151,10 +168,19 @@ public interface RendererWakeupListener { /* maxImages= */ CAPACITY, /* usage= */ SDK_INT >= 29 ? HardwareBuffer.USAGE_GPU_SAMPLED_IMAGE : 0); Handler playbackHandler = new Handler(playbackLooper); - playbackExecutor = new PlaybackExecutor(playbackHandler); + playbackExecutor = + new HandlerExecutor(playbackHandler, e -> listenerHandler.post(() -> listener.onError(e))); rendererWakeupListenerList = new ArrayList<>(); - imageReader.setOnImageAvailableListener(playbackExecutor, playbackHandler); pendingFrameInfo = new ArrayDeque<>(); + imageReader.setOnImageAvailableListener( + reader -> { + try { + pollImage(); + } catch (RuntimeException e) { + listenerHandler.post(() -> listener.onError(e)); + } + }, + playbackHandler); } /** Returns a Surface which can be used to produce frames into. */ @@ -223,12 +249,14 @@ void queueFrameViaSurface( // into account. void outputBitmap( Bitmap bitmap, TimestampIterator timestampIterator, long sequenceOffsetUs, int indexOfItem) { + String sampleMimeType = + SDK_INT >= 34 && Api34.hasGainmap(bitmap) ? MimeTypes.IMAGE_JPEG_R : MimeTypes.IMAGE_RAW; Format format = new Format.Builder() - .setSampleMimeType(MimeTypes.IMAGE_RAW) + .setSampleMimeType(sampleMimeType) .setWidth(bitmap.getWidth()) .setHeight(bitmap.getHeight()) - .setColorInfo(ColorInfo.SRGB_BT709_FULL) + .setColorInfo(resolveColorInfoFromBitmap(bitmap)) .setFrameRate(/* frameRate= */ DEFAULT_FRAME_RATE) .build(); synchronized (this) { @@ -276,7 +304,7 @@ void pollImage() { itemPresentationTimeUs * 1000); frameConsumer.accept( - createHardwareBufferFrameFromImage( + createAsyncFrameFromImage( image, itemPresentationTimeUs, sequenceOffsetUs, indexOfItem, frameInfo.format)); maybeOutputPendingBitmaps(); maybeWakeupVideoRenderer(); @@ -289,8 +317,8 @@ void pollImage() { } /** - * Forwards a {@link HardwareBufferFrame#END_OF_STREAM_FRAME} to the downstream consumer after all - * pending frames have been forwarded. + * Forwards a {@link TransformerUtil#END_OF_STREAM_ASYNC_FRAME} to the downstream consumer after + * all pending frames have been forwarded. */ void queueEndOfStream() { synchronized (this) { @@ -323,6 +351,9 @@ void removeRendererWakeupListener(RendererWakeupListener rendererWakeupListener) void flush() { synchronized (this) { pendingFrameInfo.clear(); + if (SDK_INT >= 26 && bitmapToHardwareBufferConverter != null) { + bitmapToHardwareBufferConverter.flush(); + } } } @@ -332,6 +363,10 @@ void release() { imageReaderSurface.release(); } imageReader.close(); + if (SDK_INT >= 26 && bitmapToHardwareBufferConverter != null) { + bitmapToHardwareBufferConverter.close(); + bitmapToHardwareBufferConverter = null; + } } /** Sets a default {@link ColorInfo} on the given {@link Format}. */ @@ -364,14 +399,14 @@ private void maybeOutputPendingBitmaps() { long itemPresentationTimeUs = frameInfo.timestampIterator.next(); long sequenceOffsetUs = frameInfo.sequenceOffsetUs; framesInUse++; - HardwareBufferFrame hardwareBufferFrame = - createHardwareBufferFrameFromBitmap( + AsyncFrame asyncFrame = + createAsyncFrameFromBitmap( checkNotNull(frameInfo.bitmap), itemPresentationTimeUs, sequenceOffsetUs, frameInfo.itemIndex, frameInfo.format); - frameConsumer.accept(hardwareBufferFrame); + frameConsumer.accept(asyncFrame); } } maybeOutputEndOfStream(); @@ -381,113 +416,138 @@ private void maybeOutputEndOfStream() { synchronized (this) { @Nullable FrameInfo frameInfo = pendingFrameInfo.peek(); while (frameInfo == FrameInfo.END_OF_STREAM) { - frameConsumer.accept(HardwareBufferFrame.END_OF_STREAM_FRAME); + frameConsumer.accept(END_OF_STREAM_ASYNC_FRAME); pendingFrameInfo.remove(); frameInfo = pendingFrameInfo.peek(); } } } - private HardwareBufferFrame createHardwareBufferFrameFromImage( + @SuppressWarnings("NewApi") + private AsyncFrame createAsyncFrameFromImage( ImageAdapter image, long presentationTimeUs, long sequenceOffsetUs, int indexOfItem, Format format) { - HardwareBufferFrame.Builder frameBuilder; // TODO: b/449956936 - Add support for HardwareBuffer on API 26 using Media NDK methods such as // AImage_getHardwareBuffer. - if (SDK_INT >= 28) { - HardwareBuffer hardwareBuffer = checkNotNull(image.getHardwareBuffer()); - checkState(!hardwareBuffer.isClosed()); - frameBuilder = - new HardwareBufferFrame.Builder( - hardwareBuffer, - playbackExecutor, - /* releaseCallback= */ (releaseFence) -> { - // TODO: b/449956936 - Notify the video renderer's WakeupListener that new capacity - // is freed up, and run another render loop. - releaseFrame(image, hardwareBuffer, releaseFence); - }); - } else { - // TODO: b/449956936 - Support earlier API levels via HardwareBufferFrame.internalFrame. - frameBuilder = - new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, - playbackExecutor, - /* releaseCallback= */ (releaseFence) -> { - releaseFrame(image, /* hardwareBuffer= */ null, releaseFence); - }); + // TODO(b/531653682): Support frame types not backed by HardwareBuffer. + HardwareBuffer hardwareBuffer = checkNotNull(image.getHardwareBuffer()); + checkState(!hardwareBuffer.isClosed()); + return createAsyncFrame( + hardwareBuffer, + image.getInternalImage(), + playbackExecutor, + /* releaseCallback= */ (releaseFence) -> releaseFrame(image, hardwareBuffer, releaseFence), + presentationTimeUs, + sequenceOffsetUs, + indexOfItem, + format); + } + + @Nullable + private BitmapToHardwareBufferConverter getOrCreateBitmapToHardwareBufferConverter() { + if (SDK_INT >= 26 && bitmapToHardwareBufferConverter == null) { + bitmapToHardwareBufferConverter = + new BitmapToHardwareBufferConverter( + hardwareBufferJniWrapper, + /* internalExecutor= */ Util.newSingleThreadExecutor( + "BitmapToHardwareBufferConverter::Thread"), + /* errorExecutor= */ playbackExecutor, + /* errorCallback= */ listener::onError); } - // TODO: b/449956936 - Set the acquire fence from image on the frameBuilder. - frameBuilder.setInternalFrame(image.getInternalImage()); - return createHardwareBufferFrame( - frameBuilder, presentationTimeUs, sequenceOffsetUs, indexOfItem, format); + return bitmapToHardwareBufferConverter; } - private HardwareBufferFrame createHardwareBufferFrameFromBitmap( + private AsyncFrame createAsyncFrameFromBitmap( Bitmap bitmap, long presentationTimeUs, long sequenceOffsetUs, int itemIndex, Format format) { - HardwareBufferFrame.Builder frameBuilder; // TODO: b/449956936 - Copy the Bitmap into a HardwareBuffer using NDK on earlier API levels. if (SDK_INT >= 31 && checkNotNull(bitmap).getConfig() == Bitmap.Config.HARDWARE) { - HardwareBuffer hardwareBuffer = checkNotNull(bitmap).getHardwareBuffer(); + HardwareBuffer hardwareBuffer = checkNotNull(bitmap.getHardwareBuffer()); checkState(!hardwareBuffer.isClosed()); - frameBuilder = - new HardwareBufferFrame.Builder( - hardwareBuffer, - playbackExecutor, - /* releaseCallback= */ (releaseFence) -> { - // Do not manually release the hardware buffer backing the bitmap, it will be reused - // when the bitmap is repeated, and cleaned up when the bitmap is garbage collected. - releaseFrame(/* image= */ null, /* hardwareBuffer= */ null, releaseFence); - }); - } else { - frameBuilder = - new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, - playbackExecutor, - /* releaseCallback= */ (releaseFence) -> { - releaseFrame(/* image= */ null, /* hardwareBuffer= */ null, releaseFence); - }); + return createAsyncFrame( + hardwareBuffer, + bitmap, + playbackExecutor, + /* releaseCallback= */ (releaseFence) -> { + // Do not manually release the hardware buffer backing the bitmap, it will be reused + // when the bitmap is repeated, and cleaned up when the bitmap is garbage collected. + releaseFrame(/* image= */ null, /* hardwareBuffer= */ null, releaseFence); + }, + presentationTimeUs, + sequenceOffsetUs, + itemIndex, + format); } - frameBuilder.setInternalFrame(bitmap); - return createHardwareBufferFrame( - frameBuilder, presentationTimeUs, sequenceOffsetUs, itemIndex, format); + // TODO(b/531653682): Support frame types not backed by HardwareBuffer. + DefaultHardwareBufferFrame retainedHandle = + checkNotNull(getOrCreateBitmapToHardwareBufferConverter()).getOrCreateRetainedFrame(bitmap); + return createAsyncFrame( + checkNotNull(retainedHandle.getHardwareBuffer()), + bitmap, + playbackExecutor, + /* releaseCallback= */ (releaseFence) -> { + retainedHandle.release(releaseFence); + releaseFrame(/* image= */ null, /* hardwareBuffer= */ null, /* releaseFence= */ null); + }, + presentationTimeUs, + sequenceOffsetUs, + itemIndex, + format); } - private HardwareBufferFrame createHardwareBufferFrame( - HardwareBufferFrame.Builder frameBuilder, + @SuppressWarnings({"deprecation", "NewApi"}) // Uses deprecated CompositionFrameMetadata. + private AsyncFrame createAsyncFrame( + HardwareBuffer hardwareBuffer, + @Nullable Object internalImage, + Executor releaseExecutor, + ReleaseCallback releaseCallback, long presentationTimeUs, long sequenceOffsetUs, int itemIndex, Format format) { // COLOR_TRANSFER_SRGB may not be supported by the encoder or display, but is equivalent to // COLOR_TRANSFER_SDR on Android which is widely supported. - if (format.colorInfo != null && format.colorInfo.colorTransfer == C.COLOR_TRANSFER_SRGB) { + if (format.colorInfo != null + && format.colorInfo.colorTransfer == C.COLOR_TRANSFER_SRGB + && !Objects.equals(format.sampleMimeType, MimeTypes.IMAGE_JPEG_R)) { ColorInfo adjustedColorInfo = format.colorInfo.buildUpon().setColorTransfer(C.COLOR_TRANSFER_SDR).build(); format = format.buildUpon().setColorInfo(adjustedColorInfo).build(); } long sequencePresentationTimeUs = presentationTimeUs + sequenceOffsetUs; - return frameBuilder - .setPresentationTimeUs(presentationTimeUs) - .setSequencePresentationTimeUs(sequencePresentationTimeUs) - .setMetadata(new CompositionFrameMetadata(composition, sequenceIndex, itemIndex)) - .setFormat(format) - .build(); + CompositionFrameMetadata compositionFrameMetadata = + new CompositionFrameMetadata(composition, sequenceIndex, itemIndex); + ImmutableMap metadata = + ImmutableMap.builder() + .put(Frame.KEY_PRESENTATION_TIME_US, presentationTimeUs) + .put(CompositionFrameMetadata.KEY_COMPOSITION_FRAME_METADATA, compositionFrameMetadata) + .putAll(asFrameMetadata(compositionFrameMetadata)) + .buildOrThrow(); + DefaultHardwareBufferFrame frame = + new DefaultHardwareBufferFrame.Builder(hardwareBuffer, releaseExecutor, releaseCallback) + .setContentTimeUs(sequencePresentationTimeUs) + .setMetadata(metadata) + .setFormat(format) + .setInternalImage(internalImage) + .build(); + return new AsyncFrame(frame, /* acquireFence= */ null); } private void releaseFrame( @Nullable ImageAdapter image, @Nullable HardwareBuffer hardwareBuffer, @Nullable SyncFenceWrapper releaseFence) { + if (releaseFence != null) { + // TODO: b/475744934 - Use the NDK to set the fence on the Image. + if (!releaseFence.awaitMs(RELEASE_TIMEOUT_MS)) { + Log.w(TAG, "Timed out waiting for release fence to signal."); + } + releaseFence.close(); + } synchronized (this) { framesInUse--; - if (releaseFence != null) { - // TODO: b/475744934 - Use the NDK to set the fence on the Image. - checkState(releaseFence.awaitMs(500)); - releaseFence.close(); - } if (SDK_INT >= 26 && hardwareBuffer != null) { hardwareBuffer.close(); } @@ -505,38 +565,11 @@ private void maybeWakeupVideoRenderer() { } } - /** - * A {@link Executor} which executes commands on a {@link Handler} and propagates {@linkplain - * RuntimeException errors} to the {@link Listener}. - */ - private class PlaybackExecutor implements Consumer, Executor { - - final Handler playbackHandler; - - PlaybackExecutor(Handler playbackHandler) { - this.playbackHandler = playbackHandler; - } - - @Override - public void execute(Runnable command) { - playbackHandler.post( - () -> { - try { - command.run(); - } catch (RuntimeException e) { - listenerHandler.post(() -> listener.onError(e)); - } - }); - } - - @Override - public void accept(ImageReaderAdapter reader) { - try { - pollImage(); - } catch (RuntimeException e) { - listenerHandler.post(() -> listener.onError(e)); - } + private static ColorInfo resolveColorInfoFromBitmap(Bitmap bitmap) { + if (SDK_INT >= 33) { + return Api33.resolveColorInfoFromBitmap(bitmap); } + return ColorInfo.SRGB_BT709_FULL; } private static final class FrameInfo { @@ -598,4 +631,47 @@ private FrameInfo( this.format = format; } } + + @RequiresApi(33) + private static final class Api33 { + private static final ColorInfo BT2020_HLG_FULL = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorRange(C.COLOR_RANGE_FULL) + .setColorTransfer(C.COLOR_TRANSFER_HLG) + .build(); + + private static final ColorInfo BT2020_PQ_FULL = + new ColorInfo.Builder() + .setColorSpace(C.COLOR_SPACE_BT2020) + .setColorRange(C.COLOR_RANGE_FULL) + .setColorTransfer(C.COLOR_TRANSFER_ST2084) + .build(); + + private static ColorInfo resolveColorInfoFromBitmap(Bitmap bitmap) { + ColorSpace colorSpace = bitmap.getColorSpace(); + if (colorSpace != null) { + return dataSpaceToColorInfo(colorSpace.getDataSpace()); + } + return ColorInfo.SRGB_BT709_FULL; + } + + private static ColorInfo dataSpaceToColorInfo(int dataSpace) { + switch (dataSpace) { + case DataSpace.STANDARD_BT2020 | DataSpace.TRANSFER_HLG | DataSpace.RANGE_FULL: + return BT2020_HLG_FULL; + case DataSpace.STANDARD_BT2020 | DataSpace.TRANSFER_ST2084 | DataSpace.RANGE_FULL: + return BT2020_PQ_FULL; + default: + return ColorInfo.SRGB_BT709_FULL; + } + } + } + + @RequiresApi(34) + private static final class Api34 { + private static boolean hasGainmap(Bitmap bitmap) { + return bitmap.hasGainmap(); + } + } } diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/HardwareBufferSampleConsumer.java b/libraries/transformer/src/main/java/androidx/media3/transformer/HardwareBufferSampleConsumer.java index 734b76b7064..6b293b11179 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/HardwareBufferSampleConsumer.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/HardwareBufferSampleConsumer.java @@ -15,6 +15,7 @@ */ package androidx.media3.transformer; +import static androidx.media3.transformer.TransformerUtil.END_OF_STREAM_ASYNC_FRAME; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.base.Preconditions.checkState; @@ -28,14 +29,19 @@ import androidx.media3.common.util.Consumer; import androidx.media3.common.util.HandlerWrapper; import androidx.media3.common.util.TimestampIterator; -import androidx.media3.effect.HardwareBufferFrame; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.DefaultHardwareBufferFrame; +import androidx.media3.common.video.Frame; +import androidx.media3.effect.HardwareBufferJniWrapper; import androidx.media3.transformer.HardwareBufferFrameReader.Listener; +import java.util.HashMap; +import java.util.Map; /** * A {@link GraphInput} that wraps {@link HardwareBufferFrameReader}. * - *

    Offsets {@link HardwareBufferFrame#releaseTimeNs} based on preceding media item durations to - * ensure a continuous timeline within the sequence. + *

    Offsets {@link Frame#KEY_DISPLAY_TIME_NS} based on preceding media item durations to ensure a + * continuous timeline within the sequence. */ /* package */ final class HardwareBufferSampleConsumer implements GraphInput { @@ -57,32 +63,40 @@ * @param playbackLooper The {@link Looper} for playback operations. * @param listenerHandler The {@link HandlerWrapper} for scheduling to dispatch {@link Listener} * callbacks. - * @param frameConsumer The {@link Consumer} to which processed frames are - * output. + * @param frameConsumer The {@link Consumer} to which processed frames are output. * @param errorConsumer A consumer to accept {@link ExportException}s if errors occur. + * @param hardwareBufferJniWrapper The {@link HardwareBufferJniWrapper} used to convert software + * bitmaps to hardware buffers. */ public HardwareBufferSampleConsumer( Composition composition, int sequenceIndex, Looper playbackLooper, HandlerWrapper listenerHandler, - Consumer frameConsumer, - Consumer errorConsumer) { + Consumer frameConsumer, + Consumer errorConsumer, + HardwareBufferJniWrapper hardwareBufferJniWrapper) { this.composition = composition; this.sequenceIndex = sequenceIndex; // Modify the release times of frames exiting the hardwareBufferFrameReader to account for // item duration. - Consumer intermediateConsumer = - (hardwareBufferFrame) -> { - if (hardwareBufferFrame != HardwareBufferFrame.END_OF_STREAM_FRAME) { + Consumer intermediateConsumer = + (asyncFrame) -> { + if (asyncFrame != END_OF_STREAM_ASYNC_FRAME) { // The frame's sequencePresentationTimeUs is already adjusted by the // HardwareBufferFrameReader to be monotonic across the entire sequence. // Adjust the release times so they are valid for the encoder. - long releaseTimeNs = hardwareBufferFrame.sequencePresentationTimeUs * 1000; - hardwareBufferFrame = - hardwareBufferFrame.buildUpon().setReleaseTimeNs(releaseTimeNs).build(); + long releaseTimeNs = asyncFrame.frame.getContentTimeUs() * 1000; + Map updatedMetadata = new HashMap<>(asyncFrame.frame.getMetadata()); + updatedMetadata.put(Frame.KEY_DISPLAY_TIME_NS, releaseTimeNs); + Frame frame = + ((DefaultHardwareBufferFrame) asyncFrame.frame) + .buildUpon() + .setMetadata(updatedMetadata) + .build(); + asyncFrame = new AsyncFrame(frame, asyncFrame.acquireFence); } - frameConsumer.accept(hardwareBufferFrame); + frameConsumer.accept(asyncFrame); }; this.hardwareBufferFrameReader = new HardwareBufferFrameReader( @@ -93,7 +107,8 @@ public HardwareBufferSampleConsumer( /* defaultSurfacePixelFormat= */ ImageFormat.PRIVATE, new DefaultImageReaderAdapter.Factory(), e -> errorConsumer.accept(ExportException.createForUnexpected(e)), - listenerHandler); + listenerHandler, + hardwareBufferJniWrapper); } @Override @@ -107,7 +122,9 @@ public void onMediaItemChanged( // The editedMediaItems passed to this method are different instances to those in the // composition, so they cannot be directly compared. Assume that media items are processed in // order, so every onMediaItemChanged call increments the media item index. - currentMediaItemIndex++; + EditedMediaItemSequence sequence = composition.sequences.get(sequenceIndex); + currentMediaItemIndex = + EditedMediaItemSequence.getEditedMediaItemIndex(sequence, currentMediaItemIndex + 1); currentTimeOffsetUs += nextTimeOffsetUs; nextTimeOffsetUs = durationUs; } @@ -138,8 +155,8 @@ public boolean registerVideoFrame(long presentationTimeUs) { @Override public void signalEndOfVideoInput() { - if (currentMediaItemIndex - == composition.sequences.get(sequenceIndex).editedMediaItems.size() - 1) { + EditedMediaItemSequence sequence = composition.sequences.get(sequenceIndex); + if (sequence.isLooping || currentMediaItemIndex == sequence.editedMediaItems.size() - 1) { hardwareBufferFrameReader.queueEndOfStream(); } } diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/InAppFragmentedMp4Muxer.java b/libraries/transformer/src/main/java/androidx/media3/transformer/InAppFragmentedMp4Muxer.java index e52c94899cf..d201aa43217 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/InAppFragmentedMp4Muxer.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/InAppFragmentedMp4Muxer.java @@ -15,6 +15,10 @@ */ package androidx.media3.transformer; +import static com.google.common.base.Preconditions.checkArgument; + +import android.os.ParcelFileDescriptor; +import androidx.annotation.Nullable; import androidx.media3.common.C; import androidx.media3.common.Format; import androidx.media3.common.MediaLibraryInfo; @@ -32,11 +36,11 @@ import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.io.FileNotFoundException; import java.io.FileOutputStream; +import java.io.IOException; import java.nio.ByteBuffer; import java.util.Locale; /** {@link Muxer} implementation that uses a {@link FragmentedMp4Muxer}. */ -// TODO: b/372417042 - Add E2E tests for producing fragmented MP4 output. @UnstableApi public final class InAppFragmentedMp4Muxer implements Muxer { /** {@link Muxer.Factory} for {@link InAppFragmentedMp4Muxer}. */ @@ -100,6 +104,48 @@ public InAppFragmentedMp4Muxer create(String path) throws MuxerException { return new InAppFragmentedMp4Muxer(muxer, videoDurationUs); } + @Override + public InAppFragmentedMp4Muxer create(ParcelFileDescriptor pfd) throws MuxerException { + @Nullable FragmentedMp4Muxer muxer = null; + try { + checkArgument(pfd.getFileDescriptor().valid(), "ParcelFileDescriptor is closed or invalid"); + FragmentedMp4Muxer.Builder builder = + new FragmentedMp4Muxer.Builder( + new ParcelFileDescriptor.AutoCloseOutputStream(pfd).getChannel()); + if (fragmentDurationMs != C.TIME_UNSET) { + builder.setFragmentDurationMs(fragmentDurationMs); + } + muxer = builder.build(); + + return new InAppFragmentedMp4Muxer(muxer, videoDurationUs); + } catch (RuntimeException e) { + MuxerException muxerException = + new MuxerException( + "Error creating InAppFragmentedMp4Muxer from ParcelFileDescriptor", e); + closeResourceAndSuppressException(muxerException, muxer, pfd); + throw muxerException; + } + } + + private static void closeResourceAndSuppressException( + MuxerException muxerException, + @Nullable FragmentedMp4Muxer muxer, + ParcelFileDescriptor pfd) { + if (muxer != null) { + try { + muxer.close(); + } catch (MuxerException e) { + muxerException.addSuppressed(e); + } + } else { + try { + pfd.close(); + } catch (IOException e) { + muxerException.addSuppressed(e); + } + } + } + @Override public ImmutableList getSupportedSampleMimeTypes(@C.TrackType int trackType) { if (trackType == C.TRACK_TYPE_VIDEO) { @@ -129,7 +175,7 @@ private InAppFragmentedMp4Muxer(FragmentedMp4Muxer muxer, long videoDurationUs) } @Override - public int addTrack(Format format) { + public int addTrack(Format format) throws MuxerException { int trackId = muxer.addTrack(format); if (MimeTypes.isVideo(format.sampleMimeType)) { muxer.addMetadataEntry(new Mp4OrientationData(format.rotationDegrees)); diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/InAppMp4Muxer.java b/libraries/transformer/src/main/java/androidx/media3/transformer/InAppMp4Muxer.java index 3742c8bef68..5ca9673ffcc 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/InAppMp4Muxer.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/InAppMp4Muxer.java @@ -15,6 +15,9 @@ */ package androidx.media3.transformer; +import static com.google.common.base.Preconditions.checkArgument; + +import android.os.ParcelFileDescriptor; import androidx.annotation.Nullable; import androidx.media3.common.C; import androidx.media3.common.Format; @@ -34,6 +37,7 @@ import com.google.errorprone.annotations.CanIgnoreReturnValue; import java.io.FileNotFoundException; import java.io.FileOutputStream; +import java.io.IOException; import java.nio.ByteBuffer; import java.util.LinkedHashSet; import java.util.Locale; @@ -156,6 +160,62 @@ public InAppMp4Muxer create(String path) throws MuxerException { return new InAppMp4Muxer(muxer, metadataProvider, videoDurationUs); } + @Override + public InAppMp4Muxer create(ParcelFileDescriptor pfd) throws MuxerException { + @Nullable Mp4Muxer muxer = null; + try { + checkArgument(pfd.getFileDescriptor().valid(), "ParcelFileDescriptor is closed or invalid"); + SeekableMuxerOutput seekableMuxerOutput = SeekableMuxerOutput.of(pfd); + validateSeekable(seekableMuxerOutput); + Mp4Muxer.Builder builder = + new Mp4Muxer.Builder(seekableMuxerOutput) + .setAttemptStreamableOutputEnabled(attemptStreamableOutputEnabled); + if (freeSpaceAfterFileTypeBoxBytes > 0) { + builder.experimentalSetFreeSpaceAfterFileTypeBox(freeSpaceAfterFileTypeBoxBytes); + } + muxer = builder.build(); + + return new InAppMp4Muxer(muxer, metadataProvider, videoDurationUs); + } catch (MuxerException e) { + closeResourceAndSuppressException(e, muxer, pfd); + throw e; + } catch (RuntimeException e) { + MuxerException muxerException = + new MuxerException("Error creating InAppMp4Muxer from ParcelFileDescriptor", e); + closeResourceAndSuppressException(muxerException, muxer, pfd); + throw muxerException; + } + } + + private static void validateSeekable(SeekableMuxerOutput seekableMuxerOutput) + throws MuxerException { + try { + long unused = seekableMuxerOutput.getPosition(); + } catch (IOException e) { + throw new MuxerException( + "The provided ParcelFileDescriptor is not seekable. FragmentedMp4Muxer is required" + + " for non-seekable output.", + e); + } + } + + private static void closeResourceAndSuppressException( + MuxerException muxerException, @Nullable Mp4Muxer muxer, ParcelFileDescriptor pfd) { + if (muxer != null) { + try { + muxer.close(); + } catch (MuxerException e) { + muxerException.addSuppressed(e); + } + } else { + try { + pfd.close(); + } catch (IOException e) { + muxerException.addSuppressed(e); + } + } + } + @Override public ImmutableList getSupportedSampleMimeTypes(@C.TrackType int trackType) { if (trackType == C.TRACK_TYPE_VIDEO) { diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/InFlightFrameManager.java b/libraries/transformer/src/main/java/androidx/media3/transformer/InFlightFrameManager.java new file mode 100644 index 00000000000..7e7752b2185 --- /dev/null +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/InFlightFrameManager.java @@ -0,0 +1,93 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import androidx.annotation.Nullable; +import androidx.media3.common.util.Log; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.Frame; +import androidx.media3.common.video.ReferenceCounter; +import androidx.media3.common.video.SyncFenceWrapper; +import com.google.common.base.Predicate; +import com.google.common.collect.ImmutableList; +import com.google.errorprone.annotations.CanIgnoreReturnValue; +import java.util.HashSet; +import java.util.List; +import java.util.Set; + +/** + * Manages {@link Frame} instances currently in flight downstream, releasing them when processed if + * they implement {@link ReferenceCounter}. + * + *

    All methods must be called on the playback thread. + */ +/* package */ final class InFlightFrameManager { + + private static final String TAG = "InFlightFrameManager"; + + private final Set inFlightFrames = new HashSet<>(); + + /** + * Tracks a list of {@link AsyncFrame} instances during an attempt to queue them, removing them + * from tracking if queuing fails or throws an exception. + * + * @param queueAction The action attempting to queue the frames. + * @param packet The packet of {@link AsyncFrame} instances. + * @return {@code true} if the packet was successfully queued, {@code false} otherwise. + */ + @CanIgnoreReturnValue + > boolean trackIfSuccessful(Predicate queueAction, T packet) { + for (int i = 0; i < packet.size(); i++) { + inFlightFrames.add(packet.get(i).frame); + } + boolean success = false; + try { + success = queueAction.apply(packet); + return success; + } finally { + if (!success) { + for (int i = 0; i < packet.size(); i++) { + inFlightFrames.remove(packet.get(i).frame); + } + } + } + } + + /** + * Called when a downstream frame has been processed. Releases the matching {@link + * ReferenceCounter} if applicable. + */ + void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper releaseFence) { + // If FrameProcessor passes decorated frame, fetching the frame will silently fail. + // TODO(b/539914074): throw if frame is not found, and/or document expectations for decorated + // frames. + if (inFlightFrames.remove(frame)) { + TransformerUtil.releaseIfNeeded(frame, releaseFence); + } else { + if (releaseFence != null) { + releaseFence.close(); + } + Log.d(TAG, "onFrameProcessed: Frame not found: " + frame); + } + } + + /** Releases all currently in-flight frames. */ + void releaseAll() { + ImmutableList framesToRelease = ImmutableList.copyOf(inFlightFrames); + inFlightFrames.clear(); + TransformerUtil.releaseIfNeeded(framesToRelease); + } +} diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/MediaProjectionAssetLoader.java b/libraries/transformer/src/main/java/androidx/media3/transformer/MediaProjectionAssetLoader.java index b8b90d424fa..cb6b679c14b 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/MediaProjectionAssetLoader.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/MediaProjectionAssetLoader.java @@ -171,6 +171,10 @@ public ImmutableMap getDecoderNames() { @Override public void release() { surfaceAssetLoader.release(); + if (virtualDisplay != null) { + virtualDisplay.release(); + virtualDisplay = null; + } } private final class ComponentListener diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/MuxerWrapper.java b/libraries/transformer/src/main/java/androidx/media3/transformer/MuxerWrapper.java index 141b7129600..abeeaf117c2 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/MuxerWrapper.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/MuxerWrapper.java @@ -31,6 +31,7 @@ import static java.lang.Math.min; import static java.lang.annotation.ElementType.TYPE_USE; +import android.os.ParcelFileDescriptor; import android.util.SparseArray; import androidx.annotation.IntDef; import androidx.annotation.IntRange; @@ -146,7 +147,8 @@ void onTrackEnded( */ private static final long MAX_TRACK_WRITE_AHEAD_US = Util.msToUs(500); - private final String outputPath; + @Nullable private final String outputPath; + @Nullable private final ParcelFileDescriptor parcelFileDescriptor; private final Muxer.Factory muxerFactory; private final Listener listener; private final boolean dropSamplesBeforeFirstVideoSample; @@ -192,7 +194,62 @@ public MuxerWrapper( @MuxerMode int muxerMode, boolean dropSamplesBeforeFirstVideoSample, @Nullable Format appendVideoFormat) { + this( + outputPath, + /* parcelFileDescriptor= */ null, + muxerFactory, + listener, + muxerMode, + dropSamplesBeforeFirstVideoSample, + appendVideoFormat); + } + + /** + * Creates an instance with a {@link ParcelFileDescriptor}. + * + *

    {@code appendVideoFormat} must be non-{@code null} when using {@link + * #MUXER_MODE_MUX_PARTIAL}. + * + * @param parcelFileDescriptor The {@link ParcelFileDescriptor} to write output to. + * @param muxerFactory A {@link Muxer.Factory} to create a {@link Muxer}. + * @param listener A {@link MuxerWrapper.Listener}. + * @param muxerMode The {@link MuxerMode}. The initial mode must be {@link #MUXER_MODE_DEFAULT} or + * {@link #MUXER_MODE_MUX_PARTIAL}. + * @param dropSamplesBeforeFirstVideoSample Whether to drop any non-video samples with + * presentation timestamps before the first video sample. + * @param appendVideoFormat The format which will be used to write samples after transitioning + * from {@link #MUXER_MODE_MUX_PARTIAL} to {@link #MUXER_MODE_APPEND}. + */ + public MuxerWrapper( + ParcelFileDescriptor parcelFileDescriptor, + Muxer.Factory muxerFactory, + Listener listener, + @MuxerMode int muxerMode, + boolean dropSamplesBeforeFirstVideoSample, + @Nullable Format appendVideoFormat) { + this( + /* outputPath= */ null, + parcelFileDescriptor, + muxerFactory, + listener, + muxerMode, + dropSamplesBeforeFirstVideoSample, + appendVideoFormat); + } + + private MuxerWrapper( + @Nullable String outputPath, + @Nullable ParcelFileDescriptor parcelFileDescriptor, + Muxer.Factory muxerFactory, + Listener listener, + @MuxerMode int muxerMode, + boolean dropSamplesBeforeFirstVideoSample, + @Nullable Format appendVideoFormat) { + checkArgument( + (outputPath == null) != (parcelFileDescriptor == null), + "Exactly one of outputPath or parcelFileDescriptor must be provided."); this.outputPath = outputPath; + this.parcelFileDescriptor = parcelFileDescriptor; this.muxerFactory = muxerFactory; this.listener = listener; checkArgument(muxerMode == MUXER_MODE_DEFAULT || muxerMode == MUXER_MODE_MUX_PARTIAL); @@ -676,18 +733,12 @@ public void finishWritingAndMaybeRelease(@MuxerReleaseReason int releaseReason) } isReady = false; if (muxer != null) { - try { - muxer.close(); - } catch (MuxerException e) { - if (releaseReason == MUXER_RELEASE_REASON_CANCELLED - && checkNotNull(e.getMessage()) - .equals(FrameworkMuxer.MUXER_STOPPING_FAILED_ERROR_MESSAGE)) { - // When releasing the muxer, FrameworkMuxer may sometimes fail before the actual release. - // When the release is due to cancellation, swallow this exception. - return; - } - throw e; + if (muxer instanceof FrameworkMuxer && releaseReason == MUXER_RELEASE_REASON_CANCELLED) { + ((FrameworkMuxer) muxer).setSuppressReleaseException(); } + muxer.close(); + } else if (parcelFileDescriptor != null) { + Util.closeQuietly(parcelFileDescriptor); } if (releaseReason == MUXER_RELEASE_REASON_COMPLETED) { // Make sure to only read the output file size after the muxer and underlying file are closed. @@ -737,14 +788,24 @@ && contains(trackTypeToInfo, C.TRACK_TYPE_VIDEO) @EnsuresNonNull("muxer") private void ensureMuxerInitialized() throws MuxerException { if (muxer == null) { - muxer = muxerFactory.create(outputPath); + if (parcelFileDescriptor != null) { + muxer = muxerFactory.create(parcelFileDescriptor); + } else if (outputPath != null) { + muxer = muxerFactory.create(outputPath); + } else { + throw new MuxerException( + "Unsupported output destination for initialization", new IllegalArgumentException()); + } } } /** Returns the current size in bytes of the output, or {@link C#LENGTH_UNSET} if unavailable. */ private long getCurrentOutputSizeBytes() { - long fileSize = new File(outputPath).length(); - return fileSize > 0 ? fileSize : C.LENGTH_UNSET; + if (outputPath != null) { + long fileSize = new File(outputPath).length(); + return fileSize > 0 ? fileSize : C.LENGTH_UNSET; + } + return C.LENGTH_UNSET; } @Nullable diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/PacketConsumerToFrameProcessorAdapter.java b/libraries/transformer/src/main/java/androidx/media3/transformer/PacketConsumerToFrameProcessorAdapter.java deleted file mode 100644 index f70b5ec5788..00000000000 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/PacketConsumerToFrameProcessorAdapter.java +++ /dev/null @@ -1,147 +0,0 @@ -/* - * Copyright 2026 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package androidx.media3.transformer; - -import static com.google.common.base.Preconditions.checkState; -import static com.google.common.util.concurrent.MoreExecutors.directExecutor; -import static java.util.concurrent.TimeUnit.MILLISECONDS; - -import androidx.annotation.RequiresApi; -import androidx.media3.common.VideoFrameProcessingException; -import androidx.media3.common.util.ExperimentalApi; -import androidx.media3.common.util.Util; -import androidx.media3.common.video.AsyncFrame; -import androidx.media3.common.video.DefaultHardwareBufferFrame; -import androidx.media3.common.video.FrameProcessor; -import androidx.media3.effect.HardwareBufferFrame; -import androidx.media3.effect.PacketConsumer; -import androidx.media3.effect.PacketConsumerCaller; -import androidx.media3.effect.PacketConsumerUtil; -import androidx.media3.effect.RenderingPacketConsumer; -import com.google.common.collect.ImmutableList; -import java.util.List; -import java.util.concurrent.ExecutionException; -import java.util.concurrent.Executor; -import java.util.concurrent.ExecutorService; -import java.util.concurrent.TimeoutException; -import org.checkerframework.checker.nullness.qual.MonotonicNonNull; - -/** Adapts a {@link RenderingPacketConsumer} to the {@link FrameProcessor} interface. */ -@RequiresApi(26) -@ExperimentalApi // TODO: b/449956776 - Remove once FrameConsumer API is finalized. -public final class PacketConsumerToFrameProcessorAdapter implements FrameProcessor { - - private static final long RELEASE_TIMEOUT_MS = 1000L; - private final PacketConsumer> packetConsumer; - private final PacketConsumerCaller> packetConsumerCaller; - private final ExecutorService executorService; - - private volatile @MonotonicNonNull Exception pendingException; - private final Executor listenerExecutor; - private final Listener listener; - - @SuppressWarnings("nullness:methodref.receiver.bound.invalid") // "this" reference - public PacketConsumerToFrameProcessorAdapter( - PacketConsumer> packetConsumer, - Executor listenerExecutor, - Listener listener) { - this.packetConsumer = packetConsumer; - this.listenerExecutor = listenerExecutor; - this.listener = listener; - this.executorService = - Util.newSingleThreadExecutor("PacketConsumerToFrameProcessorAdapter::Thread"); - this.packetConsumerCaller = - PacketConsumerCaller.create(packetConsumer, executorService, this::onException); - this.packetConsumerCaller.run(); - } - - @Override - public boolean queue(List frames) { - if (pendingException != null) { - return false; - } - - ImmutableList.Builder frameListBuilder = ImmutableList.builder(); - for (AsyncFrame asyncFrame : frames) { - checkState( - asyncFrame.frame instanceof DefaultHardwareBufferFrame, - "Expected DefaultHardwareBufferFrame"); - DefaultHardwareBufferFrame hardwareBufferFrame = - (DefaultHardwareBufferFrame) asyncFrame.frame; - - HardwareBufferFrame.Builder effectFrameBuilder = - new HardwareBufferFrame.Builder( - hardwareBufferFrame.getHardwareBuffer(), - directExecutor(), - releaseFence -> - listenerExecutor.execute( - () -> listener.onFrameProcessed(asyncFrame.frame, releaseFence))); - - effectFrameBuilder.setPresentationTimeUs(hardwareBufferFrame.getContentTimeUs()); - - effectFrameBuilder.setSequencePresentationTimeUs(hardwareBufferFrame.getContentTimeUs()); - // When encoding, releaseTime and contentTime are the same. - effectFrameBuilder.setReleaseTimeNs(hardwareBufferFrame.getContentTimeUs() * 1000); - - effectFrameBuilder.setAcquireFence(asyncFrame.acquireFence); - effectFrameBuilder.setFormat(hardwareBufferFrame.getFormat()); - effectFrameBuilder.setInternalFrame(hardwareBufferFrame.getInternalImage()); - - Object metadata = - hardwareBufferFrame - .getMetadata() - .get(CompositionFrameMetadata.KEY_COMPOSITION_FRAME_METADATA); - if (metadata instanceof CompositionFrameMetadata) { - effectFrameBuilder.setMetadata((CompositionFrameMetadata) metadata); - } - - frameListBuilder.add(effectFrameBuilder.build()); - } - - packetConsumerCaller.queuePacket(PacketConsumer.Packet.Companion.of(frameListBuilder.build())); - - return true; - } - - @Override - public void signalEndOfStream() { - packetConsumerCaller.queueEndOfStream(); - } - - @Override - public void close() { - packetConsumerCaller.release(); - try { - PacketConsumerUtil.release(packetConsumer, executorService) - .get(RELEASE_TIMEOUT_MS, MILLISECONDS); - } catch (InterruptedException e) { - Thread.currentThread().interrupt(); - throw new IllegalStateException("Failed to release packet consumer", e); - } catch (ExecutionException | TimeoutException e) { - throw new IllegalStateException("Failed to release packet consumer", e); - } finally { - executorService.shutdown(); - } - } - - private void onException(Exception exception) { - if (pendingException == null) { - pendingException = exception; - listenerExecutor.execute( - () -> listener.onError(VideoFrameProcessingException.from(exception))); - } - } -} diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/PacketConsumerVideoSampleExporter.java b/libraries/transformer/src/main/java/androidx/media3/transformer/PacketConsumerVideoSampleExporter.java index 9782ac5664e..435a64af292 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/PacketConsumerVideoSampleExporter.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/PacketConsumerVideoSampleExporter.java @@ -17,15 +17,15 @@ import static android.os.Build.VERSION.SDK_INT; import static androidx.media3.common.C.TRACK_TYPE_VIDEO; -import static androidx.media3.effect.HardwareBufferFrame.END_OF_STREAM_FRAME; -import static androidx.media3.transformer.CompositionFrameMetadata.asFrameMetadata; +import static androidx.media3.transformer.FrameAggregator.STRATEGY_EXPECT_NO_FRAMES; +import static androidx.media3.transformer.FrameAggregator.STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET; +import static androidx.media3.transformer.FrameAggregator.STRATEGY_MATCH_FRAME_CLOSEST_TO_TARGET; +import static androidx.media3.transformer.TransformerUtil.END_OF_STREAM_ASYNC_FRAME; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.base.Preconditions.checkState; -import static com.google.common.util.concurrent.MoreExecutors.directExecutor; import static com.google.common.util.concurrent.MoreExecutors.listeningDecorator; import android.content.Context; -import android.hardware.HardwareBuffer; import android.media.MediaCodec.BufferInfo; import android.media.metrics.LogSessionId; import android.os.Handler; @@ -38,25 +38,22 @@ import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.VideoFrameProcessor; import androidx.media3.common.util.Consumer; +import androidx.media3.common.util.HandlerExecutor; import androidx.media3.common.util.HandlerWrapper; import androidx.media3.common.util.Util; import androidx.media3.common.video.AsyncFrame; -import androidx.media3.common.video.DefaultHardwareBufferFrame; import androidx.media3.common.video.Frame; import androidx.media3.common.video.FrameProcessor; import androidx.media3.common.video.FrameWriter; import androidx.media3.common.video.SyncFenceWrapper; import androidx.media3.decoder.DecoderInputBuffer; -import androidx.media3.effect.BitmapToHardwareBufferProcessor; import androidx.media3.effect.DefaultGlObjectsProvider; -import androidx.media3.effect.HardwareBufferFrame; import androidx.media3.effect.HardwareBufferJniWrapper; import androidx.media3.transformer.Codec.EncoderFactory; +import androidx.media3.transformer.FrameAggregator.AggregationStrategy; import com.google.common.collect.ImmutableList; -import com.google.common.collect.ImmutableMap; import java.util.ArrayDeque; -import java.util.HashMap; -import java.util.Map; +import java.util.List; import java.util.Queue; import java.util.concurrent.Executor; import org.checkerframework.checker.initialization.qual.Initialized; @@ -69,7 +66,6 @@ private static final String DEFAULT_OUTPUT_MIME_TYPE = MimeTypes.VIDEO_H265; private final DecoderInputBuffer encoderOutputBuffer; - private final Consumer errorConsumer; private final FrameProcessor frameProcessor; private final FrameAggregator frameAggregator; @@ -79,10 +75,9 @@ private final MuxerWrapper muxerWrapper; private final TransformationRequest transformationRequest; private final Format firstInputFormat; - @Nullable private final BitmapToHardwareBufferProcessor hardwareBufferPostProcessor; - private final Queue pendingQueueCalls; - private final Map inFlightFrames; + private final Queue> pendingPackets; + private final InFlightFrameManager inFlightFrameManager; private boolean hasPendingEos; private int outputRotationDegrees; private volatile boolean released; @@ -105,7 +100,7 @@ public PacketConsumerVideoSampleExporter( Format firstInputFormat, TransformationRequest transformationRequest, FrameProcessor.Factory frameProcessorFactory, - @Nullable HardwareBufferJniWrapper hardwareBufferJniWrapper, + HardwareBufferJniWrapper hardwareBufferJniWrapper, EncoderFactory encoderFactory, MuxerWrapper muxerWrapper, Consumer errorConsumer, @@ -121,8 +116,8 @@ public PacketConsumerVideoSampleExporter( this.errorConsumer = errorConsumer; this.muxerWrapper = muxerWrapper; this.firstInputFormat = firstInputFormat; - this.pendingQueueCalls = new ArrayDeque<>(); - this.inFlightFrames = new HashMap<>(); + this.pendingPackets = new ArrayDeque<>(); + this.inFlightFrameManager = new InFlightFrameManager(); finalFramePresentationTimeUs = C.TIME_UNSET; lastMuxerInputBufferTimestampUs = C.TIME_UNSET; encoderOutputBuffer = @@ -146,22 +141,22 @@ public PacketConsumerVideoSampleExporter( .build(); } + Executor playbackExecutor = new HandlerExecutor(playbackHandler, componentListener); if (SDK_INT >= 33) { frameWriter = new EncoderFrameWriter( strictEncoderFactory, componentListener, - playbackHandler::post, + playbackExecutor, playbackHandler, logSessionId); } else { - checkState(hardwareBufferJniWrapper != null); frameWriter = new GlEncoderFrameWriter( context, strictEncoderFactory, componentListener, - playbackHandler::post, + playbackExecutor, new DefaultGlObjectsProvider(), listeningDecorator(Util.newSingleThreadExecutor("GlEncoderFrameWriter::Thread")), hardwareBufferJniWrapper, @@ -170,24 +165,11 @@ public PacketConsumerVideoSampleExporter( frameProcessor = frameProcessorFactory.create( frameWriter, listenerExecutor, /* listener= */ componentListener); - if (hardwareBufferJniWrapper != null) { - hardwareBufferPostProcessor = - new BitmapToHardwareBufferProcessor( - hardwareBufferJniWrapper, - /* internalExecutor= */ Util.newSingleThreadExecutor( - "BitmapToHardwareBufferProcessor::Thread"), - /* errorExecutor= */ directExecutor(), - /* errorCallback= */ (e) -> - errorConsumer.accept( - ExportException.createForVideoFrameProcessingException( - VideoFrameProcessingException.from(e)))); - } else { - hardwareBufferPostProcessor = null; - } frameAggregator = new FrameAggregator( composition.sequences.size(), + composition.videoFrameAggregationParameters.frameRate, thisRef::queueAggregatedFrames, /* onFlush= */ (unused) -> {}); // Create the per sequence consumers that feed buffers from the decoders into the @@ -196,28 +178,22 @@ public PacketConsumerVideoSampleExporter( new ImmutableList.Builder<>(); for (int i = 0; i < composition.sequences.size(); i++) { int sequenceIndex = i; - Consumer frameConsumer = + Consumer frameConsumer = // TODO: b/478781219 - Remove the handlerWrapper.post once HardwareBufferSampleConsumer is // only accessed from a single thread. - (frame) -> + (asyncFrame) -> handlerWrapper.post( () -> { // Frames may be produced by the underlying players after Transformer has been // canceled. Immediately release these frames. if (released) { - if (frame != HardwareBufferFrame.END_OF_STREAM_FRAME) { - frame.release(/* releaseFence= */ null); - } + TransformerUtil.releaseIfNeeded(asyncFrame.frame, /* releaseFence= */ null); return; } - if (frame == HardwareBufferFrame.END_OF_STREAM_FRAME) { + if (asyncFrame == END_OF_STREAM_ASYNC_FRAME) { checkNotNull(frameAggregator).queueEndOfStream(sequenceIndex); - } else if (hardwareBufferPostProcessor != null) { - HardwareBufferFrame processedFrame = - hardwareBufferPostProcessor.process(frame); - checkNotNull(frameAggregator).queueFrame(processedFrame, sequenceIndex); } else { - checkNotNull(frameAggregator).queueFrame(frame, sequenceIndex); + checkNotNull(frameAggregator).queueFrame(asyncFrame, sequenceIndex); } }); HardwareBufferSampleConsumer sampleConsumer = @@ -227,21 +203,31 @@ public PacketConsumerVideoSampleExporter( playbackLooper, handlerWrapper, frameConsumer, - errorConsumer); + errorConsumer, + hardwareBufferJniWrapper); sampleConsumerBuilder.add(sampleConsumer); // TODO: b/496585841 - Handle single asset items with TRACK_TYPE_NONE. - // Ensure the FrameAggregator ignores audio only sequences. - boolean shouldAggregateSequence = + boolean sequenceContainsVideo = composition.sequences.get(sequenceIndex).trackTypes.contains(TRACK_TYPE_VIDEO); - frameAggregator.registerSequence(sequenceIndex, shouldAggregateSequence); + @AggregationStrategy int aggregationStrategy; + if (!sequenceContainsVideo) { + // Audio only sequences never produce video frames. + aggregationStrategy = STRATEGY_EXPECT_NO_FRAMES; + } else if (HardwareBufferFrameReader.CAPACITY >= 3) { + // Retaining the previous frame in FrameAggregator requires reader capacity >= 3 since + // InFlightFrameManager also holds a frame in flight downstream. + aggregationStrategy = STRATEGY_MATCH_FRAME_CLOSEST_TO_TARGET; + } else { + aggregationStrategy = STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET; + } + frameAggregator.registerSequence(sequenceIndex, aggregationStrategy); } sampleConsumers = sampleConsumerBuilder.build(); } - @SuppressWarnings("deprecation") - private void queueAggregatedFrames(ImmutableList frames) { - if (frames.get(0) == END_OF_STREAM_FRAME) { - if (pendingQueueCalls.isEmpty()) { + private void queueAggregatedFrames(ImmutableList frames) { + if (frames.get(0) == END_OF_STREAM_ASYNC_FRAME) { + if (pendingPackets.isEmpty()) { frameProcessor.signalEndOfStream(); } else { hasPendingEos = true; @@ -249,39 +235,8 @@ private void queueAggregatedFrames(ImmutableList frames) { return; } - ImmutableList.Builder asyncFrameListBuilder = ImmutableList.builder(); - for (int i = 0; i < frames.size(); i++) { - HardwareBufferFrame effectFrame = frames.get(i); - // When encoding, releaseTime and contentTime are the same. - checkState(effectFrame.releaseTimeNs == effectFrame.sequencePresentationTimeUs * 1000); - - ImmutableMap.Builder metadataBuilder = ImmutableMap.builder(); - metadataBuilder - .put(Frame.KEY_PRESENTATION_TIME_US, effectFrame.presentationTimeUs) - .put(Frame.KEY_DISPLAY_TIME_NS, effectFrame.releaseTimeNs); - if (effectFrame.getMetadata() instanceof CompositionFrameMetadata) { - CompositionFrameMetadata compositionFrameMetadata = - (CompositionFrameMetadata) effectFrame.getMetadata(); - metadataBuilder - .put(CompositionFrameMetadata.KEY_COMPOSITION_FRAME_METADATA, compositionFrameMetadata) - .putAll(asFrameMetadata(compositionFrameMetadata)); - } - - HardwareBuffer hardwareBuffer = checkNotNull(effectFrame.hardwareBuffer); - DefaultHardwareBufferFrame commonFrame = - new DefaultHardwareBufferFrame.Builder(hardwareBuffer) - .setFormat(effectFrame.format) - .setContentTimeUs(effectFrame.sequencePresentationTimeUs) - .setMetadata(metadataBuilder.buildOrThrow()) - .setInternalImage(effectFrame.internalFrame) - .build(); - - asyncFrameListBuilder.add(new AsyncFrame(commonFrame, effectFrame.acquireFence)); - } - - PendingQueueCall call = new PendingQueueCall(asyncFrameListBuilder.build(), frames); - pendingQueueCalls.add(call); - drainPendingQueueCalls(); + pendingPackets.add(frames); + drainPendingPackets(); } @Override @@ -295,13 +250,11 @@ public void release() { return; } released = true; - releasePendingQueueCalls(); + releasePendingPackets(); + inFlightFrameManager.releaseAll(); for (int i = 0; i < sampleConsumers.size(); i++) { sampleConsumers.get(i).release(); } - if (hardwareBufferPostProcessor != null) { - hardwareBufferPostProcessor.close(); - } frameAggregator.close(); try { frameProcessor.close(); @@ -375,7 +328,8 @@ protected boolean isMuxerInputEnded() { private final class ComponentListener implements GlEncoderFrameWriter.Listener, EncoderFrameWriter.Listener, - FrameProcessor.Listener { + FrameProcessor.Listener, + HandlerExecutor.Listener { @Override public Format onConfigure(Format requestedFormat) { @@ -421,52 +375,63 @@ public void onEndOfStream() { finalFramePresentationTimeUs = C.TIME_UNSET; } - @Override - public void onError(VideoFrameProcessingException e) { - errorConsumer.accept(ExportException.createForVideoFrameProcessingException(e)); - } - // FrameProcessor.Listener methods @Override public void onWakeup() { - drainPendingQueueCalls(); + drainPendingPackets(); } @Override public void onFrameProcessed(Frame frame, @Nullable SyncFenceWrapper releaseFence) { - checkNotNull(inFlightFrames.remove(frame)).release(releaseFence); + inFlightFrameManager.onFrameProcessed(frame, releaseFence); + } + + @Override + public void onError(VideoFrameProcessingException e) { + errorConsumer.accept(ExportException.createForVideoFrameProcessingException(e)); + } + + // HandlerExecutor.Listener methods + + @Override + public void onError(RuntimeException e) { + onError(VideoFrameProcessingException.from(e)); } } - private void drainPendingQueueCalls() { - while (!pendingQueueCalls.isEmpty()) { - PendingQueueCall call = pendingQueueCalls.peek(); - if (call == null) { + private void drainPendingPackets() { + while (!pendingPackets.isEmpty()) { + ImmutableList packet = pendingPackets.peek(); + if (packet == null) { break; } - boolean queued = frameProcessor.queue(call.asyncFrames); + boolean queued = inFlightFrameManager.trackIfSuccessful(frameProcessor::queue, packet); if (queued) { - for (int i = 0; i < call.asyncFrames.size(); i++) { - inFlightFrames.put(call.asyncFrames.get(i).frame, call.hardwareBufferFrames.get(i)); - } - pendingQueueCalls.poll(); + pendingPackets.poll(); } else { break; } } - if (pendingQueueCalls.isEmpty() && hasPendingEos) { + if (pendingPackets.isEmpty() && hasPendingEos) { frameProcessor.signalEndOfStream(); hasPendingEos = false; } } - private void releasePendingQueueCalls() { - PendingQueueCall call; - while ((call = pendingQueueCalls.poll()) != null) { - for (HardwareBufferFrame frame : call.hardwareBufferFrames) { - frame.release(/* releaseFence= */ null); - } + private void releasePendingPackets() { + @Nullable ImmutableList packet; + while ((packet = pendingPackets.poll()) != null) { + releasePacket(packet); + } + } + + private static void releasePacket(@Nullable List packet) { + if (packet == null) { + return; + } + for (int i = 0; i < packet.size(); i++) { + TransformerUtil.releaseIfNeeded(packet.get(i).frame, /* releaseFence= */ null); } } @@ -481,38 +446,4 @@ private static String getRequestedOutputMimeType( return inputSampleMimeType; } } - - private static final class PendingQueueCall { - final ImmutableList asyncFrames; - final ImmutableList hardwareBufferFrames; - - PendingQueueCall( - ImmutableList asyncFrames, - ImmutableList hardwareBufferFrames) { - this.asyncFrames = asyncFrames; - this.hardwareBufferFrames = hardwareBufferFrames; - } - } - - private static final class HandlerExecutor implements Executor { - private final HandlerWrapper handler; - private final ComponentListener componentListener; - - private HandlerExecutor(HandlerWrapper handler, ComponentListener componentListener) { - this.handler = handler; - this.componentListener = componentListener; - } - - @Override - public void execute(Runnable command) { - handler.post( - () -> { - try { - command.run(); - } catch (RuntimeException e) { - componentListener.onError(VideoFrameProcessingException.from(e)); - } - }); - } - } } diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/PassthroughAudioCodec.java b/libraries/transformer/src/main/java/androidx/media3/transformer/PassthroughAudioCodec.java new file mode 100644 index 00000000000..7aaa7386705 --- /dev/null +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/PassthroughAudioCodec.java @@ -0,0 +1,179 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.transformer; + +import static com.google.common.base.Preconditions.checkArgument; +import static com.google.common.base.Preconditions.checkState; + +import android.media.MediaCodec; +import android.media.MediaCodec.BufferInfo; +import android.view.Surface; +import androidx.annotation.Nullable; +import androidx.media3.common.Format; +import androidx.media3.common.MimeTypes; +import androidx.media3.common.util.Util; +import androidx.media3.decoder.DecoderInputBuffer; +import java.nio.ByteBuffer; +import java.nio.ByteOrder; +import java.util.Objects; + +/** A passthrough {@link Codec} implementation for PCM audio samples. */ +/* package */ final class PassthroughAudioCodec implements Codec { + + private static final int DEFAULT_BUFFER_SIZE_FRAMES = 4096; + + private final Format format; + private final ByteBuffer buffer; + private final BufferInfo bufferInfo; + + private boolean isInputBufferDequeued; + private boolean isOutputBufferAvailable; + private boolean isInputStreamEnded; + private boolean isReleased; + + public PassthroughAudioCodec(Format format) { + checkArgument(Objects.equals(format.sampleMimeType, MimeTypes.AUDIO_RAW)); + this.format = format; + int frameSize = Util.getPcmFrameSize(format.pcmEncoding, format.channelCount); + this.buffer = + ByteBuffer.allocateDirect(DEFAULT_BUFFER_SIZE_FRAMES * frameSize) + .order(ByteOrder.nativeOrder()); + this.bufferInfo = new BufferInfo(); + } + + @Override + public String getName() { + return "androidx.media3.transformer.PassthroughAudioCodec"; + } + + @Override + public Format getConfigurationFormat() { + return format; + } + + @Override + public Surface getInputSurface() { + throw new UnsupportedOperationException(); + } + + @Override + public boolean maybeDequeueInputBuffer(DecoderInputBuffer inputBuffer) { + checkState(!isReleased); + if (isInputBufferDequeued || isOutputBufferAvailable || isInputStreamEnded) { + return false; + } + inputBuffer.clear(); + inputBuffer.data = buffer; + isInputBufferDequeued = true; + return true; + } + + @Override + // We explicitly want to compare instances between inputBuffer.data and buffer and avoid comparing + // the contents of the buffer. + @SuppressWarnings("ReferenceEquality") + public void queueInputBuffer(DecoderInputBuffer inputBuffer) { + checkState(!isReleased); + checkState(isInputBufferDequeued, "Input buffer is not dequeued."); + checkState(!isInputStreamEnded, "Input buffer cannot be queued after EoS."); + checkArgument(inputBuffer.data != null); + checkArgument(inputBuffer.data == buffer); + + int offset = 0; + int size = 0; + int flags = 0; + + // Ignore data when EoS is queued. + if (inputBuffer.isEndOfStream()) { + checkState(!inputBuffer.data.hasRemaining(), "EoS buffer should have no remaining data."); + isInputStreamEnded = true; + flags = MediaCodec.BUFFER_FLAG_END_OF_STREAM; + } else { + offset = inputBuffer.data.position(); + size = inputBuffer.data.remaining(); + } + + bufferInfo.set(offset, size, inputBuffer.timeUs, flags); + isInputBufferDequeued = false; + isOutputBufferAvailable = true; + + // Clear state or release reference from inputBuffer + inputBuffer.data = null; + } + + @Override + public void signalEndOfInputStream() { + throw new UnsupportedOperationException(); + } + + @Override + public Format getInputFormat() { + return format; + } + + @Override + public Format getOutputFormat() { + return format; + } + + @Override + @Nullable + public ByteBuffer getOutputBuffer() { + checkState(!isReleased); + if (!isOutputBufferAvailable) { + return null; + } + // Return a duplicate to prevent the caller from modifying buffer's positions. + return buffer.duplicate().order(buffer.order()); + } + + @Override + @Nullable + public BufferInfo getOutputBufferInfo() { + checkState(!isReleased); + return isOutputBufferAvailable ? bufferInfo : null; + } + + @Override + public void releaseOutputBuffer(boolean render) { + checkState(!isReleased); + checkArgument(!render); + checkState(isOutputBufferAvailable); + isOutputBufferAvailable = false; + buffer.clear(); + } + + @Override + public void releaseOutputBuffer(long renderPresentationTimeUs) { + throw new UnsupportedOperationException(); + } + + @Override + public boolean isEnded() { + return isInputStreamEnded && !isOutputBufferAvailable; + } + + @Override + public void release() { + checkState(!isReleased); + isReleased = true; + isInputBufferDequeued = false; + isOutputBufferAvailable = false; + isInputStreamEnded = false; + buffer.clear(); + } +} diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/RendererTracker.java b/libraries/transformer/src/main/java/androidx/media3/transformer/RendererTracker.java new file mode 100644 index 00000000000..3955c56c741 --- /dev/null +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/RendererTracker.java @@ -0,0 +1,140 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static com.google.common.base.Preconditions.checkNotNull; + +import androidx.annotation.Nullable; +import androidx.media3.common.C; +import androidx.media3.common.Timeline; +import androidx.media3.exoplayer.Renderer; +import androidx.media3.exoplayer.source.MediaSource.MediaPeriodId; +import java.util.HashSet; +import java.util.Set; + +/** + * Tracker for active media periods in the player. + * + *

    This class tracks the set of {@linkplain Renderer#STATE_ENABLED enabled} {@link Renderer}s and + * determines which one is currently "active" (i.e., corresponds to the earliest period in the + * player's timeline). + * + *

    Methods in this class are not thread-safe and must be called from the ExoPlayer playback + * thread. + */ +/* package */ final class RendererTracker { + + /** + * A {@link Renderer} that provides its current {@link MediaPeriodId} and {@link Timeline} for + * tracking. + */ + /* package */ interface TrackedRenderer extends Renderer { + /** Returns the {@link MediaPeriodId} of the renderer, or null if unset. */ + @Nullable + MediaPeriodId getTrackedMediaPeriodId(); + + /** Returns the current {@link Timeline} containing the renderer's stream, or null if unset. */ + @Nullable + Timeline getTrackedTimeline(); + } + + private final Set renderers = new HashSet<>(); + + public RendererTracker() {} + + /** + * Registers a renderer. + * + *

    Should be called when the renderer is enabled. + */ + public void addRenderer(TrackedRenderer renderer) { + renderers.add(renderer); + } + + /** + * Unregisters a renderer. + * + *

    Should be called when the renderer is disabled. + */ + public void removeRenderer(TrackedRenderer renderer) { + renderers.remove(renderer); + } + + /** + * Returns the currently active renderer from the registered ones, or null if none are active. + * + *

    The active renderer is the one associated with the earliest period in its current {@link + * Timeline}. + */ + @Nullable + public Renderer getActiveRenderer() { + if (renderers.isEmpty()) { + return null; + } + + @Nullable Renderer activeRenderer = null; + @Nullable MediaPeriodId activePeriodId = null; + int activePeriodIndex = C.INDEX_UNSET; + + for (TrackedRenderer renderer : renderers) { + Timeline timeline = renderer.getTrackedTimeline(); + if (timeline == null || timeline.isEmpty()) { + continue; + } + MediaPeriodId periodId = renderer.getTrackedMediaPeriodId(); + if (periodId == null) { + // The renderer is enabled but has not yet been assigned a media period. + continue; + } + int periodIndex = timeline.getIndexOfPeriod(periodId.periodUid); + if (periodIndex == C.INDEX_UNSET) { + // The renderer's period does not exist in the current timeline (e.g. from an earlier + // playlist state or a removed item), so ignore it. + continue; + } + + if (activeRenderer == null + || comparePeriods(periodId, periodIndex, checkNotNull(activePeriodId), activePeriodIndex) + < 0) { + activeRenderer = renderer; + activePeriodId = periodId; + activePeriodIndex = periodIndex; + } + } + + return activeRenderer; + } + + /** + * Compares two periods to determine which one is earlier in playback order. + * + *

    Comparison prioritizes {@link MediaPeriodId#windowSequenceNumber} to correctly order periods + * across playlist window transitions and repeated/looping sequences. If window sequence numbers + * are identical, relative order is determined by period index within the {@link Timeline}. + */ + private static int comparePeriods( + MediaPeriodId p1, int periodIndex1, MediaPeriodId p2, int periodIndex2) { + // Window sequence numbers are assigned monotonically by ExoPlayer in playback order. + // Periods with smaller window sequence numbers are played earlier, even if periods repeat + // in a loop or playlist. + if (p1.windowSequenceNumber != p2.windowSequenceNumber) { + return Long.compare(p1.windowSequenceNumber, p2.windowSequenceNumber); + } + // For periods within the same window sequence, order is determined by their index in the + // timeline. + return Integer.compare(periodIndex1, periodIndex2); + } +} diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/ResumedExportOperation.java b/libraries/transformer/src/main/java/androidx/media3/transformer/ResumedExportOperation.java index ba4ddd62a36..732e6af60ea 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/ResumedExportOperation.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/ResumedExportOperation.java @@ -138,7 +138,7 @@ @Nullable private final FrameProcessor.Factory frameProcessorFactory; - @Nullable HardwareBufferJniWrapper hardwareBufferJniWrapper; + @Nullable private final HardwareBufferJniWrapper hardwareBufferJniWrapper; @Nullable private final LogSessionId logSessionId; private final boolean applyMp4EditListTrim; diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/SampleExporter.java b/libraries/transformer/src/main/java/androidx/media3/transformer/SampleExporter.java index 0e9d18dd274..ab7172cece7 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/SampleExporter.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/SampleExporter.java @@ -31,9 +31,11 @@ import androidx.media3.common.MimeTypes; import androidx.media3.decoder.DecoderInputBuffer; import androidx.media3.muxer.MuxerException; +import androidx.media3.transformer.Codec.EncoderFactory; import com.google.common.collect.ImmutableList; import com.google.common.collect.ImmutableSet; import java.util.List; +import java.util.Objects; /** * Exporter that processes media data. @@ -152,6 +154,19 @@ private boolean feedMuxer() throws ExportException { return true; } + /** + * Finds a {@linkplain MimeTypes MIME type} that is supported by the encoder and the muxer. + * + *

    This is identical to calling {@code + * #findSupportedMimeTypeForEncoderAndMuxer(requestedFormat, muxerSupportedMimeTypes, null)}. + */ + @Nullable + protected static String findSupportedMimeTypeForEncoderAndMuxer( + Format requestedFormat, List muxerSupportedMimeTypes) { + return findSupportedMimeTypeForEncoderAndMuxer( + requestedFormat, muxerSupportedMimeTypes, /* encoderFactory= */ null); + } + /** * Finds a {@linkplain MimeTypes MIME type} that is supported by the encoder and the muxer. * @@ -169,11 +184,14 @@ private boolean feedMuxer() throws ExportException { * @param requestedFormat The {@link Format} requested. * @param muxerSupportedMimeTypes The list of sample {@linkplain MimeTypes MIME types} that the * muxer supports. + * @param encoderFactory The provided {@link EncoderFactory}. * @return A supported {@linkplain MimeTypes MIME type}, or {@code null} if none are supported. */ @Nullable protected static String findSupportedMimeTypeForEncoderAndMuxer( - Format requestedFormat, List muxerSupportedMimeTypes) { + Format requestedFormat, + List muxerSupportedMimeTypes, + @Nullable EncoderFactory encoderFactory) { boolean isVideo = MimeTypes.isVideo(checkNotNull(requestedFormat.sampleMimeType)); ImmutableSet.Builder mimeTypesToCheckSetBuilder = @@ -195,7 +213,10 @@ protected static String findSupportedMimeTypeForEncoderAndMuxer( if (!getSupportedEncodersForHdrEditing(mimeType, requestedFormat.colorInfo).isEmpty()) { return mimeType; } - } else if (!getSupportedEncoders(mimeType).isEmpty()) { + } else if ((Objects.equals(requestedFormat.sampleMimeType, MimeTypes.AUDIO_RAW) + && encoderFactory != null + && !encoderFactory.audioNeedsEncoding()) + || !getSupportedEncoders(mimeType).isEmpty()) { return mimeType; } } diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/SequenceAssetLoader.java b/libraries/transformer/src/main/java/androidx/media3/transformer/SequenceAssetLoader.java index d76e65ec2bd..189bd1055c9 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/SequenceAssetLoader.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/SequenceAssetLoader.java @@ -312,7 +312,7 @@ public boolean onTrackAdded(Format inputFormat, @SupportedOutputTypes int suppor @Nullable @Override - public SampleConsumerWrapper onOutputFormat(Format format) throws ExportException { + public SampleConsumer onOutputFormat(Format format) throws ExportException { @C.TrackType int trackType = getProcessedTrackType(format.sampleMimeType); DebugTraceUtil.logEvent( COMPONENT_ASSET_LOADER, @@ -517,8 +517,7 @@ private final class SampleConsumerWrapper implements SampleConsumer { private final @C.TrackType int trackType; private long totalDurationUs; - private boolean audioLoopingEnded; - private boolean videoLoopingEnded; + private boolean isLoopingEnded; public SampleConsumerWrapper(SampleConsumer sampleConsumer, @C.TrackType int trackType) { this.sampleConsumer = sampleConsumer; @@ -528,21 +527,23 @@ public SampleConsumerWrapper(SampleConsumer sampleConsumer, @C.TrackType int tra @Nullable @Override public DecoderInputBuffer getInputBuffer() { + checkState(trackType == C.TRACK_TYPE_AUDIO || !decodeVideo); return sampleConsumer.getInputBuffer(); } @Override public boolean queueInputBuffer() { + checkState(trackType == C.TRACK_TYPE_AUDIO || !decodeVideo); DecoderInputBuffer inputBuffer = checkNotNull(sampleConsumer.getInputBuffer()); long globalTimestampUs = totalDurationUs + inputBuffer.timeUs; - if (isLooping && (globalTimestampUs >= maxSequenceDurationUs || audioLoopingEnded)) { - if (isMaxSequenceDurationUsFinal && !audioLoopingEnded) { + if (isLooping && (globalTimestampUs >= maxSequenceDurationUs || isLoopingEnded)) { + if (isMaxSequenceDurationUsFinal && !isLoopingEnded) { checkNotNull(inputBuffer.data).limit(0); inputBuffer.setFlags(C.BUFFER_FLAG_END_OF_STREAM); // We know that queueInputBuffer() will always return true for the underlying // SampleConsumer so there is no need to handle the case where the sample wasn't queued. checkState(sampleConsumer.queueInputBuffer()); - audioLoopingEnded = true; + isLoopingEnded = true; nonEndedTrackCount.decrementAndGet(); } return false; @@ -574,6 +575,7 @@ public boolean queueInputBuffer() { @Override public @InputResult int queueInputBitmap( Bitmap inputBitmap, TimestampIterator timestampIterator) { + checkState(trackType == C.TRACK_TYPE_VIDEO && decodeVideo); if (isLooping) { long lastOffsetUs = C.TIME_UNSET; while (timestampIterator.hasNext()) { @@ -583,15 +585,15 @@ public boolean queueInputBuffer() { return INPUT_RESULT_TRY_AGAIN_LATER; } if (lastOffsetUs == C.TIME_UNSET) { - if (!videoLoopingEnded) { - videoLoopingEnded = true; + if (!isLoopingEnded) { + isLoopingEnded = true; signalEndOfVideoInput(); return INPUT_RESULT_END_OF_STREAM; } return INPUT_RESULT_TRY_AGAIN_LATER; } timestampIterator = new ClippingIterator(timestampIterator.copyOf(), lastOffsetUs); - videoLoopingEnded = true; + isLoopingEnded = true; break; } lastOffsetUs = offsetUs; @@ -602,20 +604,23 @@ public boolean queueInputBuffer() { @Override public void setOnInputFrameProcessedListener(OnInputFrameProcessedListener listener) { + checkState(trackType == C.TRACK_TYPE_VIDEO && decodeVideo); sampleConsumer.setOnInputFrameProcessedListener(listener); } @Override public void setOnInputSurfaceReadyListener(Runnable runnable) { + checkState(trackType == C.TRACK_TYPE_VIDEO && decodeVideo); sampleConsumer.setOnInputSurfaceReadyListener(runnable); } @Override public @InputResult int queueInputTexture(int texId, long presentationTimeUs) { + checkState(trackType == C.TRACK_TYPE_VIDEO && decodeVideo); long globalTimestampUs = totalDurationUs + presentationTimeUs; if (isLooping && globalTimestampUs >= maxSequenceDurationUs) { - if (isMaxSequenceDurationUsFinal && !videoLoopingEnded) { - videoLoopingEnded = true; + if (isMaxSequenceDurationUsFinal && !isLoopingEnded) { + isLoopingEnded = true; signalEndOfVideoInput(); return INPUT_RESULT_END_OF_STREAM; } @@ -626,20 +631,23 @@ public void setOnInputSurfaceReadyListener(Runnable runnable) { @Override public Surface getInputSurface() { + checkState(trackType == C.TRACK_TYPE_VIDEO && decodeVideo); return sampleConsumer.getInputSurface(); } @Override public int getPendingVideoFrameCount() { + checkState(trackType == C.TRACK_TYPE_VIDEO && decodeVideo); return sampleConsumer.getPendingVideoFrameCount(); } @Override public boolean registerVideoFrame(long presentationTimeUs) { + checkState(trackType == C.TRACK_TYPE_VIDEO && decodeVideo); long globalTimestampUs = totalDurationUs + presentationTimeUs; if (isLooping && globalTimestampUs >= maxSequenceDurationUs) { - if (isMaxSequenceDurationUsFinal && !videoLoopingEnded) { - videoLoopingEnded = true; + if (isMaxSequenceDurationUsFinal && !isLoopingEnded) { + isLoopingEnded = true; signalEndOfVideoInput(); } return false; @@ -650,8 +658,9 @@ public boolean registerVideoFrame(long presentationTimeUs) { @Override public void signalEndOfVideoInput() { + checkState(trackType == C.TRACK_TYPE_VIDEO && decodeVideo); nonEndedTrackCount.decrementAndGet(); - boolean videoEnded = isLooping ? videoLoopingEnded : isLastMediaItemInSequence(); + boolean videoEnded = isLooping ? isLoopingEnded : isLastMediaItemInSequence(); if (videoEnded) { sampleConsumer.signalEndOfVideoInput(); } else if (nonEndedTrackCount.get() == 0) { @@ -660,6 +669,7 @@ public void signalEndOfVideoInput() { } private void onAudioGapSignalled() { + checkState(trackType == C.TRACK_TYPE_AUDIO); int nonEndedTracks = nonEndedTrackCount.decrementAndGet(); if (nonEndedTracks == 0 && !isLastMediaItemInSequence()) { switchAssetLoader(); @@ -811,19 +821,17 @@ private void outputFormatToSequenceAssetLoader() { try { boolean shouldRetry = false; if (audioPending) { - @Nullable - SampleConsumerWrapper sampleConsumerWrapper = onOutputFormat(audioTrackDecodedFormat); - if (sampleConsumerWrapper == null) { + @Nullable SampleConsumer sampleConsumer = onOutputFormat(audioTrackDecodedFormat); + if (sampleConsumer == null) { shouldRetry = true; } else { - sampleConsumerWrapper.onAudioGapSignalled(); + ((SampleConsumerWrapper) sampleConsumer).onAudioGapSignalled(); producedAudio = true; } } if (videoPending) { - @Nullable - SampleConsumerWrapper sampleConsumerWrapper = onOutputFormat(BLANK_IMAGE_BITMAP_FORMAT); - if (sampleConsumerWrapper == null) { + @Nullable SampleConsumer sampleConsumer = onOutputFormat(BLANK_IMAGE_BITMAP_FORMAT); + if (sampleConsumer == null) { shouldRetry = true; } else { insertBlankFrames(getBlankImageBitmap()); diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/SequenceRenderersFactory.java b/libraries/transformer/src/main/java/androidx/media3/transformer/SequenceRenderersFactory.java index 6daeb46ed34..1ae570a5259 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/SequenceRenderersFactory.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/SequenceRenderersFactory.java @@ -25,6 +25,7 @@ import static androidx.media3.exoplayer.video.VideoSink.RELEASE_FIRST_FRAME_WHEN_PREVIOUS_STREAM_PROCESSED; import static androidx.media3.exoplayer.video.VideoSink.RELEASE_FIRST_FRAME_WHEN_STARTED; import static androidx.media3.transformer.TransformerUtil.getEditedMediaItem; +import static androidx.media3.transformer.TransformerUtil.getEditedMediaItemIndex; import static androidx.media3.transformer.TransformerUtil.getOffsetToCompositionTimeUs; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.base.Preconditions.checkState; @@ -32,7 +33,6 @@ import android.content.Context; import android.graphics.Bitmap; -import android.media.MediaCodec; import android.media.MediaFormat; import android.os.Handler; import android.os.SystemClock; @@ -45,7 +45,6 @@ import androidx.media3.common.Timeline; import androidx.media3.common.util.ConstantRateTimestampIterator; import androidx.media3.common.util.NullableType; -import androidx.media3.decoder.DecoderInputBuffer; import androidx.media3.exoplayer.ExoPlaybackException; import androidx.media3.exoplayer.Renderer; import androidx.media3.exoplayer.RenderersFactory; @@ -121,6 +120,8 @@ void onRender( @Nullable private final CompositionRendererListener compositionRendererListener; private final Supplier<@NullableType HardwareBufferFrameReader> hardwareBufferFrameReaderSupplier; private final long lateThresholdToDropInputUs; + private final RendererTracker rendererTracker; + private final OutputTracker outputTracker; private @MonotonicNonNull SequenceAudioRenderer audioRenderer; private @MonotonicNonNull SequenceVideoRenderer primaryVideoRenderer; @@ -189,6 +190,8 @@ private SequenceRenderersFactory( this.compositionRendererListener = compositionRendererListener; this.hardwareBufferFrameReaderSupplier = hardwareBufferFrameReaderSupplier; this.lateThresholdToDropInputUs = lateThresholdToDropInputUs; + rendererTracker = new RendererTracker(); + outputTracker = new OutputTracker(); } public void setRequestMediaCodecToneMapping(boolean requestMediaCodecToneMapping) { @@ -238,12 +241,15 @@ public Renderer[] createRenderers( videoRendererEventListener, checkNotNull(compositionRendererListener), hardwareBufferFrameReaderSupplier, - lateThresholdToDropInputUs)); + lateThresholdToDropInputUs, + rendererTracker, + outputTracker)); renderers.add( new HardwareBufferImageRenderer( checkNotNull(imageDecoderFactory), checkNotNull(compositionRendererListener), - hardwareBufferFrameReaderSupplier)); + hardwareBufferFrameReaderSupplier, + rendererTracker)); } return renderers.toArray(new Renderer[0]); } @@ -275,7 +281,16 @@ public Renderer createSecondaryRenderer( videoRendererEventListener, checkNotNull(compositionRendererListener), hardwareBufferFrameReaderSupplier, - lateThresholdToDropInputUs); + lateThresholdToDropInputUs, + rendererTracker, + outputTracker); + } + if (renderer instanceof HardwareBufferImageRenderer) { + return new HardwareBufferImageRenderer( + checkNotNull(imageDecoderFactory), + checkNotNull(compositionRendererListener), + hardwareBufferFrameReaderSupplier, + rendererTracker); } return null; } @@ -407,17 +422,6 @@ protected void onStreamChanged( super.onStreamChanged(formats, startPositionUs, offsetUs, mediaPeriodId); } - @Override - protected void onQueueInputBuffer(DecoderInputBuffer buffer) throws ExoPlaybackException { - targetFrameRateHelper.onQueueInputBuffer(buffer, getCodecInputFormat()); - super.onQueueInputBuffer(buffer); - } - - @Override - protected int getCodecBufferFlags(DecoderInputBuffer buffer) { - return super.getCodecBufferFlags(buffer) | targetFrameRateHelper.getCodecBufferFlags(buffer); - } - @Override protected boolean processOutputBuffer( long positionUs, @@ -581,6 +585,7 @@ private static final class SequenceImageRenderer extends ImageRenderer { @Nullable private ImageMetadataListener imageMetadataListener; private @MonotonicNonNull Format outputFormat; private long streamOffsetUs; + private boolean listenerSet; public SequenceImageRenderer(ImageDecoder.Factory imageDecoderFactory, VideoSink videoSink) { super(imageDecoderFactory, ImageOutput.NO_OP); @@ -600,17 +605,21 @@ protected void onEnabled(boolean joining, boolean mayRenderStartOfStream) mayRenderStartOfStream ? RELEASE_FIRST_FRAME_IMMEDIATELY : RELEASE_FIRST_FRAME_WHEN_STARTED; - // TODO: b/328444280 - Unregister as a listener when the renderer is not used anymore - videoSink.setListener( - new VideoSink.Listener() { - @Override - public void onFrameAvailableForRendering() { - if (wakeupListener != null) { - wakeupListener.onWakeup(); - } - } - }, - directExecutor()); + if (mayRenderStartOfStream) { + maybeSetVideoSinkListener(); + } + } + + @Override + protected void onStarted() throws ExoPlaybackException { + super.onStarted(); + maybeSetVideoSinkListener(); + } + + @Override + protected void onDisabled() { + super.onDisabled(); + listenerSet = false; } @Override @@ -755,6 +764,23 @@ public void handleMessage(@MessageType int messageType, @Nullable Object message } } + private void maybeSetVideoSinkListener() { + if (!listenerSet) { + // TODO: b/328444280 - Unregister as a listener when the renderer is not used anymore + videoSink.setListener( + new VideoSink.Listener() { + @Override + public void onFrameAvailableForRendering() { + if (wakeupListener != null) { + wakeupListener.onWakeup(); + } + } + }, + directExecutor()); + listenerSet = true; + } + } + private ConstantRateTimestampIterator createTimestampIterator(long positionUs) { EditedMediaItem editedMediaItem = getEditedMediaItem(getTimeline(), checkNotNull(getMediaPeriodId())); @@ -771,8 +797,10 @@ private ConstantRateTimestampIterator createTimestampIterator(long positionUs) { * HardwareBufferFrameReader}. */ private static final class HardwareBufferVideoRenderer extends MediaCodecVideoRenderer - implements RendererWakeupListener { + implements RendererWakeupListener, RendererTracker.TrackedRenderer { private final CompositionRendererListener compositionRendererListener; + private final RendererTracker rendererTracker; + private final OutputTracker outputTracker; private @MonotonicNonNull HardwareBufferFrameReader hardwareBufferFrameReader; private final long lateThresholdToDropInputUs; private final TargetFrameRateHelper targetFrameRateHelper; @@ -780,12 +808,12 @@ private static final class HardwareBufferVideoRenderer extends MediaCodecVideoRe private final Supplier<@NullableType HardwareBufferFrameReader> hardwareBufferFrameReaderSupplier; - private MediaSource.@MonotonicNonNull MediaPeriodId mediaPeriodId; private @MonotonicNonNull Format nextFormat; @Nullable private VideoFrameMetadataListener frameMetadataListener; private long streamStartPositionUs; private long offsetToCompositionTimeUs; - private boolean hasOutputSurface = false; + + private boolean isReadyToRenderFirstFrame; private HardwareBufferVideoRenderer( Context context, @@ -793,7 +821,9 @@ private HardwareBufferVideoRenderer( VideoRendererEventListener videoRendererEventListener, CompositionRendererListener compositionRendererListener, Supplier<@NullableType HardwareBufferFrameReader> hardwareBufferFrameReaderSupplier, - long lateThresholdToDropInputUs) { + long lateThresholdToDropInputUs, + RendererTracker rendererTracker, + OutputTracker outputTracker) { super( new Builder(context) .setMediaCodecSelector(MediaCodecSelector.DEFAULT) @@ -804,11 +834,26 @@ private HardwareBufferVideoRenderer( .setEventListener(videoRendererEventListener) .setMaxDroppedFramesToNotify(MAX_DROPPED_VIDEO_FRAME_COUNT_TO_NOTIFY) .setAssumedMinimumCodecOperatingRate(DEFAULT_FRAME_RATE) - .experimentalSetLateThresholdToDropDecoderInputUs(lateThresholdToDropInputUs)); + .experimentalSetLateThresholdToDropDecoderInputUs(lateThresholdToDropInputUs) + .setSkipBuffersWithIdenticalReleaseTime(false)); this.compositionRendererListener = compositionRendererListener; this.hardwareBufferFrameReaderSupplier = hardwareBufferFrameReaderSupplier; this.lateThresholdToDropInputUs = lateThresholdToDropInputUs; this.targetFrameRateHelper = new TargetFrameRateHelper(); + this.rendererTracker = rendererTracker; + this.outputTracker = outputTracker; + } + + @Override + @Nullable + public MediaPeriodId getTrackedMediaPeriodId() { + return getMediaPeriodId(); + } + + @Override + @Nullable + public Timeline getTrackedTimeline() { + return getTimeline(); } @Override @@ -820,6 +865,7 @@ protected void onEnabled(boolean joining, boolean mayRenderStartOfStream) } hardwareBufferFrameReader.addRendererWakeupListener(/* rendererWakeupListener= */ this); super.onEnabled(joining, mayRenderStartOfStream); + rendererTracker.addRenderer(this); } @Override @@ -827,6 +873,7 @@ protected void onDisabled() { super.onDisabled(); checkNotNull(hardwareBufferFrameReader) .removeRendererWakeupListener(/* rendererWakeupListener= */ this); + rendererTracker.removeRenderer(this); } @Override @@ -846,6 +893,7 @@ protected void onPositionReset( throws ExoPlaybackException { super.onPositionReset(positionUs, joining, sampleStreamIsResetToKeyFrame); targetFrameRateHelper.onPositionReset(); + isReadyToRenderFirstFrame = false; } @Override @@ -856,9 +904,10 @@ protected void onStreamChanged( MediaSource.MediaPeriodId mediaPeriodId) throws ExoPlaybackException { checkState(getTimeline().getWindowCount() == 1); - this.mediaPeriodId = mediaPeriodId; + // The media item might have been repeated in the sequence. EditedMediaItem editedMediaItem = getEditedMediaItem(getTimeline(), mediaPeriodId); + isReadyToRenderFirstFrame = false; // The renderer has started processing this item, VideoGraph might still be processing the // previous one. streamStartPositionUs = startPositionUs; @@ -868,17 +917,6 @@ protected void onStreamChanged( super.onStreamChanged(formats, startPositionUs, offsetUs, mediaPeriodId); } - @Override - protected void onQueueInputBuffer(DecoderInputBuffer buffer) throws ExoPlaybackException { - targetFrameRateHelper.onQueueInputBuffer(buffer, getCodecInputFormat()); - super.onQueueInputBuffer(buffer); - } - - @Override - protected int getCodecBufferFlags(DecoderInputBuffer buffer) { - return super.getCodecBufferFlags(buffer) | targetFrameRateHelper.getCodecBufferFlags(buffer); - } - @Override protected boolean processOutputBuffer( long positionUs, @@ -893,13 +931,34 @@ protected boolean processOutputBuffer( boolean isLastBuffer, Format format) throws ExoPlaybackException { + checkNotNull(codec); + // Allow decode only buffers to be dropped before this renderer is ready to output frames, + // so the first frame that will be processed is guaranteed to be output. + if (isDecodeOnlyBuffer && !isLastBuffer) { + skipOutputBuffer(codec, bufferIndex, bufferPresentationTimeUs); + return true; + } // When prewarming is enabled this method will be called when the renderer is enabled, which // is well before item should be displayed. Frames should not be rendered until a Surface is - // set on this renderer. - if (!hasOutputSurface) { + // set on this renderer and this renderer is either started, or enabled with + // mayRenderStartOfStream. + Renderer activeRenderer = rendererTracker.getActiveRenderer(); + if (this != activeRenderer) { return false; } - checkNotNull(codec); + if (!outputTracker.hasOutput()) { + return false; + } + if (outputTracker.shouldTransferOutput(/* renderer= */ this)) { + // Transferring the Surface switches the producer of the shared reader BufferQueue; doing + // so while a buffer is still pending over the surface causes the Codec2 output BufferQueue + // to reject the next buffer on some devices below API 33. + // TODO: b/546521739 - Investigate allowing the early Surface transfer below API 33. + if (SDK_INT < 33) { + return false; + } + outputTracker.transferOutput(/* target= */ this); + } nextFormat = format; long outputStreamOffsetUs = getOutputStreamOffsetUs(); long presentationTimeUs = bufferPresentationTimeUs - outputStreamOffsetUs; @@ -907,10 +966,7 @@ protected boolean processOutputBuffer( skipOutputBuffer(codec, bufferIndex, presentationTimeUs); return true; } - if (isDecodeOnlyBuffer && !isLastBuffer) { - skipOutputBuffer(codec, bufferIndex, bufferPresentationTimeUs); - return true; - } + isReadyToRenderFirstFrame = true; if (!checkNotNull(hardwareBufferFrameReader).canAcceptFrameViaSurface()) { return false; } @@ -928,6 +984,11 @@ protected boolean processOutputBuffer( return true; } + @Override + public boolean isReady() { + return isReadyForDecoding() && isReadyToRenderFirstFrame; + } + @Override protected void renderOutputBufferV21( MediaCodecAdapter codec, int index, long presentationTimeUs, long releaseTimeNs) { @@ -965,12 +1026,6 @@ protected boolean shouldDropBuffersToKeyframe( return super.shouldDropBuffersToKeyframe(earlyUs, elapsedRealtimeUs, isLastBuffer); } - @Override - protected boolean shouldSkipBuffersWithIdenticalReleaseTime() { - // Do not skip buffers with identical vsync times as we aren't rendering to a screen. - return false; - } - @Override protected void renderToEndOfStream() { super.renderToEndOfStream(); @@ -984,7 +1039,7 @@ public boolean isEnded() { // Wait until the listener has also ended before ending the renderer, to avoid frames being // stuck between the renderer and listener if this renderer ends too early. return super.isEnded() - && (!isLastInSequence(getTimeline(), checkNotNull(mediaPeriodId)) + && (!isLastInSequence(getTimeline(), checkNotNull(getMediaPeriodId())) || compositionRendererListener.isEnded()); } @@ -993,10 +1048,15 @@ public void handleMessage(@MessageType int messageType, @Nullable Object message throws ExoPlaybackException { switch (messageType) { case MSG_SET_VIDEO_OUTPUT: - hasOutputSurface = (message != null); - break; + super.handleMessage(messageType, message); + outputTracker.onPlayerSetOutput(this, message); + return; case MSG_TRANSFER_RESOURCES: - hasOutputSurface = false; + // TODO: b/546051614 - Investigate whether we can avoid having to transfer resources. + // The Surface may already have been manually transferred. + if (outputTracker.shouldTransferOutput(this)) { + return; + } break; case MSG_SET_VIDEO_FRAME_METADATA_LISTENER: frameMetadataListener = (VideoFrameMetadataListener) checkNotNull(message); @@ -1018,56 +1078,101 @@ public void onWakeup() { } private int indexOfCurrentItem() { - return getTimeline().getIndexOfPeriod(checkNotNull(getMediaPeriodId()).periodUid); + return getEditedMediaItemIndex(getTimeline(), checkNotNull(getMediaPeriodId())); } } /** * An {@link ImageRenderer} that outputs decoded images to a {@link HardwareBufferFrameReader}. */ - private static final class HardwareBufferImageRenderer extends ImageRenderer { + private static final class HardwareBufferImageRenderer extends ImageRenderer + implements RendererWakeupListener, RendererTracker.TrackedRenderer { private final CompositionRendererListener compositionRendererListener; private final Supplier<@NullableType HardwareBufferFrameReader> hardwareBufferFrameReaderSupplier; + private final RendererTracker rendererTracker; private @MonotonicNonNull HardwareBufferFrameReader hardwareBufferFrameReader; private @MonotonicNonNull ConstantRateTimestampIterator timestampIterator; - private MediaSource.@MonotonicNonNull MediaPeriodId mediaPeriodId; + private @MonotonicNonNull Format outputFormat; + private @MonotonicNonNull MediaPeriodId mediaPeriodId; + private long streamStartPositionUs; private long offsetToCompositionTimeUs; - @Nullable private ImageMetadataListener imageMetadataListener; - private @MonotonicNonNull Format outputFormat; private long streamOffsetUs; + @Nullable private ImageMetadataListener imageMetadataListener; + @Nullable private WakeupListener wakeupListener; + HardwareBufferImageRenderer( ImageDecoder.Factory imageDecoderFactory, CompositionRendererListener compositionRendererListener, - Supplier<@NullableType HardwareBufferFrameReader> hardwareBufferFrameReaderSupplier) { + Supplier<@NullableType HardwareBufferFrameReader> hardwareBufferFrameReaderSupplier, + RendererTracker rendererTracker) { super(imageDecoderFactory, ImageOutput.NO_OP); this.compositionRendererListener = compositionRendererListener; this.hardwareBufferFrameReaderSupplier = hardwareBufferFrameReaderSupplier; + this.rendererTracker = rendererTracker; streamStartPositionUs = C.TIME_UNSET; } + @Override + @Nullable + public MediaPeriodId getTrackedMediaPeriodId() { + return getMediaPeriodId(); + } + + @Override + @Nullable + public Timeline getTrackedTimeline() { + return getTimeline(); + } + @Override public void handleMessage(@Renderer.MessageType int messageType, @Nullable Object message) throws ExoPlaybackException { - if (messageType == Renderer.MSG_SET_IMAGE_METADATA_LISTENER) { - imageMetadataListener = (ImageMetadataListener) message; - return; + switch (messageType) { + case Renderer.MSG_SET_WAKEUP_LISTENER: + this.wakeupListener = (WakeupListener) checkNotNull(message); + break; + case Renderer.MSG_SET_IMAGE_METADATA_LISTENER: + imageMetadataListener = (ImageMetadataListener) message; + break; + default: + super.handleMessage(messageType, message); + break; } - super.handleMessage(messageType, message); } // ImageRenderer methods @Override protected void onEnabled(boolean joining, boolean mayRenderStartOfStream) throws ExoPlaybackException { - super.onEnabled(joining, mayRenderStartOfStream); + // Force enable mayRenderStartOfStream for the image renderer so processOutputBuffer is called + // when the renderer is enabled. If this is not set processOutputBuffer is only called once it + // is started and results in dropped frames at renderer transition. + super.onEnabled(joining, /* mayRenderStartOfStream= */ true); if (hardwareBufferFrameReader == null) { // Initialize hardwareBufferFrameReader on the first onEnabled() call. this.hardwareBufferFrameReader = checkNotNull(hardwareBufferFrameReaderSupplier.get()); } + hardwareBufferFrameReader.addRendererWakeupListener(/* rendererWakeupListener= */ this); + rendererTracker.addRenderer(this); + } + + @Override + protected void onDisabled() { + super.onDisabled(); + checkNotNull(hardwareBufferFrameReader) + .removeRendererWakeupListener(/* rendererWakeupListener= */ this); + rendererTracker.removeRenderer(this); + } + + @Override + public void onWakeup() { + if (wakeupListener != null) { + wakeupListener.onWakeup(); + } } @Override @@ -1114,12 +1219,16 @@ public void render(long positionUs, long elapsedRealtimeUs) throws ExoPlaybackEx @Override protected boolean processOutputBuffer( long positionUs, long elapsedRealtimeUs, Bitmap outputImage, long timeUs) { + Renderer activeRenderer = rendererTracker.getActiveRenderer(); + if (this != activeRenderer) { + return false; + } checkNotNull(timestampIterator); - int indexOfItem = getTimeline().getIndexOfPeriod(checkNotNull(getMediaPeriodId()).periodUid); + int indexOfItem = getEditedMediaItemIndex(getTimeline(), checkNotNull(getMediaPeriodId())); long sequenceOffsetUs = getStreamOffsetUs() + offsetToCompositionTimeUs; checkNotNull(hardwareBufferFrameReader) .outputBitmap(outputImage, timestampIterator, sequenceOffsetUs, indexOfItem); - if (isLastInSequence(getTimeline(), checkNotNull(mediaPeriodId))) { + if (isLastInSequence(getTimeline(), checkNotNull(getMediaPeriodId()))) { hardwareBufferFrameReader.queueEndOfStream(); } if (imageMetadataListener != null) { @@ -1168,22 +1277,16 @@ private ConstantRateTimestampIterator createTimestampIterator(long positionUs) { /** Helper class that encapsulates frame dropping logic for video renderers. */ private static final class TargetFrameRateHelper { - private long nextDecoderInputExpectedTimestampUs; private long nextDecoderOutputExpectedTimestampUs; private long expectedTimestampDeltaUs; - private long decodeOnlyBufferTimestampUs; private TargetFrameRateHelper() { - nextDecoderInputExpectedTimestampUs = C.TIME_UNSET; nextDecoderOutputExpectedTimestampUs = C.TIME_UNSET; expectedTimestampDeltaUs = C.TIME_UNSET; - decodeOnlyBufferTimestampUs = C.TIME_UNSET; } private void onPositionReset() { - nextDecoderInputExpectedTimestampUs = C.TIME_UNSET; nextDecoderOutputExpectedTimestampUs = C.TIME_UNSET; - decodeOnlyBufferTimestampUs = C.TIME_UNSET; } private void onStreamChanged(EditedMediaItem editedMediaItem) { @@ -1193,31 +1296,6 @@ private void onStreamChanged(EditedMediaItem editedMediaItem) { : C.MICROS_PER_SECOND / editedMediaItem.frameRate; } - private void onQueueInputBuffer(DecoderInputBuffer buffer, @Nullable Format codecInputFormat) { - if (SDK_INT >= 34 && shouldMaintainTargetFrameRate()) { - if (shouldDropDecoderInputFrameToMaintainTargetFrameRate( - buffer.timeUs, nextDecoderInputExpectedTimestampUs, codecInputFormat) - && !buffer.isEndOfStream() - && !buffer.isLastSample()) { - // Mark this buffer as DECODE_ONLY. The frame will be dropped by the renderer. We track - // the timestamp to later add the DECODE_ONLY flag in getCodecBufferFlags. - decodeOnlyBufferTimestampUs = buffer.timeUs; - } else { - nextDecoderInputExpectedTimestampUs = - (nextDecoderInputExpectedTimestampUs == C.TIME_UNSET) - ? (buffer.timeUs + expectedTimestampDeltaUs) - : (nextDecoderInputExpectedTimestampUs + expectedTimestampDeltaUs); - } - } - } - - private int getCodecBufferFlags(DecoderInputBuffer buffer) { - if (SDK_INT >= 34 && decodeOnlyBufferTimestampUs == buffer.timeUs) { - return MediaCodec.BUFFER_FLAG_DECODE_ONLY; - } - return 0; - } - private boolean shouldDropOutputFrame(long presentationTimeUs) { return shouldDropFrameToMaintainTargetFrameRate( presentationTimeUs, nextDecoderOutputExpectedTimestampUs); @@ -1236,17 +1314,6 @@ private boolean shouldMaintainTargetFrameRate() { return expectedTimestampDeltaUs != C.TIME_UNSET; } - private boolean shouldDropDecoderInputFrameToMaintainTargetFrameRate( - long presentationTimeUs, - long nextExpectedPresentationTimeUs, - @Nullable Format codecInputFormat) { - checkNotNull(codecInputFormat); - boolean mediaItemContainsBFrames = codecInputFormat.maxNumReorderSamples > 0; - return !mediaItemContainsBFrames - && shouldDropFrameToMaintainTargetFrameRate( - presentationTimeUs, nextExpectedPresentationTimeUs); - } - private boolean shouldDropFrameToMaintainTargetFrameRate( long presentationTimeUs, long nextExpectedPresentationTimeUs) { // This algorithm will always pick the first sample that is after desired timestamp and then @@ -1261,4 +1328,36 @@ private boolean shouldDropFrameToMaintainTargetFrameRate( && presentationTimeUs < nextExpectedPresentationTimeUs; } } + + /** + * Tracker for the renderer that currently holds the video output. + * + *

    This class manages transferring the output between video renderers as the player transitions + * between different items in the sequence. + * + *

    Methods in this class must be called from the ExoPlayer playback thread. + */ + private static final class OutputTracker { + @Nullable private HardwareBufferVideoRenderer rendererWithOutput; + + void onPlayerSetOutput(HardwareBufferVideoRenderer renderer, @Nullable Object output) { + if (output != null) { + this.rendererWithOutput = renderer; + } else if (this.rendererWithOutput == renderer) { + this.rendererWithOutput = null; + } + } + + boolean hasOutput() { + return rendererWithOutput != null; + } + + boolean shouldTransferOutput(HardwareBufferVideoRenderer renderer) { + return rendererWithOutput != renderer; + } + + void transferOutput(HardwareBufferVideoRenderer target) throws ExoPlaybackException { + checkNotNull(rendererWithOutput).handleMessage(Renderer.MSG_TRANSFER_RESOURCES, target); + } + } } diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/SpeedChangingMediaSource.java b/libraries/transformer/src/main/java/androidx/media3/transformer/SpeedChangingMediaSource.java index 194659d7e4b..b4222d9fd53 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/SpeedChangingMediaSource.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/SpeedChangingMediaSource.java @@ -108,6 +108,11 @@ public Period getPeriod(int periodIndex, Period period, boolean setIds) { getAdjustedPeriodTimeUs( wrappedPeriod.durationUs, speedProviderMapper, clipStartUs); } + if (wrappedPeriod.originalDurationUs != C.TIME_UNSET) { + wrappedPeriod.originalDurationUs = + getAdjustedPeriodTimeUs( + wrappedPeriod.originalDurationUs, speedProviderMapper, clipStartUs); + } return wrappedPeriod; } }; diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/Transformer.java b/libraries/transformer/src/main/java/androidx/media3/transformer/Transformer.java index 36868552760..c8c8b195e63 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/Transformer.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/Transformer.java @@ -30,12 +30,11 @@ import androidx.annotation.IntDef; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; -import androidx.annotation.RestrictTo; -import androidx.annotation.RestrictTo.Scope; import androidx.annotation.VisibleForTesting; import androidx.media3.common.C; import androidx.media3.common.DebugViewProvider; import androidx.media3.common.Effect; +import androidx.media3.common.Format; import androidx.media3.common.MediaItem; import androidx.media3.common.MediaLibraryInfo; import androidx.media3.common.MimeTypes; @@ -54,12 +53,11 @@ import androidx.media3.common.video.FrameProcessor; import androidx.media3.effect.DebugTraceUtil; import androidx.media3.effect.DefaultVideoFrameProcessor; -import androidx.media3.effect.HardwareBufferFrame; -import androidx.media3.effect.HardwareBufferFrameQueue; +import androidx.media3.effect.HardwareBufferJni; import androidx.media3.effect.HardwareBufferJniWrapper; -import androidx.media3.effect.RenderingPacketConsumer; import androidx.media3.exoplayer.source.DefaultMediaSourceFactory; import androidx.media3.muxer.Muxer; +import androidx.media3.transformer.Codec.EncoderFactory; import com.google.common.collect.ImmutableList; import com.google.common.collect.Iterables; import com.google.errorprone.annotations.CanIgnoreReturnValue; @@ -69,6 +67,7 @@ import java.lang.annotation.Target; import java.util.ArrayList; import java.util.List; +import java.util.Objects; import org.checkerframework.checker.nullness.qual.EnsuresNonNull; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; @@ -129,10 +128,6 @@ public static final class Builder { private EditingMetricsCollector.MetricsReporter.@MonotonicNonNull Factory metricsReporterFactory; - @Nullable - private RenderingPacketConsumer, HardwareBufferFrameQueue> - packetProcessor; - @Nullable private HardwareBufferJniWrapper hardwareBufferJniWrapper; @Nullable private FrameProcessor.Factory frameProcessorFactory; @@ -190,7 +185,6 @@ private Builder(Transformer transformer) { this.debugViewProvider = transformer.debugViewProvider; this.clock = transformer.clock; this.metricsReporterFactory = transformer.metricsReporterFactory; - this.packetProcessor = transformer.packetProcessor; this.hardwareBufferJniWrapper = transformer.hardwareBufferJniWrapper; this.frameProcessorFactory = transformer.frameProcessorFactory; } @@ -207,6 +201,7 @@ private Builder(Transformer transformer) { *

  • {@link MimeTypes#AUDIO_AAC} *
  • {@link MimeTypes#AUDIO_AMR_NB} *
  • {@link MimeTypes#AUDIO_AMR_WB} + *
  • {@link MimeTypes#AUDIO_RAW} * * * If the MIME type is not supported, {@link Transformer} will fallback to a supported MIME type @@ -346,9 +341,11 @@ public Builder setPortraitEncodingEnabled(boolean enabled) { /** * Sets the {@link HardwareBufferJniWrapper} used to provide native helpers. * + *

    The default value is {@link HardwareBufferJni#INSTANCE}. + * *

    This method is experimental and will be renamed or removed in a future release. * - *

    This will only be used if {@link #setHardwareBufferEffectsPipeline} is set. + *

    This will only be used if {@link #setFrameProcessorFactory} is set. * * @param hardwareBufferJniWrapper The {@link HardwareBufferJniWrapper} to provide native * helpers. @@ -362,33 +359,6 @@ public Builder setNativeHardwareBufferHelpers( return this; } - /** - * Sets the {@link RenderingPacketConsumer} used to process {@link HardwareBufferFrame}s. - * - *

    Setting this parameter forces video transcoding. It has no effect on audio processing. - * - *

    This method is experimental and will be renamed or removed in a future release. - * - *

    If using this method, do not {@linkplain #setVideoFrameProcessorFactory set} a {@link - * VideoFrameProcessor.Factory}. - * - *

    For multi-sequence compositions, the index of a frame in the aggregated {@link - * HardwareBufferFrame} list is not guaranteed to correspond to its originating sequence's - * index. An input sequence that has finished outputting frames will no longer contribute to the - * aggregated packet. - * - * @param packetProcessor The {@link RenderingPacketConsumer} to process frames. - * @return This builder. - */ - @CanIgnoreReturnValue - @RestrictTo(Scope.LIBRARY_GROUP) // TODO: b/498547782 - Remove once usages have been migrated. - public Builder setHardwareBufferEffectsPipeline( - RenderingPacketConsumer, HardwareBufferFrameQueue> - packetProcessor) { - this.packetProcessor = packetProcessor; - return this; - } - /** * Sets the {@link FrameProcessor.Factory} to be used to create {@link FrameProcessor} * instances. @@ -710,9 +680,11 @@ public Transformer build() { !mp4EditListTrimEnabled || muxerFactory.supportsWritingNegativeTimestampsInEditList(), "Muxer.Factory %s does not support writing negative timestamps to an edit list.", muxerFactory); - if (packetProcessor != null) { - checkState(hardwareBufferJniWrapper != null || SDK_INT >= 33); - } + @Nullable + HardwareBufferJniWrapper hardwareBufferJniWrapper = + SDK_INT >= 26 && frameProcessorFactory != null && this.hardwareBufferJniWrapper == null + ? HardwareBufferJni.INSTANCE + : this.hardwareBufferJniWrapper; return new Transformer( context, transformationRequest, @@ -737,7 +709,6 @@ public Transformer build() { debugViewProvider, clock, metricsReporterFactory, - packetProcessor, hardwareBufferJniWrapper, frameProcessorFactory); } @@ -863,11 +834,6 @@ default void onFallbackApplied( private final ExportOperationListener exportOperationListener; @Nullable private final EditingMetricsCollector.MetricsReporter.Factory metricsReporterFactory; - @Nullable - private final RenderingPacketConsumer< - ImmutableList, HardwareBufferFrameQueue> - packetProcessor; - @Nullable private final HardwareBufferJniWrapper hardwareBufferJniWrapper; @Nullable private final FrameProcessor.Factory frameProcessorFactory; @@ -875,7 +841,7 @@ default void onFallbackApplied( private boolean exportResumed; private @MonotonicNonNull Composition composition; private @MonotonicNonNull Composition originalComposition; - private @MonotonicNonNull String outputFilePath; + private @MonotonicNonNull TransformerOutput transformerOutput; private @MonotonicNonNull String oldFilePath; private @MonotonicNonNull EditingMetricsCollector editingMetricsCollector; @Nullable private WatchdogTimer exportWatchdogTimer; @@ -904,9 +870,6 @@ private Transformer( DebugViewProvider debugViewProvider, Clock clock, @Nullable EditingMetricsCollector.MetricsReporter.Factory metricsReporterFactory, - @Nullable - RenderingPacketConsumer, HardwareBufferFrameQueue> - packetProcessor, @Nullable HardwareBufferJniWrapper hardwareBufferJniWrapper, @Nullable FrameProcessor.Factory frameProcessorFactory) { checkState(!removeAudio || !removeVideo, "Audio and video cannot both be removed."); @@ -927,12 +890,11 @@ private Transformer( this.assetLoaderFactory = assetLoaderFactory; this.audioMixerFactory = audioMixerFactory; this.videoFrameProcessorFactory = videoFrameProcessorFactory; - this.encoderFactory = encoderFactory; + this.encoderFactory = new TransformerEncoderFactory(encoderFactory); this.muxerFactory = muxerFactory; this.looper = looper; this.debugViewProvider = debugViewProvider; this.clock = clock; - this.packetProcessor = packetProcessor; this.hardwareBufferJniWrapper = hardwareBufferJniWrapper; this.frameProcessorFactory = frameProcessorFactory; this.metricsReporterFactory = metricsReporterFactory; @@ -1040,9 +1002,39 @@ public void removeAllListeners() { * @throws IllegalStateException If an export is already in progress. */ public void start(Composition composition, String path) { + start(composition, TransformerOutput.create(path)); + } + + /** + * Starts an asynchronous operation to export the given {@link Composition} to a {@link + * TransformerOutput}. + * + *

    The export state is notified through the {@linkplain Builder#addListener(Listener) + * listener}. + * + *

    Concurrent exports on the same Transformer object are not allowed. + * + *

    If no custom {@link Transformer.Builder#setMuxerFactory(Muxer.Factory) Muxer.Factory} is + * specified, the output is an MP4 file. + * + *

    The output can contain at most one video track and one audio track. Other track types are + * ignored. For adaptive bitrate inputs, if no custom {@link + * Transformer.Builder#setAssetLoaderFactory(AssetLoader.Factory) AssetLoader.Factory} is + * specified, the highest bitrate video and audio streams are selected. + * + *

    If exporting the video track entails transcoding, the output frames' dimensions will be + * swapped if the output video's height is larger than the width. This is to improve compatibility + * among different device encoders. + * + * @param composition The {@link Composition} to export. + * @param output The {@link TransformerOutput} destination. + * @throws IllegalStateException If this method is called from the wrong thread. + * @throws IllegalStateException If an export is already in progress. + */ + public void start(Composition composition, TransformerOutput output) { verifyApplicationThread(); checkState(currentExportOperation == null, "There is already an export in progress."); - initialize(composition, path); + initialize(composition, output); startExportOperation(); } @@ -1171,7 +1163,8 @@ public void cancel() { if (editingMetricsCollector == null) { LogSessionId unused = setUpMetricsCollection(); } - checkNotNull(editingMetricsCollector).onExportCancelled(progressPercentage); + checkNotNull(editingMetricsCollector) + .onExportCancelled(checkNotNull(composition), progressPercentage); } currentExportOperation = null; @@ -1208,7 +1201,7 @@ public void resume(Composition composition, String outputFilePath, String oldFil verifyApplicationThread(); checkState(currentExportOperation == null, "There is already an export in progress."); checkArgument(!compositionContainsSpeedChangingEffects(composition)); - initialize(composition, outputFilePath); + initialize(composition, TransformerOutput.create(outputFilePath)); this.oldFilePath = oldFilePath; exportResumed = true; startExportOperation(); @@ -1324,12 +1317,12 @@ private void maybeStopExportWatchdogTimer() { } } - @EnsuresNonNull({"this.composition", "this.outputFilePath", "this.originalComposition"}) - private void initialize(Composition composition, String outputFilePath) { + @EnsuresNonNull({"this.composition", "this.originalComposition", "this.transformerOutput"}) + private void initialize(Composition composition, TransformerOutput transformerOutput) { maybeInitializeExportWatchdogTimer(); this.originalComposition = composition; this.composition = applyPreProcessingEffects(composition); - this.outputFilePath = outputFilePath; + this.transformerOutput = transformerOutput; } private boolean isMultiAsset() { @@ -1362,6 +1355,7 @@ private void startExportOperation() { transformationRequest); AssetLoader.Factory assetLoaderFactory = this.assetLoaderFactory; DebugTraceUtil.reset(); + TransformerOutput transformerOutput = checkNotNull(this.transformerOutput); if (exportResumed) { currentExportOperation = new ResumedExportOperation( @@ -1379,12 +1373,12 @@ private void startExportOperation() { applicationHandler, debugViewProvider, clock, - getFrameProcessorFactory(), + frameProcessorFactory, hardwareBufferJniWrapper, logSessionId, shouldApplyMp4EditListTrim(), muxerFactory, - checkNotNull(outputFilePath), + checkNotNull(transformerOutput.path), checkNotNull(oldFilePath)); } else if (trimOptimizationEnabled && isSingleAssetTrimming()) { currentExportOperation = @@ -1403,12 +1397,12 @@ private void startExportOperation() { applicationHandler, debugViewProvider, clock, - getFrameProcessorFactory(), + frameProcessorFactory, hardwareBufferJniWrapper, logSessionId, shouldApplyMp4EditListTrim(), muxerFactory, - checkNotNull(outputFilePath)); + checkNotNull(transformerOutput.path)); } else { currentExportOperation = new DefaultExportOperation( @@ -1426,12 +1420,12 @@ private void startExportOperation() { applicationHandler, debugViewProvider, clock, - getFrameProcessorFactory(), + frameProcessorFactory, hardwareBufferJniWrapper, logSessionId, shouldApplyMp4EditListTrim(), muxerFactory, - checkNotNull(outputFilePath), + transformerOutput, fileStartsOnVideoFrameEnabled); } currentExportOperation.start(); @@ -1492,6 +1486,33 @@ private EditingMetricsCollector prepareEditingMetricsCollector( compositionHasVideoEffects); } + /** + * An {@link EncoderFactory} implementation that wraps another {@link EncoderFactory} instance and + * adds workarounds for existing issues. + * + *

    {@link DefaultEncoderFactory} returns {@link DefaultCodec} instances, which does not allow + * other implementations of {@link Codec}. + */ + // TODO(b/550318509): Consolidate this logic in a real EncoderFactory once the "default" encoder + // factory can return non-MediaCodec encoders. + /* package */ static final class TransformerEncoderFactory extends ForwardingEncoderFactory { + + public TransformerEncoderFactory(EncoderFactory factory) { + super(factory); + } + + @Override + public Codec createForAudioEncoding(Format format, @Nullable LogSessionId logSessionId) + throws ExportException { + if (Objects.equals(format.sampleMimeType, MimeTypes.AUDIO_RAW) + && !super.audioNeedsEncoding()) { + return new PassthroughAudioCodec(format); + } + + return super.createForAudioEncoding(format, logSessionId); + } + } + private final class ExportOperationListener implements ExportOperation.Listener { @Override public void onCompleted(ExportResult exportResult) { @@ -1499,7 +1520,8 @@ public void onCompleted(ExportResult exportResult) { listeners.sendEvent( listener -> listener.onCompleted(checkNotNull(composition), exportResult)); if (canCollectEditingMetrics()) { - checkNotNull(editingMetricsCollector).onExportSuccess(exportResult, exportResumed); + checkNotNull(editingMetricsCollector) + .onExportSuccess(checkNotNull(composition), exportResult, exportResumed); } currentExportOperation = null; exportResumed = false; @@ -1521,7 +1543,12 @@ public void onError(ExportResult exportResult, ExportException exportException) LogSessionId unused = setUpMetricsCollection(); } checkNotNull(editingMetricsCollector) - .onExportError(progressPercentage, exportException, exportResult, exportResumed); + .onExportError( + checkNotNull(composition), + progressPercentage, + exportException, + exportResult, + exportResumed); } currentExportOperation = null; exportResumed = false; @@ -1536,26 +1563,4 @@ public void onSampleWrittenOrDropped() { } } } - - @Nullable - private FrameProcessor.Factory getFrameProcessorFactory() { - FrameProcessor.Factory factory = frameProcessorFactory; - RenderingPacketConsumer, HardwareBufferFrameQueue> - localPacketProcessor = packetProcessor; - if (factory == null && localPacketProcessor != null) { - if (SDK_INT >= 26) { - factory = - (output, listenerExecutor, listener) -> { - HardwareBufferFrameQueue adaptedQueue = - new FrameWriterToHardwareBufferFrameQueueAdapter(output); - localPacketProcessor.setRenderOutput(adaptedQueue); - return new PacketConsumerToFrameProcessorAdapter( - localPacketProcessor, listenerExecutor, listener); - }; - } else { - throw new IllegalStateException("API 26+ required to use PacketProcessor in Transformer"); - } - } - return factory; - } } diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/TransformerInternal.java b/libraries/transformer/src/main/java/androidx/media3/transformer/TransformerInternal.java index 83f3bca5f45..fd8654aabcd 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/TransformerInternal.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/TransformerInternal.java @@ -58,6 +58,7 @@ import androidx.media3.common.C; import androidx.media3.common.ColorInfo; import androidx.media3.common.DebugViewProvider; +import androidx.media3.common.Flags; import androidx.media3.common.Format; import androidx.media3.common.MediaItem; import androidx.media3.common.MediaLibraryInfo; @@ -245,7 +246,8 @@ public TransformerInternal( + MediaLibraryInfo.VERSION_SLASHY + "] [" + Util.DEVICE_DEBUG_INFO - + "]"); + + "] " + + Flags.getLogString()); internalHandlerThread = new HandlerThread("Transformer:Internal"); internalHandlerThread.start(); sequenceAssetLoaders = new ArrayList<>(); @@ -453,27 +455,24 @@ private void endInternal(@EndReason int endReason, @Nullable ExportException exp + "] [" + MediaLibraryInfo.registeredModules() + "]"); - // VideoSampleExporter can hold buffers from the asset loader's decoder in a surface texture, - // so we release the VideoSampleExporter first to avoid releasing the codec while its buffers - // are pending processing. - for (int i = 0; i < sampleExporters.size(); i++) { + for (int i = 0; i < sequenceAssetLoaders.size(); i++) { try { - sampleExporters.get(i).release(); + sequenceAssetLoaders.get(i).release(); } catch (RuntimeException e) { if (releaseExportException == null) { releaseExportException = ExportException.createForUnexpected(e); - // cancelException is not reported through a listener. It is thrown in cancel(), as this - // method is blocking. cancelException = e; } } } - for (int i = 0; i < sequenceAssetLoaders.size(); i++) { + for (int i = 0; i < sampleExporters.size(); i++) { try { - sequenceAssetLoaders.get(i).release(); + sampleExporters.get(i).release(); } catch (RuntimeException e) { if (releaseExportException == null) { releaseExportException = ExportException.createForUnexpected(e); + // cancelException is not reported through a listener. It is thrown in cancel(), as this + // method is blocking. cancelException = e; } } @@ -483,6 +482,7 @@ private void endInternal(@EndReason int endReason, @Nullable ExportException exp } catch (MuxerException e) { if (releaseExportException == null) { releaseExportException = ExportException.createForMuxer(e, ERROR_CODE_MUXING_FAILED); + cancelException = new RuntimeException(e); } } catch (RuntimeException e) { if (releaseExportException == null) { @@ -796,7 +796,7 @@ private void createDecodedSampleExporter(Format assetLoaderOutputFormat) firstFormat, transformationRequest, checkNotNull(frameProcessorFactory), - hardwareBufferJniWrapper, + checkNotNull(hardwareBufferJniWrapper), encoderFactory, muxerWrapper, /* errorConsumer= */ this::onError, diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/TransformerOutput.java b/libraries/transformer/src/main/java/androidx/media3/transformer/TransformerOutput.java new file mode 100644 index 00000000000..da8ad71c4de --- /dev/null +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/TransformerOutput.java @@ -0,0 +1,81 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static java.lang.annotation.ElementType.TYPE_USE; + +import android.content.ContentValues; +import android.net.Uri; +import androidx.annotation.IntDef; +import androidx.annotation.Nullable; +import androidx.media3.common.util.UnstableApi; +import java.lang.annotation.Documented; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** Represents the output destination for a Transformer export. */ +@UnstableApi +public final class TransformerOutput { + + /** Indicates the output target type. */ + @Documented + @Retention(RetentionPolicy.SOURCE) + @Target(TYPE_USE) + @IntDef({TYPE_FILE_PATH, TYPE_MEDIA_STORE}) + /* package */ @interface OutputType {} + + /* package */ static final int TYPE_FILE_PATH = 1; + /* package */ static final int TYPE_MEDIA_STORE = 2; + + /* package */ final @OutputType int outputType; + @Nullable /* package */ final String path; + @Nullable /* package */ final Uri collectionUri; + @Nullable /* package */ final ContentValues contentValues; + + /** + * Creates a {@link TransformerOutput} targeting a local file path. + * + * @param path The local file path. + * @return A {@link TransformerOutput} instance. + */ + public static TransformerOutput create(String path) { + return new TransformerOutput( + TYPE_FILE_PATH, path, /* collectionUri= */ null, /* contentValues= */ null); + } + + /** + * Creates a {@link TransformerOutput} targeting the Android MediaStore. + * + * @param collectionUri The collection {@link Uri} to insert the media into. + * @param contentValues The metadata values for the new MediaStore entry. + * @return A {@link TransformerOutput} instance. + */ + public static TransformerOutput create(Uri collectionUri, ContentValues contentValues) { + return new TransformerOutput(TYPE_MEDIA_STORE, /* path= */ null, collectionUri, contentValues); + } + + private TransformerOutput( + @OutputType int outputType, + @Nullable String path, + @Nullable Uri collectionUri, + @Nullable ContentValues contentValues) { + this.outputType = outputType; + this.path = path; + this.collectionUri = collectionUri; + this.contentValues = contentValues != null ? new ContentValues(contentValues) : null; + } +} diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/TransformerOutputResolver.java b/libraries/transformer/src/main/java/androidx/media3/transformer/TransformerOutputResolver.java new file mode 100644 index 00000000000..38b4274dbe0 --- /dev/null +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/TransformerOutputResolver.java @@ -0,0 +1,144 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import android.content.ContentResolver; +import android.content.ContentValues; +import android.content.Context; +import android.database.Cursor; +import android.media.MediaScannerConnection; +import android.net.Uri; +import android.os.Build; +import android.os.ParcelFileDescriptor; +import android.provider.MediaStore; +import androidx.annotation.Nullable; +import androidx.media3.common.util.UnstableApi; +import java.io.FileNotFoundException; +import java.io.IOException; + +/** Internal helper for resolving {@link TransformerOutput} options to low-level handles. */ +@UnstableApi +/* package */ final class TransformerOutputResolver { + + private TransformerOutputResolver() {} + + /** + * Opens a {@link ParcelFileDescriptor} with {@code "rwt"} mode for the given {@link Uri}. + * + * @param context The {@link Context}. + * @param uri The {@link Uri} to open. + * @return The opened {@link ParcelFileDescriptor}. + * @throws FileNotFoundException If the file descriptor could not be opened. + */ + @SuppressWarnings("ContentResolverUri") // Open source Media3 component consuming Uri + public static ParcelFileDescriptor openFileDescriptor(Context context, Uri uri) + throws FileNotFoundException { + ContentResolver resolver = context.getContentResolver(); + ParcelFileDescriptor pfd = resolver.openFileDescriptor(uri, "rwt"); + if (pfd == null) { + throw new FileNotFoundException("Could not open file descriptor for URI: " + uri); + } + return pfd; + } + + /** + * Inserts a MediaStore entry and returns the inserted {@link Uri}. + * + * @param context The {@link Context}. + * @param collectionUri The collection {@link Uri} to insert into. + * @param contentValues The metadata values for the new entry. + * @return The {@link Uri} of the inserted item. + * @throws IOException If the insertion into MediaStore failed. + */ + @SuppressWarnings("ContentResolverUri") // Open source Media3 component consuming MediaStore Uri + public static Uri insertMediaStoreEntry( + Context context, Uri collectionUri, ContentValues contentValues) throws IOException { + ContentValues values = new ContentValues(contentValues); + if (Build.VERSION.SDK_INT >= 29) { + values.put(MediaStore.MediaColumns.IS_PENDING, 1); + } + ContentResolver resolver = context.getContentResolver(); + Uri itemUri = resolver.insert(collectionUri, values); + if (itemUri == null) { + throw new IOException("Failed to insert MediaStore entry into " + collectionUri); + } + return itemUri; + } + + /** + * Clears the {@code IS_PENDING} flag on API 29+ or requests a media scan on API < 29 for the + * specified MediaStore entry after successful writing. + * + * @param context The {@link Context}. + * @param itemUri The {@link Uri} of the MediaStore item. + */ + @SuppressWarnings("ContentResolverUri") // Open source Media3 component updating MediaStore Uri + public static void clearMediaStorePendingFlag(Context context, Uri itemUri) { + if (Build.VERSION.SDK_INT >= 29) { + ContentValues values = new ContentValues(); + values.put(MediaStore.MediaColumns.IS_PENDING, 0); + context + .getContentResolver() + .update(itemUri, values, /* where= */ null, /* selectionArgs= */ null); + } else { + String filePath = getFilePathFromUri(context, itemUri); + if (filePath != null) { + MediaScannerConnection.scanFile( + context, new String[] {filePath}, /* mimeTypes= */ null, /* callback= */ null); + } + } + } + + /** + * Deletes the specified MediaStore entry, removing both the database row and the underlying file. + * + *

    This is the counterpart to {@link #insertMediaStoreEntry} and is used to clean up an entry + * whose export did not complete successfully. Deletion is best effort: failures are ignored so + * that they do not mask the error that triggered the cleanup. + * + * @param context The {@link Context}. + * @param itemUri The {@link Uri} of the MediaStore item. + */ + @SuppressWarnings("ContentResolverUri") // Open source Media3 component deleting MediaStore Uri + public static void deleteMediaStoreEntry(Context context, Uri itemUri) { + try { + context.getContentResolver().delete(itemUri, /* where= */ null, /* selectionArgs= */ null); + } catch (RuntimeException e) { + // Deletion failure; on API 30+ the pending entry is later reclaimed by the system. + } + } + + @Nullable + private static String getFilePathFromUri(Context context, Uri uri) { + try (Cursor cursor = + context + .getContentResolver() + .query( + uri, + new String[] {MediaStore.MediaColumns.DATA}, + /* selection= */ null, + /* selectionArgs= */ null, + /* sortOrder= */ null)) { + if (cursor != null && cursor.moveToFirst()) { + int columnIndex = cursor.getColumnIndexOrThrow(MediaStore.MediaColumns.DATA); + return cursor.getString(columnIndex); + } + } catch (RuntimeException e) { + // Query failure on legacy device; let system background media scanner discover the file. + } + return null; + } +} diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/TransformerUtil.java b/libraries/transformer/src/main/java/androidx/media3/transformer/TransformerUtil.java index f95437bc3b8..972857540a7 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/TransformerUtil.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/TransformerUtil.java @@ -45,7 +45,12 @@ import androidx.media3.common.audio.AudioProcessor; import androidx.media3.common.audio.SpeedChangingAudioProcessor; import androidx.media3.common.audio.SpeedProvider; +import androidx.media3.common.util.CodecSpecificDataUtil; import androidx.media3.common.util.UnstableApi; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.Frame; +import androidx.media3.common.video.ReferenceCounter; +import androidx.media3.common.video.SyncFenceWrapper; import androidx.media3.effect.GlEffect; import androidx.media3.effect.ScaleAndRotateTransformation; import androidx.media3.effect.SpeedChangeEffect; @@ -56,12 +61,37 @@ import androidx.media3.transformer.Composition.HdrMode; import com.google.common.base.Ascii; import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import java.util.List; import java.util.Objects; /** Utility methods for Transformer. */ @UnstableApi public final class TransformerUtil { + /** An empty {@link AsyncFrame} representing the end of a stream. */ + /* package */ static final AsyncFrame END_OF_STREAM_ASYNC_FRAME = + new AsyncFrame( + new Frame() { + private final Format format = new Format.Builder().build(); + + @Override + public Format getFormat() { + return format; + } + + @Override + public ImmutableMap getMetadata() { + return ImmutableMap.of(); + } + + @Override + public long getContentTimeUs() { + return C.TIME_END_OF_SOURCE; + } + }, + /* acquireFence= */ null); + private TransformerUtil() {} /** @@ -168,6 +198,10 @@ private static boolean containsSlowMotionData(Format format) { return true; } } + if (Objects.equals(inputFormat.sampleMimeType, MimeTypes.VIDEO_DOLBY_VISION) + && CodecSpecificDataUtil.getCodecProfileAndLevel(inputFormat) == null) { + return true; + } if (requestedMimeType == null && !muxerWrapper.supportsSampleMimeType(inputFormat.sampleMimeType) && !muxerWrapper.supportsSampleMimeType(getAlternativeCodecMimeType(inputFormat))) { @@ -355,6 +389,7 @@ public static String getImageMimeType(Context context, MediaItem mediaItem) { * *

    If no speed changing effects are present, this method returns {@code true}. */ + @SuppressWarnings("deprecation") // Uses deprecated TimestampAdjustment. public static boolean validateSpeedChangingEffects(Effects effects, SpeedProvider speedProvider) { if (!effects.audioProcessors.isEmpty()) { AudioProcessor firstProcessor = effects.audioProcessors.get(0); @@ -385,6 +420,7 @@ public static boolean validateSpeedChangingEffects(Effects effects, SpeedProvide * SpeedChangingAudioProcessor} are ignored as first elements of the video and audio pipelines, * respectively. */ + @SuppressWarnings("deprecation") // Uses deprecated SpeedChangeEffect and TimestampAdjustment. public static boolean containsSpeedChangingEffects(Effects effects, boolean ignoreFirstEffect) { for (int i = ignoreFirstEffect ? 1 : 0; i < effects.audioProcessors.size(); i++) { if (effects.audioProcessors.get(i) instanceof SpeedChangingAudioProcessor) { @@ -458,9 +494,51 @@ public static EditedMediaItemSequence getEditedMediaItemSequence( * @param mediaPeriodId The {@link MediaPeriodId} associated to the {@link EditedMediaItem}. */ public static EditedMediaItem getEditedMediaItem(Timeline timeline, MediaPeriodId mediaPeriodId) { + EditedMediaItemSequence sequence = getEditedMediaItemSequence(timeline, mediaPeriodId); + return sequence.editedMediaItems.get(getEditedMediaItemIndex(timeline, mediaPeriodId)); + } + + /** + * Returns the index of the {@link EditedMediaItem} corresponding to {@code mediaPeriodId} within + * {@code timeline}. + * + * @param timeline The {@link Timeline} associated to the {@link EditedMediaItem}. + * @param mediaPeriodId The {@link MediaPeriodId} associated to the {@link EditedMediaItem}. + */ + public static int getEditedMediaItemIndex(Timeline timeline, MediaPeriodId mediaPeriodId) { int index = timeline.getIndexOfPeriod(mediaPeriodId.periodUid); EditedMediaItemSequence sequence = getEditedMediaItemSequence(timeline, mediaPeriodId); - return EditedMediaItemSequence.getEditedMediaItem(sequence, index); + return EditedMediaItemSequence.getEditedMediaItemIndex(sequence, index); + } + + /** + * Releases {@code object} if it implements {@link ReferenceCounter}. + * + *

    If {@code object} is not a {@link ReferenceCounter} and {@code releaseFence} is non-null, + * the fence is closed silently to prevent leaks. + * + * @param object The object to release, or {@code null}. + * @param releaseFence An optional {@link SyncFenceWrapper} signaling when hardware resources are + * safe for reuse. + */ + /* package */ static void releaseIfNeeded( + @Nullable Object object, @Nullable SyncFenceWrapper releaseFence) { + if (object instanceof ReferenceCounter) { + ((ReferenceCounter) object).release(releaseFence); + } else if (releaseFence != null) { + releaseFence.close(); + } + } + + /** + * Releases each object in {@code objects} if it implements {@link ReferenceCounter}. + * + * @param objects The collection of objects to release. + */ + /* package */ static void releaseIfNeeded(List objects) { + for (int i = 0; i < objects.size(); i++) { + releaseIfNeeded(objects.get(i), /* releaseFence= */ null); + } } @Nullable diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/TrimOptimizedExportOperation.java b/libraries/transformer/src/main/java/androidx/media3/transformer/TrimOptimizedExportOperation.java index c34e1c96f9a..704940066dd 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/TrimOptimizedExportOperation.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/TrimOptimizedExportOperation.java @@ -92,7 +92,7 @@ @Nullable private final FrameProcessor.Factory frameProcessorFactory; - @Nullable HardwareBufferJniWrapper hardwareBufferJniWrapper; + @Nullable private final HardwareBufferJniWrapper hardwareBufferJniWrapper; @Nullable private final LogSessionId logSessionId; private final Muxer.Factory muxerFactory; @@ -161,6 +161,10 @@ public void start() { @Override public @ProgressState int getProgress(ProgressHolder progressHolder) { + if (state == STATE_PROCESS_FULL_INPUT) { + return getNextAccumulatedProgress( + /* progressSoFar= */ 0, /* nextProgressWeight= */ 1, progressHolder); + } if (mediaItemInfo == null) { return PROGRESS_STATE_WAITING_FOR_AVAILABILITY; } diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/VideoEncoderSettings.java b/libraries/transformer/src/main/java/androidx/media3/transformer/VideoEncoderSettings.java index f9c672c1c57..eb3f0ec531e 100644 --- a/libraries/transformer/src/main/java/androidx/media3/transformer/VideoEncoderSettings.java +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/VideoEncoderSettings.java @@ -84,6 +84,7 @@ public static final class Builder { private int maxBFrames; private int numNonBidirectionalTemporalLayers; private int numBidirectionalTemporalLayers; + private int complexity; /** Creates a new instance. */ public Builder() { @@ -98,6 +99,7 @@ public Builder() { this.maxBFrames = NO_VALUE; this.numNonBidirectionalTemporalLayers = NO_VALUE; this.numBidirectionalTemporalLayers = NO_VALUE; + this.complexity = NO_VALUE; } private Builder(VideoEncoderSettings videoEncoderSettings) { @@ -113,6 +115,7 @@ private Builder(VideoEncoderSettings videoEncoderSettings) { this.numNonBidirectionalTemporalLayers = videoEncoderSettings.numNonBidirectionalTemporalLayers; this.numBidirectionalTemporalLayers = videoEncoderSettings.numBidirectionalTemporalLayers; + this.complexity = videoEncoderSettings.complexity; } /** @@ -233,20 +236,46 @@ public Builder setMaxBFrames(int maxBFrames) { * encoder. * * @param numNonBidirectionalLayers the number of predictive layers to have. This value must be - * stricly positive. A value of '0' explicitly requests no temporal layers from the encoder, - * regardless of the requested 'numBidirectionalLayers'. + * greater than or equal to zero, or {@link #NO_VALUE} if {@code numBidirectionalLayers} is + * also {@link #NO_VALUE}. A value of '0' explicitly requests no temporal layers from the + * encoder, in which case {@code numBidirectionalLayers} must be '0' or {@link #NO_VALUE}. * @param numBidirectionalLayers the number of bi-directional layers to have. This value must be - * greater than or equal to zero. A value greater than 1 constructs a hierarchical-B coding + * greater than or equal to zero, or {@link #NO_VALUE} if {@code numNonBidirectionalLayers} + * is '0' or {@link #NO_VALUE}. A value greater than 1 constructs a hierarchical-B coding * structure. * @return This builder. */ @CanIgnoreReturnValue public Builder setTemporalLayers(int numNonBidirectionalLayers, int numBidirectionalLayers) { + checkArgument( + (numNonBidirectionalLayers == NO_VALUE && numBidirectionalLayers == NO_VALUE) + || (numNonBidirectionalLayers == 0 + && (numBidirectionalLayers == 0 || numBidirectionalLayers == NO_VALUE)) + || (numNonBidirectionalLayers > 0 && numBidirectionalLayers >= 0), + "Invalid temporal layers: numNonBidirectionalLayers=%s, numBidirectionalLayers=%s", + numNonBidirectionalLayers, + numBidirectionalLayers); this.numNonBidirectionalTemporalLayers = numNonBidirectionalLayers; this.numBidirectionalTemporalLayers = numBidirectionalLayers; return this; } + /** + * Sets the encoding complexity. The default value is {@link #NO_VALUE} which indicates that no + * {@linkplain MediaFormat#KEY_COMPLEXITY complexity} will be set for the encoder. + * + *

    The valid range of values depends on the encoder, and is exposed by {@link + * MediaCodecInfo.EncoderCapabilities#getComplexityRange()}. + * + * @param complexity The {@linkplain MediaFormat#KEY_COMPLEXITY encoding complexity}. + * @return This builder. + */ + @CanIgnoreReturnValue + public Builder setComplexity(int complexity) { + this.complexity = complexity; + return this; + } + /** Builds the instance. */ public VideoEncoderSettings build() { return new VideoEncoderSettings( @@ -260,7 +289,8 @@ public VideoEncoderSettings build() { repeatPreviousFrameIntervalUs, maxBFrames, numNonBidirectionalTemporalLayers, - numBidirectionalTemporalLayers); + numBidirectionalTemporalLayers, + complexity); } } @@ -303,6 +333,9 @@ public VideoEncoderSettings build() { /** The requested number of bidirectional temporal layers requested from the encoder. */ public final int numBidirectionalTemporalLayers; + /** The {@linkplain MediaFormat#KEY_COMPLEXITY encoding complexity}. */ + public final int complexity; + private VideoEncoderSettings( int bitrate, int bitrateMode, @@ -314,7 +347,8 @@ private VideoEncoderSettings( long repeatPreviousFrameIntervalUs, int maxBFrames, int numNonBidirectionalTemporalLayers, - int numBidirectionalTemporalLayers) { + int numBidirectionalTemporalLayers, + int complexity) { this.bitrate = bitrate; this.bitrateMode = bitrateMode; this.profile = profile; @@ -326,6 +360,7 @@ private VideoEncoderSettings( this.maxBFrames = maxBFrames; this.numNonBidirectionalTemporalLayers = numNonBidirectionalTemporalLayers; this.numBidirectionalTemporalLayers = numBidirectionalTemporalLayers; + this.complexity = complexity; } /** @@ -354,7 +389,8 @@ public boolean equals(@Nullable Object o) { && repeatPreviousFrameIntervalUs == that.repeatPreviousFrameIntervalUs && maxBFrames == that.maxBFrames && numNonBidirectionalTemporalLayers == that.numNonBidirectionalTemporalLayers - && numBidirectionalTemporalLayers == that.numBidirectionalTemporalLayers; + && numBidirectionalTemporalLayers == that.numBidirectionalTemporalLayers + && complexity == that.complexity; } @Override @@ -373,6 +409,7 @@ public int hashCode() { result = 31 * result + maxBFrames; result = 31 * result + numNonBidirectionalTemporalLayers; result = 31 * result + numBidirectionalTemporalLayers; + result = 31 * result + complexity; return result; } @@ -401,6 +438,8 @@ public String toString() { + numNonBidirectionalTemporalLayers + ", numBidirectionalTemporalLayers=" + numBidirectionalTemporalLayers + + ", complexity=" + + complexity + '}'; } } diff --git a/libraries/transformer/src/main/java/androidx/media3/transformer/VideoFrameAggregationParameters.java b/libraries/transformer/src/main/java/androidx/media3/transformer/VideoFrameAggregationParameters.java new file mode 100644 index 00000000000..f0817fe5487 --- /dev/null +++ b/libraries/transformer/src/main/java/androidx/media3/transformer/VideoFrameAggregationParameters.java @@ -0,0 +1,133 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static com.google.common.base.Preconditions.checkArgument; + +import android.util.Rational; +import androidx.annotation.Nullable; +import androidx.media3.common.util.ExperimentalApi; +import androidx.media3.common.util.Log; +import com.google.errorprone.annotations.CanIgnoreReturnValue; +import java.util.Objects; +import org.json.JSONException; +import org.json.JSONObject; + +/** + * Parameters for video frame aggregation. + * + *

    These parameters control how video frames from multiple sequences are aligned, dropped, or + * duplicated before entering the processing pipeline. + */ +@ExperimentalApi // TODO: b/526781983 - Remove @ExperimentalApi. +public final class VideoFrameAggregationParameters { + + /** A builder for {@link VideoFrameAggregationParameters} instances. */ + public static final class Builder { + + @Nullable private Rational frameRate; + + /** Creates a new instance. */ + public Builder() {} + + private Builder(VideoFrameAggregationParameters parameters) { + this.frameRate = parameters.frameRate; + } + + /** + * Sets the target frame rate for aggregating video frames. + * + *

    If set, the aggregation process will drop or reuse frames (with appropriate retiming) to + * match this rate. This allows the pipeline to guarantee a single, constant frame rate before + * frames enter the processing layer. For example, if the target frame rate is 30 FPS, a 24 FPS + * video will have some frames reused, and a 60 FPS video will have some frames dropped to meet + * the target. + * + *

    If {@code null}, the aggregation follows the primary sequence's physical timestamps. The + * default is {@code null}. + * + * @param frameRate The target frame rate, in frames per second. Must be strictly positive or + * {@code null}. + * @return This builder. + * @throws IllegalArgumentException If {@code frameRate} is not strictly positive and not {@code + * null}. + */ + @CanIgnoreReturnValue + public Builder setFrameRate(@Nullable Rational frameRate) { + checkArgument( + frameRate == null || (frameRate.getNumerator() > 0 && frameRate.getDenominator() > 0), + "frameRate must be strictly positive or null"); + this.frameRate = frameRate; + return this; + } + + /** Builds a {@link VideoFrameAggregationParameters} instance. */ + public VideoFrameAggregationParameters build() { + return new VideoFrameAggregationParameters(this); + } + } + + /** An instance with default parameters. */ + public static final VideoFrameAggregationParameters DEFAULT = new Builder().build(); + + /** The target frame rate, in frames per second, or {@code null} if no target rate is set. */ + @Nullable public final Rational frameRate; + + private VideoFrameAggregationParameters(Builder builder) { + this.frameRate = builder.frameRate; + } + + /** Returns a new {@link Builder} initialized with the values of this instance. */ + public Builder buildUpon() { + return new Builder(this); + } + + @Override + public boolean equals(@Nullable Object o) { + if (this == o) { + return true; + } + if (!(o instanceof VideoFrameAggregationParameters)) { + return false; + } + VideoFrameAggregationParameters that = (VideoFrameAggregationParameters) o; + return Objects.equals(frameRate, that.frameRate); + } + + @Override + public int hashCode() { + return Objects.hashCode(frameRate); + } + + @Override + public String toString() { + return toJsonObject().toString(); + } + + /** Returns a {@link JSONObject} that represents the {@code VideoFrameAggregationParameters}. */ + /* package */ JSONObject toJsonObject() { + JSONObject jsonObject = new JSONObject(); + try { + if (frameRate != null) { + jsonObject.put("frameRate", frameRate.getNumerator() + "/" + frameRate.getDenominator()); + } + } catch (JSONException e) { + Log.w(/* tag= */ "VideoFrameAggParams", "JSON conversion failed.", e); + return new JSONObject(); + } + return jsonObject; + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/BitmapToHardwareBufferConverterTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/BitmapToHardwareBufferConverterTest.java new file mode 100644 index 00000000000..792f69d9ea5 --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/BitmapToHardwareBufferConverterTest.java @@ -0,0 +1,126 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static androidx.media3.transformer.TransformerUtil.releaseIfNeeded; +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.util.concurrent.MoreExecutors.directExecutor; +import static org.junit.Assert.assertThrows; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; + +import android.graphics.Bitmap; +import androidx.media3.common.video.HardwareBufferFrame; +import androidx.media3.effect.HardwareBufferJniWrapper; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.util.concurrent.ExecutorService; +import java.util.concurrent.Executors; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.annotation.Config; + +/** Robolectric tests for {@link BitmapToHardwareBufferConverter}. */ +@RunWith(AndroidJUnit4.class) +@Config(minSdk = 26) // HardwareBuffers only exist on API26+. +public final class BitmapToHardwareBufferConverterTest { + + private ExecutorService executorService; + private BitmapToHardwareBufferConverter converter; + private HardwareBufferJniWrapper mockJniWrapper; + + @Before + public void setUp() { + executorService = Executors.newSingleThreadExecutor(); + mockJniWrapper = mock(HardwareBufferJniWrapper.class); + when(mockJniWrapper.nativeCopyBitmapToHardwareBuffer(any(), any())).thenReturn(true); + + converter = + new BitmapToHardwareBufferConverter( + mockJniWrapper, + /* internalExecutor= */ executorService, + /* errorExecutor= */ directExecutor(), + /* errorCallback= */ (e) -> { + throw new AssertionError(e); + }); + } + + @After + public void tearDown() { + if (converter != null) { + converter.close(); + } + if (executorService != null) { + executorService.shutdown(); + } + } + + @Test + public void getOrCreateRetainedFrame_sameBitmap_reusesCachedFrame() { + Bitmap bitmap = Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888); + + HardwareBufferFrame frame1 = converter.getOrCreateRetainedFrame(bitmap); + HardwareBufferFrame frame2 = converter.getOrCreateRetainedFrame(bitmap); + + assertThat(frame1.getHardwareBuffer()).isNotNull(); + assertThat(frame1.getHardwareBuffer()).isSameInstanceAs(frame2.getHardwareBuffer()); + + releaseIfNeeded(frame1, /* releaseFence= */ null); + releaseIfNeeded(frame2, /* releaseFence= */ null); + } + + @Test + public void getOrCreateRetainedFrame_differentBitmap_createsNewFrame() { + Bitmap bitmap1 = Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888); + Bitmap bitmap2 = Bitmap.createBitmap(20, 20, Bitmap.Config.ARGB_8888); + + HardwareBufferFrame frame1 = converter.getOrCreateRetainedFrame(bitmap1); + HardwareBufferFrame frame2 = converter.getOrCreateRetainedFrame(bitmap2); + + assertThat(frame1.getHardwareBuffer()).isNotNull(); + assertThat(frame2.getHardwareBuffer()).isNotNull(); + assertThat(frame1.getHardwareBuffer()).isNotSameInstanceAs(frame2.getHardwareBuffer()); + + releaseIfNeeded(frame1, /* releaseFence= */ null); + releaseIfNeeded(frame2, /* releaseFence= */ null); + } + + @Test + public void flush_withoutErrors_releasesCachedFrame() { + Bitmap bitmap = Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888); + + HardwareBufferFrame frame1 = converter.getOrCreateRetainedFrame(bitmap); + converter.flush(); + HardwareBufferFrame frame2 = converter.getOrCreateRetainedFrame(bitmap); + + assertThat(frame1.getHardwareBuffer()).isNotNull(); + assertThat(frame2.getHardwareBuffer()).isNotNull(); + assertThat(frame1.getHardwareBuffer()).isNotSameInstanceAs(frame2.getHardwareBuffer()); + + releaseIfNeeded(frame1, /* releaseFence= */ null); + releaseIfNeeded(frame2, /* releaseFence= */ null); + } + + @Test + public void getOrCreateRetainedFrame_nativeCopyFails_throwsIllegalStateException() { + when(mockJniWrapper.nativeCopyBitmapToHardwareBuffer(any(), any())).thenReturn(false); + Bitmap bitmap = Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888); + + assertThrows(IllegalStateException.class, () -> converter.getOrCreateRetainedFrame(bitmap)); + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/CodecDbLiteTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/CodecDbLiteTest.java index 9d267397117..311201cee66 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/CodecDbLiteTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/CodecDbLiteTest.java @@ -21,7 +21,6 @@ import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; import androidx.test.ext.junit.runners.AndroidJUnit4; -import org.junit.BeforeClass; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.shadows.ShadowBuild; @@ -30,12 +29,6 @@ @RunWith(AndroidJUnit4.class) public class CodecDbLiteTest { - @BeforeClass - public static void setUp() { - ShadowBuild.setSystemOnChipManufacturer("Google"); - ShadowBuild.setSystemOnChipModel("Tensor G3"); - } - @Test public void getRecommendedVideoEncoderSettings_noMimeType_throwsIllegalArgumentException() { assertThrows( @@ -54,6 +47,10 @@ public void getRecommendedVideoEncoderSettings_nonVideoMimeType_throwsIllegalArg @Test public void getRecommendedVideoEncoderSettings_avc1080p30_returnsRecommendedBitrate() { + // Samsung Exynos 850 opts in to the rate model at 1080p. + ShadowBuild.setSystemOnChipManufacturer("Samsung"); + ShadowBuild.setSystemOnChipModel("Exynos 850"); + Format format = new Format.Builder() .setSampleMimeType(MimeTypes.VIDEO_H264) @@ -68,37 +65,48 @@ public void getRecommendedVideoEncoderSettings_avc1080p30_returnsRecommendedBitr } @Test - public void getRecommendedVideoEncoderSettings_hevc1080p30_returnsRecommendedBitrate() { + public void getRecommendedVideoEncoderSettings_hevc720p30_returnsRecommendedBitrate() { + // Samsung s5e9925 keeps its H.265 recommendation and is approved at 720p. + ShadowBuild.setSystemOnChipManufacturer("Samsung"); + ShadowBuild.setSystemOnChipModel("s5e9925"); + Format format = new Format.Builder() .setSampleMimeType(MimeTypes.VIDEO_H265) - .setWidth(1920) - .setHeight(1080) + .setWidth(1280) + .setHeight(720) .setFrameRate(30) .build(); VideoEncoderSettings settings = CodecDbLite.getRecommendedVideoEncoderSettings(format); - assertThat(settings.bitrate).isEqualTo(12_116_617); + assertThat(settings.bitrate).isEqualTo(8_904_992); } @Test - public void getRecommendedVideoEncoderSettings_av11080p30_returnsHevcRecommendedBitrate() { + public void getRecommendedVideoEncoderSettings_av14k30_returnsHevcRecommendedBitrate() { + // Google Tensor G4 has an AV1 entry that opts in to the rate model at 4k. + ShadowBuild.setSystemOnChipManufacturer("Google"); + ShadowBuild.setSystemOnChipModel("Tensor G4"); + Format format = new Format.Builder() .setSampleMimeType(MimeTypes.VIDEO_AV1) - .setWidth(1920) - .setHeight(1080) + .setWidth(3840) + .setHeight(2160) .setFrameRate(30) .build(); VideoEncoderSettings settings = CodecDbLite.getRecommendedVideoEncoderSettings(format); - assertThat(settings.bitrate).isEqualTo(12_116_617); + assertThat(settings.bitrate).isEqualTo(18_947_248); } @Test public void getRecommendedVideoEncoderSettings_unsupportedVideoCodec_returnsNoBitrate() { + ShadowBuild.setSystemOnChipManufacturer("Google"); + ShadowBuild.setSystemOnChipModel("Tensor G3"); + Format format = new Format.Builder() .setSampleMimeType(MimeTypes.VIDEO_VP9) @@ -113,7 +121,7 @@ public void getRecommendedVideoEncoderSettings_unsupportedVideoCodec_returnsNoBi } @Test - public void getRecommendedVideoEncoderSettings_unknownChipset_returnsRecommendedBitrate() { + public void getRecommendedVideoEncoderSettings_unknownChipset_returnsNoBitrate() { ShadowBuild.setSystemOnChipManufacturer("Unknown"); ShadowBuild.setSystemOnChipModel("Unknown"); @@ -127,6 +135,222 @@ public void getRecommendedVideoEncoderSettings_unknownChipset_returnsRecommended VideoEncoderSettings settings = CodecDbLite.getRecommendedVideoEncoderSettings(format); - assertThat(settings.bitrate).isEqualTo(15_502_805); + assertThat(settings.bitrate).isEqualTo(VideoEncoderSettings.NO_VALUE); + } + + @Test + public void getRecommendedVideoEncoderSettings_chipsetWithoutRateModel1080p_returnsNoBitrate() { + // Mediatek MT6761 does not opt in to the rate model in any scope. + ShadowBuild.setSystemOnChipManufacturer("Mediatek"); + ShadowBuild.setSystemOnChipModel("MT6761"); + + Format format = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H264) + .setWidth(1920) + .setHeight(1080) + .setFrameRate(30) + .build(); + + VideoEncoderSettings settings = CodecDbLite.getRecommendedVideoEncoderSettings(format); + + assertThat(settings.bitrate).isEqualTo(VideoEncoderSettings.NO_VALUE); + } + + @Test + public void getRecommendedVideoEncoderSettings_chipsetApprovedAt4kOnly_1080p_returnsNoBitrate() { + // Mediatek MT6785 only opts in to the rate model in the 4k scope. + ShadowBuild.setSystemOnChipManufacturer("Mediatek"); + ShadowBuild.setSystemOnChipModel("MT6785"); + + Format format = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H264) + .setWidth(1920) + .setHeight(1080) + .setFrameRate(30) + .build(); + + VideoEncoderSettings settings = CodecDbLite.getRecommendedVideoEncoderSettings(format); + + assertThat(settings.bitrate).isEqualTo(VideoEncoderSettings.NO_VALUE); + } + + @Test + public void + getRecommendedVideoEncoderSettings_chipsetApprovedAt4kOnly_4k_returnsRecommendedBitrate() { + ShadowBuild.setSystemOnChipManufacturer("Mediatek"); + ShadowBuild.setSystemOnChipModel("MT6785"); + + Format format = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H264) + .setWidth(3840) + .setHeight(2160) + .setFrameRate(30) + .build(); + + VideoEncoderSettings settings = CodecDbLite.getRecommendedVideoEncoderSettings(format); + + assertThat(settings.bitrate).isEqualTo(29_044_193); + } + + @Test + public void + getRecommendedVideoEncoderSettings_chipsetApprovedAt720pNot1080p_720p_returnsRecommendedBitrate() { + // Samsung s5e9925 opts in to the rate model at 720p and 4k but not at 1080p. + ShadowBuild.setSystemOnChipManufacturer("Samsung"); + ShadowBuild.setSystemOnChipModel("s5e9925"); + + Format format = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H264) + .setWidth(1280) + .setHeight(720) + .setFrameRate(30) + .build(); + + VideoEncoderSettings settings = CodecDbLite.getRecommendedVideoEncoderSettings(format); + + assertThat(settings.bitrate).isEqualTo(10_549_473); + } + + @Test + public void + getRecommendedVideoEncoderSettings_chipsetApprovedAt720pNot1080p_1080p_returnsNoBitrate() { + ShadowBuild.setSystemOnChipManufacturer("Samsung"); + ShadowBuild.setSystemOnChipModel("s5e9925"); + + Format format = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H264) + .setWidth(1920) + .setHeight(1080) + .setFrameRate(30) + .build(); + + VideoEncoderSettings settings = CodecDbLite.getRecommendedVideoEncoderSettings(format); + + assertThat(settings.bitrate).isEqualTo(VideoEncoderSettings.NO_VALUE); + } + + @Test + public void getRecommendedVideoMimeType_chipsetWithoutHevcEntry_returnsAvc() { + // QTI SM6225 has no H.265 entry in the dataset, so H.264 is recommended. + ShadowBuild.setSystemOnChipManufacturer("QTI"); + ShadowBuild.setSystemOnChipModel("SM6225"); + + assertThat(CodecDbLite.getRecommendedVideoMimeType()).isEqualTo(MimeTypes.VIDEO_H264); + } + + @Test + public void getRecommendedVideoMimeType_chipsetWithHevcEntry_returnsHevc() { + // Samsung s5e9925 has an H.265 entry ahead of its H.264 entry. + ShadowBuild.setSystemOnChipManufacturer("Samsung"); + ShadowBuild.setSystemOnChipModel("s5e9925"); + + assertThat(CodecDbLite.getRecommendedVideoMimeType()).isEqualTo(MimeTypes.VIDEO_H265); + } + + @Test + public void + getRecommendedVideoEncoderSettings_tensorG3_720p_hevcReturnsNoBitrate_h264ReturnsBitrate() { + // Google Tensor G3 carries the 720p rate model flag on its H.264 entry but not on its H.265 + // entry. + ShadowBuild.setSystemOnChipManufacturer("Google"); + ShadowBuild.setSystemOnChipModel("Tensor G3"); + + Format hevcFormat = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H265) + .setWidth(1280) + .setHeight(720) + .setFrameRate(30) + .build(); + assertThat(CodecDbLite.getRecommendedVideoEncoderSettings(hevcFormat).bitrate) + .isEqualTo(VideoEncoderSettings.NO_VALUE); + + Format h264Format = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H264) + .setWidth(1280) + .setHeight(720) + .setFrameRate(30) + .build(); + assertThat(CodecDbLite.getRecommendedVideoEncoderSettings(h264Format).bitrate) + .isEqualTo(10_549_473); + } + + @Test + public void getRecommendedVideoEncoderSettings_wideAspectRatioIn1080pScope_returnsNoBitrate() { + // 1920x816 is a 2.35:1 crop of 1080p. It is classified as 1080p because its long edge is + // 1920, and Samsung s5e9925 does not carry the 1080p rate model flag. + // + // Classifying by pixel count instead would place its 1,566,720 pixels below the 1080p pixel + // count and into the 720p scope, where s5e9925 does carry the flag, applying the rate model + // at a scope it was not validated for. + ShadowBuild.setSystemOnChipManufacturer("Samsung"); + ShadowBuild.setSystemOnChipModel("s5e9925"); + + Format format = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H264) + .setWidth(1920) + .setHeight(816) + .setFrameRate(30) + .build(); + + VideoEncoderSettings settings = CodecDbLite.getRecommendedVideoEncoderSettings(format); + + assertThat(settings.bitrate).isEqualTo(VideoEncoderSettings.NO_VALUE); + } + + @Test + public void + getRecommendedVideoEncoderSettings_wideAspectRatioIn4kScope_returnsRecommendedBitrate() + throws Exception { + // 3840x1608 is a 2.39:1 crop of 4k. It is classified as 4k because its long edge is 3840, and + // Mediatek MT6785 carries the 4k rate model flag but not the 1080p one. + // + // This is the same classifier disagreement as the 1080p case above but in the opposite + // direction: its 6,174,720 pixels are fewer than a full 4k frame, so bucketing by pixel count + // would withhold a recommendation the dataset does allow. The exact bitrate is not pinned here + // because the formula itself is covered by the canonical-resolution tests above; what matters + // is that the 4k scope flag is honoured. + ShadowBuild.setSystemOnChipManufacturer("Mediatek"); + ShadowBuild.setSystemOnChipModel("MT6785"); + + Format format = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H264) + .setWidth(3840) + .setHeight(1608) + .setFrameRate(30) + .build(); + + VideoEncoderSettings settings = CodecDbLite.getRecommendedVideoEncoderSettings(format); + + assertThat(settings.bitrate).isNotEqualTo(VideoEncoderSettings.NO_VALUE); + assertThat(settings.bitrate).isGreaterThan(0); + } + + @Test + public void getRecommendedVideoEncoderSettings_belowSmallestModelledScope_returnsNoBitrate() { + // 854x480 is below the 720p scope, the smallest scope the rate model is validated for, so no + // recommendation is made even for a chipset that carries every scope flag. + ShadowBuild.setSystemOnChipManufacturer("Samsung"); + ShadowBuild.setSystemOnChipModel("Exynos 850"); + + Format format = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_H264) + .setWidth(854) + .setHeight(480) + .setFrameRate(30) + .build(); + + VideoEncoderSettings settings = CodecDbLite.getRecommendedVideoEncoderSettings(format); + + assertThat(settings.bitrate).isEqualTo(VideoEncoderSettings.NO_VALUE); } } diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionExportTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionExportTest.java index 45fa644d16e..32c03fd75db 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionExportTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionExportTest.java @@ -169,9 +169,7 @@ public void start_loopingTransmuxedVideo_producesExpectedResult() throws Excepti @Test public void start_longVideoCompositionWithLoopingAudio_producesExpectedResult() throws Exception { - CapturingMuxer.Factory muxerFactory = new CapturingMuxer.Factory(/* handleAudioAsPcm= */ true); - Transformer transformer = - new TestTransformerBuilder(context).setMuxerFactory(muxerFactory).build(); + Transformer transformer = new TestTransformerBuilder(context).build(); EditedMediaItemSequence loopingAudioSequence = new EditedMediaItemSequence.Builder(ImmutableSet.of(TRACK_TYPE_AUDIO)) .addItem( @@ -182,7 +180,6 @@ public void start_longVideoCompositionWithLoopingAudio_producesExpectedResult() EditedMediaItem videoEditedMediaItem = new EditedMediaItem.Builder( MediaItem.fromUri(ASSET_URI_PREFIX + FILE_AUDIO_VIDEO_INCREASING_TIMESTAMPS_15S)) - .setRemoveAudio(true) .build(); EditedMediaItemSequence videoSequence = EditedMediaItemSequence.withVideoFrom( @@ -194,8 +191,11 @@ public void start_longVideoCompositionWithLoopingAudio_producesExpectedResult() ExportResult exportResult = TransformerTestRunner.runLooper(transformer); // TODO: b/443998866 - Use MetadataRetriever to get exact duration. - assertThat(exportResult.approximateDurationMs).isEqualTo(31_065); - // FILE_AUDIO_RAW duration is 1000ms. Input 32 times to cover the 31_053ms duration. + // The video's duration is 15_534ms, so the video sequence has a duration of 31_068ms. However, + // the looping sequence is not truncated mid-buffer (b/530852636), so the audio sequence outputs + // 31_100ms, aligned with WavExtractor's 100ms buffers. + assertThat(exportResult.approximateDurationMs).isEqualTo(31_100); + // FILE_AUDIO_RAW duration is 1000ms. Input 32 times to cover the 31_068ms duration + 2 videos. assertThat(exportResult.processedInputs).hasSize(34); assertThat(exportResult.channelCount).isEqualTo(1); assertThat(exportResult.fileSizeBytes).isEqualTo(5_692_714); diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionFrameMetadataTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionFrameMetadataTest.java new file mode 100644 index 00000000000..39fbf527353 --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionFrameMetadataTest.java @@ -0,0 +1,118 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static androidx.media3.effect.DefaultGlFrameProcessor.KEY_COMPOSITION_EFFECTS; +import static androidx.media3.effect.DefaultGlFrameProcessor.KEY_COMPOSITION_SEQUENCE_INDEX; +import static androidx.media3.effect.DefaultGlFrameProcessor.KEY_COMPOSITOR_SETTINGS; +import static androidx.media3.effect.DefaultGlFrameProcessor.KEY_ITEM_EFFECTS; +import static androidx.media3.test.utils.AssetInfo.MP4_ADVANCED_ASSET; +import static androidx.media3.transformer.Composition.KEY_COMPOSITION; +import static androidx.media3.transformer.Composition.KEY_COMPOSITION_ITEM_INDEX; +import static androidx.media3.transformer.EditedMediaItemSequence.withAudioFrom; +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; + +import androidx.media3.common.Effect; +import androidx.media3.common.MediaItem; +import androidx.media3.effect.Presentation; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit test for {@link CompositionFrameMetadata}. */ +@RunWith(AndroidJUnit4.class) +@SuppressWarnings("deprecation") // Uses deprecated CompositionFrameMetadata +public final class CompositionFrameMetadataTest { + + @Test + public void asFrameMetadata_validIndices_populatesAllKeysIncludingItemEffects() { + Effect testEffect = Presentation.createForHeight(720); + EditedMediaItem item = + new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)) + .setEffects( + new Effects( + /* audioProcessors= */ ImmutableList.of(), ImmutableList.of(testEffect))) + .build(); + EditedMediaItemSequence sequence = withAudioFrom(ImmutableList.of(item)); + Composition composition = new Composition.Builder(sequence).build(); + CompositionFrameMetadata metadata = + new CompositionFrameMetadata(composition, /* sequenceIndex= */ 0, /* itemIndex= */ 0); + + ImmutableMap map = CompositionFrameMetadata.asFrameMetadata(metadata); + + assertThat(map).containsEntry(KEY_COMPOSITION, composition); + assertThat(map).containsEntry(KEY_COMPOSITION_SEQUENCE_INDEX, 0); + assertThat(map).containsEntry(KEY_COMPOSITION_ITEM_INDEX, 0); + assertThat(map).containsEntry(KEY_COMPOSITOR_SETTINGS, composition.videoCompositorSettings); + assertThat(map).containsEntry(KEY_COMPOSITION_EFFECTS, composition.effects.videoEffects); + assertThat(map).containsEntry(KEY_ITEM_EFFECTS, ImmutableList.of(testEffect)); + } + + @Test + public void asFrameMetadata_negativeSequenceIndex_throwsIndexOutOfBoundsException() { + EditedMediaItem item = + new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)).build(); + Composition composition = + new Composition.Builder(withAudioFrom(ImmutableList.of(item))).build(); + CompositionFrameMetadata metadata = + new CompositionFrameMetadata(composition, /* sequenceIndex= */ -1, /* itemIndex= */ 0); + + assertThrows( + IndexOutOfBoundsException.class, () -> CompositionFrameMetadata.asFrameMetadata(metadata)); + } + + @Test + public void asFrameMetadata_outOfBoundsSequenceIndex_throwsIndexOutOfBoundsException() { + EditedMediaItem item = + new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)).build(); + Composition composition = + new Composition.Builder(withAudioFrom(ImmutableList.of(item))).build(); + CompositionFrameMetadata metadata = + new CompositionFrameMetadata(composition, /* sequenceIndex= */ 1, /* itemIndex= */ 0); + + assertThrows( + IndexOutOfBoundsException.class, () -> CompositionFrameMetadata.asFrameMetadata(metadata)); + } + + @Test + public void asFrameMetadata_negativeItemIndex_throwsIndexOutOfBoundsException() { + EditedMediaItem item = + new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)).build(); + Composition composition = + new Composition.Builder(withAudioFrom(ImmutableList.of(item))).build(); + CompositionFrameMetadata metadata = + new CompositionFrameMetadata(composition, /* sequenceIndex= */ 0, /* itemIndex= */ -1); + + assertThrows( + IndexOutOfBoundsException.class, () -> CompositionFrameMetadata.asFrameMetadata(metadata)); + } + + @Test + public void asFrameMetadata_outOfBoundsItemIndex_throwsIndexOutOfBoundsException() { + EditedMediaItem item = + new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)).build(); + Composition composition = + new Composition.Builder(withAudioFrom(ImmutableList.of(item))).build(); + CompositionFrameMetadata metadata = + new CompositionFrameMetadata(composition, /* sequenceIndex= */ 0, /* itemIndex= */ 1); + + assertThrows( + IndexOutOfBoundsException.class, () -> CompositionFrameMetadata.asFrameMetadata(metadata)); + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionPlayerAudioPlaybackTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionPlayerAudioPlaybackTest.java index c1bc957a7c6..1792617196f 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionPlayerAudioPlaybackTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionPlayerAudioPlaybackTest.java @@ -33,10 +33,14 @@ import static androidx.media3.transformer.TestUtil.createChannelCountChangingAudioProcessor; import static androidx.media3.transformer.TestUtil.createSampleRateChangingAudioProcessor; import static androidx.media3.transformer.TestUtil.createTestCompositionPlayer; +import static androidx.media3.transformer.TestUtil.createTestCompositionPlayerBuilder; import static androidx.media3.transformer.TestUtil.createVolumeScalingAudioProcessor; import static com.google.common.base.Preconditions.checkState; import static com.google.common.collect.Iterables.getLast; import static com.google.common.truth.Truth.assertThat; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; import android.content.Context; import androidx.media3.common.C; @@ -45,6 +49,7 @@ import androidx.media3.common.Player; import androidx.media3.common.audio.AudioProcessor; import androidx.media3.common.audio.SpeedProvider; +import androidx.media3.exoplayer.analytics.AnalyticsListener; import androidx.media3.exoplayer.audio.AudioSink; import androidx.media3.test.utils.CapturingAudioSink; import androidx.media3.test.utils.DumpFileAsserts; @@ -74,6 +79,7 @@ *

    These tests focus on audio because the video pipeline doesn't work in Robolectric. */ @RunWith(RobolectricTestParameterInjector.class) +// TODO(b/540736479): Implement better instrumentation for seeking tests. public final class CompositionPlayerAudioPlaybackTest { @Rule public final Expect expect = Expect.create(); @@ -1443,6 +1449,33 @@ public void seek_withMultipleAudioSequences_reportsExpectedPositionToEachSequenc expect.that(getLast(secondSequenceRecorder.positionOffsetsUs)).isEqualTo(100_000L); } + @Test + public void seek_withScrubbingModeEnabled_disablesAudioTrackWhileScrubbing() throws Exception { + AnalyticsListener listener = mock(AnalyticsListener.class); + player = + createTestCompositionPlayerBuilder().setSequencePlayerAnalyticsListener(listener).build(); + EditedMediaItem item = + new EditedMediaItem.Builder(MediaItem.fromUri(WAV_ASSET.uri)) + .setDurationUs(1_000_000L) + .build(); + Composition composition = + new Composition.Builder(withAudioFrom(ImmutableList.of(item))).build(); + + player.setComposition(composition); + player.prepare(); + advance(player).untilState(STATE_READY); + + verify(listener).onAudioEnabled(any(), any()); + + player.setScrubbingModeEnabled(true); + advance(player).untilPendingCommandsAreFullyHandled(); + verify(listener).onAudioDisabled(any(), any()); + + player.setScrubbingModeEnabled(false); + advance(player).untilState(STATE_READY); + verify(listener).onAudioEnabled(any(), any()); + } + private static class ForwardingAudioMixer implements AudioMixer { private final AudioMixer wrappedAudioMixer; diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionPlayerSeekTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionPlayerSeekTest.java new file mode 100644 index 00000000000..d85ee12c7b0 --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionPlayerSeekTest.java @@ -0,0 +1,321 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static androidx.media3.common.Player.STATE_ENDED; +import static androidx.media3.common.Player.STATE_READY; +import static androidx.media3.test.utils.AssetInfo.MP4_15FPS; +import static androidx.media3.test.utils.AssetInfo.MP4_ADVANCED_ASSET; +import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_WITH_INCREASING_TIMESTAMPS; +import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_5S; +import static androidx.media3.test.utils.robolectric.RobolectricUtil.runMainLooperUntil; +import static androidx.media3.test.utils.robolectric.ShadowMediaCodecConfig.CODEC_INFO_AVC; +import static androidx.media3.test.utils.robolectric.ShadowMediaCodecConfig.CODEC_INFO_RAW; +import static androidx.media3.test.utils.robolectric.TestPlayerRunHelper.advance; +import static androidx.media3.transformer.TestUtil.FPS_10; +import static androidx.media3.transformer.TestUtil.FPS_30; +import static androidx.media3.transformer.TestUtil.FPS_60; +import static androidx.media3.transformer.TestUtil.assertTimestampsMatchFrameRate; +import static androidx.media3.transformer.TestUtil.buildComposition; +import static androidx.media3.transformer.TestUtil.getQueuedContentTimesUs; +import static androidx.media3.transformer.TestUtil.setupAndPrepareHardwareBufferPlayer; +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.truth.Truth.assertWithMessage; + +import android.util.Rational; +import androidx.media3.test.utils.CapturingFrameProcessor; +import androidx.media3.test.utils.FakeFrameProcessor; +import androidx.media3.test.utils.robolectric.ShadowMediaCodecConfig; +import com.google.common.collect.ImmutableList; +import com.google.testing.junit.testparameterinjector.TestParameter; +import java.util.List; +import org.checkerframework.checker.nullness.qual.MonotonicNonNull; +import org.junit.After; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.runner.RunWith; +import org.robolectric.RobolectricTestParameterInjector; + +/** Robolectric parameterized seek tests for {@link CompositionPlayer}. */ +@RunWith(RobolectricTestParameterInjector.class) +public final class CompositionPlayerSeekTest { + + @Rule + public ShadowMediaCodecConfig shadowMediaCodecConfig = + ShadowMediaCodecConfig.withCodecs( + /* decoders= */ ImmutableList.of(CODEC_INFO_AVC), + /* encoders= */ ImmutableList.of(CODEC_INFO_RAW)); + + @TestParameter boolean isScrubbingModeEnabled; + + @MonotonicNonNull CompositionPlayer player; + + private CapturingFrameProcessor.Factory frameProcessorFactory; + + @Before + public void setUp() { + frameProcessorFactory = + new CapturingFrameProcessor.Factory( + new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ true)); + } + + @After + public void tearDown() { + if (player != null) { + player.release(); + } + } + + @Test + public void seek_withAggregationFpsHigherThanIntrinsic_outputsFramesAtTargetFps() + throws Exception { + // Asset (30): ... | 167 | | 200 | | 233 | ... | 733 | | 767 | ... + // Target (60): ... | 167 | 183 | 200 | 217 | 233 | ... | 733 | 750 | 767 | ... + // ^ ^ + // Seek request: 200ms 750ms + // | | + // | +-----+ + // v v + // First output: 200ms 767ms + // (Aggregator isn't aware of original seek + // time and assumes it based on first + // decoded frame of 767. Not important to + // fix since content of a 750 would be + // identical anyway.) + Composition composition = + buildComposition( + ImmutableList.of(ImmutableList.of(MP4_ASSET_WITH_INCREASING_TIMESTAMPS)), FPS_60); + + runSeekAndPlaybackAssertingTimestamps( + composition, + ImmutableList.of(750L, 200L), + /* expectedPostSeekTimestampsUs= */ ImmutableList.of(766_667L, 200_000L), + /* targetFps= */ 60); + } + + @Test + public void seek_withAggregationFpsLowerThanIntrinsic_outputsFramesAtTargetFps() + throws Exception { + // Asset (30): ... | 200 | 233 | 267 | 300 | ... | 700 | 733 | 767 | 800 | ... + // Target (10): ... | 200 | | 300 | ... | 700 | | 800 | ... + // ^ ^ ^ + // Seek request: 200ms 233ms 750ms + // | | | + // | +-----------+ +--------+ + // v v v + // First output: 200ms 300ms 800ms + Composition composition = + buildComposition( + ImmutableList.of(ImmutableList.of(MP4_ASSET_WITH_INCREASING_TIMESTAMPS)), FPS_10); + + runSeekAndPlaybackAssertingTimestamps( + composition, + ImmutableList.of(750L, 200L, 233L), + /* expectedPostSeekTimestampsUs= */ ImmutableList.of(800_000L, 200_000L, 300_000L), + /* targetFps= */ 10); + } + + @Test + public void seek_withAggregationFpsMatchingIntrinsic_outputsFramesAtTargetFps() throws Exception { + // Asset (30): ... | 167 | | 200 | | 233 | ... | 733 | | 767 | ... + // Target (30): ... | 167 | | 200 | | 233 | ... | 733 | | 767 | ... + // ^ ^ + // Seek request: 200ms 750ms + // | | + // | +-----+ + // v v + // First output: 200ms 767ms + Composition composition = + buildComposition( + ImmutableList.of(ImmutableList.of(MP4_ASSET_WITH_INCREASING_TIMESTAMPS)), FPS_30); + + runSeekAndPlaybackAssertingTimestamps( + composition, + ImmutableList.of(750L, 200L), + /* expectedPostSeekTimestampsUs= */ ImmutableList.of(766_667L, 200_000L), + /* targetFps= */ 30); + } + + @Test + public void seek_withAggregationFpsHigherThanIntrinsicAndBFrame_outputsFramesAtTargetFps() + throws Exception { + // Asset (30): ... |200.2p| ... |734.1b| |767.4b| |800.8b| |834.2p| ... + // Target (60): ... | 200 | 217 | ... | 750 | 767 | 783 | 800 | 817 | 833 | 850 | ... + // ^ ^ + // Seek request: 200ms 750ms + // | | + // +-----+ +--------------------------------------+ + // v v + // First output: 217ms 850ms + // (Aggregator isn't aware of original seek (B-frames 767.4 and 800.8 + // time and assumes it based on first are discarded post-seek; + // decoded frame of 200.2. Not important to first output is based + // fix since content of a 200.2 would be on P-frame 834.2 and + // identical anyway.) next target tick is 850) + Composition composition = + buildComposition(ImmutableList.of(ImmutableList.of(MP4_ADVANCED_ASSET)), FPS_60); + + runSeekAndPlaybackAssertingTimestamps( + composition, + ImmutableList.of(750L, 200L), + /* expectedPostSeekTimestampsUs= */ ImmutableList.of(850_000L, 216_667L), + /* targetFps= */ 60); + } + + @Test + public void seek_withAggregationFpsLowerThanIntrinsicAndBFrame_outputsFramesAtTargetFps() + throws Exception { + // Asset (30): ... |200.2p| ... |701.7p|734.1b|767.4b|800.8b|834.2p|867.5b|900.9b| ... + // Target (10): ... | 200 | 300 | ... | 700 | | 800 | | 900 | ... + // ^ ^ + // Seek request: 200ms 750ms + // | | + // +-----+ +-------------------------------+ + // v v + // First output: 300ms 900ms + // (Aggregator isn't aware of original seek (B-frames 767.4 and 800.8 + // time and assumes it based on first are discarded post-seek; + // decoded frame of 200.2. Not important to first output is based + // fix since content of a 200.2 would be on P-frame 834.2 and + // identical anyway.) next target tick is 900) + Composition composition = + buildComposition(ImmutableList.of(ImmutableList.of(MP4_ADVANCED_ASSET)), FPS_10); + + runSeekAndPlaybackAssertingTimestamps( + composition, + ImmutableList.of(750L, 200L), + /* expectedPostSeekTimestampsUs= */ ImmutableList.of(900_000L, 300_000L), + /* targetFps= */ 10); + } + + @Test + public void seek_withAggregationFpsAndVaryingFpsInSingleSequence_outputsFramesAtTargetFps() + throws Exception { + // Seek 400ms & 500ms (Asset 1 with 15 FPS): + // Asset (15): ... | 400 | | 467 | | 533 | ... + // Target (30): ... | 400 | 433 | 467 | 500 | 533 | ... + // ^ ^ + // Seek request: 400ms 500ms + // | | + // | +-----+ + // v v + // First output: 400ms 533ms + // + // Seek 1490ms & 1600ms (Asset 2 with 30 FPS): + // Asset (30): ... | 1497 | ... | 1597 | 1630 | ... + // Target (30): ... | 1500 | ... | 1600 | 1633 | ... + // ^ ^ + // Seek request: 1490ms 1600ms + // | | + // +-----+ +-------+ + // v v + // First output: 1500ms 1633ms + // + // Seek 2130ms & 2500ms (Asset 3 with 60 FPS): + // Asset (60): ... | 2130 | ... | 2497 | 2513 | 2530 | 2547 | 2563 | 2580 | 2597 | ... + // Target (30): ... | 2133 | ... | 2500 | | 2533 | | 2567 | | 2600 | ... + // ^ ^ ^ + // Seek request: 2130ms 2500ms 2590ms + // | | | + // ++ +-------------+ +-----------+ + // v v v + // First output: 2133ms 2533ms 2600ms + Composition composition = + buildComposition( + ImmutableList.of( + ImmutableList.of( + MP4_15FPS, // 15 FPS + MP4_ASSET_WITH_INCREASING_TIMESTAMPS, // 30 FPS + MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_5S)), // 60 FPS + FPS_30); + + runSeekAndPlaybackAssertingTimestamps( + composition, + ImmutableList.of(1600L, 2590L, 400L, 1490L, 2130L, 500L, 2500L), + /* expectedPostSeekTimestampsUs= */ ImmutableList.of( + 1_633_333L, 2_600_000L, 400_000L, 1_500_000L, 2_133_333L, 533_333L, 2_533_333L), + /* targetFps= */ 30); + } + + @Test + public void seek_withAggregationFpsAndVaryingFpsInMultiSequence_outputsFramesAtTargetFps() + throws Exception { + // Parallel tracks: + // 15 FPS Track: ... | 267 | | 333 | ... ... | 733 | | 800 | ... + // 30 FPS Track: ... | 267 | 300 | 333 | ... | 700 | | 733 | | 767 | ... + // 60 FPS Track: ... | 267 | 300 | 333 | ... | 700 | 717 | 733 | 750 | 767 | ... + // Target (30): ... | 267 | 300 | 333 | ... | 700 | | 733 | | 767 | ... + // ^ ^ ^ + // Seek request: 300ms 710ms 750ms + // | +--------+ +-----+ + // v v v + // First output: 300ms 733ms 767ms + Composition composition = + buildComposition( + ImmutableList.of( + ImmutableList.of(MP4_15FPS), // 15 FPS + ImmutableList.of(MP4_ASSET_WITH_INCREASING_TIMESTAMPS), // 30 FPS + ImmutableList.of(MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_5S)), // 60 FPS + FPS_30); + + runSeekAndPlaybackAssertingTimestamps( + composition, + ImmutableList.of(710L, 300L, 750L), + /* expectedPostSeekTimestampsUs= */ ImmutableList.of(733_333L, 300_000L, 766_667L), + /* targetFps= */ 30); + } + + private void runSeekAndPlaybackAssertingTimestamps( + Composition composition, + List seekTimesMs, + List expectedPostSeekTimestampsUs, + int targetFps) + throws Exception { + player = setupAndPrepareHardwareBufferPlayer(composition, frameProcessorFactory); + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); + + runMainLooperUntil(() -> !getQueuedContentTimesUs(frameProcessor).isEmpty()); + ImmutableList initialTimestampsUs = getQueuedContentTimesUs(frameProcessor); + player.setScrubbingModeEnabled(isScrubbingModeEnabled); + int numSeeks = seekTimesMs.size(); + ImmutableList.Builder actualSeekTimestampsUs = ImmutableList.builder(); + ImmutableList queuedTimestampsUs = initialTimestampsUs; + for (int i = 0; i < numSeeks; i++) { + int indexOfSeekFrame = queuedTimestampsUs.size(); + player.seekTo(seekTimesMs.get(i)); + advance(player).untilState(STATE_READY); + runMainLooperUntil(() -> getQueuedContentTimesUs(frameProcessor).size() > indexOfSeekFrame); + queuedTimestampsUs = getQueuedContentTimesUs(frameProcessor); + actualSeekTimestampsUs.add(queuedTimestampsUs.get(indexOfSeekFrame)); + } + player.setScrubbingModeEnabled(false); + int packetsQueuedBeforePlay = queuedTimestampsUs.size(); + player.play(); + advance(player).untilState(STATE_ENDED); + + ImmutableList finalTimestampsUs = getQueuedContentTimesUs(frameProcessor); + ImmutableList resumedFrameTimestampsUs = + finalTimestampsUs.subList(packetsQueuedBeforePlay - 1, finalTimestampsUs.size()); + Rational expectedFps = new Rational(targetFps, 1); + assertWithMessage("Queued events: %s", frameProcessor.getQueuedEvents()) + .that(initialTimestampsUs) + .isNotEmpty(); + assertThat(initialTimestampsUs.get(0)).isEqualTo(0); + assertThat(actualSeekTimestampsUs.build()).isEqualTo(expectedPostSeekTimestampsUs); + assertTimestampsMatchFrameRate(resumedFrameTimestampsUs, expectedFps); + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionPlayerTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionPlayerTest.java index fd3c689e15d..adf7583a25c 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionPlayerTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionPlayerTest.java @@ -24,6 +24,8 @@ import static androidx.media3.test.utils.AssetInfo.JPG_ASSET; import static androidx.media3.test.utils.AssetInfo.MP4_12_5FPS; import static androidx.media3.test.utils.AssetInfo.MP4_15FPS; +import static androidx.media3.test.utils.AssetInfo.MP4_ADVANCED_ASSET; +import static androidx.media3.test.utils.AssetInfo.MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_5S; import static androidx.media3.test.utils.AssetInfo.MP4_SIMPLE_ASSET; import static androidx.media3.test.utils.AssetInfo.WAV_ASSET; import static androidx.media3.test.utils.TestUtil.createByteCountingAudioProcessor; @@ -38,9 +40,16 @@ import static androidx.media3.transformer.TestUtil.FILE_AUDIO_RAW; import static androidx.media3.transformer.TestUtil.FILE_AUDIO_RAW_STEREO_48000KHZ; import static androidx.media3.transformer.TestUtil.FILE_PNG; +import static androidx.media3.transformer.TestUtil.FPS_10; +import static androidx.media3.transformer.TestUtil.FPS_30; +import static androidx.media3.transformer.TestUtil.FPS_60; +import static androidx.media3.transformer.TestUtil.FPS_HALF; +import static androidx.media3.transformer.TestUtil.assertTimestampsMatchFrameRate; import static androidx.media3.transformer.TestUtil.createTestCompositionPlayer; import static androidx.media3.transformer.TestUtil.createTestCompositionPlayerBuilder; import static androidx.media3.transformer.TestUtil.createTestHardwareBufferCompositionPlayerBuilder; +import static androidx.media3.transformer.TestUtil.getQueuedContentTimesUs; +import static androidx.media3.transformer.TestUtil.setupAndPrepareHardwareBufferPlayer; import static androidx.test.core.app.ApplicationProvider.getApplicationContext; import static com.google.common.collect.ImmutableList.toImmutableList; import static com.google.common.collect.Iterables.getLast; @@ -61,10 +70,12 @@ import android.os.Handler; import android.os.HandlerThread; import android.os.Looper; +import android.util.Rational; import android.view.Surface; import android.view.TextureView; import androidx.annotation.Nullable; import androidx.media3.common.C; +import androidx.media3.common.Flags; import androidx.media3.common.MediaItem; import androidx.media3.common.PlaybackException; import androidx.media3.common.Player; @@ -74,7 +85,9 @@ import androidx.media3.common.audio.AudioProcessor; import androidx.media3.common.audio.SpeedChangingAudioProcessor; import androidx.media3.common.audio.SpeedProvider; +import androidx.media3.common.util.Clock; import androidx.media3.common.util.ConditionVariable; +import androidx.media3.common.util.HandlerWrapper; import androidx.media3.common.util.Log; import androidx.media3.common.util.NullableType; import androidx.media3.common.video.AsyncFrame; @@ -94,7 +107,14 @@ import androidx.media3.exoplayer.audio.DefaultAudioSink; import androidx.media3.exoplayer.audio.ForwardingAudioSink; import androidx.media3.exoplayer.audio.TrimmingAudioProcessor; +import androidx.media3.exoplayer.video.VideoFrameReleaseControl.FrameTimingEvaluator; +import androidx.media3.test.utils.CapturingFrameProcessor; +import androidx.media3.test.utils.CapturingFrameProcessor.FramesEvent; +import androidx.media3.test.utils.EditedMediaItemAssetInfo; +import androidx.media3.test.utils.FakeClock; import androidx.media3.test.utils.FakeFrameProcessor; +import androidx.media3.test.utils.Media3FlagsRule; +import androidx.media3.test.utils.SequenceAssetInfo; import androidx.media3.test.utils.TestSpeedProvider; import androidx.media3.test.utils.robolectric.ShadowMediaCodecConfig; import androidx.media3.test.utils.robolectric.TestPlayerRunHelper; @@ -105,10 +125,12 @@ import com.google.common.collect.ImmutableSet; import com.google.testing.junit.testparameterinjector.TestParameter; import com.google.testing.junit.testparameterinjector.TestParameterValuesProvider; +import java.nio.ByteBuffer; import java.util.List; import java.util.concurrent.Executor; import java.util.concurrent.TimeoutException; import java.util.concurrent.atomic.AtomicInteger; +import java.util.concurrent.atomic.AtomicLong; import java.util.concurrent.atomic.AtomicReference; import org.checkerframework.checker.nullness.qual.MonotonicNonNull; import org.junit.After; @@ -120,6 +142,7 @@ import org.mockito.InOrder; import org.mockito.Mockito; import org.robolectric.RobolectricTestParameterInjector; +import org.robolectric.annotation.Config; import org.robolectric.shadows.ShadowLooper; /** Unit tests for {@link CompositionPlayer}. */ @@ -421,13 +444,17 @@ public long getNextSpeedChangeTimeUs(long timeUs) { /* expectedDurationMs= */ 900, "AUDIO_ITEM_WITH_CLIPPED_START_AND_END_WITH_SET_HALF_SPEED"); + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); + @MonotonicNonNull CompositionPlayer player; - private FakeFrameProcessor.Factory frameProcessorFactory; + + private CapturingFrameProcessor.Factory frameProcessorFactory; @Before public void setUp() { frameProcessorFactory = - new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ true); + new CapturingFrameProcessor.Factory( + new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ true)); } @After @@ -1349,14 +1376,14 @@ public void frameProcessor_setComposition_rendersFirstFrameBeforeStarted() player.setComposition(composition1); player.prepare(); - advance(player).untilState(STATE_READY); + advance(player).withTimeoutMs(TEST_TIMEOUT_MS).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); assertThat(frameProcessor.getQueuedContentTimesUs()).containsExactly(ImmutableList.of(0L, 0L)); player.setComposition(composition2); - advance(player).untilState(STATE_READY); + advance(player).withTimeoutMs(TEST_TIMEOUT_MS).untilState(STATE_READY); runMainLooperUntilContentTimesUs(frameProcessor, ImmutableList.of(0L, 0L, 0L)); assertThat(frameProcessor.getQueuedContentTimesUs()) @@ -1364,7 +1391,7 @@ public void frameProcessor_setComposition_rendersFirstFrameBeforeStarted() .inOrder(); player.setComposition(composition3); - advance(player).untilState(STATE_READY); + advance(player).withTimeoutMs(TEST_TIMEOUT_MS).untilState(STATE_READY); runMainLooperUntilContentTimesUs(frameProcessor, ImmutableList.of(0L)); assertThat(frameProcessor.getQueuedContentTimesUs()) @@ -1373,6 +1400,68 @@ public void frameProcessor_setComposition_rendersFirstFrameBeforeStarted() .inOrder(); } + @Test + public void frameProcessor_setCompositionWithImageVideo_rendersFirstFrameBeforeStarted() + throws PlaybackException, TimeoutException { + Composition composition1 = + new Composition.Builder( + EditedMediaItemSequence.withAudioAndVideoFrom( + ImmutableList.of( + EditedMediaItemAssetInfo.IMAGE.getEditedMediaItem(), + EditedMediaItemAssetInfo.VIDEO.getEditedMediaItem()))) + .build(); + MaxPositionAudioSink customAudioSink = + new MaxPositionAudioSink(new DefaultAudioSink.Builder(getApplicationContext()).build()); + customAudioSink.maxPositionUs.set(0); + player = + createTestHardwareBufferCompositionPlayerBuilder(frameProcessorFactory) + .setAudioSink(customAudioSink) + .build(); + player.setComposition(composition1); + player.prepare(); + + advance(player).withTimeoutMs(TEST_TIMEOUT_MS).untilState(STATE_READY); + + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); + assertThat(frameProcessor).isNotNull(); + assertThat(frameProcessor.getQueuedContentTimesUs()).containsExactly(ImmutableList.of(0L)); + } + + @Test + @Config(minSdk = 28) + public void frameProcessor_setComposition_withLowFrameRate_rendersFirstFrame() + throws PlaybackException, TimeoutException { + MediaItem clippedMediaItem = + new MediaItem.Builder() + .setUri(MP4_SIMPLE_ASSET.uri) + .setClippingConfiguration( + new MediaItem.ClippingConfiguration.Builder() + .setStartPositionMs(430) + .setEndPositionMs(460) + .build()) + .build(); + + Composition composition = + new Composition.Builder( + EditedMediaItemSequence.withAudioAndVideoFrom( + ImmutableList.of( + new EditedMediaItem.Builder(clippedMediaItem) + .setDurationUs(MP4_SIMPLE_ASSET.videoDurationUs) + .setFrameRate(1) + .build()))) + .build(); + player = createTestHardwareBufferCompositionPlayerBuilder(frameProcessorFactory).build(); + + player.setComposition(composition); + player.prepare(); + player.play(); + advance(player).untilState(STATE_ENDED); + + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); + assertThat(frameProcessor).isNotNull(); + assertThat(getQueuedContentTimesUs(frameProcessor)).containsExactly(3766L); + } + @Test public void frameProcessor_setCompositionAfterSeek_rendersFrameAtSeekPositionBeforeStarted() throws PlaybackException, TimeoutException { @@ -1402,7 +1491,7 @@ public void frameProcessor_setCompositionAfterSeek_rendersFrameAtSeekPositionBef player.seekTo(/* positionMs= */ 500); advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); assertThat(frameProcessor.getQueuedContentTimesUs()) .containsExactly(ImmutableList.of(500_000L, 500_500L)); @@ -1440,7 +1529,7 @@ public void frameProcessor_playbackAt5xSpeedAt30Fps_dropsFramesAtRegularInterval player.play(); advance(player).untilState(STATE_ENDED); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); // Output should be a 2s long, 30fps video, because the frameRate is set on the EditedMediaItem. assertThat(frameProcessor.getQueuedContentTimesUs()) @@ -1477,7 +1566,7 @@ public void frameProcessor_playbackAt5xSpeedUnsetFrameRate_doesNotDropFrames() player.play(); advance(player).untilState(STATE_ENDED); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); // Output should be a 2s long, 30fps video, because the frameRate is set on the EditedMediaItem. assertThat(frameProcessor.getQueuedContentTimesUs()) @@ -1534,7 +1623,7 @@ public void frameProcessor_startPlaybackAfterZero_dropsFirstFrame() throws Excep play(player).untilState(STATE_ENDED); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); // The frame at 0L should be dropped because the start position is 1000L. @@ -1572,7 +1661,7 @@ public void frameProcessor_setCompositionWithNewPosition_rendersFrameAtSeekPosit player.seekTo(/* positionMs= */ 500); advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); assertThat(frameProcessor.getQueuedContentTimesUs()) .containsExactly(ImmutableList.of(500_000L, 500_500L)); @@ -1621,7 +1710,7 @@ public void frameProcessor_playback_image_endsWithExpectedNumberOfFrames() throw play(player).untilState(STATE_ENDED); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); assertThat(frameProcessor.getQueuedContentTimesUs()) .containsExactlyElementsIn(expectedTimestampsUs.build()) @@ -1654,7 +1743,7 @@ public void frameProcessor_playback_imageAndAudio_endsWithExpectedNumberOfFrames play(player).untilState(STATE_ENDED); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); assertThat(frameProcessor.getQueuedContentTimesUs()) .containsExactlyElementsIn(expectedTimestampsUs.build()) @@ -1686,7 +1775,7 @@ public void frameProcessor_playback_videoAndAudio_endsWithExpectedTimestamps() t play(player).untilState(STATE_ENDED); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); assertThat(frameProcessor.getQueuedContentTimesUs()) .containsExactlyElementsIn(expectedTimestamps) @@ -1719,9 +1808,8 @@ public void frameProcessor_playback_populatesRequiredMetadataFields() throws Exc player.prepare(); play(player).untilState(STATE_ENDED); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; - FakeFrameProcessor.FramesEvent framesEvent = - (FakeFrameProcessor.FramesEvent) frameProcessor.getQueuedEvents().get(0); + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); + FramesEvent framesEvent = (FramesEvent) frameProcessor.getQueuedEvents().get(0); Frame frame = framesEvent.frames.get(0).frame; ImmutableMap metadata = frame.getMetadata(); @@ -1737,6 +1825,289 @@ public void frameProcessor_playback_populatesRequiredMetadataFields() throws Exc .isEqualTo(composition.effects.videoEffects); } + // TODO: b/546521739 - Remove min sdk once early Surface transfer is supported below API 33. + @Config(minSdk = 33) + @Test + public void + frameProcessorWithAutomaticFrameOutput_videoVideoAndPlayToEndOfFirstItem_outputsAllFramesFromFirstAndSecondItems() + throws Exception { + ImmutableList assetInfos = + ImmutableList.of(EditedMediaItemAssetInfo.VIDEO, EditedMediaItemAssetInfo.VIDEO); + long playToPositionUs = + assetInfos.get(0).getEditedMediaItem().getPresentationDurationUs() - 200_000; + runFrameProcessorWithAutomaticFrameOutputTest( + assetInfos, + playToPositionUs, + /* expectedTimestampsAtPlayToPositionUs= */ getTimestamps(assetInfos, /* withEos= */ false), + /* expectedTimestampsAtEnd= */ getTimestamps(assetInfos, /* withEos= */ true)); + } + + @Test + public void + frameProcessorWithAutomaticFrameOutput_videoImageAndPlayToEndOfFirstItem_outputsAllFramesFromFirstAndSecondItems() + throws Exception { + ImmutableList assetInfos = + ImmutableList.of(EditedMediaItemAssetInfo.VIDEO, EditedMediaItemAssetInfo.IMAGE); + long playToPositionUs = + assetInfos.get(0).getEditedMediaItem().getPresentationDurationUs() - 200_000; + runFrameProcessorWithAutomaticFrameOutputTest( + assetInfos, + playToPositionUs, + /* expectedTimestampsAtPlayToPositionUs= */ getTimestamps(assetInfos, /* withEos= */ false), + /* expectedTimestampsAtEnd= */ getTimestamps(assetInfos, /* withEos= */ true)); + } + + @Test + public void + frameProcessorWithAutomaticFrameOutput_imageVideoAndPlayToEndOfFirstItem_outputsAllFramesFromFirstAndSecondItems() + throws Exception { + ImmutableList assetInfos = + ImmutableList.of(EditedMediaItemAssetInfo.IMAGE, EditedMediaItemAssetInfo.VIDEO); + long playToPositionUs = + assetInfos.get(0).getEditedMediaItem().getPresentationDurationUs() - 200_000; + runFrameProcessorWithAutomaticFrameOutputTest( + assetInfos, + playToPositionUs, + /* expectedTimestampsAtPlayToPositionUs= */ getTimestamps(assetInfos, /* withEos= */ false), + /* expectedTimestampsAtEnd= */ getTimestamps(assetInfos, /* withEos= */ true)); + } + + @Test + public void + frameProcessorWithAutomaticFrameOutput_imageImageAndPlayToEndOfFirstItem_outputsAllFramesFromFirstAndSecondItems() + throws Exception { + ImmutableList assetInfos = + ImmutableList.of(EditedMediaItemAssetInfo.IMAGE, EditedMediaItemAssetInfo.IMAGE); + long playToPositionUs = + assetInfos.get(0).getEditedMediaItem().getPresentationDurationUs() - 200_000; + runFrameProcessorWithAutomaticFrameOutputTest( + assetInfos, + playToPositionUs, + /* expectedTimestampsAtPlayToPositionUs= */ getTimestamps(assetInfos, /* withEos= */ false), + /* expectedTimestampsAtEnd= */ getTimestamps(assetInfos, /* withEos= */ true)); + } + + // TODO: b/546521739 - Remove min sdk once early Surface transfer is supported below API 33. + @Config(minSdk = 33) + @Test + public void + frameProcessorWithAutomaticFrameOutput_videoVideoVideoAndPlayToEndOfFirstItem_outputsAllFramesFromFirstAndSecondItems() + throws Exception { + ImmutableList assetInfos = + ImmutableList.of( + EditedMediaItemAssetInfo.VIDEO, + EditedMediaItemAssetInfo.VIDEO, + EditedMediaItemAssetInfo.VIDEO); + long playToPositionUs = + assetInfos.get(0).getEditedMediaItem().getPresentationDurationUs() - 200_000; + runFrameProcessorWithAutomaticFrameOutputTest( + assetInfos, + playToPositionUs, + /* expectedTimestampsAtPlayToPositionUs= */ getTimestamps( + assetInfos.subList(0, 2), /* withEos= */ false), + /* expectedTimestampsAtEnd= */ getTimestamps(assetInfos, /* withEos= */ true)); + } + + // TODO: b/546521739 - Remove min sdk once early Surface transfer is supported below API 33. + @Config(minSdk = 33) + @Test + public void + frameProcessorWithAutomaticFrameOutput_videoVideoImageAndPlayToEndOfFirstItem_outputsAllFramesFromFirstAndSecondItems() + throws Exception { + ImmutableList assetInfos = + ImmutableList.of( + EditedMediaItemAssetInfo.VIDEO, + EditedMediaItemAssetInfo.VIDEO, + EditedMediaItemAssetInfo.IMAGE); + long playToPositionUs = + assetInfos.get(0).getEditedMediaItem().getPresentationDurationUs() - 200_000; + runFrameProcessorWithAutomaticFrameOutputTest( + assetInfos, + playToPositionUs, + /* expectedTimestampsAtPlayToPositionUs= */ getTimestamps( + assetInfos.subList(0, 2), /* withEos= */ false), + /* expectedTimestampsAtEnd= */ getTimestamps(assetInfos, /* withEos= */ true)); + } + + @Test + public void + frameProcessorWithAutomaticFrameOutput_videoImageVideoAndPlayToEndOfFirstItem_outputsAllFramesFromFirstAndSecondItems() + throws Exception { + ImmutableList assetInfos = + ImmutableList.of( + EditedMediaItemAssetInfo.VIDEO, + EditedMediaItemAssetInfo.IMAGE, + EditedMediaItemAssetInfo.VIDEO); + long playToPositionUs = + assetInfos.get(0).getEditedMediaItem().getPresentationDurationUs() - 200_000; + runFrameProcessorWithAutomaticFrameOutputTest( + assetInfos, + playToPositionUs, + /* expectedTimestampsAtPlayToPositionUs= */ getTimestamps( + assetInfos.subList(0, 2), /* withEos= */ false), + /* expectedTimestampsAtEnd= */ getTimestamps(assetInfos, /* withEos= */ true)); + } + + @Test + public void + frameProcessorWithAutomaticFrameOutput_videoImageImageAndPlayToEndOfFirstItem_outputsAllFramesFromFirstAndSecondItems() + throws Exception { + ImmutableList assetInfos = + ImmutableList.of( + EditedMediaItemAssetInfo.VIDEO, + EditedMediaItemAssetInfo.IMAGE, + EditedMediaItemAssetInfo.IMAGE); + long playToPositionUs = + assetInfos.get(0).getEditedMediaItem().getPresentationDurationUs() - 200_000; + runFrameProcessorWithAutomaticFrameOutputTest( + assetInfos, + playToPositionUs, + /* expectedTimestampsAtPlayToPositionUs= */ getTimestamps( + assetInfos.subList(0, 2), /* withEos= */ false), + /* expectedTimestampsAtEnd= */ getTimestamps(assetInfos, /* withEos= */ true)); + } + + @Test + public void + frameProcessorWithAutomaticFrameOutput_imageVideoVideoAndPlayToEndOfFirstItem_outputsAllFramesFromFirstAndSecondItems() + throws Exception { + ImmutableList assetInfos = + ImmutableList.of( + EditedMediaItemAssetInfo.IMAGE, + EditedMediaItemAssetInfo.VIDEO, + EditedMediaItemAssetInfo.VIDEO); + long playToPositionUs = + assetInfos.get(0).getEditedMediaItem().getPresentationDurationUs() - 200_000; + runFrameProcessorWithAutomaticFrameOutputTest( + assetInfos, + playToPositionUs, + /* expectedTimestampsAtPlayToPositionUs= */ getTimestamps( + assetInfos.subList(0, 2), /* withEos= */ false), + /* expectedTimestampsAtEnd= */ getTimestamps(assetInfos, /* withEos= */ true)); + } + + @Test + public void + frameProcessorWithAutomaticFrameOutput_imageVideoImageAndPlayToEndOfFirstItem_outputsAllFramesFromFirstAndSecondItems() + throws Exception { + ImmutableList assetInfos = + ImmutableList.of( + EditedMediaItemAssetInfo.IMAGE, + EditedMediaItemAssetInfo.VIDEO, + EditedMediaItemAssetInfo.IMAGE); + long playToPositionUs = + assetInfos.get(0).getEditedMediaItem().getPresentationDurationUs() - 200_000; + runFrameProcessorWithAutomaticFrameOutputTest( + assetInfos, + playToPositionUs, + /* expectedTimestampsAtPlayToPositionUs= */ getTimestamps( + assetInfos.subList(0, 2), /* withEos= */ false), + /* expectedTimestampsAtEnd= */ getTimestamps(assetInfos, /* withEos= */ true)); + } + + @Test + public void + frameProcessorWithAutomaticFrameOutput_imageImageVideoAndPlayToEndOfFirstItem_outputsAllFramesFromFirstAndSecondItems() + throws Exception { + ImmutableList assetInfos = + ImmutableList.of( + EditedMediaItemAssetInfo.IMAGE, + EditedMediaItemAssetInfo.IMAGE, + EditedMediaItemAssetInfo.VIDEO); + long playToPositionUs = + assetInfos.get(0).getEditedMediaItem().getPresentationDurationUs() - 200_000; + runFrameProcessorWithAutomaticFrameOutputTest( + assetInfos, + playToPositionUs, + /* expectedTimestampsAtPlayToPositionUs= */ getTimestamps( + assetInfos.subList(0, 2), /* withEos= */ false), + /* expectedTimestampsAtEnd= */ getTimestamps(assetInfos, /* withEos= */ true)); + } + + @Test + public void + frameProcessorWithAutomaticFrameOutput_imageImageImageAndPlayToEndOfFirstItem_outputsAllFramesFromFirstAndSecondItems() + throws Exception { + ImmutableList assetInfos = + ImmutableList.of( + EditedMediaItemAssetInfo.IMAGE, + EditedMediaItemAssetInfo.IMAGE, + EditedMediaItemAssetInfo.IMAGE); + long playToPositionUs = + assetInfos.get(0).getEditedMediaItem().getPresentationDurationUs() - 200_000; + runFrameProcessorWithAutomaticFrameOutputTest( + assetInfos, + playToPositionUs, + /* expectedTimestampsAtPlayToPositionUs= */ getTimestamps( + assetInfos.subList(0, 2), /* withEos= */ false), + /* expectedTimestampsAtEnd= */ getTimestamps(assetInfos, /* withEos= */ true)); + } + + @Test + public void frameProcessor_videoVideoImage_seekBackToFirstVideo_playsToEnd() throws Exception { + Composition composition = + new Composition.Builder( + EditedMediaItemSequence.withAudioAndVideoFrom( + ImmutableList.of( + EditedMediaItemAssetInfo.VIDEO.getEditedMediaItem(), + EditedMediaItemAssetInfo.VIDEO.getEditedMediaItem(), + EditedMediaItemAssetInfo.IMAGE.getEditedMediaItem()))) + .build(); + Flags.enableFlag(Flags.FLAG_PER_STREAM_MEDIA_PROGRESSION); + player = createTestHardwareBufferCompositionPlayerBuilder(frameProcessorFactory).build(); + + player.setComposition(composition); + player.prepare(); + play(player).untilState(STATE_ENDED); + + player.setScrubbingModeEnabled(true); + player.seekTo(/* positionMs= */ 544); + player.setScrubbingModeEnabled(false); + + advance(player).untilState(STATE_ENDED); + + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); + assertThat(frameProcessor).isNotNull(); + + // Construct expected timestamps + ImmutableList.Builder> expectedTimestampsBuilder = ImmutableList.builder(); + // First play: Video 1 (full), Video 2 (full), Image 1 (full) + ImmutableList firstPlayAssets = + ImmutableList.of( + EditedMediaItemAssetInfo.VIDEO, + EditedMediaItemAssetInfo.VIDEO, + EditedMediaItemAssetInfo.IMAGE); + expectedTimestampsBuilder.addAll( + SequenceAssetInfo.getExpectedVideoTimestampsUs(firstPlayAssets, /* offsetUs= */ 0).stream() + .map(ImmutableList::of) + .collect(toImmutableList())); + expectedTimestampsBuilder.add(ImmutableList.of(C.TIME_UNSET)); // EOS from first play + + // Second play: Video 1 (partial, >= 544.5ms), Video 2 (full), Image 1 (full) + // Video 1 partial: + for (long t : EditedMediaItemAssetInfo.VIDEO.videoTimestampsUs) { + if (t >= 544_500L) { + expectedTimestampsBuilder.add(ImmutableList.of(t)); + } + } + // Video 2 and Image 1: + ImmutableList postSeekRemainingAssets = + ImmutableList.of(EditedMediaItemAssetInfo.VIDEO, EditedMediaItemAssetInfo.IMAGE); + long video1DurationUs = + EditedMediaItemAssetInfo.VIDEO.getEditedMediaItem().getPresentationDurationUs(); + expectedTimestampsBuilder.addAll( + SequenceAssetInfo.getExpectedVideoTimestampsUs( + postSeekRemainingAssets, /* offsetUs= */ video1DurationUs) + .stream() + .map(ImmutableList::of) + .collect(toImmutableList())); + expectedTimestampsBuilder.add(ImmutableList.of(C.TIME_UNSET)); // EOS from second play + + assertThat(frameProcessor.getQueuedContentTimesUs()) + .containsExactlyElementsIn(expectedTimestampsBuilder.build()) + .inOrder(); + } + @Test public void frameProcessor_imagePlayback_seekToMiddle_completesWithExpectedNumberOfFrames() throws Exception { @@ -1753,7 +2124,7 @@ public void frameProcessor_imagePlayback_seekToMiddle_completesWithExpectedNumbe player.seekTo(/* positionMs= */ 500); play(player).untilState(STATE_ENDED); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); // 1 initial frame + 15 frames from 500ms + 1 EOS. assertThat(frameProcessor.getQueuedEvents()).hasSize(1 + 15 + 1); @@ -1772,7 +2143,7 @@ public void frameProcessor_oneImageSequence_seekForwardsAndBackwards_outputsCorr advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); // Seek forwards @@ -1828,7 +2199,7 @@ public void frameProcessor_twoImageSequences_seekForwardsAndBackwards_outputsCor advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); // Seek forwards @@ -1884,7 +2255,7 @@ public void frameProcessor_oneImageSequence_rapidSeeking_debouncesSeeks() throws advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); // Seek forwards @@ -1934,7 +2305,7 @@ public void frameProcessor_twoImageSequences_rapidSeeking_debouncesSeeks() throw advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); // Seek forwards @@ -1985,7 +2356,7 @@ public void frameProcessor_oneImageSequence_scrubbing_debouncesSeeks() throws Ex advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); // Seek forwards @@ -2034,7 +2405,7 @@ public void frameProcessor_twoImageSequences_scrubbing_debouncesSeeks() throws E advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); // Seek forwards @@ -2084,7 +2455,7 @@ public void frameProcessor_oneImageSequence_scrubThenWait_debouncesSeeks() throw advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); // Seek forwards @@ -2135,7 +2506,7 @@ public void frameProcessor_twoImageSequences_scrubThenWait_debouncesSeeks() thro advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); // Seek forwards @@ -2187,7 +2558,7 @@ public void frameProcessor_oneImageSequence_seekThenPlay_outputsPacketAndEnds() advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); player.setScrubbingModeEnabled(true); @@ -2221,7 +2592,7 @@ public void frameProcessor_twoImageSequences_seekThenPlay_outputsPacketAndEnds() advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); player.setScrubbingModeEnabled(true); @@ -2258,7 +2629,7 @@ public void frameProcessor_oneVideoSequence_seekForwardsAndBackwards_outputsCorr advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); // Seek forwards @@ -2310,7 +2681,7 @@ public void frameProcessor_twoVideoSequences_seekForwardsAndBackwards_outputsCor advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); // Seek forwards @@ -2357,7 +2728,7 @@ public void frameProcessor_oneVideoSequence_scrubForwardsAndBackwards_debouncesF advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); // Seek forwards @@ -2408,7 +2779,7 @@ public void frameProcessor_twoVideoSequences_scrubForwardsAndBackwards_debounces advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); // Seek forwards @@ -2459,7 +2830,7 @@ public void frameProcessor_oneVideoSequence_seekThenPlay_outputsPacketAndEnds() player.setScrubbingModeEnabled(false); advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); player.play(); @@ -2505,7 +2876,7 @@ public void frameProcessor_twoVideoSequences_seekThenPlay_outputsPacketAndEnds() player.setScrubbingModeEnabled(false); advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); player.play(); @@ -2527,7 +2898,7 @@ public void frameProcessor_twoVideoSequences_seekThenPlay_outputsPacketAndEnds() @Test public void - frameProcessor_twoVideoSequencesMismatchedFrameRates_seek_decodesExtraSecondaryFrames() + frameProcessor_twoVideoSequencesMismatchedFrameRates_seek_outputsClosestSecondaryFrame() throws Exception { Composition composition = @@ -2549,11 +2920,11 @@ public void frameProcessor_twoVideoSequences_seekThenPlay_outputsPacketAndEnds() player.prepare(); advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); - // Seek to 466ms, primary will resolve to 480_000, secondary will resolve to 466_667, then - // decode an extra frame and output 533_333. + // Seek to 466ms, primary will resolve to 480_000, secondary will output the closer frame + // 466_666. player.setScrubbingModeEnabled(true); player.seekTo(466); player.setScrubbingModeEnabled(false); @@ -2561,10 +2932,10 @@ public void frameProcessor_twoVideoSequences_seekThenPlay_outputsPacketAndEnds() // Because extra frames need to be decoded, advancing until the player is ready is not enough, // we need to wait until the FrameProcessor receives the new frames. advance(player).untilState(STATE_READY); - runMainLooperUntilContentTimesUs(frameProcessor, ImmutableList.of(480_000L, 533_333L)); + runMainLooperUntilContentTimesUs(frameProcessor, ImmutableList.of(480_000L, 466_666L)); assertThat(frameProcessor.getQueuedContentTimesUs()) - .containsExactly(ImmutableList.of(0L, 0L), ImmutableList.of(480_000L, 533_333L)) + .containsExactly(ImmutableList.of(0L, 0L), ImmutableList.of(480_000L, 466_666L)) .inOrder(); } @@ -2640,7 +3011,7 @@ public void experimentalRedrawLastFrame_onFirstFramePaused_queuesFrameAgain() th player.prepare(); advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); // First frame is rendered on prepare (playback is paused). ImmutableList> queuedContentTimes = frameProcessor.getQueuedContentTimesUs(); @@ -2670,7 +3041,7 @@ public void experimentalRedrawLastFrame_afterPlayEnds_queuesLastFrame() throws E player.prepare(); play(player).untilState(STATE_ENDED); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); assertThat(frameProcessor).isNotNull(); int videoFrameCount = MP4_SIMPLE_ASSET.videoFrameCount; @@ -2702,7 +3073,7 @@ public void experimentalRedrawLastFrame_afterSeek_queuesLastFrame() throws Excep player.prepare(); advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); player.seekTo(500); advance(player).untilState(STATE_READY); @@ -2735,7 +3106,7 @@ public void experimentalRedrawLastFrame_whenPlaying_isIgnored() throws Exception advance(player).untilState(STATE_READY); player.play(); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); runMainLooperUntil(() -> frameProcessor.getQueuedContentTimesUs().size() >= 3); player.experimentalRedrawLastFrame(); @@ -2777,7 +3148,7 @@ public void experimentalRedrawLastFrame_whenPlaying_isIgnored() throws Exception player.prepare(); advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = frameProcessorFactory.createdProcessor; + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); ImmutableList> queuedContentTimes = frameProcessor.getQueuedContentTimesUs(); assertThat(queuedContentTimes).hasSize(1); @@ -2803,7 +3174,10 @@ public void experimentalRedrawLastFrame_whenPlaying_isIgnored() throws Exception public void seek_doesNotCloseInFlightHardwareBuffer() throws Exception { FakeFrameProcessor.Factory fakeFrameProcessorFactory = new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ false); - player = createTestHardwareBufferCompositionPlayerBuilder(fakeFrameProcessorFactory).build(); + CapturingFrameProcessor.Factory capturingFrameProcessorFactory = + new CapturingFrameProcessor.Factory(fakeFrameProcessorFactory); + player = + createTestHardwareBufferCompositionPlayerBuilder(capturingFrameProcessorFactory).build(); player.setComposition( new Composition.Builder( @@ -2818,9 +3192,9 @@ public void seek_doesNotCloseInFlightHardwareBuffer() throws Exception { // Advance to READY. The first frame should be queued but not completed. advance(player).untilState(STATE_READY); - FakeFrameProcessor frameProcessor = fakeFrameProcessorFactory.createdProcessor; - ImmutableList events = frameProcessor.getQueuedEvents(); - FakeFrameProcessor.FramesEvent framesEvent = (FakeFrameProcessor.FramesEvent) events.get(0); + CapturingFrameProcessor frameProcessor = capturingFrameProcessorFactory.getCreatedProcessor(); + ImmutableList events = frameProcessor.getQueuedEvents(); + FramesEvent framesEvent = (FramesEvent) events.get(0); AsyncFrame preSeekFrame = framesEvent.frames.get(0); assertThat(preSeekFrame.frame).isInstanceOf(HardwareBufferFrame.class); @@ -2834,16 +3208,294 @@ public void seek_doesNotCloseInFlightHardwareBuffer() throws Exception { // Advance player to handle seek. advance(player).untilState(STATE_READY); - AsyncFrame postSeekFrame = frameProcessor.lastFrames.get(0); + ImmutableList postSeekEvents = frameProcessor.getQueuedEvents(); + AsyncFrame postSeekFrame = + ((FramesEvent) postSeekEvents.get(postSeekEvents.size() - 1)).frames.get(0); assertThat((int) postSeekFrame.frame.getMetadata().get(KEY_FRAME_DISCONTINUITY_NUMBER)) .isGreaterThan((int) preSeekFrame.frame.getMetadata().get(KEY_FRAME_DISCONTINUITY_NUMBER)); // Verify that the pre-seek buffer is still NOT closed because we haven't released it. assertThat(preSeekBuffer.isClosed()).isFalse(); - // TODO: b/518679527 - Implemnet FakeFrameProcessor.close and close the underlying + // TODO: b/518679527 - Implement FakeFrameProcessor.close and close the underlying // HardwareBuffer instances. } + @Test + public void setComposition_doesNotCloseInFlightHardwareBufferOfPreviousComposition() + throws Exception { + FakeFrameProcessor.Factory fakeFrameProcessorFactory = + new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ false); + CapturingFrameProcessor.Factory capturingFrameProcessorFactory = + new CapturingFrameProcessor.Factory(fakeFrameProcessorFactory); + player = + createTestHardwareBufferCompositionPlayerBuilder(capturingFrameProcessorFactory).build(); + + Composition firstComposition = + new Composition.Builder( + EditedMediaItemSequence.withAudioAndVideoFrom( + ImmutableList.of( + new EditedMediaItem.Builder(MediaItem.fromUri(MP4_SIMPLE_ASSET.uri)) + .setDurationUs(MP4_SIMPLE_ASSET.videoDurationUs) + .build()))) + .build(); + player.setComposition(firstComposition); + player.prepare(); + + // Advance to READY. The first frame should be queued but not completed. + advance(player).untilState(STATE_READY); + + CapturingFrameProcessor frameProcessor = capturingFrameProcessorFactory.getCreatedProcessor(); + ImmutableList events = frameProcessor.getQueuedEvents(); + FramesEvent firstFramesEvent = (FramesEvent) events.get(0); + AsyncFrame firstFrame = firstFramesEvent.frames.get(0); + HardwareBuffer firstBuffer = ((HardwareBufferFrame) firstFrame.frame).getHardwareBuffer(); + assertThat(firstBuffer.isClosed()).isFalse(); + + Composition firstCompositionFromMetadata = + (Composition) firstFrame.frame.getMetadata().get("KEY_COMPOSITION"); + assertThat(firstCompositionFromMetadata.toString()).isEqualTo(firstComposition.toString()); + + // Set a different composition. + Composition secondComposition = + new Composition.Builder( + EditedMediaItemSequence.withAudioAndVideoFrom( + ImmutableList.of( + new EditedMediaItem.Builder(MediaItem.fromUri(MP4_15FPS.uri)) + .setDurationUs(MP4_15FPS.videoDurationUs) + .build()))) + .build(); + player.setComposition(secondComposition); + + // Advance player to handle the new composition. + advance(player).untilState(STATE_READY); + + // Verify we got a frame from the second composition. + events = frameProcessor.getQueuedEvents(); + FramesEvent secondFramesEvent = (FramesEvent) events.getLast(); + AsyncFrame secondFrame = secondFramesEvent.frames.get(0); + Composition secondCompositionFromMetadata = + (Composition) secondFrame.frame.getMetadata().get("KEY_COMPOSITION"); + assertThat(secondCompositionFromMetadata.toString()).isEqualTo(secondComposition.toString()); + // Verify that the buffer from the first composition is still NOT closed. + assertThat(firstBuffer.isClosed()).isFalse(); + } + + @Test + public void playback_withAggregationFpsLowerThanIntrinsic_downsamplesFrames() throws Exception { + Composition composition = + TestUtil.buildComposition(ImmutableList.of(ImmutableList.of(MP4_SIMPLE_ASSET)), FPS_10); + + runPlaybackAndAssertFps(composition, FPS_10); + } + + @Test + public void playback_withAggregationIntervalLongerThanDuration_outputsSingleFrame() + throws Exception { + Composition composition = + TestUtil.buildComposition(ImmutableList.of(ImmutableList.of(MP4_SIMPLE_ASSET)), FPS_HALF); + + // The aggregation interval (2 seconds) is longer than the composition duration (~1.09 + // seconds). We expect only the first frame (at 0 us) to be output, as the playback ends before + // the next interval boundary is reached. + runPlaybackAndAssert(composition, FPS_HALF, /* expectSingleFrame= */ true); + } + + @Test + public void playback_withAggregationFpsLowerThan1Fps_downsamplesFrames() throws Exception { + Composition composition = + TestUtil.buildComposition( + ImmutableList.of( + ImmutableList.of(MP4_SIMPLE_ASSET, MP4_SIMPLE_ASSET, MP4_SIMPLE_ASSET)), + FPS_HALF); + + runPlaybackAndAssertFps(composition, FPS_HALF); + } + + @Test + public void playback_withAggregationFpsHigherThanIntrinsic_upsamplesFrames() throws Exception { + Composition composition = + TestUtil.buildComposition(ImmutableList.of(ImmutableList.of(MP4_SIMPLE_ASSET)), FPS_60); + + runPlaybackAndAssertFps(composition, FPS_60); + } + + @Test + public void playback_withAggregationFpsMatchingIntrinsic_matchesIntrinsicFps() throws Exception { + Composition composition = + TestUtil.buildComposition(ImmutableList.of(ImmutableList.of(MP4_SIMPLE_ASSET)), FPS_30); + + runPlaybackAndAssertFps(composition, FPS_30); + } + + @Test + public void playback_withAggregationFpsAndVaryingFpsInMultiSequence_outputsFramesAtTargetFps() + throws Exception { + Composition composition = + TestUtil.buildComposition( + ImmutableList.of( + ImmutableList.of(MP4_15FPS), // 15 FPS + ImmutableList.of(MP4_SIMPLE_ASSET), // 29.97 FPS + ImmutableList.of(MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_5S)), // 60 FPS + FPS_30); + + runPlaybackAndAssertFps(composition, FPS_30); + } + + @Test + public void playback_withAggregationFpsAndVaryingFpsInSingleSequence_outputsFramesAtTargetFps() + throws Exception { + Composition composition = + TestUtil.buildComposition( + ImmutableList.of( + ImmutableList.of( + MP4_15FPS, // 15 FPS + MP4_SIMPLE_ASSET, // 29.97 FPS + MP4_ASSET_WITH_INCREASING_TIMESTAMPS_320W_240H_5S)), // 60 FPS + FPS_30); + + runPlaybackAndAssertFps(composition, FPS_30); + } + + @Test + public void playback_withAggregationFpsHigherThanIntrinsicAndBFrame_outputsTargetFps() + throws Exception { + Composition composition = + TestUtil.buildComposition(ImmutableList.of(ImmutableList.of(MP4_ADVANCED_ASSET)), FPS_60); + + runPlaybackAndAssertFps(composition, FPS_60); + } + + @Test + public void playback_withLoopingSecondarySequence_wrapsItemIndexInFrameMetadata() + throws Exception { + EditedMediaItemSequence primarySequence = + EditedMediaItemSequence.withVideoFrom( + ImmutableList.of( + new EditedMediaItem.Builder(MediaItem.fromUri(MP4_SIMPLE_ASSET.uri)) + .setDurationUs(1_000_000L) + .build())); + EditedMediaItemSequence loopingSecondarySequence = + new EditedMediaItemSequence.Builder(ImmutableSet.of(C.TRACK_TYPE_VIDEO)) + .addItem( + new EditedMediaItem.Builder(MediaItem.fromUri(JPG_ASSET.uri)) + .setDurationUs(200_000L) + .setFrameRate(30) + .build()) + .setIsLooping(true) + .build(); + Composition composition = + new Composition.Builder(primarySequence, loopingSecondarySequence).build(); + + player = setupAndPrepareHardwareBufferPlayer(composition, frameProcessorFactory); + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); + + player.play(); + advance(player).untilState(STATE_ENDED); + + int secondarySequenceFrameCount = 0; + for (CapturingFrameProcessor.Event event : frameProcessor.getQueuedEvents()) { + if (event instanceof FramesEvent) { + for (AsyncFrame asyncFrame : ((FramesEvent) event).frames) { + ImmutableMap metadata = asyncFrame.frame.getMetadata(); + Integer sequenceIndex = + (Integer) metadata.get(DefaultGlFrameProcessor.KEY_COMPOSITION_SEQUENCE_INDEX); + if (sequenceIndex != null && sequenceIndex == 1) { + secondarySequenceFrameCount++; + assertThat(metadata.get(Composition.KEY_COMPOSITION_ITEM_INDEX)).isEqualTo(0); + } + } + } + } + assertThat(secondarySequenceFrameCount).isGreaterThan(10); + } + + /** + * Plays {@link CompositionPlayer} to {@code playToPositionUs}, verifies the expected timestamps, + * then plays to the end. + */ + private void runFrameProcessorWithAutomaticFrameOutputTest( + ImmutableList assetInfos, + long playToPositionUs, + ImmutableList> expectedTimestampsAtPlayToPositionUs, + ImmutableList> expectedTimestampsAtEnd) + throws Exception { + Composition composition = + new Composition.Builder( + EditedMediaItemSequence.withAudioAndVideoFrom( + assetInfos.stream() + .map(asset -> asset.getEditedMediaItem()) + .collect(toImmutableList()))) + .build(); + // Allow manually setting the audio position to control which frames are output from the + // release control. + MaxPositionAudioSink customAudioSink = + new MaxPositionAudioSink(new DefaultAudioSink.Builder(getApplicationContext()).build()); + customAudioSink.maxPositionUs.set(0); + // Control the position of the clock to avoid deadlocks when the FakeClock increments time + // before frames are output. + ManualPositionClock clock = new ManualPositionClock(new FakeClock(/* isAutoAdvancing= */ true)); + // Force release every frame. + AutomaticOutputFrameTimingEvaluator frameTimingEvaluator = + new AutomaticOutputFrameTimingEvaluator(); + Flags.enableFlag(Flags.FLAG_PER_STREAM_MEDIA_PROGRESSION); + player = + createTestHardwareBufferCompositionPlayerBuilder(frameProcessorFactory) + .setAudioSink(customAudioSink) + .setClock(clock) + .setFrameTimingEvaluator(frameTimingEvaluator) + .build(); + + player.setComposition(composition); + player.prepare(); + advance(player).untilState(STATE_READY); + + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); + assertThat(frameProcessor).isNotNull(); + advance(player) + .untilBackgroundThreadCondition(() -> !frameProcessor.getQueuedContentTimesUs().isEmpty()); + + assertThat(frameProcessor.getQueuedContentTimesUs()).containsExactly(ImmutableList.of(0L)); + + assertThat(player.getCurrentPosition()).isEqualTo(0); + + // Start playback and wait until the player is actually playing and pending commands are handled + // so the internal media clock is deterministically started. + player.play(); + advance(player).untilBackgroundThreadCondition(player::isPlaying); + advance(player).untilPendingCommandsAreFullyHandled(); + + // Advance position to playToPositionUs. + customAudioSink.maxPositionUs.set(playToPositionUs); + clock.advanceElapsedRealtime(playToPositionUs / 1000); + + advance(player) + .untilBackgroundThreadCondition( + () -> + frameProcessor.getQueuedContentTimesUs().size() + >= expectedTimestampsAtPlayToPositionUs.size()); + + // All frames should be rendered because FrameTimingEvaluator will ignore audio position and + // output all available decoded frames. + assertThat(frameProcessor.getQueuedContentTimesUs()) + .containsAtLeastElementsIn(expectedTimestampsAtPlayToPositionUs) + .inOrder(); + + // Allow the player to play to the end. + customAudioSink.maxPositionUs.set(C.TIME_UNSET); + clock.setManualMode(false); + advance(player).untilPendingCommandsAreFullyHandled(); + advance(player) + .untilBackgroundThreadCondition( + () -> + frameProcessor.getQueuedContentTimesUs().size() >= expectedTimestampsAtEnd.size()); + advance(player).untilState(STATE_ENDED); + + // There should be no more frames output when the player ends. + assertThat(frameProcessor.getQueuedContentTimesUs()) + .containsExactlyElementsIn(expectedTimestampsAtEnd) + .inOrder(); + } + private static EditedMediaItem getImageItem() { return new EditedMediaItem.Builder( new MediaItem.Builder() @@ -2855,6 +3507,29 @@ private static EditedMediaItem getImageItem() { .build(); } + private void runPlaybackAndAssertFps(Composition composition, Rational expectedFps) + throws Exception { + runPlaybackAndAssert(composition, expectedFps, /* expectSingleFrame= */ false); + } + + private void runPlaybackAndAssert( + Composition composition, Rational expectedFps, boolean expectSingleFrame) throws Exception { + player = setupAndPrepareHardwareBufferPlayer(composition, frameProcessorFactory); + CapturingFrameProcessor frameProcessor = frameProcessorFactory.getCreatedProcessor(); + + player.play(); + advance(player).untilState(STATE_ENDED); + + ImmutableList queuedTimestampsUs = getQueuedContentTimesUs(frameProcessor); + assertThat(queuedTimestampsUs).isNotEmpty(); + assertThat(queuedTimestampsUs.get(0)).isEqualTo(0L); + if (expectSingleFrame) { + assertThat(queuedTimestampsUs).hasSize(1); + } else { + assertTimestampsMatchFrameRate(queuedTimestampsUs, expectedFps); + } + } + private static Composition buildComposition() { // Use raw audio-only assets which can be played in robolectric tests. EditedMediaItem editedMediaItem1 = @@ -2876,11 +3551,20 @@ private static Composition buildComposition() { * frameProcessor}. */ private static void runMainLooperUntilContentTimesUs( - FakeFrameProcessor frameProcessor, ImmutableList timestamps) throws TimeoutException { + CapturingFrameProcessor frameProcessor, ImmutableList timestamps) + throws TimeoutException { runMainLooperUntil( () -> { - List lastFrames = frameProcessor.lastFrames; - if (lastFrames == null || lastFrames.size() != timestamps.size()) { + ImmutableList events = frameProcessor.getQueuedEvents(); + if (events.isEmpty()) { + return false; + } + CapturingFrameProcessor.Event lastEvent = events.get(events.size() - 1); + if (!(lastEvent instanceof FramesEvent)) { + return false; + } + ImmutableList lastFrames = ((FramesEvent) lastEvent).frames; + if (lastFrames.size() != timestamps.size()) { return false; } for (int i = 0; i < timestamps.size(); i++) { @@ -2892,6 +3576,21 @@ private static void runMainLooperUntilContentTimesUs( }); } + private static ImmutableList> getTimestamps( + ImmutableList assetInfos, boolean withEos) { + ImmutableList.Builder> contentTimesUsBuilder = + new ImmutableList.Builder>() + .addAll( + SequenceAssetInfo.getExpectedVideoTimestampsUs(assetInfos, /* offsetUs= */ 0) + .stream() + .map(ImmutableList::of) + .collect(ImmutableList.toImmutableList())); + if (withEos) { + contentTimesUsBuilder.add(ImmutableList.of(C.TIME_UNSET)); + } + return contentTimesUsBuilder.build(); + } + private static final class CustomLoadControl extends DefaultLoadControl { public boolean prepared; @@ -3041,4 +3740,162 @@ private void verifyThread() { } } } + + /** An {@link AudioSink} that only allows buffers up to a set position to be consumed. */ + private static class MaxPositionAudioSink extends ForwardingAudioSink { + + private final AudioSink sink; + private final AtomicLong maxPositionUs; + + MaxPositionAudioSink(AudioSink sink) { + super(sink); + this.sink = sink; + maxPositionUs = new AtomicLong(C.TIME_UNSET); + } + + @Override + public boolean handleBuffer( + ByteBuffer buffer, long presentationTimeUs, int encodedAccessUnitCount) + throws InitializationException, WriteException { + long maxPositionUs = this.maxPositionUs.get(); + if (maxPositionUs != C.TIME_UNSET && presentationTimeUs >= maxPositionUs) { + return false; + } + return sink.handleBuffer(buffer, presentationTimeUs, encodedAccessUnitCount); + } + } + + /** A {@link FrameTimingEvaluator} that always force releases frames. */ + private static final class AutomaticOutputFrameTimingEvaluator implements FrameTimingEvaluator { + @Override + public boolean shouldForceReleaseFrame(long earlyUs, long elapsedSinceLastReleaseUs) { + return true; + } + + @Override + public boolean shouldDropFrame(long earlyUs, long elapsedRealtimeUs, boolean isLastFrame) { + return false; + } + + @Override + public boolean shouldIgnoreFrame( + long earlyUs, + long positionUs, + long elapsedRealtimeUs, + boolean isLastFrame, + boolean treatDroppedBuffersAsSkipped) { + return false; + } + } + + // TODO: b/545016809 - Consider moving this functionality into FakeClock. + /** + * A {@link Clock} implementation that allows manual control of the elapsed realtime, while + * delegating other clock operations to a {@link FakeClock}. + * + *

    It can operate in either manual mode or auto-advancing mode. + * + *

      + *
    • In manual mode, elapsed realtime only advances when explicitly requested via {@link + * #advanceElapsedRealtime(long)}. + *
    • In auto-advancing mode, elapsed realtime advances in sync with the underlying {@link + * FakeClock}. + *
    + */ + public static final class ManualPositionClock implements Clock { + private final FakeClock delegate; + private boolean isManualMode; + private long manualElapsedRealtimeMs; + private long delegateElapsedRealtimeAtSync; + + /** + * Creates a new {@code ManualPositionClock} in manual mode. + * + * @param delegate The underlying {@link FakeClock} to delegate to. + */ + public ManualPositionClock(FakeClock delegate) { + this(delegate, /* isManualMode= */ true); + } + + /** + * Creates a new {@code ManualPositionClock}. + * + * @param delegate The underlying {@link FakeClock} to delegate to. + * @param isManualMode Whether the clock should start in manual mode. + */ + public ManualPositionClock(FakeClock delegate, boolean isManualMode) { + this.delegate = delegate; + this.isManualMode = isManualMode; + this.manualElapsedRealtimeMs = delegate.elapsedRealtime(); + this.delegateElapsedRealtimeAtSync = delegate.elapsedRealtime(); + } + + @Override + public synchronized long elapsedRealtime() { + if (isManualMode) { + return manualElapsedRealtimeMs; + } + return manualElapsedRealtimeMs + (delegate.elapsedRealtime() - delegateElapsedRealtimeAtSync); + } + + @Override + public synchronized long nanoTime() { + return elapsedRealtime() * 1_000_000L; + } + + @Override + public long uptimeMillis() { + return delegate.uptimeMillis(); + } + + @Override + public long currentTimeMillis() { + return delegate.currentTimeMillis(); + } + + @Override + public HandlerWrapper createHandler(Looper looper, @Nullable Handler.Callback callback) { + return delegate.createHandler(looper, callback); + } + + @Override + public void onThreadBlocked() { + delegate.onThreadBlocked(); + } + + /** + * Advances the elapsed realtime by the specified duration. + * + *

    This also advances the underlying {@link FakeClock} by the same duration. + * + * @param timeDiffMs The duration to advance by, in milliseconds. + */ + public synchronized void advanceElapsedRealtime(long timeDiffMs) { + manualElapsedRealtimeMs += timeDiffMs; + delegate.advanceTime(timeDiffMs); + } + + /** + * Sets whether the clock should operate in manual mode. + * + *

    When switching from manual to auto-advancing mode, the current time of the delegate clock + * is recorded to sync future time advancement. + * + *

    When switching from auto-advancing to manual mode, the manual elapsed realtime is updated + * to the current elapsed realtime. + * + * @param isManualMode Whether to enable manual mode. + */ + public synchronized void setManualMode(boolean isManualMode) { + if (this.isManualMode == isManualMode) { + return; + } + if (!isManualMode) { + delegateElapsedRealtimeAtSync = delegate.elapsedRealtime(); + } else { + manualElapsedRealtimeMs = elapsedRealtime(); + } + this.isManualMode = isManualMode; + } + } } diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionTest.java new file mode 100644 index 00000000000..3be5b94cc72 --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionTest.java @@ -0,0 +1,117 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static com.google.common.truth.Truth.assertThat; + +import android.util.Rational; +import androidx.media3.common.MediaItem; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import org.json.JSONObject; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link Composition}. */ +@RunWith(AndroidJUnit4.class) +public class CompositionTest { + + @Test + public void builder_setsAllParameters() { + EditedMediaItem editedMediaItem = new EditedMediaItem.Builder(MediaItem.fromUri("uri")).build(); + VideoFrameAggregationParameters parameters = + new VideoFrameAggregationParameters.Builder().setFrameRate(new Rational(60, 1)).build(); + Effects effects = + new Effects( + /* audioProcessors= */ ImmutableList.of(), /* videoEffects= */ ImmutableList.of()); + Composition composition = + new Composition.Builder( + EditedMediaItemSequence.withVideoFrom(ImmutableList.of(editedMediaItem))) + .setTransmuxAudio(true) + .setTransmuxVideo(true) + .setHdrMode(Composition.HDR_MODE_KEEP_HDR) + .experimentalSetRetainHdrFromUltraHdrImage(true) + .setEffects(effects) + .setVideoFrameAggregationParameters(parameters) + .build(); + + assertThat(composition.transmuxAudio).isTrue(); + assertThat(composition.transmuxVideo).isTrue(); + assertThat(composition.hdrMode).isEqualTo(Composition.HDR_MODE_KEEP_HDR); + assertThat(composition.retainHdrFromUltraHdrImage).isTrue(); + assertThat(composition.effects).isEqualTo(effects); + assertThat(composition.videoFrameAggregationParameters).isEqualTo(parameters); + } + + @Test + public void buildUpon_preservesAllParameters() { + EditedMediaItem editedMediaItem = new EditedMediaItem.Builder(MediaItem.fromUri("uri")).build(); + VideoFrameAggregationParameters parameters = + new VideoFrameAggregationParameters.Builder().setFrameRate(new Rational(60, 1)).build(); + Effects effects = + new Effects( + /* audioProcessors= */ ImmutableList.of(), /* videoEffects= */ ImmutableList.of()); + Composition composition = + new Composition.Builder( + EditedMediaItemSequence.withVideoFrom(ImmutableList.of(editedMediaItem))) + .setTransmuxAudio(true) + .setTransmuxVideo(true) + .setHdrMode(Composition.HDR_MODE_KEEP_HDR) + .experimentalSetRetainHdrFromUltraHdrImage(true) + .setEffects(effects) + .setVideoFrameAggregationParameters(parameters) + .build(); + + Composition newComposition = composition.buildUpon().build(); + + assertThat(newComposition.transmuxAudio).isTrue(); + assertThat(newComposition.transmuxVideo).isTrue(); + assertThat(newComposition.hdrMode).isEqualTo(Composition.HDR_MODE_KEEP_HDR); + assertThat(newComposition.retainHdrFromUltraHdrImage).isTrue(); + assertThat(newComposition.effects).isEqualTo(effects); + assertThat(newComposition.videoFrameAggregationParameters).isEqualTo(parameters); + } + + @Test + public void toJsonObject_containsAllParameters() throws Exception { + EditedMediaItem editedMediaItem = new EditedMediaItem.Builder(MediaItem.fromUri("uri")).build(); + VideoFrameAggregationParameters parameters = + new VideoFrameAggregationParameters.Builder().setFrameRate(new Rational(60, 1)).build(); + Effects effects = + new Effects( + /* audioProcessors= */ ImmutableList.of(), /* videoEffects= */ ImmutableList.of()); + Composition composition = + new Composition.Builder( + EditedMediaItemSequence.withVideoFrom(ImmutableList.of(editedMediaItem))) + .setTransmuxAudio(true) + .setTransmuxVideo(true) + .setHdrMode(Composition.HDR_MODE_KEEP_HDR) + .experimentalSetRetainHdrFromUltraHdrImage(true) + .setEffects(effects) + .setVideoFrameAggregationParameters(parameters) + .build(); + + JSONObject jsonObject = composition.toJsonObject(); + + assertThat(jsonObject.getBoolean("transmuxAudio")).isTrue(); + assertThat(jsonObject.getBoolean("transmuxVideo")).isTrue(); + assertThat(jsonObject.getInt("hdrMode")).isEqualTo(Composition.HDR_MODE_KEEP_HDR); + assertThat(jsonObject.getBoolean("retainHdrFromUltraHdrImage")).isTrue(); + assertThat(jsonObject.has("effects")).isTrue(); + assertThat(jsonObject.getJSONObject("videoFrameAggregationParameters").getString("frameRate")) + .isEqualTo("60/1"); + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionVideoPacketReleaseControlTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionVideoPacketReleaseControlTest.java index 54e06b6b9cb..7f9e5ada1ad 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionVideoPacketReleaseControlTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/CompositionVideoPacketReleaseControlTest.java @@ -16,9 +16,9 @@ package androidx.media3.transformer; import static androidx.media3.common.util.Util.msToUs; -import static androidx.media3.effect.HardwareBufferFrame.END_OF_STREAM_FRAME; import static androidx.media3.exoplayer.video.VideoSink.RELEASE_FIRST_FRAME_WHEN_PREVIOUS_STREAM_PROCESSED; import static androidx.media3.exoplayer.video.VideoSink.RELEASE_FIRST_FRAME_WHEN_STARTED; +import static androidx.media3.transformer.TransformerUtil.END_OF_STREAM_ASYNC_FRAME; import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.truth.Truth.assertThat; import static com.google.common.util.concurrent.MoreExecutors.directExecutor; @@ -26,22 +26,24 @@ import android.content.Context; import android.hardware.HardwareBuffer; import androidx.annotation.Nullable; +import androidx.media3.common.C; import androidx.media3.common.Format; +import androidx.media3.common.VideoFrameProcessingException; import androidx.media3.common.video.AsyncFrame; import androidx.media3.common.video.DefaultHardwareBufferFrame; import androidx.media3.common.video.Frame; import androidx.media3.common.video.FrameProcessor; import androidx.media3.common.video.FrameWriter; import androidx.media3.common.video.SyncFenceWrapper; -import androidx.media3.effect.HardwareBufferFrame; import androidx.media3.exoplayer.ExoPlaybackException; import androidx.media3.exoplayer.video.VideoFrameReleaseControl; import androidx.media3.exoplayer.video.VideoFrameReleaseControl.FrameTimingEvaluator; +import androidx.media3.test.utils.CapturingFrameProcessor; +import androidx.media3.test.utils.CapturingFrameProcessor.EosEvent; +import androidx.media3.test.utils.CapturingFrameProcessor.Event; +import androidx.media3.test.utils.CapturingFrameProcessor.FramesEvent; import androidx.media3.test.utils.FakeClock; import androidx.media3.test.utils.FakeFrameProcessor; -import androidx.media3.test.utils.FakeFrameProcessor.EosEvent; -import androidx.media3.test.utils.FakeFrameProcessor.Event; -import androidx.media3.test.utils.FakeFrameProcessor.FramesEvent; import androidx.media3.transformer.CompositionVideoPacketReleaseControl.Listener; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; @@ -65,12 +67,12 @@ public class CompositionVideoPacketReleaseControlTest { private VideoFrameReleaseControl videoFrameReleaseControl; private FakeFrameTimingEvaluator fakeFrameTimingEvaluator; private FakeClock fakeClock; - private FakeFrameProcessor frameProcessor; + private CapturingFrameProcessor frameProcessor; private List releasedFrameTimestamps; private HardwareBuffer placeholderBuffer; // The first packet is required to be sent to the CompositionVideoPacketReleaseControl to // initialize the VideoFrameReleaseControl so subsequent behaviour can be tested. - private ImmutableList firstPacket; + private ImmutableList firstPacket; @Before public void setUp() { @@ -88,11 +90,29 @@ public void setUp() { fakeClock = new FakeClock(/* initialTimeMs= */ 0); videoFrameReleaseControl = new VideoFrameReleaseControl( - context, fakeFrameTimingEvaluator, /* allowedJoiningTimeMs= */ 0); + context, + fakeFrameTimingEvaluator, + /* allowedJoiningTimeMs= */ 0, + /* skipBuffersWithIdenticalReleaseTime= */ false); videoFrameReleaseControl.setClock(fakeClock); + FakeFrameProcessor.Factory fakeFrameProcessorFactory = + new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ false); frameProcessor = - new FakeFrameProcessor.Factory(/* shouldCompleteIncomingFrames= */ false) - .create(new NoOpFrameWriter()); + new CapturingFrameProcessor.Factory(fakeFrameProcessorFactory) + .create( + new NoOpFrameWriter(), + /* listenerExecutor= */ directExecutor(), + new FrameProcessor.Listener() { + @Override + public void onWakeup() {} + + @Override + public void onError(VideoFrameProcessingException e) {} + + @Override + public void onFrameProcessed( + Frame frame, @Nullable SyncFenceWrapper onCompleteFence) {} + }); compositionVideoPacketReleaseControl = new CompositionVideoPacketReleaseControl( videoFrameReleaseControl, @@ -108,6 +128,14 @@ public void onError(Exception e) { }); } + private ImmutableList getLastFrames() { + ImmutableList events = frameProcessor.getQueuedEvents(); + assertThat(events).isNotEmpty(); + Event lastEvent = events.get(events.size() - 1); + assertThat(lastEvent).isInstanceOf(FramesEvent.class); + return ((FramesEvent) lastEvent).frames; + } + @After public void tearDown() { if (placeholderBuffer != null) { @@ -118,7 +146,7 @@ public void tearDown() { @Test public void onRender_releaseActionDrop_releasesFrame() throws ExoPlaybackException { compositionVideoPacketReleaseControl.onStarted(); - ImmutableList packet = + ImmutableList packet = createPacket(/* presentationTimeUs= */ 49_999, /* sequencePresentationTimeUs= */ 49_999); compositionVideoPacketReleaseControl.queue(firstPacket); compositionVideoPacketReleaseControl.onRender( @@ -136,13 +164,13 @@ public void onRender_releaseActionDrop_releasesFrame() throws ExoPlaybackExcepti /* compositionTimeOutputStreamStartPositionUs= */ 0); assertOutputEvents(/* ignoreReleaseTime= */ true, toFramesEvent(firstPacket)); - assertThat(releasedFrameTimestamps).containsExactly(packet.get(0).presentationTimeUs); + assertThat(releasedFrameTimestamps).containsExactly(packet.get(0).frame.getContentTimeUs()); } @Test public void onRender_releaseActionTryAgainLater_keepsFrame() throws ExoPlaybackException { compositionVideoPacketReleaseControl.onStarted(); - ImmutableList packet = + ImmutableList packet = createPacket(/* presentationTimeUs= */ 200_000, /* sequencePresentationTimeUs= */ 200_000); compositionVideoPacketReleaseControl.queue(firstPacket); compositionVideoPacketReleaseControl.onRender( @@ -166,7 +194,7 @@ public void onRender_releaseActionTryAgainLater_keepsFrame() throws ExoPlaybackE public void onRender_releaseActionImmediate_forwardsFrameDownstream() throws ExoPlaybackException { compositionVideoPacketReleaseControl.onStarted(); - ImmutableList packet = + ImmutableList packet = createPacket(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); compositionVideoPacketReleaseControl.queue(firstPacket); compositionVideoPacketReleaseControl.onRender( @@ -191,9 +219,9 @@ public void onRender_releaseActionImmediate_forwardsFrameDownstream() public void onRender_releaseActionSchedule_forwardsFrameDownstreamWithReleaseTime() throws ExoPlaybackException { compositionVideoPacketReleaseControl.onStarted(); - ImmutableList packet = + ImmutableList packet = createPacket(/* presentationTimeUs= */ 120_000, /* sequencePresentationTimeUs= */ 120_000); - ImmutableList expectedPacket = + ImmutableList expectedPacket = updatePacketWithReleaseTime(packet, /* releaseTimeNs= */ 120_000_000); compositionVideoPacketReleaseControl.queue(firstPacket); compositionVideoPacketReleaseControl.onRender( @@ -215,7 +243,7 @@ public void onRender_releaseActionSchedule_forwardsFrameDownstreamWithReleaseTim DefaultHardwareBufferFrame firstFrame = (DefaultHardwareBufferFrame) firstEvent.frames.get(0).frame; long releaseTimeNs = (Long) firstFrame.getMetadata().get(Frame.KEY_DISPLAY_TIME_NS); - ImmutableList expectedFirstFrame = + ImmutableList expectedFirstFrame = updatePacketWithReleaseTime(firstPacket, releaseTimeNs); assertOutputEvents( /* ignoreReleaseTime= */ false, @@ -226,7 +254,7 @@ public void onRender_releaseActionSchedule_forwardsFrameDownstreamWithReleaseTim @Test public void onRender_forwardsFirstFrameImmediately() throws ExoPlaybackException { - ImmutableList packet = + ImmutableList packet = createPacket(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); compositionVideoPacketReleaseControl.queue(packet); @@ -241,9 +269,9 @@ public void onRender_forwardsFirstFrameImmediately() throws ExoPlaybackException @Test public void queue_backwardSeekWithoutFlush_doesNotReleaseHeldFrames() { - ImmutableList packet1 = + ImmutableList packet1 = createPacket(/* presentationTimeUs= */ 200, /* sequencePresentationTimeUs= */ 200); - ImmutableList packet2 = + ImmutableList packet2 = createPacket(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); compositionVideoPacketReleaseControl.queue(packet1); @@ -256,9 +284,9 @@ public void queue_backwardSeekWithoutFlush_doesNotReleaseHeldFrames() { @Test public void queueWhenStopped_afterFlush_forwardsFirstFrameAfterFlushImmediately() throws ExoPlaybackException { - ImmutableList packet1 = + ImmutableList packet1 = createPacket(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); - ImmutableList packet2 = + ImmutableList packet2 = createPacket(/* presentationTimeUs= */ 200_000, /* sequencePresentationTimeUs= */ 200_000); compositionVideoPacketReleaseControl.onStarted(); compositionVideoPacketReleaseControl.onStopped(); @@ -291,9 +319,9 @@ public void queueWhenStopped_afterFlush_forwardsFirstFrameAfterFlushImmediately( @Test public void queueWhenStopped_withoutFlush_doesNotForwardFirstFrame() throws ExoPlaybackException { - ImmutableList packet1 = + ImmutableList packet1 = createPacket(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); - ImmutableList packet2 = + ImmutableList packet2 = createPacket(/* presentationTimeUs= */ 200_000, /* sequencePresentationTimeUs= */ 200_000); compositionVideoPacketReleaseControl.onStarted(); compositionVideoPacketReleaseControl.onStopped(); @@ -318,11 +346,11 @@ public void queueWhenStopped_withoutFlush_doesNotForwardFirstFrame() throws ExoP @Test public void queue_lowerPresentationTimeButHigherSequenceTime_doesNotFlush() { // Simulate a frame from the end of the first media item. - ImmutableList packet1 = + ImmutableList packet1 = createPacket( /* presentationTimeUs= */ 2_000_000, /* sequencePresentationTimeUs= */ 2_000_000); // Simulate a frame from the start of the second media item. - ImmutableList packet2 = + ImmutableList packet2 = createPacket(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 2_033_333); compositionVideoPacketReleaseControl.queue(packet1); @@ -337,14 +365,14 @@ public void queue_lowerPresentationTimeButHigherSequenceTime_doesNotFlush() { public void queue_lowerPresentationTimeButHigherSequenceTime_forwardsFrameDownstream() throws ExoPlaybackException { // Simulate a frame from the end of the first media item. - ImmutableList packet1 = + ImmutableList packet1 = createPacket(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); // Simulate a frame from the start of the second media item. - ImmutableList packet2 = + ImmutableList packet2 = createPacket(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 220_000); - ImmutableList expectedPacket1 = + ImmutableList expectedPacket1 = updatePacketWithReleaseTime(packet1, /* releaseTimeNs= */ 120_000_000); - ImmutableList expectedPacket2 = + ImmutableList expectedPacket2 = updatePacketWithReleaseTime(packet2, /* releaseTimeNs= */ 220_000_000); compositionVideoPacketReleaseControl.onStarted(); compositionVideoPacketReleaseControl.queue(firstPacket); @@ -382,9 +410,9 @@ public void queue_lowerPresentationTimeButHigherSequenceTime_forwardsFrameDownst public void flushPrimarySequence_releasesHeldFramesAndResetsReleaseControl() throws ExoPlaybackException { compositionVideoPacketReleaseControl.onStarted(); - ImmutableList packet1 = + ImmutableList packet1 = createPacket(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); - ImmutableList packet2 = + ImmutableList packet2 = createPacket(/* presentationTimeUs= */ 200, /* sequencePresentationTimeUs= */ 200); compositionVideoPacketReleaseControl.queue(firstPacket); compositionVideoPacketReleaseControl.onRender( @@ -402,7 +430,8 @@ public void flushPrimarySequence_releasesHeldFramesAndResetsReleaseControl() // Only packet1 and packet2 (which were in queue) are released. // firstPacket (in-flight) is NOT released. assertThat(releasedFrameTimestamps) - .containsExactly(packet1.get(0).presentationTimeUs, packet2.get(0).presentationTimeUs) + .containsExactly( + packet1.get(0).frame.getContentTimeUs(), packet2.get(0).frame.getContentTimeUs()) .inOrder(); // Now complete the in-flight frame. @@ -413,18 +442,18 @@ public void flushPrimarySequence_releasesHeldFramesAndResetsReleaseControl() // Now it should be released. assertThat(releasedFrameTimestamps) .containsExactly( - packet1.get(0).presentationTimeUs, - packet2.get(0).presentationTimeUs, - firstPacket.get(0).presentationTimeUs) + packet1.get(0).frame.getContentTimeUs(), + packet2.get(0).frame.getContentTimeUs(), + firstPacket.get(0).frame.getContentTimeUs()) .inOrder(); } @Test public void flushSecondarySequence_isIgnored() throws ExoPlaybackException { compositionVideoPacketReleaseControl.onStarted(); - ImmutableList packet1 = + ImmutableList packet1 = createPacket(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); - ImmutableList packet2 = + ImmutableList packet2 = createPacket(/* presentationTimeUs= */ 200, /* sequencePresentationTimeUs= */ 200); compositionVideoPacketReleaseControl.queue(firstPacket); compositionVideoPacketReleaseControl.onRender( @@ -449,14 +478,14 @@ public void isEnded_initially_returnsFalse() { @Test public void queue_eosPacket_doesNotIsEndedWhenOnRenderNotCalled() { - compositionVideoPacketReleaseControl.queue(ImmutableList.of(END_OF_STREAM_FRAME)); + compositionVideoPacketReleaseControl.queue(ImmutableList.of(END_OF_STREAM_ASYNC_FRAME)); assertThat(compositionVideoPacketReleaseControl.isEnded()).isFalse(); } @Test public void onRender_eosPacket_setsIsEndedTrue() throws Exception { compositionVideoPacketReleaseControl.onStarted(); - compositionVideoPacketReleaseControl.queue(ImmutableList.of(END_OF_STREAM_FRAME)); + compositionVideoPacketReleaseControl.queue(ImmutableList.of(END_OF_STREAM_ASYNC_FRAME)); compositionVideoPacketReleaseControl.onRender( /* compositionTimePositionUs= */ 0, /* elapsedRealtimeUs= */ msToUs(fakeClock.elapsedRealtime()), @@ -470,7 +499,7 @@ public void onRender_eosPacket_setsIsEndedTrue() throws Exception { @Test public void onRender_eosPacketAfterFrames_setsIsEndedTrue() throws ExoPlaybackException { compositionVideoPacketReleaseControl.onStarted(); - ImmutableList packet = + ImmutableList packet = createPacket(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); compositionVideoPacketReleaseControl.queue(firstPacket); compositionVideoPacketReleaseControl.onRender( @@ -481,7 +510,7 @@ public void onRender_eosPacketAfterFrames_setsIsEndedTrue() throws ExoPlaybackEx assertOutputEvents(/* ignoreReleaseTime= */ true, toFramesEvent(firstPacket)); compositionVideoPacketReleaseControl.queue(packet); - compositionVideoPacketReleaseControl.queue(ImmutableList.of(END_OF_STREAM_FRAME)); + compositionVideoPacketReleaseControl.queue(ImmutableList.of(END_OF_STREAM_ASYNC_FRAME)); compositionVideoPacketReleaseControl.onRender( /* compositionTimePositionUs= */ 100_000, /* elapsedRealtimeUs= */ msToUs(fakeClock.elapsedRealtime()), @@ -499,7 +528,7 @@ public void onRender_eosPacketAfterFrames_setsIsEndedTrue() throws ExoPlaybackEx @Test public void onRender_eosPacketBeforeFrames_doesNotSetIsEndedTrue() throws ExoPlaybackException { compositionVideoPacketReleaseControl.onStarted(); - ImmutableList packet = + ImmutableList packet = createPacket(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); compositionVideoPacketReleaseControl.queue(firstPacket); compositionVideoPacketReleaseControl.onRender( @@ -509,7 +538,7 @@ public void onRender_eosPacketBeforeFrames_doesNotSetIsEndedTrue() throws ExoPla fakeClock.advanceTime(/* timeDiffMs= */ 100); assertOutputEvents(/* ignoreReleaseTime= */ true, toFramesEvent(firstPacket)); - compositionVideoPacketReleaseControl.queue(ImmutableList.of(END_OF_STREAM_FRAME)); + compositionVideoPacketReleaseControl.queue(ImmutableList.of(END_OF_STREAM_ASYNC_FRAME)); compositionVideoPacketReleaseControl.queue(packet); compositionVideoPacketReleaseControl.onRender( /* compositionTimePositionUs= */ 100_000, @@ -525,7 +554,7 @@ public void onRender_eosPacketBeforeFrames_doesNotSetIsEndedTrue() throws ExoPla @Test public void onStarted_afterEos_doesNotFlushIsEndedToFalse() throws ExoPlaybackException { compositionVideoPacketReleaseControl.onStarted(); - compositionVideoPacketReleaseControl.queue(ImmutableList.of(END_OF_STREAM_FRAME)); + compositionVideoPacketReleaseControl.queue(ImmutableList.of(END_OF_STREAM_ASYNC_FRAME)); compositionVideoPacketReleaseControl.onRender( /* compositionTimePositionUs= */ 0, /* elapsedRealtimeUs= */ msToUs(fakeClock.elapsedRealtime()), @@ -541,7 +570,7 @@ public void onStarted_afterEos_doesNotFlushIsEndedToFalse() throws ExoPlaybackEx @Test public void flushPrimarySequence_afterEos_resetsIsEndedToFalse() throws ExoPlaybackException { compositionVideoPacketReleaseControl.onStarted(); - compositionVideoPacketReleaseControl.queue(ImmutableList.of(END_OF_STREAM_FRAME)); + compositionVideoPacketReleaseControl.queue(ImmutableList.of(END_OF_STREAM_ASYNC_FRAME)); compositionVideoPacketReleaseControl.onRender( /* compositionTimePositionUs= */ 0, /* elapsedRealtimeUs= */ msToUs(fakeClock.elapsedRealtime()), @@ -558,7 +587,7 @@ public void flushPrimarySequence_afterEos_resetsIsEndedToFalse() throws ExoPlayb public void resetSecondarySequence_afterEos_doesNotFlushIsEndedToFalse() throws ExoPlaybackException { compositionVideoPacketReleaseControl.onStarted(); - compositionVideoPacketReleaseControl.queue(ImmutableList.of(END_OF_STREAM_FRAME)); + compositionVideoPacketReleaseControl.queue(ImmutableList.of(END_OF_STREAM_ASYNC_FRAME)); compositionVideoPacketReleaseControl.onRender( /* compositionTimePositionUs= */ 0, /* elapsedRealtimeUs= */ msToUs(fakeClock.elapsedRealtime()), @@ -574,7 +603,7 @@ public void resetSecondarySequence_afterEos_doesNotFlushIsEndedToFalse() @Test public void onRender_emptyQueueWhenEnded_isEndedRemainsTrue() throws ExoPlaybackException { compositionVideoPacketReleaseControl.onStarted(); - compositionVideoPacketReleaseControl.queue(ImmutableList.of(END_OF_STREAM_FRAME)); + compositionVideoPacketReleaseControl.queue(ImmutableList.of(END_OF_STREAM_ASYNC_FRAME)); compositionVideoPacketReleaseControl.onRender( /* compositionTimePositionUs= */ 0, /* elapsedRealtimeUs= */ msToUs(fakeClock.elapsedRealtime()), @@ -593,7 +622,7 @@ public void onRender_emptyQueueWhenEnded_isEndedRemainsTrue() throws ExoPlayback @Test public void onFrameProcessed_completionListenerCalled_releasesMatchingFrame() throws Exception { compositionVideoPacketReleaseControl.onStarted(); - ImmutableList packet = + ImmutableList packet = createPacket(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); compositionVideoPacketReleaseControl.queue(packet); compositionVideoPacketReleaseControl.onRender( @@ -601,8 +630,9 @@ public void onFrameProcessed_completionListenerCalled_releasesMatchingFrame() th /* elapsedRealtimeUs= */ msToUs(fakeClock.elapsedRealtime()), /* compositionTimeOutputStreamStartPositionUs= */ 0); - assertThat(frameProcessor.lastFrames).hasSize(1); - AsyncFrame asyncFrame = frameProcessor.lastFrames.get(0); + ImmutableList lastFrames = getLastFrames(); + assertThat(lastFrames).hasSize(1); + AsyncFrame asyncFrame = lastFrames.get(0); assertThat(releasedFrameTimestamps).isEmpty(); compositionVideoPacketReleaseControl.onFrameProcessed( @@ -620,9 +650,9 @@ public void onFrameProcessed_completionListenerCalled_releasesMatchingFrame() th public void onFrameProcessed_multipleFramesInFlight_releasesInOrderAndOutOfOrder() throws Exception { compositionVideoPacketReleaseControl.onStarted(); - ImmutableList packet1 = + ImmutableList packet1 = createPacket(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); - ImmutableList packet2 = + ImmutableList packet2 = createPacket(/* presentationTimeUs= */ 200_000, /* sequencePresentationTimeUs= */ 200_000); compositionVideoPacketReleaseControl.queue(packet1); @@ -640,16 +670,16 @@ public void onFrameProcessed_multipleFramesInFlight_releasesInOrderAndOutOfOrder assertThat(releasedFrameTimestamps).isEmpty(); - ImmutableList events = frameProcessor.getQueuedEvents(); + ImmutableList events = frameProcessor.getQueuedEvents(); assertThat(events).hasSize(2); - AsyncFrame asyncFrame2 = ((FakeFrameProcessor.FramesEvent) events.get(1)).frames.get(0); + AsyncFrame asyncFrame2 = ((FramesEvent) events.get(1)).frames.get(0); compositionVideoPacketReleaseControl.onFrameProcessed( asyncFrame2.frame, /* releaseFence= */ null); // 200_000 is still in cache, so it is not released. assertThat(releasedFrameTimestamps).isEmpty(); - AsyncFrame asyncFrame1 = ((FakeFrameProcessor.FramesEvent) events.get(0)).frames.get(0); + AsyncFrame asyncFrame1 = ((FramesEvent) events.get(0)).frames.get(0); compositionVideoPacketReleaseControl.onFrameProcessed( asyncFrame1.frame, /* releaseFence= */ null); @@ -664,7 +694,7 @@ public void onFrameProcessed_multipleFramesInFlight_releasesInOrderAndOutOfOrder @Test public void onFrameProcessed_nonMatchingFrame_ignoresFrame() throws Exception { compositionVideoPacketReleaseControl.onStarted(); - ImmutableList packet = + ImmutableList packet = createPacket(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); compositionVideoPacketReleaseControl.queue(packet); compositionVideoPacketReleaseControl.onRender( @@ -757,7 +787,7 @@ public void onRender_whenDropped_doesNotCallOnFrameReleasedIsFirstFrame() @Test public void onFrameProcessed_afterFlush_releasesFrame() throws Exception { compositionVideoPacketReleaseControl.onStarted(); - ImmutableList packet = + ImmutableList packet = createPacket(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); compositionVideoPacketReleaseControl.queue(packet); compositionVideoPacketReleaseControl.onRender( @@ -765,7 +795,7 @@ public void onFrameProcessed_afterFlush_releasesFrame() throws Exception { /* elapsedRealtimeUs= */ msToUs(fakeClock.elapsedRealtime()), /* compositionTimeOutputStreamStartPositionUs= */ 0); - AsyncFrame asyncFrame = frameProcessor.lastFrames.get(0); + AsyncFrame asyncFrame = getLastFrames().get(0); compositionVideoPacketReleaseControl.flush(/* sequenceIndex= */ 0); // Not released yet. assertThat(releasedFrameTimestamps).isEmpty(); @@ -777,134 +807,6 @@ public void onFrameProcessed_afterFlush_releasesFrame() throws Exception { assertThat(releasedFrameTimestamps).containsExactly(100_000L); } - private ImmutableList createPacket( - long presentationTimeUs, long sequencePresentationTimeUs) { - HardwareBufferFrame hardwareBufferFrame = - new HardwareBufferFrame.Builder( - checkNotNull(placeholderBuffer), - directExecutor(), - (releaseFence) -> releasedFrameTimestamps.add(presentationTimeUs)) - .setPresentationTimeUs(presentationTimeUs) - .setSequencePresentationTimeUs(sequencePresentationTimeUs) - .build(); - return ImmutableList.of(hardwareBufferFrame); - } - - private static FramesEvent toFramesEvent(List packet) { - ImmutableList.Builder asyncFrameListBuilder = ImmutableList.builder(); - for (HardwareBufferFrame effectFrame : packet) { - ImmutableMap.Builder metadataBuilder = - ImmutableMap.builder() - .put(Frame.KEY_PRESENTATION_TIME_US, effectFrame.presentationTimeUs) - .put(Frame.KEY_DISPLAY_TIME_NS, effectFrame.releaseTimeNs); - if (effectFrame.getMetadata() instanceof CompositionFrameMetadata) { - metadataBuilder.put( - CompositionFrameMetadata.KEY_COMPOSITION_FRAME_METADATA, effectFrame.getMetadata()); - } - - DefaultHardwareBufferFrame commonFrame = - new DefaultHardwareBufferFrame.Builder(checkNotNull(effectFrame.hardwareBuffer)) - .setFormat(effectFrame.format) - .setContentTimeUs(effectFrame.sequencePresentationTimeUs) - .setMetadata(metadataBuilder.buildOrThrow()) - .setInternalImage(effectFrame.internalFrame) - .build(); - asyncFrameListBuilder.add(new AsyncFrame(commonFrame, effectFrame.acquireFence)); - } - return new FramesEvent(asyncFrameListBuilder.build()); - } - - /** Verifies that the {@link #frameProcessor} received the expected {@link Event}. */ - private void assertOutputEvents(boolean ignoreReleaseTime, Event... expectedEvents) { - ImmutableList queuedEvents = frameProcessor.getQueuedEvents(); - assertThat(queuedEvents).hasSize(expectedEvents.length); - for (int i = 0; i < expectedEvents.length; i++) { - Event receivedEvent = queuedEvents.get(i); - Event expectedEvent = expectedEvents[i]; - assertThat(receivedEvent).isInstanceOf(expectedEvent.getClass()); - if (expectedEvent instanceof FramesEvent) { - assertThat(receivedEvent).isInstanceOf(FramesEvent.class); - List receivedFrames = ((FramesEvent) receivedEvent).frames; - List expectedFrames = ((FramesEvent) expectedEvent).frames; - assertThat(receivedFrames).hasSize(expectedFrames.size()); - for (int j = 0; j < receivedFrames.size(); j++) { - AsyncFrame receivedAsyncFrame = receivedFrames.get(j); - assertThat(receivedAsyncFrame.frame).isInstanceOf(DefaultHardwareBufferFrame.class); - DefaultHardwareBufferFrame receivedFrame = - (DefaultHardwareBufferFrame) receivedAsyncFrame.frame; - - AsyncFrame expectedAsyncFrame = expectedFrames.get(j); - assertThat(expectedAsyncFrame.frame).isInstanceOf(DefaultHardwareBufferFrame.class); - DefaultHardwareBufferFrame expectedFrame = - (DefaultHardwareBufferFrame) expectedAsyncFrame.frame; - - assertFramesEqual(receivedFrame, expectedFrame, ignoreReleaseTime); - } - } - } - } - - /** - * Verifies that the fields on the received and expected {@link DefaultHardwareBufferFrame} are - * equal. - */ - private static void assertFramesEqual( - DefaultHardwareBufferFrame received, - DefaultHardwareBufferFrame expected, - boolean ignoreReleaseTime) { - assertThat(received.getContentTimeUs()).isEqualTo(expected.getContentTimeUs()); - assertThat(received.getHardwareBuffer()).isEqualTo(expected.getHardwareBuffer()); - assertThat(received.getFormat()).isEqualTo(expected.getFormat()); - assertThat(received.getInternalImage()).isEqualTo(expected.getInternalImage()); - - Long receivedPresentationTimeUs = - (Long) received.getMetadata().get(Frame.KEY_PRESENTATION_TIME_US); - Long expectedPresentationTimeUs = - (Long) expected.getMetadata().get(Frame.KEY_PRESENTATION_TIME_US); - assertThat(receivedPresentationTimeUs).isEqualTo(expectedPresentationTimeUs); - - if (!ignoreReleaseTime) { - Long receivedReleaseTimeNs = (Long) received.getMetadata().get(Frame.KEY_DISPLAY_TIME_NS); - Long expectedReleaseTimeNs = (Long) expected.getMetadata().get(Frame.KEY_DISPLAY_TIME_NS); - assertThat(receivedReleaseTimeNs).isEqualTo(expectedReleaseTimeNs); - } - } - - private static ImmutableList updatePacketWithReleaseTime( - ImmutableList packet, long releaseTimeNs) { - ImmutableList.Builder updatedPacketBuilder = new ImmutableList.Builder<>(); - for (HardwareBufferFrame frame : packet) { - updatedPacketBuilder.add(frame.buildUpon().setReleaseTimeNs(releaseTimeNs).build()); - } - return updatedPacketBuilder.build(); - } - - /** Empty {@link FrameWriter} implementation the returns no frames and always succeeds. */ - private static final class NoOpFrameWriter implements FrameWriter { - @Override - public Info getInfo() { - return (format, usage) -> true; - } - - @Override - public void configure(Format format, @Frame.Usage long usage) {} - - @Override - @Nullable - public AsyncFrame dequeueInputFrame(Executor wakeupExecutor, Runnable wakeupListener) { - return null; - } - - @Override - public void queueInputFrame(Frame frame, @Nullable SyncFenceWrapper writeCompleteFence) {} - - @Override - public void signalEndOfStream() {} - - @Override - public void close() {} - } - @Test public void redraw_withNoCachedFrame_doesNotQueueDownstream() { compositionVideoPacketReleaseControl.redraw(); @@ -927,13 +829,12 @@ public void redraw_afterFrameReleased_queuesFrameImmediately() throws ExoPlaybac compositionVideoPacketReleaseControl.redraw(); // Verify that the packet was queued again (size should be 2 now). - ImmutableList events = frameProcessor.getQueuedEvents(); + ImmutableList events = frameProcessor.getQueuedEvents(); assertThat(events).hasSize(2); - AsyncFrame redrawnFrame = ((FakeFrameProcessor.FramesEvent) events.get(1)).frames.get(0); + AsyncFrame redrawnFrame = ((FramesEvent) events.get(1)).frames.get(0); assertFramesEqual( (DefaultHardwareBufferFrame) redrawnFrame.frame, - (DefaultHardwareBufferFrame) - ((FakeFrameProcessor.FramesEvent) events.get(0)).frames.get(0).frame, + (DefaultHardwareBufferFrame) ((FramesEvent) events.get(0)).frames.get(0).frame, /* ignoreReleaseTime= */ true); } @@ -976,9 +877,9 @@ public void redraw_afterReset_ignoresRedraw() throws ExoPlaybackException { @Test public void onRender_normalPlaybackAndRedraw_managesRefCountsCorrectly() throws Exception { compositionVideoPacketReleaseControl.onStarted(); - ImmutableList packet0 = + ImmutableList packet0 = createPacket(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); - ImmutableList packet1 = + ImmutableList packet1 = createPacket(/* presentationTimeUs= */ 200_000, /* sequencePresentationTimeUs= */ 200_000); compositionVideoPacketReleaseControl.queue(packet0); @@ -1024,7 +925,7 @@ public void onRender_normalPlaybackAndRedraw_managesRefCountsCorrectly() throws @Test public void onRender_multipleRedrawsWhilePaused_managesRefCountsCorrectly() throws Exception { compositionVideoPacketReleaseControl.onStarted(); - ImmutableList packet0 = + ImmutableList packet0 = createPacket(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); // Initial render @@ -1055,7 +956,7 @@ public void onRender_multipleRedrawsWhilePaused_managesRefCountsCorrectly() thro assertThat(releasedFrameTimestamps).isEmpty(); // Queueing the next packet should evict the first. - ImmutableList packet1 = + ImmutableList packet1 = createPacket(/* presentationTimeUs= */ 200_000, /* sequencePresentationTimeUs= */ 200_000); compositionVideoPacketReleaseControl.queue(packet1); compositionVideoPacketReleaseControl.onRender( @@ -1096,15 +997,21 @@ public void onError(Exception e) {} releaseControl.onStarted(); Set releasedTimestamps = new HashSet<>(); - ImmutableList packet = - ImmutableList.of( - new HardwareBufferFrame.Builder( - placeholderBuffer, - directExecutor(), - (releaseFence) -> releasedTimestamps.add(100_000L)) - .setPresentationTimeUs(100_000L) - .setSequencePresentationTimeUs(100_000L) - .build()); + DefaultHardwareBufferFrame commonFrame = + new DefaultHardwareBufferFrame.Builder( + placeholderBuffer, + directExecutor(), + (releaseFence) -> releasedTimestamps.add(100_000L)) + .setContentTimeUs(100_000L) + .setMetadata( + ImmutableMap.of( + Frame.KEY_PRESENTATION_TIME_US, + 100_000L, + Frame.KEY_DISPLAY_TIME_NS, + C.TIME_UNSET)) + .build(); + ImmutableList packet = + ImmutableList.of(new AsyncFrame(commonFrame, /* acquireFence= */ null)); releaseControl.queue(packet); @@ -1116,8 +1023,8 @@ public void onError(Exception e) {} assertThat(releasedTimestamps).isEmpty(); - for (HardwareBufferFrame frame : packet) { - frame.release(/* releaseFence= */ null); + for (AsyncFrame frame : packet) { + TransformerUtil.releaseIfNeeded(frame.frame, /* releaseFence= */ null); } assertThat(releasedTimestamps).containsExactly(100_000L); @@ -1127,9 +1034,9 @@ public void onError(Exception e) {} public void redraw_frameInFlightWhenNextPacketQueued_doesNotReleaseFrameUntilProcessed() throws Exception { compositionVideoPacketReleaseControl.onStarted(); - ImmutableList packet0 = + ImmutableList packet0 = createPacket(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); - ImmutableList packet1 = + ImmutableList packet1 = createPacket(/* presentationTimeUs= */ 200_000, /* sequencePresentationTimeUs= */ 200_000); compositionVideoPacketReleaseControl.queue(packet0); @@ -1161,8 +1068,132 @@ public void redraw_frameInFlightWhenNextPacketQueued_doesNotReleaseFrameUntilPro } private AsyncFrame getFirstQueuedFrameInPacket(int index) { - return ((FakeFrameProcessor.FramesEvent) frameProcessor.getQueuedEvents().get(index)) - .frames.get(0); + return ((FramesEvent) frameProcessor.getQueuedEvents().get(index)).frames.get(0); + } + + private ImmutableList createPacket( + long presentationTimeUs, long sequencePresentationTimeUs) { + DefaultHardwareBufferFrame commonFrame = + new DefaultHardwareBufferFrame.Builder( + checkNotNull(placeholderBuffer), + directExecutor(), + (releaseFence) -> releasedFrameTimestamps.add(presentationTimeUs)) + .setContentTimeUs(sequencePresentationTimeUs) + .setMetadata( + ImmutableMap.of( + Frame.KEY_PRESENTATION_TIME_US, + presentationTimeUs, + Frame.KEY_DISPLAY_TIME_NS, + C.TIME_UNSET)) + .build(); + return ImmutableList.of(new AsyncFrame(commonFrame, /* acquireFence= */ null)); + } + + private static FramesEvent toFramesEvent(List packet) { + return new FramesEvent(ImmutableList.copyOf(packet)); + } + + /** Verifies that the {@link #frameProcessor} received the expected {@link Event}. */ + private void assertOutputEvents(boolean ignoreReleaseTime, Event... expectedEvents) { + ImmutableList queuedEvents = frameProcessor.getQueuedEvents(); + assertThat(queuedEvents).hasSize(expectedEvents.length); + for (int i = 0; i < expectedEvents.length; i++) { + Event receivedEvent = queuedEvents.get(i); + Event expectedEvent = expectedEvents[i]; + assertThat(receivedEvent).isInstanceOf(expectedEvent.getClass()); + if (expectedEvent instanceof FramesEvent) { + FramesEvent framesEvent = (FramesEvent) expectedEvent; + assertThat(receivedEvent).isInstanceOf(FramesEvent.class); + ImmutableList receivedFrames = ((FramesEvent) receivedEvent).frames; + ImmutableList expectedFrames = framesEvent.frames; + assertThat(receivedFrames).hasSize(expectedFrames.size()); + for (int j = 0; j < receivedFrames.size(); j++) { + AsyncFrame receivedAsyncFrame = receivedFrames.get(j); + assertThat(receivedAsyncFrame.frame).isInstanceOf(DefaultHardwareBufferFrame.class); + DefaultHardwareBufferFrame receivedFrame = + (DefaultHardwareBufferFrame) receivedAsyncFrame.frame; + + AsyncFrame expectedAsyncFrame = expectedFrames.get(j); + assertThat(expectedAsyncFrame.frame).isInstanceOf(DefaultHardwareBufferFrame.class); + DefaultHardwareBufferFrame expectedFrame = + (DefaultHardwareBufferFrame) expectedAsyncFrame.frame; + + assertFramesEqual(receivedFrame, expectedFrame, ignoreReleaseTime); + } + } + } + } + + /** + * Verifies that the fields on the received and expected {@link DefaultHardwareBufferFrame} are + * equal. + */ + private static void assertFramesEqual( + DefaultHardwareBufferFrame received, + DefaultHardwareBufferFrame expected, + boolean ignoreReleaseTime) { + assertThat(received.getContentTimeUs()).isEqualTo(expected.getContentTimeUs()); + assertThat(received.getHardwareBuffer()).isEqualTo(expected.getHardwareBuffer()); + assertThat(received.getFormat()).isEqualTo(expected.getFormat()); + assertThat(received.getInternalImage()).isEqualTo(expected.getInternalImage()); + + Long receivedPresentationTimeUs = + (Long) received.getMetadata().get(Frame.KEY_PRESENTATION_TIME_US); + Long expectedPresentationTimeUs = + (Long) expected.getMetadata().get(Frame.KEY_PRESENTATION_TIME_US); + assertThat(receivedPresentationTimeUs).isEqualTo(expectedPresentationTimeUs); + + if (!ignoreReleaseTime) { + Long receivedReleaseTimeNs = (Long) received.getMetadata().get(Frame.KEY_DISPLAY_TIME_NS); + Long expectedReleaseTimeNs = (Long) expected.getMetadata().get(Frame.KEY_DISPLAY_TIME_NS); + assertThat(receivedReleaseTimeNs).isEqualTo(expectedReleaseTimeNs); + } + } + + private static ImmutableList updatePacketWithReleaseTime( + ImmutableList packet, long releaseTimeNs) { + ImmutableList.Builder updatedPacketBuilder = new ImmutableList.Builder<>(); + for (AsyncFrame asyncFrame : packet) { + DefaultHardwareBufferFrame commonFrame = (DefaultHardwareBufferFrame) asyncFrame.frame; + ImmutableMap.Builder metadataBuilder = + ImmutableMap.builder().putAll(commonFrame.getMetadata()); + metadataBuilder.put(Frame.KEY_DISPLAY_TIME_NS, releaseTimeNs); + DefaultHardwareBufferFrame updatedCommonFrame = + new DefaultHardwareBufferFrame.Builder(commonFrame.getHardwareBuffer()) + .setFormat(commonFrame.getFormat()) + .setContentTimeUs(commonFrame.getContentTimeUs()) + .setMetadata(metadataBuilder.buildKeepingLast()) + .setInternalImage(commonFrame.getInternalImage()) + .build(); + updatedPacketBuilder.add(new AsyncFrame(updatedCommonFrame, asyncFrame.acquireFence)); + } + return updatedPacketBuilder.build(); + } + + /** Empty {@link FrameWriter} implementation the returns no frames and always succeeds. */ + private static final class NoOpFrameWriter implements FrameWriter { + @Override + public Info getInfo() { + return (format, usage) -> true; + } + + @Override + public void configure(Format format, @Frame.Usage long usage) {} + + @Override + @Nullable + public AsyncFrame dequeueInputFrame(Executor wakeupExecutor, Runnable wakeupListener) { + return null; + } + + @Override + public void queueInputFrame(Frame frame, @Nullable SyncFenceWrapper writeCompleteFence) {} + + @Override + public void signalEndOfStream() {} + + @Override + public void close() {} } /** diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/DefaultDecoderFactoryTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/DefaultDecoderFactoryTest.java new file mode 100644 index 00000000000..ede186b655a --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/DefaultDecoderFactoryTest.java @@ -0,0 +1,92 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static androidx.media3.transformer.ExportException.ERROR_CODE_DECODING_FORMAT_UNSUPPORTED; +import static androidx.test.core.app.ApplicationProvider.getApplicationContext; +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; + +import android.content.Context; +import android.graphics.SurfaceTexture; +import android.view.Surface; +import androidx.media3.common.Format; +import androidx.media3.common.MimeTypes; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link DefaultDecoderFactory}. */ +@RunWith(AndroidJUnit4.class) +public final class DefaultDecoderFactoryTest { + + private final Context context = getApplicationContext(); + private SurfaceTexture surfaceTexture; + private Surface surface; + + @Before + public void setUp() { + surfaceTexture = new SurfaceTexture(/* texName= */ 0); + surface = new Surface(surfaceTexture); + } + + @After + public void tearDown() { + surface.release(); + surfaceTexture.release(); + } + + @Test + public void createForVideoDecoding_withDolbyVisionProfile20_throws() { + Format format = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION) + .setCodecs("dvh1.20.01") + .setWidth(1920) + .setHeight(1080) + .build(); + DefaultDecoderFactory decoderFactory = new DefaultDecoderFactory.Builder(context).build(); + + ExportException exception = + assertThrows( + ExportException.class, + () -> + decoderFactory.createForVideoDecoding( + format, surface, /* requestSdrToneMapping= */ false, /* logSessionId= */ null)); + assertThat(exception.errorCode).isEqualTo(ERROR_CODE_DECODING_FORMAT_UNSUPPORTED); + } + + @Test + public void createForVideoDecoding_withDolbyVisionNullCodecs_throws() { + Format format = + new Format.Builder() + .setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION) + .setWidth(1920) + .setHeight(1080) + .build(); + DefaultDecoderFactory decoderFactory = new DefaultDecoderFactory.Builder(context).build(); + + ExportException exception = + assertThrows( + ExportException.class, + () -> + decoderFactory.createForVideoDecoding( + format, surface, /* requestSdrToneMapping= */ false, /* logSessionId= */ null)); + assertThat(exception.errorCode).isEqualTo(ERROR_CODE_DECODING_FORMAT_UNSUPPORTED); + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/DefaultEncoderFactoryTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/DefaultEncoderFactoryTest.java index 4cf1f2b9799..2c895e0c650 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/DefaultEncoderFactoryTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/DefaultEncoderFactoryTest.java @@ -22,92 +22,48 @@ import static androidx.media3.common.C.COLOR_SPACE_BT2020; import static androidx.media3.common.C.COLOR_TRANSFER_ST2084; import static androidx.media3.exoplayer.mediacodec.MediaCodecUtil.createCodecProfileLevel; +import static androidx.media3.test.utils.robolectric.ShadowMediaCodecConfig.CODEC_INFO_AAC; import static androidx.media3.transformer.ExportException.ERROR_CODE_ENCODING_FORMAT_UNSUPPORTED; import static androidx.test.core.app.ApplicationProvider.getApplicationContext; import static com.google.common.truth.Truth.assertThat; import static org.junit.Assert.assertThrows; import android.content.Context; -import android.media.MediaCodecInfo; -import android.media.MediaCodecInfo.CodecProfileLevel; +import android.media.MediaCodecInfo.CodecCapabilities; import android.media.MediaFormat; +import androidx.media3.common.C; import androidx.media3.common.ColorInfo; import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; +import androidx.media3.test.utils.robolectric.ShadowMediaCodecConfig; +import androidx.media3.test.utils.robolectric.ShadowMediaCodecConfig.CodecInfo; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.collect.ImmutableList; -import org.junit.After; -import org.junit.Before; +import org.junit.Rule; import org.junit.Test; import org.junit.runner.RunWith; import org.robolectric.annotation.Config; -import org.robolectric.shadows.MediaCodecInfoBuilder; import org.robolectric.shadows.ShadowBuild; -import org.robolectric.shadows.ShadowMediaCodec; -import org.robolectric.shadows.ShadowMediaCodecList; /** Unit test for {@link DefaultEncoderFactory}. */ @RunWith(AndroidJUnit4.class) public class DefaultEncoderFactoryTest { - private final Context context = getApplicationContext(); - - @Before - public void setUp() { - createShadowH264Encoder(); - createShadowAacEncoder(); - } - - @After - public void tearDown() { - ShadowMediaCodec.clearCodecs(); - ShadowMediaCodecList.reset(); - EncoderUtil.clearCachedEncoders(); - } - private static void createShadowH264Encoder() { - MediaFormat avcFormat = new MediaFormat(); - avcFormat.setString(MediaFormat.KEY_MIME, MediaFormat.MIMETYPE_VIDEO_AVC); - // Using Level4 gives us 8192 16x16 blocks. If using width 1920 uses 120 blocks, 8192 / 120 = 68 - // blocks will be left for encoding height 1088. - CodecProfileLevel profileLevel = createCodecProfileLevel(AVCProfileHigh, AVCLevel4); - - createShadowVideoEncoder(avcFormat, profileLevel, "test.transformer.avc.encoder"); - } - - private static void createShadowAacEncoder() { - MediaFormat format = new MediaFormat(); - format.setString(MediaFormat.KEY_MIME, MediaFormat.MIMETYPE_AUDIO_AAC); - MediaCodecInfo.CodecCapabilities capabilities = - MediaCodecInfoBuilder.CodecCapabilitiesBuilder.newBuilder() - .setMediaFormat(format) - .setIsEncoder(true) - .build(); - createShadowEncoder("test.transformer.aac.encoder", capabilities); - } - - private static void createShadowVideoEncoder( - MediaFormat supportedFormat, CodecProfileLevel supportedProfileLevel, String name) { - MediaCodecInfo.CodecCapabilities capabilities = - MediaCodecInfoBuilder.CodecCapabilitiesBuilder.newBuilder() - .setMediaFormat(supportedFormat) - .setIsEncoder(true) - .setColorFormats( - new int[] {MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Flexible}) - .setProfileLevels(new CodecProfileLevel[] {supportedProfileLevel}) - .build(); - createShadowEncoder(name, capabilities); - } + private static final CodecInfo CODEC_INFO_AVC_LEVEL4 = + new CodecInfo( + /* codecName= */ "test.transformer.avc.encoder", + MediaFormat.MIMETYPE_VIDEO_AVC, + // Using Level4 gives us 8192 16x16 blocks. If using width 1920 uses 120 blocks, 8192 / + // 120 = 68 + // blocks will be left for encoding height 1088. + ImmutableList.of(createCodecProfileLevel(AVCProfileHigh, AVCLevel4)), + ImmutableList.of(CodecCapabilities.COLOR_FormatYUV420Flexible)); + private final Context context = getApplicationContext(); - private static void createShadowEncoder( - String name, MediaCodecInfo.CodecCapabilities... capabilities) { - // ShadowMediaCodecList is static. The added encoders will be visible for every test. - ShadowMediaCodecList.addCodec( - MediaCodecInfoBuilder.newBuilder() - .setName(name) - .setIsEncoder(true) - .setCapabilities(capabilities) - .build()); - } + @Rule + public ShadowMediaCodecConfig shadowMediaCodecConfig = + ShadowMediaCodecConfig.withCodecs( + ImmutableList.of(), ImmutableList.of(CODEC_INFO_AVC_LEVEL4, CODEC_INFO_AAC)); @Test public void createForVideoEncoding_withFallbackOnAndSupportedInputFormat_configuresEncoder() @@ -220,16 +176,16 @@ public void createForVideoEncoding_withFallbackOnAndUnsupportedResolution_config createForVideoEncoding_withH264EncodingOnApi31_configuresEncoderWithCorrectPerformanceSettings() throws Exception { Format requestedVideoFormat = createVideoFormat(MimeTypes.VIDEO_H264, 1920, 1080, 30); - Codec videoEncoder = + DefaultCodec videoEncoder = new DefaultEncoderFactory.Builder(context) .build() .createForVideoEncoding(requestedVideoFormat, /* logSessionId= */ null); assertThat(videoEncoder).isInstanceOf(DefaultCodec.class); - MediaFormat configurationMediaFormat = - ((DefaultCodec) videoEncoder).getConfigurationMediaFormat(); + MediaFormat configurationMediaFormat = videoEncoder.getConfigurationMediaFormat(); assertThat(configurationMediaFormat.containsKey(MediaFormat.KEY_PRIORITY)).isTrue(); - assertThat(configurationMediaFormat.getInteger(MediaFormat.KEY_PRIORITY)).isEqualTo(1); + assertThat(configurationMediaFormat.getInteger(MediaFormat.KEY_PRIORITY)) + .isEqualTo(C.MEDIA_CODEC_PRIORITY_NON_REALTIME); assertThat(configurationMediaFormat.containsKey(MediaFormat.KEY_OPERATING_RATE)).isTrue(); assertThat(configurationMediaFormat.getInteger(MediaFormat.KEY_OPERATING_RATE)) .isEqualTo(Integer.MAX_VALUE); @@ -241,7 +197,7 @@ public void createForVideoEncoding_withFallbackOnAndUnsupportedResolution_config createForVideoEncoding_withH264EncodingOnApi29AndConservativeDefault_configuresEncoderWithCorrectPerformanceSettings() throws Exception { Format requestedVideoFormat = createVideoFormat(MimeTypes.VIDEO_H264, 1920, 1080, 30); - Codec videoEncoder = + DefaultCodec videoEncoder = new DefaultEncoderFactory.Builder(context) .setRequestedVideoEncoderSettings( new VideoEncoderSettings.Builder() @@ -251,10 +207,10 @@ public void createForVideoEncoding_withFallbackOnAndUnsupportedResolution_config .createForVideoEncoding(requestedVideoFormat, /* logSessionId= */ null); assertThat(videoEncoder).isInstanceOf(DefaultCodec.class); - MediaFormat configurationMediaFormat = - ((DefaultCodec) videoEncoder).getConfigurationMediaFormat(); + MediaFormat configurationMediaFormat = videoEncoder.getConfigurationMediaFormat(); assertThat(configurationMediaFormat.containsKey(MediaFormat.KEY_PRIORITY)).isTrue(); - assertThat(configurationMediaFormat.getInteger(MediaFormat.KEY_PRIORITY)).isEqualTo(1); + assertThat(configurationMediaFormat.getInteger(MediaFormat.KEY_PRIORITY)) + .isEqualTo(C.MEDIA_CODEC_PRIORITY_NON_REALTIME); assertThat(configurationMediaFormat.containsKey(MediaFormat.KEY_OPERATING_RATE)).isTrue(); assertThat(configurationMediaFormat.getInteger(MediaFormat.KEY_OPERATING_RATE)).isEqualTo(-1); } @@ -265,7 +221,7 @@ public void createForVideoEncoding_withFallbackOnAndUnsupportedResolution_config createForVideoEncoding_withOperatingRateUnset_configuresEncoderWithCorrectPerformanceSettings() throws Exception { Format requestedVideoFormat = createVideoFormat(MimeTypes.VIDEO_H264, 1920, 1080, 30); - Codec videoEncoder = + DefaultCodec videoEncoder = new DefaultEncoderFactory.Builder(context) .setRequestedVideoEncoderSettings( new VideoEncoderSettings.Builder() @@ -276,10 +232,10 @@ public void createForVideoEncoding_withFallbackOnAndUnsupportedResolution_config .createForVideoEncoding(requestedVideoFormat, /* logSessionId= */ null); assertThat(videoEncoder).isInstanceOf(DefaultCodec.class); - MediaFormat configurationMediaFormat = - ((DefaultCodec) videoEncoder).getConfigurationMediaFormat(); + MediaFormat configurationMediaFormat = videoEncoder.getConfigurationMediaFormat(); assertThat(configurationMediaFormat.containsKey(MediaFormat.KEY_PRIORITY)).isTrue(); - assertThat(configurationMediaFormat.getInteger(MediaFormat.KEY_PRIORITY)).isEqualTo(1); + assertThat(configurationMediaFormat.getInteger(MediaFormat.KEY_PRIORITY)) + .isEqualTo(C.MEDIA_CODEC_PRIORITY_NON_REALTIME); assertThat(configurationMediaFormat.containsKey(MediaFormat.KEY_OPERATING_RATE)).isFalse(); } @@ -289,7 +245,7 @@ public void createForVideoEncoding_withFallbackOnAndUnsupportedResolution_config createForVideoEncoding_withOperatingRatePriorityUnset_configuresEncoderWithCorrectPerformanceSettings() throws Exception { Format requestedVideoFormat = createVideoFormat(MimeTypes.VIDEO_H264, 1920, 1080, 30); - Codec videoEncoder = + DefaultCodec videoEncoder = new DefaultEncoderFactory.Builder(context) .setRequestedVideoEncoderSettings( new VideoEncoderSettings.Builder() @@ -300,8 +256,7 @@ public void createForVideoEncoding_withFallbackOnAndUnsupportedResolution_config .createForVideoEncoding(requestedVideoFormat, /* logSessionId= */ null); assertThat(videoEncoder).isInstanceOf(DefaultCodec.class); - MediaFormat configurationMediaFormat = - ((DefaultCodec) videoEncoder).getConfigurationMediaFormat(); + MediaFormat configurationMediaFormat = videoEncoder.getConfigurationMediaFormat(); assertThat(configurationMediaFormat.containsKey(MediaFormat.KEY_PRIORITY)).isFalse(); assertThat(configurationMediaFormat.containsKey(MediaFormat.KEY_OPERATING_RATE)).isFalse(); } @@ -359,23 +314,6 @@ public void createForVideoEncoding_withMaxBFrames_configuresEncoderWithMaxBFrame .isEqualTo(3); } - @Test - @Config(sdk = 23) - public void createForVideoEncoding_withMaxBFramesOnApi23_doesNotConfigureMaxBFrames() - throws Exception { - Format requestedVideoFormat = createVideoFormat(MimeTypes.VIDEO_H264, 1920, 1080, 30); - - DefaultCodec videoEncoder = - new DefaultEncoderFactory.Builder(context) - .setRequestedVideoEncoderSettings( - new VideoEncoderSettings.Builder().setMaxBFrames(3).build()) - .build() - .createForVideoEncoding(requestedVideoFormat, /* logSessionId= */ null); - - assertThat(videoEncoder.getConfigurationMediaFormat().containsKey(MediaFormat.KEY_MAX_B_FRAMES)) - .isFalse(); - } - @Test @Config(sdk = 29) public void createForVideoEncoding_withDefaultEncoderSettings_doesNotConfigureMaxBFrames() @@ -457,30 +395,6 @@ public void createForVideoEncoding_withDefaultEncoderSettings_doesNotConfigureMa .isEqualTo("android.generic.1+2"); } - @Config(sdk = 23) - @Test - public void - createForVideoEncoding_withTemporalLayeringSchemaOnApi23_doesNotConfigureTemporalLayeringSchema() - throws Exception { - Format requestedVideoFormat = createVideoFormat(MimeTypes.VIDEO_H264, 1920, 1080, 30); - - DefaultCodec videoEncoder = - new DefaultEncoderFactory.Builder(context) - .setRequestedVideoEncoderSettings( - new VideoEncoderSettings.Builder() - .setTemporalLayers( - /* numNonBidirectionalLayers= */ 1, /* numBidirectionalLayers= */ 2) - .build()) - .build() - .createForVideoEncoding(requestedVideoFormat, /* logSessionId= */ null); - - assertThat( - videoEncoder - .getConfigurationMediaFormat() - .containsKey(MediaFormat.KEY_TEMPORAL_LAYERING)) - .isFalse(); - } - @Config(sdk = 29) @Test public void @@ -592,8 +506,10 @@ public void createForVideoEncoding_withCodecDbLiteDisabled_doesNotUseCodecDbLite public void createForVideoEncoding_withCodecDbLiteBitrateEnabled_configuresEncoderWithRecommendedBitrate() throws Exception { - ShadowBuild.setSystemOnChipManufacturer("QTI"); - ShadowBuild.setSystemOnChipModel("SM8550"); + // Samsung Exynos 850 opts in to the non-linear rate model at 1080p, so CodecDB Lite returns a + // recommendation rather than deferring to Kush Gauge. + ShadowBuild.setSystemOnChipManufacturer("Samsung"); + ShadowBuild.setSystemOnChipModel("Exynos 850"); Format requestedVideoFormat = createVideoFormat(MimeTypes.VIDEO_H264, 1920, 1080, 30); @@ -606,6 +522,28 @@ public void createForVideoEncoding_withCodecDbLiteDisabled_doesNotUseCodecDbLite assertThat(videoEncoder.getConfigurationFormat().averageBitrate).isEqualTo(15_502_805); } + @Test + @Config(sdk = 31) + public void + createForVideoEncoding_withCodecDbLiteBitrateEnabledAndCodecDbReturnsNoValue_fallsBackToKushGauge() + throws Exception { + // MediaTek MT6761 is in the dataset but does not opt in to the non-linear rate model, so + // CodecDB Lite returns no bitrate recommendation. + ShadowBuild.setSystemOnChipManufacturer("Mediatek"); + ShadowBuild.setSystemOnChipModel("MT6761"); + + Format requestedVideoFormat = createVideoFormat(MimeTypes.VIDEO_H264, 1920, 1080, 30); + + DefaultCodec videoEncoder = + new DefaultEncoderFactory.Builder(context) + .setEnableCodecDbLiteBitrate(true) + .build() + .createForVideoEncoding(requestedVideoFormat, /* logSessionId= */ null); + + // Kush Gauge suggested bitrate is 8_709_120. + assertThat(videoEncoder.getConfigurationFormat().averageBitrate).isEqualTo(8_709_120); + } + @Test @Config(sdk = 31) public void createForVideoEncoding_withCodecDbLiteEnabledAndBitrateDisabled_usesSuggestedBitrate() @@ -669,6 +607,11 @@ public void createForVideoEncoding_withCodecDbLiteBitrateEnabled_respectsRequest public void createForVideoEncoding_withCodecDbLiteBitrateEnabledAndNoFallback_configuresEncoderWithRecommendedBitrate() throws Exception { + // Samsung Exynos 850 opts in to the non-linear rate model at 1080p, so CodecDB Lite returns a + // recommendation rather than deferring to Kush Gauge. + ShadowBuild.setSystemOnChipManufacturer("Samsung"); + ShadowBuild.setSystemOnChipModel("Exynos 850"); + Format requestedVideoFormat = createVideoFormat(MimeTypes.VIDEO_H264, 1920, 1080, 30); DefaultCodec videoEncoder = diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/DefaultMuxerTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/DefaultMuxerTest.java new file mode 100644 index 00000000000..cfacca13c0f --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/DefaultMuxerTest.java @@ -0,0 +1,49 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static com.google.common.truth.Truth.assertThat; + +import android.os.ParcelFileDescriptor; +import androidx.media3.muxer.Muxer; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.io.File; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import org.junit.runner.RunWith; + +/** Unit tests for {@link DefaultMuxer}. */ +@RunWith(AndroidJUnit4.class) +public final class DefaultMuxerTest { + + @Rule public final TemporaryFolder temporaryFolder = new TemporaryFolder(); + + @Test + public void createAndClose_withParcelFileDescriptor_keepsDescriptorOpenUntilMuxerClosed() + throws Exception { + File tempFile = temporaryFolder.newFile("test_output.mp4"); + DefaultMuxer.Factory factory = new DefaultMuxer.Factory(); + + ParcelFileDescriptor pfd = + ParcelFileDescriptor.open(tempFile, ParcelFileDescriptor.MODE_READ_WRITE); + try (Muxer muxer = factory.create(pfd)) { + assertThat(muxer).isNotNull(); + assertThat(pfd.getFileDescriptor().valid()).isTrue(); + } + assertThat(pfd.getFileDescriptor().valid()).isFalse(); + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/EditedMediaItemTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/EditedMediaItemTest.java new file mode 100644 index 00000000000..083ecf08e7c --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/EditedMediaItemTest.java @@ -0,0 +1,85 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.media3.common.C; +import androidx.media3.common.MediaItem; +import androidx.media3.common.SpeedParameters; +import androidx.media3.common.audio.SpeedProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import org.json.JSONObject; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link EditedMediaItem}. */ +@RunWith(AndroidJUnit4.class) +public class EditedMediaItemTest { + + @Test + public void toJsonObject_editedMediaItem_containsAllParameters() throws Exception { + EditedMediaItem editedMediaItem = + new EditedMediaItem.Builder(MediaItem.fromUri("uri.mp4")) + .setRemoveAudio(true) + .setRemoveVideo(false) + .setFlattenForSlowMotion(true) + .setDurationUs(1_000_000L) + .setFrameRate(30) + .setSpeed( + new SpeedParameters( + new SpeedProvider() { + @Override + public float getSpeed(long timeUs) { + return 2.0f; + } + + @Override + public long getNextSpeedChangeTimeUs(long timeUs) { + return C.TIME_UNSET; + } + }, + /* shouldMaintainPitch= */ true)) + .build(); + + JSONObject jsonObject = editedMediaItem.toJsonObject(); + + assertThat(jsonObject.getBoolean("removeAudio")).isTrue(); + assertThat(jsonObject.getBoolean("removeVideo")).isFalse(); + assertThat(jsonObject.getBoolean("flattenForSlowMotion")).isTrue(); + assertThat(jsonObject.getLong("durationUs")).isEqualTo(1_000_000L); + assertThat(jsonObject.getInt("frameRate")).isEqualTo(30); + assertThat(jsonObject.has("speedParameters")).isTrue(); + assertThat(jsonObject.getJSONObject("speedParameters").getBoolean("shouldMaintainPitch")) + .isTrue(); + assertThat(jsonObject.getLong("presentationDuration")).isEqualTo(500_000L); + } + + @Test + public void toJsonObject_withDefaultParameters_returnsJsonObjectWithDefaultFields() + throws Exception { + EditedMediaItem editedMediaItem = + new EditedMediaItem.Builder(MediaItem.fromUri("uri.mp4")).build(); + + JSONObject jsonObject = editedMediaItem.toJsonObject(); + + assertThat(jsonObject.getBoolean("removeAudio")).isFalse(); + assertThat(jsonObject.getBoolean("removeVideo")).isFalse(); + assertThat(jsonObject.getBoolean("flattenForSlowMotion")).isFalse(); + assertThat(jsonObject.getString("frameRate")).isEqualTo("UNSET"); + assertThat(jsonObject.has("speedParameters")).isFalse(); + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/EditingMetricsCollectorTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/EditingMetricsCollectorTest.java index 8633e5826d8..340fabc11fb 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/EditingMetricsCollectorTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/EditingMetricsCollectorTest.java @@ -21,13 +21,25 @@ import android.hardware.DataSpace; import android.media.metrics.EditingEndedEvent; import android.media.metrics.MediaItemInfo; +import android.net.Uri; +import android.os.PersistableBundle; import android.util.Size; import androidx.media3.common.C; import androidx.media3.common.ColorInfo; +import androidx.media3.common.Effect; import androidx.media3.common.Format; import androidx.media3.common.MediaItem; import androidx.media3.common.MediaLibraryInfo; +import androidx.media3.common.audio.AudioProcessor; +import androidx.media3.common.audio.BaseAudioProcessor; +import androidx.media3.common.audio.SonicAudioProcessor; +import androidx.media3.effect.Crop; +import androidx.media3.effect.GaussianBlur; +import androidx.media3.effect.OverlayEffect; +import androidx.media3.effect.ScaleAndRotateTransformation; import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import java.nio.ByteBuffer; import java.util.ArrayList; import java.util.List; import java.util.concurrent.atomic.AtomicReference; @@ -78,6 +90,13 @@ public final class EditingMetricsCollectorTest { .setColorInfo(VIDEO_COLOR_INFO) .build(); + private static final Composition EMPTY_COMPOSITION = + new Composition.Builder( + EditedMediaItemSequence.withAudioAndVideoFrom( + ImmutableList.of( + new EditedMediaItem.Builder(MediaItem.fromUri(Uri.EMPTY)).build()))) + .build(); + @Test public void onExportSuccess_populatesEditingEndedEvent() { List processedInputs = new ArrayList<>(); @@ -121,7 +140,8 @@ public void close() {} /* compositionHasAudioProcessors= */ false, /* compositionHasVideoEffects= */ false); - editingMetricsCollector.onExportSuccess(exportResult, /* isExportResumed= */ false); + editingMetricsCollector.onExportSuccess( + EMPTY_COMPOSITION, exportResult, /* isExportResumed= */ false); EditingEndedEvent editingEndedEvent = editingEndedEventAtomicReference.get(); assertThat(editingEndedEvent.getFinalState()) @@ -231,7 +251,11 @@ public void close() {} int progressPercentage = 10; editingMetricsCollector.onExportError( - progressPercentage, exception, exportResult, /* isExportResumed= */ false); + EMPTY_COMPOSITION, + progressPercentage, + exception, + exportResult, + /* isExportResumed= */ false); EditingEndedEvent editingEndedEvent = editingEndedEventAtomicReference.get(); assertThat(editingEndedEvent.getFinalState()).isEqualTo(EditingEndedEvent.FINAL_STATE_ERROR); @@ -314,7 +338,7 @@ public void close() {} /* compositionHasVideoEffects= */ true); int progressPercentage = 70; - editingMetricsCollector.onExportCancelled(progressPercentage); + editingMetricsCollector.onExportCancelled(EMPTY_COMPOSITION, progressPercentage); EditingEndedEvent editingEndedEvent = editingEndedEventAtomicReference.get(); assertThat(editingEndedEvent.getFinalState()).isEqualTo(EditingEndedEvent.FINAL_STATE_CANCELED); @@ -327,4 +351,342 @@ public void close() {} EditingEndedEvent.OPERATION_TYPE_AUDIO_EDIT | EditingEndedEvent.OPERATION_TYPE_VIDEO_EDIT); } + + @Test + public void onExportSuccess_withVideoEffect_populatesSpatialComponentAndScope() { + AtomicReference bundleReference = new AtomicReference<>(); + EditingMetricsCollector collector = createCollectorForEffectMetrics(bundleReference); + Crop crop = + new Crop(/* left= */ -0.5f, /* right= */ 0.5f, /* bottom= */ -0.5f, /* top= */ 0.5f); + Composition composition = createCompositionWithVideoEffects(crop); + + collector.onExportSuccess( + composition, createDefaultExportResult(), /* isExportResumed= */ false); + + PersistableBundle bundle = bundleReference.get(); + assertThat(bundle).isNotNull(); + assertThat(bundle.getString(EditingMetricsCollector.KEY_PROCESSOR_NAME)) + .isEqualTo(EXPORTER_NAME); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_COMPONENTS)) + .asList() + .containsExactly(EditingMetricsCollector.COMPONENT_MEDIA3_EFFECT_SPATIAL); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_COMPONENT_SCOPES)) + .asList() + .containsExactly(1); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_METRICS)).isNull(); + assertThat(bundle.getLongArray(EditingMetricsCollector.KEY_METRIC_VALUES)).isNull(); + } + + @Test + public void onExportSuccess_withAudioProcessor_populatesSpeedAndPitchComponent() { + AtomicReference bundleReference = new AtomicReference<>(); + EditingMetricsCollector collector = createCollectorForEffectMetrics(bundleReference); + SonicAudioProcessor sonicAudioProcessor = new SonicAudioProcessor(); + Composition composition = createCompositionWithAudioProcessors(sonicAudioProcessor); + + collector.onExportSuccess( + composition, createDefaultExportResult(), /* isExportResumed= */ false); + + PersistableBundle bundle = bundleReference.get(); + assertThat(bundle).isNotNull(); + assertThat(bundle.getString(EditingMetricsCollector.KEY_PROCESSOR_NAME)) + .isEqualTo(EXPORTER_NAME); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_COMPONENTS)) + .asList() + .containsExactly(EditingMetricsCollector.COMPONENT_MEDIA3_AUDIO_SPEED_AND_PITCH); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_COMPONENT_SCOPES)) + .asList() + .containsExactly(1); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_METRICS)).isNull(); + assertThat(bundle.getLongArray(EditingMetricsCollector.KEY_METRIC_VALUES)).isNull(); + } + + @Test + public void onExportSuccess_withScaleAndRotate_extractsRotationMetric() { + AtomicReference bundleReference = new AtomicReference<>(); + EditingMetricsCollector collector = createCollectorForEffectMetrics(bundleReference); + ScaleAndRotateTransformation scaleAndRotate = + new ScaleAndRotateTransformation.Builder().setRotationDegrees(90).build(); + Composition composition = createCompositionWithVideoEffects(scaleAndRotate); + + collector.onExportSuccess( + composition, createDefaultExportResult(), /* isExportResumed= */ false); + + PersistableBundle bundle = bundleReference.get(); + assertThat(bundle).isNotNull(); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_COMPONENTS)) + .asList() + .containsExactly(EditingMetricsCollector.COMPONENT_MEDIA3_EFFECT_SPATIAL); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_METRICS)) + .asList() + .containsExactly(EditingMetricsCollector.METRIC_ROTATION_DEGREES); + assertThat(bundle.getLongArray(EditingMetricsCollector.KEY_METRIC_VALUES)) + .asList() + .containsExactly(90L); + } + + @Test + public void onExportSuccess_withChainedRotations_extractsAllRotationMetricsInOrder() { + AtomicReference bundleReference = new AtomicReference<>(); + EditingMetricsCollector collector = createCollectorForEffectMetrics(bundleReference); + ScaleAndRotateTransformation rotation90 = + new ScaleAndRotateTransformation.Builder().setRotationDegrees(90).build(); + ScaleAndRotateTransformation rotation180 = + new ScaleAndRotateTransformation.Builder().setRotationDegrees(180).build(); + Composition composition = createCompositionWithVideoEffects(rotation90, rotation180); + + collector.onExportSuccess( + composition, createDefaultExportResult(), /* isExportResumed= */ false); + + PersistableBundle bundle = bundleReference.get(); + assertThat(bundle).isNotNull(); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_METRICS)) + .asList() + .containsExactly( + EditingMetricsCollector.METRIC_ROTATION_DEGREES, + EditingMetricsCollector.METRIC_ROTATION_DEGREES) + .inOrder(); + assertThat(bundle.getLongArray(EditingMetricsCollector.KEY_METRIC_VALUES)) + .asList() + .containsExactly(90L, 180L) + .inOrder(); + } + + @Test + public void onExportSuccess_withGaussianBlur_extractsFilterSampleCountMetric() { + AtomicReference bundleReference = new AtomicReference<>(); + EditingMetricsCollector collector = createCollectorForEffectMetrics(bundleReference); + GaussianBlur gaussianBlur = + new GaussianBlur(/* sigma= */ 5.0f, /* numStandardDeviations= */ 2.0f); + Composition composition = createCompositionWithVideoEffects(gaussianBlur); + + collector.onExportSuccess( + composition, createDefaultExportResult(), /* isExportResumed= */ false); + + PersistableBundle bundle = bundleReference.get(); + assertThat(bundle).isNotNull(); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_COMPONENTS)) + .asList() + .containsExactly(EditingMetricsCollector.COMPONENT_MEDIA3_EFFECT_CONVOLUTION); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_METRICS)) + .asList() + .containsExactly(EditingMetricsCollector.METRIC_FILTER_SAMPLE_COUNT); + assertThat(bundle.getLongArray(EditingMetricsCollector.KEY_METRIC_VALUES)) + .asList() + .containsExactly(21L); + } + + @Test + public void onExportSuccess_withOverlayEffect_extractsOverlayCountMetric() { + AtomicReference bundleReference = new AtomicReference<>(); + EditingMetricsCollector collector = createCollectorForEffectMetrics(bundleReference); + OverlayEffect overlayEffect = new OverlayEffect(ImmutableList.of()); + Composition composition = createCompositionWithVideoEffects(overlayEffect); + + collector.onExportSuccess( + composition, createDefaultExportResult(), /* isExportResumed= */ false); + + PersistableBundle bundle = bundleReference.get(); + assertThat(bundle).isNotNull(); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_COMPONENTS)) + .asList() + .containsExactly(EditingMetricsCollector.COMPONENT_MEDIA3_EFFECT_OVERLAY); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_METRICS)) + .asList() + .containsExactly(EditingMetricsCollector.METRIC_OVERLAY_COUNT); + assertThat(bundle.getLongArray(EditingMetricsCollector.KEY_METRIC_VALUES)) + .asList() + .containsExactly(0L); + } + + @Test + public void onExportSuccess_withSonicAudioProcessorCustomSpeed_extractsSpeedMultiplierMetric() { + AtomicReference bundleReference = new AtomicReference<>(); + EditingMetricsCollector collector = createCollectorForEffectMetrics(bundleReference); + SonicAudioProcessor sonicAudioProcessor = new SonicAudioProcessor(); + sonicAudioProcessor.setSpeed(1.5f); + Composition composition = createCompositionWithAudioProcessors(sonicAudioProcessor); + + collector.onExportSuccess( + composition, createDefaultExportResult(), /* isExportResumed= */ false); + + PersistableBundle bundle = bundleReference.get(); + assertThat(bundle).isNotNull(); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_COMPONENTS)) + .asList() + .containsExactly(EditingMetricsCollector.COMPONENT_MEDIA3_AUDIO_SPEED_AND_PITCH); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_METRICS)) + .asList() + .containsExactly(EditingMetricsCollector.METRIC_SPEED_MULTIPLIER); + assertThat(bundle.getLongArray(EditingMetricsCollector.KEY_METRIC_VALUES)) + .asList() + .containsExactly(15000L); + } + + @Test + public void onExportSuccess_withSonicAudioProcessorDefaultSpeed_doesNotExtractSpeedMetric() { + AtomicReference bundleReference = new AtomicReference<>(); + EditingMetricsCollector collector = createCollectorForEffectMetrics(bundleReference); + SonicAudioProcessor sonicAudioProcessor = new SonicAudioProcessor(); + Composition composition = createCompositionWithAudioProcessors(sonicAudioProcessor); + + collector.onExportSuccess( + composition, createDefaultExportResult(), /* isExportResumed= */ false); + + PersistableBundle bundle = bundleReference.get(); + assertThat(bundle).isNotNull(); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_COMPONENTS)) + .asList() + .containsExactly(EditingMetricsCollector.COMPONENT_MEDIA3_AUDIO_SPEED_AND_PITCH); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_METRICS)).isNull(); + assertThat(bundle.getLongArray(EditingMetricsCollector.KEY_METRIC_VALUES)).isNull(); + } + + @Test + public void onExportSuccess_withMultipleEffectsSameScope_deduplicatesComponents() { + AtomicReference bundleReference = new AtomicReference<>(); + EditingMetricsCollector collector = createCollectorForEffectMetrics(bundleReference); + Crop crop = + new Crop(/* left= */ -0.5f, /* right= */ 0.5f, /* bottom= */ -0.5f, /* top= */ 0.5f); + ScaleAndRotateTransformation scaleAndRotate = + new ScaleAndRotateTransformation.Builder().setRotationDegrees(90).build(); + Composition composition = createCompositionWithVideoEffects(crop, scaleAndRotate); + + collector.onExportSuccess( + composition, createDefaultExportResult(), /* isExportResumed= */ false); + + PersistableBundle bundle = bundleReference.get(); + assertThat(bundle).isNotNull(); + // Both Crop and ScaleAndRotate are SPATIAL (101) in the same scope, so components array + // contains 101 only once + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_COMPONENTS)) + .asList() + .containsExactly(EditingMetricsCollector.COMPONENT_MEDIA3_EFFECT_SPATIAL); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_COMPONENT_SCOPES)) + .asList() + .containsExactly(1); + } + + @Test + public void onExportSuccess_withScaleAndRotateZeroRotation_doesNotExtractRotationMetric() { + AtomicReference bundleReference = new AtomicReference<>(); + EditingMetricsCollector collector = createCollectorForEffectMetrics(bundleReference); + ScaleAndRotateTransformation scaleAndRotate = + new ScaleAndRotateTransformation.Builder().setScale(2f, 2f).setRotationDegrees(0f).build(); + Composition composition = createCompositionWithVideoEffects(scaleAndRotate); + + collector.onExportSuccess( + composition, createDefaultExportResult(), /* isExportResumed= */ false); + + PersistableBundle bundle = bundleReference.get(); + assertThat(bundle).isNotNull(); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_COMPONENTS)) + .asList() + .containsExactly(EditingMetricsCollector.COMPONENT_MEDIA3_EFFECT_SPATIAL); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_METRICS)).isNull(); + assertThat(bundle.getLongArray(EditingMetricsCollector.KEY_METRIC_VALUES)).isNull(); + } + + @Test + public void onExportSuccess_withCustomEffectAndAudioProcessor_fallsBackToCustomComponents() { + AtomicReference bundleReference = new AtomicReference<>(); + EditingMetricsCollector collector = createCollectorForEffectMetrics(bundleReference); + Effect customEffect = new Effect() {}; + AudioProcessor customAudioProcessor = + new BaseAudioProcessor() { + @Override + public void queueInput(ByteBuffer inputBuffer) {} + }; + Composition composition = + new Composition.Builder( + EditedMediaItemSequence.withAudioAndVideoFrom( + ImmutableList.of( + new EditedMediaItem.Builder(MediaItem.fromUri(Uri.EMPTY)) + .setEffects( + new Effects( + /* audioProcessors= */ ImmutableList.of(customAudioProcessor), + /* videoEffects= */ ImmutableList.of(customEffect))) + .build()))) + .build(); + + collector.onExportSuccess( + composition, createDefaultExportResult(), /* isExportResumed= */ false); + + PersistableBundle bundle = bundleReference.get(); + assertThat(bundle).isNotNull(); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_COMPONENTS)) + .asList() + .containsExactly( + EditingMetricsCollector.COMPONENT_MEDIA3_EFFECT_CUSTOM, + EditingMetricsCollector.COMPONENT_MEDIA3_AUDIO_PROCESSOR_CUSTOM); + assertThat(bundle.getIntArray(EditingMetricsCollector.KEY_COMPONENT_SCOPES)) + .asList() + .containsExactly(1, 1); + } + + @Test + public void onExportSuccess_withoutEffects_doesNotReportEffectMetrics() { + AtomicReference bundleReference = new AtomicReference<>(); + EditingMetricsCollector collector = createCollectorForEffectMetrics(bundleReference); + + collector.onExportSuccess( + EMPTY_COMPOSITION, createDefaultExportResult(), /* isExportResumed= */ false); + + assertThat(bundleReference.get()).isNull(); + } + + private static EditingMetricsCollector createCollectorForEffectMetrics( + AtomicReference bundleReference) { + return new EditingMetricsCollector( + new EditingMetricsCollector.MetricsReporter() { + @Override + public void reportMetrics(EditingEndedEvent editingEndedEvent) {} + + @Override + public void reportEffectMetrics(PersistableBundle bundle) { + bundleReference.set(bundle); + } + + @Override + public void close() {} + }, + EXPORTER_NAME, + MUXER_NAME, + /* compositionHasAudioProcessors= */ true, + /* compositionHasVideoEffects= */ true); + } + + private static Composition createCompositionWithVideoEffects(Effect... effects) { + return new Composition.Builder( + EditedMediaItemSequence.withAudioAndVideoFrom( + ImmutableList.of( + new EditedMediaItem.Builder(MediaItem.fromUri(Uri.EMPTY)) + .setEffects( + new Effects( + /* audioProcessors= */ ImmutableList.of(), + /* videoEffects= */ ImmutableList.copyOf(effects))) + .build()))) + .build(); + } + + private static Composition createCompositionWithAudioProcessors( + AudioProcessor... audioProcessors) { + return new Composition.Builder( + EditedMediaItemSequence.withAudioAndVideoFrom( + ImmutableList.of( + new EditedMediaItem.Builder(MediaItem.fromUri(Uri.EMPTY)) + .setEffects( + new Effects( + /* audioProcessors= */ ImmutableList.copyOf(audioProcessors), + /* videoEffects= */ ImmutableList.of())) + .build()))) + .build(); + } + + private static ExportResult createDefaultExportResult() { + return new ExportResult.Builder() + .setAudioMimeType(AUDIO_MIME_TYPE) + .setVideoMimeType(VIDEO_MIME_TYPE) + .build(); + } } diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/EncoderFrameWriterTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/EncoderFrameWriterTest.java index e6244b68c31..17f0b0010ff 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/EncoderFrameWriterTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/EncoderFrameWriterTest.java @@ -164,6 +164,7 @@ public void configure_factoryThrowsException_notifiesListenerOnError() { encoderFrameWriter.configure(VIDEO_FORMAT, /* usage= */ 0L); // Verify the exception was intercepted and passed to the listener. + assertThat(testListener.encoder).isNull(); assertThat(testListener.exception).isNotNull(); assertThat(testListener.exception).hasCauseThat().isEqualTo(expectedException); } @@ -171,6 +172,7 @@ public void configure_factoryThrowsException_notifiesListenerOnError() { /** A fake listener that records callbacks for validation. */ private static class TestListener implements EncoderFrameWriter.Listener { @Nullable private VideoFrameProcessingException exception; + @Nullable private Codec encoder; @Override public Format onConfigure(Format requestedFormat) { @@ -178,7 +180,9 @@ public Format onConfigure(Format requestedFormat) { } @Override - public void onEncoderCreated(Codec encoder) {} + public void onEncoderCreated(Codec encoder) { + this.encoder = encoder; + } @Override public void onEndOfStream() {} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/EncoderUtilTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/EncoderUtilTest.java index 957d6670bda..b2c6e4d27ae 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/EncoderUtilTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/EncoderUtilTest.java @@ -21,14 +21,17 @@ import static androidx.media3.common.MimeTypes.VIDEO_H264; import static androidx.media3.exoplayer.mediacodec.MediaCodecUtil.createCodecProfileLevel; import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; import android.media.MediaCodecInfo; import android.media.MediaCodecInfo.CodecProfileLevel; import android.media.MediaFormat; +import android.util.Pair; import android.util.Size; import androidx.annotation.Nullable; import androidx.media3.common.C; import androidx.media3.common.ColorInfo; +import androidx.media3.common.Format; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.collect.ImmutableList; import org.junit.After; @@ -205,4 +208,127 @@ public void getSupportedEncodersForHdrEditing_returnsEmptyList() { .build())) .isEmpty(); } + + @Test + public void toTemporalLayeringSchema_withZeroNonBidirectionalLayers_returnsNone() { + assertThat( + EncoderUtil.toTemporalLayeringSchema( + /* numNonBidirectionalLayers= */ 0, /* numBidirectionalLayers= */ 0)) + .isEqualTo("none"); + } + + @Test + public void + toTemporalLayeringSchema_withZeroNonBidirectionalAndUnsetBidirectionalLayers_returnsNone() { + assertThat( + EncoderUtil.toTemporalLayeringSchema( + /* numNonBidirectionalLayers= */ 0, /* numBidirectionalLayers= */ Format.NO_VALUE)) + .isEqualTo("none"); + } + + @Test + public void + toTemporalLayeringSchema_withZeroNonBidirectionalAndPositiveBidirectionalLayers_throws() { + assertThrows( + IllegalArgumentException.class, + () -> + EncoderUtil.toTemporalLayeringSchema( + /* numNonBidirectionalLayers= */ 0, /* numBidirectionalLayers= */ 2)); + } + + @Test + public void toTemporalLayeringSchema_withoutBidirectionalLayers_returnsGenericSchema() { + assertThat( + EncoderUtil.toTemporalLayeringSchema( + /* numNonBidirectionalLayers= */ 3, /* numBidirectionalLayers= */ 0)) + .isEqualTo("android.generic.3"); + } + + @Test + public void toTemporalLayeringSchema_withBidirectionalLayers_returnsGenericSchemaWithPlus() { + assertThat( + EncoderUtil.toTemporalLayeringSchema( + /* numNonBidirectionalLayers= */ 1, /* numBidirectionalLayers= */ 2)) + .isEqualTo("android.generic.1+2"); + } + + @Test + public void toTemporalLayeringSchema_withNegativeNonBidirectionalLayers_throws() { + assertThrows( + IllegalArgumentException.class, + () -> + EncoderUtil.toTemporalLayeringSchema( + /* numNonBidirectionalLayers= */ -1, /* numBidirectionalLayers= */ 0)); + } + + @Test + public void toTemporalLayeringSchema_withNegativeBidirectionalLayers_throws() { + assertThrows( + IllegalArgumentException.class, + () -> + EncoderUtil.toTemporalLayeringSchema( + /* numNonBidirectionalLayers= */ 2, /* numBidirectionalLayers= */ -1)); + } + + @Test + public void parseTemporalLayeringSchema_withNone_returnsZeroLayers() { + assertThat(EncoderUtil.parseTemporalLayeringSchema("none")).isEqualTo(Pair.create(0, 0)); + } + + @Test + public void parseTemporalLayeringSchema_withGenericSchema_returnsLayers() { + assertThat(EncoderUtil.parseTemporalLayeringSchema("android.generic.3")) + .isEqualTo(Pair.create(3, 0)); + } + + @Test + public void parseTemporalLayeringSchema_withGenericSchemaWithPlus_returnsLayers() { + assertThat(EncoderUtil.parseTemporalLayeringSchema("android.generic.1+2")) + .isEqualTo(Pair.create(1, 2)); + } + + @Test + public void parseTemporalLayeringSchema_withGenericSchemaWithPlusZero_returnsLayers() { + assertThat(EncoderUtil.parseTemporalLayeringSchema("android.generic.2+0")) + .isEqualTo(Pair.create(2, 0)); + } + + @Test + public void parseTemporalLayeringSchema_withWebRtcSchema_throws() { + assertThrows( + UnsupportedOperationException.class, + () -> EncoderUtil.parseTemporalLayeringSchema("webrtc.vp8.2-layer")); + assertThrows( + UnsupportedOperationException.class, + () -> EncoderUtil.parseTemporalLayeringSchema("webrtc.svc.l1t3")); + } + + @Test + public void parseTemporalLayeringSchema_withMalformedSchema_throws() { + assertThrows( + IllegalArgumentException.class, + () -> EncoderUtil.parseTemporalLayeringSchema("android.generic.0")); + assertThrows( + IllegalArgumentException.class, + () -> EncoderUtil.parseTemporalLayeringSchema("vendor.specific.schema")); + } + + @Test + public void parseTemporalLayeringSchema_withLayerCountOverflow_throws() { + assertThrows( + IllegalArgumentException.class, + () -> EncoderUtil.parseTemporalLayeringSchema("android.generic.99999999999")); + assertThrows( + IllegalArgumentException.class, + () -> EncoderUtil.parseTemporalLayeringSchema("android.generic.1+99999999999")); + } + + @Test + public void parseTemporalLayeringSchema_withToTemporalLayeringSchemaOutput_roundTrips() { + String schema = + EncoderUtil.toTemporalLayeringSchema( + /* numNonBidirectionalLayers= */ 2, /* numBidirectionalLayers= */ 1); + + assertThat(EncoderUtil.parseTemporalLayeringSchema(schema)).isEqualTo(Pair.create(2, 1)); + } } diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/ForwardingEncoderFactoryTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/ForwardingEncoderFactoryTest.java new file mode 100644 index 00000000000..38479503ef4 --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/ForwardingEncoderFactoryTest.java @@ -0,0 +1,39 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static androidx.media3.test.utils.TestUtil.assertForwardingClassForwardsAllMethods; +import static androidx.media3.test.utils.TestUtil.assertSubclassOverridesAllMethods; + +import androidx.media3.transformer.Codec.EncoderFactory; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link ForwardingEncoderFactory}. */ +@RunWith(AndroidJUnit4.class) +public final class ForwardingEncoderFactoryTest { + + @Test + public void forwardsAllMethods() throws Exception { + assertForwardingClassForwardsAllMethods(EncoderFactory.class, ForwardingEncoderFactory::new); + } + + @Test + public void overridesAllMethods() throws Exception { + assertSubclassOverridesAllMethods(EncoderFactory.class, ForwardingEncoderFactory.class); + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/FrameAggregatorTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/FrameAggregatorTest.java index 7557128ba47..adde6b711e9 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/FrameAggregatorTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/FrameAggregatorTest.java @@ -15,15 +15,29 @@ */ package androidx.media3.transformer; +import static androidx.media3.transformer.FrameAggregator.STRATEGY_EXPECT_NO_FRAMES; +import static androidx.media3.transformer.FrameAggregator.STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET; +import static androidx.media3.transformer.FrameAggregator.STRATEGY_MATCH_FRAME_CLOSEST_TO_TARGET; +import static androidx.media3.transformer.TransformerUtil.END_OF_STREAM_ASYNC_FRAME; import static com.google.common.truth.Truth.assertThat; import static com.google.common.util.concurrent.MoreExecutors.directExecutor; import static org.junit.Assert.assertThrows; -import androidx.media3.effect.GlTextureFrame; -import androidx.media3.effect.HardwareBufferFrame; +import android.hardware.HardwareBuffer; +import android.util.Rational; +import androidx.annotation.Nullable; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.DefaultHardwareBufferFrame; +import androidx.media3.common.video.Frame; +import androidx.media3.transformer.FrameAggregator.AggregationStrategy; import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableMap; +import com.google.common.collect.Iterables; import java.util.ArrayList; import java.util.List; +import java.util.Objects; +import org.junit.After; import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; @@ -38,14 +52,29 @@ public class FrameAggregatorTest { */ private List releasedFrameTimestamps; - private ArrayList> outputFrames; + private ArrayList> outputFrames; private ArrayList flushedSequences; + private HardwareBuffer placeholderBuffer; @Before public void setUp() { releasedFrameTimestamps = new ArrayList<>(); outputFrames = new ArrayList<>(); flushedSequences = new ArrayList<>(); + placeholderBuffer = + HardwareBuffer.create( + /* width= */ 16, + /* height= */ 16, + /* format= */ HardwareBuffer.RGBA_8888, + /* layers= */ 1, + /* usage= */ 0); + } + + @After + public void tearDown() { + if (placeholderBuffer != null) { + placeholderBuffer.close(); + } } @Test @@ -55,7 +84,8 @@ public void constructor_withZeroSequences_throwsIllegalArgumentException() { () -> new FrameAggregator( /* numSequences= */ 0, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add)); } @@ -66,7 +96,8 @@ public void constructor_withNegativeNumSequences_throwsIllegalArgumentException( () -> new FrameAggregator( /* numSequences= */ -1, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add)); } @@ -75,9 +106,10 @@ public void queueFrame_withInvalidSequenceIndex_throwsIllegalArgumentException() FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); - HardwareBufferFrame frame = + AsyncFrame frame = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); assertThrows(IllegalArgumentException.class, () -> frameAggregator.queueFrame(frame, 2)); @@ -88,12 +120,13 @@ public void queueFrame_singleSequence_passesFramesThrough() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 1, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 1); - HardwareBufferFrame frame1 = + AsyncFrame frame1 = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); - HardwareBufferFrame frame2 = + AsyncFrame frame2 = createFrame(/* presentationTimeUs= */ 200, /* sequencePresentationTimeUs= */ 200); frameAggregator.queueFrame(frame1, /* sequenceIndex= */ 0); @@ -109,12 +142,13 @@ public void queueFrame_waitsForAllSequencesBeforeAggregating() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 2); - HardwareBufferFrame primaryFrame = + AsyncFrame primaryFrame = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); - HardwareBufferFrame secondaryFrame = + AsyncFrame secondaryFrame = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); frameAggregator.queueFrame(primaryFrame, /* sequenceIndex= */ 0); @@ -125,29 +159,31 @@ public void queueFrame_waitsForAllSequencesBeforeAggregating() { frameAggregator.queueFrame(secondaryFrame, /* sequenceIndex= */ 1); assertThat(outputFrames).hasSize(1); - List aggregatedPacket = outputFrames.get(0); + List aggregatedPacket = outputFrames.get(0); assertThat(aggregatedPacket).hasSize(2); - assertThat(aggregatedPacket.get(0).presentationTimeUs) - .isEqualTo(primaryFrame.presentationTimeUs); - assertThat(aggregatedPacket.get(1).presentationTimeUs) - .isEqualTo(secondaryFrame.presentationTimeUs); + assertThat(getPresentationTimeUs(aggregatedPacket.get(0))) + .isEqualTo(getPresentationTimeUs(primaryFrame)); + assertThat(getPresentationTimeUs(aggregatedPacket.get(1))) + .isEqualTo(getPresentationTimeUs(secondaryFrame)); } @Test - public void queueFrame_dropsSecondaryFramesWithEarlierPresentationTimeUs() { + public void queueFrame_retentionDisabled_dropsSecondaryFramesWithEarlierPresentationTimeUs() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); - registerAllSequences(frameAggregator, /* numSequences= */ 2); - HardwareBufferFrame primaryFrame = + registerAllSequences( + frameAggregator, /* numSequences= */ 2, STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET); + AsyncFrame primaryFrame = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); - HardwareBufferFrame secondaryFrame1 = + AsyncFrame secondaryFrame1 = createFrame(/* presentationTimeUs= */ 50, /* sequencePresentationTimeUs= */ 50); - HardwareBufferFrame secondaryFrame2 = + AsyncFrame secondaryFrame2 = createFrame(/* presentationTimeUs= */ 80, /* sequencePresentationTimeUs= */ 80); - HardwareBufferFrame secondaryFrame3 = + AsyncFrame secondaryFrame3 = createFrame(/* presentationTimeUs= */ 110, /* sequencePresentationTimeUs= */ 110); frameAggregator.queueFrame(secondaryFrame1, /* sequenceIndex= */ 1); @@ -156,18 +192,19 @@ public void queueFrame_dropsSecondaryFramesWithEarlierPresentationTimeUs() { // Frames before the primary frame should be dropped and released. assertThat(releasedFrameTimestamps) - .containsExactly(secondaryFrame1.presentationTimeUs, secondaryFrame2.presentationTimeUs); + .containsExactly( + getPresentationTimeUs(secondaryFrame1), getPresentationTimeUs(secondaryFrame2)); assertThat(outputFrames).isEmpty(); frameAggregator.queueFrame(secondaryFrame3, /* sequenceIndex= */ 1); assertThat(outputFrames).hasSize(1); - List aggregatedPacket = outputFrames.get(0); + List aggregatedPacket = outputFrames.get(0); assertThat(aggregatedPacket).hasSize(2); - assertThat(aggregatedPacket.get(0).presentationTimeUs) - .isEqualTo(primaryFrame.presentationTimeUs); - assertThat(aggregatedPacket.get(1).presentationTimeUs) - .isEqualTo(secondaryFrame3.presentationTimeUs); + assertThat(getPresentationTimeUs(aggregatedPacket.get(0))) + .isEqualTo(getPresentationTimeUs(primaryFrame)); + assertThat(getPresentationTimeUs(aggregatedPacket.get(1))) + .isEqualTo(getPresentationTimeUs(secondaryFrame3)); } @Test @@ -175,16 +212,17 @@ public void queueFrame_selectsSecondaryFrameWithEqualPresentationTimeUs() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 2); - HardwareBufferFrame primaryFrame = + AsyncFrame primaryFrame = createFrame(/* presentationTimeUs= */ 200, /* sequencePresentationTimeUs= */ 200); - HardwareBufferFrame secondaryFrame1 = + AsyncFrame secondaryFrame1 = createFrame(/* presentationTimeUs= */ 199, /* sequencePresentationTimeUs= */ 199); - HardwareBufferFrame secondaryFrame2 = + AsyncFrame secondaryFrame2 = createFrame(/* presentationTimeUs= */ 200, /* sequencePresentationTimeUs= */ 200); - HardwareBufferFrame secondaryFrame3 = + AsyncFrame secondaryFrame3 = createFrame(/* presentationTimeUs= */ 201, /* sequencePresentationTimeUs= */ 201); frameAggregator.queueFrame(primaryFrame, /* sequenceIndex= */ 0); @@ -193,30 +231,32 @@ public void queueFrame_selectsSecondaryFrameWithEqualPresentationTimeUs() { frameAggregator.queueFrame(secondaryFrame3, /* sequenceIndex= */ 1); assertThat(outputFrames).hasSize(1); - List aggregatedPacket = outputFrames.get(0); + List aggregatedPacket = outputFrames.get(0); assertThat(aggregatedPacket).hasSize(2); - assertThat(aggregatedPacket.get(0).presentationTimeUs) - .isEqualTo(primaryFrame.presentationTimeUs); - assertThat(aggregatedPacket.get(1).presentationTimeUs) - .isEqualTo(secondaryFrame2.presentationTimeUs); - assertThat(releasedFrameTimestamps).containsExactly(secondaryFrame1.presentationTimeUs); + assertThat(getPresentationTimeUs(aggregatedPacket.get(0))) + .isEqualTo(getPresentationTimeUs(primaryFrame)); + assertThat(getPresentationTimeUs(aggregatedPacket.get(1))) + .isEqualTo(getPresentationTimeUs(secondaryFrame2)); + assertThat(releasedFrameTimestamps).containsExactly(getPresentationTimeUs(secondaryFrame1)); } @Test - public void queueFrame_selectsSecondaryFrameWithGreaterPresentationTimeUs() { + public void queueFrame_retentionDisabled_selectsSecondaryFrameWithGreaterPresentationTimeUs() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); - registerAllSequences(frameAggregator, /* numSequences= */ 2); - HardwareBufferFrame primaryFrame = + registerAllSequences( + frameAggregator, /* numSequences= */ 2, STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET); + AsyncFrame primaryFrame = createFrame(/* presentationTimeUs= */ 200, /* sequencePresentationTimeUs= */ 200); - HardwareBufferFrame secondaryFrame1 = + AsyncFrame secondaryFrame1 = createFrame(/* presentationTimeUs= */ 199, /* sequencePresentationTimeUs= */ 199); - HardwareBufferFrame secondaryFrame2 = + AsyncFrame secondaryFrame2 = createFrame(/* presentationTimeUs= */ 201, /* sequencePresentationTimeUs= */ 201); - HardwareBufferFrame secondaryFrame3 = + AsyncFrame secondaryFrame3 = createFrame(/* presentationTimeUs= */ 202, /* sequencePresentationTimeUs= */ 202); frameAggregator.queueFrame(primaryFrame, /* sequenceIndex= */ 0); @@ -225,13 +265,158 @@ public void queueFrame_selectsSecondaryFrameWithGreaterPresentationTimeUs() { frameAggregator.queueFrame(secondaryFrame3, /* sequenceIndex= */ 1); assertThat(outputFrames).hasSize(1); - List aggregatedPacket = outputFrames.get(0); + List aggregatedPacket = outputFrames.get(0); + assertThat(aggregatedPacket).hasSize(2); + assertThat(getPresentationTimeUs(aggregatedPacket.get(0))) + .isEqualTo(getPresentationTimeUs(primaryFrame)); + assertThat(getPresentationTimeUs(aggregatedPacket.get(1))) + .isEqualTo(getPresentationTimeUs(secondaryFrame2)); + assertThat(releasedFrameTimestamps).containsExactly(getPresentationTimeUs(secondaryFrame1)); + } + + @Test + public void queueFrame_previousFrameCloserThanNextCandidate_selectsPreviousFrame() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 2); + AsyncFrame primaryFrame = + createFrame(/* presentationTimeUs= */ 200, /* sequencePresentationTimeUs= */ 200); + // 199 is 1us before the target, 205 is 5us after it. + AsyncFrame secondaryFrameBefore = + createFrame( + /* presentationTimeUs= */ 199, + /* sequencePresentationTimeUs= */ 199, + /* sequenceIndex= */ 1); + AsyncFrame secondaryFrameAfter = + createFrame( + /* presentationTimeUs= */ 205, + /* sequencePresentationTimeUs= */ 205, + /* sequenceIndex= */ 1); + + frameAggregator.queueFrame(primaryFrame, /* sequenceIndex= */ 0); + // The earlier frame is retained rather than matched, because a closer frame might follow. + frameAggregator.queueFrame(secondaryFrameBefore, /* sequenceIndex= */ 1); + frameAggregator.queueFrame(secondaryFrameAfter, /* sequenceIndex= */ 1); + + assertThat(outputFrames).hasSize(1); + List aggregatedPacket = outputFrames.get(0); assertThat(aggregatedPacket).hasSize(2); - assertThat(aggregatedPacket.get(0).presentationTimeUs) - .isEqualTo(primaryFrame.presentationTimeUs); - assertThat(aggregatedPacket.get(1).presentationTimeUs) - .isEqualTo(secondaryFrame2.presentationTimeUs); - assertThat(releasedFrameTimestamps).containsExactly(secondaryFrame1.presentationTimeUs); + assertThat(getPresentationTimeUs(aggregatedPacket.get(0))) + .isEqualTo(getPresentationTimeUs(primaryFrame)); + assertThat(getPresentationTimeUs(aggregatedPacket.get(1))) + .isEqualTo(getPresentationTimeUs(secondaryFrameBefore)); + assertThat(releasedFrameTimestamps).isEmpty(); + } + + @Test + public void queueFrame_framesWithEqualDistanceToTarget_selectsPreviousFrame() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 2); + AsyncFrame primaryFrame = + createFrame(/* presentationTimeUs= */ 50, /* sequencePresentationTimeUs= */ 50); + // Both 40 and 60 have an absolute distance of 10 from target 50. + AsyncFrame secondaryFramePrevious = + createFrame( + /* presentationTimeUs= */ 40, + /* sequencePresentationTimeUs= */ 40, + /* sequenceIndex= */ 1); + AsyncFrame secondaryFrameNext = + createFrame( + /* presentationTimeUs= */ 60, + /* sequencePresentationTimeUs= */ 60, + /* sequenceIndex= */ 1); + + frameAggregator.queueFrame(primaryFrame, /* sequenceIndex= */ 0); + frameAggregator.queueFrame(secondaryFramePrevious, /* sequenceIndex= */ 1); + frameAggregator.queueFrame(secondaryFrameNext, /* sequenceIndex= */ 1); + + assertThat(outputFrames).hasSize(1); + List outputPacket = outputFrames.get(0); + assertThat(getPresentationTimeUs(outputPacket.get(1))).isEqualTo(40); + } + + @Test + public void queueFrame_noCandidateAtOrAfterTargetTime_retainsPreviousFrame() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 2); + AsyncFrame primaryFrame0 = + createFrame(/* presentationTimeUs= */ 40, /* sequencePresentationTimeUs= */ 40); + AsyncFrame secondaryFrame = + createFrame( + /* presentationTimeUs= */ 40, + /* sequencePresentationTimeUs= */ 40, + /* sequenceIndex= */ 1); + AsyncFrame primaryFrame1 = + createFrame(/* presentationTimeUs= */ 50, /* sequencePresentationTimeUs= */ 50); + + frameAggregator.queueFrame(primaryFrame0, /* sequenceIndex= */ 0); + frameAggregator.queueFrame(secondaryFrame, /* sequenceIndex= */ 1); + frameAggregator.queueFrame(primaryFrame1, /* sequenceIndex= */ 0); + + // Primary 40 and secondary 40 are output. When primary advances to 50, secondary frame 40 is + // still retained in the queue awaiting future candidates and not released. + assertThat(outputFrames).hasSize(1); + assertThat(getPresentationTimeUs(outputFrames.get(0).get(1))).isEqualTo(40); + assertThat(releasedFrameTimestamps).doesNotContain(40L); + } + + @Test + public void + queueFrame_multiplePastFramesWithCloserCandidate_discardsOlderFramesAndMatchesCandidate() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 2); + AsyncFrame primaryFrame = + createFrame(/* presentationTimeUs= */ 35, /* sequencePresentationTimeUs= */ 35); + AsyncFrame secondaryFrame10 = + createFrame( + /* presentationTimeUs= */ 10, + /* sequencePresentationTimeUs= */ 10, + /* sequenceIndex= */ 1); + AsyncFrame secondaryFrame20 = + createFrame( + /* presentationTimeUs= */ 20, + /* sequencePresentationTimeUs= */ 20, + /* sequenceIndex= */ 1); + AsyncFrame secondaryFrame30 = + createFrame( + /* presentationTimeUs= */ 30, + /* sequencePresentationTimeUs= */ 30, + /* sequenceIndex= */ 1); + AsyncFrame secondaryFrame36 = + createFrame( + /* presentationTimeUs= */ 36, + /* sequencePresentationTimeUs= */ 36, + /* sequenceIndex= */ 1); + + frameAggregator.queueFrame(primaryFrame, /* sequenceIndex= */ 0); + frameAggregator.queueFrame(secondaryFrame10, /* sequenceIndex= */ 1); + frameAggregator.queueFrame(secondaryFrame20, /* sequenceIndex= */ 1); + frameAggregator.queueFrame(secondaryFrame30, /* sequenceIndex= */ 1); + frameAggregator.queueFrame(secondaryFrame36, /* sequenceIndex= */ 1); + + assertThat(outputFrames).hasSize(1); + List outputPacket = outputFrames.get(0); + assertThat(getPresentationTimeUs(outputPacket.get(1))).isEqualTo(36); + assertThat(releasedFrameTimestamps).containsAtLeast(10L, 20L, 30L); } @Test @@ -239,16 +424,17 @@ public void queueFrame_upsampling_reusesFutureSecondaryFrame() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 2); - HardwareBufferFrame primaryFrame1 = + AsyncFrame primaryFrame1 = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); - HardwareBufferFrame primaryFrame2 = + AsyncFrame primaryFrame2 = createFrame(/* presentationTimeUs= */ 101, /* sequencePresentationTimeUs= */ 101); - HardwareBufferFrame primaryFrame3 = + AsyncFrame primaryFrame3 = createFrame(/* presentationTimeUs= */ 102, /* sequencePresentationTimeUs= */ 102); - HardwareBufferFrame secondaryFrame = + AsyncFrame secondaryFrame = createFrame(/* presentationTimeUs= */ 102, /* sequencePresentationTimeUs= */ 102); frameAggregator.queueFrame(primaryFrame1, 0); @@ -261,29 +447,31 @@ public void queueFrame_upsampling_reusesFutureSecondaryFrame() { frameAggregator.queueFrame(secondaryFrame, 1); assertThat(outputFrames).hasSize(3); - assertThat(outputFrames.get(0).get(1).presentationTimeUs) - .isEqualTo(secondaryFrame.presentationTimeUs); - assertThat(outputFrames.get(1).get(1).presentationTimeUs) - .isEqualTo(secondaryFrame.presentationTimeUs); - assertThat(outputFrames.get(2).get(1).presentationTimeUs) - .isEqualTo(secondaryFrame.presentationTimeUs); + assertThat(getPresentationTimeUs(outputFrames.get(0).get(1))) + .isEqualTo(getPresentationTimeUs(secondaryFrame)); + assertThat(getPresentationTimeUs(outputFrames.get(1).get(1))) + .isEqualTo(getPresentationTimeUs(secondaryFrame)); + assertThat(getPresentationTimeUs(outputFrames.get(2).get(1))) + .isEqualTo(getPresentationTimeUs(secondaryFrame)); } @Test - public void queueFrame_downsampling_dropsIntermediateFrames() { + public void queueFrame_downsamplingAndRetentionDisabled_dropsIntermediateFrames() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); - registerAllSequences(frameAggregator, /* numSequences= */ 2); - HardwareBufferFrame primaryFrame = + registerAllSequences( + frameAggregator, /* numSequences= */ 2, STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET); + AsyncFrame primaryFrame = createFrame(/* presentationTimeUs= */ 102, /* sequencePresentationTimeUs= */ 102); - HardwareBufferFrame secondaryFrame1 = + AsyncFrame secondaryFrame1 = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); - HardwareBufferFrame secondaryFrame2 = + AsyncFrame secondaryFrame2 = createFrame(/* presentationTimeUs= */ 101, /* sequencePresentationTimeUs= */ 101); - HardwareBufferFrame secondaryFrame3 = + AsyncFrame secondaryFrame3 = createFrame(/* presentationTimeUs= */ 102, /* sequencePresentationTimeUs= */ 102); frameAggregator.queueFrame(primaryFrame, 0); @@ -293,16 +481,17 @@ public void queueFrame_downsampling_dropsIntermediateFrames() { // Frames 100 and 101 are strictly in the past for target 102. They should be dropped. assertThat(outputFrames).isEmpty(); assertThat(releasedFrameTimestamps) - .containsExactly(secondaryFrame1.presentationTimeUs, secondaryFrame2.presentationTimeUs); + .containsExactly( + getPresentationTimeUs(secondaryFrame1), getPresentationTimeUs(secondaryFrame2)); frameAggregator.queueFrame(secondaryFrame3, 1); assertThat(outputFrames).hasSize(1); - List aggregatedPacket = outputFrames.get(0); - assertThat(aggregatedPacket.get(0).presentationTimeUs) - .isEqualTo(primaryFrame.presentationTimeUs); - assertThat(aggregatedPacket.get(1).presentationTimeUs) - .isEqualTo(secondaryFrame3.presentationTimeUs); + List aggregatedPacket = outputFrames.get(0); + assertThat(getPresentationTimeUs(aggregatedPacket.get(0))) + .isEqualTo(getPresentationTimeUs(primaryFrame)); + assertThat(getPresentationTimeUs(aggregatedPacket.get(1))) + .isEqualTo(getPresentationTimeUs(secondaryFrame3)); } @Test @@ -310,14 +499,15 @@ public void queueFrame_mismatchedDuration_omitsEndedSecondary() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 2); - HardwareBufferFrame primaryFrame1 = + AsyncFrame primaryFrame1 = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); - HardwareBufferFrame primaryFrame2 = + AsyncFrame primaryFrame2 = createFrame(/* presentationTimeUs= */ 200, /* sequencePresentationTimeUs= */ 200); - HardwareBufferFrame secondaryFrame = + AsyncFrame secondaryFrame = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); frameAggregator.queueFrame(primaryFrame1, 0); @@ -339,7 +529,8 @@ public void queueFrame_throughputStress_1fpsSecondary() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 2); // Queue one second of primary frames (30 packets). @@ -360,18 +551,20 @@ public void queueFrame_throughputStress_1fpsSecondary() { assertThat(outputFrames).hasSize(30); // Every primary frame in the first second paired with the 1s secondary frame. for (int i = 0; i < 30; i++) { - assertThat(outputFrames.get(i).get(1).presentationTimeUs).isEqualTo(1000000L); + assertThat(getPresentationTimeUs(outputFrames.get(i).get(1))).isEqualTo(1000000L); } } @Test - public void queueFrame_interleavedTimestamps_aggregatesCorrectly() { + public void queueFrame_interleavedTimestampsAndRetentionDisabled_aggregatesCorrectly() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); - registerAllSequences(frameAggregator, /* numSequences= */ 2); + registerAllSequences( + frameAggregator, /* numSequences= */ 2, STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET); // Simulate ~30fps for primary sequence long[] primaryTimestampsUs = {0, 33_333, 66_667, 100_000, 133_333, 166_667, 200_000}; // Simulate ~20fps for secondary sequence @@ -411,20 +604,80 @@ public void queueFrame_interleavedTimestamps_aggregatesCorrectly() { assertThat(outputFrames).hasSize(primaryTimestampsUs.length); for (int i = 0; i < primaryTimestampsUs.length; i++) { - List aggregatedPacket = outputFrames.get(i); - assertThat(aggregatedPacket.get(0).presentationTimeUs).isEqualTo(primaryTimestampsUs[i]); - assertThat(aggregatedPacket.get(1).presentationTimeUs).isEqualTo(expectedSecondaryMatches[i]); + List aggregatedPacket = outputFrames.get(i); + assertThat(getPresentationTimeUs(aggregatedPacket.get(0))).isEqualTo(primaryTimestampsUs[i]); + assertThat(getPresentationTimeUs(aggregatedPacket.get(1))) + .isEqualTo(expectedSecondaryMatches[i]); } } @Test - public void queueFrame_withStalls_waitsForMissingFrames() { + public void queueFrame_interleavedTimestamps_selectsClosestSecondaryFrames() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 2); + // Simulate ~30fps for primary sequence + long[] primaryTimestampsUs = {0, 33_333, 66_667, 100_000, 133_333, 166_667, 200_000}; + // Simulate ~20fps for secondary sequence + long[] secondaryTimestampsUs = {0, 50_000, 100_000, 150_000, 200_000}; + // Each primary frame is matched with the secondary frame closest to it in time, which is the + // earlier one when both neighbours are equidistant. + long[] expectedSecondaryMatches = {0, 50_000, 50_000, 100_000, 150_000, 150_000, 200_000}; + int primaryIndex = 0; + int secondaryIndex = 0; + + // Iterate through both lists and queue the frames in chronological order + // simulating two sequences working normally. + while (primaryIndex < primaryTimestampsUs.length + || secondaryIndex < secondaryTimestampsUs.length) { + long nextPrimaryUs = + primaryIndex < primaryTimestampsUs.length + ? primaryTimestampsUs[primaryIndex] + : Long.MAX_VALUE; + long nextSecondaryUs = + secondaryIndex < secondaryTimestampsUs.length + ? secondaryTimestampsUs[secondaryIndex] + : Long.MAX_VALUE; + if (nextPrimaryUs <= nextSecondaryUs) { + frameAggregator.queueFrame( + createFrame( + /* presentationTimeUs= */ nextPrimaryUs, + /* sequencePresentationTimeUs= */ nextPrimaryUs), + /* sequenceIndex= */ 0); + primaryIndex++; + } else { + frameAggregator.queueFrame( + createFrame( + /* presentationTimeUs= */ nextSecondaryUs, + /* sequencePresentationTimeUs= */ nextSecondaryUs), + /* sequenceIndex= */ 1); + secondaryIndex++; + } + } + + assertThat(outputFrames).hasSize(primaryTimestampsUs.length); + for (int i = 0; i < primaryTimestampsUs.length; i++) { + List aggregatedPacket = outputFrames.get(i); + assertThat(getPresentationTimeUs(aggregatedPacket.get(0))).isEqualTo(primaryTimestampsUs[i]); + assertThat(getPresentationTimeUs(aggregatedPacket.get(1))) + .isEqualTo(expectedSecondaryMatches[i]); + } + } + + @Test + public void queueFrame_withStallsAndRetentionDisabled_waitsForMissingFrames() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences( + frameAggregator, /* numSequences= */ 2, STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET); long[] primaryUs = {0, 33_333, 66_667, 100_000}; long[] secondaryUs = {0, 50_000, 100_000}; @@ -480,37 +733,39 @@ public void queueFrame_withStalls_waitsForMissingFrames() { /* sequenceIndex= */ 0); assertThat(outputFrames).hasSize(4); - assertThat(outputFrames.get(0).get(1).presentationTimeUs).isEqualTo(secondaryUs[0]); - assertThat(outputFrames.get(1).get(1).presentationTimeUs).isEqualTo(secondaryUs[1]); - assertThat(outputFrames.get(2).get(1).presentationTimeUs).isEqualTo(secondaryUs[2]); - assertThat(outputFrames.get(3).get(1).presentationTimeUs).isEqualTo(secondaryUs[2]); + assertThat(getPresentationTimeUs(outputFrames.get(0).get(1))).isEqualTo(secondaryUs[0]); + assertThat(getPresentationTimeUs(outputFrames.get(1).get(1))).isEqualTo(secondaryUs[1]); + assertThat(getPresentationTimeUs(outputFrames.get(2).get(1))).isEqualTo(secondaryUs[2]); + assertThat(getPresentationTimeUs(outputFrames.get(3).get(1))).isEqualTo(secondaryUs[2]); } @Test - public void queueFrame_multiItemPrimarySequence_matchesUsingSequenceTime() { + public void queueFrame_multiItemPrimarySequenceAndRetentionDisabled_matchesUsingSequenceTime() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); - registerAllSequences(frameAggregator, /* numSequences= */ 2); + registerAllSequences( + frameAggregator, /* numSequences= */ 2, STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET); // Primary Sequence with two items: // Item 1: presentation time 0 -> 33_333, sequence time 0 -> 33_333 - HardwareBufferFrame primary1 = + AsyncFrame primary1 = createFrame(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 0); - HardwareBufferFrame primary2 = + AsyncFrame primary2 = createFrame(/* presentationTimeUs= */ 33_333, /* sequencePresentationTimeUs= */ 33_333); // Item 2: presentation time resets to 0 -> 33_333, sequence time continues 66_667 -> 100_000 - HardwareBufferFrame primary3 = + AsyncFrame primary3 = createFrame(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 66_667); - HardwareBufferFrame primary4 = + AsyncFrame primary4 = createFrame(/* presentationTimeUs= */ 33_333, /* sequencePresentationTimeUs= */ 100_000); // Secondary Sequence with one item - HardwareBufferFrame secondary1 = + AsyncFrame secondary1 = createFrame(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 0); - HardwareBufferFrame secondary2 = + AsyncFrame secondary2 = createFrame(/* presentationTimeUs= */ 50_000, /* sequencePresentationTimeUs= */ 50_000); - HardwareBufferFrame secondary3 = + AsyncFrame secondary3 = createFrame(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); frameAggregator.queueFrame(primary1, /* sequenceIndex= */ 0); @@ -521,51 +776,53 @@ public void queueFrame_multiItemPrimarySequence_matchesUsingSequenceTime() { frameAggregator.queueFrame(secondary1, /* sequenceIndex= */ 1); assertThat(outputFrames).hasSize(1); - assertThat(outputFrames.get(0).get(0).sequencePresentationTimeUs).isEqualTo(0); - assertThat(outputFrames.get(0).get(1).sequencePresentationTimeUs).isEqualTo(0); + assertThat(outputFrames.get(0).get(0).frame.getContentTimeUs()).isEqualTo(0); + assertThat(outputFrames.get(0).get(1).frame.getContentTimeUs()).isEqualTo(0); // Matches primary 33_333 with secondary 50_000 frameAggregator.queueFrame(secondary2, /* sequenceIndex= */ 1); assertThat(outputFrames).hasSize(2); - assertThat(outputFrames.get(1).get(0).sequencePresentationTimeUs).isEqualTo(33_333); - assertThat(outputFrames.get(1).get(1).sequencePresentationTimeUs).isEqualTo(50_000); + assertThat(outputFrames.get(1).get(0).frame.getContentTimeUs()).isEqualTo(33_333); + assertThat(outputFrames.get(1).get(1).frame.getContentTimeUs()).isEqualTo(50_000); // Matches primary 66_667 with secondary 100_000, and primary 100_000 with secondary 100_000 frameAggregator.queueFrame(secondary3, /* sequenceIndex= */ 1); assertThat(outputFrames).hasSize(4); - assertThat(outputFrames.get(2).get(0).sequencePresentationTimeUs).isEqualTo(66_667); - assertThat(outputFrames.get(2).get(1).sequencePresentationTimeUs).isEqualTo(100_000); - assertThat(outputFrames.get(3).get(0).sequencePresentationTimeUs).isEqualTo(100_000); - assertThat(outputFrames.get(3).get(1).sequencePresentationTimeUs).isEqualTo(100_000); + assertThat(outputFrames.get(2).get(0).frame.getContentTimeUs()).isEqualTo(66_667); + assertThat(outputFrames.get(2).get(1).frame.getContentTimeUs()).isEqualTo(100_000); + assertThat(outputFrames.get(3).get(0).frame.getContentTimeUs()).isEqualTo(100_000); + assertThat(outputFrames.get(3).get(1).frame.getContentTimeUs()).isEqualTo(100_000); } @Test - public void queueFrame_multiItemSecondarySequence_matchesUsingSequenceTime() { + public void queueFrame_multiItemSecondarySequenceAndRetentionDisabled_matchesUsingSequenceTime() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); - registerAllSequences(frameAggregator, /* numSequences= */ 2); + registerAllSequences( + frameAggregator, /* numSequences= */ 2, STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET); // Primary Sequence with one item spanning the whole duration - HardwareBufferFrame primary1 = + AsyncFrame primary1 = createFrame(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 0); - HardwareBufferFrame primary2 = + AsyncFrame primary2 = createFrame(/* presentationTimeUs= */ 50_000, /* sequencePresentationTimeUs= */ 50_000); - HardwareBufferFrame primary3 = + AsyncFrame primary3 = createFrame(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); // Secondary Sequence with two items: // Item 1: presentation time 0 -> 33_333, sequence time 0 -> 33_333 - HardwareBufferFrame secondary1 = + AsyncFrame secondary1 = createFrame(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 0); - HardwareBufferFrame secondary2 = + AsyncFrame secondary2 = createFrame(/* presentationTimeUs= */ 33_333, /* sequencePresentationTimeUs= */ 33_333); // Item 2: presentation time resets to 0 -> 33_333, sequence time continues 66_667 -> 100_000 - HardwareBufferFrame secondary3 = + AsyncFrame secondary3 = createFrame(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 66_667); - HardwareBufferFrame secondary4 = + AsyncFrame secondary4 = createFrame(/* presentationTimeUs= */ 33_333, /* sequencePresentationTimeUs= */ 100_000); // Queue primary frames @@ -576,8 +833,8 @@ public void queueFrame_multiItemSecondarySequence_matchesUsingSequenceTime() { frameAggregator.queueFrame(secondary1, /* sequenceIndex= */ 1); assertThat(outputFrames).hasSize(1); - assertThat(outputFrames.get(0).get(0).sequencePresentationTimeUs).isEqualTo(0); - assertThat(outputFrames.get(0).get(1).sequencePresentationTimeUs).isEqualTo(0); + assertThat(outputFrames.get(0).get(0).frame.getContentTimeUs()).isEqualTo(0); + assertThat(outputFrames.get(0).get(1).frame.getContentTimeUs()).isEqualTo(0); frameAggregator.queueFrame(secondary2, /* sequenceIndex= */ 1); frameAggregator.queueFrame(secondary3, /* sequenceIndex= */ 1); @@ -586,15 +843,63 @@ public void queueFrame_multiItemSecondarySequence_matchesUsingSequenceTime() { // secondary3 (66_667) >= primary2 (50_000), so it correctly matches. // If we relied on the reset presentation time, secondary3's `0` would be dropped! assertThat(outputFrames).hasSize(2); - assertThat(outputFrames.get(1).get(0).sequencePresentationTimeUs).isEqualTo(50_000); - assertThat(outputFrames.get(1).get(1).sequencePresentationTimeUs).isEqualTo(66_667); + assertThat(outputFrames.get(1).get(0).frame.getContentTimeUs()).isEqualTo(50_000); + assertThat(outputFrames.get(1).get(1).frame.getContentTimeUs()).isEqualTo(66_667); // Matches primary 100_000 with secondary 100_000 frameAggregator.queueFrame(secondary4, /* sequenceIndex= */ 1); assertThat(outputFrames).hasSize(3); - assertThat(outputFrames.get(2).get(0).sequencePresentationTimeUs).isEqualTo(100_000); - assertThat(outputFrames.get(2).get(1).sequencePresentationTimeUs).isEqualTo(100_000); + assertThat(outputFrames.get(2).get(0).frame.getContentTimeUs()).isEqualTo(100_000); + assertThat(outputFrames.get(2).get(1).frame.getContentTimeUs()).isEqualTo(100_000); + } + + @Test + public void queueFrame_multiItemSecondarySequence_matchesClosestUsingSequenceTime() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 2); + // Primary Sequence with one item spanning the whole duration + AsyncFrame primary1 = + createFrame(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 0); + AsyncFrame primary2 = + createFrame(/* presentationTimeUs= */ 50_000, /* sequencePresentationTimeUs= */ 50_000); + AsyncFrame primary3 = + createFrame(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); + // Secondary Sequence with two items: + // Item 1: presentation time 0 -> 33_333, sequence time 0 -> 33_333 + AsyncFrame secondary1 = + createFrame(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 0); + AsyncFrame secondary2 = + createFrame(/* presentationTimeUs= */ 33_333, /* sequencePresentationTimeUs= */ 33_333); + // Item 2: presentation time resets to 0 -> 33_333, sequence time continues 66_667 -> 100_000 + AsyncFrame secondary3 = + createFrame(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 66_667); + AsyncFrame secondary4 = + createFrame(/* presentationTimeUs= */ 33_333, /* sequencePresentationTimeUs= */ 100_000); + + frameAggregator.queueFrame(primary1, /* sequenceIndex= */ 0); + frameAggregator.queueFrame(primary2, /* sequenceIndex= */ 0); + frameAggregator.queueFrame(primary3, /* sequenceIndex= */ 0); + frameAggregator.queueFrame(secondary1, /* sequenceIndex= */ 1); + frameAggregator.queueFrame(secondary2, /* sequenceIndex= */ 1); + frameAggregator.queueFrame(secondary3, /* sequenceIndex= */ 1); + frameAggregator.queueFrame(secondary4, /* sequenceIndex= */ 1); + + assertThat(outputFrames).hasSize(3); + assertThat(outputFrames.get(0).get(0).frame.getContentTimeUs()).isEqualTo(0); + assertThat(outputFrames.get(0).get(1).frame.getContentTimeUs()).isEqualTo(0); + // secondary2 (33_333) and secondary3 (66_667) are equidistant from primary2 (50_000), so the + // earlier one is selected even though it precedes the primary frame. The selection spans the + // item boundary because it uses sequence time, not the presentation time that resets to 0. + assertThat(outputFrames.get(1).get(0).frame.getContentTimeUs()).isEqualTo(50_000); + assertThat(outputFrames.get(1).get(1).frame.getContentTimeUs()).isEqualTo(33_333); + assertThat(outputFrames.get(2).get(0).frame.getContentTimeUs()).isEqualTo(100_000); + assertThat(outputFrames.get(2).get(1).frame.getContentTimeUs()).isEqualTo(100_000); } @Test @@ -602,18 +907,19 @@ public void close_releasesAllHeldFrames() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 3, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 3); - HardwareBufferFrame frame0 = + AsyncFrame frame0 = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); - HardwareBufferFrame frame1 = + AsyncFrame frame1 = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); - HardwareBufferFrame frame2 = + AsyncFrame frame2 = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); - HardwareBufferFrame frame3 = + AsyncFrame frame3 = createFrame(/* presentationTimeUs= */ 150, /* sequencePresentationTimeUs= */ 150); - HardwareBufferFrame frame4 = + AsyncFrame frame4 = createFrame(/* presentationTimeUs= */ 150, /* sequencePresentationTimeUs= */ 150); frameAggregator.queueFrame(frame0, /* sequenceIndex= */ 0); @@ -623,25 +929,56 @@ public void close_releasesAllHeldFrames() { frameAggregator.queueFrame(frame4, /* sequenceIndex= */ 1); assertThat(outputFrames).hasSize(1); - List aggregatedPacket = outputFrames.get(0); + List aggregatedPacket = outputFrames.get(0); assertThat(aggregatedPacket).hasSize(3); - assertThat(aggregatedPacket.get(0).presentationTimeUs).isEqualTo(frame0.presentationTimeUs); - assertThat(aggregatedPacket.get(1).presentationTimeUs).isEqualTo(frame1.presentationTimeUs); - assertThat(aggregatedPacket.get(2).presentationTimeUs).isEqualTo(frame2.presentationTimeUs); + assertThat(getPresentationTimeUs(aggregatedPacket.get(0))) + .isEqualTo(getPresentationTimeUs(frame0)); + assertThat(getPresentationTimeUs(aggregatedPacket.get(1))) + .isEqualTo(getPresentationTimeUs(frame1)); + assertThat(getPresentationTimeUs(aggregatedPacket.get(2))) + .isEqualTo(getPresentationTimeUs(frame2)); frameAggregator.close(); assertThat(releasedFrameTimestamps) - .containsExactly(frame3.presentationTimeUs, frame4.presentationTimeUs); + .containsExactly(getPresentationTimeUs(frame3), getPresentationTimeUs(frame4)); assertThat(outputFrames).hasSize(1); } + @Test + public void close_withRetainedPreviousFrame_releasesRetainedFrame() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 2); + AsyncFrame primaryFrame = + createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); + AsyncFrame secondaryFrame = + createFrame( + /* presentationTimeUs= */ 30, + /* sequencePresentationTimeUs= */ 30, + /* sequenceIndex= */ 1); + // The secondary frame is retained rather than matched, because a closer candidate might still + // arrive. + frameAggregator.queueFrame(primaryFrame, /* sequenceIndex= */ 0); + frameAggregator.queueFrame(secondaryFrame, /* sequenceIndex= */ 1); + + frameAggregator.close(); + + assertThat(outputFrames).isEmpty(); + assertThat(releasedFrameTimestamps).containsExactly(100L, 30L); + } + @Test public void registerSequence_afterClose_throwsIllegalStateException() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 1, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); frameAggregator.close(); @@ -649,7 +986,8 @@ public void registerSequence_afterClose_throwsIllegalStateException() { assertThrows( IllegalStateException.class, () -> - frameAggregator.registerSequence(/* sequenceIndex= */ 0, /* shouldAggregate= */ true)); + frameAggregator.registerSequence( + /* sequenceIndex= */ 0, STRATEGY_MATCH_FRAME_CLOSEST_TO_TARGET)); } @Test @@ -657,18 +995,19 @@ public void queueFrame_afterClose_releasesFrameImmediately() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 1, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 1); frameAggregator.close(); - HardwareBufferFrame frame = + AsyncFrame frame = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); frameAggregator.queueFrame(frame, /* sequenceIndex= */ 0); assertThat(outputFrames).isEmpty(); - assertThat(releasedFrameTimestamps).containsExactly(frame.presentationTimeUs); + assertThat(releasedFrameTimestamps).containsExactly(getPresentationTimeUs(frame)); } @Test @@ -676,7 +1015,8 @@ public void queueEndOfStream_afterClose_isIgnored() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 1, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 1); frameAggregator.close(); @@ -691,7 +1031,8 @@ public void flush_afterClose_isIgnored() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 1, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 1); frameAggregator.close(); @@ -706,45 +1047,94 @@ public void flush_thenQueueFramesWithEarlierPresentationTimeUs_aggregatesCorrect FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 2); - HardwareBufferFrame primaryFrame1 = + AsyncFrame primaryFrame1 = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); - HardwareBufferFrame secondaryFrame1 = + AsyncFrame secondaryFrame1 = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); frameAggregator.queueFrame(primaryFrame1, /* sequenceIndex= */ 0); frameAggregator.queueFrame(secondaryFrame1, /* sequenceIndex= */ 1); assertThat(outputFrames).hasSize(1); - List aggregatedPacket1 = outputFrames.get(0); + List aggregatedPacket1 = outputFrames.get(0); assertThat(aggregatedPacket1).hasSize(2); - assertThat(aggregatedPacket1.get(0).presentationTimeUs) - .isEqualTo(primaryFrame1.presentationTimeUs); - assertThat(aggregatedPacket1.get(1).presentationTimeUs) - .isEqualTo(secondaryFrame1.presentationTimeUs); + assertThat(getPresentationTimeUs(aggregatedPacket1.get(0))) + .isEqualTo(getPresentationTimeUs(primaryFrame1)); + assertThat(getPresentationTimeUs(aggregatedPacket1.get(1))) + .isEqualTo(getPresentationTimeUs(secondaryFrame1)); frameAggregator.flush(/* sequenceIndex= */ 0); frameAggregator.flush(/* sequenceIndex= */ 1); assertThat(flushedSequences).containsExactly(0, 1).inOrder(); - HardwareBufferFrame primaryFrame2 = + AsyncFrame primaryFrame2 = createFrame(/* presentationTimeUs= */ 50, /* sequencePresentationTimeUs= */ 50); - HardwareBufferFrame secondaryFrame2 = + AsyncFrame secondaryFrame2 = createFrame(/* presentationTimeUs= */ 50, /* sequencePresentationTimeUs= */ 50); frameAggregator.queueFrame(primaryFrame2, /* sequenceIndex= */ 0); frameAggregator.queueFrame(secondaryFrame2, /* sequenceIndex= */ 1); assertThat(outputFrames).hasSize(2); - List aggregatedPacket2 = outputFrames.get(1); + List aggregatedPacket2 = outputFrames.get(1); assertThat(aggregatedPacket2).hasSize(2); - assertThat(aggregatedPacket2.get(0).presentationTimeUs) - .isEqualTo(primaryFrame2.presentationTimeUs); - assertThat(aggregatedPacket2.get(1).presentationTimeUs) - .isEqualTo(secondaryFrame2.presentationTimeUs); + assertThat(getPresentationTimeUs(aggregatedPacket2.get(0))) + .isEqualTo(getPresentationTimeUs(primaryFrame2)); + assertThat(getPresentationTimeUs(aggregatedPacket2.get(1))) + .isEqualTo(getPresentationTimeUs(secondaryFrame2)); + } + + @Test + public void flush_withQueuedFrames_releasesQueuedFrames() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 2); + List secondaryFrames = + createFrameList(/* numFrames= */ 3, /* frameRate= */ 30, /* sequenceIndex= */ 1); + secondaryFrames.forEach(frame -> frameAggregator.queueFrame(frame, /* sequenceIndex= */ 1)); + + frameAggregator.flush(/* sequenceIndex= */ 1); + + assertThat(outputFrames).isEmpty(); + assertThat(releasedFrameTimestamps).containsExactly(0L, 33_333L, 66_667L); + } + + @Test + public void flush_withRetainedPreviousFrame_releasesRetainedFrame() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 2); + AsyncFrame primaryFrame = + createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); + AsyncFrame secondaryFrame = + createFrame( + /* presentationTimeUs= */ 30, + /* sequencePresentationTimeUs= */ 30, + /* sequenceIndex= */ 1); + + // The secondary frame is retained rather than matched, because a closer candidate might still + // arrive. + frameAggregator.queueFrame(primaryFrame, /* sequenceIndex= */ 0); + frameAggregator.queueFrame(secondaryFrame, /* sequenceIndex= */ 1); + + frameAggregator.flush(/* sequenceIndex= */ 0); + frameAggregator.flush(/* sequenceIndex= */ 1); + + assertThat(outputFrames).isEmpty(); + assertThat(releasedFrameTimestamps).containsExactly(100L, 30L); } @Test @@ -752,7 +1142,8 @@ public void queueEndOfStream_withInvalidSequenceIndex_throwsIllegalArgumentExcep FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 2); @@ -764,10 +1155,11 @@ public void queueEndOfStream_secondaryStream_aggregatesWithoutSecondaryFrame() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 2); - HardwareBufferFrame primaryFrame = + AsyncFrame primaryFrame = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); frameAggregator.queueFrame(primaryFrame, /* sequenceIndex= */ 0); @@ -785,12 +1177,13 @@ public void queueEndOfStream_oneOfTwoSecondaryStreams_aggregatesWithAvailableFra FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 3, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 3); - HardwareBufferFrame primaryFrame = + AsyncFrame primaryFrame = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); - HardwareBufferFrame secondaryFrame1 = + AsyncFrame secondaryFrame1 = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); frameAggregator.queueFrame(primaryFrame, /* sequenceIndex= */ 0); @@ -801,12 +1194,12 @@ public void queueEndOfStream_oneOfTwoSecondaryStreams_aggregatesWithAvailableFra frameAggregator.queueEndOfStream(/* sequenceIndex= */ 2); assertThat(outputFrames).hasSize(1); - List aggregatedPacket = outputFrames.get(0); + List aggregatedPacket = outputFrames.get(0); assertThat(aggregatedPacket).hasSize(2); - assertThat(aggregatedPacket.get(0).presentationTimeUs) - .isEqualTo(primaryFrame.presentationTimeUs); - assertThat(aggregatedPacket.get(1).presentationTimeUs) - .isEqualTo(secondaryFrame1.presentationTimeUs); + assertThat(getPresentationTimeUs(aggregatedPacket.get(0))) + .isEqualTo(getPresentationTimeUs(primaryFrame)); + assertThat(getPresentationTimeUs(aggregatedPacket.get(1))) + .isEqualTo(getPresentationTimeUs(secondaryFrame1)); } @Test @@ -814,14 +1207,15 @@ public void queueEndOfStream_thenFlush_resetsEndedState() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 2); - HardwareBufferFrame primaryFrame1 = + AsyncFrame primaryFrame1 = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); - HardwareBufferFrame primaryFrame2 = + AsyncFrame primaryFrame2 = createFrame(/* presentationTimeUs= */ 200, /* sequencePresentationTimeUs= */ 200); - HardwareBufferFrame secondaryFrame = + AsyncFrame secondaryFrame = createFrame(/* presentationTimeUs= */ 200, /* sequencePresentationTimeUs= */ 200); frameAggregator.queueFrame(primaryFrame1, /* sequenceIndex= */ 0); @@ -838,12 +1232,12 @@ public void queueEndOfStream_thenFlush_resetsEndedState() { frameAggregator.queueFrame(secondaryFrame, /* sequenceIndex= */ 1); assertThat(outputFrames).hasSize(2); - List aggregatedPacket = outputFrames.get(1); + List aggregatedPacket = outputFrames.get(1); assertThat(aggregatedPacket).hasSize(2); - assertThat(aggregatedPacket.get(0).presentationTimeUs) - .isEqualTo(primaryFrame2.presentationTimeUs); - assertThat(aggregatedPacket.get(1).presentationTimeUs) - .isEqualTo(secondaryFrame.presentationTimeUs); + assertThat(getPresentationTimeUs(aggregatedPacket.get(0))) + .isEqualTo(getPresentationTimeUs(primaryFrame2)); + assertThat(getPresentationTimeUs(aggregatedPacket.get(1))) + .isEqualTo(getPresentationTimeUs(secondaryFrame)); assertThat(flushedSequences).containsExactly(1); } @@ -853,14 +1247,15 @@ public void queueEndOfStream_secondaryStreamAlreadyHasFrames_usesFramesBeforeSki FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 2); - HardwareBufferFrame primaryFrame1 = + AsyncFrame primaryFrame1 = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); - HardwareBufferFrame primaryFrame2 = + AsyncFrame primaryFrame2 = createFrame(/* presentationTimeUs= */ 200, /* sequencePresentationTimeUs= */ 200); - HardwareBufferFrame secondaryFrame1 = + AsyncFrame secondaryFrame1 = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); frameAggregator.queueFrame(primaryFrame1, 0); @@ -869,12 +1264,12 @@ public void queueEndOfStream_secondaryStreamAlreadyHasFrames_usesFramesBeforeSki frameAggregator.queueFrame(secondaryFrame1, 1); assertThat(outputFrames).hasSize(1); - List aggregatedPacket = outputFrames.get(0); + List aggregatedPacket = outputFrames.get(0); assertThat(aggregatedPacket).hasSize(2); - assertThat(aggregatedPacket.get(0).presentationTimeUs) - .isEqualTo(primaryFrame1.presentationTimeUs); - assertThat(aggregatedPacket.get(1).presentationTimeUs) - .isEqualTo(secondaryFrame1.presentationTimeUs); + assertThat(getPresentationTimeUs(aggregatedPacket.get(0))) + .isEqualTo(getPresentationTimeUs(primaryFrame1)); + assertThat(getPresentationTimeUs(aggregatedPacket.get(1))) + .isEqualTo(getPresentationTimeUs(secondaryFrame1)); frameAggregator.queueEndOfStream(1); @@ -882,19 +1277,50 @@ public void queueEndOfStream_secondaryStreamAlreadyHasFrames_usesFramesBeforeSki assertThat(outputFrames.get(1)).containsExactly(primaryFrame2); } + @Test + public void queueEndOfStream_secondarySequenceEndsEarlier_retainsLastSecondaryFrame() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 2); + List primaryFrames = + createFrameList(/* numFrames= */ 2, /* frameRate= */ 10, /* sequenceIndex= */ 0); + List secondaryFrames = + createFrameList(/* numFrames= */ 2, /* frameRate= */ 30, /* sequenceIndex= */ 1); + + primaryFrames.forEach(frame -> frameAggregator.queueFrame(frame, /* sequenceIndex= */ 0)); + secondaryFrames.forEach(frame -> frameAggregator.queueFrame(frame, /* sequenceIndex= */ 1)); + frameAggregator.queueEndOfStream(/* sequenceIndex= */ 1); + outputFrames.forEach( + packet -> + packet.forEach( + frame -> TransformerUtil.releaseIfNeeded(frame.frame, /* releaseFence= */ null))); + + assertThat(outputFrames).hasSize(2); + assertThat(outputFrames.get(1)).containsExactly(primaryFrames.get(1)); + // The superseded secondary frame is released, while the last frame remains queued until the + // aggregator is flushed or closed. + assertThat(releasedFrameTimestamps).contains(0L); + assertThat(releasedFrameTimestamps).doesNotContain(33_333L); + } + @Test public void queueEndOfStream_primarySequence_outputsEndOfStreamFrame() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 1, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 1); frameAggregator.queueEndOfStream(/* sequenceIndex= */ 0); assertThat(outputFrames).hasSize(1); - assertThat(outputFrames.get(0)).containsExactly(HardwareBufferFrame.END_OF_STREAM_FRAME); + assertThat(outputFrames.get(0)).containsExactly(END_OF_STREAM_ASYNC_FRAME); } @Test @@ -902,10 +1328,11 @@ public void queueEndOfStream_primarySequenceAfterFrames_outputsFramesThenEndOfSt FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 1, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 1); - HardwareBufferFrame frame1 = + AsyncFrame frame1 = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); frameAggregator.queueFrame(frame1, /* sequenceIndex= */ 0); @@ -913,7 +1340,7 @@ public void queueEndOfStream_primarySequenceAfterFrames_outputsFramesThenEndOfSt assertThat(outputFrames).hasSize(2); assertThat(outputFrames.get(0)).containsExactly(frame1); - assertThat(outputFrames.get(1)).containsExactly(HardwareBufferFrame.END_OF_STREAM_FRAME); + assertThat(outputFrames.get(1)).containsExactly(END_OF_STREAM_ASYNC_FRAME); } @Test @@ -921,15 +1348,16 @@ public void queueEndOfStream_primarySequence_ignoresSubsequentFrames() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 1, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 1); - HardwareBufferFrame frame1 = + AsyncFrame frame1 = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); frameAggregator.queueEndOfStream(/* sequenceIndex= */ 0); assertThat(outputFrames).hasSize(1); - assertThat(outputFrames.get(0)).containsExactly(HardwareBufferFrame.END_OF_STREAM_FRAME); + assertThat(outputFrames.get(0)).containsExactly(END_OF_STREAM_ASYNC_FRAME); frameAggregator.queueFrame(frame1, /* sequenceIndex= */ 0); @@ -941,7 +1369,8 @@ public void queueEndOfStream_primarySequence_idempotent() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 1, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 1); @@ -949,7 +1378,7 @@ public void queueEndOfStream_primarySequence_idempotent() { frameAggregator.queueEndOfStream(/* sequenceIndex= */ 0); assertThat(outputFrames).hasSize(1); - assertThat(outputFrames.get(0)).containsExactly(HardwareBufferFrame.END_OF_STREAM_FRAME); + assertThat(outputFrames.get(0)).containsExactly(END_OF_STREAM_ASYNC_FRAME); } @Test @@ -957,16 +1386,17 @@ public void queueEndOfStream_thenFlushPrimary_allowsNewFrames() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 1, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 1); frameAggregator.queueEndOfStream(/* sequenceIndex= */ 0); - assertThat(outputFrames.get(0)).containsExactly(HardwareBufferFrame.END_OF_STREAM_FRAME); + assertThat(outputFrames.get(0)).containsExactly(END_OF_STREAM_ASYNC_FRAME); frameAggregator.flush(/* sequenceIndex= */ 0); - HardwareBufferFrame frame1 = + AsyncFrame frame1 = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); frameAggregator.queueFrame(frame1, /* sequenceIndex= */ 0); @@ -981,11 +1411,12 @@ public void queueEndOfStream_primaryEndsWhileWaitingForSecondary_outputsFramesTh FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 2); - HardwareBufferFrame primaryFrame = createFrame(100, /* sequencePresentationTimeUs= */ 100); - HardwareBufferFrame secondaryFrame = createFrame(100, /* sequencePresentationTimeUs= */ 100); + AsyncFrame primaryFrame = createFrame(100, /* sequencePresentationTimeUs= */ 100); + AsyncFrame secondaryFrame = createFrame(100, /* sequencePresentationTimeUs= */ 100); frameAggregator.queueFrame(primaryFrame, 0); frameAggregator.queueEndOfStream(0); @@ -995,12 +1426,14 @@ public void queueEndOfStream_primaryEndsWhileWaitingForSecondary_outputsFramesTh frameAggregator.queueFrame(secondaryFrame, 1); assertThat(outputFrames).hasSize(2); - List dataPacket = outputFrames.get(0); + List dataPacket = outputFrames.get(0); assertThat(dataPacket).hasSize(2); - assertThat(dataPacket.get(0).presentationTimeUs).isEqualTo(primaryFrame.presentationTimeUs); - assertThat(dataPacket.get(1).presentationTimeUs).isEqualTo(secondaryFrame.presentationTimeUs); - List eosPacket = outputFrames.get(1); - assertThat(eosPacket).containsExactly(HardwareBufferFrame.END_OF_STREAM_FRAME); + assertThat(getPresentationTimeUs(dataPacket.get(0))) + .isEqualTo(getPresentationTimeUs(primaryFrame)); + assertThat(getPresentationTimeUs(dataPacket.get(1))) + .isEqualTo(getPresentationTimeUs(secondaryFrame)); + List eosPacket = outputFrames.get(1); + assertThat(eosPacket).containsExactly(END_OF_STREAM_ASYNC_FRAME); } @Test @@ -1008,9 +1441,10 @@ public void queueFrame_beforeRegisterSequence_throwsIllegalStateException() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); - HardwareBufferFrame primaryFrame = + AsyncFrame primaryFrame = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); assertThrows( @@ -1023,7 +1457,8 @@ public void queueEndOfStream_beforeRegisterSequence_throwsIllegalStateException( FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); assertThrows( @@ -1036,32 +1471,32 @@ public void registerSequence_withInvalidSequenceIndex_throwsIllegalArgumentExcep FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); assertThrows( IllegalArgumentException.class, - () -> - frameAggregator.registerSequence(/* sequenceIndex= */ 2, /* shouldAggregate= */ false)); + () -> frameAggregator.registerSequence(/* sequenceIndex= */ 2, STRATEGY_EXPECT_NO_FRAMES)); assertThrows( IllegalArgumentException.class, - () -> - frameAggregator.registerSequence( - /* sequenceIndex= */ -1, /* shouldAggregate= */ false)); + () -> frameAggregator.registerSequence(/* sequenceIndex= */ -1, STRATEGY_EXPECT_NO_FRAMES)); } @Test - public void registerSequence_withShouldAggregateFalse_doesNotWaitForSecondarySequence() { + public void registerSequence_withExpectNoFramesStrategy_doesNotWaitForSecondarySequence() { FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); - HardwareBufferFrame primaryFrame = + AsyncFrame primaryFrame = createFrame(/* presentationTimeUs= */ 100, /* sequencePresentationTimeUs= */ 100); - frameAggregator.registerSequence(/* sequenceIndex= */ 0, /* shouldAggregate= */ true); - frameAggregator.registerSequence(/* sequenceIndex= */ 1, /* shouldAggregate= */ false); + frameAggregator.registerSequence( + /* sequenceIndex= */ 0, STRATEGY_MATCH_FRAME_CLOSEST_TO_TARGET); + frameAggregator.registerSequence(/* sequenceIndex= */ 1, STRATEGY_EXPECT_NO_FRAMES); frameAggregator.queueFrame(primaryFrame, /* sequenceIndex= */ 0); // The aggregator should not wait for sequence 1 and should output the primary frame @@ -1075,12 +1510,13 @@ public void handlePrimaryEndOfStream_withPendingSecondaryFrames_doesNotReleaseSe FrameAggregator frameAggregator = new FrameAggregator( /* numSequences= */ 2, - /* downstreamConsumer= */ outputFrames::add, + /* frameRate= */ null, + /* downstreamConsumer= */ this::recordOutputFrames, /* onFlush= */ flushedSequences::add); registerAllSequences(frameAggregator, /* numSequences= */ 2); - HardwareBufferFrame primary1 = + AsyncFrame primary1 = createFrame(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 0); - HardwareBufferFrame secondary1 = + AsyncFrame secondary1 = createFrame(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 0); frameAggregator.queueFrame(primary1, /* sequenceIndex= */ 0); @@ -1091,7 +1527,7 @@ public void handlePrimaryEndOfStream_withPendingSecondaryFrames_doesNotReleaseSe // Simulate a seek: The secondary sequence flushes first and queues new frames BEFORE the // primary EOS arrives. frameAggregator.flush(/* sequenceIndex= */ 1); - HardwareBufferFrame postSeekSecondary = + AsyncFrame postSeekSecondary = createFrame(/* presentationTimeUs= */ 50_000, /* sequencePresentationTimeUs= */ 50_000); frameAggregator.queueFrame(postSeekSecondary, /* sequenceIndex= */ 1); // A delayed EOS arrives from the slow primary sequence. @@ -1101,36 +1537,905 @@ public void handlePrimaryEndOfStream_withPendingSecondaryFrames_doesNotReleaseSe // The primary sequence flushes and queues its new frames. frameAggregator.flush(/* sequenceIndex= */ 0); - HardwareBufferFrame postSeekPrimary = + AsyncFrame postSeekPrimary = createFrame(/* presentationTimeUs= */ 50_000, /* sequencePresentationTimeUs= */ 50_000); frameAggregator.queueFrame(postSeekPrimary, /* sequenceIndex= */ 0); // The post-seek frames should successfully aggregate. // outputFrames now contains: [initial data packet], [EOS packet], [post-seek data packet] assertThat(outputFrames).hasSize(3); - assertThat(outputFrames.get(0).get(0).sequencePresentationTimeUs).isEqualTo(0); - assertThat(outputFrames.get(0).get(1).sequencePresentationTimeUs).isEqualTo(0); - assertThat(outputFrames.get(1)).containsExactly(HardwareBufferFrame.END_OF_STREAM_FRAME); - assertThat(outputFrames.get(2).get(0).sequencePresentationTimeUs).isEqualTo(50_000); - assertThat(outputFrames.get(2).get(1).sequencePresentationTimeUs).isEqualTo(50_000); - } - - /** Creates a {@link GlTextureFrame} for testing. */ - private HardwareBufferFrame createFrame( - long presentationTimeUs, long sequencePresentationTimeUs) { - return new HardwareBufferFrame.Builder( - /* hardwareBuffer= */ null, - directExecutor(), - (releaseFence) -> releasedFrameTimestamps.add(presentationTimeUs)) - .setPresentationTimeUs(presentationTimeUs) - .setSequencePresentationTimeUs(sequencePresentationTimeUs) - .setInternalFrame(this) - .build(); + assertThat(outputFrames.get(0).get(0).frame.getContentTimeUs()).isEqualTo(0); + assertThat(outputFrames.get(0).get(1).frame.getContentTimeUs()).isEqualTo(0); + assertThat(outputFrames.get(1)).containsExactly(END_OF_STREAM_ASYNC_FRAME); + assertThat(outputFrames.get(2).get(0).frame.getContentTimeUs()).isEqualTo(50_000); + assertThat(outputFrames.get(2).get(1).frame.getContentTimeUs()).isEqualTo(50_000); + } + + @Test + public void + queueFrame_withFrameRateAndRetentionDisabled_upsamplesMatchesAndDownsamplesInputStreams() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 3, + /* frameRate= */ new Rational(30, 1), + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences( + frameAggregator, /* numSequences= */ 3, STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET); + + /* + * Pacing / Retiming Timeline (Target Virtual Clock: 30 FPS): + * + * Virtual Ticks: [Tick 0: 0us] [Tick 1: 33_333us] [Tick 2: 66_667us] + * ---------------------------------------------------------------------------------- + * Seq 0 (15 FPS, up): |--- 0us ---| |--- 66_667us ---| | (hold 66_667us)| + * Seq 1 (30 FPS, exact): |--- 0us ---| |--- 33_333us ---| |--- 66_667us ---| + * Seq 2 (60 FPS, drop): |0| (16_667)| |--- 33_333us ---| |(50_000)| 66_667| + */ + + List seq0Frames = + createFrameList(/* numFrames= */ 2, /* frameRate= */ 15, /* sequenceIndex= */ 0); + List seq1Frames = + createFrameList(/* numFrames= */ 3, /* frameRate= */ 30, /* sequenceIndex= */ 1); + List seq2Frames = + createFrameList(/* numFrames= */ 5, /* frameRate= */ 60, /* sequenceIndex= */ 2); + + // Queue initial frame batch for Virtual Tick 0 (0us) + frameAggregator.queueFrame(seq0Frames.get(0), /* sequenceIndex= */ 0); + frameAggregator.queueFrame(seq1Frames.get(0), /* sequenceIndex= */ 1); + frameAggregator.queueFrame(seq2Frames.get(0), /* sequenceIndex= */ 2); + + assertOutputPacket( + Iterables.getLast(outputFrames), + /* expectedSize= */ 3, + /* expectedTimeUs= */ 0, + new SourceFrame( + /* sequenceIndex= */ 0, + /* presentationTimeUs= */ 0, + /* sequencePresentationTimeUs= */ 0), + new SourceFrame( + /* sequenceIndex= */ 1, + /* presentationTimeUs= */ 0, + /* sequencePresentationTimeUs= */ 0), + new SourceFrame( + /* sequenceIndex= */ 2, + /* presentationTimeUs= */ 0, + /* sequencePresentationTimeUs= */ 0)); + + // Queue next frame batch for Virtual Tick 1 (33_333us) + // Seq 0 (15 FPS): Queues Frame 2 (66_667us >= 33_333us), which is matched for Virtual Tick 1 + // (33_333us). + // Seq 2 (60 FPS): Frame 2 (16_667us) is dropped (downsampled). + frameAggregator.queueFrame(seq0Frames.get(1), /* sequenceIndex= */ 0); + frameAggregator.queueFrame(seq1Frames.get(1), /* sequenceIndex= */ 1); + frameAggregator.queueFrame(seq2Frames.get(1), /* sequenceIndex= */ 2); + frameAggregator.queueFrame(seq2Frames.get(2), /* sequenceIndex= */ 2); + + assertThat(outputFrames).hasSize(2); // Virtual Ticks 0 and 1 emitted + assertOutputPacket( + Iterables.getLast(outputFrames), + /* expectedSize= */ 3, + /* expectedTimeUs= */ 33_333, + new SourceFrame( + /* sequenceIndex= */ 0, + /* presentationTimeUs= */ 66_667, + /* sequencePresentationTimeUs= */ 66_667), + new SourceFrame( + /* sequenceIndex= */ 1, + /* presentationTimeUs= */ 33_333, + /* sequencePresentationTimeUs= */ 33_333), + new SourceFrame( + /* sequenceIndex= */ 2, + /* presentationTimeUs= */ 33_333, + /* sequencePresentationTimeUs= */ 33_333)); + assertThat(releasedFrameTimestamps).contains(getPresentationTimeUs(seq2Frames.get(1))); + + // Queue final frame batch for Virtual Tick 2 (66_667us) + // Seq 0 (15 FPS): Holds (reuses) Frame 2 (66_667us == 66_667us). + // Seq 2 (60 FPS): Frame 4 (50_000us) is dropped. + frameAggregator.queueFrame(seq1Frames.get(2), /* sequenceIndex= */ 1); + frameAggregator.queueFrame(seq2Frames.get(3), /* sequenceIndex= */ 2); + frameAggregator.queueFrame(seq2Frames.get(4), /* sequenceIndex= */ 2); + + assertThat(outputFrames).hasSize(3); // Virtual Ticks 0, 1, and 2 emitted + assertOutputPacket( + Iterables.getLast(outputFrames), + /* expectedSize= */ 3, + /* expectedTimeUs= */ 66_667, + new SourceFrame( + /* sequenceIndex= */ 0, + /* presentationTimeUs= */ 66_667, + /* sequencePresentationTimeUs= */ 66_667), + new SourceFrame( + /* sequenceIndex= */ 1, + /* presentationTimeUs= */ 66_667, + /* sequencePresentationTimeUs= */ 66_667), + new SourceFrame( + /* sequenceIndex= */ 2, + /* presentationTimeUs= */ 66_667, + /* sequencePresentationTimeUs= */ 66_667)); + assertThat(releasedFrameTimestamps).contains(getPresentationTimeUs(seq2Frames.get(3))); + + frameAggregator.queueEndOfStream(/* sequenceIndex= */ 0); + frameAggregator.queueEndOfStream(/* sequenceIndex= */ 1); + frameAggregator.queueEndOfStream(/* sequenceIndex= */ 2); + + assertThat(outputFrames).hasSize(4); + assertThat(Iterables.getLast(outputFrames)).containsExactly(END_OF_STREAM_ASYNC_FRAME); + } + + @Test + public void queueFrame_withFrameRateAndSecondarySequenceEndsEarly_outputsOnlyPrimaryFrame() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ new Rational(30, 1), + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 2); + + /* + * Pacing / Retiming Timeline (Target Virtual Clock: 30 FPS): + * + * Virtual Ticks: [Tick 0: 0us] [Tick 1: 33_333us] + * --------------------------------------------------------------- + * Seq 0 (Primary): |--- 0us ---| |--- 33_333us ---| + * Seq 1 (Secondary): |--- 0us ---| (EOS) + */ + + List primaryFrames = + createFrameList(/* numFrames= */ 2, /* frameRate= */ 30, /* sequenceIndex= */ 0); + List secondaryFrames = + createFrameList(/* numFrames= */ 1, /* frameRate= */ 30, /* sequenceIndex= */ 1); + + frameAggregator.queueFrame(primaryFrames.get(0), /* sequenceIndex= */ 0); + frameAggregator.queueFrame(secondaryFrames.get(0), /* sequenceIndex= */ 1); + + assertThat(outputFrames).hasSize(1); // Virtual Tick 0 emitted + assertOutputPacket( + Iterables.getLast(outputFrames), /* expectedSize= */ 2, /* expectedTimeUs= */ 0); + + frameAggregator.queueEndOfStream(/* sequenceIndex= */ 1); + frameAggregator.queueFrame(primaryFrames.get(1), /* sequenceIndex= */ 0); + + assertThat(outputFrames).hasSize(2); // Virtual Ticks 0 and 1 emitted + assertOutputPacket( + Iterables.getLast(outputFrames), + /* expectedSize= */ 1, + /* expectedTimeUs= */ 33_333, + new SourceFrame( + /* sequenceIndex= */ 0, + /* presentationTimeUs= */ 33_333, + /* sequencePresentationTimeUs= */ 33_333)); + } + + @Test + public void queueFrame_withFrameRateAndPrimarySequenceEndsEarly_outputsOnlySecondaryFrame() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ new Rational(30, 1), + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 2); + + /* + * Pacing / Retiming Timeline (Target Virtual Clock: 30 FPS): + * + * Virtual Ticks: [Tick 0: 0us] [Tick 1: 33_333us] + * --------------------------------------------------------------- + * Seq 0 (Primary): |--- 0us ---| (EOS) + * Seq 1 (Secondary): |--- 0us ---| |--- 33_333us ---| + */ + + List primaryFrames = + createFrameList(/* numFrames= */ 1, /* frameRate= */ 30, /* sequenceIndex= */ 0); + List secondaryFrames = + createFrameList(/* numFrames= */ 2, /* frameRate= */ 30, /* sequenceIndex= */ 1); + + frameAggregator.queueFrame(primaryFrames.get(0), /* sequenceIndex= */ 0); + frameAggregator.queueFrame(secondaryFrames.get(0), /* sequenceIndex= */ 1); + + assertThat(outputFrames).hasSize(1); // Virtual Tick 0 emitted + assertOutputPacket( + Iterables.getLast(outputFrames), /* expectedSize= */ 2, /* expectedTimeUs= */ 0); + + frameAggregator.queueEndOfStream(/* sequenceIndex= */ 0); + frameAggregator.queueFrame(secondaryFrames.get(1), /* sequenceIndex= */ 1); + + assertThat(outputFrames).hasSize(2); // Virtual Ticks 0 and 1 emitted + assertOutputPacket( + Iterables.getLast(outputFrames), + /* expectedSize= */ 1, + /* expectedTimeUs= */ 33_333, + new SourceFrame( + /* sequenceIndex= */ 1, + /* presentationTimeUs= */ 33_333, + /* sequencePresentationTimeUs= */ 33_333)); + } + + @Test + public void queueFrame_withFrameRateAndStartsAtVirtualTick_exactMatch() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ new Rational(30, 1), + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 2); + + /* + * Pacing / Retiming Timeline (Target Virtual Clock: 30 FPS): + * + * Virtual Ticks: [Tick 1: 33_333us] + * ----------------------------------------- + * Seq 0 (Primary): |--- 33_333us ---| + * Seq 1 (Secondary): |--- 33_333us ---| + */ + + AsyncFrame primaryFrame = + createFrame(/* presentationTimeUs= */ 33_333, /* sequencePresentationTimeUs= */ 33_333); + AsyncFrame secondaryFrame = + createFrame(/* presentationTimeUs= */ 33_333, /* sequencePresentationTimeUs= */ 33_333); + + frameAggregator.queueFrame(primaryFrame, /* sequenceIndex= */ 0); + frameAggregator.queueFrame(secondaryFrame, /* sequenceIndex= */ 1); + + assertThat(outputFrames).hasSize(1); + assertOutputPacket( + Iterables.getLast(outputFrames), /* expectedSize= */ 2, /* expectedTimeUs= */ 33_333); + } + + @Test + public void queueFrame_withFrameRateAndStartsAtVirtualTick_retainsAndOutputsFrame() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ new Rational(30, 1), + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 2); + + /* + * Pacing / Retiming Timeline (Target Virtual Clock: 30 FPS): + * + * Virtual Ticks: [Tick 2: 66_667us] [Tick 3: 100_000us] + * ---------------------------------------------------------------- + * Seq 0 (Primary): | 66_667us | + * Seq 1 (Secondary): | 66_667us | + */ + AsyncFrame primaryFrame = + createFrame(/* presentationTimeUs= */ 66_667, /* sequencePresentationTimeUs= */ 66_667); + AsyncFrame secondaryFrame = + createFrame(/* presentationTimeUs= */ 66_667, /* sequencePresentationTimeUs= */ 66_667); + + frameAggregator.queueFrame(primaryFrame, /* sequenceIndex= */ 0); + frameAggregator.queueFrame(secondaryFrame, /* sequenceIndex= */ 1); + + // Frame 2 at 66_667us is retained and emitted for Tick 2. + assertThat(outputFrames).hasSize(1); + assertOutputPacket( + outputFrames.get(0), + /* expectedSize= */ 2, + /* expectedPresentationTimeUs= */ 66_667, + /* expectedSequencePresentationTimeUs= */ 66_667); + assertThat(releasedFrameTimestamps).isEmpty(); + } + + @Test + public void + queueFrame_withFrameRateAndTruncatedTimestampsAtFrameBoundary_retainsAndOutputsFrame() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 1, + /* frameRate= */ new Rational(30, 1), + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 1); + + // A rounding error on microsecond presentation time (e.g. 66_666 us vs 66_667 us at 30 fps) + // should not cause a frame drop. + frameAggregator.queueFrame( + createFrame(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 0), + /* sequenceIndex= */ 0); + frameAggregator.queueFrame( + createFrame(/* presentationTimeUs= */ 33_333, /* sequencePresentationTimeUs= */ 33_333), + /* sequenceIndex= */ 0); + frameAggregator.queueFrame( + createFrame(/* presentationTimeUs= */ 66_666, /* sequencePresentationTimeUs= */ 66_666), + /* sequenceIndex= */ 0); + frameAggregator.queueFrame( + createFrame(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000), + /* sequenceIndex= */ 0); + + assertThat(releasedFrameTimestamps).doesNotContain(66_666L); + assertThat(outputFrames).hasSize(4); + assertOutputPacket( + outputFrames.get(2), + /* expectedSize= */ 1, + /* expectedPresentationTimeUs= */ 66_667, + /* expectedSequencePresentationTimeUs= */ 66_667, + new SourceFrame( + /* sequenceIndex= */ 0, + /* presentationTimeUs= */ 66_666, + /* sequencePresentationTimeUs= */ 66_666)); + } + + @Test + public void queueFrame_withFrameRateAndTruncatedTimestampAndRetentionDisabled_dropsFrame() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 1, + /* frameRate= */ new Rational(30, 1), + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences( + frameAggregator, /* numSequences= */ 1, STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET); + AsyncFrame frame0 = + createFrame(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 0); + AsyncFrame frame1 = + createFrame(/* presentationTimeUs= */ 33_333, /* sequencePresentationTimeUs= */ 33_333); + AsyncFrame frame2 = + createFrame(/* presentationTimeUs= */ 66_666, /* sequencePresentationTimeUs= */ 66_666); + AsyncFrame frame3 = + createFrame(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); + + frameAggregator.queueFrame(frame0, /* sequenceIndex= */ 0); + frameAggregator.queueFrame(frame1, /* sequenceIndex= */ 0); + frameAggregator.queueFrame(frame2, /* sequenceIndex= */ 0); + frameAggregator.queueFrame(frame3, /* sequenceIndex= */ 0); + + assertThat(outputFrames).hasSize(4); + // Without retention, Frame 2 (66_666us) is dropped immediately because 66_666 < 66_667. + assertThat(releasedFrameTimestamps).contains(66_666L); + // Frame 3 (100_000us) is matched to tick 2 (66_667us). + assertOutputPacket( + outputFrames.get(2), + /* expectedSize= */ 1, + /* expectedPresentationTimeUs= */ 66_667, + /* expectedSequencePresentationTimeUs= */ 66_667, + new SourceFrame( + /* sequenceIndex= */ 0, + /* presentationTimeUs= */ 100_000, + /* sequencePresentationTimeUs= */ 100_000)); + } + + @Test + public void + queueFrame_withFrameRateAndRoundsUpToVirtualTickAndRetentionDisabled_dropsPrecedingFrames() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ new Rational(30, 1), + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences( + frameAggregator, /* numSequences= */ 2, STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET); + + /* + * Pacing / Retiming Timeline (Target Virtual Clock: 30 FPS): + * + * Virtual Ticks (30 FPS): [ Tick 3: 100_000us ] [ Tick 4: 133_333us ] + * --------------------------------------------------------------------------------- + * Seq 0 (Primary): | 82_000us | 100_000us| |---- 133_333us ----| + * Seq 1 (Secondary): | (Wait) | 133_333us| |---- 133_333us ----| + */ + + AsyncFrame primaryFrame1 = + createFrame(/* presentationTimeUs= */ 82_000, /* sequencePresentationTimeUs= */ 82_000); + AsyncFrame primaryFrame2 = + createFrame(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000); + AsyncFrame primaryFrame3 = + createFrame(/* presentationTimeUs= */ 133_333, /* sequencePresentationTimeUs= */ 133_333); + AsyncFrame secondaryFrame1 = + createFrame(/* presentationTimeUs= */ 133_333, /* sequencePresentationTimeUs= */ 133_333); + + // Queue secondary frame [133_333]. Output is empty. + frameAggregator.queueFrame(secondaryFrame1, /* sequenceIndex= */ 1); + + assertThat(outputFrames).isEmpty(); + + // Queue primary frame [82_000]. + // timeUs * frameRate / 1_000_000.0 = 82_000 * 30 / 1M = 2.46. + // This ceils to index 3 (target time 100_000us) to ensure the virtual clock + // never produces frames with timestamps earlier than the current available frames. + // Because 82_000 < 100_000, it is dropped. + frameAggregator.queueFrame(primaryFrame1, /* sequenceIndex= */ 0); + + assertThat(outputFrames).isEmpty(); + assertThat(releasedFrameTimestamps).containsExactly(82_000L); + + // Queue primary frame [100_000]. Output matches S1[100_000] + S2[133_333] at Target 100_000. + frameAggregator.queueFrame(primaryFrame2, /* sequenceIndex= */ 0); + + assertThat(outputFrames).hasSize(1); + assertOutputPacket(outputFrames.get(0), /* expectedSize= */ 2, /* expectedTimeUs= */ 100_000); + + // Queue primary frame [133_333]. Output matches S1[133_333] + S2[133_333] at Target 133_333. + // The previous primary frame [100_000] is released. + frameAggregator.queueFrame(primaryFrame3, /* sequenceIndex= */ 0); + + assertThat(outputFrames).hasSize(2); + assertOutputPacket(outputFrames.get(1), /* expectedSize= */ 2, /* expectedTimeUs= */ 133_333); + assertThat(releasedFrameTimestamps).containsExactly(82_000L); + } + + @Test + public void flush_withFrameRate_resetsToSeekTimestampVirtualTick() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ new Rational(30, 1), + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 2); + + /* + * Pacing / Retiming Timeline (Target Virtual Clock: 30 FPS): + * + * Virtual Ticks: [Tick 0: 0us] [Tick 3: 100_000us] + * --------------------------------------------------------------------- + * Seq 0 (Primary): |--- 0us ---| |--- 100_000us ---| + * Seq 1 (Secondary): |--- 0us ---| |--- 100_000us ---| + */ + + frameAggregator.queueFrame( + createFrame(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 0), + /* sequenceIndex= */ 0); + frameAggregator.queueFrame( + createFrame(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 0), + /* sequenceIndex= */ 1); + + assertThat(outputFrames).hasSize(1); // Pre-seek Virtual Tick emitted + + // Seek to 100_000us + frameAggregator.flush(/* sequenceIndex= */ 0); + frameAggregator.flush(/* sequenceIndex= */ 1); + + frameAggregator.queueFrame( + createFrame(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000), + /* sequenceIndex= */ 0); + frameAggregator.queueFrame( + createFrame(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000), + /* sequenceIndex= */ 1); + + assertThat(outputFrames).hasSize(2); // Post-seek Virtual Tick emitted + assertOutputPacket( + Iterables.getLast(outputFrames), /* expectedSize= */ 2, /* expectedTimeUs= */ 100_000); + } + + @Test + public void + flush_withFrameRateAndSeekToFrameAlignedWithRoundedMicrosecond_retainsAndOutputsFrame() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 1, + /* frameRate= */ new Rational(30, 1), + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 1); + + frameAggregator.queueFrame( + createFrame(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 0), + /* sequenceIndex= */ 0); + + assertThat(outputFrames).hasSize(1); // Pre-seek Virtual Tick emitted + + // Seek to 66_667us (Frame 2 at 30fps) + frameAggregator.flush(/* sequenceIndex= */ 0); + + frameAggregator.queueFrame( + createFrame(/* presentationTimeUs= */ 66_667, /* sequencePresentationTimeUs= */ 66_667), + /* sequenceIndex= */ 0); + + assertThat(outputFrames).hasSize(2); + assertOutputPacket( + Iterables.getLast(outputFrames), /* expectedSize= */ 1, /* expectedTimeUs= */ 66_667); + } + + @Test + public void queueFrame_withFrameRateAndMultipleItemsAndRetentionDisabled_retimesTimestamps() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 1, + /* frameRate= */ new Rational(10, 1), + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences( + frameAggregator, /* numSequences= */ 1, STRATEGY_MATCH_FRAME_AT_OR_AFTER_TARGET); + + /* + * Pacing / Retiming Timeline (Target Virtual Clock: 10 FPS): + * + * Virtual Ticks (10 FPS): [ Tick 0: 0us ] [ Tick 1: 100_000us ] + * ----------------------------------------------------------------------------------- + * Item 1 (30 FPS): |--- 0us ---| (33) | (66) | + * Item 2 (30 FPS, offset): | (90) | |---- 123_333us ----| + */ + + List item1Frames = + createFrameList(/* numFrames= */ 3, /* frameRate= */ 30, /* sequenceIndex= */ 0); + List item2Frames = + createFrameList( + /* numFrames= */ 2, + /* frameRate= */ 30, + /* sequenceIndex= */ 0, + /* sequencePresentationTimeOffsetUs= */ 90_000); + + item1Frames.forEach(frame -> frameAggregator.queueFrame(frame, /* sequenceIndex= */ 0)); + item2Frames.forEach(frame -> frameAggregator.queueFrame(frame, /* sequenceIndex= */ 0)); + + assertThat(outputFrames).hasSize(2); // Virtual Ticks 0 and 1 emitted + + // Virtual Tick 0 (0us): Matches Item 1 Frame 0 (SeqPTS = 0us). + assertOutputPacket( + outputFrames.get(0), + /* expectedSize= */ 1, + /* expectedTimeUs= */ 0, + new SourceFrame( + /* sequenceIndex= */ 0, + /* presentationTimeUs= */ 0, + /* sequencePresentationTimeUs= */ 0)); + + // Virtual Tick 1 (100_000us): + // Intermediate 30 FPS frames (Item 1 Frames 1 & 2 at 33_333us & 66_667us; + // Item 2 Frame 0 at 90_000us) are dropped because SeqPTS < 100_000us. + // Item 2 Frame 1 (SeqPTS = 90_000 + 33_333 = 123_333us >= 100_000us) matches. + // shiftDeltaUs = 100_000 - 123_333 = -23_333us. + // Retimed PTS = 33_333 - 23_333 = 10_000us. + assertOutputPacket( + outputFrames.get(1), + /* expectedSize= */ 1, + /* expectedPresentationTimeUs= */ 10_000, + /* expectedSequencePresentationTimeUs= */ 100_000, + new SourceFrame( + /* sequenceIndex= */ 0, + /* presentationTimeUs= */ 33_333, + /* sequencePresentationTimeUs= */ 123_333)); + } + + @Test + public void queueFrame_withFrameRateAndMultipleItems_matchesClosestAndRetimesTimestamps() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 1, + /* frameRate= */ new Rational(10, 1), + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 1); + + /* + * Pacing / Retiming Timeline (Target Virtual Clock: 10 FPS): + * + * Virtual Ticks: [ Tick 0: 0us ] [ Tick 1: 100_000us ] + * -------------------------------------------------------------------------------------------- + * Seq 0: |-- 0us --| (33_333us) | |-- 90_000us --| (123_333us) + * \_______ Item 1 _______/ \_____________________ Item 2 _____________________/ + */ + + List item1Frames = + createFrameList(/* numFrames= */ 2, /* frameRate= */ 30, /* sequenceIndex= */ 0); + List item2Frames = + createFrameList( + /* numFrames= */ 2, + /* frameRate= */ 30, + /* sequenceIndex= */ 0, + /* sequencePresentationTimeOffsetUs= */ 90_000); + + item1Frames.forEach(frame -> frameAggregator.queueFrame(frame, /* sequenceIndex= */ 0)); + item2Frames.forEach(frame -> frameAggregator.queueFrame(frame, /* sequenceIndex= */ 0)); + + assertThat(outputFrames).hasSize(2); // Virtual Ticks 0 and 1 emitted + + // Virtual Tick 0 (0us): Matches Item 1 Frame 0 (SeqPTS = 0us). + assertOutputPacket( + outputFrames.get(0), + /* expectedSize= */ 1, + /* expectedTimeUs= */ 0, + new SourceFrame( + /* sequenceIndex= */ 0, + /* presentationTimeUs= */ 0, + /* sequencePresentationTimeUs= */ 0)); + + // Virtual Tick 1 (100_000us): Item 2 Frame 0 (SeqPTS = 90_000us) is 10_000us before the tick + // and Item 2 Frame 1 (SeqPTS = 123_333us) is 23_333us after it, so Frame 0 is selected. + // Its sequence timestamp is retimed onto the tick (100_000us), and its item-relative timestamp + // is shifted by the same delta (0 + (100_000 - 90_000) = 10_000us). + assertOutputPacket( + outputFrames.get(1), + /* expectedSize= */ 1, + /* expectedPresentationTimeUs= */ 10_000, + /* expectedSequencePresentationTimeUs= */ 100_000, + new SourceFrame( + /* sequenceIndex= */ 0, + /* presentationTimeUs= */ 0, + /* sequencePresentationTimeUs= */ 90_000)); + } + + @Test + public void queueEndOfStream_withFrameRateAndNoQueuedFrames_outputsEndOfStream() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ new Rational(30, 1), + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 2); + + frameAggregator.queueEndOfStream(/* sequenceIndex= */ 0); + frameAggregator.queueEndOfStream(/* sequenceIndex= */ 1); + + // The virtual clock never started, so no tick is generated and no aggregation pass can report + // the end of the stream. + assertThat(outputFrames).hasSize(1); + assertThat(outputFrames.get(0)).containsExactly(END_OF_STREAM_ASYNC_FRAME); + } + + @Test + public void queueEndOfStream_withFrameRateAfterFlush_outputsEndOfStream() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ new Rational(30, 1), + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 2); + frameAggregator.queueFrame( + createFrame(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 0), + /* sequenceIndex= */ 0); + frameAggregator.queueFrame( + createFrame( + /* presentationTimeUs= */ 0, + /* sequencePresentationTimeUs= */ 0, + /* sequenceIndex= */ 1), + /* sequenceIndex= */ 1); + frameAggregator.flush(/* sequenceIndex= */ 0); + frameAggregator.flush(/* sequenceIndex= */ 1); + outputFrames.clear(); + + frameAggregator.queueEndOfStream(/* sequenceIndex= */ 0); + frameAggregator.queueEndOfStream(/* sequenceIndex= */ 1); + + // Flushing resets the virtual clock, so this is the seek-to-the-end case. + assertThat(outputFrames).hasSize(1); + assertThat(outputFrames.get(0)).containsExactly(END_OF_STREAM_ASYNC_FRAME); + } + + @Test + public void queueEndOfStream_withFrameRateAndFrameAheadOfVirtualClock_outputsCatchUpFrames() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ new Rational(30, 1), + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 2); + + /* + * Pacing / Retiming Timeline (Target Virtual Clock: 30 FPS): + * + * Virtual Ticks: [Tick 0: 0us] [Tick 1: 33_333us] [Tick 2: 66_667us] [Tick 3: 100_000us] + * ------------------------------------------------------------------------------------------- + * Seq 0 (Primary): |--- 0us ---| |----- 0us ------| |-- 100_000us ---| |--- 100_000us ---| + * Seq 1 (Secondary): |--- 0us ---| |---- (Ended) ---| |---- (Ended) ---| |---- (Ended) ----| + */ + + frameAggregator.queueFrame( + createFrame(/* presentationTimeUs= */ 0, /* sequencePresentationTimeUs= */ 0), + /* sequenceIndex= */ 0); + frameAggregator.queueFrame( + createFrame( + /* presentationTimeUs= */ 0, + /* sequencePresentationTimeUs= */ 0, + /* sequenceIndex= */ 1), + /* sequenceIndex= */ 1); + frameAggregator.queueFrame( + createFrame(/* presentationTimeUs= */ 100_000, /* sequencePresentationTimeUs= */ 100_000), + /* sequenceIndex= */ 0); + // Only the tick at 0us can be aggregated so far, because Sequence 1 has neither a frame at or + // after 33_333us nor ended. + + frameAggregator.queueEndOfStream(/* sequenceIndex= */ 0); + frameAggregator.queueEndOfStream(/* sequenceIndex= */ 1); + + // The virtual clock is at 33_333us when the sequences end, so it keeps ticking until it passes + // the frame queued at 100_000us, which is duplicated at 66_667us and 100_000us. The end of the + // stream is reported by the first tick that matches no frame at all, rather than by the + // sequences ending. + assertThat(outputFrames).hasSize(5); + assertOutputPacket(outputFrames.get(0), /* expectedSize= */ 2, /* expectedTimeUs= */ 0); + assertOutputPacket( + outputFrames.get(1), + /* expectedSize= */ 1, + /* expectedTimeUs= */ 33_333, + new SourceFrame( + /* sequenceIndex= */ 0, + /* presentationTimeUs= */ 0, + /* sequencePresentationTimeUs= */ 0)); + assertOutputPacket( + outputFrames.get(2), + /* expectedSize= */ 1, + /* expectedTimeUs= */ 66_667, + new SourceFrame( + /* sequenceIndex= */ 0, + /* presentationTimeUs= */ 100_000, + /* sequencePresentationTimeUs= */ 100_000)); + assertOutputPacket( + outputFrames.get(3), + /* expectedSize= */ 1, + /* expectedTimeUs= */ 100_000, + new SourceFrame( + /* sequenceIndex= */ 0, + /* presentationTimeUs= */ 100_000, + /* sequencePresentationTimeUs= */ 100_000)); + assertThat(outputFrames.get(4)).containsExactly(END_OF_STREAM_ASYNC_FRAME); + } + + @Test + public void queueEndOfStream_withFrameRateAfterAllFramesEmitted_outputsEndOfStream() { + FrameAggregator frameAggregator = + new FrameAggregator( + /* numSequences= */ 2, + /* frameRate= */ new Rational(30, 1), + /* downstreamConsumer= */ this::recordOutputFrames, + /* onFlush= */ flushedSequences::add); + registerAllSequences(frameAggregator, /* numSequences= */ 2); + List primaryFrames = + createFrameList(/* numFrames= */ 3, /* frameRate= */ 30, /* sequenceIndex= */ 0); + List secondaryFrames = + createFrameList(/* numFrames= */ 2, /* frameRate= */ 30, /* sequenceIndex= */ 1); + + primaryFrames.forEach(frame -> frameAggregator.queueFrame(frame, /* sequenceIndex= */ 0)); + secondaryFrames.forEach(frame -> frameAggregator.queueFrame(frame, /* sequenceIndex= */ 1)); + frameAggregator.queueEndOfStream(/* sequenceIndex= */ 1); + frameAggregator.queueEndOfStream(/* sequenceIndex= */ 0); + + assertThat(outputFrames).hasSize(4); + assertOutputPacket(outputFrames.get(0), /* expectedSize= */ 2, /* expectedTimeUs= */ 0); + assertOutputPacket(outputFrames.get(1), /* expectedSize= */ 2, /* expectedTimeUs= */ 33_333); + assertOutputPacket(outputFrames.get(2), /* expectedSize= */ 1, /* expectedTimeUs= */ 66_667); + assertThat(outputFrames.get(3)).containsExactly(END_OF_STREAM_ASYNC_FRAME); + } + + private static void assertOutputPacket( + List outputPacket, + int expectedSize, + long expectedTimeUs, + SourceFrame... expectedSourceFrames) { + assertOutputPacket( + outputPacket, + expectedSize, + /* expectedPresentationTimeUs= */ expectedTimeUs, + /* expectedSequencePresentationTimeUs= */ expectedTimeUs, + expectedSourceFrames); + } + + private static void assertOutputPacket( + List outputPacket, + int expectedSize, + long expectedPresentationTimeUs, + long expectedSequencePresentationTimeUs, + SourceFrame... expectedSourceFrames) { + assertThat(outputPacket).hasSize(expectedSize); + for (AsyncFrame asyncFrame : outputPacket) { + assertThat(getPresentationTimeUs(asyncFrame)).isEqualTo(expectedPresentationTimeUs); + assertThat(asyncFrame.frame.getContentTimeUs()).isEqualTo(expectedSequencePresentationTimeUs); + } + if (expectedSourceFrames.length > 0) { + List actualSourceFrames = new ArrayList<>(); + for (AsyncFrame asyncFrame : outputPacket) { + actualSourceFrames.add(getSourceFrame(asyncFrame)); + } + assertThat(actualSourceFrames).containsExactlyElementsIn(expectedSourceFrames).inOrder(); + } + } + + private AsyncFrame createFrame(long presentationTimeUs, long sequencePresentationTimeUs) { + return createFrame(presentationTimeUs, sequencePresentationTimeUs, /* sequenceIndex= */ 0); + } + + private AsyncFrame createFrame( + long presentationTimeUs, long sequencePresentationTimeUs, int sequenceIndex) { + DefaultHardwareBufferFrame frame = + new DefaultHardwareBufferFrame.Builder( + placeholderBuffer, + directExecutor(), + (releaseFence) -> releasedFrameTimestamps.add(presentationTimeUs)) + .setContentTimeUs(sequencePresentationTimeUs) + .setMetadata( + ImmutableMap.of( + Frame.KEY_PRESENTATION_TIME_US, + presentationTimeUs, + Frame.KEY_DISPLAY_TIME_NS, + sequencePresentationTimeUs * 1_000L, + "SOURCE_FRAME_METADATA", + new SourceFrame(sequenceIndex, presentationTimeUs, sequencePresentationTimeUs))) + .setInternalImage(this) + .build(); + return new AsyncFrame(frame, /* acquireFence= */ null); + } + + private List createFrameList(int numFrames, float frameRate, int sequenceIndex) { + return createFrameList( + numFrames, frameRate, sequenceIndex, /* sequencePresentationTimeOffsetUs= */ 0); + } + + private List createFrameList( + int numFrames, float frameRate, int sequenceIndex, long sequencePresentationTimeOffsetUs) { + List frames = new ArrayList<>(); + for (int i = 0; i < numFrames; i++) { + long timeUs = Math.round(i * 1_000_000.0 / frameRate); + frames.add( + createFrame( + /* presentationTimeUs= */ timeUs, + /* sequencePresentationTimeUs= */ sequencePresentationTimeOffsetUs + timeUs, + sequenceIndex)); + } + return frames; + } + + private static SourceFrame getSourceFrame(AsyncFrame frame) { + return (SourceFrame) frame.frame.getMetadata().get("SOURCE_FRAME_METADATA"); + } + + private static long getPresentationTimeUs(AsyncFrame asyncFrame) { + return (Long) asyncFrame.frame.getMetadata().get(Frame.KEY_PRESENTATION_TIME_US); } private static void registerAllSequences(FrameAggregator frameAggregator, int numSequences) { + registerAllSequences(frameAggregator, numSequences, STRATEGY_MATCH_FRAME_CLOSEST_TO_TARGET); + } + + private static void registerAllSequences( + FrameAggregator frameAggregator, + int numSequences, + @AggregationStrategy int aggregationStrategy) { for (int i = 0; i < numSequences; i++) { - frameAggregator.registerSequence(i, /* shouldAggregate= */ true); + frameAggregator.registerSequence(i, aggregationStrategy); + } + } + + private static final class SourceFrame { + final int sequenceIndex; + final long presentationTimeUs; + final long sequencePresentationTimeUs; + + SourceFrame(int sequenceIndex, long presentationTimeUs, long sequencePresentationTimeUs) { + this.sequenceIndex = sequenceIndex; + this.presentationTimeUs = presentationTimeUs; + this.sequencePresentationTimeUs = sequencePresentationTimeUs; } + + @Override + @SuppressWarnings("PatternMatchingInstanceof") + public boolean equals(@Nullable Object obj) { + if (!(obj instanceof SourceFrame)) { + return false; + } + SourceFrame other = (SourceFrame) obj; + return sequenceIndex == other.sequenceIndex + && presentationTimeUs == other.presentationTimeUs + && sequencePresentationTimeUs == other.sequencePresentationTimeUs; + } + + @Override + public int hashCode() { + return Objects.hash(sequenceIndex, presentationTimeUs, sequencePresentationTimeUs); + } + + @Override + public String toString() { + return "SourceFrame[sequence=" + + sequenceIndex + + ", pts=" + + presentationTimeUs + + ", seqPts=" + + sequencePresentationTimeUs + + "]"; + } + } + + private void recordOutputFrames(ImmutableList asyncFrames) { + outputFrames.add(asyncFrames); } } diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/HardwareBufferFrameQueueToFrameWriterAdapterTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/HardwareBufferFrameQueueToFrameWriterAdapterTest.java deleted file mode 100644 index dfab59e761d..00000000000 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/HardwareBufferFrameQueueToFrameWriterAdapterTest.java +++ /dev/null @@ -1,119 +0,0 @@ -/* - * Copyright 2026 The Android Open Source Project - * - * Licensed under the Apache License, Version 2.0 (the "License"); - * you may not use this file except in compliance with the License. - * You may obtain a copy of the License at - * - * https://www.apache.org/licenses/LICENSE-2.0 - * - * Unless required by applicable law or agreed to in writing, software - * distributed under the License is distributed on an "AS IS" BASIS, - * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. - * See the License for the specific language governing permissions and - * limitations under the License. - */ -package androidx.media3.transformer; - -import static com.google.common.truth.Truth.assertThat; -import static com.google.common.util.concurrent.MoreExecutors.directExecutor; - -import android.hardware.HardwareBuffer; -import androidx.annotation.Nullable; -import androidx.annotation.OptIn; -import androidx.media3.common.C; -import androidx.media3.common.ColorInfo; -import androidx.media3.common.Format; -import androidx.media3.common.util.ExperimentalApi; -import androidx.media3.common.video.AsyncFrame; -import androidx.media3.effect.HardwareBufferFrame; -import androidx.media3.effect.HardwareBufferFrameQueue; -import androidx.test.ext.junit.runners.AndroidJUnit4; -import org.junit.Test; -import org.junit.runner.RunWith; - -/** Unit tests for {@link HardwareBufferFrameQueueToFrameWriterAdapter}. */ -@RunWith(AndroidJUnit4.class) -@OptIn(markerClass = ExperimentalApi.class) -public final class HardwareBufferFrameQueueToFrameWriterAdapterTest { - - @Test - public void dequeueInputFrame_propagatesPixelFormat() { - FakeHardwareBufferFrameQueue fakeQueue = new FakeHardwareBufferFrameQueue(); - HardwareBufferFrameQueueToFrameWriterAdapter adapter = - new HardwareBufferFrameQueueToFrameWriterAdapter(fakeQueue); - Format format = - new Format.Builder() - .setWidth(100) - .setHeight(200) - .setPixelFormat(HardwareBuffer.RGBA_FP16) - .setColorInfo(ColorInfo.SDR_BT709_LIMITED) - .build(); - adapter.configure(format, /* usage= */ 0); - - AsyncFrame unused = adapter.dequeueInputFrame(directExecutor(), () -> {}); - - assertThat(fakeQueue.lastFormat).isNotNull(); - assertThat(fakeQueue.lastFormat.pixelFormat).isEqualTo(HardwareBuffer.RGBA_FP16); - } - - @Test - public void dequeueInputFrame_noPixelFormat_defaultsToSdr() { - FakeHardwareBufferFrameQueue fakeQueue = new FakeHardwareBufferFrameQueue(); - HardwareBufferFrameQueueToFrameWriterAdapter adapter = - new HardwareBufferFrameQueueToFrameWriterAdapter(fakeQueue); - Format format = - new Format.Builder() - .setWidth(100) - .setHeight(200) - .setColorInfo(ColorInfo.SDR_BT709_LIMITED) - .build(); - adapter.configure(format, /* usage= */ 0); - - AsyncFrame unused = adapter.dequeueInputFrame(directExecutor(), () -> {}); - - assertThat(fakeQueue.lastFormat).isNotNull(); - assertThat(fakeQueue.lastFormat.pixelFormat).isEqualTo(HardwareBuffer.RGBA_8888); - } - - @Test - public void dequeueInputFrame_noPixelFormat_defaultsToHdr() { - FakeHardwareBufferFrameQueue fakeQueue = new FakeHardwareBufferFrameQueue(); - HardwareBufferFrameQueueToFrameWriterAdapter adapter = - new HardwareBufferFrameQueueToFrameWriterAdapter(fakeQueue); - ColorInfo hdrColorInfo = - new ColorInfo.Builder() - .setColorSpace(C.COLOR_SPACE_BT2020) - .setColorRange(C.COLOR_RANGE_LIMITED) - .setColorTransfer(C.COLOR_TRANSFER_ST2084) - .build(); - Format format = - new Format.Builder().setWidth(100).setHeight(200).setColorInfo(hdrColorInfo).build(); - adapter.configure(format, /* usage= */ 0); - - AsyncFrame unused = adapter.dequeueInputFrame(directExecutor(), () -> {}); - - assertThat(fakeQueue.lastFormat).isNotNull(); - assertThat(fakeQueue.lastFormat.pixelFormat).isEqualTo(HardwareBuffer.RGBA_1010102); - } - - private static final class FakeHardwareBufferFrameQueue implements HardwareBufferFrameQueue { - @Nullable private FrameFormat lastFormat; - - @Nullable - @Override - public HardwareBufferFrame dequeue(FrameFormat format, Runnable wakeupListener) { - lastFormat = format; - return null; - } - - @Override - public void queue(HardwareBufferFrame frame) {} - - @Override - public void signalEndOfStream() {} - - @Override - public void release() {} - } -} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/HardwareBufferFrameReaderTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/HardwareBufferFrameReaderTest.java index 4891dff7e8e..c46a4c3fc49 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/HardwareBufferFrameReaderTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/HardwareBufferFrameReaderTest.java @@ -17,19 +17,34 @@ import static androidx.media3.test.utils.AssetInfo.MP4_ADVANCED_ASSET; import static androidx.media3.transformer.EditedMediaItemSequence.withAudioFrom; +import static androidx.media3.transformer.HardwareBufferFrameReader.CAPACITY; +import static androidx.media3.transformer.TransformerUtil.END_OF_STREAM_ASYNC_FRAME; +import static androidx.media3.transformer.TransformerUtil.releaseIfNeeded; import static com.google.common.truth.Truth.assertThat; +import static java.lang.Math.round; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; import static org.robolectric.Shadows.shadowOf; import android.graphics.Bitmap; +import android.graphics.Gainmap; import android.graphics.ImageFormat; +import android.hardware.HardwareBuffer; import android.os.HandlerThread; +import androidx.media3.common.C; import androidx.media3.common.Format; import androidx.media3.common.MediaItem; +import androidx.media3.common.MimeTypes; import androidx.media3.common.util.ConstantRateTimestampIterator; import androidx.media3.common.util.SystemClock; import androidx.media3.common.util.TimestampIterator; import androidx.media3.common.util.Util; -import androidx.media3.effect.HardwareBufferFrame; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.DefaultHardwareBufferFrame; +import androidx.media3.common.video.Frame; +import androidx.media3.common.video.HardwareBufferFrame; +import androidx.media3.effect.HardwareBufferJniWrapper; import androidx.media3.transformer.HardwareBufferFrameReader.RendererWakeupListener; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.collect.ImmutableList; @@ -41,39 +56,46 @@ import org.junit.Before; import org.junit.Test; import org.junit.runner.RunWith; +import org.robolectric.annotation.Config; /** Robolectric tests for {@link HardwareBufferFrameReader}. */ @RunWith(AndroidJUnit4.class) public class HardwareBufferFrameReaderTest { - private List receivedFrames; + private List receivedFrames; private HardwareBufferFrameReader hardwareBufferFrameReader; private HandlerThread handlerThread; private AtomicReference hardwareBufferFrameReaderException; @Before public void setUp() { - EditedMediaItem editedMediaItem = + EditedMediaItem editedMediaItem1 = + new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)).build(); + EditedMediaItem editedMediaItem2 = new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)).build(); - EditedMediaItemSequence sequence = withAudioFrom(ImmutableList.of(editedMediaItem)); + EditedMediaItemSequence sequence = + withAudioFrom(ImmutableList.of(editedMediaItem1, editedMediaItem2)); Composition composition = new Composition.Builder(sequence).build(); handlerThread = new HandlerThread("HardwareBufferFrameReaderTest"); handlerThread.start(); hardwareBufferFrameReaderException = new AtomicReference<>(); receivedFrames = new ArrayList<>(); + HardwareBufferJniWrapper mockJniWrapper = mock(HardwareBufferJniWrapper.class); + when(mockJniWrapper.nativeCopyBitmapToHardwareBuffer(any(), any())).thenReturn(true); hardwareBufferFrameReader = new HardwareBufferFrameReader( composition, /* sequenceIndex= */ 0, - /* frameConsumer= */ hardwareBufferFrame -> { - receivedFrames.add(hardwareBufferFrame); + /* frameConsumer= */ asyncFrame -> { + receivedFrames.add(asyncFrame); }, handlerThread.getLooper(), /* defaultSurfacePixelFormat= */ ImageFormat.YUV_420_888, new DefaultImageReaderAdapter.Factory(), /* listener= */ e -> hardwareBufferFrameReaderException.set(e), - SystemClock.DEFAULT.createHandler(Util.getCurrentOrMainLooper(), /* callback= */ null)); + SystemClock.DEFAULT.createHandler(Util.getCurrentOrMainLooper(), /* callback= */ null), + mockJniWrapper); } @After @@ -142,11 +164,9 @@ public void outputBitmap_outputsBitmapsUpToCapacityImmediately() { // Trying to output 30 frames. Stop outputting when the output capacity is reached. assertThat(thirtyFrames.hasNext()).isTrue(); assertThat(hardwareBufferFrameReader.canAcceptFrameViaSurface()).isFalse(); - assertThat(receivedFrames).hasSize(2); - assertThat(receivedFrames.get(0).presentationTimeUs).isEqualTo(0); - assertThat(receivedFrames.get(0).sequencePresentationTimeUs).isEqualTo(0); - assertThat(receivedFrames.get(1).presentationTimeUs).isEqualTo(33_333); - assertThat(receivedFrames.get(1).sequencePresentationTimeUs).isEqualTo(33_333); + assertThat(receivedFrames).hasSize(CAPACITY); + assertFramePresentationTimes( + receivedFrames, /* firstFrameIndex= */ 0, /* sequenceOffsetUs= */ 0); assertThat(hardwareBufferFrameReaderException.get()).isNull(); } @@ -162,29 +182,24 @@ public void releaseFrame_outputsMoreBitmaps() { /* indexOfItem= */ 0); // Trying to output 30 frames. Stop outputting when the output capacity is reached. - assertThat(receivedFrames).hasSize(2); + assertThat(receivedFrames).hasSize(CAPACITY); - receivedFrames.get(0).release(/* releaseFence= */ null); + releaseIfNeeded(receivedFrames.get(0).frame, /* releaseFence= */ null); shadowOf(handlerThread.getLooper()).idle(); // Once a frame is released, more output can be generated. - assertThat(receivedFrames).hasSize(3); - assertThat(receivedFrames.get(0).presentationTimeUs).isEqualTo(0); - assertThat(receivedFrames.get(0).sequencePresentationTimeUs).isEqualTo(0); - assertThat(receivedFrames.get(1).presentationTimeUs).isEqualTo(33_333); - assertThat(receivedFrames.get(1).sequencePresentationTimeUs).isEqualTo(33_333); - assertThat(receivedFrames.get(2).presentationTimeUs).isEqualTo(66_667); - assertThat(receivedFrames.get(2).sequencePresentationTimeUs).isEqualTo(66_667); + assertThat(receivedFrames).hasSize(CAPACITY + 1); + assertFramePresentationTimes( + receivedFrames, /* firstFrameIndex= */ 0, /* sequenceOffsetUs= */ 0); assertThat(hardwareBufferFrameReaderException.get()).isNull(); } @Test public void addWakeupListenerProvider_releaseFrame_callsRendererWakeupListener() { - TimestampIterator twoFrames = - new ConstantRateTimestampIterator(/* durationUs= */ 1_000_000, /* frameRate= */ 2f); + TimestampIterator capacityFrames = createTimestampIterator(CAPACITY); hardwareBufferFrameReader.outputBitmap( Bitmap.createBitmap(/* width= */ 1, /* height= */ 1, Bitmap.Config.ARGB_8888), - /* timestampIterator= */ twoFrames, + /* timestampIterator= */ capacityFrames, /* sequenceOffsetUs= */ 0, /* indexOfItem= */ 0); @@ -192,7 +207,7 @@ public void addWakeupListenerProvider_releaseFrame_callsRendererWakeupListener() hardwareBufferFrameReader.addRendererWakeupListener(() -> onWakeupCalled.set(true)); assertThat(hardwareBufferFrameReader.canAcceptFrameViaSurface()).isFalse(); - receivedFrames.get(0).release(/* releaseFence= */ null); + releaseIfNeeded(receivedFrames.get(0).frame, /* releaseFence= */ null); shadowOf(handlerThread.getLooper()).idle(); assertThat(onWakeupCalled.get()).isTrue(); @@ -203,12 +218,11 @@ public void addWakeupListenerProvider_releaseFrame_callsRendererWakeupListener() public void releaseFrame_afterRemoveWakeupListenerProvider_doesNotRendererWakeupListener() { AtomicBoolean onWakeupCalled = new AtomicBoolean(); RendererWakeupListener rendererWakeupListener = () -> onWakeupCalled.set(true); - TimestampIterator twoFrames = - new ConstantRateTimestampIterator(/* durationUs= */ 1_000_000, /* frameRate= */ 2f); + TimestampIterator capacityFrames = createTimestampIterator(CAPACITY); hardwareBufferFrameReader.outputBitmap( Bitmap.createBitmap(/* width= */ 1, /* height= */ 1, Bitmap.Config.ARGB_8888), - /* timestampIterator= */ twoFrames, + /* timestampIterator= */ capacityFrames, /* sequenceOffsetUs= */ 0, /* indexOfItem= */ 0); @@ -216,7 +230,7 @@ public void releaseFrame_afterRemoveWakeupListenerProvider_doesNotRendererWakeup assertThat(hardwareBufferFrameReader.canAcceptFrameViaSurface()).isFalse(); hardwareBufferFrameReader.removeRendererWakeupListener(rendererWakeupListener); - receivedFrames.get(0).release(/* releaseFence= */ null); + releaseIfNeeded(receivedFrames.get(0).frame, /* releaseFence= */ null); shadowOf(handlerThread.getLooper()).idle(); assertThat(onWakeupCalled.get()).isFalse(); @@ -229,7 +243,7 @@ public void queueEndOfStream_outputsEosFrame() { shadowOf(handlerThread.getLooper()).idle(); assertThat(hardwareBufferFrameReaderException.get()).isNull(); - assertThat(receivedFrames).containsExactly(HardwareBufferFrame.END_OF_STREAM_FRAME); + assertThat(receivedFrames).containsExactly(END_OF_STREAM_ASYNC_FRAME); } @Test @@ -245,10 +259,10 @@ public void outputBitmap_thenQueueEndOfStream_outputsBitmapThenEos() { assertThat(hardwareBufferFrameReaderException.get()).isNull(); assertThat(receivedFrames).hasSize(2); - assertThat(receivedFrames.get(0).internalFrame).isInstanceOf(Bitmap.class); - assertThat(receivedFrames.get(0).presentationTimeUs).isEqualTo(0); - assertThat(receivedFrames.get(0).sequencePresentationTimeUs).isEqualTo(0); - assertThat(receivedFrames.get(1)).isEqualTo(HardwareBufferFrame.END_OF_STREAM_FRAME); + assertThat(getInternalImage(receivedFrames.get(0))).isInstanceOf(Bitmap.class); + assertThat(getPresentationTimeUs(receivedFrames.get(0))).isEqualTo(0); + assertThat(receivedFrames.get(0).frame.getContentTimeUs()).isEqualTo(0); + assertThat(receivedFrames.get(1)).isEqualTo(END_OF_STREAM_ASYNC_FRAME); } @Test @@ -263,20 +277,15 @@ public void outputBitmap_withSequenceOffset_outputsFramesWithIncreasingSequenceT sequenceOffsetUs, /* indexOfItem= */ 1); - assertThat(receivedFrames).hasSize(2); - assertThat(receivedFrames.get(0).presentationTimeUs).isEqualTo(0); - assertThat(receivedFrames.get(0).sequencePresentationTimeUs).isEqualTo(100_000); - - assertThat(receivedFrames.get(1).presentationTimeUs).isEqualTo(33_333); - assertThat(receivedFrames.get(1).sequencePresentationTimeUs).isEqualTo(133_333); + assertThat(receivedFrames).hasSize(CAPACITY); + assertFramePresentationTimes(receivedFrames, /* firstFrameIndex= */ 0, sequenceOffsetUs); - receivedFrames.get(0).release(/* releaseFence= */ null); + releaseIfNeeded(receivedFrames.get(0).frame, /* releaseFence= */ null); shadowOf(handlerThread.getLooper()).idle(); - // Check that as presentation time increments, sequence time also correctly increments. - assertThat(receivedFrames).hasSize(3); - assertThat(receivedFrames.get(2).presentationTimeUs).isEqualTo(66_667); - assertThat(receivedFrames.get(2).sequencePresentationTimeUs).isEqualTo(166_667); + // Check that as presentation time increments, sequence time also correctly increments.\ + assertThat(receivedFrames).hasSize(CAPACITY + 1); + assertFramePresentationTimes(receivedFrames, /* firstFrameIndex= */ 0, sequenceOffsetUs); } @Test @@ -293,10 +302,10 @@ public void queueEndOfStream_thenOutputBitmap_outputsEosThenBitmap() { assertThat(hardwareBufferFrameReaderException.get()).isNull(); assertThat(receivedFrames).hasSize(2); - assertThat(receivedFrames.get(0)).isEqualTo(HardwareBufferFrame.END_OF_STREAM_FRAME); - assertThat(receivedFrames.get(1).internalFrame).isInstanceOf(Bitmap.class); - assertThat(receivedFrames.get(1).presentationTimeUs).isEqualTo(0); - assertThat(receivedFrames.get(1).sequencePresentationTimeUs).isEqualTo(0); + assertThat(receivedFrames.get(0)).isEqualTo(END_OF_STREAM_ASYNC_FRAME); + assertThat(getInternalImage(receivedFrames.get(1))).isInstanceOf(Bitmap.class); + assertThat(getPresentationTimeUs(receivedFrames.get(1))).isEqualTo(0); + assertThat(receivedFrames.get(1).frame.getContentTimeUs()).isEqualTo(0); } @Test @@ -307,45 +316,44 @@ public void queueEndOfStream_multipleTimes_outputsMultipleEosFrames() { assertThat(hardwareBufferFrameReaderException.get()).isNull(); assertThat(receivedFrames) - .containsExactly( - HardwareBufferFrame.END_OF_STREAM_FRAME, HardwareBufferFrame.END_OF_STREAM_FRAME) + .containsExactly(END_OF_STREAM_ASYNC_FRAME, END_OF_STREAM_ASYNC_FRAME) .inOrder(); } @Test public void queueEndOfStream_afterBitmapsFillCapacity_isOutputAfterBitmapsReleased() { - TimestampIterator threeFrames = - new ConstantRateTimestampIterator(/* durationUs= */ 100_000, /* frameRate= */ 30f); + TimestampIterator capacityPlusOneFrames = createTimestampIterator(CAPACITY + 1); Bitmap bitmap = Bitmap.createBitmap(1, 1, Bitmap.Config.ARGB_8888); - // Queue 3 bitmaps and EOS. Only the first 2 should be output immediately, filling the capacity. + // Queue one more bitmap than the capacity, and EOS. Only the bitmaps filling the capacity + // should be output immediately. hardwareBufferFrameReader.outputBitmap( - bitmap, threeFrames, /* sequenceOffsetUs= */ 0, /* indexOfItem= */ 0); + bitmap, capacityPlusOneFrames, /* sequenceOffsetUs= */ 0, /* indexOfItem= */ 0); hardwareBufferFrameReader.queueEndOfStream(); shadowOf(handlerThread.getLooper()).idle(); - assertThat(receivedFrames).hasSize(2); - assertThat(receivedFrames.get(0).presentationTimeUs).isEqualTo(0); - assertThat(receivedFrames.get(0).sequencePresentationTimeUs).isEqualTo(0); - assertThat(receivedFrames.get(1).presentationTimeUs).isEqualTo(33_333); - assertThat(receivedFrames.get(1).sequencePresentationTimeUs).isEqualTo(33_333); + assertThat(receivedFrames).hasSize(CAPACITY); + assertFramePresentationTimes( + receivedFrames, /* firstFrameIndex= */ 0, /* sequenceOffsetUs= */ 0); - receivedFrames.remove(0).release(/* releaseFence= */ null); + releaseIfNeeded(receivedFrames.remove(0).frame, /* releaseFence= */ null); shadowOf(handlerThread.getLooper()).idle(); - assertThat(receivedFrames).hasSize(2); - assertThat(receivedFrames.get(0).presentationTimeUs).isEqualTo(33_333); - assertThat(receivedFrames.get(0).sequencePresentationTimeUs).isEqualTo(33_333); - assertThat(receivedFrames.get(1).presentationTimeUs).isEqualTo(66_667); - assertThat(receivedFrames.get(1).sequencePresentationTimeUs).isEqualTo(66_667); - assertThat(threeFrames.hasNext()).isFalse(); + // The last bitmap is output, keeping the capacity filled. + assertThat(receivedFrames).hasSize(CAPACITY); + assertFramePresentationTimes( + receivedFrames, /* firstFrameIndex= */ 1, /* sequenceOffsetUs= */ 0); + assertThat(capacityPlusOneFrames.hasNext()).isFalse(); - receivedFrames.remove(0).release(/* releaseFence= */ null); + releaseIfNeeded(receivedFrames.remove(0).frame, /* releaseFence= */ null); shadowOf(handlerThread.getLooper()).idle(); - assertThat(receivedFrames).hasSize(2); - assertThat(receivedFrames.get(0).presentationTimeUs).isEqualTo(66_667); - assertThat(receivedFrames.get(0).sequencePresentationTimeUs).isEqualTo(66_667); - assertThat(receivedFrames.get(1)).isEqualTo(HardwareBufferFrame.END_OF_STREAM_FRAME); + // With no bitmaps left, EOS is output after the remaining bitmaps. + assertThat(receivedFrames).hasSize(CAPACITY); + assertFramePresentationTimes( + receivedFrames.subList(0, CAPACITY - 1), + /* firstFrameIndex= */ 2, + /* sequenceOffsetUs= */ 0); + assertThat(receivedFrames.get(CAPACITY - 1)).isEqualTo(END_OF_STREAM_ASYNC_FRAME); assertThat(hardwareBufferFrameReaderException.get()).isNull(); } @@ -360,19 +368,201 @@ public void flush_preventsMoreFramesFromBeingOutput() { /* sequenceOffsetUs= */ 0, /* indexOfItem= */ 0); - // Trying to output 30 frames, but HardwareBufferFrameReader#CAPACITY is only 2. - assertThat(receivedFrames).hasSize(2); + // Trying to output 30 frames, but HardwareBufferFrameReader#CAPACITY is smaller. + assertThat(receivedFrames).hasSize(CAPACITY); hardwareBufferFrameReader.flush(); - receivedFrames.get(0).release(/* releaseFence= */ null); + releaseIfNeeded(receivedFrames.get(0).frame, /* releaseFence= */ null); shadowOf(handlerThread.getLooper()).idle(); // Once a frame is released, more output can be accepted downstream. But calling flush() // clears the remaining frames from the thirtyFrames timestamp iterator. - assertThat(receivedFrames).hasSize(2); - assertThat(receivedFrames.get(0).presentationTimeUs).isEqualTo(0); - assertThat(receivedFrames.get(1).presentationTimeUs).isEqualTo(33_333); + assertThat(receivedFrames).hasSize(CAPACITY); + assertFramePresentationTimes( + receivedFrames, /* firstFrameIndex= */ 0, /* sequenceOffsetUs= */ 0); assertThat(hardwareBufferFrameReaderException.get()).isNull(); } + + @Test + public void outputBitmap_withSdrBitmap_outputsImageRawFormatWithSdrColorTransfer() { + Bitmap bitmap = Bitmap.createBitmap(1, 1, Bitmap.Config.ARGB_8888); + TimestampIterator singleFrame = + new ConstantRateTimestampIterator(/* durationUs= */ 1_000, /* frameRate= */ 1f); + + hardwareBufferFrameReader.outputBitmap( + bitmap, + /* timestampIterator= */ singleFrame, + /* sequenceOffsetUs= */ 0, + /* indexOfItem= */ 0); + + assertThat(receivedFrames).hasSize(1); + Format format = receivedFrames.get(0).frame.getFormat(); + assertThat(format.sampleMimeType).isEqualTo(MimeTypes.IMAGE_RAW); + assertThat(format.colorInfo).isNotNull(); + assertThat(format.colorInfo.colorTransfer).isEqualTo(C.COLOR_TRANSFER_SDR); + assertThat(hardwareBufferFrameReaderException.get()).isNull(); + } + + @Config(sdk = 34) + @Test + public void outputBitmap_withUltraHdrBitmap_outputsImageJpegRFormatWithSrgbColorTransfer() { + Bitmap bitmap = Bitmap.createBitmap(1, 1, Bitmap.Config.ARGB_8888); + Bitmap gainmapBitmap = Bitmap.createBitmap(1, 1, Bitmap.Config.ALPHA_8); + Gainmap gainmap = new Gainmap(gainmapBitmap); + bitmap.setGainmap(gainmap); + TimestampIterator singleFrame = + new ConstantRateTimestampIterator(/* durationUs= */ 1_000, /* frameRate= */ 1f); + + hardwareBufferFrameReader.outputBitmap( + bitmap, + /* timestampIterator= */ singleFrame, + /* sequenceOffsetUs= */ 0, + /* indexOfItem= */ 0); + + assertThat(receivedFrames).hasSize(1); + Format format = receivedFrames.get(0).frame.getFormat(); + assertThat(format.sampleMimeType).isEqualTo(MimeTypes.IMAGE_JPEG_R); + assertThat(format.colorInfo).isNotNull(); + assertThat(format.colorInfo.colorTransfer).isEqualTo(C.COLOR_TRANSFER_SRGB); + assertThat(format.colorInfo.colorSpace).isEqualTo(C.COLOR_SPACE_BT709); + assertThat(format.colorInfo.colorRange).isEqualTo(C.COLOR_RANGE_FULL); + assertThat(hardwareBufferFrameReaderException.get()).isNull(); + } + + @Test + public void + outputBitmap_withBitmapToHardwareBufferConverter_reusesHardwareBufferForRepeatedBitmap() { + HardwareBufferJniWrapper mockJniWrapper = mock(HardwareBufferJniWrapper.class); + when(mockJniWrapper.nativeCopyBitmapToHardwareBuffer(any(), any())).thenReturn(true); + + EditedMediaItem editedMediaItem = + new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)).build(); + Composition composition = + new Composition.Builder(withAudioFrom(ImmutableList.of(editedMediaItem))).build(); + List frames = new ArrayList<>(); + HardwareBufferFrameReader frameReader = + new HardwareBufferFrameReader( + composition, + /* sequenceIndex= */ 0, + /* frameConsumer= */ frames::add, + handlerThread.getLooper(), + /* defaultSurfacePixelFormat= */ ImageFormat.YUV_420_888, + new DefaultImageReaderAdapter.Factory(), + /* listener= */ e -> hardwareBufferFrameReaderException.set(e), + SystemClock.DEFAULT.createHandler(Util.getCurrentOrMainLooper(), /* callback= */ null), + mockJniWrapper); + + TimestampIterator thirtyFrames = + new ConstantRateTimestampIterator(/* durationUs= */ 1_000_000, /* frameRate= */ 30f); + Bitmap bitmap = Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888); + + frameReader.outputBitmap(bitmap, thirtyFrames, /* sequenceOffsetUs= */ 0, /* indexOfItem= */ 0); + + assertThat(frames).hasSize(CAPACITY); + for (int i = 0; i < CAPACITY; i++) { + DefaultHardwareBufferFrame frame = (DefaultHardwareBufferFrame) frames.get(i).frame; + assertThat(frame).isNotNull(); + assertThat(frame.getHardwareBuffer()) + .isSameInstanceAs(((DefaultHardwareBufferFrame) frames.get(0).frame).getHardwareBuffer()); + assertThat(frame.getInternalImage()).isSameInstanceAs(bitmap); + } + assertFramePresentationTimes(frames, /* firstFrameIndex= */ 0, /* sequenceOffsetUs= */ 0); + + releaseIfNeeded(frames.get(0).frame, /* releaseFence= */ null); + shadowOf(handlerThread.getLooper()).idle(); + + assertThat(frames).hasSize(CAPACITY + 1); + DefaultHardwareBufferFrame frame = (DefaultHardwareBufferFrame) frames.get(CAPACITY).frame; + assertThat(frame.getHardwareBuffer()) + .isSameInstanceAs(((DefaultHardwareBufferFrame) frames.get(0).frame).getHardwareBuffer()); + assertThat(frame.getInternalImage()).isSameInstanceAs(bitmap); + assertFramePresentationTimes(frames, /* firstFrameIndex= */ 0, /* sequenceOffsetUs= */ 0); + assertThat(hardwareBufferFrameReaderException.get()).isNull(); + + for (int i = 1; i < CAPACITY; i++) { + releaseIfNeeded(frames.get(1).frame, /* releaseFence= */ null); + } + frameReader.release(); + } + + @Test + public void outputBitmap_withConverterAndDifferentBitmap_createsNewHardwareBuffer() { + HardwareBufferJniWrapper mockJniWrapper = mock(HardwareBufferJniWrapper.class); + when(mockJniWrapper.nativeCopyBitmapToHardwareBuffer(any(), any())).thenReturn(true); + + EditedMediaItem editedMediaItem1 = + new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)).build(); + EditedMediaItem editedMediaItem2 = + new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)).build(); + Composition composition = + new Composition.Builder(withAudioFrom(ImmutableList.of(editedMediaItem1, editedMediaItem2))) + .build(); + List frames = new ArrayList<>(); + HardwareBufferFrameReader frameReader = + new HardwareBufferFrameReader( + composition, + /* sequenceIndex= */ 0, + /* frameConsumer= */ frames::add, + handlerThread.getLooper(), + /* defaultSurfacePixelFormat= */ ImageFormat.YUV_420_888, + new DefaultImageReaderAdapter.Factory(), + /* listener= */ e -> hardwareBufferFrameReaderException.set(e), + SystemClock.DEFAULT.createHandler(Util.getCurrentOrMainLooper(), /* callback= */ null), + mockJniWrapper); + + TimestampIterator singleFrame1 = + new ConstantRateTimestampIterator(/* durationUs= */ 1_000, /* frameRate= */ 1f); + TimestampIterator singleFrame2 = + new ConstantRateTimestampIterator(/* durationUs= */ 1_000, /* frameRate= */ 1f); + Bitmap bitmap1 = Bitmap.createBitmap(10, 10, Bitmap.Config.ARGB_8888); + Bitmap bitmap2 = Bitmap.createBitmap(20, 20, Bitmap.Config.ARGB_8888); + + frameReader.outputBitmap( + bitmap1, singleFrame1, /* sequenceOffsetUs= */ 0, /* indexOfItem= */ 0); + frameReader.outputBitmap( + bitmap2, singleFrame2, /* sequenceOffsetUs= */ 10_000, /* indexOfItem= */ 1); + + assertThat(frames).hasSize(2); + assertThat(getHardwareBuffer(frames.get(0))).isNotNull(); + assertThat(getHardwareBuffer(frames.get(1))).isNotNull(); + assertThat(getHardwareBuffer(frames.get(0))) + .isNotSameInstanceAs(getHardwareBuffer(frames.get(1))); + + frameReader.release(); + } + + private static long getPresentationTimeUs(AsyncFrame asyncFrame) { + Long presentationTimeUs = + (Long) asyncFrame.frame.getMetadata().get(Frame.KEY_PRESENTATION_TIME_US); + return presentationTimeUs != null ? presentationTimeUs : asyncFrame.frame.getContentTimeUs(); + } + + private static Object getInternalImage(AsyncFrame asyncFrame) { + return ((DefaultHardwareBufferFrame) asyncFrame.frame).getInternalImage(); + } + + private static HardwareBuffer getHardwareBuffer(AsyncFrame asyncFrame) { + return ((HardwareBufferFrame) asyncFrame.frame).getHardwareBuffer(); + } + + /** Returns a 30 fps {@link TimestampIterator} producing exactly {@code frameCount} timestamps. */ + private static TimestampIterator createTimestampIterator(int frameCount) { + return new ConstantRateTimestampIterator( + /* durationUs= */ round(frameCount * (C.MICROS_PER_SECOND / 30f)), /* frameRate= */ 30f); + } + + /** + * Asserts that {@code frames} have the presentation times of the 30 fps frames starting at {@code + * firstFrameIndex}, with sequence presentation times shifted by {@code sequenceOffsetUs}. + */ + private static void assertFramePresentationTimes( + List frames, int firstFrameIndex, long sequenceOffsetUs) { + for (int i = 0; i < frames.size(); i++) { + long presentationTimeUs = round((firstFrameIndex + i) * (C.MICROS_PER_SECOND / 30f)); + assertThat(getPresentationTimeUs(frames.get(i))).isEqualTo(presentationTimeUs); + assertThat(frames.get(i).frame.getContentTimeUs()) + .isEqualTo(presentationTimeUs + sequenceOffsetUs); + } + } } diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/HardwareBufferSampleConsumerTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/HardwareBufferSampleConsumerTest.java index a2e2703d85e..c6765ef5fbc 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/HardwareBufferSampleConsumerTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/HardwareBufferSampleConsumerTest.java @@ -17,22 +17,34 @@ import static androidx.media3.test.utils.AssetInfo.MP4_ADVANCED_ASSET; import static androidx.media3.transformer.EditedMediaItemSequence.withAudioFrom; +import static androidx.media3.transformer.HardwareBufferFrameReader.CAPACITY; +import static androidx.media3.transformer.TransformerUtil.END_OF_STREAM_ASYNC_FRAME; +import static androidx.media3.transformer.TransformerUtil.releaseIfNeeded; +import static com.google.common.base.Preconditions.checkNotNull; import static com.google.common.truth.Truth.assertThat; +import static java.lang.Math.round; import static org.junit.Assert.assertThrows; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; import static org.robolectric.Shadows.shadowOf; import android.graphics.Bitmap; import android.os.HandlerThread; import android.os.Looper; +import androidx.media3.common.C; import androidx.media3.common.Format; import androidx.media3.common.MediaItem; import androidx.media3.common.MimeTypes; import androidx.media3.common.util.ConstantRateTimestampIterator; import androidx.media3.common.util.HandlerWrapper; import androidx.media3.common.util.SystemClock; -import androidx.media3.effect.HardwareBufferFrame; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.Frame; +import androidx.media3.effect.HardwareBufferJniWrapper; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableSet; import com.google.common.collect.Iterables; import java.util.ArrayList; import java.util.List; @@ -46,7 +58,7 @@ public class HardwareBufferSampleConsumerTest { private HardwareBufferSampleConsumer sampleConsumer; - private List receivedFrames; + private List receivedFrames; private HandlerThread handlerThread; private AtomicReference errorRef; @@ -57,22 +69,7 @@ public void setUp() { errorRef = new AtomicReference<>(); receivedFrames = new ArrayList<>(); - EditedMediaItem editedMediaItem = - new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)).build(); - EditedMediaItemSequence sequence = withAudioFrom(ImmutableList.of(editedMediaItem)); - Composition composition = new Composition.Builder(sequence).build(); - - Looper looper = handlerThread.getLooper(); - HandlerWrapper handlerWrapper = SystemClock.DEFAULT.createHandler(looper, /* callback= */ null); - - sampleConsumer = - new HardwareBufferSampleConsumer( - composition, - /* sequenceIndex= */ 0, - looper, - handlerWrapper, - frame -> receivedFrames.add(frame), - error -> errorRef.set(error)); + sampleConsumer = createSampleConsumer(createComposition(/* itemCount= */ 1)); } @After @@ -100,29 +97,29 @@ public void queueInputBitmap_singleItem_outputsFramesWithCorrectReleaseTimeAndIn .isEqualTo(GraphInput.INPUT_RESULT_SUCCESS); shadowOf(handlerThread.getLooper()).idle(); - assertThat(receivedFrames).hasSize(2); - - HardwareBufferFrame frame1 = receivedFrames.get(0); - // The frame.hardwareBuffer should also be non null, but in roboelectric HardwareBuffers are not - // created properly so this is tested in the AndroidTest. - assertThat(frame1.presentationTimeUs).isEqualTo(0); - assertThat(frame1.sequencePresentationTimeUs).isEqualTo(0); - assertThat(frame1.releaseTimeNs).isEqualTo(0); - assertThat(((CompositionFrameMetadata) frame1.getMetadata()).itemIndex).isEqualTo(0); - // For bitmaps, the format is currently hardcoded in HardwareBufferFrameReader. - assertThat(frame1.format).isNotNull(); - assertThat(frame1.format.sampleMimeType).isEqualTo(MimeTypes.IMAGE_RAW); - - HardwareBufferFrame frame2 = receivedFrames.get(1); - assertThat(frame2.presentationTimeUs).isEqualTo(33_333); - assertThat(frame2.sequencePresentationTimeUs).isEqualTo(33_333); - assertThat(frame2.releaseTimeNs).isEqualTo(33_333 * 1000); - assertThat(((CompositionFrameMetadata) frame2.getMetadata()).itemIndex).isEqualTo(0); - assertThat(frame2.format.sampleMimeType).isEqualTo(MimeTypes.IMAGE_RAW); + // The reader outputs frames up to its capacity before any frame is released. The capacity is 2 + // below API 29 and 3 from API 29, and the timestamp iterator produces 3 frames, so the reader + // outputs exactly CAPACITY frames. + assertThat(receivedFrames).hasSize(CAPACITY); + for (int i = 0; i < CAPACITY; i++) { + AsyncFrame frame = receivedFrames.get(i); + long expectedPresentationTimeUs = round(i * (C.MICROS_PER_SECOND / 30f)); + // The frame.hardwareBuffer should also be non null, but in Robolectric HardwareBuffers are + // not created properly so this is tested in the AndroidTest. + assertThat(getPresentationTimeUs(frame)).isEqualTo(expectedPresentationTimeUs); + assertThat(frame.frame.getContentTimeUs()).isEqualTo(expectedPresentationTimeUs); + assertThat(getDisplayTimeNs(frame)).isEqualTo(expectedPresentationTimeUs * 1000); + assertThat(getCompositionFrameMetadata(frame).itemIndex).isEqualTo(0); + // For bitmaps, the format is currently hardcoded in HardwareBufferFrameReader. + assertThat(frame.frame.getFormat()).isNotNull(); + assertThat(frame.frame.getFormat().sampleMimeType).isEqualTo(MimeTypes.IMAGE_RAW); + } } @Test public void onMediaItemChanged_updatesFormatAndAdjustsReleaseTime() { + sampleConsumer.release(); + sampleConsumer = createSampleConsumer(createComposition(/* itemCount= */ 2)); Bitmap bitmap = Bitmap.createBitmap(1, 1, Bitmap.Config.ARGB_8888); long item1DurationUs = 10_000; Format format1 = new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build(); @@ -151,16 +148,18 @@ public void onMediaItemChanged_updatesFormatAndAdjustsReleaseTime() { shadowOf(handlerThread.getLooper()).idle(); assertThat(receivedFrames).hasSize(1); - HardwareBufferFrame frame = receivedFrames.get(0); + AsyncFrame frame = receivedFrames.get(0); - assertThat(frame.presentationTimeUs).isEqualTo(0); - assertThat(frame.sequencePresentationTimeUs).isEqualTo(item1DurationUs); - assertThat(frame.releaseTimeNs).isEqualTo(item1DurationUs * 1000); - assertThat(((CompositionFrameMetadata) frame.getMetadata()).itemIndex).isEqualTo(1); + assertThat(getPresentationTimeUs(frame)).isEqualTo(0); + assertThat(frame.frame.getContentTimeUs()).isEqualTo(item1DurationUs); + assertThat(getDisplayTimeNs(frame)).isEqualTo(item1DurationUs * 1000); + assertThat(getCompositionFrameMetadata(frame).itemIndex).isEqualTo(1); } @Test public void onMediaItemChanged_accumulatesOffsetsAcrossMultipleItems() { + sampleConsumer.release(); + sampleConsumer = createSampleConsumer(createComposition(/* itemCount= */ 3)); Bitmap bitmap = Bitmap.createBitmap(1, 1, Bitmap.Config.ARGB_8888); long item1DurationUs = 1_000_000; long item2DurationUs = 500_000; @@ -180,14 +179,13 @@ public void onMediaItemChanged_accumulatesOffsetsAcrossMultipleItems() { shadowOf(handlerThread.getLooper()).idle(); assertThat(receivedFrames).hasSize(1); - HardwareBufferFrame frame1 = receivedFrames.get(0); - assertThat(frame1.sequencePresentationTimeUs).isEqualTo(0); - assertThat(frame1.releaseTimeNs).isEqualTo(0); - assertThat(frame1.getMetadata()).isInstanceOf(CompositionFrameMetadata.class); - assertThat(((CompositionFrameMetadata) frame1.getMetadata()).itemIndex).isEqualTo(0); - - for (HardwareBufferFrame frame : receivedFrames) { - frame.release(/* releaseFence= */ null); + AsyncFrame frame1 = receivedFrames.get(0); + assertThat(frame1.frame.getContentTimeUs()).isEqualTo(0); + assertThat(getDisplayTimeNs(frame1)).isEqualTo(0); + assertThat(getCompositionFrameMetadata(frame1).itemIndex).isEqualTo(0); + + for (AsyncFrame frame : receivedFrames) { + releaseIfNeeded(frame.frame, /* releaseFence= */ null); } shadowOf(handlerThread.getLooper()).idle(); receivedFrames.clear(); @@ -206,14 +204,14 @@ public void onMediaItemChanged_accumulatesOffsetsAcrossMultipleItems() { shadowOf(handlerThread.getLooper()).idle(); assertThat(receivedFrames).hasSize(1); - HardwareBufferFrame frame2 = receivedFrames.get(0); - assertThat(frame2.presentationTimeUs).isEqualTo(0); - assertThat(frame2.sequencePresentationTimeUs).isEqualTo(item1DurationUs); - assertThat(frame2.releaseTimeNs).isEqualTo(item1DurationUs * 1000); - assertThat(((CompositionFrameMetadata) frame2.getMetadata()).itemIndex).isEqualTo(1); - - for (HardwareBufferFrame frame : receivedFrames) { - frame.release(/* releaseFence= */ null); + AsyncFrame frame2 = receivedFrames.get(0); + assertThat(getPresentationTimeUs(frame2)).isEqualTo(0); + assertThat(frame2.frame.getContentTimeUs()).isEqualTo(item1DurationUs); + assertThat(getDisplayTimeNs(frame2)).isEqualTo(item1DurationUs * 1000); + assertThat(getCompositionFrameMetadata(frame2).itemIndex).isEqualTo(1); + + for (AsyncFrame frame : receivedFrames) { + releaseIfNeeded(frame.frame, /* releaseFence= */ null); } shadowOf(handlerThread.getLooper()).idle(); receivedFrames.clear(); @@ -232,12 +230,49 @@ public void onMediaItemChanged_accumulatesOffsetsAcrossMultipleItems() { shadowOf(handlerThread.getLooper()).idle(); assertThat(receivedFrames).hasSize(1); - HardwareBufferFrame frame3 = receivedFrames.get(0); + AsyncFrame frame3 = receivedFrames.get(0); long expectedOffsetUs3 = item1DurationUs + item2DurationUs; - assertThat(frame3.presentationTimeUs).isEqualTo(0); - assertThat(frame3.sequencePresentationTimeUs).isEqualTo(expectedOffsetUs3); - assertThat(frame3.releaseTimeNs).isEqualTo(expectedOffsetUs3 * 1000); - assertThat(((CompositionFrameMetadata) frame3.getMetadata()).itemIndex).isEqualTo(2); + assertThat(getPresentationTimeUs(frame3)).isEqualTo(0); + assertThat(frame3.frame.getContentTimeUs()).isEqualTo(expectedOffsetUs3); + assertThat(getDisplayTimeNs(frame3)).isEqualTo(expectedOffsetUs3 * 1000); + assertThat(getCompositionFrameMetadata(frame3).itemIndex).isEqualTo(2); + } + + @Test + public void onMediaItemChanged_withLoopingSequence_wrapsItemIndex() { + sampleConsumer.release(); + sampleConsumer = + createSampleConsumer(createComposition(/* itemCount= */ 2, /* isLooping= */ true)); + Bitmap bitmap = Bitmap.createBitmap(1, 1, Bitmap.Config.ARGB_8888); + Format format = new Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build(); + + for (int expectedItemIndex : new int[] {0, 1, 0}) { + sampleConsumer.onMediaItemChanged( + /* editedMediaItem= */ null, + /* durationUs= */ 100_000, + /* decodedFormat= */ format, + /* isLast= */ false, + /* positionOffsetUs= */ 0); + assertThat( + sampleConsumer.queueInputBitmap( + bitmap, + new ConstantRateTimestampIterator(/* durationUs= */ 1_000, /* frameRate= */ 30f))) + .isEqualTo(GraphInput.INPUT_RESULT_SUCCESS); + shadowOf(handlerThread.getLooper()).idle(); + + assertThat(receivedFrames).hasSize(1); + assertThat(getCompositionFrameMetadata(receivedFrames.get(0)).itemIndex) + .isEqualTo(expectedItemIndex); + releaseIfNeeded(receivedFrames.get(0).frame, /* releaseFence= */ null); + shadowOf(handlerThread.getLooper()).idle(); + receivedFrames.clear(); + } + + sampleConsumer.signalEndOfVideoInput(); + shadowOf(handlerThread.getLooper()).idle(); + + assertThat(receivedFrames).isNotEmpty(); + assertThat(Iterables.getLast(receivedFrames)).isEqualTo(END_OF_STREAM_ASYNC_FRAME); } @Test @@ -275,7 +310,65 @@ public void signalEndOfVideoInput_queuesEndOfStream() { shadowOf(handlerThread.getLooper()).idle(); assertThat(receivedFrames).isNotEmpty(); - assertThat(Iterables.getLast(receivedFrames)) - .isEqualTo(HardwareBufferFrame.END_OF_STREAM_FRAME); + assertThat(Iterables.getLast(receivedFrames)).isEqualTo(END_OF_STREAM_ASYNC_FRAME); + } + + private HardwareBufferSampleConsumer createSampleConsumer(Composition composition) { + return createSampleConsumer(composition, /* sequenceIndex= */ composition.sequences.size() - 1); + } + + private HardwareBufferSampleConsumer createSampleConsumer( + Composition composition, int sequenceIndex) { + Looper looper = handlerThread.getLooper(); + HandlerWrapper handlerWrapper = SystemClock.DEFAULT.createHandler(looper, /* callback= */ null); + + HardwareBufferJniWrapper mockJniWrapper = mock(HardwareBufferJniWrapper.class); + when(mockJniWrapper.nativeCopyBitmapToHardwareBuffer(any(), any())).thenReturn(true); + return new HardwareBufferSampleConsumer( + composition, + sequenceIndex, + looper, + handlerWrapper, + frame -> receivedFrames.add(frame), + error -> errorRef.set(error), + mockJniWrapper); + } + + private static Composition createComposition(int itemCount) { + return createComposition(itemCount, /* isLooping= */ false); + } + + private static Composition createComposition(int itemCount, boolean isLooping) { + ImmutableList.Builder mediaItems = ImmutableList.builder(); + for (int i = 0; i < itemCount; i++) { + mediaItems.add( + new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)).build()); + } + EditedMediaItemSequence sequence = + new EditedMediaItemSequence.Builder(ImmutableSet.of(C.TRACK_TYPE_AUDIO)) + .addItems(mediaItems.build()) + .setIsLooping(isLooping) + .build(); + if (isLooping) { + EditedMediaItem primaryItem = + new EditedMediaItem.Builder(MediaItem.fromUri(MP4_ADVANCED_ASSET.uri)).build(); + EditedMediaItemSequence primarySequence = withAudioFrom(ImmutableList.of(primaryItem)); + return new Composition.Builder(primarySequence, sequence).build(); + } + return new Composition.Builder(sequence).build(); + } + + private static long getPresentationTimeUs(AsyncFrame asyncFrame) { + return checkNotNull((Long) asyncFrame.frame.getMetadata().get(Frame.KEY_PRESENTATION_TIME_US)); + } + + private static long getDisplayTimeNs(AsyncFrame asyncFrame) { + return checkNotNull((Long) asyncFrame.frame.getMetadata().get(Frame.KEY_DISPLAY_TIME_NS)); + } + + @SuppressWarnings("deprecation") // Uses deprecated CompositionFrameMetadata. + private static CompositionFrameMetadata getCompositionFrameMetadata(AsyncFrame asyncFrame) { + return (CompositionFrameMetadata) + asyncFrame.frame.getMetadata().get(CompositionFrameMetadata.KEY_COMPOSITION_FRAME_METADATA); } } diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/InAppFragmentedMp4MuxerTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/InAppFragmentedMp4MuxerTest.java new file mode 100644 index 00000000000..f0f10160e35 --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/InAppFragmentedMp4MuxerTest.java @@ -0,0 +1,93 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; +import static org.mockito.Mockito.verify; + +import android.os.ParcelFileDescriptor; +import androidx.media3.muxer.Muxer; +import androidx.media3.muxer.MuxerException; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.io.File; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import org.junit.runner.RunWith; +import org.mockito.Mockito; + +/** Unit tests for {@link InAppFragmentedMp4Muxer}. */ +@RunWith(AndroidJUnit4.class) +public final class InAppFragmentedMp4MuxerTest { + + @Rule public final TemporaryFolder temporaryFolder = new TemporaryFolder(); + + @Test + public void createAndClose_withParcelFileDescriptor_keepsDescriptorOpenUntilMuxerClosed() + throws Exception { + File tempFile = temporaryFolder.newFile("test_output.mp4"); + InAppFragmentedMp4Muxer.Factory factory = new InAppFragmentedMp4Muxer.Factory(); + + ParcelFileDescriptor pfd = + ParcelFileDescriptor.open(tempFile, ParcelFileDescriptor.MODE_READ_WRITE); + try (Muxer muxer = factory.create(pfd)) { + assertThat(muxer).isNotNull(); + assertThat(pfd.getFileDescriptor().valid()).isTrue(); + } + assertThat(pfd.getFileDescriptor().valid()).isFalse(); + } + + @Test + public void createAndClose_withPipe_keepsPipeOpenUntilMuxerClosed() throws Exception { + ParcelFileDescriptor[] pipe = ParcelFileDescriptor.createPipe(); + InAppFragmentedMp4Muxer.Factory factory = new InAppFragmentedMp4Muxer.Factory(); + try (ParcelFileDescriptor readPipe = pipe[0]; + Muxer muxer = factory.create(pipe[1])) { + assertThat(muxer).isNotNull(); + assertThat(pipe[1].getFileDescriptor().valid()).isTrue(); + } + assertThat(pipe[1].getFileDescriptor().valid()).isFalse(); + } + + @Test + public void create_withClosedParcelFileDescriptor_throwsMuxerException() throws Exception { + File tempFile = temporaryFolder.newFile("test_output.mp4"); + InAppFragmentedMp4Muxer.Factory factory = new InAppFragmentedMp4Muxer.Factory(); + ParcelFileDescriptor pfd = + ParcelFileDescriptor.open(tempFile, ParcelFileDescriptor.MODE_READ_WRITE); + pfd.close(); + + assertThrows(MuxerException.class, () -> factory.create(pfd)); + } + + @Test + public void create_withUnexpectedRuntimeException_throwsMuxerExceptionAndClosesPfd() + throws Exception { + File tempFile = temporaryFolder.newFile("test_output.mp4"); + InAppFragmentedMp4Muxer.Factory factory = new InAppFragmentedMp4Muxer.Factory(); + ParcelFileDescriptor pfd = + ParcelFileDescriptor.open(tempFile, ParcelFileDescriptor.MODE_READ_WRITE); + ParcelFileDescriptor spyPfd = Mockito.spy(pfd); + Mockito.doThrow(new RuntimeException("Simulated unexpected failure")) + .when(spyPfd) + .getFileDescriptor(); + + MuxerException exception = assertThrows(MuxerException.class, () -> factory.create(spyPfd)); + assertThat(exception).hasCauseThat().isInstanceOf(RuntimeException.class); + verify(spyPfd).close(); + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/InAppMp4MuxerTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/InAppMp4MuxerTest.java new file mode 100644 index 00000000000..fbc79126469 --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/InAppMp4MuxerTest.java @@ -0,0 +1,105 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; +import static org.mockito.Mockito.verify; + +import android.os.ParcelFileDescriptor; +import androidx.media3.muxer.Muxer; +import androidx.media3.muxer.MuxerException; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.io.File; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import org.junit.runner.RunWith; +import org.mockito.Mockito; + +/** Unit tests for {@link InAppMp4Muxer}. */ +@RunWith(AndroidJUnit4.class) +public final class InAppMp4MuxerTest { + + @Rule public final TemporaryFolder temporaryFolder = new TemporaryFolder(); + + @Test + public void createAndClose_withParcelFileDescriptor_keepsDescriptorOpenUntilMuxerClosed() + throws Exception { + File tempFile = temporaryFolder.newFile("test_output.mp4"); + InAppMp4Muxer.Factory factory = new InAppMp4Muxer.Factory(); + + ParcelFileDescriptor pfd = + ParcelFileDescriptor.open(tempFile, ParcelFileDescriptor.MODE_READ_WRITE); + try (Muxer muxer = factory.create(pfd)) { + assertThat(muxer).isNotNull(); + assertThat(pfd.getFileDescriptor().valid()).isTrue(); + } + assertThat(pfd.getFileDescriptor().valid()).isFalse(); + } + + @Test + public void create_withNonSeekableParcelFileDescriptor_throwsMuxerExceptionAndClosesPfd() + throws Exception { + File fifoFile = new File(temporaryFolder.getRoot(), "test_fifo"); + Process mkfifo = new ProcessBuilder("mkfifo", fifoFile.getAbsolutePath()).start(); + if (mkfifo.waitFor() != 0) { + return; + } + + ParcelFileDescriptor pfd = + ParcelFileDescriptor.open(fifoFile, ParcelFileDescriptor.MODE_READ_WRITE); + ParcelFileDescriptor spyPfd = Mockito.spy(pfd); + InAppMp4Muxer.Factory factory = new InAppMp4Muxer.Factory(); + + try { + MuxerException exception = assertThrows(MuxerException.class, () -> factory.create(spyPfd)); + assertThat(exception).hasMessageThat().contains("not seekable"); + verify(spyPfd, Mockito.atLeastOnce()).close(); + assertThat(pfd.getFileDescriptor().valid()).isFalse(); + } finally { + fifoFile.delete(); + } + } + + @Test + public void create_withClosedParcelFileDescriptor_throwsMuxerException() throws Exception { + File tempFile = temporaryFolder.newFile("test_output.mp4"); + InAppMp4Muxer.Factory factory = new InAppMp4Muxer.Factory(); + ParcelFileDescriptor pfd = + ParcelFileDescriptor.open(tempFile, ParcelFileDescriptor.MODE_READ_WRITE); + pfd.close(); + + assertThrows(MuxerException.class, () -> factory.create(pfd)); + } + + @Test + public void create_withUnexpectedRuntimeException_throwsMuxerExceptionAndClosesPfd() + throws Exception { + File tempFile = temporaryFolder.newFile("test_output.mp4"); + InAppMp4Muxer.Factory factory = new InAppMp4Muxer.Factory(); + ParcelFileDescriptor pfd = + ParcelFileDescriptor.open(tempFile, ParcelFileDescriptor.MODE_READ_WRITE); + ParcelFileDescriptor spyPfd = Mockito.spy(pfd); + Mockito.doThrow(new RuntimeException("Simulated unexpected failure")) + .when(spyPfd) + .getFileDescriptor(); + + MuxerException exception = assertThrows(MuxerException.class, () -> factory.create(spyPfd)); + assertThat(exception).hasCauseThat().isInstanceOf(RuntimeException.class); + verify(spyPfd).close(); + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/InFlightFrameManagerTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/InFlightFrameManagerTest.java new file mode 100644 index 00000000000..7dedf1dd084 --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/InFlightFrameManagerTest.java @@ -0,0 +1,256 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.verify; +import static org.mockito.Mockito.verifyNoInteractions; +import static org.mockito.Mockito.verifyNoMoreInteractions; + +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.common.video.Frame; +import androidx.media3.common.video.ReferenceCounter; +import androidx.media3.common.video.SyncFenceWrapper; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link InFlightFrameManager}. */ +@RunWith(AndroidJUnit4.class) +public final class InFlightFrameManagerTest { + + private interface ReferenceCountedFrame extends Frame, ReferenceCounter {} + + @Test + public void trackIfSuccessful_whenQueueSucceeds_tracksFrames() { + InFlightFrameManager manager = new InFlightFrameManager(); + ReferenceCountedFrame frame = mock(ReferenceCountedFrame.class); + + boolean queued = manager.trackIfSuccessful(p -> true, createPacket(frame)); + + assertThat(queued).isTrue(); + manager.releaseAll(); + verify(frame).release(null); + } + + @Test + public void trackIfSuccessful_whenQueueFails_removesFramesWithoutReleasing() { + InFlightFrameManager manager = new InFlightFrameManager(); + ReferenceCountedFrame frame = mock(ReferenceCountedFrame.class); + + boolean queued = manager.trackIfSuccessful(p -> false, createPacket(frame)); + + assertThat(queued).isFalse(); + manager.releaseAll(); + verifyNoInteractions(frame); + } + + @Test + public void trackIfSuccessful_whenQueueThrows_removesFramesWithoutReleasing() { + InFlightFrameManager manager = new InFlightFrameManager(); + ReferenceCountedFrame frame = mock(ReferenceCountedFrame.class); + ImmutableList packet = createPacket(frame); + + assertThrows( + RuntimeException.class, + () -> + manager.trackIfSuccessful( + p -> { + throw new RuntimeException("Queue error"); + }, + packet)); + + manager.releaseAll(); + verifyNoInteractions(frame); + } + + @Test + public void trackIfSuccessful_whenSubsequentQueueFails_preservesExistingTrackedFrames() { + InFlightFrameManager manager = new InFlightFrameManager(); + ReferenceCountedFrame frame1 = mock(ReferenceCountedFrame.class); + ReferenceCountedFrame frame2 = mock(ReferenceCountedFrame.class); + + manager.trackIfSuccessful(p -> true, createPacket(frame1)); + manager.trackIfSuccessful(p -> false, createPacket(frame2)); + + manager.releaseAll(); + verify(frame1).release(null); + verifyNoInteractions(frame2); + } + + @Test + public void trackIfSuccessful_withEmptyPacket_succeeds() { + InFlightFrameManager manager = new InFlightFrameManager(); + + boolean queued = manager.trackIfSuccessful(p -> true, ImmutableList.of()); + + assertThat(queued).isTrue(); + manager.releaseAll(); + } + + @Test + public void onFrameProcessed_withRegisteredReferenceCounter_releasesFrame() { + InFlightFrameManager manager = new InFlightFrameManager(); + ReferenceCountedFrame frame = mock(ReferenceCountedFrame.class); + manager.trackIfSuccessful(p -> true, createPacket(frame)); + + manager.onFrameProcessed(frame, /* releaseFence= */ null); + + verify(frame).release(null); + } + + @Test + public void onFrameProcessed_withReleaseFence_releasesFrame() { + SyncFenceWrapper releaseFence = mock(SyncFenceWrapper.class); + InFlightFrameManager manager = new InFlightFrameManager(); + ReferenceCountedFrame frame = mock(ReferenceCountedFrame.class); + manager.trackIfSuccessful(p -> true, createPacket(frame)); + + manager.onFrameProcessed(frame, releaseFence); + + verify(frame).release(releaseFence); + } + + @Test + public void onFrameProcessed_withUnmanagedFrame_removesWithoutError() { + InFlightFrameManager manager = new InFlightFrameManager(); + Frame unmanagedFrame = mock(Frame.class); + manager.trackIfSuccessful(p -> true, createPacket(unmanagedFrame)); + + manager.onFrameProcessed(unmanagedFrame, /* releaseFence= */ null); + + verifyNoInteractions(unmanagedFrame); + } + + @Test + public void onFrameProcessed_withUnmanagedTrackedFrame_closesReleaseFence() { + SyncFenceWrapper releaseFence = mock(SyncFenceWrapper.class); + InFlightFrameManager manager = new InFlightFrameManager(); + Frame unmanagedFrame = mock(Frame.class); + manager.trackIfSuccessful(p -> true, createPacket(unmanagedFrame)); + + manager.onFrameProcessed(unmanagedFrame, releaseFence); + + verify(releaseFence).close(); + } + + @Test + public void onFrameProcessed_withUnregisteredFrame_doesNotThrow() { + InFlightFrameManager manager = new InFlightFrameManager(); + Frame frame = mock(Frame.class); + + manager.onFrameProcessed(frame, /* releaseFence= */ null); + + verifyNoInteractions(frame); + } + + @Test + public void onFrameProcessed_withUnregisteredFrame_closesReleaseFence() { + SyncFenceWrapper releaseFence = mock(SyncFenceWrapper.class); + InFlightFrameManager manager = new InFlightFrameManager(); + Frame frame = mock(Frame.class); + + manager.onFrameProcessed(frame, releaseFence); + + verify(releaseFence).close(); + } + + @Test + public void onFrameProcessed_calledTwiceForSameFrame_releasesOnlyOnceAndClosesSecondFence() { + InFlightFrameManager manager = new InFlightFrameManager(); + ReferenceCountedFrame frame = mock(ReferenceCountedFrame.class); + SyncFenceWrapper fence1 = mock(SyncFenceWrapper.class); + SyncFenceWrapper fence2 = mock(SyncFenceWrapper.class); + manager.trackIfSuccessful(p -> true, createPacket(frame)); + + manager.onFrameProcessed(frame, fence1); + manager.onFrameProcessed(frame, fence2); + + verify(frame).release(fence1); + verify(fence2).close(); + verifyNoMoreInteractions(frame); + } + + @Test + public void onFrameProcessed_withOneOfMultipleFrames_remainsInFlight() { + InFlightFrameManager manager = new InFlightFrameManager(); + ReferenceCountedFrame frame1 = mock(ReferenceCountedFrame.class); + ReferenceCountedFrame frame2 = mock(ReferenceCountedFrame.class); + manager.trackIfSuccessful(p -> true, createPacket(frame1, frame2)); + + manager.onFrameProcessed(frame1, /* releaseFence= */ null); + verify(frame1).release(null); + + manager.releaseAll(); + verify(frame2).release(null); + verifyNoMoreInteractions(frame1); + } + + @Test + public void onFrameProcessed_withAllMultipleFrames_clearsInFlightFrames() { + InFlightFrameManager manager = new InFlightFrameManager(); + ReferenceCountedFrame frame1 = mock(ReferenceCountedFrame.class); + ReferenceCountedFrame frame2 = mock(ReferenceCountedFrame.class); + manager.trackIfSuccessful(p -> true, createPacket(frame1, frame2)); + + manager.onFrameProcessed(frame1, /* releaseFence= */ null); + manager.onFrameProcessed(frame2, /* releaseFence= */ null); + + verify(frame1).release(null); + verify(frame2).release(null); + + manager.releaseAll(); + verifyNoMoreInteractions(frame1); + verifyNoMoreInteractions(frame2); + } + + @Test + public void releaseAll_withInFlightFrames_releasesAllFrames() { + InFlightFrameManager manager = new InFlightFrameManager(); + ReferenceCountedFrame frame1 = mock(ReferenceCountedFrame.class); + ReferenceCountedFrame frame2 = mock(ReferenceCountedFrame.class); + manager.trackIfSuccessful(p -> true, createPacket(frame1, frame2)); + + manager.releaseAll(); + + verify(frame1).release(null); + verify(frame2).release(null); + } + + @Test + public void releaseAll_calledMultipleTimes_releasesFramesOnlyOnce() { + InFlightFrameManager manager = new InFlightFrameManager(); + ReferenceCountedFrame frame = mock(ReferenceCountedFrame.class); + manager.trackIfSuccessful(p -> true, createPacket(frame)); + + manager.releaseAll(); + manager.releaseAll(); + + verify(frame).release(null); + verifyNoMoreInteractions(frame); + } + + private static ImmutableList createPacket(Frame... frames) { + ImmutableList.Builder builder = ImmutableList.builder(); + for (Frame frame : frames) { + builder.add(new AsyncFrame(frame, /* acquireFence= */ null)); + } + return builder.build(); + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/MediaItemExportTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/MediaItemExportTest.java index a06e9489af7..44bc213138e 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/MediaItemExportTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/MediaItemExportTest.java @@ -16,6 +16,7 @@ package androidx.media3.transformer; +import static androidx.media3.test.utils.AssetInfo.WAV_ASSET; import static androidx.media3.test.utils.TestUtil.createByteCountingAudioProcessor; import static androidx.media3.test.utils.robolectric.RobolectricUtil.runLooperUntil; import static androidx.media3.test.utils.robolectric.ShadowMediaCodecConfig.CODEC_INFO_AAC; @@ -69,11 +70,14 @@ import android.view.Surface; import androidx.annotation.Nullable; import androidx.media3.common.C; +import androidx.media3.common.C.TrackType; import androidx.media3.common.Effect; +import androidx.media3.common.Flags; import androidx.media3.common.Format; import androidx.media3.common.MediaItem; import androidx.media3.common.MimeTypes; import androidx.media3.common.audio.SonicAudioProcessor; +import androidx.media3.common.audio.SpeedProvider; import androidx.media3.common.audio.ToInt16PcmAudioProcessor; import androidx.media3.common.util.Clock; import androidx.media3.effect.Contrast; @@ -87,8 +91,16 @@ import androidx.media3.extractor.ExtractorOutput; import androidx.media3.extractor.ExtractorsFactory; import androidx.media3.extractor.PositionHolder; +import androidx.media3.inspector.MetadataRetriever; +import androidx.media3.muxer.Muxer; +import androidx.media3.muxer.MuxerException; +import androidx.media3.muxer.SeekableMuxerOutput; +import androidx.media3.muxer.WavMuxer; import androidx.media3.test.utils.DumpFileAsserts; import androidx.media3.test.utils.FakeClock; +import androidx.media3.test.utils.Media3FlagsRule; +import androidx.media3.test.utils.PassthroughAudioProcessor; +import androidx.media3.test.utils.TestSpeedProvider; import androidx.media3.test.utils.TestTransformerBuilder; import androidx.media3.test.utils.robolectric.ShadowMediaCodecConfig; import androidx.test.core.app.ApplicationProvider; @@ -145,6 +157,7 @@ public void onConfigured( } }); + @Rule public final Media3FlagsRule flagsRule = new Media3FlagsRule(this); @Rule public final TemporaryFolder outputDir = new TemporaryFolder(); private final Context context = ApplicationProvider.getApplicationContext(); @@ -251,6 +264,9 @@ public void start_audioAndVideoPassthrough_withClippingStartAtKeyFrame_completes @Test public void start_withClippingStartAndEndEqual_completesSuccessfully( @TestParameter boolean enableClippingInMediaPeriod) throws Exception { + if (enableClippingInMediaPeriod) { + Flags.enableFlag(Flags.FLAG_ENABLE_CLIPPING_IN_MEDIA_PERIOD); + } CapturingMuxer.Factory muxerFactory = new CapturingMuxer.Factory(/* handleAudioAsPcm= */ false); Clock clock = new FakeClock(/* isAutoAdvancing= */ true); Codec.DecoderFactory decoderFactory = new DefaultDecoderFactory.Builder(context).build(); @@ -259,11 +275,7 @@ public void start_withClippingStartAndEndEqual_completesSuccessfully( .setMuxerFactory(muxerFactory) .setAssetLoaderFactory( new ExoPlayerAssetLoader.Factory( - context, - decoderFactory, - clock, - new DefaultMediaSourceFactory(context) - .setEnableClippingInMediaPeriod(enableClippingInMediaPeriod))) + context, decoderFactory, clock, new DefaultMediaSourceFactory(context))) .setClock(clock) .build(); MediaItem mediaItem = @@ -307,6 +319,59 @@ public void start_trimOptimizationEnabled_clippingConfigurationUnset_doesNotOpti assertThat(result.optimizationResult).isEqualTo(OPTIMIZATION_NONE); } + @Test + public void export_wavFileWithAudioProcessor_outputsRawPcmSamples() throws Exception { + // TODO(b/562454033): Use default muxer factory once pipeline signals exact audio duration. + Transformer transformer = + new TestTransformerBuilder(context).setMuxerFactory(new WavMuxerFactory()).build(); + EditedMediaItem item = + new EditedMediaItem.Builder(MediaItem.fromUri(ASSET_URI_PREFIX + FILE_AUDIO_RAW)) + .setEffects(createAudioEffects(new PassthroughAudioProcessor())) + .build(); + String outputFile = outputDir.newFile().getPath(); + + transformer.start(item, outputFile); + ExportResult result = TransformerTestRunner.runLooper(transformer); + + assertThat(result.audioMimeType).isEqualTo(MimeTypes.AUDIO_RAW); + assertThat(result.approximateDurationMs).isEqualTo(1_000); + + try (MetadataRetriever r = + new MetadataRetriever.Builder(context, MediaItem.fromUri(outputFile)).build()) { + assertThat(r.retrieveTrackGroups().get().get(0).getFormat(0).sampleMimeType) + .isEqualTo(MimeTypes.AUDIO_RAW); + assertThat(r.retrieveDurationUs().get()).isEqualTo(1_000_000); + } + } + + @Test + public void setSpeed_onRawAudioTrackWithNoOtherEffects_appliesEffect() throws Exception { + // TODO(b/562454033): Use default muxer factory once pipeline signals exact audio duration. + Transformer transformer = + new TestTransformerBuilder(context).setMuxerFactory(new WavMuxerFactory()).build(); + SpeedProvider speedProvider = + TestSpeedProvider.createWithStartTimes( + new long[] {0L, 300_000, 600_000}, new float[] {4f, 0.5f, 2f}); + // Use a WAV file without any effects so that the EditedMediaItem is a candidate for + // transmuxing. + EditedMediaItem input = + new EditedMediaItem.Builder(MediaItem.fromUri(WAV_ASSET.uri)) + .setSpeed(speedProvider) + .build(); + + String outputPath = outputDir.newFile().getPath(); + transformer.start(input, outputPath); + TransformerTestRunner.runLooper(transformer); + + try (MetadataRetriever retriever = + new MetadataRetriever.Builder(context, MediaItem.fromUri(outputPath)).build()) { + assertThat(retriever.retrieveTrackGroups().get().get(0).getFormat(0).sampleMimeType) + .isEqualTo(MimeTypes.AUDIO_RAW); + // Allow one sample of tolerance per speed region (3 * 1_000_000 / 44_100 Hz = ~68us) + assertThat(retriever.retrieveDurationUs().get()).isWithin(68).of(875_000); + } + } + @Test public void start_trimOptimizationEnabled_withClippingStartAtKeyFrame_completesSuccessfully() throws Exception { @@ -918,20 +983,18 @@ public void start_withSlowOutputSampleRate_completesWithError() throws Exception CapturingMuxer.Factory muxerFactory = new CapturingMuxer.Factory(/* handleAudioAsPcm= */ false); // TODO: b/484325763 - Investigate refactoring `WatchdogTimer` to use `Clock` interface for // deterministic testing. + FakeClock clock = new FakeClock(/* isAutoAdvancing= */ true); MediaSource.Factory mediaSourceFactory = new DefaultMediaSourceFactory( - context, new SlowExtractorsFactory(/* delayBetweenReadsMs= */ 100)); + context, new SlowExtractorsFactory(/* delayBetweenReadsMs= */ 1000)); Codec.DecoderFactory decoderFactory = new DefaultDecoderFactory.Builder(context).build(); AssetLoader.Factory assetLoaderFactory = - new ExoPlayerAssetLoader.Factory( - context, - decoderFactory, - new FakeClock(/* isAutoAdvancing= */ true), - mediaSourceFactory); + new ExoPlayerAssetLoader.Factory(context, decoderFactory, clock, mediaSourceFactory); Transformer transformer = new TestTransformerBuilder(context) + .setClock(clock) .setMuxerFactory(muxerFactory) - .setMaxDelayBetweenMuxerSamplesMs(10) + .setMaxDelayBetweenMuxerSamplesMs(500) .setAssetLoaderFactory(assetLoaderFactory) .build(); MediaItem mediaItem = MediaItem.fromUri(ASSET_URI_PREFIX + FILE_AUDIO_VIDEO); @@ -1183,7 +1246,7 @@ public void export_withRawAacFile_transmuxesIntoMp4() throws Exception { context, muxerFactory.getCreatedMuxer(), getDumpFileName( - /* originalFileName= */ FILE_AUDIO_RAW_AAC, /* modifications= */ "transmuxed")); + /* originalFileName= */ FILE_AUDIO_RAW_AAC, /* modifications...= */ "transmuxed")); } @Test @@ -1439,6 +1502,52 @@ public void getProgress_fromWrongThread_throwsError() throws Exception { assertThat(stateAfterTransform).isEqualTo(PROGRESS_STATE_NOT_STARTED); } + @Test + public void getProgress_trimOptimizationEnabledButAbandoned_givesIncreasingPercentages() + throws Exception { + Transformer transformer = + new TestTransformerBuilder(context).experimentalSetTrimOptimizationEnabled(true).build(); + MediaItem mediaItem = + new MediaItem.Builder() + .setUri(ASSET_URI_PREFIX + FILE_AUDIO_VIDEO_INCREASING_TIMESTAMPS_15S) + .setClippingConfiguration( + new MediaItem.ClippingConfiguration.Builder().setEndPositionMs(14_000).build()) + .build(); + + transformer.start(mediaItem, outputDir.newFile().getPath()); + ImmutableList progressValues = + runTransformerForProgressStateAndValueUpdates(transformer).second; + + assertThat(progressValues.size()).isAtLeast(2); + assertThat(progressValues.get(0)).isAtLeast(0); + assertThat(progressValues).isInStrictOrder(); + assertThat(Iterables.getLast(progressValues)).isAtMost(100); + } + + @Test + public void getProgress_trimOptimizationEnabledButAbandoned_returnsConsistentStates() + throws Exception { + Transformer transformer = + new TestTransformerBuilder(context).experimentalSetTrimOptimizationEnabled(true).build(); + MediaItem mediaItem = + new MediaItem.Builder() + .setUri(ASSET_URI_PREFIX + FILE_AUDIO_VIDEO_INCREASING_TIMESTAMPS_15S) + .setClippingConfiguration( + new MediaItem.ClippingConfiguration.Builder().setEndPositionMs(14_000).build()) + .build(); + + transformer.start(mediaItem, outputDir.newFile().getPath()); + ImmutableList<@Transformer.ProgressState Integer> progressStates = + runTransformerForProgressStateAndValueUpdates(transformer).first; + + assertThat(progressStates) + .containsExactly( + PROGRESS_STATE_WAITING_FOR_AVAILABILITY, + PROGRESS_STATE_AVAILABLE, + PROGRESS_STATE_NOT_STARTED) + .inOrder(); + } + @Test public void cancel_afterCompletion_doesNotThrow() throws Exception { CapturingMuxer.Factory muxerFactory = new CapturingMuxer.Factory(/* handleAudioAsPcm= */ false); @@ -1549,6 +1658,22 @@ public void transmux_trimsFirstIDRDuration() throws Exception { /* modifications...= */ "transmuxed")); } + private static final class WavMuxerFactory implements Muxer.Factory { + @Override + public Muxer create(String path) throws MuxerException { + try { + return new WavMuxer(SeekableMuxerOutput.of(path)); + } catch (Exception ex) { + throw new MuxerException("Failed to create WavMuxer.", ex); + } + } + + @Override + public ImmutableList getSupportedSampleMimeTypes(@TrackType int trackType) { + return ImmutableList.of(MimeTypes.AUDIO_RAW); + } + } + private Pair, ImmutableList> runTransformerForProgressStateAndValueUpdates(Transformer transformer) throws ExportException, TimeoutException { diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/Mp4InfoTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/Mp4InfoTest.java index 97dd701eadc..2fb372548ea 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/Mp4InfoTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/Mp4InfoTest.java @@ -83,7 +83,7 @@ public void durationUs_ofMp4File_outputMatchesExpected() throws Exception { String mp4FilePath = "asset:///media/mp4/hdr10-720p.mp4"; Mp4Info mp4Info = Mp4Info.create(context, mp4FilePath); - assertThat(mp4Info.durationUs).isEqualTo(4_236_600L); // The duration of hdr10-720p.mp4. + assertThat(mp4Info.durationUs).isEqualTo(4_236_588L); // The duration of hdr10-720p.mp4. } @Test @@ -109,7 +109,7 @@ public void firstSyncSampleTimestampUsAfterTimeUs_timeUsNotASyncSample_returnsCo public void firstSyncSampleTimestampUsAfterTimeUs_timeUsSetToDuration_returnsTimeEndOfSource() throws IOException { String mp4FilePath = "asset:///media/mp4/hdr10-720p.mp4"; - Mp4Info mp4Info = Mp4Info.create(context, mp4FilePath, /* timeUs= */ 4_236_600L); + Mp4Info mp4Info = Mp4Info.create(context, mp4FilePath, /* timeUs= */ 4_236_588L); assertThat(mp4Info.firstSyncSampleTimestampUsAfterTimeUs).isEqualTo(C.TIME_END_OF_SOURCE); } diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/MuxerWrapperTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/MuxerWrapperTest.java index 2c59457fd6f..8b5edd8e9da 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/MuxerWrapperTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/MuxerWrapperTest.java @@ -30,6 +30,9 @@ import androidx.media3.common.C; import androidx.media3.common.ColorInfo; import androidx.media3.common.Format; +import androidx.media3.common.Metadata; +import androidx.media3.muxer.BufferInfo; +import androidx.media3.muxer.Muxer; import androidx.media3.muxer.MuxerException; import androidx.media3.test.utils.DumpFileAsserts; import androidx.test.core.app.ApplicationProvider; @@ -712,6 +715,80 @@ public void release_withReleaseReasonErrorInMuxPartialMode_releasesResources() t muxerWrapper = null; } + @Test + public void finishWritingAndMaybeRelease_cancelledOnNonFrameworkMuxer_throwsException() + throws Exception { + Muxer.Factory throwingMuxerFactory = + new Muxer.Factory() { + @Override + public Muxer create(String path) { + return new Muxer() { + @Override + public int addTrack(Format format) { + return 0; + } + + @Override + public void writeSampleData( + int trackId, ByteBuffer byteBuffer, BufferInfo bufferInfo) {} + + @Override + public void addMetadataEntry(Metadata.Entry metadataEntry) {} + + @Override + public void close() throws MuxerException { + throw new MuxerException( + FrameworkMuxer.MUXER_STOPPING_FAILED_ERROR_MESSAGE, new Exception()); + } + }; + } + + @Override + public ImmutableList getSupportedSampleMimeTypes(@C.TrackType int trackType) { + return ImmutableList.of(VIDEO_H264); + } + }; + muxerWrapper = + new MuxerWrapper( + temporaryFolder.newFile().getPath(), + throwingMuxerFactory, + new NoOpMuxerListenerImpl(), + MUXER_MODE_DEFAULT, + /* dropSamplesBeforeFirstVideoSample= */ false, + /* appendVideoFormat= */ null); + muxerWrapper.setTrackCount(1); + muxerWrapper.addTrackFormat(FAKE_VIDEO_TRACK_FORMAT); + muxerWrapper.writeSample( + C.TRACK_TYPE_VIDEO, + FAKE_SAMPLE.duplicate(), + /* isKeyFrame= */ true, + /* presentationTimeUs= */ 0); + + assertThrows( + MuxerException.class, + () -> + muxerWrapper.finishWritingAndMaybeRelease(MuxerWrapper.MUXER_RELEASE_REASON_CANCELLED)); + muxerWrapper = null; + } + + @Test + public void finishWritingAndMaybeRelease_cancelledOnFrameworkMuxer_doesNotThrow() + throws Exception { + muxerWrapper = + new MuxerWrapper( + temporaryFolder.newFile().getPath(), + new FrameworkMuxer.Factory(), + new NoOpMuxerListenerImpl(), + MUXER_MODE_DEFAULT, + /* dropSamplesBeforeFirstVideoSample= */ false, + /* appendVideoFormat= */ null); + muxerWrapper.setTrackCount(1); + muxerWrapper.addTrackFormat(FAKE_VIDEO_TRACK_FORMAT); + + muxerWrapper.finishWritingAndMaybeRelease(MuxerWrapper.MUXER_RELEASE_REASON_CANCELLED); + muxerWrapper = null; + } + private static final class NoOpMuxerListenerImpl implements MuxerWrapper.Listener { @Override diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/ParameterizedCodecDbLiteMimeTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/ParameterizedCodecDbLiteMimeTest.java index 79aa957d87e..78e0675be07 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/ParameterizedCodecDbLiteMimeTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/ParameterizedCodecDbLiteMimeTest.java @@ -38,12 +38,12 @@ public class ParameterizedCodecDbLiteMimeTest { @Parameters(name = "{index}: {0}") public static ImmutableList parameters() { return ImmutableList.of( - new TestParameters("QTI", "SM6375", MimeTypes.VIDEO_H265), + new TestParameters("QTI", "SM6375", MimeTypes.VIDEO_H264), new TestParameters("QTI", "SM8350", MimeTypes.VIDEO_H265), new TestParameters("Mediatek", "MT6769T", MimeTypes.VIDEO_H265), new TestParameters("Google", "Tensor G3", MimeTypes.VIDEO_H265), new TestParameters("Mediatek", "MT6762", MimeTypes.VIDEO_H264), - new TestParameters("QTI", "SM6225", MimeTypes.VIDEO_H265), + new TestParameters("QTI", "SM6225", MimeTypes.VIDEO_H264), new TestParameters("QTI", "SM8475", MimeTypes.VIDEO_H265), new TestParameters("Mediatek", "MT6893", MimeTypes.VIDEO_H265), new TestParameters("Spreadtrum", "SC9863A", MimeTypes.VIDEO_H264), @@ -52,17 +52,17 @@ public static ImmutableList parameters() { new TestParameters("Mediatek", "MT6789V/CD", MimeTypes.VIDEO_H265), new TestParameters("QTI", "SM8250", MimeTypes.VIDEO_H265), new TestParameters("Google", "Tensor G2", MimeTypes.VIDEO_H265), - new TestParameters("Mediatek", "MT6983", MimeTypes.VIDEO_H265), + new TestParameters("Mediatek", "MT6983", MimeTypes.VIDEO_H264), new TestParameters("Mediatek", "MT6769Z", MimeTypes.VIDEO_H265), - new TestParameters("Samsung", "Exynos 850", MimeTypes.VIDEO_H265), + new TestParameters("Samsung", "Exynos 850", MimeTypes.VIDEO_H264), new TestParameters("QTI", "SM8650", MimeTypes.VIDEO_H265), new TestParameters("QTI", "SDM450", MimeTypes.VIDEO_H264), new TestParameters("Spreadtrum", "T606", MimeTypes.VIDEO_H264), new TestParameters("Samsung", "s5e9925", MimeTypes.VIDEO_H265), new TestParameters("QTI", "SM8550", MimeTypes.VIDEO_H265), - new TestParameters("QTI", "SM4350", MimeTypes.VIDEO_H265), + new TestParameters("QTI", "SM4350", MimeTypes.VIDEO_H264), new TestParameters("Samsung", "s5e8825", MimeTypes.VIDEO_H265), - new TestParameters("QTI", "SM6125", MimeTypes.VIDEO_H265), + new TestParameters("QTI", "SM6125", MimeTypes.VIDEO_H264), new TestParameters("Mediatek", "MT6833V/NZA", MimeTypes.VIDEO_H265), new TestParameters("Mediatek", "MT6761", MimeTypes.VIDEO_H264), new TestParameters("Mediatek", "MT6785", MimeTypes.VIDEO_H265), diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/ParameterizedCodecDbLiteVideoEncoderSettingsTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/ParameterizedCodecDbLiteVideoEncoderSettingsTest.java index 64d79470e9d..8d63692d18f 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/ParameterizedCodecDbLiteVideoEncoderSettingsTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/ParameterizedCodecDbLiteVideoEncoderSettingsTest.java @@ -80,7 +80,7 @@ public static ImmutableList parameters() { .setHeight(540) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(1).setBitrate(7939243).build()), + new VideoEncoderSettings.Builder().setMaxBFrames(1).build()), new TestParameters( "Google", "Tensor G3", @@ -90,7 +90,7 @@ public static ImmutableList parameters() { .setHeight(480) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(1).setBitrate(7045659).build()), + new VideoEncoderSettings.Builder().setMaxBFrames(1).build()), new TestParameters( "Google", "Tensor G3", @@ -100,7 +100,7 @@ public static ImmutableList parameters() { .setHeight(360) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(1).setBitrate(5199819).build()), + new VideoEncoderSettings.Builder().setMaxBFrames(1).build()), new TestParameters( "Google", "Tensor G3", @@ -130,7 +130,7 @@ public static ImmutableList parameters() { .setHeight(720) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(1).setBitrate(8904992).build()), + new VideoEncoderSettings.Builder().setMaxBFrames(1).build()), new TestParameters( "Google", "Tensor G3", @@ -140,7 +140,7 @@ public static ImmutableList parameters() { .setHeight(540) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(1).setBitrate(6971975).build()), + new VideoEncoderSettings.Builder().setMaxBFrames(1).build()), new TestParameters( "Google", "Tensor G3", @@ -150,7 +150,7 @@ public static ImmutableList parameters() { .setHeight(480) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(1).setBitrate(6259729).build()), + new VideoEncoderSettings.Builder().setMaxBFrames(1).build()), new TestParameters( "Google", "Tensor G3", @@ -160,7 +160,7 @@ public static ImmutableList parameters() { .setHeight(360) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(1).setBitrate(4684288).build()), + new VideoEncoderSettings.Builder().setMaxBFrames(1).build()), new TestParameters( "QTI", "SM8650", @@ -214,7 +214,6 @@ public static ImmutableList parameters() { .setMaxBFrames(1) .setTemporalLayers( /* numNonBidirectionalLayers= */ 1, /* numBidirectionalLayers= */ 2) - .setBitrate(7939243) .build()), new TestParameters( "QTI", @@ -229,7 +228,6 @@ public static ImmutableList parameters() { .setMaxBFrames(1) .setTemporalLayers( /* numNonBidirectionalLayers= */ 1, /* numBidirectionalLayers= */ 2) - .setBitrate(7045659) .build()), new TestParameters( "QTI", @@ -244,7 +242,6 @@ public static ImmutableList parameters() { .setMaxBFrames(1) .setTemporalLayers( /* numNonBidirectionalLayers= */ 1, /* numBidirectionalLayers= */ 2) - .setBitrate(5199819) .build()), new TestParameters( "QTI", @@ -294,7 +291,6 @@ public static ImmutableList parameters() { .setMaxBFrames(1) .setTemporalLayers( /* numNonBidirectionalLayers= */ 1, /* numBidirectionalLayers= */ 2) - .setBitrate(6971975) .build()), new TestParameters( "QTI", @@ -309,7 +305,6 @@ public static ImmutableList parameters() { .setMaxBFrames(1) .setTemporalLayers( /* numNonBidirectionalLayers= */ 1, /* numBidirectionalLayers= */ 2) - .setBitrate(6259729) .build()), new TestParameters( "QTI", @@ -324,7 +319,6 @@ public static ImmutableList parameters() { .setMaxBFrames(1) .setTemporalLayers( /* numNonBidirectionalLayers= */ 1, /* numBidirectionalLayers= */ 2) - .setBitrate(4684288) .build()), new TestParameters( "Mediatek", @@ -365,7 +359,7 @@ public static ImmutableList parameters() { .setHeight(540) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(1).setBitrate(7939243).build()), + new VideoEncoderSettings.Builder().setMaxBFrames(1).build()), new TestParameters( "Mediatek", "MT6983", @@ -375,7 +369,7 @@ public static ImmutableList parameters() { .setHeight(480) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(1).setBitrate(7045659).build()), + new VideoEncoderSettings.Builder().setMaxBFrames(1).build()), new TestParameters( "Mediatek", "MT6983", @@ -385,7 +379,7 @@ public static ImmutableList parameters() { .setHeight(360) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(1).setBitrate(5199819).build()), + new VideoEncoderSettings.Builder().setMaxBFrames(1).build()), new TestParameters( "Mediatek", "MT6983", @@ -395,7 +389,7 @@ public static ImmutableList parameters() { .setHeight(2160) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setBitrate(18947248).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Mediatek", "MT6983", @@ -405,7 +399,7 @@ public static ImmutableList parameters() { .setHeight(1080) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setBitrate(12116617).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Mediatek", "MT6983", @@ -415,7 +409,7 @@ public static ImmutableList parameters() { .setHeight(720) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(1).setBitrate(8904992).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Mediatek", "MT6983", @@ -425,7 +419,7 @@ public static ImmutableList parameters() { .setHeight(540) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(1).setBitrate(6971975).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Mediatek", "MT6983", @@ -435,7 +429,7 @@ public static ImmutableList parameters() { .setHeight(480) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(1).setBitrate(6259729).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Mediatek", "MT6983", @@ -445,7 +439,7 @@ public static ImmutableList parameters() { .setHeight(360) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(1).setBitrate(4684288).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Samsung", "s5e9925", @@ -465,7 +459,7 @@ public static ImmutableList parameters() { .setHeight(1080) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setBitrate(15502805).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Samsung", "s5e9925", @@ -485,7 +479,7 @@ public static ImmutableList parameters() { .setHeight(540) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(2).setBitrate(7939243).build()), + new VideoEncoderSettings.Builder().setMaxBFrames(2).build()), new TestParameters( "Samsung", "s5e9925", @@ -495,7 +489,7 @@ public static ImmutableList parameters() { .setHeight(480) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(2).setBitrate(7045659).build()), + new VideoEncoderSettings.Builder().setMaxBFrames(2).build()), new TestParameters( "Samsung", "s5e9925", @@ -505,7 +499,7 @@ public static ImmutableList parameters() { .setHeight(360) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(2).setBitrate(5199819).build()), + new VideoEncoderSettings.Builder().setMaxBFrames(2).build()), new TestParameters( "Samsung", "s5e9925", @@ -525,7 +519,7 @@ public static ImmutableList parameters() { .setHeight(1080) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setBitrate(12116617).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Samsung", "s5e9925", @@ -545,7 +539,7 @@ public static ImmutableList parameters() { .setHeight(540) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(1).setBitrate(6971975).build()), + new VideoEncoderSettings.Builder().setMaxBFrames(1).build()), new TestParameters( "Samsung", "s5e9925", @@ -555,7 +549,7 @@ public static ImmutableList parameters() { .setHeight(480) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(1).setBitrate(6259729).build()), + new VideoEncoderSettings.Builder().setMaxBFrames(1).build()), new TestParameters( "Samsung", "s5e9925", @@ -565,7 +559,7 @@ public static ImmutableList parameters() { .setHeight(360) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setMaxBFrames(1).setBitrate(4684288).build()), + new VideoEncoderSettings.Builder().setMaxBFrames(1).build()), new TestParameters( "Unknown", "Chipset", @@ -575,7 +569,7 @@ public static ImmutableList parameters() { .setHeight(360) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setBitrate(5199819).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Unknown", "Chipset", @@ -585,7 +579,7 @@ public static ImmutableList parameters() { .setHeight(480) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setBitrate(7045659).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Unknown", "Chipset", @@ -595,7 +589,7 @@ public static ImmutableList parameters() { .setHeight(540) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setBitrate(7939243).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Unknown", "Chipset", @@ -605,7 +599,7 @@ public static ImmutableList parameters() { .setHeight(720) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setBitrate(10549473).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Unknown", "Chipset", @@ -615,7 +609,7 @@ public static ImmutableList parameters() { .setHeight(1080) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setBitrate(15502805).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Unknown", "Chipset", @@ -625,7 +619,7 @@ public static ImmutableList parameters() { .setHeight(2160) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setBitrate(29044193).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Unknown", "Chipset", @@ -635,7 +629,7 @@ public static ImmutableList parameters() { .setHeight(360) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setBitrate(4684288).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Unknown", "Chipset", @@ -645,7 +639,7 @@ public static ImmutableList parameters() { .setHeight(480) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setBitrate(6259729).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Unknown", "Chipset", @@ -655,7 +649,7 @@ public static ImmutableList parameters() { .setHeight(540) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setBitrate(6971975).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Unknown", "Chipset", @@ -665,7 +659,7 @@ public static ImmutableList parameters() { .setHeight(720) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setBitrate(8904992).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Unknown", "Chipset", @@ -675,7 +669,7 @@ public static ImmutableList parameters() { .setHeight(1080) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setBitrate(12116617).build()), + new VideoEncoderSettings.Builder().build()), new TestParameters( "Unknown", "Chipset", @@ -685,7 +679,7 @@ public static ImmutableList parameters() { .setHeight(2160) .setFrameRate(30.00f) .build(), - new VideoEncoderSettings.Builder().setBitrate(18947248).build())); + new VideoEncoderSettings.Builder().build())); } @Test diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/PassthroughAudioCodecTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/PassthroughAudioCodecTest.java new file mode 100644 index 00000000000..72cc90384b3 --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/PassthroughAudioCodecTest.java @@ -0,0 +1,264 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static androidx.media3.test.utils.TestUtil.buildTestData; +import static androidx.media3.test.utils.TestUtil.createByteBuffer; +import static androidx.media3.test.utils.TestUtil.createShortArray; +import static com.google.common.base.Preconditions.checkNotNull; +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; + +import android.media.MediaCodec; +import android.media.MediaCodec.BufferInfo; +import androidx.media3.common.C; +import androidx.media3.common.Format; +import androidx.media3.common.MimeTypes; +import androidx.media3.decoder.DecoderInputBuffer; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.nio.ByteBuffer; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link PassthroughAudioCodec}. */ +@RunWith(AndroidJUnit4.class) +public final class PassthroughAudioCodecTest { + + private static final Format FORMAT = + new Format.Builder() + .setSampleMimeType(MimeTypes.AUDIO_RAW) + .setPcmEncoding(C.ENCODING_PCM_16BIT) + .setChannelCount(2) + .build(); + + private PassthroughAudioCodec codec; + + @Before + public void setup() { + codec = new PassthroughAudioCodec(FORMAT); + } + + @After + public void tearDown() { + if (codec != null) { + codec.release(); + } + } + + @Test + public void getConfigurationFormat_returnsFormat() { + assertThat(codec.getConfigurationFormat()).isEqualTo(FORMAT); + } + + @Test + public void getInputFormat_returnsFormat() { + assertThat(codec.getInputFormat()).isEqualTo(FORMAT); + } + + @Test + public void getOutputFormat_returnsFormat() { + assertThat(codec.getOutputFormat()).isEqualTo(FORMAT); + } + + @Test + public void maybeDequeueInputBuffer_withNoPendingInputBuffer_returnsTrueAndAssignsBuffer() { + DecoderInputBuffer inputBuffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DIRECT); + + assertThat(codec.maybeDequeueInputBuffer(inputBuffer)).isTrue(); + assertThat(inputBuffer.data).isNotNull(); + assertThat(inputBuffer.data.position()).isEqualTo(0); + } + + @Test + public void maybeDequeueInputBuffer_calledTwiceWithoutQueueing_returnsFalse() { + DecoderInputBuffer inputBuffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DIRECT); + + assertThat(codec.maybeDequeueInputBuffer(inputBuffer)).isTrue(); + assertThat(codec.maybeDequeueInputBuffer(inputBuffer)).isFalse(); + } + + @Test + public void queueInputBuffer_updatesOutputAndClearsInputBuffer() { + DecoderInputBuffer inputBuffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DIRECT); + assertThat(codec.maybeDequeueInputBuffer(inputBuffer)).isTrue(); + + assertThat(inputBuffer.data).isNotNull(); + inputBuffer.data.put(createByteBuffer(new short[] {10, 20, 30, 40})).flip(); + inputBuffer.timeUs = 123456L; + codec.queueInputBuffer(inputBuffer); + + assertThat(inputBuffer.data).isNull(); + assertThat(createShortArray(checkNotNull(codec.getOutputBuffer()))) + .isEqualTo(new short[] {10, 20, 30, 40}); + + BufferInfo outputBufferInfo = codec.getOutputBufferInfo(); + assertThat(outputBufferInfo).isNotNull(); + assertThat(outputBufferInfo.offset).isEqualTo(0); + assertThat(outputBufferInfo.size).isEqualTo(8); + assertThat(outputBufferInfo.presentationTimeUs).isEqualTo(123456L); + assertThat(outputBufferInfo.flags).isEqualTo(0); + } + + @Test + public void releaseOutputBuffer_clearsOutputBufferAndUnlocksDequeuing() { + DecoderInputBuffer inputBuffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DIRECT); + assertThat(codec.maybeDequeueInputBuffer(inputBuffer)).isTrue(); + inputBuffer.data.put(buildTestData(/* length= */ 20)).flip(); + codec.queueInputBuffer(inputBuffer); + + assertThat(codec.getOutputBuffer()).isNotNull(); + assertThat(codec.getOutputBufferInfo()).isNotNull(); + + codec.releaseOutputBuffer(/* render= */ false); + + assertThat(codec.getOutputBuffer()).isNull(); + assertThat(codec.getOutputBufferInfo()).isNull(); + assertThat(codec.maybeDequeueInputBuffer(inputBuffer)).isTrue(); + } + + @Test + public void queueEndOfStream_endsCodecAfterConsumingOutput() { + DecoderInputBuffer inputBuffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DIRECT); + assertThat(codec.maybeDequeueInputBuffer(inputBuffer)).isTrue(); + + inputBuffer.addFlag(C.BUFFER_FLAG_END_OF_STREAM); + // Flip buffer so that there is no readable data. + inputBuffer.data.flip(); + codec.queueInputBuffer(inputBuffer); + + assertThat(codec.isEnded()).isFalse(); + + BufferInfo outputBufferInfo = codec.getOutputBufferInfo(); + assertThat(outputBufferInfo).isNotNull(); + assertThat(outputBufferInfo.flags).isEqualTo(MediaCodec.BUFFER_FLAG_END_OF_STREAM); + + codec.releaseOutputBuffer(/* render= */ false); + + // After releasing the output buffer, stream should be ended. + assertThat(codec.isEnded()).isTrue(); + assertThat(codec.getOutputBuffer()).isNull(); + assertThat(codec.getOutputBufferInfo()).isNull(); + } + + @Test + public void queueInputBuffer_withoutDequeuing_throws() { + DecoderInputBuffer inputBuffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DIRECT); + + assertThrows(IllegalStateException.class, () -> codec.queueInputBuffer(inputBuffer)); + } + + @Test + public void releaseOutputBuffer_withRenderTrue_throws() { + DecoderInputBuffer inputBuffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DIRECT); + assertThat(codec.maybeDequeueInputBuffer(inputBuffer)).isTrue(); + codec.queueInputBuffer(inputBuffer); + + assertThrows( + IllegalArgumentException.class, () -> codec.releaseOutputBuffer(/* render= */ true)); + } + + @Test + public void getInputSurface_throwsException() { + assertThrows(UnsupportedOperationException.class, codec::getInputSurface); + } + + @Test + public void signalEndOfInputStream_throwsException() { + assertThrows(UnsupportedOperationException.class, codec::signalEndOfInputStream); + } + + @Test + public void maybeDequeueInputBuffer_afterEndOfStream_returnsFalse() { + DecoderInputBuffer inputBuffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DIRECT); + assertThat(codec.maybeDequeueInputBuffer(inputBuffer)).isTrue(); + // Flip buffer so that there is no readable data. + inputBuffer.data.flip(); + inputBuffer.addFlag(C.BUFFER_FLAG_END_OF_STREAM); + codec.queueInputBuffer(inputBuffer); + + assertThat(codec.maybeDequeueInputBuffer(inputBuffer)).isFalse(); + } + + @Test + public void queueInputBuffer_afterEndOfStream_throwsException() throws Exception { + DecoderInputBuffer inputBuffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DIRECT); + assertThat(codec.maybeDequeueInputBuffer(inputBuffer)).isTrue(); + // Flip buffer so that there is no readable data. + inputBuffer.data.flip(); + inputBuffer.addFlag(C.BUFFER_FLAG_END_OF_STREAM); + codec.queueInputBuffer(inputBuffer); + + assertThrows(IllegalStateException.class, () -> codec.queueInputBuffer(inputBuffer)); + } + + @Test + public void release_resetsStateAndThrowsOnSubsequentOperations() { + DecoderInputBuffer inputBuffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DIRECT); + assertThat(codec.maybeDequeueInputBuffer(inputBuffer)).isTrue(); + inputBuffer.data.put(buildTestData(/* length= */ 20)).flip(); + codec.queueInputBuffer(inputBuffer); + + codec.release(); + + assertThat(codec.isEnded()).isFalse(); + + assertThrows(IllegalStateException.class, () -> codec.maybeDequeueInputBuffer(inputBuffer)); + assertThrows(IllegalStateException.class, codec::release); + assertThrows(IllegalStateException.class, codec::getOutputBuffer); + assertThrows(IllegalStateException.class, codec::getOutputBufferInfo); + + // Set to null to avoid releasing twice. + codec = null; + } + + @Test + public void releaseOutputBuffer_withNoQueuedData_throws() { + assertThrows(IllegalStateException.class, () -> codec.releaseOutputBuffer(/* render= */ false)); + } + + @Test + public void releaseOutputBuffer_calledTwice_throws() { + DecoderInputBuffer inputBuffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DIRECT); + assertThat(codec.maybeDequeueInputBuffer(inputBuffer)).isTrue(); + codec.queueInputBuffer(inputBuffer); + + codec.releaseOutputBuffer(/* render= */ false); + assertThrows(IllegalStateException.class, () -> codec.releaseOutputBuffer(/* render= */ false)); + } + + @Test + public void queueInputBuffer_withDifferentBuffer_throwsException() { + DecoderInputBuffer inputBuffer = + new DecoderInputBuffer(DecoderInputBuffer.BUFFER_REPLACEMENT_MODE_DIRECT); + assertThat(codec.maybeDequeueInputBuffer(inputBuffer)).isTrue(); + inputBuffer.data = ByteBuffer.allocate(10); + + assertThrows(IllegalArgumentException.class, () -> codec.queueInputBuffer(inputBuffer)); + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/RendererTrackerTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/RendererTrackerTest.java new file mode 100644 index 00000000000..9ddf32c7812 --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/RendererTrackerTest.java @@ -0,0 +1,305 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static com.google.common.truth.Truth.assertThat; + +import androidx.annotation.Nullable; +import androidx.media3.common.C; +import androidx.media3.common.Timeline; +import androidx.media3.exoplayer.source.MediaSource.MediaPeriodId; +import androidx.media3.test.utils.FakeRenderer; +import androidx.media3.test.utils.FakeTimeline; +import androidx.media3.test.utils.FakeTimeline.TimelineWindowDefinition; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link RendererTracker}. */ +@RunWith(AndroidJUnit4.class) +public final class RendererTrackerTest { + + @Test + public void getActiveRenderer_emptyTracker_returnsNull() { + RendererTracker tracker = new RendererTracker(); + + assertThat(tracker.getActiveRenderer()).isNull(); + } + + @Test + public void getActiveRenderer_singleRegisteredRenderer_returnsIt() { + RendererTracker tracker = new RendererTracker(); + Timeline timeline = createFakeTimelineWithPeriods(2); + Object periodUid = timeline.getUidOfPeriod(0); + MediaPeriodId periodId = new MediaPeriodId(periodUid); + FakeTrackedRenderer renderer = createFakeRenderer(timeline, periodId); + + tracker.addRenderer(renderer); + + assertThat(tracker.getActiveRenderer()).isEqualTo(renderer); + } + + @Test + public void getActiveRenderer_multipleRegisteredRenderers_returnsEarliest() { + RendererTracker tracker = new RendererTracker(); + Timeline timeline = createFakeTimelineWithPeriods(3); + MediaPeriodId p0 = new MediaPeriodId(timeline.getUidOfPeriod(0)); + MediaPeriodId p1 = new MediaPeriodId(timeline.getUidOfPeriod(1)); + MediaPeriodId p2 = new MediaPeriodId(timeline.getUidOfPeriod(2)); + FakeTrackedRenderer r0 = createFakeRenderer(timeline, p0); + FakeTrackedRenderer r1 = createFakeRenderer(timeline, p1); + FakeTrackedRenderer r2 = createFakeRenderer(timeline, p2); + + // Register out of order + tracker.addRenderer(r2); + tracker.addRenderer(r1); + + // Should return r1 as it is earlier than r2 + assertThat(tracker.getActiveRenderer()).isEqualTo(r1); + + tracker.addRenderer(r0); + + // Should now return r0 + assertThat(tracker.getActiveRenderer()).isEqualTo(r0); + } + + @Test + public void getActiveRenderer_multipleWindows_returnsEarliest() { + RendererTracker tracker = new RendererTracker(); + // Timeline with 2 windows: Window 0 has 1 period, Window 1 has 1 period + Timeline timeline = createFakeTimelineWithWindows(1, 1); + MediaPeriodId p0 = new MediaPeriodId(timeline.getUidOfPeriod(0)); // Window 0, Period 0 + MediaPeriodId p1 = new MediaPeriodId(timeline.getUidOfPeriod(1)); // Window 1, Period 0 + FakeTrackedRenderer r0 = createFakeRenderer(timeline, p0); + FakeTrackedRenderer r1 = createFakeRenderer(timeline, p1); + + // Register out of order + tracker.addRenderer(r1); + tracker.addRenderer(r0); + + // Should return r0 as it is in Window 0 (earlier than Window 1) + assertThat(tracker.getActiveRenderer()).isEqualTo(r0); + } + + @Test + public void getActiveRenderer_afterUnregister_returnsNextEarliest() { + RendererTracker tracker = new RendererTracker(); + Timeline timeline = createFakeTimelineWithPeriods(3); + MediaPeriodId p0 = new MediaPeriodId(timeline.getUidOfPeriod(0)); + MediaPeriodId p1 = new MediaPeriodId(timeline.getUidOfPeriod(1)); + FakeTrackedRenderer r0 = createFakeRenderer(timeline, p0); + FakeTrackedRenderer r1 = createFakeRenderer(timeline, p1); + + tracker.addRenderer(r0); + tracker.addRenderer(r1); + + assertThat(tracker.getActiveRenderer()).isEqualTo(r0); + + tracker.removeRenderer(r0); + + assertThat(tracker.getActiveRenderer()).isEqualTo(r1); + } + + @Test + public void getActiveRenderer_unrecognizedPeriod_ignored() { + RendererTracker tracker = new RendererTracker(); + Timeline timeline = createFakeTimelineWithPeriods(2); + MediaPeriodId unrecognizedPeriod = new MediaPeriodId(new Object()); + MediaPeriodId p1 = new MediaPeriodId(timeline.getUidOfPeriod(1)); + FakeTrackedRenderer rUnrecognized = createFakeRenderer(timeline, unrecognizedPeriod); + FakeTrackedRenderer r1 = createFakeRenderer(timeline, p1); + + tracker.addRenderer(rUnrecognized); + tracker.addRenderer(r1); + + // unrecognizedPeriod is not in timeline, so it should be ignored. Only r1 is resolved. + assertThat(tracker.getActiveRenderer()).isEqualTo(r1); + } + + @Test + public void getActiveRenderer_nullPeriod_ignored() { + RendererTracker tracker = new RendererTracker(); + Timeline timeline = createFakeTimelineWithPeriods(2); + MediaPeriodId p1 = new MediaPeriodId(timeline.getUidOfPeriod(1)); + FakeTrackedRenderer rNull = createFakeRenderer(timeline, /* periodId= */ null); + FakeTrackedRenderer r1 = createFakeRenderer(timeline, p1); + + tracker.addRenderer(rNull); + tracker.addRenderer(r1); + + // rNull returns null period ID, so it should be ignored. Only r1 is resolved. + assertThat(tracker.getActiveRenderer()).isEqualTo(r1); + } + + @Test + public void addRenderer_multipleTimes_ignoredDuplicates() { + RendererTracker tracker = new RendererTracker(); + Timeline timeline = createFakeTimelineWithPeriods(2); + MediaPeriodId p0 = new MediaPeriodId(timeline.getUidOfPeriod(0)); + FakeTrackedRenderer r0 = createFakeRenderer(timeline, p0); + + tracker.addRenderer(r0); + tracker.addRenderer(r0); // Register twice + + tracker.removeRenderer(r0); + + // Since we use a Set now and registered the SAME renderer object, it is registered only once. + // Unregistering once should remove it completely. + assertThat(tracker.getActiveRenderer()).isNull(); + } + + @Test + public void getActiveRenderer_updateRendererPeriod_resorts() { + RendererTracker tracker = new RendererTracker(); + Timeline timeline = createFakeTimelineWithPeriods(3); + MediaPeriodId p0 = new MediaPeriodId(timeline.getUidOfPeriod(0)); + MediaPeriodId p1 = new MediaPeriodId(timeline.getUidOfPeriod(1)); + MediaPeriodId p2 = new MediaPeriodId(timeline.getUidOfPeriod(2)); + FakeTrackedRenderer r0 = createFakeRenderer(timeline, p1); + FakeTrackedRenderer r1 = createFakeRenderer(timeline, p2); + + // Initially r0 is at p1, r1 is at p2. Active is r0. + tracker.addRenderer(r0); + tracker.addRenderer(r1); + + assertThat(tracker.getActiveRenderer()).isEqualTo(r0); + + // Update r1 to p0. Now active should be r1 (at p0). + r1.setMediaPeriodId(p0); + + assertThat(tracker.getActiveRenderer()).isEqualTo(r1); + } + + @Test + public void getActiveRenderer_timelineChange_resorts() { + RendererTracker tracker = new RendererTracker(); + Timeline timeline1 = createFakeTimelineWithWindows(1, 1); + MediaPeriodId p0 = new MediaPeriodId(timeline1.getUidOfPeriod(0)); + MediaPeriodId p1 = new MediaPeriodId(timeline1.getUidOfPeriod(1)); + FakeTrackedRenderer r0 = createFakeRenderer(timeline1, p0); + FakeTrackedRenderer r1 = createFakeRenderer(timeline1, p1); + + tracker.addRenderer(r0); + tracker.addRenderer(r1); + + // In timeline1, r0 (p0) is at index 0, r1 (p1) is at index 1. Active is r0. + assertThat(tracker.getActiveRenderer()).isEqualTo(r0); + + // Create timeline2 with swapped windows (Window 0 has UID 1, Window 1 has UID 0) + TimelineWindowDefinition w0 = + new TimelineWindowDefinition.Builder().setPeriodCount(1).setUid(1).build(); + TimelineWindowDefinition w1 = + new TimelineWindowDefinition.Builder().setPeriodCount(1).setUid(0).build(); + Timeline timeline2 = new FakeTimeline(w0, w1); + + // In timeline2, r1 (p1) is at index 0, r0 (p0) is at index 1. Active should be r1. + r0.setTimeline(timeline2); + r1.setTimeline(timeline2); + assertThat(tracker.getActiveRenderer()).isEqualTo(r1); + } + + @Test + public void getActiveRenderer_emptyTimeline_returnsNull() { + RendererTracker tracker = new RendererTracker(); + FakeTrackedRenderer r0 = createFakeRenderer(Timeline.EMPTY, new MediaPeriodId(new Object())); + + tracker.addRenderer(r0); + + assertThat(tracker.getActiveRenderer()).isNull(); + } + + @Test + public void getActiveRenderer_allUnrecognizedPeriods_returnsNull() { + RendererTracker tracker = new RendererTracker(); + Timeline timeline = createFakeTimelineWithPeriods(2); + MediaPeriodId unrecognizedPeriod0 = new MediaPeriodId(new Object()); + MediaPeriodId unrecognizedPeriod1 = new MediaPeriodId(new Object()); + FakeTrackedRenderer r0 = createFakeRenderer(timeline, unrecognizedPeriod0); + FakeTrackedRenderer r1 = createFakeRenderer(timeline, unrecognizedPeriod1); + + tracker.addRenderer(r0); + tracker.addRenderer(r1); + + assertThat(tracker.getActiveRenderer()).isNull(); + } + + @Test + public void getActiveRenderer_differentWindowSequenceNumbers_sortedBySequenceNumber() { + RendererTracker tracker = new RendererTracker(); + Timeline timeline = createFakeTimelineWithPeriods(2); + // p0 is at index 0 but has later windowSequenceNumber (10) + MediaPeriodId p0 = + new MediaPeriodId(timeline.getUidOfPeriod(0), /* windowSequenceNumber= */ 10); + // p1 is at index 1 but has earlier windowSequenceNumber (5) + MediaPeriodId p1 = new MediaPeriodId(timeline.getUidOfPeriod(1), /* windowSequenceNumber= */ 5); + FakeTrackedRenderer r0 = createFakeRenderer(timeline, p0); + FakeTrackedRenderer r1 = createFakeRenderer(timeline, p1); + + tracker.addRenderer(r0); + tracker.addRenderer(r1); + + // Active should be r1 because its windowSequenceNumber is smaller (5 < 10), + // even though r0 is at index 0 in the timeline. + assertThat(tracker.getActiveRenderer()).isEqualTo(r1); + } + + private static final class FakeTrackedRenderer extends FakeRenderer + implements RendererTracker.TrackedRenderer { + @Nullable private MediaPeriodId mediaPeriodId; + + FakeTrackedRenderer(@Nullable MediaPeriodId mediaPeriodId) { + super(C.TRACK_TYPE_VIDEO); + this.mediaPeriodId = mediaPeriodId; + } + + void setMediaPeriodId(@Nullable MediaPeriodId mediaPeriodId) { + this.mediaPeriodId = mediaPeriodId; + } + + @Override + @Nullable + public MediaPeriodId getTrackedMediaPeriodId() { + return mediaPeriodId; + } + + @Override + @Nullable + public Timeline getTrackedTimeline() { + return getTimeline(); + } + } + + private static Timeline createFakeTimelineWithPeriods(int periodCount) { + return new FakeTimeline( + new TimelineWindowDefinition.Builder().setPeriodCount(periodCount).build()); + } + + private static Timeline createFakeTimelineWithWindows(int... periodCounts) { + TimelineWindowDefinition[] definitions = new TimelineWindowDefinition[periodCounts.length]; + for (int i = 0; i < periodCounts.length; i++) { + definitions[i] = + new TimelineWindowDefinition.Builder().setPeriodCount(periodCounts[i]).setUid(i).build(); + } + return new FakeTimeline(definitions); + } + + private static FakeTrackedRenderer createFakeRenderer( + Timeline timeline, @Nullable MediaPeriodId periodId) { + FakeTrackedRenderer renderer = new FakeTrackedRenderer(periodId); + renderer.setTimeline(timeline); + return renderer; + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/SampleExporterTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/SampleExporterTest.java new file mode 100644 index 00000000000..56982e5a36e --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/SampleExporterTest.java @@ -0,0 +1,79 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static com.google.common.truth.Truth.assertThat; + +import android.content.Context; +import androidx.media3.common.C; +import androidx.media3.common.Format; +import androidx.media3.common.MimeTypes; +import androidx.media3.common.util.Util; +import androidx.media3.test.utils.robolectric.ShadowMediaCodecConfig; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link SampleExporter}. */ +@RunWith(AndroidJUnit4.class) +public class SampleExporterTest { + + @Rule + public ShadowMediaCodecConfig shadowMediaCodecConfig = + ShadowMediaCodecConfig.withCodecs( + /* decoders= */ ImmutableList.of(), + /* encoders= */ ImmutableList.of(ShadowMediaCodecConfig.CODEC_INFO_AAC)); + + private Context context; + + @Before + public void setUp() { + context = ApplicationProvider.getApplicationContext(); + } + + @Test + public void + findSupportedMimeTypeForEncoderAndMuxer_withRawAudioAndNoEncodingNeeded_returnsRawAudio() { + Format pcmFormat = + Util.getPcmFormat(C.ENCODING_PCM_16BIT, /* channels= */ 1, /* sampleRate= */ 44100); + String sampleMimeType = + SampleExporter.findSupportedMimeTypeForEncoderAndMuxer( + pcmFormat, + new DefaultMuxer.Factory().getSupportedSampleMimeTypes(C.TRACK_TYPE_AUDIO), + new DefaultEncoderFactory.Builder(context).build()); + assertThat(sampleMimeType).isEqualTo(MimeTypes.AUDIO_RAW); + } + + @Test + public void + findSupportedMimeTypeForEncoderAndMuxer_withRawAudioAndSetBitrate_returnsEncodedAudio() { + Format pcmFormat = + Util.getPcmFormat(C.ENCODING_PCM_16BIT, /* channels= */ 1, /* sampleRate= */ 44100); + String sampleMimeType = + SampleExporter.findSupportedMimeTypeForEncoderAndMuxer( + pcmFormat, + new DefaultMuxer.Factory().getSupportedSampleMimeTypes(C.TRACK_TYPE_AUDIO), + new DefaultEncoderFactory.Builder(context) + .setRequestedAudioEncoderSettings( + new AudioEncoderSettings.Builder().setBitrate(1000).build()) + .build()); + assertThat(sampleMimeType).isEqualTo(MimeTypes.AUDIO_AAC); + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/SequenceRenderersFactoryTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/SequenceRenderersFactoryTest.java new file mode 100644 index 00000000000..6951963bb13 --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/SequenceRenderersFactoryTest.java @@ -0,0 +1,338 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static androidx.media3.test.utils.FakeSampleStream.FakeSampleStreamItem.END_OF_STREAM_ITEM; +import static androidx.media3.test.utils.FakeSampleStream.FakeSampleStreamItem.oneByteSample; +import static androidx.media3.transformer.EditedMediaItemSequence.withAudioFrom; +import static androidx.media3.transformer.TransformerUtil.releaseIfNeeded; +import static androidx.test.core.app.ApplicationProvider.getApplicationContext; +import static com.google.common.truth.Truth.assertThat; +import static com.google.common.util.concurrent.Futures.immediateFuture; +import static org.mockito.ArgumentMatchers.any; +import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.when; +import static org.robolectric.Shadows.shadowOf; + +import android.graphics.Bitmap; +import android.graphics.ImageFormat; +import android.os.Handler; +import android.os.HandlerThread; +import android.os.Looper; +import androidx.media3.common.C; +import androidx.media3.common.Format; +import androidx.media3.common.MediaItem; +import androidx.media3.common.MimeTypes; +import androidx.media3.common.Timeline; +import androidx.media3.common.util.ConstantRateTimestampIterator; +import androidx.media3.common.util.SystemClock; +import androidx.media3.common.util.Util; +import androidx.media3.common.video.AsyncFrame; +import androidx.media3.effect.HardwareBufferJniWrapper; +import androidx.media3.exoplayer.ExoPlaybackException; +import androidx.media3.exoplayer.Renderer; +import androidx.media3.exoplayer.RendererConfiguration; +import androidx.media3.exoplayer.analytics.PlayerId; +import androidx.media3.exoplayer.audio.AudioRendererEventListener; +import androidx.media3.exoplayer.audio.DefaultAudioSink; +import androidx.media3.exoplayer.drm.DrmSessionEventListener; +import androidx.media3.exoplayer.drm.DrmSessionManager; +import androidx.media3.exoplayer.image.ExternallyLoadedImageDecoder; +import androidx.media3.exoplayer.metadata.MetadataOutput; +import androidx.media3.exoplayer.source.MediaSource; +import androidx.media3.exoplayer.text.TextOutput; +import androidx.media3.exoplayer.upstream.DefaultAllocator; +import androidx.media3.exoplayer.video.VideoRendererEventListener; +import androidx.media3.test.utils.FakeSampleStream; +import androidx.media3.test.utils.FakeTimeline; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.base.Supplier; +import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableSet; +import java.util.ArrayList; +import java.util.List; +import org.junit.After; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit test for {@link SequenceRenderersFactory}. */ +@RunWith(AndroidJUnit4.class) +public final class SequenceRenderersFactoryTest { + + private HandlerThread handlerThread; + + @Before + public void setUp() { + handlerThread = new HandlerThread("SequenceRenderersFactoryTest"); + handlerThread.start(); + } + + @After + public void tearDown() { + handlerThread.quitSafely(); + } + + @Test + public void hardwareBufferImageRenderer_implementsRendererWakeupListener_andForwardsWakeup() + throws Exception { + SequenceRenderersFactory factory = + createFactoryForHardwareBuffer(/* hardwareBufferFrameReaderSupplier= */ () -> null); + Renderer imageRenderer = createRenderer(factory, C.TRACK_TYPE_IMAGE); + + assertThat(imageRenderer).isInstanceOf(HardwareBufferFrameReader.RendererWakeupListener.class); + FakeWakeupListener fakeWakeupListener = new FakeWakeupListener(); + + imageRenderer.handleMessage(Renderer.MSG_SET_WAKEUP_LISTENER, fakeWakeupListener); + ((HardwareBufferFrameReader.RendererWakeupListener) imageRenderer).onWakeup(); + + assertThat(fakeWakeupListener.onWakeupCalled).isTrue(); + } + + @Test + public void hardwareBufferImageRenderer_onEnabledRegistersWakeupListener_onDisabledUnregisters() + throws Exception { + List receivedFrames = new ArrayList<>(); + EditedMediaItemSequence sequence = createSequence(); + HardwareBufferFrameReader hardwareBufferFrameReader = + createHardwareBufferFrameReader(sequence, receivedFrames, handlerThread.getLooper()); + SequenceRenderersFactory factory = + createFactoryForHardwareBuffer(() -> hardwareBufferFrameReader); + Renderer imageRenderer = createRenderer(factory, C.TRACK_TYPE_IMAGE); + Timeline timeline = + new CompositionPlayer.CompositionForwardingTimeline(new FakeTimeline(), sequence); + enableRenderer(imageRenderer, timeline); + FakeWakeupListener fakeWakeupListener = new FakeWakeupListener(); + imageRenderer.handleMessage(Renderer.MSG_SET_WAKEUP_LISTENER, fakeWakeupListener); + + hardwareBufferFrameReader.outputBitmap( + Bitmap.createBitmap(/* width= */ 1, /* height= */ 1, Bitmap.Config.ARGB_8888), + new ConstantRateTimestampIterator(/* durationUs= */ 1_000_000, /* frameRate= */ 2f), + /* sequenceOffsetUs= */ 0, + /* indexOfItem= */ 0); + releaseIfNeeded(receivedFrames.get(0).frame, /* releaseFence= */ null); + shadowOf(handlerThread.getLooper()).idle(); + + assertThat(fakeWakeupListener.onWakeupCalled).isTrue(); + + fakeWakeupListener.onWakeupCalled = false; + imageRenderer.disable(); + + releaseIfNeeded(receivedFrames.get(1).frame, /* releaseFence= */ null); + shadowOf(handlerThread.getLooper()).idle(); + + assertThat(fakeWakeupListener.onWakeupCalled).isFalse(); + } + + @Test + public void hardwareBufferVideoRenderer_implementsRendererWakeupListener_andForwardsWakeup() + throws Exception { + SequenceRenderersFactory factory = + createFactoryForHardwareBuffer(/* hardwareBufferFrameReaderSupplier= */ () -> null); + Renderer videoRenderer = createRenderer(factory, C.TRACK_TYPE_VIDEO); + + assertThat(videoRenderer).isInstanceOf(HardwareBufferFrameReader.RendererWakeupListener.class); + FakeWakeupListener fakeWakeupListener = new FakeWakeupListener(); + + videoRenderer.handleMessage(Renderer.MSG_SET_WAKEUP_LISTENER, fakeWakeupListener); + ((HardwareBufferFrameReader.RendererWakeupListener) videoRenderer).onWakeup(); + + assertThat(fakeWakeupListener.onWakeupCalled).isTrue(); + } + + @Test + public void hardwareBufferVideoRenderer_onEnabledRegistersWakeupListener_onDisabledUnregisters() + throws Exception { + List receivedFrames = new ArrayList<>(); + EditedMediaItemSequence sequence = createSequence(); + HardwareBufferFrameReader hardwareBufferFrameReader = + createHardwareBufferFrameReader(sequence, receivedFrames, handlerThread.getLooper()); + + SequenceRenderersFactory factory = + createFactoryForHardwareBuffer(() -> hardwareBufferFrameReader); + Renderer videoRenderer = createRenderer(factory, C.TRACK_TYPE_VIDEO); + + Timeline timeline = + new CompositionPlayer.CompositionForwardingTimeline(new FakeTimeline(), sequence); + enableRenderer(videoRenderer, timeline); + FakeWakeupListener fakeWakeupListener = new FakeWakeupListener(); + videoRenderer.handleMessage(Renderer.MSG_SET_WAKEUP_LISTENER, fakeWakeupListener); + + hardwareBufferFrameReader.outputBitmap( + Bitmap.createBitmap(/* width= */ 1, /* height= */ 1, Bitmap.Config.ARGB_8888), + new ConstantRateTimestampIterator(/* durationUs= */ 1_000_000, /* frameRate= */ 2f), + /* sequenceOffsetUs= */ 0, + /* indexOfItem= */ 0); + releaseIfNeeded(receivedFrames.get(0).frame, /* releaseFence= */ null); + shadowOf(handlerThread.getLooper()).idle(); + + assertThat(fakeWakeupListener.onWakeupCalled).isTrue(); + + fakeWakeupListener.onWakeupCalled = false; + videoRenderer.disable(); + + releaseIfNeeded(receivedFrames.get(1).frame, /* releaseFence= */ null); + shadowOf(handlerThread.getLooper()).idle(); + + assertThat(fakeWakeupListener.onWakeupCalled).isFalse(); + } + + @Test + @SuppressWarnings("deprecation") // Uses deprecated CompositionFrameMetadata. + public void hardwareBufferImageRenderer_loopingSequence_wrapsItemIndexInFrameMetadata() + throws Exception { + List receivedFrames = new ArrayList<>(); + EditedMediaItemSequence loopingSequence = + new EditedMediaItemSequence.Builder(ImmutableSet.of(C.TRACK_TYPE_AUDIO)) + .addItems(createSequence().editedMediaItems) + .setIsLooping(true) + .build(); + HardwareBufferFrameReader hardwareBufferFrameReader = + createHardwareBufferFrameReader(loopingSequence, receivedFrames, handlerThread.getLooper()); + Renderer imageRenderer = + createRenderer( + createFactoryForHardwareBuffer(() -> hardwareBufferFrameReader), C.TRACK_TYPE_IMAGE); + Timeline timeline = + new CompositionPlayer.CompositionForwardingTimeline( + new FakeTimeline( + new FakeTimeline.TimelineWindowDefinition(/* periodCount= */ 2, /* id= */ 0)), + loopingSequence); + enableRenderer(imageRenderer, timeline, /* periodIndex= */ 1); + + while (receivedFrames.isEmpty()) { + imageRenderer.render(/* positionUs= */ 0L, /* elapsedRealtimeUs= */ 0L); + } + + CompositionFrameMetadata metadata = + (CompositionFrameMetadata) + receivedFrames + .get(0) + .frame + .getMetadata() + .get(CompositionFrameMetadata.KEY_COMPOSITION_FRAME_METADATA); + assertThat(metadata.itemIndex).isEqualTo(0); + } + + private static SequenceRenderersFactory createFactoryForHardwareBuffer( + Supplier hardwareBufferFrameReaderSupplier) { + return SequenceRenderersFactory.createForHardwareBuffer( + getApplicationContext(), + new PlaybackAudioGraphWrapper( + new DefaultAudioMixer.Factory(), + new DefaultAudioSink.Builder(getApplicationContext()).build()), + new ExternallyLoadedImageDecoder.Factory( + request -> + immediateFuture( + Bitmap.createBitmap(/* width= */ 1, /* height= */ 1, Bitmap.Config.ARGB_8888))), + /* inputIndex= */ 0, + /* videoPrewarmingEnabled= */ false, + /* compositionRendererListener= */ mock( + SequenceRenderersFactory.CompositionRendererListener.class), + hardwareBufferFrameReaderSupplier, + /* lateThresholdToDropInputUs= */ 0); + } + + private static Renderer createRenderer( + SequenceRenderersFactory factory, @C.TrackType int trackType) { + Renderer[] renderers = + factory.createRenderers( + /* eventHandler= */ new Handler(Looper.getMainLooper()), + /* videoRendererEventListener= */ mock(VideoRendererEventListener.class), + /* audioRendererEventListener= */ mock(AudioRendererEventListener.class), + /* textRendererOutput= */ mock(TextOutput.class), + /* metadataRendererOutput= */ mock(MetadataOutput.class)); + for (Renderer renderer : renderers) { + if (renderer.getTrackType() == trackType) { + return renderer; + } + } + throw new IllegalStateException("No renderer found for track type: " + trackType); + } + + private static EditedMediaItemSequence createSequence() { + EditedMediaItem editedMediaItem = + new EditedMediaItem.Builder(MediaItem.fromUri("https://example.com")) + .setDurationUs(1_000_000) + .build(); + return withAudioFrom(ImmutableList.of(editedMediaItem)); + } + + private static HardwareBufferFrameReader createHardwareBufferFrameReader( + EditedMediaItemSequence sequence, List receivedFrames, Looper looper) { + Composition composition = new Composition.Builder(sequence, createSequence()).build(); + HardwareBufferJniWrapper mockJniWrapper = mock(HardwareBufferJniWrapper.class); + when(mockJniWrapper.nativeCopyBitmapToHardwareBuffer(any(), any())).thenReturn(true); + return new HardwareBufferFrameReader( + composition, + /* sequenceIndex= */ 0, + /* frameConsumer= */ receivedFrames::add, + looper, + /* defaultSurfacePixelFormat= */ ImageFormat.YUV_420_888, + new DefaultImageReaderAdapter.Factory(), + /* listener= */ e -> {}, + SystemClock.DEFAULT.createHandler(Util.getCurrentOrMainLooper(), /* callback= */ null), + mockJniWrapper); + } + + private static void enableRenderer(Renderer renderer, Timeline timeline) + throws ExoPlaybackException { + enableRenderer(renderer, timeline, /* periodIndex= */ 0); + } + + private static void enableRenderer(Renderer renderer, Timeline timeline, int periodIndex) + throws ExoPlaybackException { + Format imageFormat = + new Format.Builder() + .setSampleMimeType(MimeTypes.APPLICATION_EXTERNALLY_LOADED_IMAGE) + .setTileCountHorizontal(1) + .setTileCountVertical(1) + .build(); + FakeSampleStream fakeSampleStream = + new FakeSampleStream( + new DefaultAllocator(/* trimOnReset= */ true, /* individualAllocationSize= */ 1024), + /* mediaSourceEventDispatcher= */ null, + DrmSessionManager.DRM_UNSUPPORTED, + new DrmSessionEventListener.EventDispatcher(), + imageFormat, + ImmutableList.of( + oneByteSample(/* timeUs= */ 0L, C.BUFFER_FLAG_KEY_FRAME), END_OF_STREAM_ITEM)); + fakeSampleStream.writeData(/* startPositionUs= */ 0); + renderer.init(/* index= */ 0, PlayerId.UNSET, SystemClock.DEFAULT); + renderer.setTimeline(timeline); + renderer.enable( + RendererConfiguration.DEFAULT, + new Format[] {imageFormat}, + fakeSampleStream, + /* positionUs= */ 0L, + /* joining= */ false, + /* mayRenderStartOfStream= */ true, + /* startPositionUs= */ 0L, + /* offsetUs= */ 0L, + new MediaSource.MediaPeriodId(timeline.getUidOfPeriod(periodIndex))); + } + + private static final class FakeWakeupListener implements Renderer.WakeupListener { + private boolean onWakeupCalled; + + @Override + public void onSleep() {} + + @Override + public void onWakeup() { + onWakeupCalled = true; + } + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/TestUtil.java b/libraries/transformer/src/test/java/androidx/media3/transformer/TestUtil.java index 25b482440b3..c3f258ba970 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/TestUtil.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/TestUtil.java @@ -18,38 +18,51 @@ import static android.os.Build.VERSION.SDK_INT; import static androidx.media3.test.utils.TestUtil.extractAllSamplesFromFilePath; import static androidx.test.core.app.ApplicationProvider.getApplicationContext; +import static com.google.common.base.Preconditions.checkArgument; import static com.google.common.base.Preconditions.checkNotNull; +import static com.google.common.collect.ImmutableList.toImmutableList; +import static com.google.common.truth.Truth.assertWithMessage; -import android.graphics.Bitmap; import android.graphics.SurfaceTexture; import android.hardware.HardwareBuffer; import android.media.Image; import android.os.Handler; +import android.util.Rational; import android.view.Surface; import androidx.annotation.Nullable; import androidx.media3.common.C; +import androidx.media3.common.MediaItem; +import androidx.media3.common.PlaybackException; +import androidx.media3.common.Player; import androidx.media3.common.audio.AudioProcessor; import androidx.media3.common.audio.ChannelMixingAudioProcessor; import androidx.media3.common.audio.ChannelMixingMatrix; import androidx.media3.common.audio.SonicAudioProcessor; import androidx.media3.common.util.Consumer; import androidx.media3.common.util.UnstableApi; +import androidx.media3.common.util.Util; import androidx.media3.common.video.FrameProcessor; import androidx.media3.effect.HardwareBufferJniWrapper; import androidx.media3.extractor.mp4.Mp4Extractor; import androidx.media3.extractor.text.DefaultSubtitleParserFactory; +import androidx.media3.test.utils.AssetInfo; +import androidx.media3.test.utils.CapturingFrameProcessor; import androidx.media3.test.utils.FakeClock; import androidx.media3.test.utils.FakeExtractorOutput; +import androidx.media3.test.utils.FakeHardwareBufferJniWrapper; import androidx.media3.test.utils.FakeTrackOutput; import androidx.media3.test.utils.PassthroughAudioProcessor; +import androidx.media3.test.utils.robolectric.TestPlayerRunHelper; import androidx.test.core.app.ApplicationProvider; import com.google.common.collect.ImmutableList; import com.google.common.collect.Iterables; import java.io.IOException; +import java.math.RoundingMode; import java.util.ArrayDeque; import java.util.List; import java.util.Queue; import java.util.StringJoiner; +import java.util.concurrent.TimeoutException; import java.util.concurrent.atomic.AtomicReference; /** Utility class for {@link Transformer} unit tests */ @@ -89,6 +102,10 @@ public final class TestUtil { public static final String FILE_PNG = "png/media3test.png"; private static final String DUMP_FILE_OUTPUT_DIRECTORY = "transformerdumps"; private static final String DUMP_FILE_EXTENSION = "dump"; + public static final Rational FPS_10 = new Rational(10, 1); + public static final Rational FPS_30 = new Rational(30, 1); + public static final Rational FPS_60 = new Rational(60, 1); + public static final Rational FPS_HALF = new Rational(1, 2); private TestUtil() {} @@ -244,6 +261,107 @@ public static CompositionPlayer.Builder createTestHardwareBufferCompositionPlaye .experimentalSetLateThresholdToDropInputUs(C.TIME_UNSET); } + /** + * Sets up a {@link CompositionPlayer} with fake hardware buffer components, sets the given {@link + * Composition}, and prepares the player, waiting for it to reach {@link Player#STATE_READY}. + * + * @param composition The {@link Composition} to set. + * @param frameProcessorFactory The {@link FrameProcessor.Factory} to use. + * @return The prepared {@link CompositionPlayer} in {@link Player#STATE_READY} state. + * @throws PlaybackException If a fatal playback error occurs during preparation. + * @throws TimeoutException If the player fails to reach {@link Player#STATE_READY} within the + * timeout. + */ + public static CompositionPlayer setupAndPrepareHardwareBufferPlayer( + Composition composition, FrameProcessor.Factory frameProcessorFactory) + throws PlaybackException, TimeoutException { + CompositionPlayer player = + createTestHardwareBufferCompositionPlayerBuilder(frameProcessorFactory).build(); + player.setComposition(composition); + player.prepare(); + TestPlayerRunHelper.advance(player).untilState(Player.STATE_READY); + return player; + } + + /** + * Asserts that the deltas between consecutive timestamps match the expected frame rate. + * + * @param timestampsUs The list of timestamps in microseconds. + * @param expectedFps The expected frame rate. + */ + public static void assertTimestampsMatchFrameRate(List timestampsUs, Rational expectedFps) { + assertWithMessage("Expected at least two timestamps to verify frame rate") + .that(timestampsUs.size()) + .isAtLeast(2); + long expectedDeltaUs = + Util.scaleLargeValue( + 1_000_000L, + expectedFps.getDenominator(), + expectedFps.getNumerator(), + RoundingMode.HALF_UP); + for (int i = 1; i < timestampsUs.size(); i++) { + long deltaUs = timestampsUs.get(i) - timestampsUs.get(i - 1); + // Absolute timestamps rounded to the nearest microsecond can cause + // deltas to fluctuate by 1us from the exact mathematical duration. + assertWithMessage( + "Time between frames %s (%s) and %s (%s) does not match expected %s fps", + i - 1, timestampsUs.get(i - 1), i, timestampsUs.get(i), expectedFps) + .that(deltaUs) + .isWithin(1L) + .of(expectedDeltaUs); + } + } + + /** + * Extracts the first timestamp of each queued frame group from the {@link + * CapturingFrameProcessor}. + * + * @param frameProcessor The {@link CapturingFrameProcessor} to query. + * @return An {@link ImmutableList} of queued content timestamps in microseconds. + */ + public static ImmutableList getQueuedContentTimesUs( + CapturingFrameProcessor frameProcessor) { + return frameProcessor.getQueuedContentTimesUs().stream() + .map(list -> Iterables.getFirst(list, C.TIME_UNSET)) + .filter(timeUs -> timeUs != C.TIME_UNSET) + .collect(toImmutableList()); + } + + /** + * Builds a {@link Composition} from a list of asset sequences. + * + *

    The composition is configured with the specified video frame aggregation frame rate. + * + * @param sequencesAssets A list of sequences, where each sequence is a list of {@link + * AssetInfo}s. + * @param aggregationFrameRate The target frame rate for video frame aggregation, or {@code null} + * to use the default aggregation behavior (follows the primary sequence's physical + * timestamps). + * @return The constructed {@link Composition}. + */ + public static Composition buildComposition( + List> sequencesAssets, @Nullable Rational aggregationFrameRate) { + ImmutableList.Builder sequences = ImmutableList.builder(); + for (List sequenceAssets : sequencesAssets) { + ImmutableList.Builder editedMediaItems = ImmutableList.builder(); + for (AssetInfo asset : sequenceAssets) { + checkArgument( + asset.videoDurationUs != C.TIME_UNSET, "Asset duration is unset: %s", asset.uri); + editedMediaItems.add( + new EditedMediaItem.Builder(MediaItem.fromUri(asset.uri)) + .setDurationUs(asset.videoDurationUs) + .build()); + } + sequences.add(EditedMediaItemSequence.withAudioAndVideoFrom(editedMediaItems.build())); + } + return new Composition.Builder(sequences.build()) + .setVideoFrameAggregationParameters( + new VideoFrameAggregationParameters.Builder() + .setFrameRate(aggregationFrameRate) + .build()) + .build(); + } + public static final class FormatCapturingAudioProcessor extends PassthroughAudioProcessor { public final AtomicReference inputFormat = new AtomicReference<>(); @@ -377,34 +495,4 @@ public ImageReaderAdapter create(int width, int height, int format, int maxImage return new FakeImageReaderAdapter(); } } - - /** A no-op {@link HardwareBufferJniWrapper} that always succeeds. */ - private static final class FakeHardwareBufferJniWrapper implements HardwareBufferJniWrapper { - @Override - public long nativeCreateEglImageFromHardwareBuffer( - long displayHandle, HardwareBuffer hardwareBuffer) { - return 1L; - } - - @Override - public boolean nativeBindEGLImage(int target, long eglImageHandle) { - return true; - } - - @Override - public boolean nativeDestroyEGLImage(long displayHandle, long imageHandle) { - return true; - } - - @Override - public boolean nativeCopyBitmapToHardwareBuffer(Bitmap bitmap, HardwareBuffer hb) { - return true; - } - - @Override - public boolean nativeCopyHardwareBufferToHardwareBuffer( - HardwareBuffer srcHb, HardwareBuffer dstHb) { - return true; - } - } } diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/TransformerOutputResolverTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/TransformerOutputResolverTest.java new file mode 100644 index 00000000000..9d1e7a2e060 --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/TransformerOutputResolverTest.java @@ -0,0 +1,99 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; + +import android.content.ContentValues; +import android.content.Context; +import android.net.Uri; +import android.provider.MediaStore; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import java.io.FileNotFoundException; +import org.junit.Before; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link TransformerOutputResolver}. */ +@RunWith(AndroidJUnit4.class) +public final class TransformerOutputResolverTest { + + private Context context; + + @Before + public void setUp() { + context = ApplicationProvider.getApplicationContext(); + } + + @Test + public void insertMediaStoreEntry_validValues_returnsItemUri() throws Exception { + ContentValues values = new ContentValues(); + values.put(MediaStore.MediaColumns.DISPLAY_NAME, "test_video.mp4"); + values.put(MediaStore.MediaColumns.MIME_TYPE, "video/mp4"); + + Uri itemUri = + TransformerOutputResolver.insertMediaStoreEntry( + context, MediaStore.Video.Media.EXTERNAL_CONTENT_URI, values); + + assertThat(itemUri).isNotNull(); + assertThat(itemUri.toString()) + .startsWith(MediaStore.Video.Media.EXTERNAL_CONTENT_URI.toString()); + } + + @Test + public void clearMediaStorePendingFlag_doesNotThrow() throws Exception { + ContentValues values = new ContentValues(); + values.put(MediaStore.MediaColumns.DISPLAY_NAME, "test_clear.mp4"); + values.put(MediaStore.MediaColumns.MIME_TYPE, "video/mp4"); + + Uri itemUri = + TransformerOutputResolver.insertMediaStoreEntry( + context, MediaStore.Video.Media.EXTERNAL_CONTENT_URI, values); + + TransformerOutputResolver.clearMediaStorePendingFlag(context, itemUri); + } + + @Test + public void deleteMediaStoreEntry_afterInsert_doesNotThrow() throws Exception { + ContentValues values = new ContentValues(); + values.put(MediaStore.MediaColumns.DISPLAY_NAME, "test_delete.mp4"); + values.put(MediaStore.MediaColumns.MIME_TYPE, "video/mp4"); + Uri itemUri = + TransformerOutputResolver.insertMediaStoreEntry( + context, MediaStore.Video.Media.EXTERNAL_CONTENT_URI, values); + + TransformerOutputResolver.deleteMediaStoreEntry(context, itemUri); + } + + @Test + public void deleteMediaStoreEntry_invalidUri_doesNotThrow() { + Uri invalidUri = Uri.parse("content://media/external/video/media/99999999"); + + // Deletion is best effort, so that a cleanup failure cannot mask the error that triggered it. + TransformerOutputResolver.deleteMediaStoreEntry(context, invalidUri); + } + + @Test + public void openFileDescriptor_invalidUri_throwsFileNotFoundException() { + Uri invalidUri = Uri.parse("content://media/external/video/media/99999999"); + + assertThrows( + FileNotFoundException.class, + () -> TransformerOutputResolver.openFileDescriptor(context, invalidUri)); + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/TransformerOutputTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/TransformerOutputTest.java new file mode 100644 index 00000000000..4315e8090a3 --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/TransformerOutputTest.java @@ -0,0 +1,67 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static com.google.common.truth.Truth.assertThat; + +import android.content.ContentValues; +import android.net.Uri; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link TransformerOutput}. */ +@RunWith(AndroidJUnit4.class) +public final class TransformerOutputTest { + + @Test + public void create_withValidPath_setsPathCorrectly() { + String path = "/tmp/test_output.mp4"; + + TransformerOutput output = TransformerOutput.create(path); + + assertThat(output.outputType).isEqualTo(TransformerOutput.TYPE_FILE_PATH); + assertThat(output.path).isEqualTo(path); + assertThat(output.collectionUri).isNull(); + assertThat(output.contentValues).isNull(); + } + + @Test + public void create_withValidMediaStoreParameters_setsParametersCorrectly() { + Uri collectionUri = Uri.parse("content://media/external/video/media"); + ContentValues values = new ContentValues(); + values.put("title", "test_video"); + + TransformerOutput output = TransformerOutput.create(collectionUri, values); + + assertThat(output.outputType).isEqualTo(TransformerOutput.TYPE_MEDIA_STORE); + assertThat(output.path).isNull(); + assertThat(output.collectionUri).isEqualTo(collectionUri); + assertThat(output.contentValues).isEqualTo(values); + } + + @Test + public void create_withMediaStoreParameters_defensivelyCopiesContentValues() { + Uri collectionUri = Uri.parse("content://media/external/video/media"); + ContentValues values = new ContentValues(); + values.put("title", "initial_title"); + + TransformerOutput output = TransformerOutput.create(collectionUri, values); + values.put("title", "mutated_title"); + + assertThat(output.contentValues.getAsString("title")).isEqualTo("initial_title"); + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/TransformerUtilTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/TransformerUtilTest.java index df2edcf7398..6785ca0a6c3 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/TransformerUtilTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/TransformerUtilTest.java @@ -17,6 +17,7 @@ import static androidx.media3.common.MimeTypes.VIDEO_H264; import static androidx.media3.transformer.EditedMediaItemSequence.withAudioAndVideoFrom; +import static androidx.media3.transformer.EditedMediaItemSequence.withAudioFrom; import static androidx.media3.transformer.MuxerWrapper.MUXER_MODE_DEFAULT; import static androidx.media3.transformer.TestUtil.ASSET_URI_PREFIX; import static androidx.media3.transformer.TestUtil.FILE_AUDIO_VIDEO; @@ -29,14 +30,20 @@ import androidx.media3.common.Effect; import androidx.media3.common.Format; import androidx.media3.common.MediaItem; +import androidx.media3.common.MimeTypes; +import androidx.media3.common.Timeline; import androidx.media3.common.audio.SpeedProvider; import androidx.media3.common.audio.ToInt16PcmAudioProcessor; import androidx.media3.common.util.Util; import androidx.media3.effect.GlEffect; import androidx.media3.effect.Presentation; import androidx.media3.effect.ScaleAndRotateTransformation; +import androidx.media3.exoplayer.source.MediaSource; +import androidx.media3.test.utils.FakeTimeline; +import androidx.media3.test.utils.FakeTimeline.TimelineWindowDefinition; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.collect.ImmutableList; +import com.google.common.collect.ImmutableSet; import org.junit.Rule; import org.junit.Test; import org.junit.rules.TemporaryFolder; @@ -103,6 +110,68 @@ public void shouldTranscodeVideo_regularRotationAndTranscodingPresentation_retur .isTrue(); } + @Test + public void shouldTranscodeVideo_dolbyVisionProfile20_returnsTrue() throws Exception { + MediaItem mediaItem = MediaItem.fromUri(ASSET_URI_PREFIX + FILE_AUDIO_VIDEO); + EditedMediaItem editedMediaItem = new EditedMediaItem.Builder(mediaItem).build(); + Composition composition = + new Composition.Builder(withAudioAndVideoFrom(ImmutableList.of(editedMediaItem))).build(); + MuxerWrapper muxerWrapper = + new MuxerWrapper( + temporaryFolder.newFile().getPath(), + new InAppMp4Muxer.Factory(), + new NoOpMuxerListenerImpl(), + MUXER_MODE_DEFAULT, + /* dropSamplesBeforeFirstVideoSample= */ false, + /* appendVideoFormat= */ null); + Format dolbyVisionProfile20Format = + FORMAT + .buildUpon() + .setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION) + .setCodecs("dvh1.20.01") + .build(); + + assertThat( + shouldTranscodeVideo( + dolbyVisionProfile20Format, + composition, + /* sequenceIndex= */ 0, + new TransformationRequest.Builder().build(), + new DefaultEncoderFactory.Builder(getApplicationContext()).build(), + muxerWrapper, + /* hasFrameProcessorFactory= */ false)) + .isTrue(); + } + + @Test + public void shouldTranscodeVideo_dolbyVisionWithNullCodecs_returnsTrue() throws Exception { + MediaItem mediaItem = MediaItem.fromUri(ASSET_URI_PREFIX + FILE_AUDIO_VIDEO); + EditedMediaItem editedMediaItem = new EditedMediaItem.Builder(mediaItem).build(); + Composition composition = + new Composition.Builder(withAudioAndVideoFrom(ImmutableList.of(editedMediaItem))).build(); + MuxerWrapper muxerWrapper = + new MuxerWrapper( + temporaryFolder.newFile().getPath(), + new InAppMp4Muxer.Factory(), + new NoOpMuxerListenerImpl(), + MUXER_MODE_DEFAULT, + /* dropSamplesBeforeFirstVideoSample= */ false, + /* appendVideoFormat= */ null); + Format dolbyVisionNullCodecsFormat = + FORMAT.buildUpon().setSampleMimeType(MimeTypes.VIDEO_DOLBY_VISION).setCodecs(null).build(); + + assertThat( + shouldTranscodeVideo( + dolbyVisionNullCodecsFormat, + composition, + /* sequenceIndex= */ 0, + new TransformationRequest.Builder().build(), + new DefaultEncoderFactory.Builder(getApplicationContext()).build(), + muxerWrapper, + /* hasFrameProcessorFactory= */ false)) + .isTrue(); + } + @Test public void shouldTranscodeVideo_irregularRotationAndPresentation_returnsTrue() throws Exception { MediaItem mediaItem = MediaItem.fromUri(ASSET_URI_PREFIX + FILE_AUDIO_VIDEO); @@ -225,6 +294,67 @@ public void shouldTranscodeVideo_withPacketProcessor_returnsTrue() throws Except .isTrue(); } + @Test + public void getEditedMediaItemIndex_nonLoopingSequence_returnsPeriodIndex() { + EditedMediaItem item1 = + new EditedMediaItem.Builder(MediaItem.fromUri(ASSET_URI_PREFIX + FILE_AUDIO_VIDEO)).build(); + EditedMediaItem item2 = + new EditedMediaItem.Builder(MediaItem.fromUri(ASSET_URI_PREFIX + FILE_AUDIO_VIDEO)).build(); + EditedMediaItemSequence sequence = withAudioFrom(ImmutableList.of(item1, item2)); + Timeline timeline = + new CompositionPlayer.CompositionForwardingTimeline( + new FakeTimeline(new TimelineWindowDefinition.Builder().setPeriodCount(2).build()), + sequence); + + assertThat( + TransformerUtil.getEditedMediaItemIndex( + timeline, new MediaSource.MediaPeriodId(timeline.getUidOfPeriod(0)))) + .isEqualTo(0); + assertThat( + TransformerUtil.getEditedMediaItemIndex( + timeline, new MediaSource.MediaPeriodId(timeline.getUidOfPeriod(1)))) + .isEqualTo(1); + } + + @Test + public void getEditedMediaItemIndex_loopingSequence_returnsModuloWrappedIndex() { + EditedMediaItem item1 = + new EditedMediaItem.Builder(MediaItem.fromUri(ASSET_URI_PREFIX + FILE_AUDIO_VIDEO)).build(); + EditedMediaItem item2 = + new EditedMediaItem.Builder(MediaItem.fromUri(ASSET_URI_PREFIX + FILE_AUDIO_VIDEO)).build(); + EditedMediaItemSequence sequence = + new EditedMediaItemSequence.Builder(ImmutableSet.of(C.TRACK_TYPE_AUDIO)) + .addItems(ImmutableList.of(item1, item2)) + .setIsLooping(true) + .build(); + Timeline timeline = + new CompositionPlayer.CompositionForwardingTimeline( + new FakeTimeline(new TimelineWindowDefinition.Builder().setPeriodCount(5).build()), + sequence); + + // Repetitions wrap: 0, 1, 0, 1, 0 + assertThat( + TransformerUtil.getEditedMediaItemIndex( + timeline, new MediaSource.MediaPeriodId(timeline.getUidOfPeriod(0)))) + .isEqualTo(0); + assertThat( + TransformerUtil.getEditedMediaItemIndex( + timeline, new MediaSource.MediaPeriodId(timeline.getUidOfPeriod(1)))) + .isEqualTo(1); + assertThat( + TransformerUtil.getEditedMediaItemIndex( + timeline, new MediaSource.MediaPeriodId(timeline.getUidOfPeriod(2)))) + .isEqualTo(0); + assertThat( + TransformerUtil.getEditedMediaItemIndex( + timeline, new MediaSource.MediaPeriodId(timeline.getUidOfPeriod(3)))) + .isEqualTo(1); + assertThat( + TransformerUtil.getEditedMediaItemIndex( + timeline, new MediaSource.MediaPeriodId(timeline.getUidOfPeriod(4)))) + .isEqualTo(0); + } + private static final class NoOpMuxerListenerImpl implements MuxerWrapper.Listener { @Override diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/TransformerWithInAppFragmentedMp4MuxerEndToEndTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/TransformerWithInAppFragmentedMp4MuxerEndToEndTest.java new file mode 100644 index 00000000000..948a57da4e4 --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/TransformerWithInAppFragmentedMp4MuxerEndToEndTest.java @@ -0,0 +1,124 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static androidx.media3.common.MimeTypes.isVideo; + +import android.content.Context; +import android.net.Uri; +import androidx.media3.common.Format; +import androidx.media3.common.MediaItem; +import androidx.media3.common.Metadata; +import androidx.media3.common.util.ConditionVariable; +import androidx.media3.muxer.BufferInfo; +import androidx.media3.muxer.Muxer; +import androidx.media3.muxer.MuxerException; +import androidx.media3.test.utils.TestTransformerBuilder; +import androidx.media3.test.utils.robolectric.RobolectricUtil; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import com.google.common.collect.ImmutableList; +import java.nio.ByteBuffer; +import org.junit.Before; +import org.junit.Rule; +import org.junit.Test; +import org.junit.rules.TemporaryFolder; +import org.junit.runner.RunWith; + +/** End-to-end test for {@link Transformer} with {@link InAppFragmentedMp4Muxer}. */ +@RunWith(AndroidJUnit4.class) +public final class TransformerWithInAppFragmentedMp4MuxerEndToEndTest { + private static final String MP4_FILE_PATH = "asset:///media/mp4/sample.mp4"; + + @Rule public final TemporaryFolder temporaryFolder = new TemporaryFolder(); + + private Context context; + private String outputPath; + + @Before + public void setUp() throws Exception { + context = ApplicationProvider.getApplicationContext(); + outputPath = temporaryFolder.newFile().getAbsolutePath(); + } + + @Test + public void cancel_immediatelyAfterVideoTrackAdded_doesNotCrash() throws Exception { + InAppFragmentedMp4Muxer.Factory inAppMuxerFactory = new InAppFragmentedMp4Muxer.Factory(); + inAppMuxerFactory.setVideoDurationUs(2_000_000L); // Triggers EOS write on close + ConditionVariable videoTrackAddedCondition = new ConditionVariable(); + TestMuxerFactory testMuxerFactory = + new TestMuxerFactory(inAppMuxerFactory, videoTrackAddedCondition); + Transformer transformer = + new TestTransformerBuilder(context).setMuxerFactory(testMuxerFactory).build(); + MediaItem mediaItem = MediaItem.fromUri(Uri.parse(MP4_FILE_PATH)); + + transformer.start(mediaItem, outputPath); + RobolectricUtil.runLooperUntil( + transformer.getApplicationLooper(), videoTrackAddedCondition::isOpen); + transformer.cancel(); + } + + private static final class TestMuxerFactory implements Muxer.Factory { + private final Muxer.Factory nestedFactory; + private final ConditionVariable videoTrackAddedCondition; + + TestMuxerFactory(Muxer.Factory nestedFactory, ConditionVariable videoTrackAddedCondition) { + this.nestedFactory = nestedFactory; + this.videoTrackAddedCondition = videoTrackAddedCondition; + } + + @Override + public Muxer create(String path) throws MuxerException { + Muxer realMuxer = nestedFactory.create(path); + return new Muxer() { + @Override + public int addTrack(Format format) throws MuxerException { + int trackId = realMuxer.addTrack(format); + if (isVideo(format.sampleMimeType)) { + videoTrackAddedCondition.open(); + } + return trackId; + } + + @Override + public void writeSampleData(int trackId, ByteBuffer byteBuffer, BufferInfo bufferInfo) + throws MuxerException { + realMuxer.writeSampleData(trackId, byteBuffer, bufferInfo); + } + + @Override + public void addMetadataEntry(Metadata.Entry metadataEntry) { + realMuxer.addMetadataEntry(metadataEntry); + } + + @Override + public void close() throws MuxerException { + realMuxer.close(); + } + }; + } + + @Override + public ImmutableList getSupportedSampleMimeTypes(int trackType) { + return nestedFactory.getSupportedSampleMimeTypes(trackType); + } + + @Override + public boolean supportsWritingNegativeTimestampsInEditList() { + return nestedFactory.supportsWritingNegativeTimestampsInEditList(); + } + } +} diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/TransformerWithInAppMp4MuxerEndToEndTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/TransformerWithInAppMp4MuxerEndToEndTest.java index 296db896070..328c30d7f24 100644 --- a/libraries/transformer/src/test/java/androidx/media3/transformer/TransformerWithInAppMp4MuxerEndToEndTest.java +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/TransformerWithInAppMp4MuxerEndToEndTest.java @@ -15,6 +15,7 @@ */ package androidx.media3.transformer; +import static androidx.media3.common.MimeTypes.isVideo; import static androidx.media3.test.utils.TestUtil.extractAllSamplesFromFilePath; import static androidx.media3.test.utils.TestUtil.retrieveTrackFormat; import static com.google.common.base.Preconditions.checkState; @@ -28,6 +29,8 @@ import androidx.media3.common.Format; import androidx.media3.common.MediaItem; import androidx.media3.common.Metadata; +import androidx.media3.common.MimeTypes; +import androidx.media3.common.util.ConditionVariable; import androidx.media3.common.util.Util; import androidx.media3.container.MdtaMetadataEntry; import androidx.media3.container.Mp4LocationData; @@ -35,15 +38,20 @@ import androidx.media3.container.XmpData; import androidx.media3.extractor.mp4.Mp4Extractor; import androidx.media3.extractor.text.DefaultSubtitleParserFactory; +import androidx.media3.muxer.BufferInfo; import androidx.media3.muxer.Muxer; +import androidx.media3.muxer.MuxerException; import androidx.media3.test.utils.DumpFileAsserts; import androidx.media3.test.utils.FakeExtractorOutput; import androidx.media3.test.utils.FakeTrackOutput; import androidx.media3.test.utils.TestTransformerBuilder; +import androidx.media3.test.utils.robolectric.RobolectricUtil; import androidx.test.core.app.ApplicationProvider; import androidx.test.ext.junit.runners.AndroidJUnit4; import com.google.common.base.Predicate; +import com.google.common.collect.ImmutableList; import java.io.File; +import java.nio.ByteBuffer; import java.util.concurrent.ExecutionException; import org.junit.Before; import org.junit.Ignore; @@ -125,6 +133,29 @@ public void transmux_withLocationMetadata_writesSameLocationMetadata() throws Ex assertThat(actualLocationData).isEqualTo(expectedLocationData); } + @Test + public void transmux_withLocationMetadataWithAltitude_writesSameLocationMetadataWithAltitude() + throws Exception { + Mp4LocationData expectedLocationData = + new Mp4LocationData(/* latitude= */ 45f, /* longitude= */ -90f, /* altitude= */ 120.5f); + Muxer.Factory inAppMuxerFactory = + new InAppMp4Muxer.Factory( + metadataEntries -> { + metadataEntries.removeIf((Metadata.Entry entry) -> entry instanceof Mp4LocationData); + metadataEntries.add(expectedLocationData); + }); + Transformer transformer = + new TestTransformerBuilder(context).setMuxerFactory(inAppMuxerFactory).build(); + MediaItem mediaItem = MediaItem.fromUri(Uri.parse(MP4_FILE_PATH)); + + transformer.start(mediaItem, outputPath); + TransformerTestRunner.runLooper(transformer); + + Mp4LocationData actualLocationData = + retrieveMetadata(context, outputPath, Mp4LocationData.class); + assertThat(actualLocationData).isEqualTo(expectedLocationData); + } + @Test public void transmux_withXmpData_completesSuccessfully() throws Exception { String xmpSampleData = "media/xmp/sample_datetime_xmp.xmp"; @@ -289,6 +320,43 @@ public void transmux_audioWithEditList_preservesDuration() throws Exception { .isEqualTo(1_555_736); } + @Test + public void cancel_immediatelyAfterVideoTrackAdded_doesNotCrash() throws Exception { + InAppMp4Muxer.Factory inAppMuxerFactory = new InAppMp4Muxer.Factory(); + inAppMuxerFactory.setVideoDurationUs(2_000_000L); // Triggers EOS write on close + + ConditionVariable videoTrackAddedCondition = new ConditionVariable(); + TestMuxerFactory testMuxerFactory = + new TestMuxerFactory(inAppMuxerFactory, videoTrackAddedCondition); + + Transformer transformer = + new TestTransformerBuilder(context).setMuxerFactory(testMuxerFactory).build(); + + MediaItem mediaItem = MediaItem.fromUri(Uri.parse(MP4_FILE_PATH)); + transformer.start(mediaItem, outputPath); + + RobolectricUtil.runLooperUntil( + transformer.getApplicationLooper(), videoTrackAddedCondition::isOpen); + transformer.cancel(); + } + + @Test + public void transmux_withEAc3Joc_completesSuccessfully() throws Exception { + String eac3JocFilePath = "asset:///media/mp4/sample_eac3joc.mp4"; + Transformer transformer = + new TestTransformerBuilder(context).setMuxerFactory(new InAppMp4Muxer.Factory()).build(); + MediaItem mediaItem = MediaItem.fromUri(Uri.parse(eac3JocFilePath)); + + transformer.start(mediaItem, outputPath); + TransformerTestRunner.runLooper(transformer); + + Format audioFormat = retrieveTrackFormat(context, outputPath, C.TRACK_TYPE_AUDIO); + assertThat(audioFormat.sampleMimeType).isEqualTo(MimeTypes.AUDIO_E_AC3_JOC); + assertThat(audioFormat.initializationData).hasSize(1); + assertThat(audioFormat.initializationData.get(0)) + .isEqualTo(new byte[] {0x14, 0x00, 0x20, 0x0F, 0x00, 0x01, 0x10}); + } + /** * Returns specific {@linkplain Metadata.Entry metadata} from the media file. * @@ -345,4 +413,56 @@ private static void ensureSameMetadataAcrossTracks( checkState(firstTrackMetadata.equals(secondTrackMetadata)); } } + + private static final class TestMuxerFactory implements Muxer.Factory { + private final InAppMp4Muxer.Factory nestedFactory; + private final ConditionVariable videoTrackAddedCondition; + + TestMuxerFactory( + InAppMp4Muxer.Factory nestedFactory, ConditionVariable videoTrackAddedCondition) { + this.nestedFactory = nestedFactory; + this.videoTrackAddedCondition = videoTrackAddedCondition; + } + + @Override + public Muxer create(String path) throws MuxerException { + Muxer realMuxer = nestedFactory.create(path); + return new Muxer() { + @Override + public int addTrack(Format format) throws MuxerException { + int trackId = realMuxer.addTrack(format); + if (isVideo(format.sampleMimeType)) { + videoTrackAddedCondition.open(); + } + return trackId; + } + + @Override + public void writeSampleData(int trackId, ByteBuffer byteBuffer, BufferInfo bufferInfo) + throws MuxerException { + realMuxer.writeSampleData(trackId, byteBuffer, bufferInfo); + } + + @Override + public void addMetadataEntry(Metadata.Entry metadataEntry) { + realMuxer.addMetadataEntry(metadataEntry); + } + + @Override + public void close() throws MuxerException { + realMuxer.close(); + } + }; + } + + @Override + public ImmutableList getSupportedSampleMimeTypes(int trackType) { + return nestedFactory.getSupportedSampleMimeTypes(trackType); + } + + @Override + public boolean supportsWritingNegativeTimestampsInEditList() { + return nestedFactory.supportsWritingNegativeTimestampsInEditList(); + } + } } diff --git a/libraries/transformer/src/test/java/androidx/media3/transformer/VideoFrameAggregationParametersTest.java b/libraries/transformer/src/test/java/androidx/media3/transformer/VideoFrameAggregationParametersTest.java new file mode 100644 index 00000000000..5051101c86d --- /dev/null +++ b/libraries/transformer/src/test/java/androidx/media3/transformer/VideoFrameAggregationParametersTest.java @@ -0,0 +1,102 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.transformer; + +import static com.google.common.truth.Truth.assertThat; +import static org.junit.Assert.assertThrows; + +import android.util.Rational; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Unit tests for {@link VideoFrameAggregationParameters}. */ +@RunWith(AndroidJUnit4.class) +public class VideoFrameAggregationParametersTest { + + @Test + public void builder_setFrameRate_setsFrameRate() { + VideoFrameAggregationParameters parameters = + new VideoFrameAggregationParameters.Builder().setFrameRate(new Rational(60, 1)).build(); + + assertThat(parameters.frameRate).isEqualTo(new Rational(60, 1)); + } + + @Test + public void builder_unsetFrameRate_setsToNull() { + VideoFrameAggregationParameters.Builder builder = new VideoFrameAggregationParameters.Builder(); + builder.setFrameRate(new Rational(60, 1)); + builder.setFrameRate(null); + + VideoFrameAggregationParameters parameters = builder.build(); + + assertThat(parameters.frameRate).isNull(); + } + + @Test + public void builder_setFrameRate_withZero_throwsIllegalArgumentException() { + VideoFrameAggregationParameters.Builder builder = new VideoFrameAggregationParameters.Builder(); + Rational zeroFrameRate = new Rational(0, 1); + + assertThrows(IllegalArgumentException.class, () -> builder.setFrameRate(zeroFrameRate)); + } + + @Test + public void builder_setFrameRate_withNegative_throwsIllegalArgumentException() { + VideoFrameAggregationParameters.Builder builder = new VideoFrameAggregationParameters.Builder(); + Rational negativeFrameRate = new Rational(-30, 1); + + assertThrows(IllegalArgumentException.class, () -> builder.setFrameRate(negativeFrameRate)); + } + + @Test + public void build_withoutSetters_returnsDefaultValues() { + VideoFrameAggregationParameters parameters = + new VideoFrameAggregationParameters.Builder().build(); + + assertThat(parameters).isEqualTo(VideoFrameAggregationParameters.DEFAULT); + assertThat(parameters.frameRate).isNull(); + } + + @Test + public void buildUpon_createsEqualInstance() { + VideoFrameAggregationParameters parameters = + new VideoFrameAggregationParameters.Builder().setFrameRate(new Rational(60, 1)).build(); + + assertThat(parameters.buildUpon().build()).isEqualTo(parameters); + } + + @Test + public void equals_identicalParameters_returnsTrue() { + VideoFrameAggregationParameters params1 = + new VideoFrameAggregationParameters.Builder().setFrameRate(new Rational(30, 1)).build(); + VideoFrameAggregationParameters params2 = + new VideoFrameAggregationParameters.Builder().setFrameRate(new Rational(30, 1)).build(); + + assertThat(params1).isEqualTo(params2); + assertThat(params1.hashCode()).isEqualTo(params2.hashCode()); + } + + @Test + public void equals_differentParameters_returnsFalse() { + VideoFrameAggregationParameters params1 = + new VideoFrameAggregationParameters.Builder().setFrameRate(new Rational(30, 1)).build(); + VideoFrameAggregationParameters params2 = + new VideoFrameAggregationParameters.Builder().setFrameRate(new Rational(60, 1)).build(); + + assertThat(params1).isNotEqualTo(params2); + } +} diff --git a/libraries/ui/src/main/java/androidx/media3/ui/DefaultTimeBar.java b/libraries/ui/src/main/java/androidx/media3/ui/DefaultTimeBar.java index ffde6e8a618..af2533ea668 100644 --- a/libraries/ui/src/main/java/androidx/media3/ui/DefaultTimeBar.java +++ b/libraries/ui/src/main/java/androidx/media3/ui/DefaultTimeBar.java @@ -41,9 +41,11 @@ import androidx.annotation.ColorInt; import androidx.annotation.Nullable; import androidx.annotation.RequiresApi; +import androidx.annotation.VisibleForTesting; import androidx.media3.common.C; import androidx.media3.common.util.UnstableApi; import androidx.media3.common.util.Util; +import java.util.Arrays; import java.util.Collections; import java.util.Formatter; import java.util.Locale; @@ -581,6 +583,16 @@ public void setAdGroupTimesMs( update(); } + @VisibleForTesting + /* package */ long[] getAdGroupTimesMs() { + return adGroupTimesMs == null ? new long[0] : Arrays.copyOf(adGroupTimesMs, adGroupCount); + } + + @VisibleForTesting + /* package */ boolean[] getPlayedAdGroups() { + return playedAdGroups == null ? new boolean[0] : Arrays.copyOf(playedAdGroups, adGroupCount); + } + // View methods. @Override diff --git a/libraries/ui/src/main/java/androidx/media3/ui/DefaultTrackNameProvider.java b/libraries/ui/src/main/java/androidx/media3/ui/DefaultTrackNameProvider.java index d77bd536f1e..8c6d0b44247 100644 --- a/libraries/ui/src/main/java/androidx/media3/ui/DefaultTrackNameProvider.java +++ b/libraries/ui/src/main/java/androidx/media3/ui/DefaultTrackNameProvider.java @@ -24,7 +24,6 @@ import androidx.media3.common.Format; import androidx.media3.common.MimeTypes; import androidx.media3.common.util.UnstableApi; -import androidx.media3.common.util.Util; import java.util.Locale; /** A default {@link TrackNameProvider}. */ @@ -117,7 +116,7 @@ private String buildLanguageString(Format format) { return ""; } Locale languageLocale = Locale.forLanguageTag(language); - Locale displayLocale = Util.getDefaultDisplayLocale(); + Locale displayLocale = Locale.getDefault(Locale.Category.DISPLAY); String languageName = languageLocale.getDisplayName(displayLocale); if (TextUtils.isEmpty(languageName)) { return ""; diff --git a/libraries/ui/src/main/java/androidx/media3/ui/LegacyPlayerControlView.java b/libraries/ui/src/main/java/androidx/media3/ui/LegacyPlayerControlView.java index 8138945a910..15f7c74be4e 100644 --- a/libraries/ui/src/main/java/androidx/media3/ui/LegacyPlayerControlView.java +++ b/libraries/ui/src/main/java/androidx/media3/ui/LegacyPlayerControlView.java @@ -993,6 +993,10 @@ private void updateTimeline() { int removedGroups = period.getRemovedAdGroupCount(); int totalGroups = period.getAdGroupCount(); for (int adGroupIndex = removedGroups; adGroupIndex < totalGroups; adGroupIndex++) { + if (period.isLivePostrollPlaceholder(adGroupIndex)) { + // Don't show ad markers for live stream placeholders. + continue; + } long adGroupTimeInPeriodUs = period.getAdGroupTimeUs(adGroupIndex); if (adGroupTimeInPeriodUs == C.TIME_END_OF_SOURCE) { if (period.durationUs == C.TIME_UNSET) { diff --git a/libraries/ui/src/main/java/androidx/media3/ui/PlayerControlView.java b/libraries/ui/src/main/java/androidx/media3/ui/PlayerControlView.java index f1efc67156d..5bc91962b29 100644 --- a/libraries/ui/src/main/java/androidx/media3/ui/PlayerControlView.java +++ b/libraries/ui/src/main/java/androidx/media3/ui/PlayerControlView.java @@ -1518,6 +1518,10 @@ private void updateTimeline() { int removedGroups = period.getRemovedAdGroupCount(); int totalGroups = period.getAdGroupCount(); for (int adGroupIndex = removedGroups; adGroupIndex < totalGroups; adGroupIndex++) { + if (period.isLivePostrollPlaceholder(adGroupIndex)) { + // Don't show ad markers for live stream placeholders. + continue; + } long adGroupTimeInPeriodUs = period.getAdGroupTimeUs(adGroupIndex); if (adGroupTimeInPeriodUs == C.TIME_END_OF_SOURCE) { if (period.durationUs == C.TIME_UNSET) { diff --git a/libraries/ui/src/test/java/androidx/media3/ui/LegacyPlayerControlViewTest.java b/libraries/ui/src/test/java/androidx/media3/ui/LegacyPlayerControlViewTest.java new file mode 100644 index 00000000000..b16de8bf398 --- /dev/null +++ b/libraries/ui/src/test/java/androidx/media3/ui/LegacyPlayerControlViewTest.java @@ -0,0 +1,105 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.ui; + +import static androidx.media3.test.utils.FakeMultiPeriodLiveTimeline.AD_PERIOD_DURATION_MS; +import static androidx.media3.test.utils.FakeMultiPeriodLiveTimeline.PERIOD_DURATION_MS; +import static com.google.common.truth.Truth.assertThat; + +import android.os.Looper; +import androidx.media3.common.Player; +import androidx.media3.common.SimpleBasePlayer; +import androidx.media3.common.Timeline; +import androidx.media3.common.Tracks; +import androidx.media3.test.utils.FakeMultiPeriodLiveTimeline; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Tests for {@link LegacyPlayerControlView}. */ +@RunWith(AndroidJUnit4.class) +public final class LegacyPlayerControlViewTest { + + @Test + public void + setPlayer_withMultiPeriodLiveTimelineWithAds_ignoresLivePostrollPlaceholdersInTimeBar() { + LegacyPlayerControlView playerControlView = + new LegacyPlayerControlView(ApplicationProvider.getApplicationContext()); + DefaultTimeBar timeBar = playerControlView.findViewById(R.id.exo_progress); + Timeline timeline = + new FakeMultiPeriodLiveTimeline( + /* availabilityStartTimeMs= */ 0L, + /* liveWindowDurationUs= */ 60_000_000L, + /* nowUs= */ 60_000_000L, + /* adSequencePattern= */ new boolean[] {false, true, true}, + /* periodDurationMsPattern= */ new long[] { + PERIOD_DURATION_MS, AD_PERIOD_DURATION_MS, AD_PERIOD_DURATION_MS + }, + /* isContentTimeline= */ false, + /* populateAds= */ true, + /* playedAds= */ false); + Player player = + new SimpleBasePlayer(Looper.getMainLooper()) { + @Override + protected State getState() { + return new State.Builder() + .setAvailableCommands(new Commands.Builder().add(COMMAND_GET_TIMELINE).build()) + .setPlaylist(timeline, Tracks.EMPTY, /* currentMetadata= */ null) + .build(); + } + }; + + playerControlView.setPlayer(player); + + assertThat(timeBar.getAdGroupTimesMs()).asList().containsExactly(30_000L, 40_000L).inOrder(); + assertThat(timeBar.getPlayedAdGroups()).asList().containsExactly(false, false).inOrder(); + } + + @Test + public void setPlayer_withMultiPeriodLiveTimelineWithoutAds_setsEmptyAdGroupTimesInTimeBar() { + LegacyPlayerControlView playerControlView = + new LegacyPlayerControlView(ApplicationProvider.getApplicationContext()); + DefaultTimeBar timeBar = playerControlView.findViewById(R.id.exo_progress); + Timeline timeline = + new FakeMultiPeriodLiveTimeline( + /* availabilityStartTimeMs= */ 0L, + /* liveWindowDurationUs= */ 60_000_000L, + /* nowUs= */ 60_000_000L, + /* adSequencePattern= */ new boolean[] {false, true, true}, + /* periodDurationMsPattern= */ new long[] { + PERIOD_DURATION_MS, AD_PERIOD_DURATION_MS, AD_PERIOD_DURATION_MS + }, + /* isContentTimeline= */ false, + /* populateAds= */ false, + /* playedAds= */ false); + Player player = + new SimpleBasePlayer(Looper.getMainLooper()) { + @Override + protected State getState() { + return new State.Builder() + .setAvailableCommands(new Commands.Builder().add(COMMAND_GET_TIMELINE).build()) + .setPlaylist(timeline, Tracks.EMPTY, /* currentMetadata= */ null) + .build(); + } + }; + + playerControlView.setPlayer(player); + + assertThat(timeBar.getAdGroupTimesMs()).isEmpty(); + assertThat(timeBar.getPlayedAdGroups()).isEmpty(); + } +} diff --git a/libraries/ui/src/test/java/androidx/media3/ui/PlayerControlViewTest.java b/libraries/ui/src/test/java/androidx/media3/ui/PlayerControlViewTest.java new file mode 100644 index 00000000000..1721e3630f8 --- /dev/null +++ b/libraries/ui/src/test/java/androidx/media3/ui/PlayerControlViewTest.java @@ -0,0 +1,105 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package androidx.media3.ui; + +import static androidx.media3.test.utils.FakeMultiPeriodLiveTimeline.AD_PERIOD_DURATION_MS; +import static androidx.media3.test.utils.FakeMultiPeriodLiveTimeline.PERIOD_DURATION_MS; +import static com.google.common.truth.Truth.assertThat; + +import android.os.Looper; +import androidx.media3.common.Player; +import androidx.media3.common.SimpleBasePlayer; +import androidx.media3.common.Timeline; +import androidx.media3.common.Tracks; +import androidx.media3.test.utils.FakeMultiPeriodLiveTimeline; +import androidx.test.core.app.ApplicationProvider; +import androidx.test.ext.junit.runners.AndroidJUnit4; +import org.junit.Test; +import org.junit.runner.RunWith; + +/** Tests for {@link PlayerControlView}. */ +@RunWith(AndroidJUnit4.class) +public final class PlayerControlViewTest { + + @Test + public void + setPlayer_withMultiPeriodLiveTimelineWithAds_ignoresLivePostrollPlaceholdersInTimeBar() { + PlayerControlView playerControlView = + new PlayerControlView(ApplicationProvider.getApplicationContext()); + DefaultTimeBar timeBar = playerControlView.findViewById(R.id.exo_progress); + Timeline timeline = + new FakeMultiPeriodLiveTimeline( + /* availabilityStartTimeMs= */ 0L, + /* liveWindowDurationUs= */ 60_000_000L, + /* nowUs= */ 60_000_000L, + /* adSequencePattern= */ new boolean[] {false, true, true}, + /* periodDurationMsPattern= */ new long[] { + PERIOD_DURATION_MS, AD_PERIOD_DURATION_MS, AD_PERIOD_DURATION_MS + }, + /* isContentTimeline= */ false, + /* populateAds= */ true, + /* playedAds= */ false); + Player player = + new SimpleBasePlayer(Looper.getMainLooper()) { + @Override + protected State getState() { + return new State.Builder() + .setAvailableCommands(new Commands.Builder().add(COMMAND_GET_TIMELINE).build()) + .setPlaylist(timeline, Tracks.EMPTY, /* currentMetadata= */ null) + .build(); + } + }; + + playerControlView.setPlayer(player); + + assertThat(timeBar.getAdGroupTimesMs()).asList().containsExactly(30_000L, 40_000L).inOrder(); + assertThat(timeBar.getPlayedAdGroups()).asList().containsExactly(false, false).inOrder(); + } + + @Test + public void setPlayer_withMultiPeriodLiveTimelineWithoutAds_setsEmptyAdGroupTimesInTimeBar() { + PlayerControlView playerControlView = + new PlayerControlView(ApplicationProvider.getApplicationContext()); + DefaultTimeBar timeBar = playerControlView.findViewById(R.id.exo_progress); + Timeline timeline = + new FakeMultiPeriodLiveTimeline( + /* availabilityStartTimeMs= */ 0L, + /* liveWindowDurationUs= */ 60_000_000L, + /* nowUs= */ 60_000_000L, + /* adSequencePattern= */ new boolean[] {false, true, true}, + /* periodDurationMsPattern= */ new long[] { + PERIOD_DURATION_MS, AD_PERIOD_DURATION_MS, AD_PERIOD_DURATION_MS + }, + /* isContentTimeline= */ false, + /* populateAds= */ false, + /* playedAds= */ false); + Player player = + new SimpleBasePlayer(Looper.getMainLooper()) { + @Override + protected State getState() { + return new State.Builder() + .setAvailableCommands(new Commands.Builder().add(COMMAND_GET_TIMELINE).build()) + .setPlaylist(timeline, Tracks.EMPTY, /* currentMetadata= */ null) + .build(); + } + }; + + playerControlView.setPlayer(player); + + assertThat(timeBar.getAdGroupTimesMs()).isEmpty(); + assertThat(timeBar.getPlayedAdGroups()).isEmpty(); + } +} diff --git a/libraries/ui_compose/build.gradle.kts b/libraries/ui_compose/build.gradle.kts index 2bca901c97c..1b7bc58153d 100644 --- a/libraries/ui_compose/build.gradle.kts +++ b/libraries/ui_compose/build.gradle.kts @@ -31,9 +31,13 @@ dependencies { api(platform(libs.androidx.compose.bom)) api(libs.androidx.compose.foundation) + implementation(libs.kotlinx.coroutines.guava) + testImplementation(libs.androidx.compose.ui.test) testImplementation(libs.androidx.compose.ui.test.junit4) testImplementation(project(":test-utils")) testImplementation(project(":test-utils-robolectric")) testImplementation(libs.robolectric) + testImplementation(libs.test.parameter.injector) + testImplementation(libs.coil.compose) } diff --git a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/Artwork.kt b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/Artwork.kt new file mode 100644 index 00000000000..3f743306866 --- /dev/null +++ b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/Artwork.kt @@ -0,0 +1,303 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.ui.compose + +import android.content.ContentResolver +import android.content.Context +import android.graphics.Bitmap +import android.graphics.BitmapFactory +import androidx.compose.foundation.Image +import androidx.compose.foundation.layout.Box +import androidx.compose.runtime.Composable +import androidx.compose.runtime.LaunchedEffect +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.rememberUpdatedState +import androidx.compose.runtime.setValue +import androidx.compose.ui.Alignment +import androidx.compose.ui.Modifier +import androidx.compose.ui.graphics.ColorFilter +import androidx.compose.ui.graphics.DefaultAlpha +import androidx.compose.ui.graphics.FilterQuality +import androidx.compose.ui.graphics.asImageBitmap +import androidx.compose.ui.graphics.drawscope.DrawScope.Companion.DefaultFilterQuality +import androidx.compose.ui.graphics.painter.BitmapPainter +import androidx.compose.ui.graphics.painter.Painter +import androidx.compose.ui.layout.ContentScale +import androidx.compose.ui.platform.LocalContext +import androidx.media3.common.MediaMetadata +import androidx.media3.common.Player +import androidx.media3.common.util.BitmapLoader +import androidx.media3.common.util.UnstableApi +import androidx.media3.ui.compose.state.CurrentMediaItemState +import androidx.media3.ui.compose.text.CurrentMediaItemBox +import kotlin.time.Duration.Companion.milliseconds +import kotlinx.coroutines.CoroutineDispatcher +import kotlinx.coroutines.Dispatchers +import kotlinx.coroutines.delay +import kotlinx.coroutines.guava.await +import kotlinx.coroutines.launch +import kotlinx.coroutines.withContext + +/** + * Displays the artwork of the current [MediaItem][androidx.media3.common.MediaItem] from the + * [player]. + * + * This overload is designed for custom image loading. It delegates the rendering of the artwork to + * the provided [painter] composable function, which has access to the [CurrentMediaItemState]. + * + * @param player The [Player] to get the current media item from. + * @param modifier The [Modifier] to be applied to this composable. + * @param contentDescription The content description for the artwork. Defaults to title or null. + * @param alignment Optional scale alignment within the container. + * @param contentScale Optional scale type for the artwork image. + * @param alpha Optional opacity to be applied to the artwork image. + * @param colorFilter Optional [ColorFilter] to apply for the artwork image. + * @param painter A composable function that returns a [Painter] to display the artwork. + */ +@UnstableApi +@Composable +fun Artwork( + player: Player?, + modifier: Modifier = Modifier, + contentDescription: String? = null, + alignment: Alignment = Alignment.Center, + contentScale: ContentScale = ContentScale.Fit, + alpha: Float = DefaultAlpha, + colorFilter: ColorFilter? = null, + painter: @Composable CurrentMediaItemState.() -> Painter?, +) { + CurrentMediaItemBox(player) { + painter()?.let { + Image( + painter = it, + contentDescription = contentDescription ?: mediaMetadata.title?.toString(), + modifier = modifier, + alignment = alignment, + contentScale = contentScale, + alpha = alpha, + colorFilter = colorFilter, + ) + } ?: Box(modifier) + } +} + +/** + * Displays the artwork of the current [MediaItem][androidx.media3.common.MediaItem] from the + * [player]. + * + * This overload is designed for displaying artwork using a built-in or custom [BitmapLoader]. It + * automatically tracks the player's current media item. If [bitmapLoader] is not provided, it falls + * back to a default implementation that only supports reading local URIs and byte arrays. + * + * @param player The [Player] to get the current media item from. + * @param modifier The [Modifier] to be applied to this composable. + * @param contentDescription The content description for the artwork. Defaults to title or null. + * @param alignment Optional scale alignment within the container. + * @param contentScale Optional scale type for the artwork image. + * @param alpha Optional opacity to be applied to the artwork image. + * @param colorFilter Optional [ColorFilter] to apply for the artwork image. + * @param filterQuality Optional [FilterQuality] to apply to the artwork image bitmap. + * @param bitmapLoader Optional custom artwork loader. If `null`, defaults to reading artwork data + * or URI from [MediaMetadata] using internal BitmapFactory. + * @param placeholder Optional [Painter] to display while the artwork is loading (after the 1-second + * delay). + * @param error Optional [Painter] to display if the artwork loading throws an exception. + * @param fallback Optional [Painter] to display if the media item has no artwork. Defaults to + * [error]. + * @param coroutineDispatcher Optional [CoroutineDispatcher] to execute background artwork loading. + * Defaults to [Dispatchers.IO]. + */ +@UnstableApi +@Composable +fun Artwork( + player: Player?, + modifier: Modifier = Modifier, + contentDescription: String? = null, + alignment: Alignment = Alignment.Center, + contentScale: ContentScale = ContentScale.Fit, + alpha: Float = DefaultAlpha, + colorFilter: ColorFilter? = null, + filterQuality: FilterQuality = DefaultFilterQuality, + bitmapLoader: BitmapLoader? = null, + placeholder: Painter? = null, + error: Painter? = null, + fallback: Painter? = error, + coroutineDispatcher: CoroutineDispatcher = Dispatchers.IO, +) { + CurrentMediaItemBox(player) { + Artwork( + metadata = mediaMetadata, + modifier = modifier, + contentDescription = contentDescription ?: mediaMetadata.title?.toString(), + alignment = alignment, + contentScale = contentScale, + alpha = alpha, + colorFilter = colorFilter, + filterQuality = filterQuality, + bitmapLoader = bitmapLoader, + placeholder = placeholder, + error = error, + fallback = fallback, + coroutineDispatcher = coroutineDispatcher, + ) + } +} + +/** + * Displays artwork for the given [MediaMetadata]. + * + * This overload is designed for displaying artwork from a specific [MediaMetadata] instance, + * independent of the player's current item (e.g. for displaying artwork in a playlist). If + * [bitmapLoader] is not provided, it falls back to a default implementation that only supports + * reading local URIs and byte arrays. + * + * @param metadata The [MediaMetadata] containing artwork info. + * @param modifier The [Modifier] to be applied to this composable. + * @param contentDescription The content description for the artwork. Defaults to metadata title. + * @param alignment Optional scale alignment within the container. + * @param contentScale Optional scale type for the artwork image. + * @param alpha Optional opacity to be applied to the artwork image. + * @param colorFilter Optional [ColorFilter] to apply for the artwork image. + * @param filterQuality Optional [FilterQuality] to apply to the artwork image bitmap. + * @param bitmapLoader Optional custom artwork loader. If `null`, defaults to reading artwork data + * or URI from [MediaMetadata] using internal BitmapFactory. + * @param placeholder Optional [Painter] to display while the artwork is loading (after the 1-second + * delay). + * @param error Optional [Painter] to display if the artwork loading throws an exception. + * @param fallback Optional [Painter] to display if the media item has no artwork. Defaults to + * [error]. + * @param coroutineDispatcher Optional [CoroutineDispatcher] to execute background artwork loading. + * Defaults to [Dispatchers.IO]. + */ +@UnstableApi +@Composable +fun Artwork( + metadata: MediaMetadata?, + modifier: Modifier = Modifier, + contentDescription: String? = metadata?.title?.toString(), + alignment: Alignment = Alignment.Center, + contentScale: ContentScale = ContentScale.Fit, + alpha: Float = DefaultAlpha, + colorFilter: ColorFilter? = null, + filterQuality: FilterQuality = DefaultFilterQuality, + bitmapLoader: BitmapLoader? = null, + placeholder: Painter? = null, + error: Painter? = null, + fallback: Painter? = error, + coroutineDispatcher: CoroutineDispatcher = Dispatchers.IO, +) { + val context = LocalContext.current + var bitmapState by remember { mutableStateOf(null) } + var artworkState by remember { mutableStateOf(ArtworkState.LOADING) } + val currentBitmapLoader by rememberUpdatedState(bitmapLoader) + + LaunchedEffect(metadata, currentBitmapLoader, coroutineDispatcher) { + if (metadata == null) { + bitmapState = null + artworkState = ArtworkState.FALLBACK + return@LaunchedEffect + } + + val clearJob = launch { + delay(1000.milliseconds) + bitmapState = null + artworkState = ArtworkState.LOADING + } + + val result = runCatching { + currentBitmapLoader?.loadBitmapFromMetadata(metadata)?.await() + ?: defaultLoadArtwork(context, metadata, coroutineDispatcher) + } + + clearJob.cancel() + if (result.isSuccess) { + val loadedBitmap = result.getOrNull() + if (loadedBitmap != null) { + bitmapState = loadedBitmap + artworkState = ArtworkState.SUCCESS + } else { + bitmapState = null + artworkState = ArtworkState.FALLBACK + } + } else { + bitmapState = null + artworkState = ArtworkState.ERROR + } + } + + val painter = + when (artworkState) { + ArtworkState.SUCCESS -> + bitmapState?.asImageBitmap()?.let { + BitmapPainter(image = it, filterQuality = filterQuality) + } + ArtworkState.LOADING -> placeholder + ArtworkState.ERROR -> error + ArtworkState.FALLBACK -> fallback + } + + painter?.let { + Image( + painter = it, + contentDescription = contentDescription, + modifier = modifier, + alignment = alignment, + contentScale = contentScale, + alpha = alpha, + colorFilter = colorFilter, + ) + } ?: Box(modifier) +} + +private enum class ArtworkState { + LOADING, + SUCCESS, + ERROR, + FALLBACK, +} + +@SuppressWarnings("UNSAFE_URI") +private suspend fun defaultLoadArtwork( + context: Context, + metadata: MediaMetadata, + coroutineDispatcher: CoroutineDispatcher = Dispatchers.IO, +): Bitmap? = + withContext(coroutineDispatcher) { + runCatching { + val data = metadata.artworkData + if (data != null) { + return@runCatching BitmapFactory.decodeByteArray(data, 0, data.size) + } + + val uri = metadata.artworkUri ?: return@runCatching null + val scheme = uri.scheme?.lowercase() + if ( + scheme != ContentResolver.SCHEME_CONTENT && + scheme != ContentResolver.SCHEME_ANDROID_RESOURCE && + scheme != ContentResolver.SCHEME_FILE + ) { + return@runCatching null + } + + context.contentResolver.openInputStream(uri)?.use { inputStream -> + BitmapFactory.decodeStream(inputStream) + } + } + .getOrNull() + } diff --git a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/ContentFrame.kt b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/ContentFrame.kt index ab9545d411d..a7baedff626 100644 --- a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/ContentFrame.kt +++ b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/ContentFrame.kt @@ -43,6 +43,8 @@ import androidx.media3.ui.compose.state.rememberPresentationState * @param contentScale The [ContentScale] strategy for the container. * @param keepContentOnReset If `true`, the last rendered frame will remain visible when the player * is reset. If `false`, the surface will be cleared. + * @param artwork Optional composable slot to render artwork for the current media item. + * @param overlay A composable drawn on top of the media content, but under the shutter. * @param shutter A composable that is displayed when the video surface needs to be covered. By * default, this is a black background. */ @@ -54,17 +56,32 @@ fun ContentFrame( surfaceType: @SurfaceType Int = SURFACE_TYPE_SURFACE_VIEW, contentScale: ContentScale = ContentScale.Fit, keepContentOnReset: Boolean = false, + artwork: (@Composable (Player?) -> Unit)? = { + Artwork(it, modifier = Modifier.fillMaxSize(), contentScale = contentScale) + }, + overlay: @Composable () -> Unit = {}, shutter: @Composable () -> Unit = { Box(Modifier.fillMaxSize().background(Color.Black)) }, ) { val presentationState: PresentationState = rememberPresentationState(player, keepContentOnReset) - val scaledModifier = modifier.resizeWithContentScale(contentScale, presentationState.videoSizeDp) + val scaledModifier = + Modifier.resizeWithContentScale(contentScale, presentationState.videoAspectRatio) - // Always leave PlayerSurface to be part of the Compose tree because it will be initialised in - // the process. If this composable is guarded by some condition, it might never become visible - // because the Player will not emit the relevant event, e.g. the first frame being ready. - PlayerSurface(player, scaledModifier, surfaceType) + Box(modifier) { + Box(scaledModifier) { + // Always leave PlayerSurface to be part of the Compose tree because it will be initialised in + // the process. If this composable is guarded by some condition, it might never become visible + // because the Player will not emit the relevant event, e.g. the first frame being ready. + PlayerSurface(player, Modifier.fillMaxSize(), surfaceType) - if (presentationState.coverSurface) { - shutter() + overlay() + } + + if (presentationState.coverSurface) { + shutter() + } + + if (presentationState.showArtwork) { + artwork?.invoke(player) + } } } diff --git a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/buttons/PlayPauseButton.kt b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/buttons/PlayPauseButton.kt index a0eabeb7718..60d11fc3d02 100644 --- a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/buttons/PlayPauseButton.kt +++ b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/buttons/PlayPauseButton.kt @@ -19,7 +19,11 @@ package androidx.media3.ui.compose.buttons import androidx.compose.runtime.Composable import androidx.media3.common.Player import androidx.media3.common.util.UnstableApi +import androidx.media3.ui.compose.state.DisplayMode import androidx.media3.ui.compose.state.PlayPauseButtonState +import androidx.media3.ui.compose.state.SHOW_BUFFERING_ALWAYS +import androidx.media3.ui.compose.state.SHOW_REPLAY_ON_ENDED +import androidx.media3.ui.compose.state.SHOW_RETRY_ON_ERROR import androidx.media3.ui.compose.state.rememberPlayPauseButtonState /** @@ -29,10 +33,18 @@ import androidx.media3.ui.compose.state.rememberPlayPauseButtonState * provided by the [content] lambda, which has access to the [PlayPauseButtonState]. * * @param player The [Player] to control. + * @param displayMode The [DisplayMode] determining under which conditions buffering, replay, or + * retry indicators should be shown. Defaults to [SHOW_BUFFERING_ALWAYS] combined with + * [SHOW_REPLAY_ON_ENDED] and [SHOW_RETRY_ON_ERROR]. * @param content The composable content to be displayed for the button. */ @UnstableApi @Composable -fun PlayPauseButton(player: Player?, content: @Composable PlayPauseButtonState.() -> Unit) { - rememberPlayPauseButtonState(player).content() +fun PlayPauseButton( + player: Player?, + displayMode: @DisplayMode Int = + SHOW_BUFFERING_ALWAYS or SHOW_REPLAY_ON_ENDED or SHOW_RETRY_ON_ERROR, + content: @Composable PlayPauseButtonState.() -> Unit, +) { + rememberPlayPauseButtonState(player, displayMode).content() } diff --git a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/modifiers/extensions.kt b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/modifiers/extensions.kt index 2d1a5eb6d03..a6713918e31 100644 --- a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/modifiers/extensions.kt +++ b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/modifiers/extensions.kt @@ -24,6 +24,7 @@ import androidx.compose.ui.geometry.Size import androidx.compose.ui.layout.ContentScale import androidx.compose.ui.layout.layout import androidx.compose.ui.platform.LocalDensity +import androidx.compose.ui.unit.Constraints import androidx.compose.ui.unit.Density import androidx.compose.ui.unit.Dp import androidx.media3.common.util.UnstableApi @@ -32,33 +33,147 @@ import kotlin.math.roundToInt /** * Attempts to size the original content rectangle to be inscribed into a destination by applying a * specified [ContentScale] type. + * + * @param contentScale The [ContentScale] to apply to the content. + * @param aspectRatio The aspect ratio of the content (width / height). If null, no scaling is + * applied. + */ +@UnstableApi +fun Modifier.resizeWithContentScale(contentScale: ContentScale, aspectRatio: Float?): Modifier { + return if (aspectRatio == null) { + this.then(Modifier.fillMaxSize().wrapContentSize()) + } else { + this.resizeWithContentScaleCommon(contentScale, sourceSizePx = null, aspectRatio = aspectRatio) + } +} + +/** + * Attempts to size the original content rectangle to be inscribed into a destination by applying a + * specified [ContentScale] type. + * + * @param contentScale The [ContentScale] to apply to the content. + * @param sourceSizeDp The size of the content in DP. + * @param density The [Density] to use for conversions. + * @deprecated Use the overload that takes aspectRatio instead. */ @UnstableApi +@Deprecated( + "Use the overload that takes aspectRatio instead.", + ReplaceWith( + "this.resizeWithContentScale(contentScale, sourceSizeDp?.run { if (width > 0 && height > 0) width / height else null })" + ), +) @Composable fun Modifier.resizeWithContentScale( contentScale: ContentScale, sourceSizeDp: Size?, density: Density = LocalDensity.current, ): Modifier = - this.then( + if (sourceSizeDp == null) { + this.then(Modifier.fillMaxSize().wrapContentSize()) + } else if ( + !sourceSizeDp.width.isFinite() || !sourceSizeDp.height.isFinite() || sourceSizeDp.isEmpty() + ) { + // Grouping !isFinite() with <= 0f ensures that all invalid/unspecified sizes (Zero, NaN, + // Infinity) behave consistently by falling back to minimum constraints, rather than having Zero + // hide the layout and NaN/null expand it. + this.then( + Modifier.layout { measurable, constraints -> + val placeable = + measurable.measure(Constraints.fixed(constraints.minWidth, constraints.minHeight)) + layout(placeable.width, placeable.height) { placeable.place(0, 0) } + } + ) + } else { + val sourceSizePx = + with(density) { Size(Dp(sourceSizeDp.width).toPx(), Dp(sourceSizeDp.height).toPx()) } + val aspectRatio = sourceSizePx.width / sourceSizePx.height + this.resizeWithContentScaleCommon( + contentScale, + sourceSizePx = sourceSizePx, + aspectRatio = aspectRatio, + ) + } + +private fun Modifier.resizeWithContentScaleCommon( + contentScale: ContentScale, + sourceSizePx: Size?, + aspectRatio: Float, +): Modifier { + require(aspectRatio > 0 && aspectRatio.isFinite()) { + "aspectRatio must be positive and finite: $aspectRatio" + } + return this.then( Modifier.fillMaxSize() .wrapContentSize() .then( - sourceSizeDp?.let { srcSizeDp -> - Modifier.layout { measurable, constraints -> - val srcSizePx = - with(density) { Size(Dp(srcSizeDp.width).toPx(), Dp(srcSizeDp.height).toPx()) } - val dstSizePx = Size(constraints.maxWidth.toFloat(), constraints.maxHeight.toFloat()) - val scaleFactor = contentScale.computeScaleFactor(srcSizePx, dstSizePx) - val placeable = + Modifier.layout { measurable, constraints -> + val placeable = + if (constraints.hasBoundedWidth || constraints.hasBoundedHeight) { + val dstWidth = + if (constraints.hasBoundedWidth) { + constraints.maxWidth.toFloat() + } else { + constraints.maxHeight * aspectRatio + } + val dstHeight = + if (constraints.hasBoundedHeight) { + constraints.maxHeight.toFloat() + } else { + constraints.maxWidth / aspectRatio + } + val dstSizePx = Size(dstWidth, dstHeight) + + val syntheticallyScaledSrcSize = + if (constraints.hasBoundedWidth && constraints.hasBoundedHeight) { + if (dstHeight * aspectRatio < dstWidth) { + Size(dstHeight * aspectRatio, dstHeight) + } else { + Size(dstWidth, dstWidth / aspectRatio) + } + } else { + dstSizePx + } + val srcSizeForScaling = sourceSizePx ?: syntheticallyScaledSrcSize + + val scaleFactor = contentScale.computeScaleFactor(srcSizeForScaling, dstSizePx) measurable.measure( constraints.copy( - maxWidth = (srcSizePx.width * scaleFactor.scaleX).roundToInt(), - maxHeight = (srcSizePx.height * scaleFactor.scaleY).roundToInt(), + maxWidth = (srcSizeForScaling.width * scaleFactor.scaleX).roundToInt(), + maxHeight = (srcSizeForScaling.height * scaleFactor.scaleY).roundToInt(), ) ) - layout(placeable.width, placeable.height) { placeable.place(0, 0) } + } else { + // constraints width & height are both unbounded + if (sourceSizePx != null) { + // Use source size as a fallback for bounds + measurable.measure( + constraints.copy( + maxWidth = sourceSizePx.width.roundToInt(), + maxHeight = sourceSizePx.height.roundToInt(), + ) + ) + } else { + // no external or specified bounds, measure with original constraints + val measuredPlaceable = measurable.measure(constraints) + if (measuredPlaceable.width == 0 || measuredPlaceable.height == 0) { + throw IllegalArgumentException( + "resizeWithContentScale measured to 0x0 under unbounded constraints. " + + "Ensure the parent layout has a bounded dimension or the child has a non-zero size." + ) + } + measuredPlaceable + } + } + + val layoutWidth = placeable.width.coerceIn(constraints.minWidth, constraints.maxWidth) + val layoutHeight = placeable.height.coerceIn(constraints.minHeight, constraints.maxHeight) + layout(layoutWidth, layoutHeight) { + val x = (layoutWidth - placeable.width) / 2 + val y = (layoutHeight - placeable.height) / 2 + placeable.place(x, y) } - } ?: Modifier + } ) ) +} diff --git a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/CuesState.kt b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/CuesState.kt new file mode 100644 index 00000000000..f36b1310ce4 --- /dev/null +++ b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/CuesState.kt @@ -0,0 +1,66 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.ui.compose.state + +import androidx.compose.runtime.Composable +import androidx.compose.runtime.LaunchedEffect +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.setValue +import androidx.media3.common.Player +import androidx.media3.common.text.Cue +import androidx.media3.common.util.UnstableApi + +/** + * Remembers the value of [CuesState] created based on the passed [Player] and launches a coroutine + * to listen to [Player's][Player] cue changes. If the [Player] instance changes between + * compositions, produces and remembers a new value. + */ +@UnstableApi +@Composable +fun rememberCuesState(player: Player?): CuesState { + val cuesState = remember(player) { CuesState(player) } + LaunchedEffect(player) { cuesState.observe() } + return cuesState +} + +/** + * State holder that converts [Player.Listener.onCues] events into a State of [List] and tracks + * the availability of text commands. + * + * @param player The [Player] whose cues are observed. + * @property cues The current list of cues to be displayed. Defaults to an empty list and is also + * empty when [Player.COMMAND_GET_TEXT] is not available. + */ +@UnstableApi +class CuesState(private val player: Player?) { + var cues: List by mutableStateOf(emptyList()) + private set + + private val playerStateObserver: PlayerStateObserver? = + player?.observeState(Player.EVENT_CUES, Player.EVENT_AVAILABLE_COMMANDS_CHANGED) { + val isEnabled = player.isCommandAvailable(Player.COMMAND_GET_TEXT) + cues = if (isEnabled) player.currentCues.cues else emptyList() + } + + /** + * Subscribes to updates from [Player.Events] and listens to [Player.EVENT_CUES] and + * [Player.EVENT_AVAILABLE_COMMANDS_CHANGED] to update the [cues] state. + */ + suspend fun observe() = playerStateObserver?.observe() +} diff --git a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/PlayPauseButtonState.kt b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/PlayPauseButtonState.kt index 9dff49f8630..9f449ae4582 100644 --- a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/PlayPauseButtonState.kt +++ b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/PlayPauseButtonState.kt @@ -16,6 +16,7 @@ package androidx.media3.ui.compose.state +import androidx.annotation.IntDef import androidx.compose.runtime.Composable import androidx.compose.runtime.LaunchedEffect import androidx.compose.runtime.getValue @@ -32,11 +33,21 @@ import androidx.media3.common.util.Util.shouldShowPlayButton * Remembers the value of [PlayPauseButtonState] created based on the passed [Player] and launch a * coroutine to listen to [Player's][Player] changes. If the [Player] instance changes between * compositions, produce and remember a new value. + * + * @param player The [Player] to observe. + * @param displayMode The [@DisplayMode][DisplayMode] specifying when to show buffering, replay, or + * retry. */ @UnstableApi @Composable -fun rememberPlayPauseButtonState(player: Player?): PlayPauseButtonState { - val playPauseButtonState = remember(player) { PlayPauseButtonState(player) } +fun rememberPlayPauseButtonState( + player: Player?, + displayMode: @DisplayMode Int = + SHOW_BUFFERING_ALWAYS or SHOW_REPLAY_ON_ENDED or SHOW_RETRY_ON_ERROR, +): PlayPauseButtonState { + val playPauseButtonState = + remember(player) { PlayPauseButtonState(player, displayMode) } + .apply { this.displayMode = displayMode } LaunchedEffect(player) { playPauseButtonState.observe() } return playPauseButtonState } @@ -48,24 +59,67 @@ fun rememberPlayPauseButtonState(player: Player?): PlayPauseButtonState { * @property[isEnabled] true if [player] is not `null`, [Player.COMMAND_PLAY_PAUSE] is available and * we have something in the [Timeline][androidx.media3.common.Timeline] to play. See * [shouldEnablePlayPauseButton] for more details. - * @property[showPlay] true if [player] is `null` or [shouldShowPlayButton] is true. + * @property[showPlay] true if [player] is `null` or [shouldShowPlayButton] is true. Note that + * [showReplay] and [showRetry] are further specifications of this state, meaning if either is + * true, [showPlay] will also be true. Callers may prefer to prioritize showing retry and replay + * indicators over a generic play indicator. + * @property[showReplay] true if the replay indicator should be shown based on the [displayMode]. + * @property[showRetry] true if the retry indicator should be shown based on the [displayMode]. + * @param displayMode The [@DisplayMode][DisplayMode] specifying when to show buffering, replay, or + * retry. */ @UnstableApi -class PlayPauseButtonState(private val player: Player?) { +class PlayPauseButtonState( + private val player: Player?, + displayMode: @DisplayMode Int = + SHOW_BUFFERING_ALWAYS or SHOW_REPLAY_ON_ENDED or SHOW_RETRY_ON_ERROR, +) { + + internal var displayMode: @DisplayMode Int = displayMode + set(value) { + if (field != value) { + field = value + updateDisplayModeState() + } + } + var isEnabled by mutableStateOf(false) private set var showPlay by mutableStateOf(true) private set + var showBuffering by mutableStateOf(false) + private set + + var showReplay by mutableStateOf(false) + private set + + var showRetry by mutableStateOf(false) + private set + + private fun updateDisplayModeState() { + val isPlayerBuffering = player?.playbackState == Player.STATE_BUFFERING + val hasBufferingIntent = + ((displayMode and SHOW_BUFFERING_ALWAYS) != 0) || + (((displayMode and SHOW_BUFFERING_WHEN_PLAYING) != 0) && !showPlay) + showBuffering = isPlayerBuffering && hasBufferingIntent + val isEnded = player?.playbackState == Player.STATE_ENDED + val hasError = player?.playerError != null + showReplay = isEnded && ((displayMode and SHOW_REPLAY_ON_ENDED) != 0) + showRetry = hasError && ((displayMode and SHOW_RETRY_ON_ERROR) != 0) + } + private val playerStateObserver: PlayerStateObserver? = player?.observeState( Player.EVENT_PLAYBACK_STATE_CHANGED, Player.EVENT_PLAY_WHEN_READY_CHANGED, Player.EVENT_AVAILABLE_COMMANDS_CHANGED, + Player.EVENT_PLAYER_ERROR, ) { isEnabled = shouldEnablePlayPauseButton(player) showPlay = shouldShowPlayButton(player) + updateDisplayModeState() } /** @@ -99,3 +153,25 @@ class PlayPauseButtonState(private val player: Player?) { playerStateObserver?.observe() } } + +@UnstableApi +@Retention(AnnotationRetention.SOURCE) +@Target(AnnotationTarget.TYPE, AnnotationTarget.VALUE_PARAMETER, AnnotationTarget.PROPERTY) +@IntDef( + flag = true, + value = + [SHOW_BUFFERING_WHEN_PLAYING, SHOW_BUFFERING_ALWAYS, SHOW_REPLAY_ON_ENDED, SHOW_RETRY_ON_ERROR], +) +annotation class DisplayMode + +/** + * The buffering view is shown when the player is in the buffering state and shouldShowPlayButton is + * false. + */ +@UnstableApi const val SHOW_BUFFERING_WHEN_PLAYING = 1 shl 0 +/** The buffering view is always shown when the player is in the buffering state. */ +@UnstableApi const val SHOW_BUFFERING_ALWAYS = 1 shl 1 +/** The replay icon is shown when the player is in the ended state. */ +@UnstableApi const val SHOW_REPLAY_ON_ENDED = 1 shl 2 +/** The retry icon is shown when the player has encountered an error. */ +@UnstableApi const val SHOW_RETRY_ON_ERROR = 1 shl 3 diff --git a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/PlaybackState.kt b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/PlaybackState.kt new file mode 100644 index 00000000000..af75c33cf20 --- /dev/null +++ b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/PlaybackState.kt @@ -0,0 +1,96 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.ui.compose.state + +import androidx.compose.runtime.Composable +import androidx.compose.runtime.LaunchedEffect +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableIntStateOf +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.setValue +import androidx.media3.common.Player +import androidx.media3.common.util.UnstableApi + +/** Creates and remembers a [PlaybackState] for the given [Player]. */ +@UnstableApi +@Composable +fun rememberPlaybackState(player: Player?): PlaybackState { + val playbackState = remember(player) { PlaybackState(player) } + LaunchedEffect(player) { playbackState.observe() } + return playbackState +} + +/** + * A general-purpose, read-only state holder that observes and exposes the core playback states of a + * [Player] reactively for Compose UIs. + * + * This state is decoupled from any specific UI component and is ideal for driving overlays, + * shimmers, or custom control layouts. + * + * @property[playbackState] The raw [@Player.State] of the player. + * @property[playWhenReady] Whether the player should play when ready. + * @property[playbackSuppressionReason] The reason why playback is suppressed even if + * [playWhenReady] is true, or [Player.PLAYBACK_SUPPRESSION_REASON_NONE] if playback is not + * suppressed. + * @property[isPlaying] Whether the player is actively playing. + * @property[playerError] The last error that caused playback to fail, or null if there is no error. + */ +@UnstableApi +class PlaybackState(private val player: Player?) { + var playbackState by mutableIntStateOf(player?.playbackState ?: Player.STATE_IDLE) + private set + + var playWhenReady by mutableStateOf(player?.playWhenReady ?: false) + private set + + var playbackSuppressionReason by + mutableIntStateOf(player?.playbackSuppressionReason ?: Player.PLAYBACK_SUPPRESSION_REASON_NONE) + private set + + var isPlaying by mutableStateOf(player?.isPlaying ?: false) + private set + + var playerError by mutableStateOf(player?.playerError) + private set + + private val playerStateObserver: PlayerStateObserver? = + player?.observeState( + Player.EVENT_PLAYBACK_STATE_CHANGED, + Player.EVENT_PLAY_WHEN_READY_CHANGED, + Player.EVENT_PLAYBACK_SUPPRESSION_REASON_CHANGED, + Player.EVENT_IS_PLAYING_CHANGED, + Player.EVENT_PLAYER_ERROR, + ) { + playbackState = player.playbackState + playWhenReady = player.playWhenReady + playbackSuppressionReason = player.playbackSuppressionReason + isPlaying = player.isPlaying + playerError = player.playerError + } + + /** + * Subscribes to updates from [Player.Events] and listens to: + * * [Player.EVENT_PLAYBACK_STATE_CHANGED], [Player.EVENT_PLAY_WHEN_READY_CHANGED] and + * [Player.EVENT_PLAYBACK_SUPPRESSION_REASON_CHANGED] in order to determine whether the player + * is buffering, has ended, or is playing. + * * [Player.EVENT_IS_PLAYING_CHANGED] in order to determine whether the player is actively + * playing. + * * [Player.EVENT_PLAYER_ERROR] in order to determine whether a playback error occurred. + */ + suspend fun observe() = playerStateObserver?.observe() +} diff --git a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/PlayerStateObserver.kt b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/PlayerStateObserver.kt index 977246a3aba..7013c57562c 100644 --- a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/PlayerStateObserver.kt +++ b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/PlayerStateObserver.kt @@ -15,6 +15,7 @@ */ package androidx.media3.ui.compose.state +import androidx.media3.common.FlagSet import androidx.media3.common.MediaLibraryInfo import androidx.media3.common.Player import androidx.media3.common.listenTo @@ -23,33 +24,100 @@ import androidx.media3.common.util.UnstableApi /** * Utility to observe [Player] states by listening to events. * - * @param player The [Player] - * @param firstEvent The first [Player.Event] to listen to - * @param otherEvents Additional [Player.Event] types to listen to - * @param stateUpdater The operation to trigger initially and whenever one of the configured events - * happen + * @param player The [Player]. + * @param firstEvent The first [Player.Event] to listen to. + * @param otherEvents Additional [Player.Event] types to listen to. + * @param initialStateUpdater The operation to trigger initially on creation and when [observe] + * starts. + * @param stateUpdater The operation to trigger whenever one of the configured events happen. The + * provided [Player.Events] contains only the subset of configured events that occurred. */ @UnstableApi class PlayerStateObserver( private val player: Player, private val firstEvent: @Player.Event Int, - vararg otherEvents: @Player.Event Int, - private val stateUpdater: (Player) -> Unit, + private vararg val otherEvents: @Player.Event Int, + private val initialStateUpdater: (Player) -> Unit, + private val stateUpdater: (Player, Player.Events) -> Unit, ) { - private val otherEventsArray = otherEvents + /** + * Creates a [PlayerStateObserver] that triggers [stateUpdater] initially (with all configured + * [Player.Events]) and whenever one of the configured events happen. + * + * @param player The [Player]. + * @param firstEvent The first [Player.Event] to listen to. + * @param otherEvents Additional [Player.Event] types to listen to. + * @param stateUpdater The operation to trigger initially (with all configured [Player.Events]) + * and whenever one of the configured events happen. + */ + constructor( + player: Player, + firstEvent: @Player.Event Int, + vararg otherEvents: @Player.Event Int, + stateUpdater: (Player, Player.Events) -> Unit, + ) : this( + player, + firstEvent, + *otherEvents, + initialStateUpdater = { p -> stateUpdater(p, createEvents(firstEvent, *otherEvents)) }, + stateUpdater = stateUpdater, + ) + + /** + * Creates a [PlayerStateObserver] that triggers [stateUpdater] initially and whenever one of the + * configured events happen. + * + * @param player The [Player]. + * @param firstEvent The first [Player.Event] to listen to. + * @param otherEvents Additional [Player.Event] types to listen to. + * @param stateUpdater The operation to trigger initially and whenever one of the configured + * events happen. + */ + constructor( + player: Player, + firstEvent: @Player.Event Int, + vararg otherEvents: @Player.Event Int, + stateUpdater: (Player) -> Unit, + ) : this( + player, + firstEvent, + *otherEvents, + initialStateUpdater = stateUpdater, + stateUpdater = { p, _ -> stateUpdater(p) }, + ) + + private val configuredEvents = createEvents(firstEvent, *otherEvents) init { - stateUpdater.invoke(player) + initialStateUpdater.invoke(player) } /** Observes updates from the configured [Player.Events]. */ suspend fun observe(): Nothing { - stateUpdater.invoke(player) - player.listenTo(firstEvent, *otherEventsArray) { stateUpdater.invoke(player) } + initialStateUpdater.invoke(player) + player.listenTo(firstEvent, *otherEvents) { events -> + stateUpdater.invoke(player, filterEvents(events)) + } + } + + private fun filterEvents(events: Player.Events): Player.Events { + val flagSetBuilder = FlagSet.Builder() + for (i in 0 until events.size()) { + val event = events.get(i) + if (configuredEvents.contains(event)) { + flagSetBuilder.add(event) + } + } + return Player.Events(flagSetBuilder.build()) } companion object { + private fun createEvents( + firstEvent: @Player.Event Int, + vararg otherEvents: @Player.Event Int, + ): Player.Events = Player.Events(FlagSet.Builder().add(firstEvent).addAll(*otherEvents).build()) + init { MediaLibraryInfo.registerModule("media3.ui.compose") } @@ -59,10 +127,10 @@ class PlayerStateObserver( /** * Utility to observe [Player] states by listening to events. * - * @param firstEvent The first [Player.Event] to listen to - * @param otherEvents Additional [Player.Event] types to listen to + * @param firstEvent The first [Player.Event] to listen to. + * @param otherEvents Additional [Player.Event] types to listen to. * @param stateUpdater The operation to trigger initially and whenever one of the configured events - * happen + * happen. */ @UnstableApi fun Player.observeState( @@ -70,3 +138,42 @@ fun Player.observeState( vararg otherEvents: @Player.Event Int, stateUpdater: (Player) -> Unit, ) = PlayerStateObserver(player = this, firstEvent, *otherEvents, stateUpdater = stateUpdater) + +/** + * Utility to observe [Player] states by listening to events. + * + * @param firstEvent The first [Player.Event] to listen to. + * @param otherEvents Additional [Player.Event] types to listen to. + * @param stateUpdater The operation to trigger initially (with all configured [Player.Events]) and + * whenever one of the configured events happen. + */ +@UnstableApi +fun Player.observeState( + firstEvent: @Player.Event Int, + vararg otherEvents: @Player.Event Int, + stateUpdater: (Player, Player.Events) -> Unit, +) = PlayerStateObserver(player = this, firstEvent, *otherEvents, stateUpdater = stateUpdater) + +/** + * Utility to observe [Player] states by listening to events. + * + * @param firstEvent The first [Player.Event] to listen to. + * @param otherEvents Additional [Player.Event] types to listen to. + * @param initialStateUpdater The operation to trigger initially on creation and when + * [PlayerStateObserver.observe] starts. + * @param stateUpdater The operation to trigger whenever one of the configured events happen. + */ +@UnstableApi +fun Player.observeState( + firstEvent: @Player.Event Int, + vararg otherEvents: @Player.Event Int, + initialStateUpdater: (Player) -> Unit, + stateUpdater: (Player, Player.Events) -> Unit, +) = + PlayerStateObserver( + player = this, + firstEvent, + *otherEvents, + initialStateUpdater = initialStateUpdater, + stateUpdater = stateUpdater, + ) diff --git a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/PresentationState.kt b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/PresentationState.kt index 38569902a1e..6c13e481247 100644 --- a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/PresentationState.kt +++ b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/PresentationState.kt @@ -29,7 +29,6 @@ import androidx.media3.common.MediaLibraryInfo import androidx.media3.common.Player import androidx.media3.common.Timeline import androidx.media3.common.VideoSize -import androidx.media3.common.listen import androidx.media3.common.util.UnstableApi /** @@ -63,42 +62,68 @@ fun rememberPresentationState( * * @param[keepContentOnReset] whether the currently displayed video frame or media artwork is kept * visible when tracks change or player changes. Defaults to false. - * @property[videoSizeDp] wraps [Player.getVideoSize] in Compose's [Size], becomes `null` when - * either height or width of the video is zero, or [player] is null. Takes into account - * [VideoSize.pixelWidthHeightRatio] to return a Size in [Dp][androidx.compose.ui.unit.Dp], i.e. - * device-independent pixel. To use this measurement in Compose's Drawing and Layout stages, - * convert it into pixels using [Density.toPx][androidx.compose.ui.unit.Density.toPx]. Note that - * for cases where `pixelWidthHeightRatio` is not equal to 1, the rescaling will be down, i.e. - * reducing the width or the height to achieve the same aspect ratio in square pixels. + * @property[videoAspectRatio] the aspect ratio of the video (width / height), adjusted for + * [VideoSize.pixelWidthHeightRatio]. Becomes `null` when either height or width of the video is + * zero, or [player] is null. * @property[coverSurface] set to false when the Player emits [Player.EVENT_RENDERED_FIRST_FRAME] * and reset back to true on [Player.EVENT_TRACKS_CHANGED] depending on the number and type of * tracks. + * @property[showArtwork] set to true when artwork should be displayed. It does not guarantee that + * the current media item actually contains artwork data; the caller is responsible for validating + * the presence of artwork before rendering. */ @UnstableApi class PresentationState(keepContentOnReset: Boolean = false) { - var videoSizeDp: Size? by mutableStateOf(null) - private set + private var _adjustedVideoSize by mutableStateOf(null) + + /** + * Wraps the video size in Compose's [Size]. + * + * @deprecated Use [videoAspectRatio] instead. + */ + @Deprecated("Use videoAspectRatio instead.") + val videoSizeDp: Size? + get() = _adjustedVideoSize + + val videoAspectRatio: Float? + get() = _adjustedVideoSize?.run { width / height } var coverSurface by mutableStateOf(true) private set - var keepContentOnReset: Boolean = keepContentOnReset + var showArtwork by mutableStateOf(false) + private set + + internal var keepContentOnReset: Boolean = keepContentOnReset set(value) { if (value != field) { field = value - maybeHideSurface(player) + updateForCurrentTrackSelections(player) } } + private var playerStateObserver: PlayerStateObserver? = null + @VisibleForTesting(otherwise = VisibleForTesting.PRIVATE) var player: Player? = null set(value) { + if (field == value) return field = value - // Only update the size if we are attaching a player OR if we don't need to keep the content - if (value != null || !keepContentOnReset) { - videoSizeDp = getVideoSizeDp(value) + playerStateObserver = + value?.observeState( + Player.EVENT_VIDEO_SIZE_CHANGED, + Player.EVENT_RENDERED_FIRST_FRAME, + Player.EVENT_TRACKS_CHANGED, + Player.EVENT_TRACK_SELECTION_PARAMETERS_CHANGED, + initialStateUpdater = ::initializeState, + stateUpdater = ::updateState, + ) + if (value == null) { + if (!keepContentOnReset) { + _adjustedVideoSize = null + } + updateForCurrentTrackSelections(null) } - maybeHideSurface(value) } private var lastPeriodUidWithTracks: Any? = null @@ -106,35 +131,47 @@ class PresentationState(keepContentOnReset: Boolean = false) { /** * Subscribes to updates from [Player.Events] and listens to * * [Player.EVENT_VIDEO_SIZE_CHANGED] to determine pixelWidthHeightRatio-adjusted video size - * * [Player.EVENT_RENDERED_FIRST_FRAME] and [Player.EVENT_TRACKS_CHANGED]to determine whether the - * surface is ready to be shown + * * [Player.EVENT_RENDERED_FIRST_FRAME], [Player.EVENT_TRACKS_CHANGED], and + * [Player.EVENT_TRACK_SELECTION_PARAMETERS_CHANGED] to determine whether the surface is ready + * to be shown */ suspend fun observe(player: Player?) { try { this@PresentationState.player = player - player?.listen { events -> - if (events.contains(Player.EVENT_VIDEO_SIZE_CHANGED)) { - if (videoSize != VideoSize.UNKNOWN && playbackState != Player.STATE_IDLE) { - this@PresentationState.videoSizeDp = getVideoSizeDp(player) - } - } - if (events.contains(Player.EVENT_RENDERED_FIRST_FRAME)) { - // open shutter, video available - coverSurface = false - } - if (events.contains(Player.EVENT_TRACKS_CHANGED)) { - if (!shouldKeepSurfaceVisible(player)) { - maybeHideSurface(player) - } - } - } + playerStateObserver?.observe() } finally { this@PresentationState.player = null } } - private fun getVideoSizeDp(player: Player?): Size? { - player ?: return null + private fun initializeState(player: Player) { + _adjustedVideoSize = getAdjustedVideoSize(player) + updateForCurrentTrackSelections(player) + } + + private fun updateState(player: Player, events: Player.Events) { + if (events.contains(Player.EVENT_VIDEO_SIZE_CHANGED)) { + if (player.videoSize != VideoSize.UNKNOWN && player.playbackState != Player.STATE_IDLE) { + _adjustedVideoSize = getAdjustedVideoSize(player) + } + } + if (events.contains(Player.EVENT_RENDERED_FIRST_FRAME)) { + // open shutter, video available + coverSurface = false + } + if ( + events.containsAny( + Player.EVENT_TRACKS_CHANGED, + Player.EVENT_TRACK_SELECTION_PARAMETERS_CHANGED, + ) + ) { + if (!shouldKeepSurfaceVisible(player)) { + updateForCurrentTrackSelections(player) + } + } + } + + private fun getAdjustedVideoSize(player: Player): Size? { var videoSize = Size(player.videoSize.width.toFloat(), player.videoSize.height.toFloat()) if (videoSize.width == 0f || videoSize.height == 0f) return null @@ -147,18 +184,29 @@ class PresentationState(keepContentOnReset: Boolean = false) { return videoSize } - private fun maybeHideSurface(player: Player?) { + @VisibleForTesting(otherwise = VisibleForTesting.PRIVATE) + fun updateForCurrentTrackSelections(player: Player?) { if (player != null) { val hasTracks = player.isCommandAvailable(Player.COMMAND_GET_TRACKS) && !player.currentTracks.isEmpty if (!keepContentOnReset && !hasTracks) { coverSurface = true + showArtwork = false } - if (hasTracks && !hasSelectedVideoTrack(player)) { - coverSurface = true + if (hasTracks) { + if (hasSelectedVideoTrack(player)) { + // We don't lift the shutter here; we wait for EVENT_RENDERED_FIRST_FRAME instead. + } else if (hasSelectedTextTrack(player)) { + // No video track, but text (subtitles) is selected, lift the shutter to show them + coverSurface = false + } else { + coverSurface = true + } + showArtwork = !hasSelectedVideoTrack(player) } } else { coverSurface = coverSurface || !keepContentOnReset + showArtwork = showArtwork && keepContentOnReset } } @@ -199,6 +247,10 @@ class PresentationState(keepContentOnReset: Boolean = false) { player.isCommandAvailable(Player.COMMAND_GET_TRACKS) && player.currentTracks.isTypeSelected(C.TRACK_TYPE_VIDEO) + private fun hasSelectedTextTrack(player: Player): Boolean = + player.isCommandAvailable(Player.COMMAND_GET_TRACKS) && + player.currentTracks.isTypeSelected(C.TRACK_TYPE_TEXT) + companion object { init { MediaLibraryInfo.registerModule("media3.ui.compose") diff --git a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/ProgressStateJob.kt b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/ProgressStateJob.kt index 2e881a9e7a2..28aff745c0e 100644 --- a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/ProgressStateJob.kt +++ b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/ProgressStateJob.kt @@ -20,6 +20,7 @@ import androidx.annotation.VisibleForTesting import androidx.compose.runtime.withFrameMillis import androidx.media3.common.C import androidx.media3.common.Player +import androidx.media3.common.Timeline import androidx.media3.common.util.UnstableApi import kotlinx.coroutines.CoroutineScope import kotlinx.coroutines.Job @@ -133,6 +134,31 @@ internal fun getDurationMsOrDefault(player: Player): Long { } } +internal fun getOriginalDurationMsOrDefault( + player: Player, + window: Timeline.Window = Timeline.Window(), + period: Timeline.Period = Timeline.Period(), +): Long { + if (!player.isCommandAvailable(Player.COMMAND_GET_CURRENT_MEDIA_ITEM)) { + return C.TIME_UNSET + } + if (!player.isPlayingAd && player.isCommandAvailable(Player.COMMAND_GET_TIMELINE)) { + val timeline = player.currentTimeline + if (!timeline.isEmpty) { + timeline.getWindow(player.currentMediaItemIndex, window) + // Clipping is only supported for single-period media items. + if (window.firstPeriodIndex == window.lastPeriodIndex) { + val originalDurationMs = + timeline.getPeriod(window.firstPeriodIndex, period).originalDurationMs + if (originalDurationMs != C.TIME_UNSET) { + return originalDurationMs + } + } + } + } + return player.duration +} + internal fun isReadyOrBuffering(player: Player): Boolean = player.playbackState == Player.STATE_READY || player.playbackState == Player.STATE_BUFFERING diff --git a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/ProgressStateWithTickCount.kt b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/ProgressStateWithTickCount.kt index 8bb28b0ae3f..4886114310e 100644 --- a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/ProgressStateWithTickCount.kt +++ b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/ProgressStateWithTickCount.kt @@ -28,8 +28,8 @@ import androidx.compose.runtime.rememberCoroutineScope import androidx.compose.runtime.setValue import androidx.media3.common.C import androidx.media3.common.Player +import androidx.media3.common.Timeline import androidx.media3.common.util.UnstableApi -import com.google.common.base.Preconditions.checkState import kotlinx.coroutines.CoroutineScope /** @@ -99,6 +99,9 @@ fun rememberProgressStateWithTickCount( * rounded to the nearest multiple of 1/totalTickCount. * @property[durationMs] The duration of the current content or ad in milliseconds, matches * [Player.getDuration]. + * @property[originalDurationMs] The duration of the current media item in milliseconds before any + * clipping of its start or end. Equal to [durationMs] if the media item isn't clipped, its + * original duration isn't known, it has multiple periods, or an ad is playing. */ @UnstableApi class ProgressStateWithTickCount( @@ -115,6 +118,9 @@ class ProgressStateWithTickCount( var durationMs by mutableLongStateOf(C.TIME_UNSET) private set + var originalDurationMs by mutableLongStateOf(C.TIME_UNSET) + private set + /** * Whether the user is allowed to change the progress of the player, for example by dragging or * tapping a slider on the UI side or programmatically calling [updateCurrentPositionProgress]. @@ -126,6 +132,9 @@ class ProgressStateWithTickCount( var changingProgressEnabled by mutableStateOf(false) private set + private val window = Timeline.Window() + private val period = Timeline.Period() + private val updateJob = player?.let { ProgressStateJob( player = it, @@ -180,8 +189,8 @@ class ProgressStateWithTickCount( suspend fun observe() = updateJob?.observeProgress() private fun nextMediaWakeUpPositionMs(player: Player): Long { - checkState(totalTickCount != 0) - checkState(durationMs > 0) + check(totalTickCount != 0) + check(durationMs > 0) val currentPositionTick = getPositionTick(getCurrentPositionMsOrDefault(player), durationMs, totalTickCount) val nextTickIndex = currentPositionTick + 1 @@ -191,6 +200,7 @@ class ProgressStateWithTickCount( private fun updateProgress(player: Player) { durationMs = getDurationMsOrDefault(player) + originalDurationMs = getOriginalDurationMsOrDefault(player, window, period) currentPositionProgress = positionToProgress(getCurrentPositionMsOrDefault(player), durationMs, totalTickCount) bufferedPositionProgress = diff --git a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/ProgressStateWithTickInterval.kt b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/ProgressStateWithTickInterval.kt index e224c168a32..bf5d03e8fad 100644 --- a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/ProgressStateWithTickInterval.kt +++ b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/ProgressStateWithTickInterval.kt @@ -26,6 +26,7 @@ import androidx.compose.runtime.rememberCoroutineScope import androidx.compose.runtime.setValue import androidx.media3.common.C import androidx.media3.common.Player +import androidx.media3.common.Timeline import androidx.media3.common.util.UnstableApi import kotlinx.coroutines.CoroutineScope @@ -94,6 +95,9 @@ fun rememberProgressStateWithTickInterval( * which data is buffered, in milliseconds. * @property[durationMs] The duration of the current content or ad in milliseconds, matches * [Player.getDuration]. + * @property[originalDurationMs] The duration of the current media item in milliseconds before any + * clipping of its start or end. Equal to [durationMs] if the media item isn't clipped, its + * original duration isn't known, it has multiple periods, or an ad is playing. */ @UnstableApi class ProgressStateWithTickInterval( @@ -110,6 +114,12 @@ class ProgressStateWithTickInterval( var durationMs by mutableLongStateOf(C.TIME_UNSET) private set + var originalDurationMs by mutableLongStateOf(C.TIME_UNSET) + private set + + private val window = Timeline.Window() + private val period = Timeline.Period() + private val updateJob = player?.let { ProgressStateJob( player = it, @@ -149,6 +159,7 @@ class ProgressStateWithTickInterval( currentPositionMs = snapPositionToNearestTick(player, ::getCurrentPositionMsOrDefault) bufferedPositionMs = snapPositionToNearestTick(player, ::getBufferedPositionMsOrDefault) durationMs = getDurationMsOrDefault(player) + originalDurationMs = getOriginalDurationMsOrDefault(player, window, period) } /** diff --git a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/TrackSelectionParametersState.kt b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/TrackSelectionParametersState.kt new file mode 100644 index 00000000000..d5178e5cb27 --- /dev/null +++ b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/TrackSelectionParametersState.kt @@ -0,0 +1,110 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.ui.compose.state + +import androidx.compose.runtime.Composable +import androidx.compose.runtime.LaunchedEffect +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.setValue +import androidx.media3.common.Player +import androidx.media3.common.TrackSelectionParameters +import androidx.media3.common.Tracks +import androidx.media3.common.util.UnstableApi + +/** + * Remembers the value of [TrackSelectionParametersState] created based on the passed [Player] and + * launches a coroutine to listen to [Player] changes. If the [Player] instance changes between + * compositions, a new value is produced and remembered. + * + * @param player The [Player] to observe, or null. + * @return A [TrackSelectionParametersState] instance holding the current state. + * @see [TrackSelectionParametersState] + */ +@UnstableApi +@Composable +fun rememberTrackSelectionParametersState(player: Player?): TrackSelectionParametersState { + val trackSelectionParametersState = remember(player) { TrackSelectionParametersState(player) } + LaunchedEffect(player) { trackSelectionParametersState.observe() } + return trackSelectionParametersState +} + +/** + * A state holder that exposes the current [Tracks] and [TrackSelectionParameters] of a [Player] as + * Compose state. + * + * This state is suitable for advanced use cases where direct access to the player's track + * information and parameters is needed, without any additional UI abstraction. + * + * @property tracks The current [Tracks] available in the player. + * @property trackSelectionParameters The current [TrackSelectionParameters] acting on the player. + * @property canSetTrackSelectionParameters Whether [Player.COMMAND_SET_TRACK_SELECTION_PARAMETERS] + * is available on the player. + */ +@UnstableApi +class TrackSelectionParametersState(private val player: Player?) { + + var tracks: Tracks by mutableStateOf(player.getTracksIfCommandAvailable()) + private set + + var trackSelectionParameters: TrackSelectionParameters by + mutableStateOf(player?.trackSelectionParameters ?: TrackSelectionParameters.DEFAULT) + private set + + var canSetTrackSelectionParameters: Boolean by + mutableStateOf( + player?.isCommandAvailable(Player.COMMAND_SET_TRACK_SELECTION_PARAMETERS) == true + ) + private set + + private val playerStateObserver: PlayerStateObserver? = + player?.observeState( + Player.EVENT_TRACKS_CHANGED, + Player.EVENT_TRACK_SELECTION_PARAMETERS_CHANGED, + Player.EVENT_AVAILABLE_COMMANDS_CHANGED, + ) { + tracks = it.getTracksIfCommandAvailable() + trackSelectionParameters = it.trackSelectionParameters + canSetTrackSelectionParameters = + it.isCommandAvailable(Player.COMMAND_SET_TRACK_SELECTION_PARAMETERS) + } + + /** + * Applies new track selection parameters to the player. + * + * @param params The new [TrackSelectionParameters] to apply. + * @see [Player.getTrackSelectionParameters] + */ + fun updateTrackSelectionParameters(params: TrackSelectionParameters) { + player + ?.takeIf { it.isCommandAvailable(Player.COMMAND_SET_TRACK_SELECTION_PARAMETERS) } + ?.trackSelectionParameters = params + } + + /** + * Subscribes to updates from [Player.Events] and listens to [Player.EVENT_TRACKS_CHANGED], + * [Player.EVENT_TRACK_SELECTION_PARAMETERS_CHANGED], and + * [Player.EVENT_AVAILABLE_COMMANDS_CHANGED] to update the state properties. + * + * This is a suspending function that runs indefinitely until cancelled. + */ + suspend fun observe() = playerStateObserver?.observe() + + private fun Player?.getTracksIfCommandAvailable(): Tracks = + this?.takeIf { it.isCommandAvailable(Player.COMMAND_GET_TRACKS) }?.currentTracks ?: Tracks.EMPTY +} diff --git a/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/TrackSelectionState.kt b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/TrackSelectionState.kt new file mode 100644 index 00000000000..e8459bb3f42 --- /dev/null +++ b/libraries/ui_compose/src/main/java/androidx/media3/ui/compose/state/TrackSelectionState.kt @@ -0,0 +1,443 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.ui.compose.state + +import androidx.annotation.IntDef +import androidx.compose.runtime.Composable +import androidx.compose.runtime.derivedStateOf +import androidx.compose.runtime.getValue +import androidx.compose.runtime.remember +import androidx.media3.common.C +import androidx.media3.common.Format +import androidx.media3.common.Player +import androidx.media3.common.TrackSelectionParameters +import androidx.media3.common.Tracks +import androidx.media3.common.util.Log +import androidx.media3.common.util.UnstableApi + +private const val TAG = "TrackSelectionState" + +/** + * Represents the track selection state for a specific track type. + * + * @see TRACK_SELECTION_OFF + * @see TRACK_SELECTION_AUTO + * @see TRACK_SELECTION_ON + */ +@UnstableApi +@Retention(AnnotationRetention.SOURCE) +@Target(AnnotationTarget.TYPE) +@MustBeDocumented +@IntDef(TRACK_SELECTION_OFF, TRACK_SELECTION_AUTO, TRACK_SELECTION_ON) +annotation class OnOffState + +/** The track type is explicitly disabled. */ +@UnstableApi const val TRACK_SELECTION_OFF = 0 +/** + * No specific preferences or constraints are applied for the active filters (the player chooses + * automatically). + */ +@UnstableApi const val TRACK_SELECTION_AUTO = 1 +/** Specific preferences or constraints are applied for the active filters. */ +@UnstableApi const val TRACK_SELECTION_ON = 2 + +/** + * Supported filters for selecting specific tracks. + * + * @see TRACK_FILTER_LANGUAGE + * @see TRACK_FILTER_RESOLUTION + * @see TRACK_FILTER_MIME_TYPE + */ +@UnstableApi +@Retention(AnnotationRetention.SOURCE) +@Target(AnnotationTarget.TYPE) +@MustBeDocumented +@IntDef( + flag = true, + value = [TRACK_FILTER_LANGUAGE, TRACK_FILTER_RESOLUTION, TRACK_FILTER_MIME_TYPE], +) +annotation class TrackFilter + +/** Filter for selecting track language (applies to audio or text depending on track type). */ +@UnstableApi const val TRACK_FILTER_LANGUAGE = 1 shl 0 +/** Filter for selecting video resolution (`width` and `height`). */ +@UnstableApi const val TRACK_FILTER_RESOLUTION = 1 shl 1 +/** Filter for selecting video MIME type (`sampleMimeType`). */ +@UnstableApi const val TRACK_FILTER_MIME_TYPE = 1 shl 2 + +/** + * Represents a unique track configuration available for selection, based on the active + * [TrackFilter]s. + * + * This configuration encapsulates specific track properties (like language or resolution) and + * indicates whether it is currently supported and selected by the player. + * + * @property isSupported Whether the option is supported by the device. + * @property isSelected Whether the option is currently selected. + * @property language The language associated with this option, or `null`. + * @property width The video width associated with this option, or `null`. + * @property height The video height associated with this option, or `null`. + * @property sampleMimeType The sample MIME type associated with this option, or `null`. + * @property onOffState The [OnOffState] representing this option. + */ +@UnstableApi +class TrackSelectionOption( + val isSupported: Boolean, + val isSelected: Boolean, + val language: String? = null, + val width: Int? = null, + val height: Int? = null, + val sampleMimeType: String? = null, + val onOffState: @OnOffState Int = TRACK_SELECTION_ON, +) { + internal fun copyInternal( + isSupported: Boolean = this.isSupported, + isSelected: Boolean = this.isSelected, + ): TrackSelectionOption = + TrackSelectionOption( + isSupported, + isSelected, + language, + width, + height, + sampleMimeType, + onOffState, + ) + + override fun equals(other: Any?): Boolean { + if (this === other) return true + if (other !is TrackSelectionOption) return false + return isSupported == other.isSupported && + isSelected == other.isSelected && + language == other.language && + width == other.width && + height == other.height && + sampleMimeType == other.sampleMimeType && + onOffState == other.onOffState + } + + override fun hashCode(): Int { + var result = isSupported.hashCode() + result = 31 * result + isSelected.hashCode() + result = 31 * result + (language?.hashCode() ?: 0) + result = 31 * result + (width?.hashCode() ?: 0) + result = 31 * result + (height?.hashCode() ?: 0) + result = 31 * result + (sampleMimeType?.hashCode() ?: 0) + result = 31 * result + onOffState + return result + } + + override fun toString(): String { + return "TrackSelectionOption(isSupported=$isSupported, isSelected=$isSelected, language=$language, width=$width, height=$height, sampleMimeType=$sampleMimeType, onOffState=$onOffState)" + } +} + +/** + * Remembers a [TrackSelectionState] for the given [trackType] and [trackFilters]. + * + * @param player The [Player] instance to observe and update track selection state. + * @param trackType The type of track to manage (e.g., [C.TRACK_TYPE_VIDEO]). + * @param trackFilters The bitmask of filters to apply to the track selection. + * @param hasOffOption Whether the UI includes an "Off" option. + * @param hasAutoOption Whether the UI includes an "Auto" option. + * @param hasOnOption Whether the UI includes explicit track options. + * @return A [TrackSelectionState] instance. + * @see [TrackSelectionState] + */ +@UnstableApi +@Composable +fun rememberTrackSelectionState( + player: Player?, + trackType: @C.TrackType Int, + trackFilters: @TrackFilter Int = 0, + hasOffOption: Boolean = false, + hasAutoOption: Boolean = false, + hasOnOption: Boolean = true, +): TrackSelectionState { + val internalTrackSelectionParametersState = rememberTrackSelectionParametersState(player) + return remember( + internalTrackSelectionParametersState, + trackType, + trackFilters, + hasOffOption, + hasAutoOption, + hasOnOption, + ) { + TrackSelectionState( + internalTrackSelectionParametersState, + trackType, + trackFilters, + hasOffOption, + hasAutoOption, + hasOnOption, + ) + } +} + +/** + * A state object that manages the track selection for a specific [trackType]. + * + * Availability of track selection can be determined by checking [canSetTrackSelectionParameters] + * and whether [selectionOptions] has sufficient options for the desired UI. + * + * @property trackType The type of track this state manages (e.g., [C.TRACK_TYPE_VIDEO]). + * @property trackFilters The bitmask of filters supported for selecting tracks of this type. + * @property hasOffOption Whether the UI includes an "Off" option. + * @property hasAutoOption Whether the UI includes an "Auto" option. + * @property hasOnOption Whether the UI includes explicit track options. + * @property tracks The current [Tracks] available in the player. + * @property trackSelectionParameters The current [TrackSelectionParameters] acting on the player. + * @property canSetTrackSelectionParameters Whether [Player.COMMAND_SET_TRACK_SELECTION_PARAMETERS] + * is available on the player. + * @property selectionOptions Available [TrackSelectionOption]s based on [trackFilters] and options. + * @property selectedOption The selected option, or null if no option is selected. + */ +@UnstableApi +class TrackSelectionState( + private val internalTrackSelectionParametersState: TrackSelectionParametersState, + val trackType: @C.TrackType Int, + val trackFilters: @TrackFilter Int = 0, + val hasOffOption: Boolean = false, + val hasAutoOption: Boolean = false, + val hasOnOption: Boolean = true, +) { + + val tracks: Tracks + get() = internalTrackSelectionParametersState.tracks + + val trackSelectionParameters: TrackSelectionParameters + get() = internalTrackSelectionParametersState.trackSelectionParameters + + val canSetTrackSelectionParameters: Boolean + get() = internalTrackSelectionParametersState.canSetTrackSelectionParameters + + val selectionOptions: List by derivedStateOf { + val trackSelectionParams = internalTrackSelectionParametersState.trackSelectionParameters + val currentOnOffState = getCurrentOnOffState(trackSelectionParams) + buildList { + if (hasOffOption) { + add( + TrackSelectionOption( + isSupported = true, + isSelected = (currentOnOffState == TRACK_SELECTION_OFF), + onOffState = TRACK_SELECTION_OFF, + ) + ) + } + + if (hasAutoOption) { + add( + TrackSelectionOption( + isSupported = true, + isSelected = (currentOnOffState == TRACK_SELECTION_AUTO), + onOffState = TRACK_SELECTION_AUTO, + ) + ) + } + + if (hasOnOption && trackFilters != 0) { + val optionsMap = mutableMapOf() + val filteredGroups = getFilteredGroups() + for (group in filteredGroups) { + for (trackIndex in 0 until group.length) { + val format = group.getTrackFormat(trackIndex) + + val language = format.language.takeIf { isFilterEnabled(TRACK_FILTER_LANGUAGE) } + val width = + format.width.takeIf { + isFilterEnabled(TRACK_FILTER_RESOLUTION) && it != Format.NO_VALUE + } + val height = + format.height.takeIf { + isFilterEnabled(TRACK_FILTER_RESOLUTION) && it != Format.NO_VALUE + } + val sampleMimeType = + format.sampleMimeType.takeIf { isFilterEnabled(TRACK_FILTER_MIME_TYPE) } + + val isSupported = group.isTrackSupported(trackIndex) + val isSelected = + (currentOnOffState == TRACK_SELECTION_ON || !hasAutoOption) && + group.isTrackSelected(trackIndex) + val key = OptionKey(language, width, height, sampleMimeType) + + optionsMap[key] = + optionsMap[key]?.let { + it.copyInternal( + isSupported = it.isSupported || isSupported, + isSelected = it.isSelected || isSelected, + ) + } + ?: TrackSelectionOption( + isSupported = isSupported, + isSelected = isSelected, + language = language, + width = width, + height = height, + sampleMimeType = sampleMimeType, + onOffState = TRACK_SELECTION_ON, + ) + } + } + addAll(optionsMap.values) + } + } + } + + val selectedOption: TrackSelectionOption? by derivedStateOf { + selectionOptions.firstOrNull { it.isSelected } + } + + /** + * Selects an option for this track type. + * + * @param option The [TrackSelectionOption] to select, or `null` to clear the selection (treated + * as [TRACK_SELECTION_AUTO]). + */ + @UnstableApi + fun selectOption(option: TrackSelectionOption?) { + if (option != null && !selectionOptions.contains(option)) { + Log.w(TAG, "Ignoring selection of option not present in selectionOptions: $option") + return + } + + val newParams = + internalTrackSelectionParametersState.trackSelectionParameters + .buildUpon() + .apply { + if (option == null || option.onOffState == TRACK_SELECTION_AUTO) { + setTrackTypeDisabled(trackType, false) + clearOverridesAndConstraints(this) + } else { + when (option.onOffState) { + TRACK_SELECTION_OFF -> { + setTrackTypeDisabled(trackType, true) + } + TRACK_SELECTION_ON -> { + setTrackTypeDisabled(trackType, false) + clearOverridesOfType(trackType) + + if (isFilterEnabled(TRACK_FILTER_LANGUAGE)) { + if (trackType == C.TRACK_TYPE_AUDIO) { + if (option.language != null) { + setPreferredAudioLanguages(option.language) + } else { + setPreferredAudioLanguages() + } + } else if (trackType == C.TRACK_TYPE_TEXT) { + if (option.language != null) { + setPreferredTextLanguages(option.language).setSelectTextByDefault(false) + } else { + setPreferredTextLanguages().setSelectTextByDefault(true) + } + } + } + + if (isFilterEnabled(TRACK_FILTER_MIME_TYPE)) { + if (option.sampleMimeType != null) { + setPreferredVideoMimeTypes(option.sampleMimeType) + } else { + setPreferredVideoMimeTypes() + } + } + + if (isFilterEnabled(TRACK_FILTER_RESOLUTION)) { + // TODO: Continuous dimensions like resolution are currently treated as + // max constraints (`setMaxVideoSize`). Consider adding an option on + // TrackSelectionState to configure whether resolution or channel count + // should act as max, min, or exact constraints. + setMaxVideoSize(option.width ?: Int.MAX_VALUE, option.height ?: Int.MAX_VALUE) + } + } + } + } + } + .build() + + internalTrackSelectionParametersState.updateTrackSelectionParameters(newParams) + } + + private fun getCurrentOnOffState(params: TrackSelectionParameters): @OnOffState Int { + if (params.disabledTrackTypes.contains(trackType)) { + return TRACK_SELECTION_OFF + } + return if (hasOverridesOrConstraints(params)) { + TRACK_SELECTION_ON + } else { + TRACK_SELECTION_AUTO + } + } + + private fun hasOverridesOrConstraints(params: TrackSelectionParameters): Boolean { + if (getFilteredGroups().any { params.overrides.containsKey(it.mediaTrackGroup) }) { + return true + } + return when (trackType) { + C.TRACK_TYPE_AUDIO -> { + isFilterEnabled(TRACK_FILTER_LANGUAGE) && params.preferredAudioLanguages.isNotEmpty() + } + C.TRACK_TYPE_TEXT -> { + isFilterEnabled(TRACK_FILTER_LANGUAGE) && + (params.selectTextByDefault || params.preferredTextLanguages.isNotEmpty()) + } + C.TRACK_TYPE_VIDEO -> { + val hasResolutionConstraint = + isFilterEnabled(TRACK_FILTER_RESOLUTION) && + (params.maxVideoWidth != Int.MAX_VALUE || params.maxVideoHeight != Int.MAX_VALUE) + val hasMimeTypeConstraint = + isFilterEnabled(TRACK_FILTER_MIME_TYPE) && params.preferredVideoMimeTypes.isNotEmpty() + hasResolutionConstraint || hasMimeTypeConstraint + } + else -> false + } + } + + private fun clearOverridesAndConstraints(builder: TrackSelectionParameters.Builder) { + builder.clearOverridesOfType(trackType) + when (trackType) { + C.TRACK_TYPE_AUDIO -> { + if (isFilterEnabled(TRACK_FILTER_LANGUAGE)) { + builder.setPreferredAudioLanguages() + } + } + C.TRACK_TYPE_TEXT -> { + if (isFilterEnabled(TRACK_FILTER_LANGUAGE)) { + builder.setPreferredTextLanguages().setSelectTextByDefault(false) + } + } + C.TRACK_TYPE_VIDEO -> { + if (isFilterEnabled(TRACK_FILTER_RESOLUTION)) { + builder.setMaxVideoSize(Int.MAX_VALUE, Int.MAX_VALUE) + } + if (isFilterEnabled(TRACK_FILTER_MIME_TYPE)) { + builder.setPreferredVideoMimeTypes() + } + } + } + } + + private fun isFilterEnabled(filter: @TrackFilter Int): Boolean = (trackFilters and filter) != 0 + + private fun getFilteredGroups(): Sequence = + internalTrackSelectionParametersState.tracks.groups.asSequence().filter { it.type == trackType } + + private data class OptionKey( + val language: String?, + val width: Int?, + val height: Int?, + val sampleMimeType: String?, + ) +} diff --git a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/ArtworkTest.kt b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/ArtworkTest.kt new file mode 100644 index 00000000000..9945d3a9666 --- /dev/null +++ b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/ArtworkTest.kt @@ -0,0 +1,329 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.ui.compose + +import android.graphics.Bitmap +import android.graphics.Canvas +import android.graphics.Color +import android.net.Uri +import androidx.compose.ui.Modifier +import androidx.compose.ui.graphics.painter.ColorPainter +import androidx.compose.ui.platform.testTag +import androidx.compose.ui.test.ExperimentalTestApi +import androidx.compose.ui.test.assertContentDescriptionEquals +import androidx.compose.ui.test.assertIsDisplayed +import androidx.compose.ui.test.hasContentDescription +import androidx.compose.ui.test.hasTestTag +import androidx.compose.ui.test.onNodeWithTag +import androidx.compose.ui.test.v2.runComposeUiTest +import androidx.core.graphics.createBitmap +import androidx.core.net.toUri +import androidx.media3.common.MediaItem +import androidx.media3.common.MediaMetadata +import androidx.media3.common.Player +import androidx.media3.common.SimpleBasePlayer.MediaItemData +import androidx.media3.common.util.BitmapLoader +import androidx.media3.test.utils.FakePlayer +import androidx.test.ext.junit.runners.AndroidJUnit4 +import coil.compose.rememberAsyncImagePainter +import com.google.common.util.concurrent.Futures +import com.google.common.util.concurrent.ListenableFuture +import java.io.ByteArrayOutputStream +import java.io.File +import kotlinx.coroutines.ExperimentalCoroutinesApi +import kotlinx.coroutines.test.UnconfinedTestDispatcher +import org.junit.Test +import org.junit.runner.RunWith + +/** Unit tests for [Artwork]. */ +@Suppress("RedundantNullableReturnType") // Wrong Lint for loadBitmapFromMetadata +@RunWith(AndroidJUnit4::class) +@OptIn(ExperimentalTestApi::class, ExperimentalCoroutinesApi::class) +class ArtworkTest { + + @Test + fun artwork_withArtworkData_displaysBitmapImage() = runComposeUiTest { + val artworkData = createTestBitmapBytes() + val mediaMetadata = + MediaMetadata.Builder() + .setArtworkData(artworkData, MediaMetadata.PICTURE_TYPE_FRONT_COVER) + .build() + + setContent { + Artwork( + metadata = mediaMetadata, + contentDescription = "Test Artwork", + modifier = Modifier.testTag("ArtworkTag"), + coroutineDispatcher = UnconfinedTestDispatcher(), + ) + } + + onNodeWithTag("ArtworkTag").assertIsDisplayed().assertContentDescriptionEquals("Test Artwork") + } + + @Test + fun artwork_withPlayerAndArtworkData_displaysBitmapImage() = runComposeUiTest { + val artworkData = createTestBitmapBytes() + val mediaItem = + MediaItem.Builder() + .setMediaMetadata( + MediaMetadata.Builder() + .setArtworkData(artworkData, MediaMetadata.PICTURE_TYPE_FRONT_COVER) + .build() + ) + .build() + val player: Player = + FakePlayer( + playlist = listOf(MediaItemData.Builder(mediaItem.mediaId).setMediaItem(mediaItem).build()) + ) + + setContent { + Artwork( + player = player, + contentDescription = "Test Artwork", + modifier = Modifier.testTag("ArtworkTag"), + coroutineDispatcher = UnconfinedTestDispatcher(), + ) + } + + onNodeWithTag("ArtworkTag").assertIsDisplayed().assertContentDescriptionEquals("Test Artwork") + } + + @Test + fun artwork_withCoil_displaysImage() = runComposeUiTest { + val artworkData = createTestBitmapBytes() + val mediaItem = + MediaItem.Builder() + .setMediaMetadata( + MediaMetadata.Builder() + .setArtworkData(artworkData, MediaMetadata.PICTURE_TYPE_FRONT_COVER) + .build() + ) + .build() + val player: Player = + FakePlayer( + playlist = listOf(MediaItemData.Builder(mediaItem.mediaId).setMediaItem(mediaItem).build()) + ) + + setContent { + Artwork( + player = player, + contentDescription = "Test Artwork", + modifier = Modifier.testTag("ArtworkTag"), + ) { + rememberAsyncImagePainter(model = mediaMetadata.artworkUri ?: mediaMetadata.artworkData) + } + } + + onNodeWithTag("ArtworkTag").assertIsDisplayed().assertContentDescriptionEquals("Test Artwork") + } + + @Test + fun artwork_customBitmapLoader_usesCustomLoader() = runComposeUiTest { + val testBitmap = createBitmap(10, 10) + val mediaItem = MediaItem.Builder().setMediaMetadata(MediaMetadata.EMPTY).build() + val player: Player = + FakePlayer( + playlist = listOf(MediaItemData.Builder(mediaItem.mediaId).setMediaItem(mediaItem).build()) + ) + val customLoader = + object : BitmapLoader { + override fun supportsMimeType(mimeType: String): Boolean = true + + override fun decodeBitmap(data: ByteArray): ListenableFuture = + throw UnsupportedOperationException() + + override fun loadBitmap(uri: android.net.Uri): ListenableFuture = + throw UnsupportedOperationException() + + override fun loadBitmapFromMetadata(metadata: MediaMetadata): ListenableFuture? = + Futures.immediateFuture(testBitmap) + } + + setContent { + Artwork( + player = player, + contentDescription = "Test Artwork", + bitmapLoader = customLoader, + modifier = Modifier.testTag("ArtworkTag"), + coroutineDispatcher = UnconfinedTestDispatcher(), + ) + } + + onNodeWithTag("ArtworkTag").assertIsDisplayed().assertContentDescriptionEquals("Test Artwork") + } + + @Test + fun artwork_customBitmapLoaderThrows_showsErrorPainter() = runComposeUiTest { + val artworkData = createTestBitmapBytes() + val mediaItem = + MediaItem.Builder() + .setMediaMetadata( + MediaMetadata.Builder() + .setArtworkData(artworkData, MediaMetadata.PICTURE_TYPE_FRONT_COVER) + .build() + ) + .build() + val player: Player = + FakePlayer( + playlist = listOf(MediaItemData.Builder(mediaItem.mediaId).setMediaItem(mediaItem).build()) + ) + val errorPainter = ColorPainter(androidx.compose.ui.graphics.Color.Red) + val failingLoader = + object : BitmapLoader { + override fun supportsMimeType(mimeType: String): Boolean = true + + override fun decodeBitmap(data: ByteArray): ListenableFuture = + throw UnsupportedOperationException() + + override fun loadBitmap(uri: android.net.Uri): ListenableFuture = + throw UnsupportedOperationException() + + override fun loadBitmapFromMetadata(metadata: MediaMetadata): ListenableFuture? = + Futures.immediateFailedFuture(RuntimeException("Failed to load")) + } + + setContent { + Artwork( + player = player, + contentDescription = "Test Artwork", + bitmapLoader = failingLoader, + error = errorPainter, + modifier = Modifier.testTag("ArtworkTag"), + coroutineDispatcher = UnconfinedTestDispatcher(), + ) + } + + onNodeWithTag("ArtworkTag").assertIsDisplayed().assertContentDescriptionEquals("Test Artwork") + } + + @Test + fun artwork_customBitmapLoaderReturnsNull_fallsBackToDefaultLoadArtwork() = runComposeUiTest { + val artworkData = createTestBitmapBytes() + val mediaItem = + MediaItem.Builder() + .setMediaMetadata( + MediaMetadata.Builder() + .setArtworkData(artworkData, MediaMetadata.PICTURE_TYPE_FRONT_COVER) + .build() + ) + .build() + val player: Player = + FakePlayer( + playlist = listOf(MediaItemData.Builder(mediaItem.mediaId).setMediaItem(mediaItem).build()) + ) + val fallbackPainter = ColorPainter(androidx.compose.ui.graphics.Color.Red) + val nullLoader = + object : BitmapLoader { + override fun supportsMimeType(mimeType: String): Boolean = true + + override fun decodeBitmap(data: ByteArray): ListenableFuture = + throw UnsupportedOperationException() + + override fun loadBitmap(uri: android.net.Uri): ListenableFuture = + throw UnsupportedOperationException() + + override fun loadBitmapFromMetadata(metadata: MediaMetadata): ListenableFuture? = + null + } + + setContent { + Artwork( + player = player, + contentDescription = "Test Artwork", + bitmapLoader = nullLoader, + fallback = fallbackPainter, + modifier = Modifier.testTag("ArtworkTag"), + coroutineDispatcher = UnconfinedTestDispatcher(), + ) + } + + onNodeWithTag("ArtworkTag").assertIsDisplayed().assertContentDescriptionEquals("Test Artwork") + } + + @Test + fun artwork_withUnsupportedUriScheme_doesNotLoadBitmap() = runComposeUiTest { + val mediaMetadata = + MediaMetadata.Builder().setArtworkUri("https://example.com/image.png".toUri()).build() + val fallbackPainter = ColorPainter(androidx.compose.ui.graphics.Color.Red) + + setContent { + Artwork( + metadata = mediaMetadata, + fallback = fallbackPainter, + contentDescription = "Test Artwork", + modifier = Modifier.testTag("ArtworkTag"), + coroutineDispatcher = UnconfinedTestDispatcher(), + ) + } + + onNodeWithTag("ArtworkTag").assertIsDisplayed().assertContentDescriptionEquals("Test Artwork") + } + + @Test + fun artwork_withFileUriScheme_loadsBitmap() = runComposeUiTest { + val artworkData = createTestBitmapBytes() + val file = File.createTempFile("artwork", ".png") + file.writeBytes(artworkData) + val mediaMetadata = MediaMetadata.Builder().setArtworkUri(Uri.fromFile(file)).build() + + setContent { + Artwork( + metadata = mediaMetadata, + contentDescription = "Test Artwork", + modifier = Modifier.testTag("ArtworkTag"), + coroutineDispatcher = UnconfinedTestDispatcher(), + ) + } + + onNodeWithTag("ArtworkTag").assertIsDisplayed().assertContentDescriptionEquals("Test Artwork") + } + + private fun createTestBitmapBytes(): ByteArray { + val bitmap = createBitmap(10, 10) + val canvas = Canvas(bitmap) + canvas.drawColor(Color.RED) + val stream = ByteArrayOutputStream() + bitmap.compress(Bitmap.CompressFormat.PNG, 100, stream) + return stream.toByteArray() + } + + @Test + fun artwork_withUnsupportedUriScheme_isRejectedEarly() = runComposeUiTest { + val uri = Uri.parse("custom://example.com/image.png") + val mediaMetadata = MediaMetadata.Builder().setArtworkUri(uri).build() + + setContent { + Artwork( + metadata = mediaMetadata, + contentDescription = "Test Artwork", + modifier = Modifier.testTag("ArtworkTag"), + coroutineDispatcher = UnconfinedTestDispatcher(), + ) + } + + // Because the custom scheme is unsupported, it should be rejected early and return null. + // With no fallback painter provided, this results in a Box without a content description. + onNode( + hasTestTag("ArtworkTag") and hasContentDescription("Test Artwork"), + useUnmergedTree = true, + ) + .assertDoesNotExist() + + onNodeWithTag("ArtworkTag", useUnmergedTree = true).assertExists() + } +} diff --git a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/ContentFrameTest.kt b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/ContentFrameTest.kt index 71b812e55f0..8e83b9c7422 100644 --- a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/ContentFrameTest.kt +++ b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/ContentFrameTest.kt @@ -19,17 +19,25 @@ package androidx.media3.ui.compose import android.view.SurfaceView import android.view.TextureView import androidx.compose.foundation.layout.Box +import androidx.compose.foundation.layout.fillMaxSize import androidx.compose.runtime.MutableState import androidx.compose.runtime.mutableStateOf import androidx.compose.runtime.remember import androidx.compose.ui.Modifier import androidx.compose.ui.layout.ContentScale import androidx.compose.ui.platform.testTag +import androidx.compose.ui.test.assertIsDisplayed import androidx.compose.ui.test.getBoundsInRoot import androidx.compose.ui.test.junit4.v2.createComposeRule import androidx.compose.ui.test.onNodeWithTag import androidx.compose.ui.unit.height import androidx.compose.ui.unit.width +import androidx.media3.common.C +import androidx.media3.common.Format +import androidx.media3.common.MimeTypes +import androidx.media3.common.SimpleBasePlayer.MediaItemData +import androidx.media3.common.TrackGroup +import androidx.media3.common.Tracks import androidx.media3.common.VideoSize import androidx.media3.test.utils.FakePlayer import androidx.test.ext.junit.runners.AndroidJUnit4 @@ -71,16 +79,20 @@ class ContentFrameTest { fun contentFrame_withIdlePlayer_shutterShown() { val player = FakePlayer() - composeTestRule.setContent { ContentFrame(player) { Box(Modifier.testTag("Shutter")) } } + composeTestRule.setContent { + ContentFrame(player) { Box(Modifier.fillMaxSize().testTag("Shutter")) } + } - composeTestRule.onNodeWithTag("Shutter").assertExists() + composeTestRule.onNodeWithTag("Shutter").assertIsDisplayed() } @Test fun contentFrame_withFirstFrameRendered_shutterOpens() { val player = FakePlayer() - composeTestRule.setContent { ContentFrame(player) { Box(Modifier.testTag("Shutter")) } } - composeTestRule.onNodeWithTag("Shutter").assertExists() + composeTestRule.setContent { + ContentFrame(player) { Box(Modifier.fillMaxSize().testTag("Shutter")) } + } + composeTestRule.onNodeWithTag("Shutter").assertIsDisplayed() player.renderFirstFrame(true) composeTestRule.waitForIdle() @@ -99,20 +111,134 @@ class ContentFrameTest { ContentFrame( player, contentScale = contentScale.value, - modifier = Modifier.testTag("ContentFrame"), + modifier = Modifier.testTag("ContentFrameOuter"), + overlay = { Box(Modifier.fillMaxSize().testTag("ScaledVideoBounds")) }, ) } - val initialBounds = composeTestRule.onNodeWithTag("ContentFrame").getBoundsInRoot() + val initialBounds = composeTestRule.onNodeWithTag("ScaledVideoBounds").getBoundsInRoot() val initialAspectRatio = initialBounds.width / initialBounds.height contentScale.value = ContentScale.Crop composeTestRule.waitForIdle() - val croppedBounds = composeTestRule.onNodeWithTag("ContentFrame").getBoundsInRoot() + val croppedBounds = composeTestRule.onNodeWithTag("ScaledVideoBounds").getBoundsInRoot() val croppedAspectRatio = croppedBounds.width / croppedBounds.height assertThat(croppedBounds).isNotEqualTo(initialBounds) assertThat(abs(initialAspectRatio - aspectRatio) / aspectRatio).isLessThan(0.01f) assertThat(abs(croppedAspectRatio - aspectRatio) / aspectRatio).isLessThan(0.01f) } + + @Test + fun contentFrame_withAudioOnlyTrack_showsArtwork() { + val audioTrack = + Tracks.Group( + TrackGroup(Format.Builder().setSampleMimeType(MimeTypes.AUDIO_AAC).build()), + /* adaptiveSupported= */ true, + /* trackSupport= */ intArrayOf(C.FORMAT_HANDLED), + /* trackSelected= */ booleanArrayOf(true), + ) + val player = + FakePlayer( + playlist = + listOf(MediaItemData.Builder("First").setTracks(Tracks(listOf(audioTrack))).build()) + ) + + composeTestRule.setContent { + ContentFrame(player, artwork = { Box(Modifier.fillMaxSize().testTag("ArtworkTag")) }) + } + + composeTestRule.onNodeWithTag("ArtworkTag").assertIsDisplayed() + } + + @Test + fun contentFrame_withVideoTrack_hidesArtwork() { + val videoTrack = + Tracks.Group( + TrackGroup(Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build()), + /* adaptiveSupported= */ true, + /* trackSupport= */ intArrayOf(C.FORMAT_HANDLED), + /* trackSelected= */ booleanArrayOf(true), + ) + val player = + FakePlayer( + playlist = + listOf(MediaItemData.Builder("First").setTracks(Tracks(listOf(videoTrack))).build()) + ) + + composeTestRule.setContent { + ContentFrame(player, artwork = { Box(Modifier.fillMaxSize().testTag("ArtworkTag")) }) + } + + composeTestRule.onNodeWithTag("ArtworkTag").assertDoesNotExist() + } + + @Test + fun contentFrame_withUnselectedVideoTrack_showsArtwork() { + val videoTrack = + Tracks.Group( + TrackGroup(Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build()), + /* adaptiveSupported= */ true, + /* trackSupport= */ intArrayOf(C.FORMAT_HANDLED), + /* trackSelected= */ booleanArrayOf(false), + ) + val player = + FakePlayer( + playlist = + listOf(MediaItemData.Builder("First").setTracks(Tracks(listOf(videoTrack))).build()) + ) + + composeTestRule.setContent { + ContentFrame(player, artwork = { Box(Modifier.fillMaxSize().testTag("ArtworkTag")) }) + } + + composeTestRule.onNodeWithTag("ArtworkTag").assertIsDisplayed() + } + + @Test + fun contentFrame_withNullArtworkParam_doesNotDisplayArtwork() { + val audioTrack = + Tracks.Group( + TrackGroup(Format.Builder().setSampleMimeType(MimeTypes.AUDIO_AAC).build()), + /* adaptiveSupported= */ true, + /* trackSupport= */ intArrayOf(C.FORMAT_HANDLED), + /* trackSelected= */ booleanArrayOf(true), + ) + val player = + FakePlayer( + playlist = + listOf(MediaItemData.Builder("First").setTracks(Tracks(listOf(audioTrack))).build()) + ) + + composeTestRule.setContent { ContentFrame(player) } + + composeTestRule.onNodeWithTag("ArtworkTag").assertDoesNotExist() + } + + @Test + fun contentFrame_withNoTracks_doesNotDisplayArtwork() { + val audioTrack = + Tracks.Group( + TrackGroup(Format.Builder().setSampleMimeType(MimeTypes.AUDIO_AAC).build()), + /* adaptiveSupported= */ true, + /* trackSupport= */ intArrayOf(C.FORMAT_HANDLED), + /* trackSelected= */ booleanArrayOf(true), + ) + val player = + FakePlayer( + playlist = + listOf(MediaItemData.Builder("First").setTracks(Tracks(listOf(audioTrack))).build()) + ) + + composeTestRule.setContent { + ContentFrame(player, artwork = { Box(Modifier.fillMaxSize().testTag("ArtworkTag")) }) + } + + composeTestRule.onNodeWithTag("ArtworkTag").assertIsDisplayed() + + player.setMediaItems(emptyList()) + composeTestRule.waitForIdle() + + composeTestRule.onNodeWithTag("ArtworkTag").assertDoesNotExist() + } } diff --git a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/modifiers/ModifiersExtensionsTest.kt b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/modifiers/ModifiersExtensionsTest.kt index e69b75881d2..ea8eb4e7160 100644 --- a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/modifiers/ModifiersExtensionsTest.kt +++ b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/modifiers/ModifiersExtensionsTest.kt @@ -16,25 +16,33 @@ package androidx.media3.ui.compose.modifiers -import androidx.compose.foundation.layout.Box +import android.annotation.SuppressLint import androidx.compose.foundation.layout.fillMaxSize +import androidx.compose.foundation.layout.size import androidx.compose.runtime.Composable +import androidx.compose.runtime.CompositionLocalProvider import androidx.compose.ui.Modifier import androidx.compose.ui.geometry.Size import androidx.compose.ui.layout.ContentScale import androidx.compose.ui.layout.Layout import androidx.compose.ui.layout.layout +import androidx.compose.ui.platform.LocalDensity import androidx.compose.ui.test.ExperimentalTestApi import androidx.compose.ui.test.v2.runComposeUiTest import androidx.compose.ui.unit.Constraints +import androidx.compose.ui.unit.Density import androidx.compose.ui.unit.IntSize -import androidx.test.ext.junit.runners.AndroidJUnit4 +import androidx.compose.ui.unit.dp import com.google.common.truth.Truth.assertThat +import com.google.testing.junit.testparameterinjector.KotlinTestParameters.testValues +import com.google.testing.junit.testparameterinjector.TestParameter +import org.junit.Assert.assertThrows import org.junit.Test import org.junit.runner.RunWith +import org.robolectric.RobolectricTestParameterInjector @OptIn(ExperimentalTestApi::class) -@RunWith(AndroidJUnit4::class) +@RunWith(RobolectricTestParameterInjector::class) class ModifiersExtensionsTest { @Test @@ -44,7 +52,7 @@ class ModifiersExtensionsTest { setContent { TestLayout( constraints = Constraints.fixed(300, 100), - sourceSizeDp = Size(200f, 100f), + aspectRatio = 2f, contentScale = ContentScale.Fit, onPlaceableMeasured = { measuredSize = it }, ) @@ -54,7 +62,18 @@ class ModifiersExtensionsTest { } @Test - fun resizeWithContentScale_unboundedWidth_scalesToFitHeight() = runComposeUiTest { + fun resizeWithContentScale_unboundedWidth_scalesToHeight_deprecatedApi( + @TestParameter + scale: ContentScale = + testValues( + ContentScale.Fit, + ContentScale.Crop, + ContentScale.FillWidth, + ContentScale.FillHeight, + ContentScale.FillBounds, + ), + @TestParameter sourceSize: Size? = testValues(Size(400f, 200f), Size(20f, 10f)), + ) = runComposeUiTest { var measuredSize: IntSize? = null setContent { @@ -66,8 +85,42 @@ class ModifiersExtensionsTest { minHeight = 0, maxHeight = 100, ), - sourceSizeDp = Size(400f, 200f), - contentScale = ContentScale.Fit, + sourceSizeDp = sourceSize, + contentScale = scale, + onPlaceableMeasured = { measuredSize = it }, + ) + } + + assertThat(measuredSize).isEqualTo(IntSize(200, 100)) + } + + @Test + fun resizeWithContentScale_unboundedWidth_scalesToHeight( + @TestParameter + scale: ContentScale = + testValues( + ContentScale.Fit, + ContentScale.Crop, + ContentScale.FillWidth, + ContentScale.FillHeight, + ContentScale.FillBounds, + ContentScale.Inside, + ContentScale.None, + ) + ) = runComposeUiTest { + var measuredSize: IntSize? = null + + setContent { + TestLayout( + constraints = + Constraints( + minWidth = 0, + maxWidth = Constraints.Infinity, + minHeight = 0, + maxHeight = 100, + ), + aspectRatio = 2f, + contentScale = scale, onPlaceableMeasured = { measuredSize = it }, ) } @@ -76,7 +129,52 @@ class ModifiersExtensionsTest { } @Test - fun resizeWithContentScale_unboundedHeight_scalesToFitWidth() = runComposeUiTest { + fun resizeWithContentScale_unboundedHeight_scalesToWidth_deprecatedApi( + @TestParameter + scale: ContentScale = + testValues( + ContentScale.Fit, + ContentScale.Crop, + ContentScale.FillWidth, + ContentScale.FillHeight, + ContentScale.FillBounds, + ), + @TestParameter sourceSize: Size? = testValues(Size(200f, 100f), Size(10f, 5f)), + ) = runComposeUiTest { + var measuredSize: IntSize? = null + + setContent { + TestLayout( + constraints = + Constraints( + minWidth = 0, + maxWidth = 100, + minHeight = 0, + maxHeight = Constraints.Infinity, + ), + sourceSizeDp = sourceSize, + contentScale = scale, + onPlaceableMeasured = { measuredSize = it }, + ) + } + + assertThat(measuredSize).isEqualTo(IntSize(100, 50)) + } + + @Test + fun resizeWithContentScale_unboundedHeight_scalesToWidth( + @TestParameter + scale: ContentScale = + testValues( + ContentScale.Fit, + ContentScale.Crop, + ContentScale.FillWidth, + ContentScale.FillHeight, + ContentScale.FillBounds, + ContentScale.Inside, + ContentScale.None, + ) + ) = runComposeUiTest { var measuredSize: IntSize? = null setContent { @@ -88,53 +186,348 @@ class ModifiersExtensionsTest { minHeight = 0, maxHeight = Constraints.Infinity, ), + aspectRatio = 2f, + contentScale = scale, + onPlaceableMeasured = { measuredSize = it }, + ) + } + + assertThat(measuredSize).isEqualTo(IntSize(100, 50)) + } + + @Test + fun resizeWithContentScale_bothUnbounded_usesSourceSize_deprecatedApi() = runComposeUiTest { + var measuredSize: IntSize? = null + + setContent { + TestLayout( + constraints = Constraints(), sourceSizeDp = Size(200f, 100f), contentScale = ContentScale.Fit, onPlaceableMeasured = { measuredSize = it }, ) } + assertThat(measuredSize).isEqualTo(IntSize(200, 100)) + } + + @Test + fun resizeWithContentScale_bothUnbounded_throws() = runComposeUiTest { + assertThrows(IllegalArgumentException::class.java) { + setContent { + TestLayout( + constraints = Constraints(), + aspectRatio = 2f, + contentScale = ContentScale.Fit, + onPlaceableMeasured = {}, + ) + } + } + } + + @Test + fun resizeWithContentScale_crop_scalesToCrop() = runComposeUiTest { + var measuredSize: IntSize? = null + var outerSize: IntSize? = null + + setContent { + TestLayout( + constraints = Constraints.fixed(120, 80), + aspectRatio = 2f, + contentScale = ContentScale.Crop, + onPlaceableMeasured = { measuredSize = it }, + onOuterMeasured = { outerSize = it }, + ) + } + + // Child scaled down, but still overflowing in width compared to the parent container + assertThat(measuredSize).isEqualTo(IntSize(160, 80)) + assertThat(outerSize).isEqualTo(IntSize(120, 80)) + } + + @Test + fun resizeWithContentScale_scaledWidthSmallerThanMinWidth_keepsScaledSize() = runComposeUiTest { + var measuredSize: IntSize? = null + var outerSize: IntSize? = null + + setContent { + TestLayout( + constraints = Constraints(minWidth = 150, maxWidth = 300, minHeight = 0, maxHeight = 50), + aspectRatio = 2f, + contentScale = ContentScale.Fit, + onPlaceableMeasured = { measuredSize = it }, + onOuterMeasured = { outerSize = it }, + ) + } + + // Even though minWidth is 150, we use the smaller maxHeight (50) to scale the source. assertThat(measuredSize).isEqualTo(IntSize(100, 50)) + assertThat(outerSize).isEqualTo(IntSize(300, 50)) } @Test - fun resizeWithContentScale_nullSourceSize_doesNotScale() = runComposeUiTest { + fun resizeWithContentScale_noneOrInside_respectsSourceSize( + @TestParameter densityValue: Float = testValues(1f, 2f), + @TestParameter scale: ContentScale = testValues(ContentScale.Inside, ContentScale.None), + ) = runComposeUiTest { + var measuredSize: IntSize? = null + var outerSize: IntSize? = null + + setContent { + CompositionLocalProvider(LocalDensity provides Density(densityValue)) { + TestLayout( + constraints = Constraints.fixed(300, 100), + sourceSizeDp = Size(10f, 5f), + contentScale = scale, + onPlaceableMeasured = { measuredSize = it }, + onOuterMeasured = { outerSize = it }, + ) + } + } + + val expectedWidth = (10 * densityValue).toInt() + val expectedHeight = (5 * densityValue).toInt() + assertThat(measuredSize).isEqualTo(IntSize(expectedWidth, expectedHeight)) + assertThat(outerSize).isEqualTo(IntSize(300, 100)) + } + + @Test + fun resizeWithContentScale_inside_behavesLikeFit() = runComposeUiTest { var measuredSize: IntSize? = null + var outerSize: IntSize? = null setContent { TestLayout( constraints = Constraints.fixed(300, 100), + aspectRatio = 2f, + contentScale = ContentScale.Inside, + onPlaceableMeasured = { measuredSize = it }, + onOuterMeasured = { outerSize = it }, + ) + } + + assertThat(measuredSize).isEqualTo(IntSize(200, 100)) + assertThat(outerSize).isEqualTo(IntSize(300, 100)) + } + + @Test + fun resizeWithContentScale_nullSourceSize_doesNotScale() = runComposeUiTest { + var measuredSize: IntSize? = null + var outerSize: IntSize? = null + + setContent { + TestLayout( + constraints = Constraints(minWidth = 10, maxWidth = 100, minHeight = 20, maxHeight = 200), sourceSizeDp = null, contentScale = ContentScale.Fit, + childModifier = Modifier, + onPlaceableMeasured = { measuredSize = it }, + onOuterMeasured = { outerSize = it }, + ) + } + + // If we only had `wrapContentSize()` but not `fillMaxSize()`, outerSize would be 10x20 + assertThat(outerSize).isEqualTo(IntSize(100, 200)) // fillMaxSize() lead to max constraints + // If we only had `fillMaxSize()` but not `wrapContentSize()`, measuredSize would be 100x200 + assertThat(measuredSize).isEqualTo(IntSize(0, 0)) // wrapContentSize() drops min constraints + } + + @Test + fun resizeWithContentScale_nullAspectRatio_doesNotScale() = runComposeUiTest { + var measuredSize: IntSize? = null + var outerSize: IntSize? = null + + setContent { + TestLayout( + constraints = Constraints(minWidth = 10, maxWidth = 100, minHeight = 20, maxHeight = 200), + aspectRatio = null, + contentScale = ContentScale.Fit, + childModifier = Modifier, + onPlaceableMeasured = { measuredSize = it }, + onOuterMeasured = { outerSize = it }, + ) + } + + // If we only had `wrapContentSize()` but not `fillMaxSize()`, outerSize would be 10x20 + assertThat(outerSize).isEqualTo(IntSize(100, 200)) // fillMaxSize() lead to max constraints + // If we only had `fillMaxSize()` but not `wrapContentSize()`, measuredSize would be 100x200 + assertThat(measuredSize).isEqualTo(IntSize(0, 0)) // wrapContentSize() drops min constraints + } + + @Test + fun resizeWithContentScale_invalidSourceSize_measuresToMinConstraints( + @TestParameter + sourceSize: Size? = + testValues( + Size(0f, 100f), + Size(100f, 0f), + Size(-10f, 20f), + Size(10f, -20f), + Size.Unspecified, + Size(Float.POSITIVE_INFINITY, 100f), + Size(100f, Float.POSITIVE_INFINITY), + Size(Float.NaN, 100f), + Size(100f, Float.NaN), + ) + ) = runComposeUiTest { + var measuredSize: IntSize? = null + var outerSize: IntSize? = null + + setContent { + TestLayout( + constraints = Constraints(minWidth = 10, maxWidth = 100, minHeight = 20, maxHeight = 200), + sourceSizeDp = sourceSize, + contentScale = ContentScale.Fit, + onPlaceableMeasured = { measuredSize = it }, + onOuterMeasured = { outerSize = it }, + ) + } + + assertThat(measuredSize).isEqualTo(IntSize(10, 20)) + assertThat(measuredSize).isEqualTo(outerSize) + } + + @Test + fun resizeWithContentScale_invalidAspectRatio_throws( + @TestParameter aspectRatio: Float = testValues(-0.5f, 0f, Float.POSITIVE_INFINITY, Float.NaN) + ) = runComposeUiTest { + assertThrows(IllegalArgumentException::class.java) { + setContent { + TestLayout( + constraints = Constraints.fixed(300, 100), + aspectRatio = aspectRatio, + contentScale = ContentScale.Fit, + onPlaceableMeasured = {}, + ) + } + } + } + + @Test + fun resizeWithContentScale_childSmallerThanScaledConstraints_respectsChildSize() = + runComposeUiTest { + var measuredSize: IntSize? = null + var outerSize: IntSize? = null + var scaledSize: IntSize? = null + + setContent { + TestLayout( + constraints = Constraints.fixed(300, 100), + aspectRatio = 2f, + contentScale = ContentScale.Fit, + childModifier = Modifier.size(50.dp, 10.dp), + onPlaceableMeasured = { measuredSize = it }, + onOuterMeasured = { outerSize = it }, + onScaledSizeCalculated = { scaledSize = it }, + ) + } + + assertThat(scaledSize).isEqualTo(IntSize(200, 100)) + assertThat(measuredSize).isEqualTo(IntSize(50, 10)) // loses the 2:1 aspect ratio + assertThat(outerSize).isEqualTo(IntSize(300, 100)) + } + + @Test + fun resizeWithContentScale_childLargerThanScaledConstraints_clampedToScaledConstraints() = + runComposeUiTest { + var measuredSize: IntSize? = null + var outerSize: IntSize? = null + var scaledSize: IntSize? = null + + setContent { + TestLayout( + constraints = Constraints.fixed(300, 100), + aspectRatio = 2f, + contentScale = ContentScale.Fit, + childModifier = Modifier.size(250.dp, 200.dp), + onPlaceableMeasured = { measuredSize = it }, + onOuterMeasured = { outerSize = it }, + onScaledSizeCalculated = { scaledSize = it }, + ) + } + + assertThat(scaledSize).isEqualTo(IntSize(200, 100)) + assertThat(measuredSize).isEqualTo(IntSize(200, 100)) // Ignoring child's larger size 250x200 + assertThat(outerSize).isEqualTo(IntSize(300, 100)) + } + + @Test + fun resizeWithContentScale_childWidthSmallerHeightLarger_clampsOnlyHeight() = runComposeUiTest { + var measuredSize: IntSize? = null + var outerSize: IntSize? = null + var scaledSize: IntSize? = null + + setContent { + TestLayout( + constraints = Constraints.fixed(300, 100), + aspectRatio = 2f, + contentScale = ContentScale.Fit, + childModifier = Modifier.size(50.dp, 150.dp), onPlaceableMeasured = { measuredSize = it }, + onOuterMeasured = { outerSize = it }, + onScaledSizeCalculated = { scaledSize = it }, ) } - // Null source size should result in no scaling, filling constraints - assertThat(measuredSize).isEqualTo(IntSize(300, 100)) + assertThat(scaledSize).isEqualTo(IntSize(200, 100)) + assertThat(measuredSize).isEqualTo(IntSize(50, 100)) // Width 50 respected, height 150 clamped + assertThat(outerSize).isEqualTo(IntSize(300, 100)) } + /** + * A helper layout to test [resizeWithContentScale] modifier sizing behavior. + * + * It uses a nested layout structure to simulate parent constraints and intercept child + * measurement: + * 1. **Outer Layout**: Receives the parent's [constraints] and measures its single child (the + * Inner Layout) with them. It reports the final outer measured size via [onOuterMeasured]. + * 2. **Inner Layout**: Simulates the content surface (like PlayerSurface) being sized. It applies + * [resizeWithContentScale] to scale incoming constraints based on [sourceSizeDp]. It then + * applies a custom [layout] modifier to intercept the scaled constraints and report the + * resulting size via [onPlaceableMeasured]. Finally, [fillMaxSize] forces the empty core + * layout to fill the scaled constraints, simulating a child that expands to fill the size + * calculated by [resizeWithContentScale]. + */ @Composable private fun TestLayout( constraints: Constraints, - sourceSizeDp: Size?, contentScale: ContentScale, + aspectRatio: Float? = null, + sourceSizeDp: Size? = null, + @SuppressLint("ModifierParameter") childModifier: Modifier = Modifier.fillMaxSize(), onPlaceableMeasured: (IntSize) -> Unit, + onOuterMeasured: (IntSize) -> Unit = {}, + onScaledSizeCalculated: (IntSize) -> Unit = {}, ) { + require(aspectRatio == null || sourceSizeDp == null) { + "Both aspectRatio and sourceSizeDp cannot be non-null" + } Layout( content = { - Box( - Modifier.resizeWithContentScale(contentScale, sourceSizeDp) - .layout { measurable, childConstraints -> - val placeable = measurable.measure(childConstraints) - onPlaceableMeasured(IntSize(placeable.width, placeable.height)) - layout(placeable.width, placeable.height) { placeable.place(0, 0) } - } - .fillMaxSize() - ) + Layout( + content = {}, + modifier = + (if (sourceSizeDp != null) { + @Suppress("DEPRECATION") Modifier.resizeWithContentScale(contentScale, sourceSizeDp) + } else { + Modifier.resizeWithContentScale(contentScale, aspectRatio) + }) + .layout { measurable, childConstraints -> + onScaledSizeCalculated( + IntSize(childConstraints.maxWidth, childConstraints.maxHeight) + ) + val placeable = measurable.measure(childConstraints) + onPlaceableMeasured(IntSize(placeable.width, placeable.height)) + layout(placeable.width, placeable.height) { placeable.place(0, 0) } + } + .then(childModifier), + ) { _, childConstraints -> + layout(childConstraints.minWidth, childConstraints.minHeight) {} + } } ) { measurables, _ -> val placeable = measurables.first().measure(constraints) + onOuterMeasured(IntSize(placeable.width, placeable.height)) layout(placeable.width, placeable.height) { placeable.place(0, 0) } } } diff --git a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/CuesStateTest.kt b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/CuesStateTest.kt new file mode 100644 index 00000000000..ae1715957dc --- /dev/null +++ b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/CuesStateTest.kt @@ -0,0 +1,146 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.ui.compose.state + +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.setValue +import androidx.compose.ui.test.ExperimentalTestApi +import androidx.compose.ui.test.v2.runComposeUiTest +import androidx.media3.common.Player +import androidx.media3.common.text.Cue +import androidx.media3.common.text.CueGroup +import androidx.media3.test.utils.FakePlayer +import androidx.test.ext.junit.runners.AndroidJUnit4 +import com.google.common.collect.ImmutableList +import com.google.common.truth.Truth.assertThat +import org.junit.Test +import org.junit.runner.RunWith + +/** Unit test for [CuesState]. */ +@OptIn(ExperimentalTestApi::class) +@RunWith(AndroidJUnit4::class) +class CuesStateTest { + + @Test + fun initialState_withNullPlayer_hasEmptyCues() = runComposeUiTest { + lateinit var state: CuesState + setContent { state = rememberCuesState(player = null) } + + assertThat(state.cues).isEmpty() + } + + @Test + fun playerWithCues_updatesStateCorrectly() = runComposeUiTest { + val cue = Cue.Builder().setText("Hello World").build() + val player = FakePlayer() + + lateinit var state: CuesState + setContent { state = rememberCuesState(player = player) } + + assertThat(state.cues).isEmpty() + + player.setCurrentCues(CueGroup(ImmutableList.of(cue), 0)) + waitForIdle() + + assertThat(state.cues).hasSize(1) + assertThat(state.cues[0].text).isEqualTo("Hello World") + } + + @Test + fun commandChanges_updateProperties() = runComposeUiTest { + val cue = Cue.Builder().setText("Hello World").build() + val player = FakePlayer() + + lateinit var state: CuesState + setContent { state = rememberCuesState(player = player) } + + player.setCurrentCues(CueGroup(ImmutableList.of(cue), 0)) + waitForIdle() + + assertThat(state.cues).hasSize(1) + + player.removeCommands(Player.COMMAND_GET_TEXT) + waitForIdle() + + assertThat(state.cues).isEmpty() + + player.addCommands(Player.COMMAND_GET_TEXT) + waitForIdle() + + assertThat(state.cues).hasSize(1) + assertThat(state.cues[0].text).isEqualTo("Hello World") + } + + @Test + fun initialState_withPopulatedPlayer_hasCuesImmediately() = runComposeUiTest { + val cue = Cue.Builder().setText("Pre-existing").build() + val player = FakePlayer() + player.setCurrentCues(CueGroup(ImmutableList.of(cue), 0)) + + lateinit var state: CuesState + setContent { state = rememberCuesState(player = player) } + + // Should be available immediately, no waitForIdle() or event needed. + assertThat(state.cues).hasSize(1) + assertThat(state.cues[0].text).isEqualTo("Pre-existing") + } + + @Test + fun playerInstanceChanged_updatesToNewPlayerCues() = runComposeUiTest { + val cue1 = Cue.Builder().setText("Player 1").build() + val player1 = FakePlayer() + player1.setCurrentCues(CueGroup(ImmutableList.of(cue1), 0)) + + val cue2 = Cue.Builder().setText("Player 2").build() + val player2 = FakePlayer() + player2.setCurrentCues(CueGroup(ImmutableList.of(cue2), 0)) + + var currentPlayer by mutableStateOf(player1) + lateinit var state: CuesState + + setContent { state = rememberCuesState(player = currentPlayer) } + + assertThat(state.cues).hasSize(1) + assertThat(state.cues[0].text).isEqualTo("Player 1") + + currentPlayer = player2 + waitForIdle() + + assertThat(state.cues).hasSize(1) + assertThat(state.cues[0].text).isEqualTo("Player 2") + } + + @Test + fun playerCuesCleared_updatesStateToEmpty() = runComposeUiTest { + val cue = Cue.Builder().setText("Hello World").build() + val player = FakePlayer() + + lateinit var state: CuesState + setContent { state = rememberCuesState(player = player) } + + player.setCurrentCues(CueGroup(ImmutableList.of(cue), 0)) + waitForIdle() + + assertThat(state.cues).hasSize(1) + + player.setCurrentCues(CueGroup.EMPTY_TIME_ZERO) + waitForIdle() + + assertThat(state.cues).isEmpty() + } +} diff --git a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/PlayPauseButtonStateTest.kt b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/PlayPauseButtonStateTest.kt index 7a0d97146df..2c838ea287b 100644 --- a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/PlayPauseButtonStateTest.kt +++ b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/PlayPauseButtonStateTest.kt @@ -22,6 +22,7 @@ import androidx.compose.runtime.mutableStateOf import androidx.compose.runtime.remember import androidx.compose.ui.test.ExperimentalTestApi import androidx.compose.ui.test.v2.runComposeUiTest +import androidx.media3.common.PlaybackException import androidx.media3.common.Player import androidx.media3.common.SimpleBasePlayer.MediaItemData import androidx.media3.test.utils.FakePlayer @@ -322,4 +323,128 @@ class PlayPauseButtonStateTest { assertThat(state.isEnabled).isTrue() } + + @Test + fun showBuffering_playerIsBuffering_isTrue() { + val player = + FakePlayer( + playbackState = Player.STATE_BUFFERING, + playlist = listOf(MediaItemData.Builder("item").build()), + ) + + val state = PlayPauseButtonState(player, displayMode = SHOW_BUFFERING_ALWAYS) + + assertThat(state.showBuffering).isTrue() + assertThat(state.showReplay).isFalse() + } + + @Test + fun showReplay_playerIsEndedWithReplayOption_isTrue() { + val player = + FakePlayer( + playbackState = Player.STATE_ENDED, + playlist = listOf(MediaItemData.Builder("item").build()), + ) + + val state = PlayPauseButtonState(player, displayMode = SHOW_REPLAY_ON_ENDED) + + assertThat(state.showReplay).isTrue() + assertThat(state.showRetry).isFalse() + assertThat(state.showBuffering).isFalse() + } + + @Test + fun showRetry_playerHasErrorWithRetryOption_isTrue() { + val player = FakePlayer(playlist = listOf(MediaItemData.Builder("item").build())) + player.setPlayerError(PlaybackException(null, null, PlaybackException.ERROR_CODE_UNSPECIFIED)) + + val state = PlayPauseButtonState(player, displayMode = SHOW_RETRY_ON_ERROR) + + assertThat(state.showRetry).isTrue() + assertThat(state.showReplay).isFalse() + assertThat(state.showBuffering).isFalse() + } + + @Test + fun showBuffering_modeAlwaysAndPlayerBuffering_isTrue() { + val player = + FakePlayer( + playbackState = Player.STATE_BUFFERING, + playlist = listOf(MediaItemData.Builder("item").build()), + ) + + val state = PlayPauseButtonState(player, displayMode = SHOW_BUFFERING_ALWAYS) + + assertThat(state.showBuffering).isTrue() + } + + @Test + fun showBuffering_modeNeverAndPlayerBuffering_isFalse() { + val player = + FakePlayer( + playbackState = Player.STATE_BUFFERING, + playlist = listOf(MediaItemData.Builder("item").build()), + ) + + val state = PlayPauseButtonState(player, displayMode = 0) + + assertThat(state.showBuffering).isFalse() + } + + @Test + fun showBuffering_modeWhenPlayingAndPlaying_isTrue() { + val player = + FakePlayer( + playbackState = Player.STATE_BUFFERING, + playWhenReady = true, + playlist = listOf(MediaItemData.Builder("item").build()), + ) + + val state = PlayPauseButtonState(player, displayMode = SHOW_BUFFERING_WHEN_PLAYING) + + assertThat(state.showPlay).isFalse() + assertThat(state.showBuffering).isTrue() + } + + @Test + fun showBuffering_modeWhenPlayingAndPaused_isFalse() { + val player = + FakePlayer( + playbackState = Player.STATE_BUFFERING, + playWhenReady = false, + playlist = listOf(MediaItemData.Builder("item").build()), + ) + + val state = PlayPauseButtonState(player, displayMode = SHOW_BUFFERING_WHEN_PLAYING) + + assertThat(state.showPlay).isTrue() + assertThat(state.showBuffering).isFalse() + } + + @Test + fun showBuffering_modeAlwaysAndPlayerReady_isFalse() { + val player = + FakePlayer( + playbackState = Player.STATE_READY, + playlist = listOf(MediaItemData.Builder("item").build()), + ) + + val state = PlayPauseButtonState(player, displayMode = SHOW_BUFFERING_ALWAYS) + + assertThat(state.showBuffering).isFalse() + } + + @Test + fun showBuffering_modeWhenPlayingAndPlayerReady_isFalse() { + val player = + FakePlayer( + playbackState = Player.STATE_READY, + playWhenReady = true, + playlist = listOf(MediaItemData.Builder("item").build()), + ) + + val state = PlayPauseButtonState(player, displayMode = SHOW_BUFFERING_WHEN_PLAYING) + + assertThat(state.showBuffering).isFalse() + } } diff --git a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/PlaybackStateTest.kt b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/PlaybackStateTest.kt new file mode 100644 index 00000000000..cd1022a8d4c --- /dev/null +++ b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/PlaybackStateTest.kt @@ -0,0 +1,170 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.ui.compose.state + +import androidx.compose.ui.test.ExperimentalTestApi +import androidx.compose.ui.test.v2.runComposeUiTest +import androidx.media3.common.PlaybackException +import androidx.media3.common.Player +import androidx.media3.common.SimpleBasePlayer.MediaItemData +import androidx.media3.test.utils.FakePlayer +import androidx.test.ext.junit.runners.AndroidJUnit4 +import com.google.common.truth.Truth.assertThat +import org.junit.Test +import org.junit.runner.RunWith + +/** Unit test for [PlaybackState]. */ +@OptIn(ExperimentalTestApi::class) +@RunWith(AndroidJUnit4::class) +class PlaybackStateTest { + + @Test + fun playbackState_whenStateIsBuffering_returnsBufferingState() { + val player = + FakePlayer( + playbackState = Player.STATE_BUFFERING, + playlist = listOf(MediaItemData.Builder("item").build()), + ) + + val state = PlaybackState(player) + + assertThat(state.playbackState).isEqualTo(Player.STATE_BUFFERING) + } + + @Test + fun playbackState_whenStateIsEnded_returnsEndedState() { + val player = + FakePlayer( + playbackState = Player.STATE_ENDED, + playlist = listOf(MediaItemData.Builder("item").build()), + ) + + val state = PlaybackState(player) + + assertThat(state.playbackState).isEqualTo(Player.STATE_ENDED) + } + + @Test + fun playerError_whenPlayerHasError_returnsError() { + val player = FakePlayer(playlist = listOf(MediaItemData.Builder("item").build())) + player.setPlayerError(PlaybackException(null, null, PlaybackException.ERROR_CODE_UNSPECIFIED)) + + val state = PlaybackState(player) + + assertThat(state.playerError).isNotNull() + } + + @Test + fun isPlaying_whenPlayerIsPlaying_returnsTrue() { + val player = + FakePlayer( + playbackState = Player.STATE_READY, + playWhenReady = true, + playlist = listOf(MediaItemData.Builder("item").build()), + ) + + val state = PlaybackState(player) + + assertThat(state.isPlaying).isTrue() + } + + @Test + fun playbackState_whenPlayerIsNull_returnsDefaultValues() { + val state = PlaybackState(player = null) + + assertThat(state.playbackState).isEqualTo(Player.STATE_IDLE) + assertThat(state.playWhenReady).isFalse() + assertThat(state.isPlaying).isFalse() + assertThat(state.playerError).isNull() + assertThat(state.playbackSuppressionReason).isEqualTo(Player.PLAYBACK_SUPPRESSION_REASON_NONE) + } + + @Test + fun playbackSuppressionReason_whenPlayerTransitionsSuppressionReason_updates() = + runComposeUiTest { + val player = FakePlayer(playlist = listOf(MediaItemData.Builder("item").build())) + lateinit var state: PlaybackState + setContent { state = rememberPlaybackState(player = player) } + + assertThat(state.playbackSuppressionReason).isEqualTo(Player.PLAYBACK_SUPPRESSION_REASON_NONE) + + player.setPlaybackSuppressionReason( + Player.PLAYBACK_SUPPRESSION_REASON_TRANSIENT_AUDIO_FOCUS_LOSS + ) + waitForIdle() + + assertThat(state.playbackSuppressionReason) + .isEqualTo(Player.PLAYBACK_SUPPRESSION_REASON_TRANSIENT_AUDIO_FOCUS_LOSS) + } + + @Test + fun playerTransitionsToBuffering_playbackStateUpdates() = runComposeUiTest { + val player = FakePlayer(playlist = listOf(MediaItemData.Builder("item").build())) + lateinit var state: PlaybackState + setContent { state = rememberPlaybackState(player = player) } + + assertThat(state.playbackState).isEqualTo(Player.STATE_IDLE) + + player.setPlaybackState(Player.STATE_BUFFERING) + waitForIdle() + + assertThat(state.playbackState).isEqualTo(Player.STATE_BUFFERING) + } + + @Test + fun playerTransitionsToEnded_playbackStateUpdates() = runComposeUiTest { + val player = FakePlayer(playlist = listOf(MediaItemData.Builder("item").build())) + lateinit var state: PlaybackState + setContent { state = rememberPlaybackState(player = player) } + + assertThat(state.playbackState).isEqualTo(Player.STATE_IDLE) + + player.setPlaybackState(Player.STATE_ENDED) + waitForIdle() + + assertThat(state.playbackState).isEqualTo(Player.STATE_ENDED) + } + + @Test + fun playerTransitionsToError_playerErrorUpdates() = runComposeUiTest { + val player = FakePlayer(playlist = listOf(MediaItemData.Builder("item").build())) + lateinit var state: PlaybackState + setContent { state = rememberPlaybackState(player = player) } + + assertThat(state.playerError).isNull() + + player.setPlayerError(PlaybackException(null, null, PlaybackException.ERROR_CODE_UNSPECIFIED)) + waitForIdle() + + assertThat(state.playerError).isNotNull() + } + + @Test + fun playerTransitionsToPlaying_isPlayingUpdates() = runComposeUiTest { + val player = FakePlayer(playlist = listOf(MediaItemData.Builder("item").build())) + lateinit var state: PlaybackState + setContent { state = rememberPlaybackState(player = player) } + + assertThat(state.isPlaying).isFalse() + + player.setPlaybackState(Player.STATE_READY) + player.play() + waitForIdle() + + assertThat(state.isPlaying).isTrue() + } +} diff --git a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/PlayerStateObserverTest.kt b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/PlayerStateObserverTest.kt index e0d1ef71098..7efe9fd86d8 100644 --- a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/PlayerStateObserverTest.kt +++ b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/PlayerStateObserverTest.kt @@ -107,4 +107,145 @@ class PlayerStateObserverTest { assertThat(stateUpdateCount).isEqualTo(initialCount) } + + @Test + fun createPlayerStateObserver_withEvents_triggersInitialStateUpdateWithConfiguredEvents() = + runTest { + val player = createReadyPlayerWithTwoItems() + var stateUpdateCount = 0 + var receivedEvents: Player.Events? = null + val unused = + player.observeState( + Player.EVENT_PLAYBACK_STATE_CHANGED, + Player.EVENT_PLAY_WHEN_READY_CHANGED, + ) { _, events -> + stateUpdateCount++ + receivedEvents = events + } + + assertThat(stateUpdateCount).isEqualTo(1) + assertThat(receivedEvents?.size()).isEqualTo(2) + assertThat(receivedEvents?.contains(Player.EVENT_PLAYBACK_STATE_CHANGED)).isTrue() + assertThat(receivedEvents?.contains(Player.EVENT_PLAY_WHEN_READY_CHANGED)).isTrue() + } + + @Test + fun observe_withEvents_triggersInitialStateUpdateWithConfiguredEvents() = runTest { + val player = createReadyPlayerWithTwoItems() + var stateUpdateCount = 0 + var receivedEvents: Player.Events? = null + val stateObserver = + player.observeState( + Player.EVENT_PLAYBACK_STATE_CHANGED, + Player.EVENT_PLAY_WHEN_READY_CHANGED, + ) { _, events -> + stateUpdateCount++ + receivedEvents = events + } + val initialCount = stateUpdateCount + receivedEvents = null + + launch(backgroundScope.coroutineContext) { stateObserver.observe() } + testScheduler.runCurrent() + + assertThat(stateUpdateCount).isEqualTo(initialCount + 1) + assertThat(receivedEvents?.size()).isEqualTo(2) + assertThat(receivedEvents?.contains(Player.EVENT_PLAYBACK_STATE_CHANGED)).isTrue() + assertThat(receivedEvents?.contains(Player.EVENT_PLAY_WHEN_READY_CHANGED)).isTrue() + } + + @Test + fun createAndObserve_withInitialStateUpdater_triggersInitialStateUpdaterOnly() = runTest { + val player = createReadyPlayerWithTwoItems() + var initialUpdateCount = 0 + var eventUpdateCount = 0 + val stateObserver = + player.observeState( + Player.EVENT_PLAYBACK_STATE_CHANGED, + Player.EVENT_PLAY_WHEN_READY_CHANGED, + initialStateUpdater = { initialUpdateCount++ }, + stateUpdater = { _, _ -> eventUpdateCount++ }, + ) + + assertThat(initialUpdateCount).isEqualTo(1) + assertThat(eventUpdateCount).isEqualTo(0) + + launch(backgroundScope.coroutineContext) { stateObserver.observe() } + testScheduler.runCurrent() + + assertThat(initialUpdateCount).isEqualTo(2) + assertThat(eventUpdateCount).isEqualTo(0) + } + + @Test + fun observe_registeredPlayerEventWithEvents_triggersStateUpdateWithEvents() = runTest { + val player = createReadyPlayerWithTwoItems() + var stateUpdateCount = 0 + var receivedEvents: Player.Events? = null + val stateObserver = + player.observeState( + Player.EVENT_PLAYBACK_STATE_CHANGED, + Player.EVENT_PLAY_WHEN_READY_CHANGED, + ) { _, events -> + stateUpdateCount++ + receivedEvents = events + } + launch(backgroundScope.coroutineContext) { stateObserver.observe() } + testScheduler.runCurrent() + val initialCount = stateUpdateCount + + player.playWhenReady = false + shadowOf(Looper.getMainLooper()).idle() + testScheduler.runCurrent() + + assertThat(stateUpdateCount).isEqualTo(initialCount + 1) + assertThat(receivedEvents?.contains(Player.EVENT_PLAY_WHEN_READY_CHANGED)).isTrue() + } + + @Test + fun observe_mixedEvents_filtersOutUnregisteredEvents() = runTest { + val player = createReadyPlayerWithTwoItems() + var receivedEvents: Player.Events? = null + val stateObserver = + player.observeState( + Player.EVENT_PLAYBACK_STATE_CHANGED, + Player.EVENT_PLAY_WHEN_READY_CHANGED, + ) { _, events -> + receivedEvents = events + } + launch(backgroundScope.coroutineContext) { stateObserver.observe() } + testScheduler.runCurrent() + receivedEvents = null + + player.playWhenReady = false + player.shuffleModeEnabled = true + shadowOf(Looper.getMainLooper()).idle() + testScheduler.runCurrent() + + assertThat(receivedEvents?.contains(Player.EVENT_PLAY_WHEN_READY_CHANGED)).isTrue() + assertThat(receivedEvents?.contains(Player.EVENT_IS_PLAYING_CHANGED)).isFalse() + assertThat(receivedEvents?.contains(Player.EVENT_SHUFFLE_MODE_ENABLED_CHANGED)).isFalse() + } + + @Test + fun observe_unregisteredPlayerEventWithEvents_doesNotTriggerStateUpdate() = runTest { + val player = createReadyPlayerWithTwoItems() + var stateUpdateCount = 0 + val stateObserver = + player.observeState( + Player.EVENT_PLAYBACK_STATE_CHANGED, + Player.EVENT_PLAY_WHEN_READY_CHANGED, + ) { _, _ -> + stateUpdateCount++ + } + launch(backgroundScope.coroutineContext) { stateObserver.observe() } + testScheduler.runCurrent() + val initialCount = stateUpdateCount + + player.shuffleModeEnabled = true + shadowOf(Looper.getMainLooper()).idle() + testScheduler.runCurrent() + + assertThat(stateUpdateCount).isEqualTo(initialCount) + } } diff --git a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/PresentationStateTest.kt b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/PresentationStateTest.kt index bd1b3c7e54a..61fad228a35 100644 --- a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/PresentationStateTest.kt +++ b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/PresentationStateTest.kt @@ -19,10 +19,12 @@ package androidx.media3.ui.compose.state import androidx.compose.runtime.LaunchedEffect import androidx.compose.runtime.MutableIntState import androidx.compose.runtime.MutableState +import androidx.compose.runtime.getValue import androidx.compose.runtime.key import androidx.compose.runtime.mutableIntStateOf import androidx.compose.runtime.mutableStateOf import androidx.compose.runtime.remember +import androidx.compose.runtime.setValue import androidx.compose.ui.geometry.Size import androidx.compose.ui.test.ExperimentalTestApi import androidx.compose.ui.test.v2.runComposeUiTest @@ -31,6 +33,9 @@ import androidx.media3.common.Format import androidx.media3.common.ForwardingPlayer import androidx.media3.common.MediaItem import androidx.media3.common.MediaLibraryInfo +import androidx.media3.common.MimeTypes +import androidx.media3.common.MimeTypes.AUDIO_AAC +import androidx.media3.common.MimeTypes.TEXT_VTT import androidx.media3.common.MimeTypes.VIDEO_VP9 import androidx.media3.common.Player import androidx.media3.common.SimpleBasePlayer.MediaItemData @@ -58,7 +63,9 @@ class PresentationStateTest { assertThat(state.coverSurface).isTrue() assertThat(state.keepContentOnReset).isFalse() - assertThat(state.videoSizeDp).isNull() + @Suppress("DEPRECATION") assertThat(state.videoSizeDp).isNull() + assertThat(state.videoAspectRatio).isNull() + assertThat(state.showArtwork).isFalse() } @Test @@ -69,7 +76,8 @@ class PresentationStateTest { lateinit var state: PresentationState setContent { state = rememberPresentationState(player) } - assertThat(state.videoSizeDp).isEqualTo(Size(1920f, 1080f)) + @Suppress("DEPRECATION") assertThat(state.videoSizeDp).isEqualTo(Size(1920f, 1080f)) + assertThat(state.videoAspectRatio).isEqualTo(1920f / 1080f) } @Test @@ -80,46 +88,48 @@ class PresentationStateTest { lateinit var state: PresentationState setContent { state = rememberPresentationState(player) } - assertThat(state.videoSizeDp).isEqualTo(Size(1920f, 1080f)) + @Suppress("DEPRECATION") assertThat(state.videoSizeDp).isEqualTo(Size(1920f, 1080f)) + assertThat(state.videoAspectRatio).isEqualTo(1920f / 1080f) } @Test - fun rememberPresentationState_recomposition_hasSizeOnFirstPass() = runComposeUiTest { + fun rememberPresentationState_recomposition_hasAspectRatioOnFirstPass() = runComposeUiTest { val player = FakePlayer(playbackState = Player.STATE_IDLE) player.setVideoSize(VideoSize(1920, 1080)) - val observedSizes = mutableSetOf() + val observedAspectRatios = mutableSetOf() setContent { val state = rememberPresentationState(player) - // Capture the value of videoSizeDp exactly as it is seen during the composition pass. + // Capture the value of videoAspectRatio exactly as it is seen during the composition pass. // This happens before LaunchedEffect gets a chance to run. - observedSizes.add(state.videoSizeDp) + observedAspectRatios.add(state.videoAspectRatio) } - // Assert that all composition passes had the correct size. - assertThat(observedSizes).containsExactly(Size(1920f, 1080f)) + // Assert that all composition passes had the correct aspect ratio. + assertThat(observedAspectRatios).containsExactly(1920f / 1080f) } @Test - fun rememberPresentationState_recomposition_syncsVideoSizeImmediately() = runComposeUiTest { - val player = FakePlayer() - player.setVideoSize(VideoSize(1920, 1080)) + fun rememberPresentationState_recomposition_syncsVideoAspectRatioImmediately() = + runComposeUiTest { + val player = FakePlayer() + player.setVideoSize(VideoSize(1920, 1080)) - lateinit var recomposeKey: MutableIntState - lateinit var state: PresentationState + lateinit var recomposeKey: MutableIntState + lateinit var state: PresentationState - setContent { - recomposeKey = remember { mutableIntStateOf(0) } - key(recomposeKey.intValue) { state = rememberPresentationState(player) } - } + setContent { + recomposeKey = remember { mutableIntStateOf(0) } + key(recomposeKey.intValue) { state = rememberPresentationState(player) } + } - assertThat(state.videoSizeDp).isEqualTo(Size(1920f, 1080f)) + assertThat(state.videoAspectRatio).isEqualTo(1920f / 1080f) - recomposeKey.intValue = 1 - waitForIdle() + recomposeKey.intValue = 1 + waitForIdle() - assertThat(state.videoSizeDp).isEqualTo(Size(1920f, 1080f)) - } + assertThat(state.videoAspectRatio).isEqualTo(1920f / 1080f) + } @Test fun playerChangesVideoSizeBeforeEventListenerRegisters_observeGetsTheLatestValues_uiInSync() = @@ -135,7 +145,7 @@ class PresentationStateTest { state = rememberPresentationState(player) } - assertThat(state.videoSizeDp).isEqualTo(Size(480f, 360f)) + assertThat(state.videoAspectRatio).isEqualTo(480f / 360f) assertThat(state.coverSurface).isTrue() assertThat(state.keepContentOnReset).isFalse() } @@ -494,4 +504,243 @@ class PresentationStateTest { // Should remain visible (shutter open) because we are in the same window on the same timeline assertThat(state.coverSurface).isFalse() } + + @Test + fun videoOnly_shutterClosedUntilFirstFrame() = runComposeUiTest { + val group = + Tracks.Group( + /* mediaTrackGroup = */ TrackGroup(Format.Builder().setSampleMimeType(VIDEO_VP9).build()), + /* adaptiveSupported = */ true, + /* trackSupport = */ intArrayOf(C.FORMAT_HANDLED), + /* trackSelected = */ booleanArrayOf(true), + ) + val player = + FakePlayer( + playlist = listOf(MediaItemData.Builder("uid").setTracks(Tracks(listOf(group))).build()) + ) + + lateinit var state: PresentationState + setContent { state = rememberPresentationState(player) } + + // With video, shutter starts closed + assertThat(state.coverSurface).isTrue() + + player.renderFirstFrame(true) + waitForIdle() + + // First frame opens the shutter + assertThat(state.coverSurface).isFalse() + } + + @Test + fun videoAndText_shutterClosedUntilFirstFrame() = runComposeUiTest { + val videoGroup = + Tracks.Group( + /* mediaTrackGroup = */ TrackGroup(Format.Builder().setSampleMimeType(VIDEO_VP9).build()), + /* adaptiveSupported = */ true, + /* trackSupport = */ intArrayOf(C.FORMAT_HANDLED), + /* trackSelected = */ booleanArrayOf(true), + ) + val textGroup = + Tracks.Group( + /* mediaTrackGroup = */ TrackGroup(Format.Builder().setSampleMimeType(TEXT_VTT).build()), + /* adaptiveSupported = */ true, + /* trackSupport = */ intArrayOf(C.FORMAT_HANDLED), + /* trackSelected = */ booleanArrayOf(true), + ) + val player = + FakePlayer( + playlist = + listOf( + MediaItemData.Builder("uid").setTracks(Tracks(listOf(videoGroup, textGroup))).build() + ) + ) + + lateinit var state: PresentationState + setContent { state = rememberPresentationState(player) } + + // Even with text, because there is video, shutter starts closed + assertThat(state.coverSurface).isTrue() + + player.renderFirstFrame(true) + waitForIdle() + + // First frame opens the shutter + assertThat(state.coverSurface).isFalse() + } + + @Test + fun audioOnly_shutterClosed() = runComposeUiTest { + val audioGroup = + Tracks.Group( + /* mediaTrackGroup = */ TrackGroup(Format.Builder().setSampleMimeType(AUDIO_AAC).build()), + /* adaptiveSupported = */ true, + /* trackSupport = */ intArrayOf(C.FORMAT_HANDLED), + /* trackSelected = */ booleanArrayOf(true), + ) + val player = + FakePlayer( + playlist = + listOf(MediaItemData.Builder("uid").setTracks(Tracks(listOf(audioGroup))).build()) + ) + + lateinit var state: PresentationState + setContent { state = rememberPresentationState(player) } + + // Audio only, no video, no text -> shutter stays closed + assertThat(state.coverSurface).isTrue() + } + + @Test + fun audioAndText_shutterOpen() = runComposeUiTest { + val audioGroup = + Tracks.Group( + /* mediaTrackGroup = */ TrackGroup(Format.Builder().setSampleMimeType(AUDIO_AAC).build()), + /* adaptiveSupported = */ true, + /* trackSupport = */ intArrayOf(C.FORMAT_HANDLED), + /* trackSelected = */ booleanArrayOf(true), + ) + val textGroup = + Tracks.Group( + /* mediaTrackGroup = */ TrackGroup(Format.Builder().setSampleMimeType(TEXT_VTT).build()), + /* adaptiveSupported = */ true, + /* trackSupport = */ intArrayOf(C.FORMAT_HANDLED), + /* trackSelected = */ booleanArrayOf(true), + ) + val player = + FakePlayer( + playlist = + listOf( + MediaItemData.Builder("uid").setTracks(Tracks(listOf(audioGroup, textGroup))).build() + ) + ) + + lateinit var state: PresentationState + setContent { state = rememberPresentationState(player) } + + // Audio and text -> shutter should be immediately open to show the subtitles over black + assertThat(state.coverSurface).isFalse() + assertThat(state.showArtwork).isTrue() + } + + @Test + fun showArtwork_videoTrackUnsupportedButSelected_isFalse() = runComposeUiTest { + val videoTrack = + Tracks.Group( + TrackGroup(Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build()), + /* adaptiveSupported= */ true, + /* trackSupport= */ intArrayOf(C.FORMAT_UNSUPPORTED_TYPE), + /* trackSelected= */ booleanArrayOf(true), + ) + val player = + FakePlayer( + playlist = + listOf(MediaItemData.Builder("First").setTracks(Tracks(listOf(videoTrack))).build()) + ) + + lateinit var state: PresentationState + setContent { state = rememberPresentationState(player) } + + assertThat(state.showArtwork).isFalse() + } + + @Test + fun showArtwork_videoTrackUnsupportedAndNotSelected_isTrue() = runComposeUiTest { + val videoTrack = + Tracks.Group( + TrackGroup(Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build()), + /* adaptiveSupported= */ true, + /* trackSupport= */ intArrayOf(C.FORMAT_UNSUPPORTED_TYPE), + /* trackSelected= */ booleanArrayOf(false), + ) + val player = + FakePlayer( + playlist = + listOf(MediaItemData.Builder("First").setTracks(Tracks(listOf(videoTrack))).build()) + ) + + lateinit var state: PresentationState + setContent { state = rememberPresentationState(player) } + + assertThat(state.showArtwork).isTrue() + } + + @Test + fun showArtwork_videoTrackSupportedAndSelected_isFalse() = runComposeUiTest { + val videoTrack = + Tracks.Group( + TrackGroup(Format.Builder().setSampleMimeType(MimeTypes.VIDEO_H264).build()), + /* adaptiveSupported= */ true, + /* trackSupport= */ intArrayOf(C.FORMAT_HANDLED), + /* trackSelected= */ booleanArrayOf(true), + ) + val player = + FakePlayer( + playlist = + listOf(MediaItemData.Builder("First").setTracks(Tracks(listOf(videoTrack))).build()) + ) + + lateinit var state: PresentationState + setContent { state = rememberPresentationState(player) } + + assertThat(state.showArtwork).isFalse() + } + + @Test + fun showArtwork_playerDetachedWithKeepContentOnReset_artworkStaysVisible() = runComposeUiTest { + val audioTrack = + Tracks.Group( + TrackGroup(Format.Builder().setSampleMimeType(AUDIO_AAC).build()), + /* adaptiveSupported= */ true, + /* trackSupport= */ intArrayOf(C.FORMAT_HANDLED), + /* trackSelected= */ booleanArrayOf(true), + ) + val player = + FakePlayer( + playlist = + listOf(MediaItemData.Builder("First").setTracks(Tracks(listOf(audioTrack))).build()) + ) + + var attachedPlayer by mutableStateOf(player) + lateinit var state: PresentationState + setContent { + state = rememberPresentationState(player = attachedPlayer, keepContentOnReset = true) + } + + assertThat(state.showArtwork).isTrue() + + attachedPlayer = null + waitForIdle() + + assertThat(state.showArtwork).isTrue() + } + + @Test + fun showArtwork_playerDetachedWithoutKeepContentOnReset_artworkHidden() = runComposeUiTest { + val audioTrack = + Tracks.Group( + TrackGroup(Format.Builder().setSampleMimeType(AUDIO_AAC).build()), + /* adaptiveSupported= */ true, + /* trackSupport= */ intArrayOf(C.FORMAT_HANDLED), + /* trackSelected= */ booleanArrayOf(true), + ) + val player = + FakePlayer( + playlist = + listOf(MediaItemData.Builder("First").setTracks(Tracks(listOf(audioTrack))).build()) + ) + + var attachedPlayer by mutableStateOf(player) + lateinit var state: PresentationState + setContent { + state = rememberPresentationState(player = attachedPlayer, keepContentOnReset = false) + } + + assertThat(state.showArtwork).isTrue() + + attachedPlayer = null + waitForIdle() + + assertThat(state.showArtwork).isFalse() + } } diff --git a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/ProgressStateWithTickCountTest.kt b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/ProgressStateWithTickCountTest.kt index 4d0b6e4bbac..7e5a8441aac 100644 --- a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/ProgressStateWithTickCountTest.kt +++ b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/ProgressStateWithTickCountTest.kt @@ -32,7 +32,11 @@ import androidx.media3.common.Player.STATE_READY import androidx.media3.common.SimpleBasePlayer.MediaItemData import androidx.media3.test.utils.FakePlayer import androidx.media3.ui.compose.testutils.advancePrecisely +import androidx.media3.ui.compose.testutils.createReadyPlayerPlayingAd +import androidx.media3.ui.compose.testutils.createReadyPlayerWithClippedItem +import androidx.media3.ui.compose.testutils.createReadyPlayerWithMultiPeriodItem import androidx.media3.ui.compose.testutils.createReadyPlayerWithSingleItem +import androidx.media3.ui.compose.testutils.createReadyPlayerWithUnknownPeriodDuration import androidx.test.ext.junit.runners.AndroidJUnit4 import com.google.common.truth.Truth.assertThat import org.junit.Test @@ -52,6 +56,75 @@ class ProgressStateWithTickCountTest { assertThat(state.bufferedPositionProgress).isEqualTo(0f) assertThat(state.changingProgressEnabled).isFalse() assertThat(state.durationMs).isEqualTo(C.TIME_UNSET) + assertThat(state.originalDurationMs).isEqualTo(C.TIME_UNSET) + } + + @Test + fun originalDurationMs_clippedItem_returnsOriginalDuration() = runComposeUiTest { + val player = createReadyPlayerWithClippedItem() + lateinit var state: ProgressStateWithTickCount + + setContent { state = rememberProgressStateWithTickCount(player, totalTickCount = 10) } + + assertThat(state.durationMs).isEqualTo(5_000L) + assertThat(state.originalDurationMs).isEqualTo(10_000L) + } + + @Test + fun originalDurationMs_multiPeriodItem_returnsDuration() = runComposeUiTest { + val player = createReadyPlayerWithMultiPeriodItem() + lateinit var state: ProgressStateWithTickCount + + setContent { state = rememberProgressStateWithTickCount(player, totalTickCount = 10) } + + assertThat(state.durationMs).isEqualTo(90_000L) + assertThat(state.originalDurationMs).isEqualTo(90_000L) + } + + @Test + fun originalDurationMs_timelineCommandUnavailable_returnsDuration() = runComposeUiTest { + val player = createReadyPlayerWithClippedItem() + player.removeCommands(Player.COMMAND_GET_TIMELINE) + lateinit var state: ProgressStateWithTickCount + + setContent { state = rememberProgressStateWithTickCount(player, totalTickCount = 10) } + + assertThat(state.durationMs).isEqualTo(5_000L) + assertThat(state.originalDurationMs).isEqualTo(5_000L) + } + + @Test + fun originalDurationMs_currentMediaItemCommandUnavailable_returnsTimeUnset() = runComposeUiTest { + val player = createReadyPlayerWithClippedItem() + player.removeCommands(COMMAND_GET_CURRENT_MEDIA_ITEM) + lateinit var state: ProgressStateWithTickCount + + setContent { state = rememberProgressStateWithTickCount(player, totalTickCount = 10) } + + assertThat(state.durationMs).isEqualTo(C.TIME_UNSET) + assertThat(state.originalDurationMs).isEqualTo(C.TIME_UNSET) + } + + @Test + fun originalDurationMs_unknownPeriodDuration_returnsDuration() = runComposeUiTest { + val player = createReadyPlayerWithUnknownPeriodDuration() + lateinit var state: ProgressStateWithTickCount + + setContent { state = rememberProgressStateWithTickCount(player, totalTickCount = 10) } + + assertThat(state.durationMs).isEqualTo(10_000L) + assertThat(state.originalDurationMs).isEqualTo(10_000L) + } + + @Test + fun originalDurationMs_playingAd_returnsAdDuration() = runComposeUiTest { + val player = createReadyPlayerPlayingAd() + lateinit var state: ProgressStateWithTickCount + + setContent { state = rememberProgressStateWithTickCount(player, totalTickCount = 10) } + + assertThat(state.durationMs).isEqualTo(1_000L) + assertThat(state.originalDurationMs).isEqualTo(1_000L) } @Test @@ -155,6 +228,7 @@ class ProgressStateWithTickCountTest { assertThat(state.currentPositionProgress).isEqualTo(0f) assertThat(state.bufferedPositionProgress).isEqualTo(0f) assertThat(state.durationMs).isEqualTo(C.TIME_UNSET) + assertThat(state.originalDurationMs).isEqualTo(C.TIME_UNSET) } @Test @@ -171,6 +245,7 @@ class ProgressStateWithTickCountTest { setContent { state = rememberProgressStateWithTickCount(player, totalTickCount = 10) } assertThat(state.currentPositionProgress).isEqualTo(0f) assertThat(state.durationMs).isEqualTo(C.TIME_UNSET) + assertThat(state.originalDurationMs).isEqualTo(C.TIME_UNSET) mainClock.advancePrecisely(2345) assertThat(state.currentPositionProgress).isEqualTo(0f) @@ -184,6 +259,7 @@ class ProgressStateWithTickCountTest { .isWithin(17) // Allowing for ~1 frame of clock advancement during waitForIdle. .of(4690) assertThat(state.durationMs).isEqualTo(10_000) + assertThat(state.originalDurationMs).isEqualTo(10_000) assertThat(state.currentPositionProgress).isEqualTo(0.5f) } @@ -204,6 +280,7 @@ class ProgressStateWithTickCountTest { assertThat(state.changingProgressEnabled).isFalse() assertThat(player.duration).isEqualTo(C.TIME_UNSET) assertThat(state.durationMs).isEqualTo(C.TIME_UNSET) + assertThat(state.originalDurationMs).isEqualTo(C.TIME_UNSET) mainClock.advancePrecisely(durationUnknownUpdate - 100) @@ -329,6 +406,7 @@ class ProgressStateWithTickCountTest { assertThat(state.currentPositionProgress).isEqualTo(0f) assertThat(state.bufferedPositionProgress).isEqualTo(0f) assertThat(state.durationMs).isEqualTo(C.TIME_UNSET) + assertThat(state.originalDurationMs).isEqualTo(C.TIME_UNSET) // Wait for any pending updates to verify the state stays the same and is not blocked on the // main thread. @@ -337,6 +415,7 @@ class ProgressStateWithTickCountTest { assertThat(state.currentPositionProgress).isEqualTo(0f) assertThat(state.bufferedPositionProgress).isEqualTo(0f) assertThat(state.durationMs).isEqualTo(C.TIME_UNSET) + assertThat(state.originalDurationMs).isEqualTo(C.TIME_UNSET) } @Test @@ -352,6 +431,7 @@ class ProgressStateWithTickCountTest { assertThat(state.currentPositionProgress).isEqualTo(1f) assertThat(state.bufferedPositionProgress).isEqualTo(1f) assertThat(state.durationMs).isEqualTo(10_000) + assertThat(state.originalDurationMs).isEqualTo(10_000) player.setPlaybackState(Player.STATE_ENDED) @@ -359,6 +439,7 @@ class ProgressStateWithTickCountTest { assertThat(state.currentPositionProgress).isEqualTo(1f) assertThat(state.bufferedPositionProgress).isEqualTo(1f) assertThat(state.durationMs).isEqualTo(10_000) + assertThat(state.originalDurationMs).isEqualTo(10_000) waitForIdle() @@ -366,6 +447,7 @@ class ProgressStateWithTickCountTest { assertThat(state.currentPositionProgress).isEqualTo(1f) assertThat(state.bufferedPositionProgress).isEqualTo(1f) assertThat(state.durationMs).isEqualTo(10_000) + assertThat(state.originalDurationMs).isEqualTo(10_000) } @Test diff --git a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/ProgressStateWithTickIntervalTest.kt b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/ProgressStateWithTickIntervalTest.kt index ead7d838750..9cea38f8f13 100644 --- a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/ProgressStateWithTickIntervalTest.kt +++ b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/ProgressStateWithTickIntervalTest.kt @@ -29,7 +29,11 @@ import androidx.media3.common.Player.COMMAND_GET_CURRENT_MEDIA_ITEM import androidx.media3.common.SimpleBasePlayer.MediaItemData import androidx.media3.test.utils.FakePlayer import androidx.media3.ui.compose.testutils.advancePrecisely +import androidx.media3.ui.compose.testutils.createReadyPlayerPlayingAd +import androidx.media3.ui.compose.testutils.createReadyPlayerWithClippedItem +import androidx.media3.ui.compose.testutils.createReadyPlayerWithMultiPeriodItem import androidx.media3.ui.compose.testutils.createReadyPlayerWithSingleItem +import androidx.media3.ui.compose.testutils.createReadyPlayerWithUnknownPeriodDuration import androidx.test.ext.junit.runners.AndroidJUnit4 import com.google.common.truth.Truth.assertThat import org.junit.Test @@ -51,8 +55,77 @@ class ProgressStateWithTickIntervalTest { assertThat(state.currentPositionMs).isEqualTo(0L) assertThat(state.bufferedPositionMs).isEqualTo(0L) assertThat(state.durationMs).isEqualTo(C.TIME_UNSET) + assertThat(state.originalDurationMs).isEqualTo(C.TIME_UNSET) } + @Test + fun originalDurationMs_clippedItem_returnsOriginalDuration() = runComposeUiTest { + val player = createReadyPlayerWithClippedItem() + lateinit var state: ProgressStateWithTickInterval + + setContent { state = rememberProgressStateWithTickInterval(player, tickIntervalMs = 1000) } + + assertThat(state.durationMs).isEqualTo(5_000L) + assertThat(state.originalDurationMs).isEqualTo(10_000L) + } + + @Test + fun originalDurationMs_multiPeriodItem_returnsDuration() = runComposeUiTest { + val player = createReadyPlayerWithMultiPeriodItem() + lateinit var state: ProgressStateWithTickInterval + + setContent { state = rememberProgressStateWithTickInterval(player, tickIntervalMs = 1000) } + + assertThat(state.durationMs).isEqualTo(90_000L) + assertThat(state.originalDurationMs).isEqualTo(90_000L) + } + + @Test + fun originalDurationMs_timelineCommandUnavailable_returnsDuration() = runComposeUiTest { + val player = createReadyPlayerWithClippedItem() + player.removeCommands(Player.COMMAND_GET_TIMELINE) + lateinit var state: ProgressStateWithTickInterval + + setContent { state = rememberProgressStateWithTickInterval(player, tickIntervalMs = 1000) } + + assertThat(state.durationMs).isEqualTo(5_000L) + assertThat(state.originalDurationMs).isEqualTo(5_000L) + } + + @Test + fun originalDurationMs_currentMediaItemCommandUnavailable_returnsTimeUnset() = runComposeUiTest { + val player = createReadyPlayerWithClippedItem() + player.removeCommands(COMMAND_GET_CURRENT_MEDIA_ITEM) + lateinit var state: ProgressStateWithTickInterval + + setContent { state = rememberProgressStateWithTickInterval(player, tickIntervalMs = 1000) } + + assertThat(state.durationMs).isEqualTo(C.TIME_UNSET) + assertThat(state.originalDurationMs).isEqualTo(C.TIME_UNSET) + } + + @Test + fun originalDurationMs_unknownPeriodDuration_returnsDuration() = runComposeUiTest { + val player = createReadyPlayerWithUnknownPeriodDuration() + lateinit var state: ProgressStateWithTickInterval + + setContent { state = rememberProgressStateWithTickInterval(player, tickIntervalMs = 1000) } + + assertThat(state.durationMs).isEqualTo(10_000L) + assertThat(state.originalDurationMs).isEqualTo(10_000L) + } + + @Test + fun originalDurationMs_playingAd_returnsAdDuration() = runComposeUiTest { + val player = createReadyPlayerPlayingAd() + lateinit var state: ProgressStateWithTickInterval + + setContent { state = rememberProgressStateWithTickInterval(player, tickIntervalMs = 1000) } + + assertThat(state.durationMs).isEqualTo(1_000L) + assertThat(state.originalDurationMs).isEqualTo(1_000L) + } + @Test fun progressUpdatingOnTheSecondMark_positionChangesByOneSecond() = runComposeUiTest { val player = createReadyPlayerWithSingleItem() @@ -221,6 +294,7 @@ class ProgressStateWithTickIntervalTest { assertThat(state.currentPositionMs).isEqualTo(0) assertThat(state.bufferedPositionMs).isEqualTo(0) assertThat(state.durationMs).isEqualTo(C.TIME_UNSET) + assertThat(state.originalDurationMs).isEqualTo(C.TIME_UNSET) player.addCommands(COMMAND_GET_CURRENT_MEDIA_ITEM) waitForIdle() @@ -229,6 +303,7 @@ class ProgressStateWithTickIntervalTest { assertThat(state.currentPositionMs).isEqualTo(4000) assertThat(state.bufferedPositionMs).isEqualTo(4000) assertThat(state.durationMs).isEqualTo(10_000) + assertThat(state.originalDurationMs).isEqualTo(10_000) player.removeCommands(COMMAND_GET_CURRENT_MEDIA_ITEM) waitForIdle() @@ -237,6 +312,7 @@ class ProgressStateWithTickIntervalTest { assertThat(state.currentPositionMs).isEqualTo(0) assertThat(state.bufferedPositionMs).isEqualTo(0) assertThat(state.durationMs).isEqualTo(C.TIME_UNSET) + assertThat(state.originalDurationMs).isEqualTo(C.TIME_UNSET) } @Test @@ -246,6 +322,7 @@ class ProgressStateWithTickIntervalTest { setContent { state = rememberProgressStateWithTickInterval(player, tickIntervalMs = 1000) } assertThat(state.durationMs).isEqualTo(10_000) + assertThat(state.originalDurationMs).isEqualTo(10_000) assertThat(player.duration).isEqualTo(10_000) } @@ -262,12 +339,14 @@ class ProgressStateWithTickIntervalTest { lateinit var state: ProgressStateWithTickInterval setContent { state = rememberProgressStateWithTickInterval(player, tickIntervalMs = 1000) } assertThat(state.durationMs).isEqualTo(C.TIME_UNSET) + assertThat(state.originalDurationMs).isEqualTo(C.TIME_UNSET) mainClock.advancePrecisely(2345) player.setDuration("SingleItem", 10_000) waitForIdle() assertThat(state.durationMs).isEqualTo(10_000) + assertThat(state.originalDurationMs).isEqualTo(10_000) assertThat(player.duration).isEqualTo(10_000) assertThat(player.currentPosition).isEqualTo(2345) } @@ -284,6 +363,7 @@ class ProgressStateWithTickIntervalTest { lateinit var state: ProgressStateWithTickInterval setContent { state = rememberProgressStateWithTickInterval(player, tickIntervalMs = 100) } assertThat(state.durationMs).isEqualTo(C.TIME_UNSET) + assertThat(state.originalDurationMs).isEqualTo(C.TIME_UNSET) mainClock.advancePrecisely(200) player.setBufferedPositionMs(123) @@ -304,6 +384,7 @@ class ProgressStateWithTickIntervalTest { lateinit var state: ProgressStateWithTickInterval setContent { state = rememberProgressStateWithTickInterval(player, tickIntervalMs = 100) } assertThat(state.durationMs).isEqualTo(C.TIME_UNSET) + assertThat(state.originalDurationMs).isEqualTo(C.TIME_UNSET) assertThat(state.currentPositionMs).isEqualTo(0) assertThat(state.bufferedPositionMs).isEqualTo(0) @@ -312,6 +393,7 @@ class ProgressStateWithTickIntervalTest { mainClock.advancePrecisely(200) assertThat(state.durationMs).isEqualTo(C.TIME_UNSET) + assertThat(state.originalDurationMs).isEqualTo(C.TIME_UNSET) assertThat(state.currentPositionMs).isEqualTo(0) assertThat(state.bufferedPositionMs).isEqualTo(0) } @@ -327,6 +409,7 @@ class ProgressStateWithTickIntervalTest { // Check state before change to ENDED assertThat(state.durationMs).isEqualTo(10_000) + assertThat(state.originalDurationMs).isEqualTo(10_000) assertThat(state.currentPositionMs).isEqualTo(10_000) assertThat(state.bufferedPositionMs).isEqualTo(10_000) @@ -334,6 +417,7 @@ class ProgressStateWithTickIntervalTest { // Immediately after the change before running the playback state update assertThat(state.durationMs).isEqualTo(10_000) + assertThat(state.originalDurationMs).isEqualTo(10_000) assertThat(state.currentPositionMs).isEqualTo(10_000) assertThat(state.bufferedPositionMs).isEqualTo(10_000) @@ -341,6 +425,7 @@ class ProgressStateWithTickIntervalTest { // After completing any pending updates to ensure the main thread is not blocked. assertThat(state.durationMs).isEqualTo(10_000) + assertThat(state.originalDurationMs).isEqualTo(10_000) assertThat(state.currentPositionMs).isEqualTo(10_000) assertThat(state.bufferedPositionMs).isEqualTo(10_000) } diff --git a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/TrackSelectionParametersStateTest.kt b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/TrackSelectionParametersStateTest.kt new file mode 100644 index 00000000000..ea2860070dd --- /dev/null +++ b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/TrackSelectionParametersStateTest.kt @@ -0,0 +1,194 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.ui.compose.state + +import androidx.compose.runtime.LaunchedEffect +import androidx.compose.ui.test.ExperimentalTestApi +import androidx.compose.ui.test.v2.runComposeUiTest +import androidx.media3.common.Player +import androidx.media3.common.TrackSelectionParameters +import androidx.media3.common.Tracks +import androidx.media3.common.util.UnstableApi +import androidx.media3.ui.compose.testutils.createPlayerWithTracks +import androidx.media3.ui.compose.testutils.createTestTracks +import androidx.test.ext.junit.runners.AndroidJUnit4 +import com.google.common.truth.Truth.assertThat +import org.junit.Test +import org.junit.runner.RunWith + +/** Unit test for [TrackSelectionParametersState]. */ +@OptIn(ExperimentalTestApi::class) +@RunWith(AndroidJUnit4::class) +@UnstableApi +class TrackSelectionParametersStateTest { + + @Test + fun constructor_nullPlayer_initializesWithDefaults() { + val state = TrackSelectionParametersState(player = null) + + assertThat(state.tracks).isEqualTo(Tracks.EMPTY) + assertThat(state.trackSelectionParameters).isEqualTo(TrackSelectionParameters.DEFAULT) + } + + @Test + fun constructor_withPlayer_initializesFromPlayerState() { + val initialTracks = createTestTracks() + val initialParams = + TrackSelectionParameters.DEFAULT.buildUpon().setMaxVideoSize(1024, 768).build() + + val player = createPlayerWithTracks(tracks = initialTracks, params = initialParams) + val state = TrackSelectionParametersState(player) + + assertThat(state.tracks).isEqualTo(initialTracks) + assertThat(state.trackSelectionParameters).isEqualTo(initialParams) + } + + @Test + fun constructor_withCommandGetTracksUnavailable_initializesTracksToEmpty() { + val initialTracks = createTestTracks() + val player = createPlayerWithTracks(tracks = initialTracks) + player.removeCommands(Player.COMMAND_GET_TRACKS) + + val state = TrackSelectionParametersState(player) + + assertThat(state.tracks).isEqualTo(Tracks.EMPTY) + } + + @Test + fun updateTrackSelectionParameters_withNewParams_updatesPlayerAndLocalState() = runComposeUiTest { + val player = createPlayerWithTracks() + lateinit var state: TrackSelectionParametersState + setContent { state = rememberTrackSelectionParametersState(player) } + + val newParams = + TrackSelectionParameters.DEFAULT.buildUpon().setPreferredVideoMimeTypes("video/avc").build() + + state.updateTrackSelectionParameters(newParams) + waitForIdle() + + assertThat(state.trackSelectionParameters).isEqualTo(newParams) + assertThat(player.trackSelectionParameters.preferredVideoMimeTypes).containsExactly("video/avc") + } + + @Test + fun updateTrackSelectionParameters_nullPlayer_doesNotUpdateState() = runComposeUiTest { + lateinit var state: TrackSelectionParametersState + setContent { state = rememberTrackSelectionParametersState(null) } + val newParams = + TrackSelectionParameters.DEFAULT.buildUpon().setPreferredAudioLanguage("fr").build() + + state.updateTrackSelectionParameters(newParams) + waitForIdle() + + assertThat(state.trackSelectionParameters).isEqualTo(TrackSelectionParameters.DEFAULT) + } + + @Test + fun updateTrackSelectionParameters_withCommandSetTrackSelectionParametersUnavailable_doesNotApplyToPlayer() = + runComposeUiTest { + val player = createPlayerWithTracks() + player.removeCommands(Player.COMMAND_SET_TRACK_SELECTION_PARAMETERS) + lateinit var state: TrackSelectionParametersState + setContent { state = rememberTrackSelectionParametersState(player) } + + val newParams = + TrackSelectionParameters.DEFAULT.buildUpon().setPreferredVideoMimeTypes("video/avc").build() + + state.updateTrackSelectionParameters(newParams) + waitForIdle() + + assertThat(player.trackSelectionParameters).isEqualTo(TrackSelectionParameters.DEFAULT) + assertThat(state.trackSelectionParameters).isEqualTo(TrackSelectionParameters.DEFAULT) + } + + @Test + fun rememberTrackSelectionParametersState_nullPlayer_initializesWithDefaults() = + runComposeUiTest { + lateinit var state: TrackSelectionParametersState + setContent { state = rememberTrackSelectionParametersState(null) } + + assertThat(state.tracks).isEqualTo(Tracks.EMPTY) + assertThat(state.trackSelectionParameters).isEqualTo(TrackSelectionParameters.DEFAULT) + } + + @Test + fun rememberTrackSelectionParametersState_withPlayer_initializesFromPlayerState() = + runComposeUiTest { + val initialTracks = createTestTracks() + val initialParams = + TrackSelectionParameters.DEFAULT.buildUpon().setMaxVideoSize(1024, 768).build() + val player = createPlayerWithTracks(tracks = initialTracks, params = initialParams) + + lateinit var state: TrackSelectionParametersState + setContent { state = rememberTrackSelectionParametersState(player) } + + assertThat(state).isNotNull() + assertThat(state.tracks).isEqualTo(initialTracks) + assertThat(state.trackSelectionParameters).isEqualTo(initialParams) + } + + @Test + fun rememberTrackSelectionParametersState_playerChangesParametersBeforeObserve_uiInSync() = + runComposeUiTest { + val player = createPlayerWithTracks() + + lateinit var state: TrackSelectionParametersState + setContent { + LaunchedEffect(player) { + player.setTrackSelectionParameters( + TrackSelectionParameters.DEFAULT.buildUpon().setMaxVideoSize(1280, 720).build() + ) + } + state = rememberTrackSelectionParametersState(player = player) + } + + waitForIdle() + assertThat(state.trackSelectionParameters.maxVideoWidth).isEqualTo(1280) + } + + @Test + fun rememberTrackSelectionParametersState_withCommandGetTracksUnavailable_tracksRemainEmpty() = + runComposeUiTest { + val player = createPlayerWithTracks(tracks = createTestTracks()) + lateinit var state: TrackSelectionParametersState + setContent { state = rememberTrackSelectionParametersState(player) } + + player.removeCommands(Player.COMMAND_GET_TRACKS) + waitForIdle() + + assertThat(state.tracks).isEqualTo(Tracks.EMPTY) + } + + @Test + fun canSetTrackSelectionParameters_commandChanges_updatesState() = runComposeUiTest { + val player = createPlayerWithTracks() + lateinit var state: TrackSelectionParametersState + setContent { state = rememberTrackSelectionParametersState(player) } + + assertThat(state.canSetTrackSelectionParameters).isTrue() + + player.removeCommands(Player.COMMAND_SET_TRACK_SELECTION_PARAMETERS) + waitForIdle() + + assertThat(state.canSetTrackSelectionParameters).isFalse() + + player.addCommands(Player.COMMAND_SET_TRACK_SELECTION_PARAMETERS) + waitForIdle() + + assertThat(state.canSetTrackSelectionParameters).isTrue() + } +} diff --git a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/TrackSelectionStateTest.kt b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/TrackSelectionStateTest.kt new file mode 100644 index 00000000000..bb17fc7501a --- /dev/null +++ b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/state/TrackSelectionStateTest.kt @@ -0,0 +1,624 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.ui.compose.state + +import androidx.compose.runtime.MutableState +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.ui.test.ExperimentalTestApi +import androidx.compose.ui.test.v2.runComposeUiTest +import androidx.media3.common.C +import androidx.media3.common.TrackSelectionParameters +import androidx.media3.common.Tracks +import androidx.media3.ui.compose.testutils.createPlayerWithTracks +import androidx.media3.ui.compose.testutils.createTestTracks +import androidx.media3.ui.compose.testutils.createTestTracksWithLanguages +import androidx.test.ext.junit.runners.AndroidJUnit4 +import com.google.common.truth.Truth.assertThat +import org.junit.Test +import org.junit.runner.RunWith + +/** Unit test for [TrackSelectionState]. */ +@OptIn(ExperimentalTestApi::class) +@RunWith(AndroidJUnit4::class) +class TrackSelectionStateTest { + + @Test + fun constructor_noTracksOfTargetType_initializesEmptyOptions() { + val player = createPlayerWithTracks() + val paramState = TrackSelectionParametersState(player) + + val state = TrackSelectionState(paramState, C.TRACK_TYPE_VIDEO) + + assertThat(state.canSetTrackSelectionParameters).isTrue() + assertThat(state.tracks).isEqualTo(paramState.tracks) + assertThat(state.trackSelectionParameters).isEqualTo(paramState.trackSelectionParameters) + assertThat(state.selectionOptions).isEmpty() + assertThat(state.selectedOption).isNull() + } + + @Test + fun selectionOptions_otherTrackTypesExistButNotTargetType_returnsEmpty() { + val player = + createPlayerWithTracks(tracks = createTestTracksWithLanguages("audio/mp4", "en", "fr", "es")) + val paramState = TrackSelectionParametersState(player) + val state = TrackSelectionState(paramState, C.TRACK_TYPE_VIDEO, TRACK_FILTER_RESOLUTION) + + val options = state.selectionOptions + + assertThat(options).isEmpty() + } + + @Test + fun selectionOptions_tracksOfTargetTypeExist_returnsOptions() { + val player = createPlayerWithTracks(tracks = createTestTracks()) + val paramState = TrackSelectionParametersState(player) + val state = TrackSelectionState(paramState, C.TRACK_TYPE_VIDEO, TRACK_FILTER_RESOLUTION) + + val options = state.selectionOptions + + assertThat(options).isNotEmpty() + } + + @Test + fun selectionOptions_trackFilterNone_returnsEmpty() { + val state = createTrackSelectionState(trackFilters = 0) + + val options = state.selectionOptions + + assertThat(options).isEmpty() + } + + @Test + fun selectionOptions_withResolutionFilter_filtersOptionsByResolution() { + val state = createTrackSelectionState(trackFilters = TRACK_FILTER_RESOLUTION) + + val options = state.selectionOptions + + assertThat(options).hasSize(4) + val resolutions = options.map { it.width to it.height } + assertThat(resolutions).containsExactly(1920 to 1080, 1280 to 720, 854 to 480, 640 to 360) + } + + @Test + fun selectionOptions_withMimeTypeFilter_filtersOptionsByMimeType() { + val state = createTrackSelectionState(trackFilters = TRACK_FILTER_MIME_TYPE) + + val options = state.selectionOptions + + assertThat(options).hasSize(3) + val mimeTypes = options.map { it.sampleMimeType } + assertThat(mimeTypes).containsExactly("video/mp4", "video/x-matroska", "video/quicktime") + } + + @Test + fun selectionOptions_withAudioLanguageFilter_filtersOptionsByLanguage() { + val state = + createTrackSelectionState( + trackType = C.TRACK_TYPE_AUDIO, + trackFilters = TRACK_FILTER_LANGUAGE, + tracks = createTestTracksWithLanguages("audio/mp4", "en", "fr", "es"), + ) + + val options = state.selectionOptions + + assertThat(options).hasSize(3) + val languages = options.map { it.language } + assertThat(languages).containsExactly("en", "fr", "es") + } + + @Test + fun selectionOptions_withTextLanguageFilter_filtersOptionsByLanguage() { + val state = + createTrackSelectionState( + trackType = C.TRACK_TYPE_TEXT, + trackFilters = TRACK_FILTER_LANGUAGE, + tracks = createTestTracksWithLanguages("text/vtt", "en", "es"), + ) + + val options = state.selectionOptions + + assertThat(options).hasSize(2) + val languages = options.map { it.language } + assertThat(languages).containsExactly("en", "es") + } + + @Test + fun selectionOptions_withMultipleFilters_returnsFlattenedCombinedOptions() { + val state = + createTrackSelectionState(trackFilters = TRACK_FILTER_RESOLUTION or TRACK_FILTER_MIME_TYPE) + + val options = state.selectionOptions + + assertThat(options).hasSize(7) + val combinations = options.map { (it.width to it.height) to it.sampleMimeType } + assertThat(combinations) + .containsExactly( + (1920 to 1080) to "video/mp4", + (1920 to 1080) to "video/x-matroska", + (1280 to 720) to "video/mp4", + (1280 to 720) to "video/x-matroska", + (854 to 480) to "video/mp4", + (854 to 480) to "video/x-matroska", + (640 to 360) to "video/quicktime", + ) + } + + @Test + fun selectionOptions_withMissingFilterMetadata_includesInvalidTracksAsNull() { + val state = + createTrackSelectionState( + trackFilters = TRACK_FILTER_RESOLUTION, + tracks = createTestTracks(includeMissingDataTrack = true), + ) + + val options = state.selectionOptions + + assertThat(options).hasSize(5) + val hasNullOption = options.any { it.width == null || it.height == null } + assertThat(hasNullOption).isTrue() + } + + @Test + fun selectionOptions_withUnsupportedTracks_marksOptionsAsUnsupported() { + val unsupportedStates = + intArrayOf( + C.FORMAT_HANDLED, + C.FORMAT_HANDLED, + C.FORMAT_HANDLED, + C.FORMAT_HANDLED, + C.FORMAT_HANDLED, + C.FORMAT_HANDLED, + C.FORMAT_EXCEEDS_CAPABILITIES, + ) + val tracks = createTestTracks(videoHandledStates = unsupportedStates) + val state = createTrackSelectionState(tracks = tracks, trackFilters = TRACK_FILTER_RESOLUTION) + val videoGroup = tracks.groups.first { it.type == C.TRACK_TYPE_VIDEO } + val unsupportedFormat = videoGroup.getTrackFormat(6) + + val options = state.selectionOptions + val option = options.first { + it.width == unsupportedFormat.width && it.height == unsupportedFormat.height + } + + assertThat(option.isSupported).isFalse() + } + + @Test + fun selectionOptions_withSelectedTracks_marksOptionsAsSelected() { + val tracks = createTestTracks(selectedVideoTrackIndex = 0) + val state = createTrackSelectionState(trackFilters = TRACK_FILTER_RESOLUTION, tracks = tracks) + + val options = state.selectionOptions + val selectionMap = options.associate { it.width to it.isSelected } + + val expectedMap = mapOf(1920 to true, 1280 to false, 854 to false, 640 to false) + assertThat(selectionMap).isEqualTo(expectedMap) + } + + @Test + fun selectedOption_textNoPreferredLanguagesOrOverrides_selectsAutoOption() { + val player = createPlayerWithTracks(tracks = createTestTracksWithLanguages("text/vtt", "en")) + val paramState = TrackSelectionParametersState(player) + val state = + TrackSelectionState( + paramState, + C.TRACK_TYPE_TEXT, + TRACK_FILTER_LANGUAGE, + hasAutoOption = true, + ) + + assertThat(state.selectedOption?.onOffState).isEqualTo(TRACK_SELECTION_AUTO) + } + + @Test + fun selectedOption_textWithPreferredLanguages_selectsLanguageOption() { + val initialParams = + TrackSelectionParameters.DEFAULT.buildUpon().setPreferredTextLanguage("en").build() + val player = + createPlayerWithTracks( + tracks = createTestTracksWithLanguages("text/vtt", "en", selectedLanguage = "en"), + params = initialParams, + ) + val paramState = TrackSelectionParametersState(player) + val state = + TrackSelectionState( + paramState, + C.TRACK_TYPE_TEXT, + TRACK_FILTER_LANGUAGE, + hasAutoOption = true, + ) + + assertThat(state.selectedOption?.onOffState).isEqualTo(TRACK_SELECTION_ON) + assertThat(state.selectedOption?.language).isEqualTo("en") + } + + @Test + fun selectedOption_audio_withPreferredLanguages_selectsLanguageOption() { + val initialParams = + TrackSelectionParameters.DEFAULT.buildUpon().setPreferredAudioLanguage("en").build() + val player = + createPlayerWithTracks( + tracks = createTestTracksWithLanguages("audio/mp4", "en", "fr", selectedLanguage = "en"), + params = initialParams, + ) + val paramState = TrackSelectionParametersState(player) + val state = + TrackSelectionState( + paramState, + C.TRACK_TYPE_AUDIO, + TRACK_FILTER_LANGUAGE, + hasAutoOption = true, + ) + + assertThat(state.selectedOption?.onOffState).isEqualTo(TRACK_SELECTION_ON) + assertThat(state.selectedOption?.language).isEqualTo("en") + } + + @Test + fun selectedOption_audio_defaultParams_selectsAutoOption() { + val player = createPlayerWithTracks(tracks = createTestTracks()) + val paramState = TrackSelectionParametersState(player) + val state = + TrackSelectionState( + paramState, + C.TRACK_TYPE_AUDIO, + TRACK_FILTER_LANGUAGE, + hasAutoOption = true, + ) + + assertThat(state.selectedOption?.onOffState).isEqualTo(TRACK_SELECTION_AUTO) + } + + @Test + fun selectedOption_video_withMimeTypeConstraintButOnlyResolutionFilterEnabled_selectsAutoOption() { + val initialParams = + TrackSelectionParameters.DEFAULT.buildUpon().setPreferredVideoMimeType("video/mp4").build() + val player = createPlayerWithTracks(tracks = createTestTracks(), params = initialParams) + val paramState = TrackSelectionParametersState(player) + val state = + TrackSelectionState( + paramState, + C.TRACK_TYPE_VIDEO, + TRACK_FILTER_RESOLUTION, + hasAutoOption = true, + ) + + assertThat(state.selectedOption?.onOffState).isEqualTo(TRACK_SELECTION_AUTO) + } + + @Test + fun selectedOption_video_withMimeTypeConstraintAndMimeTypeFilterEnabled_selectsMimeTypeOption() { + val initialParams = + TrackSelectionParameters.DEFAULT.buildUpon().setPreferredVideoMimeType("video/mp4").build() + val player = + createPlayerWithTracks( + tracks = createTestTracks(selectedVideoTrackIndex = 0), + params = initialParams, + ) + val paramState = TrackSelectionParametersState(player) + val state = + TrackSelectionState( + paramState, + C.TRACK_TYPE_VIDEO, + TRACK_FILTER_MIME_TYPE, + hasAutoOption = true, + ) + + assertThat(state.selectedOption?.onOffState).isEqualTo(TRACK_SELECTION_ON) + assertThat(state.selectedOption?.sampleMimeType).isEqualTo("video/mp4") + } + + @Test + fun selectedOption_withHasAutoOptionTrue_defaultIsAutoOption() { + val state = + createTrackSelectionState(trackFilters = TRACK_FILTER_RESOLUTION, hasAutoOption = true) + + val selectedOption = state.selectedOption + + assertThat(selectedOption).isNotNull() + assertThat(selectedOption?.onOffState).isEqualTo(TRACK_SELECTION_AUTO) + assertThat(selectedOption?.isSelected).isTrue() + } + + @Test + fun selectedOption_withHasAutoOptionFalse_returnsActiveTrackOption() { + val tracks = createTestTracks(selectedVideoTrackIndex = 0) + val state = + createTrackSelectionState( + trackFilters = TRACK_FILTER_RESOLUTION, + tracks = tracks, + hasAutoOption = false, + ) + + val selectedOption = state.selectedOption + + assertThat(selectedOption).isNotNull() + assertThat(selectedOption?.width).isEqualTo(1920) + } + + @Test + fun selectedOption_onSelectOption_updatesSelectedOptionProperty() = runComposeUiTest { + val player = createPlayerWithTracks(tracks = createTestTracks()) + lateinit var state: TrackSelectionState + setContent { + state = rememberTrackSelectionState(player, C.TRACK_TYPE_VIDEO, TRACK_FILTER_RESOLUTION) + } + val options = state.selectionOptions + val option1080p = options.first { it.width == 1920 } + + state.selectOption(option1080p) + player.setTracks(createTestTracks(selectedVideoTrackIndex = 0)) + waitForIdle() + + assertThat(state.selectedOption).isEqualTo(state.selectionOptions.first { it.width == 1920 }) + assertThat(state.selectedOption?.isSelected).isTrue() + } + + @Test + fun selectedOption_onExternalParametersUpdate_updatesSync() = runComposeUiTest { + val player = createPlayerWithTracks(tracks = createTestTracks()) + lateinit var state: TrackSelectionState + setContent { + state = rememberTrackSelectionState(player, C.TRACK_TYPE_VIDEO, TRACK_FILTER_RESOLUTION) + } + val newParams = TrackSelectionParameters.DEFAULT.buildUpon().setMaxVideoSize(1920, 1080).build() + + player.setTrackSelectionParameters(newParams) + player.setTracks(createTestTracks(selectedVideoTrackIndex = 0)) + waitForIdle() + + val expectedOption = state.selectionOptions.first { it.width == 1920 } + assertThat(state.selectedOption).isEqualTo(expectedOption) + assertThat(state.selectedOption?.isSelected).isTrue() + } + + @Test + fun selectOption_offOption_disablesTrackTypeInPlayerParams() { + val player = createPlayerWithTracks() + val paramState = TrackSelectionParametersState(player) + val state = TrackSelectionState(paramState, C.TRACK_TYPE_VIDEO, hasOffOption = true) + val offOption = state.selectionOptions.first { it.onOffState == TRACK_SELECTION_OFF } + + state.selectOption(offOption) + + assertThat(player.trackSelectionParameters.disabledTrackTypes) + .containsExactly(C.TRACK_TYPE_VIDEO) + } + + @Test + fun selectOption_onOption_enablesTrackTypeInPlayerParams() { + val initialParams = + TrackSelectionParameters.DEFAULT.buildUpon() + .setTrackTypeDisabled(C.TRACK_TYPE_VIDEO, true) + .build() + val player = createPlayerWithTracks(tracks = createTestTracks(), params = initialParams) + val paramState = TrackSelectionParametersState(player) + val state = + TrackSelectionState( + paramState, + C.TRACK_TYPE_VIDEO, + TRACK_FILTER_RESOLUTION, + hasOffOption = true, + ) + val onOption = state.selectionOptions.first { it.onOffState == TRACK_SELECTION_ON } + + state.selectOption(onOption) + + assertThat(player.trackSelectionParameters.disabledTrackTypes) + .doesNotContain(C.TRACK_TYPE_VIDEO) + } + + @Test + fun selectOption_textToAuto_setsSelectTextByDefaultFalseAndClearsOverrides() = runComposeUiTest { + val initialParams = + TrackSelectionParameters.DEFAULT.buildUpon() + .setPreferredTextLanguage("en") + .setSelectTextByDefault(true) + .build() + val player = + createPlayerWithTracks( + tracks = createTestTracksWithLanguages("text/vtt", "en", selectedLanguage = "en"), + params = initialParams, + ) + lateinit var state: TrackSelectionState + setContent { + state = + rememberTrackSelectionState( + player = player, + trackType = C.TRACK_TYPE_TEXT, + trackFilters = TRACK_FILTER_LANGUAGE, + hasAutoOption = true, + ) + } + val autoOption = state.selectionOptions.first { it.onOffState == TRACK_SELECTION_AUTO } + + state.selectOption(autoOption) + waitForIdle() + + assertThat(player.trackSelectionParameters.selectTextByDefault).isFalse() + assertThat(player.trackSelectionParameters.preferredTextLanguages).isEmpty() + assertThat(state.selectedOption?.onOffState).isEqualTo(TRACK_SELECTION_AUTO) + } + + @Test + fun selectOption_textWithLanguage_setsPreferredTextLanguage() = runComposeUiTest { + val player = + createPlayerWithTracks( + tracks = createTestTracksWithLanguages("text/vtt", "en", selectedLanguage = "en") + ) + lateinit var state: TrackSelectionState + setContent { + state = + rememberTrackSelectionState( + player = player, + trackType = C.TRACK_TYPE_TEXT, + trackFilters = TRACK_FILTER_LANGUAGE, + hasAutoOption = true, + ) + } + val onOption = state.selectionOptions.first { it.onOffState == TRACK_SELECTION_ON } + + state.selectOption(onOption) + waitForIdle() + + assertThat(player.trackSelectionParameters.preferredTextLanguages).containsExactly("en") + assertThat(player.trackSelectionParameters.selectTextByDefault).isFalse() + assertThat(state.selectedOption?.onOffState).isEqualTo(TRACK_SELECTION_ON) + assertThat(state.selectedOption?.language).isEqualTo("en") + } + + @Test + fun selectOption_withValidOption_appliesPreferencesToPlayerParams() { + val player = createPlayerWithTracks(tracks = createTestTracks()) + val paramState = TrackSelectionParametersState(player) + val state = + TrackSelectionState( + paramState, + C.TRACK_TYPE_VIDEO, + TRACK_FILTER_RESOLUTION or TRACK_FILTER_MIME_TYPE, + ) + val options = state.selectionOptions + val option1080pMp4 = options.first { + it.width == 1920 && it.height == 1080 && it.sampleMimeType == "video/mp4" + } + + state.selectOption(option1080pMp4) + + assertThat(player.trackSelectionParameters.maxVideoWidth).isEqualTo(1920) + assertThat(player.trackSelectionParameters.maxVideoHeight).isEqualTo(1080) + assertThat(player.trackSelectionParameters.preferredVideoMimeTypes).containsExactly("video/mp4") + } + + @Test + fun selectOption_optionNotInSelectionOptions_ignoresSelection() { + val player = createPlayerWithTracks() + val paramState = TrackSelectionParametersState(player) + val state = TrackSelectionState(paramState, C.TRACK_TYPE_VIDEO, TRACK_FILTER_RESOLUTION) + val unknownOption = + TrackSelectionOption( + isSupported = true, + isSelected = false, + width = 1920, + height = 1080, + sampleMimeType = "video/mp4", + ) + + state.selectOption(unknownOption) + + assertThat(player.trackSelectionParameters.maxVideoWidth).isEqualTo(Int.MAX_VALUE) + assertThat(player.trackSelectionParameters.maxVideoHeight).isEqualTo(Int.MAX_VALUE) + } + + @Test + fun selectOption_null_clearsOverridesAndConstraints() { + val initialParams = + TrackSelectionParameters.DEFAULT.buildUpon().setPreferredAudioLanguage("en").build() + val player = + createPlayerWithTracks( + tracks = createTestTracksWithLanguages("audio/mp4", "en", "fr", selectedLanguage = "en"), + params = initialParams, + ) + val paramState = TrackSelectionParametersState(player) + val state = TrackSelectionState(paramState, C.TRACK_TYPE_AUDIO, TRACK_FILTER_LANGUAGE) + + state.selectOption(null) + + assertThat(player.trackSelectionParameters.preferredAudioLanguages).isEmpty() + } + + @Test + fun rememberTrackSelectionState_nullPlayer_initializesEmptyOptionsAndCommandFalse() = + runComposeUiTest { + lateinit var state: TrackSelectionState + + setContent { + state = rememberTrackSelectionState(null, C.TRACK_TYPE_VIDEO, TRACK_FILTER_RESOLUTION) + } + waitForIdle() + + assertThat(state).isNotNull() + assertThat(state.canSetTrackSelectionParameters).isFalse() + assertThat(state.selectionOptions).isEmpty() + } + + @Test + fun rememberTrackSelectionState_withPlayer_initializesOptionsAndCommandTrue() = runComposeUiTest { + val player = createPlayerWithTracks(tracks = createTestTracks()) + lateinit var state: TrackSelectionState + + setContent { + state = rememberTrackSelectionState(player, C.TRACK_TYPE_VIDEO, TRACK_FILTER_RESOLUTION) + } + waitForIdle() + + assertThat(state).isNotNull() + assertThat(state.trackType).isEqualTo(C.TRACK_TYPE_VIDEO) + assertThat(state.canSetTrackSelectionParameters).isTrue() + assertThat(state.selectionOptions).isNotEmpty() + } + + @Test + fun rememberTrackSelectionState_playerBecomesNullRoundTrip_stateUpdatesCorrectly() = + runComposeUiTest { + val player = createPlayerWithTracks(tracks = createTestTracks()) + lateinit var isPlayerNull: MutableState + lateinit var state: TrackSelectionState + setContent { + isPlayerNull = remember { mutableStateOf(false) } + state = + rememberTrackSelectionState( + if (isPlayerNull.value) null else player, + C.TRACK_TYPE_VIDEO, + TRACK_FILTER_RESOLUTION, + ) + } + + assertThat(state.canSetTrackSelectionParameters).isTrue() + assertThat(state.selectionOptions).isNotEmpty() + + isPlayerNull.value = true + waitForIdle() + assertThat(state.canSetTrackSelectionParameters).isFalse() + assertThat(state.selectionOptions).isEmpty() + + isPlayerNull.value = false + waitForIdle() + assertThat(state.canSetTrackSelectionParameters).isTrue() + assertThat(state.selectionOptions).isNotEmpty() + } + + private fun createTrackSelectionState( + trackType: @C.TrackType Int = C.TRACK_TYPE_VIDEO, + trackFilters: @TrackFilter Int = 0, + tracks: Tracks = createTestTracks(), + hasOffOption: Boolean = false, + hasAutoOption: Boolean = false, + hasOnOption: Boolean = true, + ): TrackSelectionState { + val player = createPlayerWithTracks(tracks = tracks) + val trackSelectionParametersState = TrackSelectionParametersState(player) + return TrackSelectionState( + trackSelectionParametersState, + trackType, + trackFilters, + hasOffOption, + hasAutoOption, + hasOnOption, + ) + } +} diff --git a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/testutils/player.kt b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/testutils/player.kt index 16edd8be2d3..b7e9835ddc7 100644 --- a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/testutils/player.kt +++ b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/testutils/player.kt @@ -16,8 +16,13 @@ package androidx.media3.ui.compose.testutils +import androidx.media3.common.AdPlaybackState +import androidx.media3.common.C import androidx.media3.common.Player.STATE_READY import androidx.media3.common.SimpleBasePlayer.MediaItemData +import androidx.media3.common.SimpleBasePlayer.PeriodData +import androidx.media3.common.TrackSelectionParameters +import androidx.media3.common.Tracks import androidx.media3.test.utils.FakePlayer internal fun createReadyPlayerWithTwoItems(): FakePlayer = @@ -37,3 +42,145 @@ internal fun createReadyPlayerWithSingleItem(durationUs: Long = 10_000_000L): Fa playWhenReady = true, playlist = listOf(MediaItemData.Builder("SingleItem").setDurationUs(durationUs).build()), ) + +internal fun createPlayerWithTracks( + tracks: Tracks = Tracks.EMPTY, + params: TrackSelectionParameters = TrackSelectionParameters.DEFAULT, +): FakePlayer { + val mediaItemData = MediaItemData.Builder("id").setTracks(tracks).build() + return FakePlayer(playlist = listOf(mediaItemData), trackSelectionParameters = params) +} + +/** + * Creates a ready player with a single media item clipped from 10s to the range 2s-7s by default. + */ +internal fun createReadyPlayerWithClippedItem( + originalDurationUs: Long = 10_000_000L, + clipStartUs: Long = 2_000_000L, + clipEndUs: Long = 7_000_000L, +): FakePlayer { + require(clipStartUs >= 0 && clipEndUs >= clipStartUs && originalDurationUs >= clipEndUs) { + "Invalid clipping boundaries" + } + + val windowDurationUs = clipEndUs - clipStartUs + val positionInFirstPeriodUs = clipStartUs + val periodDurationUs = clipEndUs + + return FakePlayer( + playbackState = STATE_READY, + playWhenReady = true, + playlist = + listOf( + MediaItemData.Builder("SingleItem") + .setDurationUs(windowDurationUs) + .setPositionInFirstPeriodUs(positionInFirstPeriodUs) + .setPeriods( + listOf( + PeriodData.Builder("Period") + .setDurationUs(periodDurationUs) + .setOriginalDurationUs(originalDurationUs) + .build() + ) + ) + .build() + ), + ) +} + +/** + * Creates a ready player with a single 90s media item made of two periods of 30s and 60s by + * default. + */ +internal fun createReadyPlayerWithMultiPeriodItem( + period0DurationUs: Long = 30_000_000L, + period1DurationUs: Long = 60_000_000L, +): FakePlayer { + val totalDurationUs = period0DurationUs + period1DurationUs + + return FakePlayer( + playbackState = STATE_READY, + playWhenReady = true, + playlist = + listOf( + MediaItemData.Builder("SingleItem") + .setDurationUs(totalDurationUs) + .setPeriods( + listOf( + PeriodData.Builder("Period0").setDurationUs(period0DurationUs).build(), + PeriodData.Builder("Period1").setDurationUs(period1DurationUs).build(), + ) + ) + .build() + ), + ) +} + +/** + * Creates a ready player with a single media item, 10s by default, whose only period has an unknown + * duration. + */ +internal fun createReadyPlayerWithUnknownPeriodDuration( + durationUs: Long = 10_000_000L +): FakePlayer = + FakePlayer( + playbackState = STATE_READY, + playWhenReady = true, + playlist = + listOf( + MediaItemData.Builder("SingleItem") + .setDurationUs(durationUs) + .setPeriods(listOf(PeriodData.Builder("Period").setDurationUs(C.TIME_UNSET).build())) + .build() + ), + ) + +/** + * Creates a ready player with a single media item, clipped from 10s to the range 2s-7s by default, + * that is currently playing an ad, 1s by default. + */ +internal fun createReadyPlayerPlayingAd( + originalDurationUs: Long = 10_000_000L, + clipStartUs: Long = 2_000_000L, + clipEndUs: Long = 7_000_000L, + adDurationUs: Long = 1_000_000L, +): FakePlayer { + require( + clipStartUs >= 0 && + clipEndUs >= clipStartUs && + originalDurationUs >= clipEndUs && + adDurationUs > 0 + ) { + "Invalid clipping boundaries or ad duration" + } + + val windowDurationUs = clipEndUs - clipStartUs + val positionInFirstPeriodUs = clipStartUs + val periodDurationUs = clipEndUs + + return FakePlayer( + playbackState = STATE_READY, + playWhenReady = true, + playlist = + listOf( + MediaItemData.Builder("SingleItem") + .setDurationUs(windowDurationUs) + .setPositionInFirstPeriodUs(positionInFirstPeriodUs) + .setPeriods( + listOf( + PeriodData.Builder("Period") + .setDurationUs(periodDurationUs) + .setOriginalDurationUs(originalDurationUs) + .setAdPlaybackState( + AdPlaybackState(/* adsId= */ "ads", /* adGroupTimesUs...= */ 0L) + .withAdCount(/* adGroupIndex= */ 0, /* adCount= */ 1) + .withAdDurationsUs(/* adGroupIndex= */ 0, adDurationUs) + ) + .build() + ) + ) + .build() + ), + ) + .apply { setCurrentAd(adGroupIndex = 0, adIndexInAdGroup = 0) } +} diff --git a/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/testutils/tracks.kt b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/testutils/tracks.kt new file mode 100644 index 00000000000..9c5d21d4a80 --- /dev/null +++ b/libraries/ui_compose/src/test/java/androidx/media3/ui/compose/testutils/tracks.kt @@ -0,0 +1,104 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.ui.compose.testutils + +import androidx.media3.common.C +import androidx.media3.common.Format +import androidx.media3.common.TrackGroup +import androidx.media3.common.Tracks + +/** Creates a standard set of tracks for testing. */ +internal fun createTestTracks( + selectedVideoTrackIndex: Int? = null, + includeMissingDataTrack: Boolean = false, + videoHandledStates: IntArray? = null, +): Tracks { + val baseFormats = + listOf( + Triple(1920, 1080, "video/mp4"), + Triple(1920, 1080, "video/x-matroska"), + Triple(1280, 720, "video/mp4"), + Triple(1280, 720, "video/x-matroska"), + Triple(854, 480, "video/mp4"), + Triple(854, 480, "video/x-matroska"), + Triple(640, 360, "video/quicktime"), + ) + .map { (w, h, mime) -> + Format.Builder().setWidth(w).setHeight(h).setSampleMimeType(mime).build() + } + + val videoFormats = + if (includeMissingDataTrack) { + baseFormats + Format.Builder().setSampleMimeType("video/mp4").build() + } else { + baseFormats + } + val videoTrackGroup = TrackGroup(*videoFormats.toTypedArray()) + + val audioFormat = Format.Builder().setSampleMimeType("audio/mp4").setLanguage("en").build() + val audioTrackGroup = TrackGroup(audioFormat) + + val textFormat = Format.Builder().setSampleMimeType("text/vtt").setLanguage("es").build() + val textTrackGroup = TrackGroup(textFormat) + + val trackCount = if (includeMissingDataTrack) 8 else 7 + val defaultHandledStates = IntArray(trackCount) { C.FORMAT_HANDLED } + val actualHandledStates = videoHandledStates ?: defaultHandledStates + val isSelectedArray = BooleanArray(trackCount) { it == selectedVideoTrackIndex } + + return Tracks( + listOf( + @Suppress("WrongConstant") + Tracks.Group( + videoTrackGroup, + /* adaptiveSupported= */ true, + actualHandledStates, + isSelectedArray, + ), + Tracks.Group( + audioTrackGroup, + /* adaptiveSupported= */ false, + intArrayOf(C.FORMAT_HANDLED), + booleanArrayOf(false), + ), + Tracks.Group( + textTrackGroup, + /* adaptiveSupported= */ false, + intArrayOf(C.FORMAT_HANDLED), + booleanArrayOf(false), + ), + ) + ) +} + +/** Creates a set of tracks with specific languages for testing language-based selection. */ +internal fun createTestTracksWithLanguages( + mimeType: String, + vararg languages: String, + selectedLanguage: String? = null, +): Tracks { + val groups = languages.map { lang -> + val format = Format.Builder().setSampleMimeType(mimeType).setLanguage(lang).build() + Tracks.Group( + TrackGroup(format), + /* adaptiveSupported= */ false, + intArrayOf(C.FORMAT_HANDLED), + booleanArrayOf(lang == selectedLanguage), + ) + } + return Tracks(groups) +} diff --git a/libraries/ui_compose_material3/build.gradle.kts b/libraries/ui_compose_material3/build.gradle.kts index 9e1258b2f5c..f41a91442ad 100644 --- a/libraries/ui_compose_material3/build.gradle.kts +++ b/libraries/ui_compose_material3/build.gradle.kts @@ -35,12 +35,12 @@ dependencies { api(libs.androidx.compose.foundation) api(libs.androidx.compose.material3) - // TODO: b/509786666 - This dependency is added as part of the Artwork implementation in - // MiniController. This might need updating once the actual implementation is done. - implementation(libs.kotlinx.coroutines.guava) - testImplementation(libs.androidx.compose.ui.test) testImplementation(libs.androidx.compose.ui.test.junit4) testImplementation(project(":test-utils")) + testImplementation(project(":lib-exoplayer")) + testImplementation(project(":lib-transformer")) testImplementation(libs.robolectric) + testImplementation(libs.glide) + testImplementation(libs.glide.concurrent.integration) } diff --git a/libraries/ui_compose_material3/proguard-rules.txt b/libraries/ui_compose_material3/proguard-rules.txt new file mode 100644 index 00000000000..f75df9c9aee --- /dev/null +++ b/libraries/ui_compose_material3/proguard-rules.txt @@ -0,0 +1,27 @@ +# Proguard rules specific to the ui-compose-material3 module. + +-if class androidx.media3.ui.compose.material3.util.ScrubbingModeUtil { + *** *ScrubbingModeEnabled(...); +} +-keepnames class androidx.media3.exoplayer.ExoPlayer {} + +-if class androidx.media3.ui.compose.material3.util.ScrubbingModeUtil { + *** *ScrubbingModeEnabled(...); +} +-keepclassmembers class androidx.media3.exoplayer.ExoPlayer { + void setScrubbingModeEnabled(boolean); + boolean isScrubbingModeEnabled(); +} + +-if class androidx.media3.ui.compose.material3.util.ScrubbingModeUtil { + *** *ScrubbingModeEnabled(...); +} +-keepnames class androidx.media3.transformer.CompositionPlayer {} + +-if class androidx.media3.ui.compose.material3.util.ScrubbingModeUtil { + *** *ScrubbingModeEnabled(...); +} +-keepclassmembers class androidx.media3.transformer.CompositionPlayer { + void setScrubbingModeEnabled(boolean); + boolean isScrubbingModeEnabled(); +} diff --git a/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/MiniController.kt b/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/MiniController.kt index f564ccf8f6e..01e382d1c27 100644 --- a/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/MiniController.kt +++ b/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/MiniController.kt @@ -16,11 +16,8 @@ package androidx.media3.ui.compose.material3 -import android.graphics.Bitmap -import androidx.compose.foundation.Image import androidx.compose.foundation.basicMarquee import androidx.compose.foundation.layout.Arrangement -import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.RowScope @@ -32,31 +29,22 @@ import androidx.compose.material3.CardDefaults import androidx.compose.material3.MaterialTheme import androidx.compose.material3.Text import androidx.compose.runtime.Composable -import androidx.compose.runtime.LaunchedEffect -import androidx.compose.runtime.getValue -import androidx.compose.runtime.mutableStateOf -import androidx.compose.runtime.remember -import androidx.compose.runtime.setValue import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier -import androidx.compose.ui.graphics.asImageBitmap import androidx.compose.ui.graphics.painter.Painter import androidx.compose.ui.layout.ContentScale import androidx.compose.ui.text.font.FontWeight import androidx.media3.common.Player import androidx.media3.common.util.BitmapLoader import androidx.media3.common.util.UnstableApi +import androidx.media3.ui.compose.Artwork import androidx.media3.ui.compose.material3.buttons.NextButton import androidx.media3.ui.compose.material3.buttons.PlayPauseButton import androidx.media3.ui.compose.material3.buttons.PreviousButton import androidx.media3.ui.compose.material3.indicator.LinearProgressIndicator -import androidx.media3.ui.compose.state.rememberCurrentMediaItemState import androidx.media3.ui.compose.text.CurrentMediaItemBox -import kotlinx.coroutines.delay -import kotlinx.coroutines.guava.await -import kotlinx.coroutines.launch -private val defaultPlayerControls: @Composable RowScope.(Player) -> Unit = { player -> +private val defaultPlayerControls: @Composable RowScope.(Player?) -> Unit = { player -> Row(verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.Center) { PreviousButton(player, modifier = Modifier.size(MiniControllerTokens.ControlSize)) PlayPauseButton(player, modifier = Modifier.size(MiniControllerTokens.ControlSize)) @@ -84,12 +72,55 @@ private val defaultPlayerControls: @Composable RowScope.(Player) -> Unit = { pla @Composable @UnstableApi fun MiniController( - player: Player, + player: Player?, modifier: Modifier = Modifier, bitmapLoader: BitmapLoader? = null, defaultArtwork: Painter? = null, onClick: () -> Unit = {}, - playerControls: @Composable RowScope.(Player) -> Unit = defaultPlayerControls, + playerControls: @Composable RowScope.(Player?) -> Unit = defaultPlayerControls, +) { + MiniController( + player = player, + modifier = modifier, + onClick = onClick, + artwork = { + Artwork( + it, + contentDescription = null, + modifier = Modifier.size(MiniControllerTokens.ArtworkSize), + bitmapLoader = bitmapLoader, + placeholder = null, + error = defaultArtwork, + fallback = defaultArtwork, + contentScale = ContentScale.Crop, + ) + }, + playerControls = playerControls, + ) +} + +/** + * A composable that provides a compact control affordance for the [Player]. + * + * The mini controller displays the title and artist of the current media item, as well as its + * artwork and a progress indicator. It provides playback controls which default to previous, + * play/pause, and next buttons. + * + * @param player The [Player] to control. + * @param modifier The [Modifier] to be applied to the mini controller. + * @param onClick The action to be performed when the mini controller is clicked. + * @param artwork A composable that provides the artwork. + * @param playerControls A composable that provides the player controls. The default controls are + * previous, play/pause, and next buttons. + */ +@Composable +@UnstableApi +fun MiniController( + player: Player?, + modifier: Modifier = Modifier, + onClick: () -> Unit = {}, + artwork: @Composable (Player?) -> Unit, + playerControls: @Composable RowScope.(Player?) -> Unit = defaultPlayerControls, ) { Card( onClick = onClick, @@ -107,12 +138,7 @@ fun MiniController( verticalAlignment = Alignment.CenterVertically, horizontalArrangement = Arrangement.spacedBy(MiniControllerTokens.SpacerWidth), ) { - Artwork( - player, - modifier = Modifier.size(MiniControllerTokens.ArtworkSize), - bitmapLoader = bitmapLoader, - defaultArtwork = defaultArtwork, - ) + artwork(player) MediaDescription(player, modifier = Modifier.weight(1f)) playerControls(player) } @@ -121,56 +147,8 @@ fun MiniController( } } -// TODO: b/509786666 - Use the Artwork composable once it becomes available. -@Composable -private fun Artwork( - player: Player?, - modifier: Modifier = Modifier, - contentDescription: String? = null, - alignment: Alignment = Alignment.Center, - contentScale: ContentScale = ContentScale.Crop, - bitmapLoader: BitmapLoader? = null, - defaultArtwork: Painter? = null, -) { - val metadata = rememberCurrentMediaItemState(player).mediaMetadata - // Do not use remember(metadata) to avoid an *immediate* reset-to-null flicker - var bitmap by remember { mutableStateOf(null) } - LaunchedEffect(metadata) { - // A parallel job to clear the old bitmap if the new one takes > 1 second - val clearStaleArtworkJob = launch { - delay(1000) - bitmap = null - } - val newBitmap = - runCatching { bitmapLoader?.loadBitmapFromMetadata(metadata)?.await() }.getOrNull() - clearStaleArtworkJob.cancel() - bitmap = newBitmap - } - - bitmap?.let { - Image( - bitmap = it.asImageBitmap(), - contentDescription = contentDescription, - modifier = modifier, - alignment = alignment, - contentScale = contentScale, - ) - } - ?: defaultArtwork?.let { - Image( - painter = it, - contentDescription = contentDescription, - modifier = modifier, - alignment = alignment, - contentScale = contentScale, - ) - } - // Equivalent to hideArtwork() - keeps layout bounds but shows nothing - ?: Box(modifier) -} - @Composable -private fun MediaDescription(player: Player, modifier: Modifier = Modifier) { +private fun MediaDescription(player: Player?, modifier: Modifier = Modifier) { CurrentMediaItemBox(player) { Column(modifier = modifier) { mediaMetadata.title?.toString()?.let { title -> diff --git a/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/Player.kt b/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/Player.kt index fbb5905cb2d..535cb4d6522 100644 --- a/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/Player.kt +++ b/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/Player.kt @@ -19,6 +19,7 @@ package androidx.media3.ui.compose.material3 import androidx.compose.foundation.focusGroup import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.BoxScope +import androidx.compose.foundation.layout.fillMaxSize import androidx.compose.foundation.layout.fillMaxWidth import androidx.compose.foundation.layout.offset import androidx.compose.runtime.Composable @@ -32,10 +33,12 @@ import androidx.compose.ui.layout.ContentScale import androidx.media3.common.Player import androidx.media3.common.util.ExperimentalApi import androidx.media3.common.util.UnstableApi +import androidx.media3.ui.compose.Artwork import androidx.media3.ui.compose.ContentFrame import androidx.media3.ui.compose.SURFACE_TYPE_SURFACE_VIEW import androidx.media3.ui.compose.SurfaceType import androidx.media3.ui.compose.material3.PlayerTokens.ErrorVerticalOffsetFromControls +import androidx.media3.ui.compose.material3.text.Subtitles /** * A composable that provides a basic player UI layout with default controls and a shutter. @@ -83,8 +86,10 @@ fun Player(player: Player?, modifier: Modifier = Modifier) { * @param surfaceType The type of surface to use for video rendering. See [SurfaceType]. * @param contentScale The scaling mode to apply to the content within the [ContentFrame]. * @param keepContentOnReset Whether to keep the content visible when the player is reset. + * @param subtitleOverlay A composable for rendering subtitles. * @param shutter A composable to be displayed as a shutter over the content. The default shutter is * a black [Box]. + * @param artwork Optional composable slot to render artwork for the current media item. * @param showControls Whether the controls should be visible. False by default. * @param topControls A composable aligned with [Alignment.TopCenter], receiving the [player] and * [showControls]. @@ -103,7 +108,11 @@ fun Player( surfaceType: @SurfaceType Int = SURFACE_TYPE_SURFACE_VIEW, contentScale: ContentScale = ContentScale.Fit, keepContentOnReset: Boolean = false, + subtitleOverlay: @Composable (Player?) -> Unit = { Subtitles(it) }, shutter: @Composable () -> Unit = PlayerDefaults::Shutter, + artwork: (@Composable (Player?) -> Unit)? = { + Artwork(it, modifier = Modifier.fillMaxSize(), contentScale = contentScale) + }, showControls: Boolean = false, topControls: (@Composable BoxScope.(Player?, Boolean) -> Unit)? = { player, showControls -> PlayerDefaults.TopControls(player, showControls, Modifier.fillMaxWidth()) @@ -122,7 +131,9 @@ fun Player( surfaceType, contentScale, keepContentOnReset, + subtitleOverlay, shutter, + artwork, showControls, topControls, centerControls, @@ -138,7 +149,11 @@ private fun PlayerImpl( surfaceType: @SurfaceType Int = SURFACE_TYPE_SURFACE_VIEW, contentScale: ContentScale = ContentScale.Fit, keepContentOnReset: Boolean = false, + subtitleOverlay: @Composable (Player?) -> Unit = { Subtitles(it) }, shutter: @Composable () -> Unit = PlayerDefaults::Shutter, + artwork: (@Composable (Player?) -> Unit)? = { + Artwork(player, modifier = Modifier.fillMaxSize(), contentScale = contentScale) + }, showControls: Boolean = false, topControls: (@Composable BoxScope.(Player?, Boolean) -> Unit)? = { player, showControls -> PlayerDefaults.TopControls(player, showControls, Modifier.fillMaxWidth()) @@ -161,6 +176,8 @@ private fun PlayerImpl( surfaceType = surfaceType, contentScale = contentScale, keepContentOnReset = keepContentOnReset, + artwork = artwork, + overlay = { subtitleOverlay(player) }, shutter = shutter, ) diff --git a/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/PlayerDefaults.kt b/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/PlayerDefaults.kt index 4bc4f77a609..753a8454998 100644 --- a/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/PlayerDefaults.kt +++ b/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/PlayerDefaults.kt @@ -218,12 +218,13 @@ object PlayerDefaults { below: @Composable ColumnScope.(Player?) -> Unit = {}, progressSlider: @Composable (Player?) -> Unit = { ProgressSlider(it) }, ) { - AnimatedVisibility(visible = visible, enter = fadeIn(), exit = fadeOut()) { - Column( - modifier - .background(brush = PlayerTokens.bottomControlsGradient) - .padding(horizontal = PlayerTokens.ControlsHorizontalPadding) - ) { + AnimatedVisibility( + visible = visible, + modifier = Modifier.background(brush = PlayerTokens.bottomControlsGradient), + enter = fadeIn(), + exit = fadeOut(), + ) { + Column(modifier.padding(horizontal = PlayerTokens.ControlsHorizontalPadding)) { above(player) Row(modifier = Modifier.fillMaxWidth(), verticalAlignment = Alignment.CenterVertically) { left(player) diff --git a/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/buttons/PlayPauseButton.kt b/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/buttons/PlayPauseButton.kt index f1fc8488db8..d2f06ce57e6 100644 --- a/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/buttons/PlayPauseButton.kt +++ b/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/buttons/PlayPauseButton.kt @@ -16,9 +16,12 @@ package androidx.media3.ui.compose.material3.buttons +import androidx.compose.foundation.layout.Box +import androidx.compose.material3.CircularProgressIndicator import androidx.compose.material3.IconButtonColors import androidx.compose.material3.IconButtonDefaults import androidx.compose.runtime.Composable +import androidx.compose.ui.Alignment import androidx.compose.ui.Modifier import androidx.compose.ui.graphics.Color import androidx.compose.ui.graphics.painter.Painter @@ -26,12 +29,17 @@ import androidx.compose.ui.graphics.vector.ImageVector import androidx.compose.ui.res.painterResource import androidx.compose.ui.res.stringResource import androidx.compose.ui.unit.Dp +import androidx.compose.ui.unit.dp import androidx.media3.common.Player import androidx.media3.common.util.UnstableApi import androidx.media3.ui.compose.buttons.PlayPauseButton as PlayPauseStateContainer import androidx.media3.ui.compose.material3.PlayerTokens import androidx.media3.ui.compose.material3.R +import androidx.media3.ui.compose.state.DisplayMode import androidx.media3.ui.compose.state.PlayPauseButtonState +import androidx.media3.ui.compose.state.SHOW_BUFFERING_ALWAYS +import androidx.media3.ui.compose.state.SHOW_REPLAY_ON_ENDED +import androidx.media3.ui.compose.state.SHOW_RETRY_ON_ERROR /** * A Material3 [IconButton][androidx.compose.material3.IconButton] that plays or pauses the current @@ -42,14 +50,17 @@ import androidx.media3.ui.compose.state.PlayPauseButtonState * [PlayPauseButtonState] instance derived from the provided [player]. * * @param player The [Player] to control. - * @param modifier The [Modifier] to be applied to the button. + * @param modifier The [Modifier] to be applied to the button container. + * @param displayMode The [DisplayMode] determining under which conditions buffering, replay, or + * retry indicators should be shown. Defaults to [SHOW_BUFFERING_ALWAYS] combined with + * [SHOW_REPLAY_ON_ENDED] and [SHOW_RETRY_ON_ERROR]. * @param painter The supplier for [Painter] used for the icon displayed on the button. This is a * composable lambda with [PlayPauseButtonState] as its receiver, allowing the icon to be updated * based on the button's current state (e.g., [PlayPauseButtonState.showPlay]). * @param iconSize The size of the icon. * @param contentDescription The content description for accessibility purposes. * @param colors [IconButtonColors] that will be used to resolve the colors used for this icon - * button in different states. See [IconButtonDefaults.iconButtonColors]. + * button and buffering indicator. See [IconButtonDefaults.iconButtonColors]. * @param tint Tint to be applied to [painter]. If [Color.Unspecified] is provided, then no tint is * applied. * @param onClick The action to be performed when the button is clicked. This lambda has @@ -67,6 +78,8 @@ import androidx.media3.ui.compose.state.PlayPauseButtonState fun PlayPauseButton( player: Player?, modifier: Modifier = Modifier, + displayMode: @DisplayMode Int = + SHOW_BUFFERING_ALWAYS or SHOW_REPLAY_ON_ENDED or SHOW_RETRY_ON_ERROR, painter: @Composable PlayPauseButtonState.() -> Painter = defaultPlayPausePainterIcon, iconSize: Dp = PlayerTokens.MediumIconSize, contentDescription: @Composable PlayPauseButtonState.() -> String = @@ -79,17 +92,26 @@ fun PlayPauseButton( // This avoids shadowing the PlayPauseButtonState's onClick() *member function* // inside the PlayPauseStateContainer's lambda. val customOnClick: PlayPauseButtonState.() -> Unit = onClick - PlayPauseStateContainer(player) { - ClickableIconButton( - modifier, - isEnabled, - icon = painter(), - contentDescription = contentDescription(), - iconSize = iconSize, - colors = colors, - tint = tint, - onClick = { customOnClick() }, - ) + PlayPauseStateContainer(player, displayMode) { + Box(contentAlignment = Alignment.Center) { + if (showBuffering) { + CircularProgressIndicator( + modifier = Modifier.matchParentSize(), + color = if (isEnabled) colors.contentColor else colors.disabledContentColor, + strokeWidth = 3.dp, + ) + } + ClickableIconButton( + modifier = modifier, + enabled = isEnabled, + icon = painter(), + contentDescription = contentDescription(), + iconSize = iconSize, + colors = colors, + tint = tint, + onClick = { customOnClick() }, + ) + } } } @@ -102,14 +124,17 @@ fun PlayPauseButton( * [PlayPauseButtonState] instance derived from the provided [player]. * * @param player The [Player] to control. - * @param modifier The [Modifier] to be applied to the button. + * @param modifier The [Modifier] to be applied to the button container. + * @param displayMode The [DisplayMode] determining under which conditions buffering, replay, or + * retry indicators should be shown. Defaults to [SHOW_BUFFERING_ALWAYS] combined with + * [SHOW_REPLAY_ON_ENDED] and [SHOW_RETRY_ON_ERROR]. * @param imageVector The supplier for [ImageVector] used for the icon displayed on the button. This * is a composable lambda with [PlayPauseButtonState] as its receiver, allowing the icon to be * updated based on the button's current state (e.g., [PlayPauseButtonState.showPlay]). * @param iconSize The size of the icon. * @param contentDescription The content description for accessibility purposes. * @param colors [IconButtonColors] that will be used to resolve the colors used for this icon - * button in different states. See [IconButtonDefaults.iconButtonColors]. + * button and buffering indicator. See [IconButtonDefaults.iconButtonColors]. * @param tint Tint to be applied to [imageVector]. If [Color.Unspecified] is provided, then no tint * is applied. * @param onClick The action to be performed when the button is clicked. This lambda has @@ -128,6 +153,8 @@ fun PlayPauseButton( fun PlayPauseButton( player: Player?, modifier: Modifier = Modifier, + displayMode: @DisplayMode Int = + SHOW_BUFFERING_ALWAYS or SHOW_REPLAY_ON_ENDED or SHOW_RETRY_ON_ERROR, imageVector: PlayPauseButtonState.() -> ImageVector, iconSize: Dp = PlayerTokens.MediumIconSize, contentDescription: @Composable PlayPauseButtonState.() -> String = @@ -140,28 +167,45 @@ fun PlayPauseButton( // This avoids shadowing the PlayPauseButtonState's onClick() *member function* // inside the PlayPauseStateContainer's lambda. val customOnClick: PlayPauseButtonState.() -> Unit = onClick - PlayPauseStateContainer(player) { - ClickableIconButton( - modifier, - isEnabled, - icon = imageVector(), - contentDescription = contentDescription(), - iconSize = iconSize, - colors = colors, - tint = tint, - onClick = { customOnClick() }, - ) + PlayPauseStateContainer(player, displayMode) { + Box(contentAlignment = Alignment.Center) { + if (showBuffering) { + CircularProgressIndicator( + modifier = Modifier.matchParentSize(), + color = if (isEnabled) colors.contentColor else colors.disabledContentColor, + strokeWidth = 3.dp, + ) + } + ClickableIconButton( + modifier = modifier, + enabled = isEnabled, + icon = imageVector(), + contentDescription = contentDescription(), + iconSize = iconSize, + colors = colors, + tint = tint, + onClick = { customOnClick() }, + ) + } } } private val defaultPlayPauseContentDescription: @Composable PlayPauseButtonState.() -> String = @Composable { - if (showPlay) stringResource(R.string.playpause_button_play) - else stringResource(R.string.playpause_button_pause) + when { + showReplay -> stringResource(R.string.playpause_button_replay) + showRetry -> stringResource(R.string.playpause_button_retry) + showPlay -> stringResource(R.string.playpause_button_play) + else -> stringResource(R.string.playpause_button_pause) + } } private val defaultPlayPausePainterIcon: @Composable PlayPauseButtonState.() -> Painter = @Composable { - if (showPlay) painterResource(R.drawable.media3_icon_play) - else painterResource(R.drawable.media3_icon_pause) + when { + showRetry -> painterResource(R.drawable.media3_icon_retry) + showReplay -> painterResource(R.drawable.media3_icon_replay) + showPlay -> painterResource(R.drawable.media3_icon_play) + else -> painterResource(R.drawable.media3_icon_pause) + } } diff --git a/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/indicator/BufferingIndicator.kt b/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/indicator/BufferingIndicator.kt new file mode 100644 index 00000000000..eefdf085c44 --- /dev/null +++ b/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/indicator/BufferingIndicator.kt @@ -0,0 +1,82 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.ui.compose.material3.indicator + +import androidx.compose.animation.AnimatedVisibility +import androidx.compose.animation.fadeIn +import androidx.compose.animation.fadeOut +import androidx.compose.material3.CircularProgressIndicator +import androidx.compose.runtime.Composable +import androidx.compose.runtime.LaunchedEffect +import androidx.compose.runtime.getValue +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember +import androidx.compose.runtime.setValue +import androidx.compose.ui.Modifier +import androidx.media3.common.Player +import androidx.media3.common.util.UnstableApi +import androidx.media3.ui.compose.state.DisplayMode +import androidx.media3.ui.compose.state.SHOW_BUFFERING_ALWAYS +import androidx.media3.ui.compose.state.SHOW_BUFFERING_WHEN_PLAYING +import androidx.media3.ui.compose.state.rememberPlaybackState +import kotlin.time.Duration +import kotlin.time.Duration.Companion.milliseconds +import kotlinx.coroutines.delay + +/** + * A circular progress indicator that displays an animated spinner when the [Player] is buffering. + * + * @param player The [Player] to observe. + * @param modifier The [Modifier] to be applied to the buffering indicator. + * @param displayMode The [DisplayMode] specifying when to show buffering. Defaults to + * [SHOW_BUFFERING_ALWAYS]. + * @param delay The delay duration before the buffering indicator is shown. + */ +@UnstableApi +@Composable +fun BufferingIndicator( + player: Player?, + modifier: Modifier = Modifier, + displayMode: @DisplayMode Int = SHOW_BUFFERING_ALWAYS, + delay: Duration = 500.milliseconds, + content: @Composable () -> Unit = { CircularProgressIndicator() }, +) { + val playbackState = rememberPlaybackState(player) + val shouldShowBuffering = + playbackState.playbackState == Player.STATE_BUFFERING && + (((displayMode and SHOW_BUFFERING_ALWAYS) != 0) || + (((displayMode and SHOW_BUFFERING_WHEN_PLAYING) != 0) && + playbackState.playWhenReady && + playbackState.playbackSuppressionReason == Player.PLAYBACK_SUPPRESSION_REASON_NONE)) + + var isVisible by remember { mutableStateOf(false) } + + LaunchedEffect(shouldShowBuffering) { + if (shouldShowBuffering) { + if (delay > Duration.ZERO) { + delay(delay) + } + isVisible = true + } else { + isVisible = false + } + } + + AnimatedVisibility(visible = isVisible, enter = fadeIn(), exit = fadeOut(), modifier = modifier) { + content() + } +} diff --git a/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/indicator/ProgressSlider.kt b/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/indicator/ProgressSlider.kt index 7597d73d1bd..22ed130f95a 100644 --- a/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/indicator/ProgressSlider.kt +++ b/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/indicator/ProgressSlider.kt @@ -21,6 +21,7 @@ import androidx.compose.material3.Slider import androidx.compose.material3.SliderColors import androidx.compose.material3.SliderDefaults import androidx.compose.runtime.Composable +import androidx.compose.runtime.DisposableEffect import androidx.compose.runtime.getValue import androidx.compose.runtime.mutableFloatStateOf import androidx.compose.runtime.mutableIntStateOf @@ -34,6 +35,8 @@ import androidx.media3.common.Player import androidx.media3.common.util.ExperimentalApi import androidx.media3.common.util.UnstableApi import androidx.media3.ui.compose.indicators.ProgressIndicator +import androidx.media3.ui.compose.material3.util.isScrubbingModeEnabled +import androidx.media3.ui.compose.material3.util.setScrubbingModeEnabled import kotlinx.coroutines.CoroutineScope /** @@ -70,6 +73,8 @@ fun ProgressSlider(player: Player?, modifier: Modifier = Modifier) { * @param scope The [CoroutineScope] to use for listening to player progress updates. * @param colors [SliderColors] that will be used to resolve the colors used for this slider in * different states. See [SliderDefaults.colors]. + * @param scrubbingEnabled Whether the time bar should seek immediately as the user drags the + * scrubber around (true), or only seek when the user releases the scrubber (false). * @param interactionSource the [MutableInteractionSource] representing the stream of * [Interactions][androidx.compose.foundation.interaction.Interaction] for this slider. You can * create and pass in your own `remember`ed instance to observe `Interactions` and customize the @@ -84,6 +89,7 @@ fun ProgressSlider( onValueChangeFinished: (() -> Unit)? = null, scope: CoroutineScope = rememberCoroutineScope(), colors: SliderColors = SliderDefaults.colors(), + scrubbingEnabled: Boolean = true, interactionSource: MutableInteractionSource = remember { MutableInteractionSource() }, ) { var sliderWidthPx by remember { mutableIntStateOf(0) } @@ -91,16 +97,33 @@ fun ProgressSlider( var isDragging by remember { mutableStateOf(false) } var seekPosition by remember { mutableFloatStateOf(0f) } + // Fallback for navigating away before the trailing invoke in onValueChangeFinished is called + DisposableEffect(player) { + onDispose { + player.setScrubbingModeEnabled(false) + isDragging = false + } + } + // Cache the result to avoid repeated reflection calls + var scrubbingEnabledForThisPlayer by + remember(player, scrubbingEnabled) { mutableStateOf(false) } + Slider( value = if (isDragging) seekPosition else currentPositionProgress, onValueChange = { - // TODO: b/459444117 - Add ScrubbingMode + if (!isDragging && scrubbingEnabled) { + player.setScrubbingModeEnabled(true) + scrubbingEnabledForThisPlayer = player.isScrubbingModeEnabled() + } isDragging = true seekPosition = it + // Dispatch seeks only if they are less expensive (scrubbing mode enabled) + if (scrubbingEnabledForThisPlayer) updateCurrentPositionProgress(seekPosition) onValueChange?.invoke(it) }, onValueChangeFinished = { updateCurrentPositionProgress(seekPosition) + player.setScrubbingModeEnabled(false) isDragging = false onValueChangeFinished?.invoke() }, diff --git a/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/text/ImageCue.kt b/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/text/ImageCue.kt new file mode 100644 index 00000000000..a7f4f46b9bd --- /dev/null +++ b/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/text/ImageCue.kt @@ -0,0 +1,178 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.ui.compose.material3.text + +import androidx.compose.foundation.Image +import androidx.compose.foundation.layout.offset +import androidx.compose.runtime.Composable +import androidx.compose.runtime.remember +import androidx.compose.ui.Modifier +import androidx.compose.ui.graphics.asImageBitmap +import androidx.compose.ui.layout.ContentScale +import androidx.compose.ui.layout.Layout +import androidx.compose.ui.res.stringResource +import androidx.compose.ui.unit.Constraints +import androidx.compose.ui.unit.IntOffset +import androidx.compose.ui.unit.IntRect +import androidx.compose.ui.unit.IntSize +import androidx.media3.common.text.Cue +import androidx.media3.ui.compose.material3.R +import kotlin.math.roundToInt + +/** + * Renders a non-text subtitle cue (i.e., [Cue.bitmap]) using an [Image] composable. + * + * This component sizes and places the bitmap within the given [viewport] using a custom layout. + * + * @param cue The non-text [Cue] to render. If [Cue.bitmap] is null, this composable renders + * nothing. + * @param viewport The [IntRect] bounds representing the video frame surface. The cue's size and + * offsets are calculated relative to this rectangle. + * @param modifier The [Modifier] to be applied to the component. + */ +@Composable +internal fun ImageCue(cue: Cue, viewport: IntRect, modifier: Modifier = Modifier) { + val bitmap = cue.bitmap ?: return + + val size = + getScaledSize( + relativeCueSize = cue.size, + relativeCueHeight = cue.bitmapHeight, + absoluteBitmapWidth = bitmap.width, + absoluteBitmapHeight = bitmap.height, + viewWidth = viewport.width, + viewHeight = viewport.height, + ) + + val imageBitmap = remember(bitmap) { bitmap.asImageBitmap() } + + Layout( + modifier = Modifier.offset { IntOffset(viewport.left, viewport.top) }, + content = { + Image( + bitmap = imageBitmap, + contentDescription = stringResource(R.string.subtitle_bitmap_cue), + modifier = modifier, + // Override the default Fit to respect scaled cues size calculations. + contentScale = ContentScale.FillBounds, + ) + }, + ) { measurables, _ -> + val placeable = measurables[0].measure(Constraints.fixed(size.width, size.height)) + + layout(viewport.width, viewport.height) { + placeable.place( + x = getOffset(cue.position, cue.positionAnchor, size.width, viewport.width), + // Bitmap cues are expected to use fractional line positioning. If lineType is not + // LINE_TYPE_FRACTION, we treat the line as unset to avoid misinterpreting a line number + // as a fractional offset, and fall back to centering. + y = + getOffset( + if (cue.lineType == Cue.LINE_TYPE_FRACTION) cue.line else Cue.DIMEN_UNSET, + cue.lineAnchor, + size.height, + viewport.height, + ), + ) + } + } +} + +/** + * Calculates the pixel dimensions of a bitmap cue. + * + * If a relative cue size is specified, it scales the width proportionally to the view width. The + * height is scaled independently if a relative cue height is provided; otherwise, it maintains the + * aspect ratio of the original bitmap. If no relative sizes are provided, it falls back to the + * absolute dimensions of the original bitmap. + * + * @param relativeCueSize The fractional width of the cue relative to the view width, expressed as a + * value between 0 and 1, or [Cue.DIMEN_UNSET]. + * @param relativeCueHeight The fractional height of the cue relative to the view height, expressed + * as a value between 0 and 1, or [Cue.DIMEN_UNSET]. + * @param absoluteBitmapWidth The inherent pixel width of the bitmap. + * @param absoluteBitmapHeight The inherent pixel height of the bitmap. + * @param viewWidth The pixel width of the viewport. + * @param viewHeight The pixel height of the viewport. + * @return The calculated [IntSize] in pixels for rendering the bitmap. + */ +private fun getScaledSize( + relativeCueSize: Float, + relativeCueHeight: Float, + absoluteBitmapWidth: Int, + absoluteBitmapHeight: Int, + viewWidth: Int, + viewHeight: Int, +): IntSize { + val width = + if (relativeCueSize != Cue.DIMEN_UNSET) { + (viewWidth * relativeCueSize).roundToInt() + } else { + absoluteBitmapWidth + } + + val height = + if (relativeCueHeight != Cue.DIMEN_UNSET) { + // Prioritize relativeCueHeight over relativeCueSize + (viewHeight * relativeCueHeight).roundToInt() + } else if (relativeCueSize != Cue.DIMEN_UNSET) { + // Maintain aspect ratio with the new [width = viewWidth * relativeCueSize] + (width * (absoluteBitmapHeight.toFloat() / absoluteBitmapWidth)).roundToInt() + } else { + absoluteBitmapHeight + } + + return IntSize(width, height) +} + +/** + * Calculates the pixel offset for placing a cue along a given axis (X or Y) relative to the + * viewport. + * + * If [fractionalPosition] is [Cue.DIMEN_UNSET], the cue is centered within the viewport. Otherwise, + * the offset is determined by aligning the specified [anchorType] of the cue with the + * [fractionalPosition] of the viewport. + * + * @param fractionalPosition The position of the cue's anchor, expressed as a fraction (between 0 + * and 1) of [viewDimension], or [Cue.DIMEN_UNSET]. + * @param anchorType The anchor type (e.g., [Cue.ANCHOR_TYPE_START], [Cue.ANCHOR_TYPE_MIDDLE], + * [Cue.ANCHOR_TYPE_END]). + * @param cueDimension The pixel dimension (width or height) of the cue. + * @param viewDimension The pixel dimension (width or height) of the viewport. + * @return The absolute pixel offset of the start (top or left) of the cue, relative to the start + * (top or left) of the viewport. + */ +private fun getOffset( + fractionalPosition: Float, + anchorType: @Cue.AnchorType Int, + cueDimension: Int, + viewDimension: Int, +): Int { + if (fractionalPosition == Cue.DIMEN_UNSET) { + return (viewDimension - cueDimension) / 2 // Fallback to center + } + + val anchorPoint = viewDimension * fractionalPosition + val offset = + when (anchorType) { + Cue.ANCHOR_TYPE_END -> anchorPoint - cueDimension + Cue.ANCHOR_TYPE_MIDDLE -> anchorPoint - (cueDimension / 2f) + Cue.ANCHOR_TYPE_START -> anchorPoint + else -> anchorPoint + } + return offset.roundToInt() +} diff --git a/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/text/Subtitles.kt b/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/text/Subtitles.kt new file mode 100644 index 00000000000..0873cea415d --- /dev/null +++ b/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/text/Subtitles.kt @@ -0,0 +1,51 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.ui.compose.material3.text + +import androidx.compose.foundation.layout.BoxWithConstraints +import androidx.compose.runtime.Composable +import androidx.compose.ui.Modifier +import androidx.compose.ui.unit.IntRect +import androidx.media3.common.Player +import androidx.media3.ui.compose.state.rememberCuesState + +/** + * A Composable for rendering subtitle cues from a Media3 [Player]. + * + * This view observes cue changes from the [Player] using [rememberCuesState]. + * + * This component currently only supports bitmap cues, support for text cues will be added later. + * + * @param player The [Player] instance to observe for cues. + * @param modifier The [Modifier] to be applied to the layout. + */ +@Composable +internal fun Subtitles(player: Player?, modifier: Modifier = Modifier) { + val cues = rememberCuesState(player).cues + if (cues.isEmpty()) return + + BoxWithConstraints(modifier = modifier) { + val viewWidth = this.constraints.maxWidth + val viewHeight = this.constraints.maxHeight + for (cue in cues) { + if (cue.bitmap != null) { + ImageCue(cue, viewport = IntRect(0, 0, viewWidth, viewHeight)) + } + // TODO: Handle text cues here once TextCue is implemented + } + } +} diff --git a/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/util/ScrubbingModeUtil.kt b/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/util/ScrubbingModeUtil.kt new file mode 100644 index 00000000000..bc025cef112 --- /dev/null +++ b/libraries/ui_compose_material3/src/main/java/androidx/media3/ui/compose/material3/util/ScrubbingModeUtil.kt @@ -0,0 +1,113 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.ui.compose.material3.util + +import androidx.annotation.RestrictTo +import androidx.media3.common.Player +import androidx.media3.common.util.Log +import java.lang.reflect.Method + +/** + * A utility object that uses reflection to enable scrubbing mode on supported [Player] + * implementations (e.g., `ExoPlayer`, `CompositionPlayer`) without introducing direct dependencies + * on their respective modules. + */ +internal object ScrubbingModeUtil { + private const val TAG = "ScrubbingModeUtil" + + private val EXO_PLAYER_CLAZZ: Class<*>? = + try { + // LINT.IfChange + Class.forName("androidx.media3.exoplayer.ExoPlayer") + // LINT.ThenChange(../../../../../../../../../proguard-rules.txt) + } catch (_: ClassNotFoundException) { + null + } + + private val EXO_PLAYER_SETTER: Method? = + try { + EXO_PLAYER_CLAZZ?.getMethod("setScrubbingModeEnabled", Boolean::class.java) + } catch (_: NoSuchMethodException) { + null + } + + private val EXO_PLAYER_GETTER: Method? = + try { + EXO_PLAYER_CLAZZ?.getMethod("isScrubbingModeEnabled") + } catch (_: NoSuchMethodException) { + null + } + + private val COMPOSITION_PLAYER_CLAZZ: Class<*>? = + try { + // LINT.IfChange + Class.forName("androidx.media3.transformer.CompositionPlayer") + // LINT.ThenChange(../../../../../../../../../proguard-rules.txt) + } catch (_: ClassNotFoundException) { + null + } + + private val COMPOSITION_PLAYER_SETTER: Method? = + try { + COMPOSITION_PLAYER_CLAZZ?.getMethod("setScrubbingModeEnabled", Boolean::class.java) + } catch (_: NoSuchMethodException) { + null + } + + private val COMPOSITION_PLAYER_GETTER: Method? = + try { + COMPOSITION_PLAYER_CLAZZ?.getMethod("isScrubbingModeEnabled") + } catch (_: NoSuchMethodException) { + null + } + + fun setScrubbingModeEnabled(player: Player?, enabled: Boolean) { + try { + if (EXO_PLAYER_CLAZZ?.isInstance(player) == true) { + EXO_PLAYER_SETTER?.invoke(player, enabled) + } else if (COMPOSITION_PLAYER_CLAZZ?.isInstance(player) == true) { + COMPOSITION_PLAYER_SETTER?.invoke(player, enabled) + } + } catch (e: Exception) { + Log.w(TAG, "Failed to set scrubbing mode", e) + } + } + + fun isScrubbingModeEnabled(player: Player?): Boolean { + return try { + if (EXO_PLAYER_CLAZZ?.isInstance(player) == true) { + EXO_PLAYER_GETTER?.invoke(player) as? Boolean ?: false + } else if (COMPOSITION_PLAYER_CLAZZ?.isInstance(player) == true) { + COMPOSITION_PLAYER_GETTER?.invoke(player) as? Boolean ?: false + } else { + false + } + } catch (_: Exception) { + false + } + } +} + +@RestrictTo(RestrictTo.Scope.LIBRARY_GROUP) +fun Player?.setScrubbingModeEnabled(enabled: Boolean) { + ScrubbingModeUtil.setScrubbingModeEnabled(this, enabled) +} + +@RestrictTo(RestrictTo.Scope.LIBRARY_GROUP) +fun Player?.isScrubbingModeEnabled(): Boolean { + return ScrubbingModeUtil.isScrubbingModeEnabled(this) +} diff --git a/libraries/ui_compose_material3/src/main/res/drawable/media3_icon_replay.xml b/libraries/ui_compose_material3/src/main/res/drawable/media3_icon_replay.xml new file mode 100644 index 00000000000..0e77f38bde8 --- /dev/null +++ b/libraries/ui_compose_material3/src/main/res/drawable/media3_icon_replay.xml @@ -0,0 +1,23 @@ + + + + diff --git a/libraries/ui_compose_material3/src/main/res/drawable/media3_icon_retry.xml b/libraries/ui_compose_material3/src/main/res/drawable/media3_icon_retry.xml new file mode 100644 index 00000000000..6ec3314c645 --- /dev/null +++ b/libraries/ui_compose_material3/src/main/res/drawable/media3_icon_retry.xml @@ -0,0 +1,26 @@ + + + + + diff --git a/libraries/ui_compose_material3/src/main/res/drawable/media3_icon_subtitles.xml b/libraries/ui_compose_material3/src/main/res/drawable/media3_icon_subtitles.xml new file mode 100644 index 00000000000..cbfd2e2e111 --- /dev/null +++ b/libraries/ui_compose_material3/src/main/res/drawable/media3_icon_subtitles.xml @@ -0,0 +1,23 @@ + + + + diff --git a/libraries/ui_compose_material3/src/main/res/drawable/media3_icon_subtitles_off.xml b/libraries/ui_compose_material3/src/main/res/drawable/media3_icon_subtitles_off.xml new file mode 100644 index 00000000000..174ab51f749 --- /dev/null +++ b/libraries/ui_compose_material3/src/main/res/drawable/media3_icon_subtitles_off.xml @@ -0,0 +1,23 @@ + + + + diff --git a/libraries/ui_compose_material3/src/main/res/values/strings.xml b/libraries/ui_compose_material3/src/main/res/values/strings.xml index 4e9243a3393..6a0a39994e7 100644 --- a/libraries/ui_compose_material3/src/main/res/values/strings.xml +++ b/libraries/ui_compose_material3/src/main/res/values/strings.xml @@ -18,6 +18,10 @@ Play Pause + + Replay + + Retry Next @@ -116,4 +120,6 @@ Failed to initialize the video frame processor. Failed to process a video frame. + + Subtitle bitmap cue diff --git a/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/MiniControllerTest.kt b/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/MiniControllerTest.kt index f2abcea52be..558e382244a 100644 --- a/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/MiniControllerTest.kt +++ b/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/MiniControllerTest.kt @@ -15,6 +15,11 @@ */ package androidx.media3.ui.compose.material3 +import androidx.compose.foundation.layout.Box +import androidx.compose.foundation.layout.size +import androidx.compose.runtime.MutableState +import androidx.compose.runtime.mutableStateOf +import androidx.compose.runtime.remember import androidx.compose.ui.Modifier import androidx.compose.ui.platform.testTag import androidx.compose.ui.test.assertIsDisplayed @@ -22,8 +27,14 @@ import androidx.compose.ui.test.junit4.v2.createComposeRule import androidx.compose.ui.test.onNodeWithTag import androidx.compose.ui.test.onNodeWithText import androidx.compose.ui.test.performClick -import androidx.media3.common.MediaItem +import androidx.compose.ui.unit.dp +import androidx.media3.common.C +import androidx.media3.common.Format import androidx.media3.common.MediaMetadata +import androidx.media3.common.MimeTypes +import androidx.media3.common.SimpleBasePlayer.MediaItemData +import androidx.media3.common.TrackGroup +import androidx.media3.common.Tracks import androidx.media3.test.utils.FakePlayer import androidx.media3.ui.compose.material3.buttons.PreviousButton import androidx.test.ext.junit.runners.AndroidJUnit4 @@ -77,14 +88,71 @@ class MiniControllerTest { @Test fun miniController_displaysTitleAndArtist() { - val player = FakePlayer() val mediaMetadata = MediaMetadata.Builder().setTitle("Sample Title").setArtist("Sample Artist").build() - val mediaItem = MediaItem.Builder().setMediaMetadata(mediaMetadata).build() - player.setMediaItem(mediaItem) + val player = + FakePlayer( + playlist = listOf(MediaItemData.Builder("First").setMediaMetadata(mediaMetadata).build()) + ) + + composeTestRule.setContent { MiniController(player) } + composeTestRule.onNodeWithText("Sample Title", substring = true).assertIsDisplayed() + composeTestRule.onNodeWithText("Sample Artist", substring = true).assertIsDisplayed() + } + + @Test + fun miniController_withNullPlayer_isDisplayed() { + composeTestRule.setContent { + MiniController(player = null, modifier = Modifier.testTag(miniControllerTag)) + } + + composeTestRule.onNodeWithTag(miniControllerTag).assertIsDisplayed() + } + + @Test + fun miniController_customArtwork_reactsToPlayerChange() { + val textTrack = + Tracks.Group( + TrackGroup(Format.Builder().setSampleMimeType(MimeTypes.TEXT_VTT).build()), + /* adaptiveSupported= */ true, + /* trackSupport= */ intArrayOf(C.FORMAT_HANDLED), + /* trackSelected= */ booleanArrayOf(true), + ) + val mediaMetadata = + MediaMetadata.Builder().setTitle("Sample Title").setArtist("Sample Artist").build() + val player = + FakePlayer( + playlist = + listOf( + MediaItemData.Builder("First") + .setMediaMetadata(mediaMetadata) + .setTracks(Tracks(listOf(textTrack))) + .build() + ) + ) + + lateinit var isPlayerNull: MutableState + composeTestRule.setContent { + isPlayerNull = remember { mutableStateOf(false) } + MiniController( + player = if (isPlayerNull.value) null else player, + modifier = Modifier.testTag(miniControllerTag), + artwork = { p -> + val tag = if (p != null) "artworkWithPlayer" else "artworkWithoutPlayer" + Box(modifier = Modifier.size(100.dp).testTag(tag)) + }, + ) + } + + composeTestRule.onNodeWithTag("artworkWithPlayer", useUnmergedTree = true).assertIsDisplayed() + composeTestRule.onNodeWithText("Sample Title", substring = true).assertIsDisplayed() + composeTestRule.onNodeWithText("Sample Artist", substring = true).assertIsDisplayed() + + isPlayerNull.value = true + composeTestRule.waitForIdle() - composeTestRule.setContent { MiniController(player = player) } - composeTestRule.onNodeWithText("Sample Title").assertIsDisplayed() - composeTestRule.onNodeWithText("Sample Artist").assertIsDisplayed() + composeTestRule + .onNodeWithTag("artworkWithoutPlayer", useUnmergedTree = true) + .assertIsDisplayed() } } diff --git a/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/PlayerTest.kt b/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/PlayerTest.kt index 39330b0be2c..1cfd4d807ed 100644 --- a/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/PlayerTest.kt +++ b/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/PlayerTest.kt @@ -17,6 +17,9 @@ package androidx.media3.ui.compose.material3 import android.content.Context +import android.graphics.Bitmap +import android.graphics.Color +import android.net.Uri import androidx.compose.foundation.clickable import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.fillMaxSize @@ -37,13 +40,29 @@ import androidx.compose.ui.test.junit4.v2.createComposeRule import androidx.compose.ui.test.onAllNodesWithText import androidx.compose.ui.test.onFirst import androidx.compose.ui.test.onLast +import androidx.compose.ui.test.onNodeWithContentDescription import androidx.compose.ui.test.onNodeWithTag import androidx.compose.ui.test.performClick import androidx.compose.ui.unit.dp +import androidx.core.graphics.createBitmap +import androidx.media3.common.C +import androidx.media3.common.Format +import androidx.media3.common.MediaItem +import androidx.media3.common.MediaMetadata +import androidx.media3.common.MimeTypes +import androidx.media3.common.SimpleBasePlayer.MediaItemData +import androidx.media3.common.TrackGroup +import androidx.media3.common.Tracks +import androidx.media3.common.util.BitmapLoader import androidx.media3.test.utils.FakePlayer +import androidx.media3.ui.compose.Artwork import androidx.test.core.app.ApplicationProvider import androidx.test.ext.junit.runners.AndroidJUnit4 +import com.bumptech.glide.Glide +import com.bumptech.glide.integration.concurrent.GlideFutures import com.google.common.truth.Truth +import com.google.common.util.concurrent.ListenableFuture +import java.io.ByteArrayOutputStream import org.junit.Rule import org.junit.Test import org.junit.runner.RunWith @@ -266,4 +285,107 @@ class PlayerTest { .onNodeWithTag("errorMessageWithoutPlayer", useUnmergedTree = true) .assertIsDisplayed() } + + @Test + fun player_customArtwork_displayedWhenAudioTrackSelected() { + val audioTrack = + Tracks.Group( + TrackGroup(Format.Builder().setSampleMimeType(MimeTypes.AUDIO_AAC).build()), + /* adaptiveSupported= */ true, + /* trackSupport= */ intArrayOf(C.FORMAT_HANDLED), + /* trackSelected= */ booleanArrayOf(true), + ) + val player = + FakePlayer( + playlist = + listOf(MediaItemData.Builder("First").setTracks(Tracks(listOf(audioTrack))).build()) + ) + + composeTestRule.setContent { + Player(player, artwork = { Box(Modifier.fillMaxSize().testTag("CustomArtworkTag")) }) + } + + composeTestRule.onNodeWithTag("CustomArtworkTag").assertIsDisplayed() + } + + @Test + fun player_glideAsBitmapLoaderBackend_loadsArtworkBitmap() { + val context = ApplicationProvider.getApplicationContext() + val glideBitmapLoader = GlideBitmapLoader(context) + val originalBitmap = createBitmap(1, 1) + originalBitmap.eraseColor(Color.RED) + val outputStream = ByteArrayOutputStream() + originalBitmap.compress(Bitmap.CompressFormat.PNG, 100, outputStream) + val validImageData = outputStream.toByteArray() + val mediaItem = + MediaItem.Builder() + .setMediaMetadata( + MediaMetadata.Builder() + .setTitle("Track Title") + .setArtworkData(validImageData, MediaMetadata.PICTURE_TYPE_FRONT_COVER) + .build() + ) + .build() + val audioTrack = + Tracks.Group( + TrackGroup(Format.Builder().setSampleMimeType(MimeTypes.AUDIO_AAC).build()), + /* adaptiveSupported= */ true, + /* trackSupport= */ intArrayOf(C.FORMAT_HANDLED), + /* trackSelected= */ booleanArrayOf(true), + ) + val player = + FakePlayer( + playlist = + listOf( + MediaItemData.Builder("First") + .setMediaItem(mediaItem) + .setTracks(Tracks(listOf(audioTrack))) + .build() + ) + ) + + composeTestRule.setContent { + Player( + player, + artwork = { + Artwork( + player = player, + contentDescription = "Album Artwork", + bitmapLoader = glideBitmapLoader, + modifier = Modifier.fillMaxSize().testTag("ArtworkBackendTag"), + ) + }, + ) + } + + // We assert against the contentDescription because it is uniquely applied to the Image + // composable upon a successful load. Asserting against the testTag("ArtworkBackendTag") + // would falsely pass even on failure, as it applies to the fallback Box wrapper. + composeTestRule.waitUntil(timeoutMillis = 5000) { + composeTestRule + .onAllNodes(hasContentDescription("Album Artwork")) + .fetchSemanticsNodes() + .isNotEmpty() + } + composeTestRule.onNodeWithContentDescription("Album Artwork").assertIsDisplayed() + } + + private class GlideBitmapLoader(private val context: Context) : BitmapLoader { + override fun loadBitmap(uri: Uri): ListenableFuture { + return GlideFutures.submit(Glide.with(context).asBitmap().load(uri)) + } + + override fun decodeBitmap(data: ByteArray): ListenableFuture { + return GlideFutures.submit(Glide.with(context).asBitmap().load(data)) + } + + override fun loadBitmapFromMetadata(metadata: MediaMetadata): ListenableFuture? { + if (metadata.artworkData != null) { + return decodeBitmap(metadata.artworkData!!) + } + return metadata.artworkUri?.let { loadBitmap(it) } + } + + override fun supportsMimeType(mimeType: String): Boolean = true + } } diff --git a/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/indicator/BufferingIndicatorTest.kt b/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/indicator/BufferingIndicatorTest.kt new file mode 100644 index 00000000000..75fd54cfe92 --- /dev/null +++ b/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/indicator/BufferingIndicatorTest.kt @@ -0,0 +1,167 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.ui.compose.material3.indicator + +import androidx.compose.foundation.layout.Box +import androidx.compose.foundation.layout.size +import androidx.compose.ui.Modifier +import androidx.compose.ui.platform.testTag +import androidx.compose.ui.test.ExperimentalTestApi +import androidx.compose.ui.test.assertIsDisplayed +import androidx.compose.ui.test.onNodeWithTag +import androidx.compose.ui.test.v2.runComposeUiTest +import androidx.compose.ui.unit.dp +import androidx.media3.common.Player +import androidx.media3.common.SimpleBasePlayer.MediaItemData +import androidx.media3.test.utils.FakePlayer +import androidx.media3.ui.compose.state.SHOW_BUFFERING_ALWAYS +import androidx.media3.ui.compose.state.SHOW_BUFFERING_WHEN_PLAYING +import androidx.test.ext.junit.runners.AndroidJUnit4 +import kotlin.time.Duration +import kotlin.time.Duration.Companion.milliseconds +import org.junit.Test +import org.junit.runner.RunWith + +private const val TEST_TAG = "buffering_indicator" + +/** Unit test for [BufferingIndicator]. */ +@OptIn(ExperimentalTestApi::class) +@RunWith(AndroidJUnit4::class) +class BufferingIndicatorTest { + + @Test + fun bufferingIndicator_whenNotBuffering_isNotVisible() = runComposeUiTest { + val player = + FakePlayer( + playbackState = Player.STATE_READY, + playlist = listOf(MediaItemData.Builder("item").build()), + ) + + setContent { + BufferingIndicator( + player = player, + delay = Duration.ZERO, + content = { Box(Modifier.testTag(TEST_TAG).size(24.dp)) }, + ) + } + + onNodeWithTag(TEST_TAG).assertDoesNotExist() + } + + @Test + fun bufferingIndicator_whenBufferingAlways_showsAfterDelay() = runComposeUiTest { + val player = + FakePlayer( + playbackState = Player.STATE_BUFFERING, + playWhenReady = false, + playlist = listOf(MediaItemData.Builder("item").build()), + ) + + setContent { + BufferingIndicator( + player = player, + displayMode = SHOW_BUFFERING_ALWAYS, + delay = 500.milliseconds, + content = { Box(Modifier.testTag(TEST_TAG).size(24.dp)) }, + ) + } + + onNodeWithTag(TEST_TAG).assertDoesNotExist() + + mainClock.advanceTimeBy(600) + waitForIdle() + + onNodeWithTag(TEST_TAG).assertIsDisplayed() + } + + @Test + fun bufferingIndicator_whenBufferingWhenPlaying_andPaused_doesNotShow() = runComposeUiTest { + val player = + FakePlayer( + playbackState = Player.STATE_BUFFERING, + playWhenReady = false, + playlist = listOf(MediaItemData.Builder("item").build()), + ) + + setContent { + BufferingIndicator( + player = player, + displayMode = SHOW_BUFFERING_WHEN_PLAYING, + delay = Duration.ZERO, + content = { Box(Modifier.testTag(TEST_TAG).size(24.dp)) }, + ) + } + + onNodeWithTag(TEST_TAG).assertDoesNotExist() + } + + @Test + fun bufferingIndicator_whenBufferingWhenPlaying_andPlaying_showsAfterDelay() = runComposeUiTest { + val player = + FakePlayer( + playbackState = Player.STATE_BUFFERING, + playWhenReady = true, + playlist = listOf(MediaItemData.Builder("item").build()), + ) + + setContent { + BufferingIndicator( + player = player, + displayMode = SHOW_BUFFERING_WHEN_PLAYING, + delay = 500.milliseconds, + content = { Box(Modifier.testTag(TEST_TAG).size(24.dp)) }, + ) + } + + onNodeWithTag(TEST_TAG).assertDoesNotExist() + + mainClock.advanceTimeBy(600) + waitForIdle() + + onNodeWithTag(TEST_TAG).assertIsDisplayed() + } + + @Test + fun bufferingIndicator_whenBufferingTransitionsToReadyBeforeDelay_doesNotShow() = + runComposeUiTest { + val player = + FakePlayer( + playbackState = Player.STATE_BUFFERING, + playWhenReady = true, + playlist = listOf(MediaItemData.Builder("item").build()), + ) + + setContent { + BufferingIndicator( + player = player, + delay = 500.milliseconds, + content = { Box(Modifier.testTag(TEST_TAG).size(24.dp)) }, + ) + } + + onNodeWithTag(TEST_TAG).assertDoesNotExist() + + mainClock.advanceTimeBy(200) + player.setPlaybackState(Player.STATE_READY) + waitForIdle() + + mainClock.advanceTimeBy(500) + waitForIdle() + + onNodeWithTag(TEST_TAG).assertDoesNotExist() + } +} diff --git a/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/indicator/ProgressSliderTest.kt b/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/indicator/ProgressSliderTest.kt index 3b1f46d7c5d..b49365be6cc 100644 --- a/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/indicator/ProgressSliderTest.kt +++ b/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/indicator/ProgressSliderTest.kt @@ -16,6 +16,7 @@ package androidx.media3.ui.compose.material3.indicator +import android.content.Context import androidx.compose.runtime.mutableStateOf import androidx.compose.ui.Modifier import androidx.compose.ui.geometry.Offset @@ -29,10 +30,19 @@ import androidx.compose.ui.test.assertRangeInfoEquals import androidx.compose.ui.test.junit4.v2.createComposeRule import androidx.compose.ui.test.onNodeWithTag import androidx.compose.ui.test.performTouchInput +import androidx.media3.common.MediaItem +import androidx.media3.common.Player import androidx.media3.common.Player.STATE_READY import androidx.media3.common.SimpleBasePlayer +import androidx.media3.exoplayer.ExoPlayer import androidx.media3.test.utils.FakePlayer +import androidx.media3.transformer.Composition +import androidx.media3.transformer.CompositionPlayer +import androidx.media3.transformer.EditedMediaItem +import androidx.media3.transformer.EditedMediaItemSequence +import androidx.test.core.app.ApplicationProvider import androidx.test.ext.junit.runners.AndroidJUnit4 +import com.google.common.collect.ImmutableList import com.google.common.truth.Truth.assertThat import kotlin.time.Duration.Companion.milliseconds import kotlinx.coroutines.test.StandardTestDispatcher @@ -41,6 +51,11 @@ import kotlinx.coroutines.test.runTest import org.junit.Rule import org.junit.Test import org.junit.runner.RunWith +import org.mockito.AdditionalAnswers.delegatesTo +import org.mockito.Mockito.doReturn +import org.mockito.Mockito.mock +import org.mockito.Mockito.never +import org.mockito.Mockito.verify import org.robolectric.annotation.Config private const val SLIDER_TAG = "slider" @@ -273,6 +288,172 @@ class ProgressSliderTest { assertThatSliderValueEquals(0.75f) } + @Test + fun slider_dragAndRelease_transitionsScrubbingMode() = + runTest(testDispatcher) { + val fakePlayer = + FakePlayer( + playlist = + listOf( + SimpleBasePlayer.MediaItemData.Builder("Item") + .setDurationUs(10_000_000) + .setIsSeekable(true) + .build() + ) + ) + val player = mock(ExoPlayer::class.java, delegatesTo(fakePlayer)) + doReturn(true).`when`(player).isScrubbingModeEnabled + + var slop = 0f + + composeTestRule.setContent { + slop = LocalViewConfiguration.current.touchSlop + ProgressSlider( + player = player, + modifier = Modifier.testTag(SLIDER_TAG), + scope = backgroundScope, + scrubbingEnabled = true, + ) + } + + // Drag starts + composeTestRule.onNodeWithTag(SLIDER_TAG).performTouchInput { + down(center) + moveBy(Offset(slop, 0f)) + } + testScheduler.runCurrent() + verify(player).isScrubbingModeEnabled = true + + // Verify scrubbing position changed during drag + assertThat(player.currentPosition).isEqualTo(5_000) + + // Drag continues + composeTestRule.onNodeWithTag(SLIDER_TAG).performTouchInput { moveBy(Offset(slop, 0f)) } + testScheduler.runCurrent() + + // Finish drag, lift pointer + composeTestRule.onNodeWithTag(SLIDER_TAG).performTouchInput { up() } + testScheduler.runCurrent() + + verify(player).isScrubbingModeEnabled = false + } + + @Test + fun slider_dragAndRelease_transitionsScrubbingMode_compositionPlayer() = + runTest(testDispatcher) { + val context = ApplicationProvider.getApplicationContext() + val player = CompositionPlayer.Builder(context).build() + val mediaItem = MediaItem.fromUri("asset:///some_asset") + val editedMediaItem = EditedMediaItem.Builder(mediaItem).setDurationUs(10_000_000L).build() + val sequence = + EditedMediaItemSequence.withAudioAndVideoFrom(ImmutableList.of(editedMediaItem)) + val composition = Composition.Builder(ImmutableList.of(sequence)).build() + player.setComposition(composition) + player.prepare() + + var slop = 0f + + composeTestRule.setContent { + slop = LocalViewConfiguration.current.touchSlop + ProgressSlider( + player = player, + modifier = Modifier.testTag(SLIDER_TAG), + scope = backgroundScope, + scrubbingEnabled = true, + ) + } + + // Drag starts + composeTestRule.onNodeWithTag(SLIDER_TAG).performTouchInput { + down(center) + moveBy(Offset(slop, 0f)) + } + testScheduler.runCurrent() + assertThat(player.isScrubbingModeEnabled).isTrue() + + // Verify scrubbing position changed during drag + assertThat(player.currentPosition).isEqualTo(5_000) + + // Drag continues + composeTestRule.onNodeWithTag(SLIDER_TAG).performTouchInput { moveBy(Offset(slop, 0f)) } + testScheduler.runCurrent() + + // Finish drag, lift pointer + composeTestRule.onNodeWithTag(SLIDER_TAG).performTouchInput { up() } + testScheduler.runCurrent() + + assertThat(player.isScrubbingModeEnabled).isFalse() + } + + @Test + fun slider_receivesNewPlayerMidDrag_transitionsScrubbingMode() = + runTest(testDispatcher) { + val fakePlayer1 = + FakePlayer( + playlist = + listOf( + SimpleBasePlayer.MediaItemData.Builder("Item1") + .setDurationUs(10_000_000) + .setIsSeekable(true) + .build() + ) + ) + val fakePlayer2 = + FakePlayer( + playlist = + listOf( + SimpleBasePlayer.MediaItemData.Builder("Item2") + .setDurationUs(10_000_000) + .setIsSeekable(true) + .build() + ) + ) + val player1 = mock(ExoPlayer::class.java, delegatesTo(fakePlayer1)) + val player2 = mock(ExoPlayer::class.java, delegatesTo(fakePlayer2)) + + val currentPlayer = mutableStateOf(player1) + var slop = 0f + + composeTestRule.setContent { + slop = LocalViewConfiguration.current.touchSlop + ProgressSlider( + player = currentPlayer.value, + modifier = Modifier.testTag(SLIDER_TAG), + scope = backgroundScope, + scrubbingEnabled = true, + ) + } + + // Drag starts on player1 + composeTestRule.onNodeWithTag(SLIDER_TAG).performTouchInput { + down(center) + moveBy(Offset(slop, 0f)) + } + testScheduler.runCurrent() + verify(player1).isScrubbingModeEnabled = true + + // Swap players mid-drag + currentPlayer.value = player2 + composeTestRule.waitForIdle() + + // player1 should have scrubbing disabled when it leaves composition + verify(player1).isScrubbingModeEnabled = false + + // Drag continues + composeTestRule.onNodeWithTag(SLIDER_TAG).performTouchInput { moveBy(Offset(slop, 0f)) } + testScheduler.runCurrent() + + // player2 should have scrubbing mode enabled since we continued dragging + verify(player2).isScrubbingModeEnabled = true + + // Finish drag, lift pointer + composeTestRule.onNodeWithTag(SLIDER_TAG).performTouchInput { up() } + testScheduler.runCurrent() + + // player2 should have scrubbing mode disabled at the end + verify(player2).isScrubbingModeEnabled = false + } + @Test fun slider_dragAndRelease_firesCallbacksCorrectly() = runTest(testDispatcher) { @@ -315,6 +496,119 @@ class ProgressSliderTest { assertThat(onValueChangeFinishedCount).isEqualTo(1) } + @Test + fun slider_scrubbingDisabled_doesNotTransitionScrubbingMode() = + runTest(testDispatcher) { + val fakePlayer = + FakePlayer( + playlist = + listOf( + SimpleBasePlayer.MediaItemData.Builder("Item") + .setDurationUs(10_000_000) + .setIsSeekable(true) + .build() + ) + ) + val player = mock(ExoPlayer::class.java, delegatesTo(fakePlayer)) + + var slop = 0f + + composeTestRule.setContent { + slop = LocalViewConfiguration.current.touchSlop + ProgressSlider( + player = player, + modifier = Modifier.testTag(SLIDER_TAG), + scope = backgroundScope, + scrubbingEnabled = false, + ) + } + + // Drag starts + composeTestRule.onNodeWithTag(SLIDER_TAG).performTouchInput { + down(center) + moveBy(Offset(slop, 0f)) + } + testScheduler.runCurrent() + verify(player, never()).isScrubbingModeEnabled = true + + // Finish drag, lift pointer + composeTestRule.onNodeWithTag(SLIDER_TAG).performTouchInput { up() } + testScheduler.runCurrent() + + verify(player).isScrubbingModeEnabled = false + } + + @Test + fun slider_scrubbingEnabledByDefault_transitionsScrubbingMode() = + runTest(testDispatcher) { + val fakePlayer = + FakePlayer( + playlist = + listOf( + SimpleBasePlayer.MediaItemData.Builder("Item") + .setDurationUs(10_000_000) + .setIsSeekable(true) + .build() + ) + ) + val player = mock(ExoPlayer::class.java, delegatesTo(fakePlayer)) + composeTestRule.setContent { + ProgressSlider(player, Modifier.testTag(SLIDER_TAG), scope = backgroundScope) + } + + composeTestRule.onNodeWithTag(SLIDER_TAG).performTouchInput { + down(center) + moveTo(centerRight) + } + testScheduler.runCurrent() + + verify(player).isScrubbingModeEnabled = true + } + + @Test + fun slider_nonExoPlayer_scrubbingEnabled_doesNotScrubImmediately() = + runTest(testDispatcher) { + val player = + FakePlayer( + playlist = + listOf( + SimpleBasePlayer.MediaItemData.Builder("Item") + .setDurationUs(10_000_000) + .setIsSeekable(true) + .build() + ) + ) + + var slop = 0f + + composeTestRule.setContent { + slop = LocalViewConfiguration.current.touchSlop + ProgressSlider( + player = player, + modifier = Modifier.testTag(SLIDER_TAG), + scope = backgroundScope, + scrubbingEnabled = true, + ) + } + + // Drag starts + composeTestRule.onNodeWithTag(SLIDER_TAG).performTouchInput { + down(center) + moveBy(Offset(slop, 0f)) + } + testScheduler.runCurrent() + + // The position should NOT update during drag because scrubbing is unsupported + assertThat(player.currentPosition).isEqualTo(0) + + // Finish drag, lift pointer + composeTestRule.onNodeWithTag(SLIDER_TAG).performTouchInput { up() } + testScheduler.runCurrent() + + // The position should update ONLY after pointer is lifted + assertThat(player.currentPosition).isEqualTo(5_000) + } + private fun assertThatSliderValueEquals(value: Float) { composeTestRule .onNodeWithTag(SLIDER_TAG) diff --git a/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/text/ImageCueTest.kt b/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/text/ImageCueTest.kt new file mode 100644 index 00000000000..d688935a74d --- /dev/null +++ b/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/text/ImageCueTest.kt @@ -0,0 +1,311 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.ui.compose.material3.text + +import android.content.Context +import androidx.compose.foundation.layout.Box +import androidx.compose.foundation.layout.fillMaxSize +import androidx.compose.ui.Alignment +import androidx.compose.ui.Modifier +import androidx.compose.ui.geometry.Offset +import androidx.compose.ui.layout.onGloballyPositioned +import androidx.compose.ui.layout.positionInParent +import androidx.compose.ui.test.ExperimentalTestApi +import androidx.compose.ui.test.onNodeWithContentDescription +import androidx.compose.ui.test.v2.runComposeUiTest +import androidx.compose.ui.unit.IntRect +import androidx.core.graphics.createBitmap +import androidx.media3.common.text.Cue +import androidx.media3.ui.compose.material3.R +import androidx.test.core.app.ApplicationProvider +import androidx.test.ext.junit.runners.AndroidJUnit4 +import com.google.common.truth.Truth.assertThat +import org.junit.Test +import org.junit.runner.RunWith + +/** Unit test for [ImageCue]. */ +@OptIn(ExperimentalTestApi::class) +@RunWith(AndroidJUnit4::class) +class ImageCueTest { + + private val context: Context = ApplicationProvider.getApplicationContext() + + @Test + fun imageCue_rendersBitmap() = runComposeUiTest { + val bitmap = createBitmap(50, 20) + val cue = Cue.Builder().setBitmap(bitmap).build() + + setContent { ImageCue(cue = cue, viewport = IntRect(0, 0, 200, 150)) } + + onNodeWithContentDescription(context.getString(R.string.subtitle_bitmap_cue)).assertExists() + } + + @Test + fun textCue_doesNotRender() = runComposeUiTest { + val cue = Cue.Builder().setText("Hello").build() + + setContent { ImageCue(cue = cue, viewport = IntRect(0, 0, 200, 150)) } + + onNodeWithContentDescription(context.getString(R.string.subtitle_bitmap_cue)) + .assertDoesNotExist() + } + + @Test + fun intrinsicSize_unsetBounds() = runComposeUiTest { + val bitmap = createBitmap(50, 20) + val cue = Cue.Builder().setBitmap(bitmap).build() + + setContent { ImageCue(cue = cue, viewport = IntRect(0, 0, 200, 150)) } + + val node = + onNodeWithContentDescription(context.getString(R.string.subtitle_bitmap_cue)) + .fetchSemanticsNode() + assertThat(node.size.width).isEqualTo(50) + assertThat(node.size.height).isEqualTo(20) + } + + @Test + fun relativeWidthOnly_maintainsAspectRatio() = runComposeUiTest { + val bitmap = createBitmap(50, 20) // 5:2 aspect ratio + val cue = Cue.Builder().setBitmap(bitmap).setSize(0.5f).build() + + setContent { ImageCue(cue = cue, viewport = IntRect(0, 0, 200, 150)) } + + val node = + onNodeWithContentDescription(context.getString(R.string.subtitle_bitmap_cue)) + .fetchSemanticsNode() + assertThat(node.size.width).isEqualTo(100) // 200 * 0.5 + assertThat(node.size.height).isEqualTo(40) // Proportional to 5:2 + } + + @Test + fun explicitWidthAndHeight_disregardsAspectRatio() = runComposeUiTest { + val bitmap = createBitmap(50, 20) + val cue = Cue.Builder().setBitmap(bitmap).setSize(0.5f).setBitmapHeight(0.2f).build() + + setContent { ImageCue(cue = cue, viewport = IntRect(0, 0, 200, 150)) } + + val node = + onNodeWithContentDescription(context.getString(R.string.subtitle_bitmap_cue)) + .fetchSemanticsNode() + assertThat(node.size.width).isEqualTo(100) // 200 * 0.5 + assertThat(node.size.height).isEqualTo(30) // 150 * 0.2 + } + + @Test + fun relativeHeightOnly_fallsBackToIntrinsicWidth_disregardsAspectRatio() = runComposeUiTest { + val bitmap = createBitmap(50, 20) + val cue = Cue.Builder().setBitmap(bitmap).setBitmapHeight(0.5f).build() + + setContent { ImageCue(cue = cue, viewport = IntRect(0, 0, 200, 150)) } + + val node = + onNodeWithContentDescription(context.getString(R.string.subtitle_bitmap_cue)) + .fetchSemanticsNode() + assertThat(node.size.width).isEqualTo(50) // Intrinsic width + assertThat(node.size.height).isEqualTo(75) // 150 * 0.5 + } + + @Test + fun fallbackToCenter() = runComposeUiTest { + val bitmap = createBitmap(50, 20) + val cue = Cue.Builder().setBitmap(bitmap).build() // Position/line are DIMEN_UNSET + + setContent { + Box(modifier = Modifier.fillMaxSize(), contentAlignment = Alignment.TopStart) { + ImageCue(cue = cue, viewport = IntRect(0, 0, 200, 150)) + } + } + + val node = + onNodeWithContentDescription(context.getString(R.string.subtitle_bitmap_cue)) + .fetchSemanticsNode() + assertThat(node.positionInRoot.x).isEqualTo(75f) // (200 - 50) / 2 + assertThat(node.positionInRoot.y).isEqualTo(65f) // (150 - 20) / 2 + } + + @Test + fun anchorTypeStart() = runComposeUiTest { + val bitmap = createBitmap(50, 20) + val cue = + Cue.Builder() + .setBitmap(bitmap) + .setPosition(0.3f) + .setPositionAnchor(Cue.ANCHOR_TYPE_START) + .setLine(0.4f, Cue.LINE_TYPE_FRACTION) + .setLineAnchor(Cue.ANCHOR_TYPE_START) + .build() + + setContent { + Box(modifier = Modifier.fillMaxSize(), contentAlignment = Alignment.TopStart) { + ImageCue(cue = cue, viewport = IntRect(0, 0, 200, 150)) + } + } + + val node = + onNodeWithContentDescription(context.getString(R.string.subtitle_bitmap_cue)) + .fetchSemanticsNode() + assertThat(node.positionInRoot.x).isEqualTo(60f) // 200 * 0.3 + assertThat(node.positionInRoot.y).isEqualTo(60f) // 150 * 0.4 + } + + @Test + fun anchorTypeMiddle() = runComposeUiTest { + val bitmap = createBitmap(50, 20) + val cue = + Cue.Builder() + .setBitmap(bitmap) + .setPosition(0.5f) + .setPositionAnchor(Cue.ANCHOR_TYPE_MIDDLE) + .setLine(0.5f, Cue.LINE_TYPE_FRACTION) + .setLineAnchor(Cue.ANCHOR_TYPE_MIDDLE) + .build() + + setContent { + Box(modifier = Modifier.fillMaxSize(), contentAlignment = Alignment.TopStart) { + ImageCue(cue = cue, viewport = IntRect(0, 0, 200, 150)) + } + } + + val node = + onNodeWithContentDescription(context.getString(R.string.subtitle_bitmap_cue)) + .fetchSemanticsNode() + // Position 100, but centered around it, so subtract half the bitmap width (50 / 2 = 25) + assertThat(node.positionInRoot.x).isEqualTo(75f) // (200 - 50) / 2 + // Line 75, centered around it, subtract half height (20 / 2 = 10) + assertThat(node.positionInRoot.y).isEqualTo(65f) // (150 - 20) / 2 + } + + @Test + fun anchorTypeEnd() = runComposeUiTest { + val bitmap = createBitmap(50, 20) + val cue = + Cue.Builder() + .setBitmap(bitmap) + .setPosition(1.0f) + .setPositionAnchor(Cue.ANCHOR_TYPE_END) + .setLine(1.0f, Cue.LINE_TYPE_FRACTION) + .setLineAnchor(Cue.ANCHOR_TYPE_END) + .build() + + setContent { + Box(modifier = Modifier.fillMaxSize(), contentAlignment = Alignment.TopStart) { + ImageCue(cue = cue, viewport = IntRect(0, 0, 200, 150)) + } + } + + val node = + onNodeWithContentDescription(context.getString(R.string.subtitle_bitmap_cue)) + .fetchSemanticsNode() + // Position 200, anchored at end, subtract full width (50) + assertThat(node.positionInRoot.x).isEqualTo(150f) // 200 - 50 + // Line 150, anchored at end, subtract full height (20) + assertThat(node.positionInRoot.y).isEqualTo(130f) // 150 - 20 + } + + @Test + fun nonZeroViewportOrigin_shiftsCoordinates() = runComposeUiTest { + val bitmap = createBitmap(50, 20) + val cue = + Cue.Builder() + .setBitmap(bitmap) + .setPosition(0.3f) + .setPositionAnchor(Cue.ANCHOR_TYPE_START) + .setLine(0.4f, Cue.LINE_TYPE_FRACTION) + .setLineAnchor(Cue.ANCHOR_TYPE_START) + .build() + + var imagePositionInParent = Offset.Zero + + // Viewport is shifted by 10x and 5y + setContent { + Box(modifier = Modifier.fillMaxSize(), contentAlignment = Alignment.TopStart) { + ImageCue( + cue = cue, + viewport = IntRect(10, 5, 210, 155), // 200x150 shifted by 10, 5 + modifier = + Modifier.onGloballyPositioned { coordinates -> + imagePositionInParent = coordinates.positionInParent() + }, + ) + } + } + + val node = + onNodeWithContentDescription(context.getString(R.string.subtitle_bitmap_cue)) + .fetchSemanticsNode() + assertThat(imagePositionInParent.x).isEqualTo(60f) // 200 * 0.3 + assertThat(imagePositionInParent.y).isEqualTo(60f) // 150 * 0.4 + assertThat(node.positionInRoot.x).isEqualTo(70f) // 10 + 60 + assertThat(node.positionInRoot.y).isEqualTo(65f) // 5 + 60 + } + + @Test + fun nonZeroViewportOrigin_withAnchor_shiftsCoordinates() = runComposeUiTest { + val bitmap = createBitmap(50, 20) + val cue = + Cue.Builder() + .setBitmap(bitmap) + .setPosition(1.0f) + .setPositionAnchor(Cue.ANCHOR_TYPE_END) + .setLine(1.0f, Cue.LINE_TYPE_FRACTION) + .setLineAnchor(Cue.ANCHOR_TYPE_END) + .build() + + var imagePositionInParent = Offset.Zero + + // Viewport is shifted by 10x and 5y + setContent { + Box(modifier = Modifier.fillMaxSize(), contentAlignment = Alignment.TopStart) { + ImageCue( + cue = cue, + viewport = IntRect(10, 5, 210, 155), // 200x150 shifted by 10, 5 + modifier = + Modifier.onGloballyPositioned { coordinates -> + imagePositionInParent = coordinates.positionInParent() + }, + ) + } + } + + val node = + onNodeWithContentDescription(context.getString(R.string.subtitle_bitmap_cue)) + .fetchSemanticsNode() + assertThat(imagePositionInParent.x).isEqualTo(150f) // 200 - 50 + assertThat(imagePositionInParent.y).isEqualTo(130f) // 150 - 20 + assertThat(node.positionInRoot.x).isEqualTo(160f) // 10 + 150 + assertThat(node.positionInRoot.y).isEqualTo(135f) // 5 + 130 + } + + @Test + fun zeroSizedViewport_placedAroundZero() = runComposeUiTest { + val bitmap = createBitmap(50, 20) + val cue = Cue.Builder().setBitmap(bitmap).build() + + setContent { ImageCue(cue = cue, viewport = IntRect(0, 0, 0, 0)) } + + val node = + onNodeWithContentDescription(context.getString(R.string.subtitle_bitmap_cue)) + .fetchSemanticsNode() + + // Intrinsic size should be maintained, but placed relative to a 0x0 viewport. + assertThat(node.size.width).isEqualTo(50) + assertThat(node.size.height).isEqualTo(20) + assertThat(node.positionInRoot.x).isEqualTo(-25f) // (0 - 50) / 2 + assertThat(node.positionInRoot.y).isEqualTo(-10f) // (0 - 20) / 2 + } +} diff --git a/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/text/SubtitlesTest.kt b/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/text/SubtitlesTest.kt new file mode 100644 index 00000000000..3a14739bddb --- /dev/null +++ b/libraries/ui_compose_material3/src/test/java/androidx/media3/ui/compose/material3/text/SubtitlesTest.kt @@ -0,0 +1,49 @@ +/* + * Copyright 2026 The Android Open Source Project + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * https://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package androidx.media3.ui.compose.material3.text + +import android.content.Context +import android.graphics.Bitmap +import androidx.compose.ui.test.ExperimentalTestApi +import androidx.compose.ui.test.onNodeWithContentDescription +import androidx.compose.ui.test.v2.runComposeUiTest +import androidx.media3.common.text.Cue +import androidx.media3.common.text.CueGroup +import androidx.media3.test.utils.FakePlayer +import androidx.media3.ui.compose.material3.R +import androidx.test.core.app.ApplicationProvider +import androidx.test.ext.junit.runners.AndroidJUnit4 +import org.junit.Test +import org.junit.runner.RunWith + +/** Unit test for [Subtitles]. */ +@OptIn(ExperimentalTestApi::class) +@RunWith(AndroidJUnit4::class) +class SubtitlesTest { + private val context: Context = ApplicationProvider.getApplicationContext() + + @Test + fun subtitles_withImageCue_rendersImage() = runComposeUiTest { + val bitmap = Bitmap.createBitmap(100, 100, Bitmap.Config.ARGB_8888) + val cues = listOf(Cue.Builder().setBitmap(bitmap).build()) + val player = FakePlayer().apply { setCurrentCues(CueGroup(cues, 0)) } + + setContent { Subtitles(player = player) } + + onNodeWithContentDescription(context.getString(R.string.subtitle_bitmap_cue)).assertExists() + } +} diff --git a/testapps/controller/src/main/res/mipmap-hdpi/ic_launcher.png b/testapps/controller/src/main/res/mipmap-hdpi/ic_launcher.png index adaa93220eb..e5a228fb765 100644 Binary files a/testapps/controller/src/main/res/mipmap-hdpi/ic_launcher.png and b/testapps/controller/src/main/res/mipmap-hdpi/ic_launcher.png differ diff --git a/testapps/controller/src/main/res/mipmap-mdpi/ic_launcher.png b/testapps/controller/src/main/res/mipmap-mdpi/ic_launcher.png index 9b6f7d5e806..a058cced802 100644 Binary files a/testapps/controller/src/main/res/mipmap-mdpi/ic_launcher.png and b/testapps/controller/src/main/res/mipmap-mdpi/ic_launcher.png differ diff --git a/testapps/controller/src/main/res/mipmap-xhdpi/ic_launcher.png b/testapps/controller/src/main/res/mipmap-xhdpi/ic_launcher.png index 2101026c9fe..b2e07ad4ff8 100644 Binary files a/testapps/controller/src/main/res/mipmap-xhdpi/ic_launcher.png and b/testapps/controller/src/main/res/mipmap-xhdpi/ic_launcher.png differ diff --git a/testapps/controller/src/main/res/mipmap-xxhdpi/ic_launcher.png b/testapps/controller/src/main/res/mipmap-xxhdpi/ic_launcher.png index 223ec8bd113..c40bc4e6d37 100644 Binary files a/testapps/controller/src/main/res/mipmap-xxhdpi/ic_launcher.png and b/testapps/controller/src/main/res/mipmap-xxhdpi/ic_launcher.png differ diff --git a/testapps/controller/src/main/res/mipmap-xxxhdpi/ic_launcher.png b/testapps/controller/src/main/res/mipmap-xxxhdpi/ic_launcher.png index 698ed68c429..b5340fa2881 100644 Binary files a/testapps/controller/src/main/res/mipmap-xxxhdpi/ic_launcher.png and b/testapps/controller/src/main/res/mipmap-xxxhdpi/ic_launcher.png differ